2026-02-11T23:30:16.226+0000 7f7000069780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-11T23:30:16.226+0000 7f7000069780 10 monclient: get_monmap_and_config 2026-02-11T23:30:16.226+0000 7f7000069780 10 monclient: build_initial_monmap 2026-02-11T23:30:16.226+0000 7f7000069780 10 monclient: monmap: epoch 0 fsid cb0a9481-482d-477a-a81c-1446b6498370 last_changed 2026-02-11T23:30:16.227068+0000 created 2026-02-11T23:30:16.227068+0000 min_mon_release 0 (unknown) election_strategy: 1 0: v1:127.0.0.1:7180/0 mon.noname-a-legacy 1: v2:127.0.0.1:7180/0 mon.noname-a 2026-02-11T23:30:16.227+0000 7f7000069780 1 Processor -- start 2026-02-11T23:30:16.227+0000 7f7000069780 1 -- start start 2026-02-11T23:30:16.227+0000 7f7000069780 10 monclient: init 2026-02-11T23:30:16.227+0000 7f7000069780 10 monclient: _reopen_session rank -1 2026-02-11T23:30:16.227+0000 7f7000069780 10 monclient: _add_conns ranks=[1,0] 2026-02-11T23:30:16.227+0000 7f7000069780 1 --2- >> v2:127.0.0.1:7180/0 conn(0x3f43c00 0x4100000 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-11T23:30:16.227+0000 7f7000069780 10 monclient(hunting): picked mon.noname-a con 0x3f43c00 addr v2:127.0.0.1:7180/0 2026-02-11T23:30:16.227+0000 7f7000069780 10 monclient(hunting): picked mon.noname-a-legacy con 0x4d8c000 addr v1:127.0.0.1:7180/0 2026-02-11T23:30:16.227+0000 7f7000069780 1 -- --> v1:127.0.0.1:7180/0 -- auth(proto 0 26 bytes epoch 0) -- 0x3facf00 con 0x4d8c000 2026-02-11T23:30:16.227+0000 7f7000069780 10 monclient(hunting): start opening mon connection 2026-02-11T23:30:16.227+0000 7f7000069780 1 -- --> v2:127.0.0.1:7180/0 -- mon_getmap magic: 0 -- 0x3fc6c80 con 0x3f43c00 2026-02-11T23:30:16.227+0000 7f7000069780 10 monclient(hunting): _renew_subs 2026-02-11T23:30:16.227+0000 7f7000069780 10 monclient(hunting): authenticate will time out at 1273.915301s 2026-02-11T23:30:16.227+0000 7f6ffdbfc640 1 --2- >> v2:127.0.0.1:7180/0 conn(0x3f43c00 0x4100000 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:16.227+0000 7f6ffdbfc640 1 --2- >> v2:127.0.0.1:7180/0 conn(0x3f43c00 0x4100000 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:127.0.0.1:7180/0 says I am v2:127.0.0.1:41090/0 (socket says 127.0.0.1:41090) 2026-02-11T23:30:16.227+0000 7f6ffdbfc640 1 -- 127.0.0.1:0/2540032399 learned_addr learned my addr 127.0.0.1:0/2540032399 (peer_addr_for_me v2:127.0.0.1:0/0) 2026-02-11T23:30:16.227+0000 7f6ffdbfc640 10 monclient(hunting): get_auth_request con 0x3f43c00 auth_method 0 2026-02-11T23:30:16.227+0000 7f6ffdbfc640 10 monclient(hunting): get_auth_request method 1 preferred_modes [1] 2026-02-11T23:30:16.227+0000 7f6ffdbfc640 10 monclient(hunting): _init_auth method 1 2026-02-11T23:30:16.227+0000 7f6ffdbfc640 10 monclient(hunting): _init_auth creating new auth 2026-02-11T23:30:16.227+0000 7f6ffdbfc640 10 monclient(hunting): handle_auth_done global_id 4103 payload 0 2026-02-11T23:30:16.227+0000 7f6ffdbfc640 1 -- 127.0.0.1:0/2540032399 >> v1:127.0.0.1:7180/0 conn(0x4d8c000 legacy=0x40cd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-11T23:30:16.227+0000 7f6ffdbfc640 10 monclient: _finish_hunting 0 2026-02-11T23:30:16.227+0000 7f6ffdbfc640 1 monclient: found mon.noname-a 2026-02-11T23:30:16.227+0000 7f6ffdbfc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-11T23:30:16.227+0000 7f6ffdbfc640 10 monclient: _send_mon_message to mon.noname-a at v2:127.0.0.1:7180/0 2026-02-11T23:30:16.227+0000 7f6ffdbfc640 1 -- 127.0.0.1:0/2540032399 --> v2:127.0.0.1:7180/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x3fa6ea0 con 0x3f43c00 2026-02-11T23:30:16.227+0000 7f6ffdbfc640 1 --2- 127.0.0.1:0/2540032399 >> v2:127.0.0.1:7180/0 conn(0x3f43c00 0x4100000 crc :-1 s=READY pgs=7 gs=2 cs=0 l=1 c_cookie=6972941204099633390 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60c4dd14b60b10ee server_cookie=0 in_seq=0 out_seq=0 2026-02-11T23:30:16.227+0000 7f6ffcbfa640 1 -- 127.0.0.1:0/2540032399 <== mon.0 v2:127.0.0.1:7180/0 1 ==== mon_map magic: 0 ==== 178+0+0 (crc 0 0 0) 0x3faadc0 con 0x3f43c00 2026-02-11T23:30:16.227+0000 7f6ffcbfa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-11T23:30:16.227+0000 7f6ffcbfa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-11T23:30:16.227+0000 7f6ffcbfa640 10 monclient: dump: epoch 1 fsid cb0a9481-482d-477a-a81c-1446b6498370 last_changed 2026-02-11T23:30:15.835295+0000 created 2026-02-11T23:30:15.835295+0000 min_mon_release 20 (tentacle) election_strategy: 1 0: v2:127.0.0.1:7180/0 mon.a 2026-02-11T23:30:16.227+0000 7f6ffcbfa640 10 monclient: _finish_auth 0 2026-02-11T23:30:16.227+0000 7f6ffcbfa640 10 monclient: _check_auth_tickets 2026-02-11T23:30:16.227+0000 7f6ffcbfa640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:29:46.228035+0000) 2026-02-11T23:30:16.227+0000 7f6ffcbfa640 1 -- 127.0.0.1:0/2540032399 <== mon.0 v2:127.0.0.1:7180/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x3facf00 con 0x3f43c00 2026-02-11T23:30:16.227+0000 7f7000069780 5 monclient: authenticate success, global_id 4103 2026-02-11T23:30:16.227+0000 7f7000069780 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-11T23:30:16.227+0000 7f6ffcbfa640 10 monclient: handle_config config(0 keys) 2026-02-11T23:30:16.227+0000 7f6ffcbfa640 1 -- 127.0.0.1:0/2540032399 <== mon.0 v2:127.0.0.1:7180/0 3 ==== mon_map magic: 0 ==== 178+0+0 (crc 0 0 0) 0x3fab080 con 0x3f43c00 2026-02-11T23:30:16.227+0000 7f6ffcbfa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-11T23:30:16.227+0000 7f6ffcbfa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-11T23:30:16.227+0000 7f6ffcbfa640 10 monclient: dump: epoch 1 fsid cb0a9481-482d-477a-a81c-1446b6498370 last_changed 2026-02-11T23:30:15.835295+0000 created 2026-02-11T23:30:15.835295+0000 min_mon_release 20 (tentacle) election_strategy: 1 0: v2:127.0.0.1:7180/0 mon.a 2026-02-11T23:30:16.227+0000 7f7000069780 10 monclient: get_monmap_and_config success 2026-02-11T23:30:16.227+0000 7f7000069780 4 set_mon_vals no callback set 2026-02-11T23:30:16.227+0000 7f7000069780 10 monclient: shutdown 2026-02-11T23:30:16.227+0000 7f7000069780 20 monclient: shutdown discarding 0 pending message(s) 2026-02-11T23:30:16.227+0000 7f7000069780 1 -- 127.0.0.1:0/2540032399 >> v2:127.0.0.1:7180/0 conn(0x3f43c00 msgr2=0x4100000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-11T23:30:16.227+0000 7f7000069780 1 --2- 127.0.0.1:0/2540032399 >> v2:127.0.0.1:7180/0 conn(0x3f43c00 0x4100000 crc :-1 s=READY pgs=7 gs=2 cs=0 l=1 c_cookie=6972941204099633390 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-11T23:30:16.228+0000 7f7000069780 1 -- 127.0.0.1:0/2540032399 shutdown_connections 2026-02-11T23:30:16.228+0000 7f7000069780 1 --2- 127.0.0.1:0/2540032399 >> v2:127.0.0.1:7180/0 conn(0x3f43c00 0x4100000 unknown :-1 s=CLOSED pgs=7 gs=2 cs=0 l=1 c_cookie=6972941204099633390 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-11T23:30:16.228+0000 7f7000069780 1 -- 127.0.0.1:0/2540032399 >> 127.0.0.1:0/2540032399 conn(0x3f43800 msgr2=0x40cc800 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-11T23:30:16.228+0000 7f7000069780 1 -- 127.0.0.1:0/2540032399 shutdown_connections 2026-02-11T23:30:16.228+0000 7f7000069780 1 -- 127.0.0.1:0/2540032399 wait complete. 2026-02-11T23:30:16.228+0000 7f7000069780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-11T23:30:16.228+0000 7f7000069780 0 ceph version 20.3.0-5256-gf2a98edf (f2a98edfc002a9c9c8df236a812719da23bb2dd0) tentacle (dev - RelWithDebInfo), process ceph-osd, pid 314762 2026-02-11T23:30:16.229+0000 7f7000069780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-11T23:30:16.229+0000 7f7000069780 1 bluestore(td/osd-backfill-space/0) mkfs path td/osd-backfill-space/0 2026-02-11T23:30:16.229+0000 7f7000069780 1 bdev(0x3f42c00 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:16.229+0000 7f7000069780 -1 bdev(0x3f42c00 td/osd-backfill-space/0/block) open stat got: (1) Operation not permitted 2026-02-11T23:30:16.229+0000 7f7000069780 -1 bluestore(td/osd-backfill-space/0) _read_fsid unparsable uuid 2026-02-11T23:30:16.229+0000 7f7000069780 1 bluestore(td/osd-backfill-space/0) mkfs using provided fsid 01217804-d5cf-416e-93e4-5f1242b1c391 2026-02-11T23:30:16.229+0000 7f7000069780 1 bluestore(td/osd-backfill-space/0) _setup_block_symlink_or_file resized block file to 100 GiB 2026-02-11T23:30:16.229+0000 7f7000069780 1 bdev(0x3f42c00 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:16.229+0000 7f7000069780 0 bdev(0x3f42c00 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:16.229+0000 7f7000069780 1 bdev(0x3f42c00 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:16.230+0000 7f7000069780 1 bluestore(td/osd-backfill-space/0) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-11T23:30:16.230+0000 7f7000069780 1 bdev(0x3f43c00 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:16.230+0000 7f7000069780 0 bdev(0x3f43c00 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:16.230+0000 7f7000069780 1 bdev(0x3f43c00 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:16.230+0000 7f7000069780 1 bluefs add_block_device bdev 1 path td/osd-backfill-space/0/block size 100 GiB 2026-02-11T23:30:16.230+0000 7f7000069780 1 bluefs mkfs osd_uuid 01217804-d5cf-416e-93e4-5f1242b1c391 2026-02-11T23:30:16.230+0000 7f7000069780 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-11T23:30:16.230+0000 7f7000069780 1 bluefs mkfs super super(uuid b7b9fc14-4e19-49d9-9f75-114c28890351 osd 01217804-d5cf-416e-93e4-5f1242b1c391 seq 1 block_size 0x1000 log_fnode 0xfile(ino 1 size 0x1000 mtime 0.000000 allocated 400000 alloc_commit 0 extents [1:0x2000~400000])) 2026-02-11T23:30:16.231+0000 7f7000069780 1 bluefs mount 2026-02-11T23:30:16.231+0000 7f7000069780 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-11T23:30:16.232+0000 7f7000069780 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:16.232+0000 7f7000069780 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:16.232+0000 7f7000069780 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:16.232+0000 7f7000069780 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:16.232+0000 7f7000069780 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:16.232+0000 7f7000069780 1 bluefs mount shared_bdev_used = 4194304 2026-02-11T23:30:16.232+0000 7f7000069780 1 bluestore(td/osd-backfill-space/0) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: RocksDB version: 7.9.2 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Git sha 0 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Compile date 2026-02-11 10:30:18 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: DB SUMMARY 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: DB Session ID: J8E4HVLK4E8NZ5Y1PHHF 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: SST files in db dir, Total Num: 0, files: 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: SST files in db.slow dir, Total Num: 0, files: 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Write Ahead Log file in db.wal: 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.error_if_exists: 0 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.create_if_missing: 1 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.paranoid_checks: 1 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.flush_verify_memtable_count: 1 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.track_and_verify_wals_in_manifest: 0 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.env: 0x402bf80 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.fs: LegacyFileSystem 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.info_log: 0x3fa17e0 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.max_file_opening_threads: 16 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.statistics: (nil) 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.use_fsync: 0 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.max_log_file_size: 0 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.max_manifest_file_size: 1073741824 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.log_file_time_to_roll: 0 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.keep_log_file_num: 1000 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.recycle_log_file_num: 0 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.allow_fallocate: 1 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.allow_mmap_reads: 0 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.allow_mmap_writes: 0 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.use_direct_reads: 0 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.create_missing_column_families: 0 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.db_log_dir: 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.wal_dir: db.wal 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.table_cache_numshardbits: 6 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.WAL_ttl_seconds: 0 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.WAL_size_limit_MB: 0 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.manifest_preallocation_size: 4194304 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.is_fd_close_on_exec: 1 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.advise_random_on_open: 1 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.db_write_buffer_size: 0 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.write_buffer_manager: 0x3f3d540 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.access_hint_on_compaction_start: 1 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.use_adaptive_mutex: 0 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.rate_limiter: (nil) 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.wal_recovery_mode: 2 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.enable_thread_tracking: 0 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.enable_pipelined_write: 0 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.unordered_write: 0 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.write_thread_max_yield_usec: 100 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.row_cache: None 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.wal_filter: None 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.avoid_flush_during_recovery: 0 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.allow_ingest_behind: 0 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.two_write_queues: 0 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.manual_wal_flush: 0 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.wal_compression: 0 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.atomic_flush: 0 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.persist_stats_to_disk: 0 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.write_dbid_to_manifest: 0 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.log_readahead_size: 0 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.file_checksum_gen_factory: Unknown 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.best_efforts_recovery: 0 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.allow_data_in_errors: 0 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.db_host_id: __hostname__ 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.enforce_single_del_contracts: true 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.max_background_jobs: 4 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.max_background_compactions: -1 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.max_subcompactions: 1 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.writable_file_max_buffer_size: 0 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.delayed_write_rate : 16777216 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.max_total_wal_size: 1073741824 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.stats_dump_period_sec: 600 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.stats_persist_period_sec: 600 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.stats_history_buffer_size: 1048576 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.max_open_files: -1 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.bytes_per_sync: 0 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.wal_bytes_per_sync: 0 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.strict_bytes_per_sync: 0 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.compaction_readahead_size: 2097152 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Options.max_background_flushes: -1 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Compression algorithms supported: 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: kZSTD supported: 0 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: kXpressCompression supported: 0 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: kBZip2Compression supported: 0 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: kZSTDNotFinalCompression supported: 0 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: kLZ4Compression supported: 1 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: kZlibCompression supported: 1 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: kLZ4HCCompression supported: 1 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: kSnappyCompression supported: 1 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: Fast CRC32 supported: Supported on x86 2026-02-11T23:30:16.233+0000 7f7000069780 4 rocksdb: DMutex implementation: pthread_mutex_t 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: [db/db_impl/db_impl_open.cc:317] Creating manifest 1 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5527] Recovering from manifest file: db/MANIFEST-000001 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.merge_operator: .T:int64_array.b:bitwise_xor 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x3fa1f20) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x3f1fe10 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:16.234+0000 7f7000069780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:16.235+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5566] Recovered from manifest file:db/MANIFEST-000001 succeeded,manifest_file_number is 1, next_file_number is 3, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2026-02-11T23:30:16.235+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2026-02-11T23:30:16.235+0000 7f7000069780 4 rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: d2a13b50-f500-4af9-be39-15a6598bf960 2026-02-11T23:30:16.235+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5047] Creating manifest 5 2026-02-11T23:30:16.236+0000 7f7000069780 4 rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x3fb1180 2026-02-11T23:30:16.236+0000 7f7000069780 4 rocksdb: DB pointer 0x3fd0000 2026-02-11T23:30:16.237+0000 7f6ff3be8640 4 rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2026-02-11T23:30:16.237+0000 7f6ff3be8640 4 rocksdb: [db/db_impl/db_impl.cc:1111] ** DB Stats ** Uptime(secs): 0.0 total, 0.0 interval Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.000 H:M:S, 0.0 percent ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 0.0 total, 0.0 interval Flush(GB): cumulative 0.000, interval 0.000 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count Block cache BinnedLRUCache@0x3f1fe10#314771 capacity: 1.35 GB usage: 0.00 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.9e-05 secs_since: 0 Block cache entry stats(count,size,portion): Misc(1,0.00 KB,0%) ** File Read Latency Histogram By Level [default] ** 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [m-0]: 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x3fa1f20) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x3f1fe10 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:16.237+0000 7f7000069780 4 rocksdb: [db/db_impl/db_impl.cc:3106] Created column family [m-0] (ID 1) 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [m-1]: 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x3fa1f20) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x3f1fe10 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:16.238+0000 7f7000069780 4 rocksdb: [db/db_impl/db_impl.cc:3106] Created column family [m-1] (ID 2) 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [m-2]: 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x3fa1f20) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x3f1fe10 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:16.239+0000 7f7000069780 4 rocksdb: [db/db_impl/db_impl.cc:3106] Created column family [m-2] (ID 3) 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [p-0]: 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x3fa1f20) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x3f1fe10 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:16.240+0000 7f7000069780 4 rocksdb: [db/db_impl/db_impl.cc:3106] Created column family [p-0] (ID 4) 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [p-1]: 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x3fa1f20) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x3f1fe10 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:16.241+0000 7f7000069780 4 rocksdb: [db/db_impl/db_impl.cc:3106] Created column family [p-1] (ID 5) 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [p-2]: 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x3fa1f20) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x3f1fe10 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:16.243+0000 7f7000069780 4 rocksdb: [db/db_impl/db_impl.cc:3106] Created column family [p-2] (ID 6) 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [O-0]: 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x3f791a0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x3f1f0f0 block_cache_name: BinnedLRUCache block_cache_options: capacity : 536870912 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:16.244+0000 7f7000069780 4 rocksdb: [db/db_impl/db_impl.cc:3106] Created column family [O-0] (ID 7) 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [O-1]: 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x3f791a0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x3f1f0f0 block_cache_name: BinnedLRUCache block_cache_options: capacity : 536870912 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:16.246+0000 7f7000069780 4 rocksdb: [db/db_impl/db_impl.cc:3106] Created column family [O-1] (ID 8) 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [O-2]: 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x3f791a0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x3f1f0f0 block_cache_name: BinnedLRUCache block_cache_options: capacity : 536870912 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:16.247+0000 7f7000069780 4 rocksdb: [db/db_impl/db_impl.cc:3106] Created column family [O-2] (ID 9) 2026-02-11T23:30:16.249+0000 7f7000069780 4 rocksdb: [db/column_family.cc:635] (skipping printing options) 2026-02-11T23:30:16.249+0000 7f7000069780 4 rocksdb: [db/db_impl/db_impl.cc:3106] Created column family [L] (ID 10) 2026-02-11T23:30:16.251+0000 7f7000069780 4 rocksdb: [db/column_family.cc:635] (skipping printing options) 2026-02-11T23:30:16.251+0000 7f7000069780 4 rocksdb: [db/db_impl/db_impl.cc:3106] Created column family [P] (ID 11) 2026-02-11T23:30:16.253+0000 7f7000069780 1 bluestore(td/osd-backfill-space/0) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-11T23:30:16.253+0000 7f7000069780 1 freelist create size 0x1900000000 bytes_per_block 0x1000 blocks 0x1900000 blocks_per_key 0x80 2026-02-11T23:30:16.254+0000 7f7000069780 1 bluestore(td/osd-backfill-space/0) _open_fm effective freelist_type = null, freelist_alloc_size = 0x1000, min_alloc_size = 0x1000 2026-02-11T23:30:16.256+0000 7f7000069780 1 freelist shutdown 2026-02-11T23:30:16.256+0000 7f7000069780 4 rocksdb: [db/db_impl/db_impl.cc:496] Shutdown: canceling all background work 2026-02-11T23:30:16.256+0000 7f7000069780 4 rocksdb: [db/db_impl/db_impl.cc:704] Shutdown complete 2026-02-11T23:30:16.256+0000 7f7000069780 1 bluefs umount 2026-02-11T23:30:16.256+0000 7f7000069780 1 bdev(0x3f43c00 td/osd-backfill-space/0/block) close 2026-02-11T23:30:16.287+0000 7f7000069780 1 bdev(0x3f42c00 td/osd-backfill-space/0/block) close 2026-02-11T23:30:16.307+0000 7f7000069780 1 bdev(0x3f42c00 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:16.307+0000 7f7000069780 0 bdev(0x3f42c00 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:16.307+0000 7f7000069780 1 bdev(0x3f42c00 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:16.307+0000 7f7000069780 1 bdev(0x3f42c00 td/osd-backfill-space/0/block) close 2026-02-11T23:30:16.329+0000 7f7000069780 1 bdev(0x3f42c00 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:16.329+0000 7f7000069780 0 bdev(0x3f42c00 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:16.329+0000 7f7000069780 1 bdev(0x3f42c00 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:16.329+0000 7f7000069780 1 bluestore(td/osd-backfill-space/0) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-11T23:30:16.329+0000 7f7000069780 1 bdev(0x3f43c00 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:16.329+0000 7f7000069780 0 bdev(0x3f43c00 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:16.329+0000 7f7000069780 1 bdev(0x3f43c00 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:16.329+0000 7f7000069780 1 bluefs add_block_device bdev 1 path td/osd-backfill-space/0/block size 100 GiB 2026-02-11T23:30:16.329+0000 7f7000069780 1 bluefs mount 2026-02-11T23:30:16.329+0000 7f7000069780 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-11T23:30:16.330+0000 7f7000069780 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:16.330+0000 7f7000069780 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:16.330+0000 7f7000069780 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:16.330+0000 7f7000069780 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:16.330+0000 7f7000069780 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:16.330+0000 7f7000069780 1 bluefs mount shared_bdev_used = 27197440 2026-02-11T23:30:16.330+0000 7f7000069780 1 bluestore(td/osd-backfill-space/0) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: RocksDB version: 7.9.2 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Git sha 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Compile date 2026-02-11 10:30:18 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: DB SUMMARY 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: DB Session ID: J8E4HVLK4E8NZ5Y1PHHE 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: CURRENT file: CURRENT 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: IDENTITY file: IDENTITY 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: MANIFEST file: MANIFEST-000005 size: 605 Bytes 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: SST files in db dir, Total Num: 0, files: 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: SST files in db.slow dir, Total Num: 0, files: 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Write Ahead Log file in db.wal: 000004.log size: 276 ; 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.error_if_exists: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.create_if_missing: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.paranoid_checks: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.flush_verify_memtable_count: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.track_and_verify_wals_in_manifest: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.env: 0x4d643f0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.fs: LegacyFileSystem 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.info_log: 0x3fae020 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_file_opening_threads: 16 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.statistics: (nil) 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.use_fsync: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_log_file_size: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_manifest_file_size: 1073741824 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.log_file_time_to_roll: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.keep_log_file_num: 1000 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.recycle_log_file_num: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.allow_fallocate: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.allow_mmap_reads: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.allow_mmap_writes: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.use_direct_reads: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.create_missing_column_families: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.db_log_dir: 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.wal_dir: db.wal 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.table_cache_numshardbits: 6 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.WAL_ttl_seconds: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.WAL_size_limit_MB: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.manifest_preallocation_size: 4194304 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.is_fd_close_on_exec: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.advise_random_on_open: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.db_write_buffer_size: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.write_buffer_manager: 0x3f3d540 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.access_hint_on_compaction_start: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.use_adaptive_mutex: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.rate_limiter: (nil) 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.wal_recovery_mode: 2 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.enable_thread_tracking: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.enable_pipelined_write: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.unordered_write: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.write_thread_max_yield_usec: 100 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.row_cache: None 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.wal_filter: None 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.avoid_flush_during_recovery: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.allow_ingest_behind: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.two_write_queues: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.manual_wal_flush: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.wal_compression: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.atomic_flush: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.persist_stats_to_disk: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.write_dbid_to_manifest: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.log_readahead_size: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.file_checksum_gen_factory: Unknown 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.best_efforts_recovery: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.allow_data_in_errors: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.db_host_id: __hostname__ 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.enforce_single_del_contracts: true 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_background_jobs: 4 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_background_compactions: -1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_subcompactions: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.writable_file_max_buffer_size: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.delayed_write_rate : 16777216 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_total_wal_size: 1073741824 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.stats_dump_period_sec: 600 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.stats_persist_period_sec: 600 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.stats_history_buffer_size: 1048576 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_open_files: -1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bytes_per_sync: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.wal_bytes_per_sync: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.strict_bytes_per_sync: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_readahead_size: 2097152 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_background_flushes: -1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Compression algorithms supported: 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: kZSTD supported: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: kXpressCompression supported: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: kBZip2Compression supported: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: kZSTDNotFinalCompression supported: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: kLZ4Compression supported: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: kZlibCompression supported: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: kLZ4HCCompression supported: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: kSnappyCompression supported: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Fast CRC32 supported: Supported on x86 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: DMutex implementation: pthread_mutex_t 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: [db/db_impl/db_impl_readonly.cc:25] Opening the db in read only mode 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5527] Recovering from manifest file: db/MANIFEST-000005 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.merge_operator: .T:int64_array.b:bitwise_xor 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x4da2300) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x3f1fe10 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [m-0]: 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x4da2300) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x3f1fe10 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [m-1]: 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x4da2300) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x3f1fe10 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [m-2]: 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x4da2300) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x3f1fe10 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [p-0]: 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x4da2300) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x3f1fe10 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [p-1]: 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x4da2300) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x3f1fe10 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.331+0000 7f7000069780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [p-2]: 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x4da2300) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x3f1fe10 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [O-0]: 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x3f79520) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x3f1f0f0 block_cache_name: BinnedLRUCache block_cache_options: capacity : 536870912 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [O-1]: 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x3f79520) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x3f1f0f0 block_cache_name: BinnedLRUCache block_cache_options: capacity : 536870912 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [O-2]: 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x3f79520) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x3f1f0f0 block_cache_name: BinnedLRUCache block_cache_options: capacity : 536870912 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: [db/column_family.cc:635] (skipping printing options) 2026-02-11T23:30:16.332+0000 7f7000069780 4 rocksdb: [db/column_family.cc:635] (skipping printing options) 2026-02-11T23:30:16.336+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5566] Recovered from manifest file:db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 29, last_sequence is 0, log_number is 4,prev_log_number is 0,max_column_family is 11,min_log_number_to_keep is 0 2026-02-11T23:30:16.336+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2026-02-11T23:30:16.336+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5581] Column family [m-0] (ID 1), log number is 4 2026-02-11T23:30:16.336+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5581] Column family [m-1] (ID 2), log number is 4 2026-02-11T23:30:16.336+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5581] Column family [m-2] (ID 3), log number is 4 2026-02-11T23:30:16.336+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5581] Column family [p-0] (ID 4), log number is 4 2026-02-11T23:30:16.336+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5581] Column family [p-1] (ID 5), log number is 4 2026-02-11T23:30:16.336+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5581] Column family [p-2] (ID 6), log number is 4 2026-02-11T23:30:16.336+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5581] Column family [O-0] (ID 7), log number is 4 2026-02-11T23:30:16.336+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5581] Column family [O-1] (ID 8), log number is 4 2026-02-11T23:30:16.336+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5581] Column family [O-2] (ID 9), log number is 4 2026-02-11T23:30:16.336+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5581] Column family [L] (ID 10), log number is 4 2026-02-11T23:30:16.336+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5581] Column family [P] (ID 11), log number is 4 2026-02-11T23:30:16.336+0000 7f7000069780 4 rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: d2a13b50-f500-4af9-be39-15a6598bf960 2026-02-11T23:30:16.336+0000 7f7000069780 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1770852616336671, "job": 1, "event": "recovery_started", "wal_files": [4]} 2026-02-11T23:30:16.336+0000 7f7000069780 4 rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2026-02-11T23:30:16.336+0000 7f7000069780 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1770852616336968, "job": 1, "event": "recovery_finished"} 2026-02-11T23:30:16.336+0000 7f7000069780 1 bluestore(td/osd-backfill-space/0) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-11T23:30:16.336+0000 7f7000069780 1 bluestore(td/osd-backfill-space/0) _open_super_meta old nid_max 0 2026-02-11T23:30:16.336+0000 7f7000069780 1 bluestore(td/osd-backfill-space/0) _open_super_meta old blobid_max 0 2026-02-11T23:30:16.336+0000 7f7000069780 1 bluestore(td/osd-backfill-space/0) _open_super_meta ondisk_format 4 compat_ondisk_format 3 2026-02-11T23:30:16.336+0000 7f7000069780 1 bluestore(td/osd-backfill-space/0) _open_super_meta min_alloc_size 0x1000 2026-02-11T23:30:16.337+0000 7f7000069780 1 freelist init 2026-02-11T23:30:16.337+0000 7f7000069780 1 freelist _read_cfg 2026-02-11T23:30:16.337+0000 7f7000069780 1 bluestore(td/osd-backfill-space/0) _open_fm effective freelist_type = null, freelist_alloc_size = 0x1000, min_alloc_size = 0x1000 2026-02-11T23:30:16.337+0000 7f7000069780 1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2026-02-11T23:30:16.337+0000 7f7000069780 1 fbmap_alloc 0x408c360 shutdown 2026-02-11T23:30:16.337+0000 7f7000069780 0 bluestore(td/osd-backfill-space/0) _init_alloc::NCB::restore_allocator() failed! Run Full Recovery from ONodes (might take a while) ... 2026-02-11T23:30:16.337+0000 7f7000069780 1 bluestore::NCB::read_allocation_from_drive_on_startup::::Allocation Recovery was completed in 0.000190 seconds, extent_count=1 2026-02-11T23:30:16.337+0000 7f7000069780 1 bluestore(td/osd-backfill-space/0) _init_alloc loaded 0 B in 0 extents, allocator type hybrid, capacity 0x1900000000, block size 0x1000, free 0x18ffffe000, fragmentation 0 2026-02-11T23:30:16.337+0000 7f7000069780 4 rocksdb: [db/db_impl/db_impl.cc:496] Shutdown: canceling all background work 2026-02-11T23:30:16.338+0000 7f7000069780 4 rocksdb: [db/db_impl/db_impl.cc:704] Shutdown complete 2026-02-11T23:30:16.338+0000 7f7000069780 1 bluefs umount 2026-02-11T23:30:16.338+0000 7f7000069780 1 bdev(0x3f43c00 td/osd-backfill-space/0/block) close 2026-02-11T23:30:16.362+0000 7f7000069780 1 bdev(0x3f43c00 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:16.362+0000 7f7000069780 0 bdev(0x3f43c00 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:16.362+0000 7f7000069780 1 bdev(0x3f43c00 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:16.362+0000 7f7000069780 1 bluefs add_block_device bdev 1 path td/osd-backfill-space/0/block size 100 GiB 2026-02-11T23:30:16.362+0000 7f7000069780 1 bluefs mount 2026-02-11T23:30:16.362+0000 7f7000069780 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-11T23:30:16.363+0000 7f7000069780 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:16.363+0000 7f7000069780 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:16.363+0000 7f7000069780 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:16.363+0000 7f7000069780 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:16.363+0000 7f7000069780 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:16.363+0000 7f7000069780 1 bluefs mount shared_bdev_used = 27197440 2026-02-11T23:30:16.363+0000 7f7000069780 1 bluestore(td/osd-backfill-space/0) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: RocksDB version: 7.9.2 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Git sha 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Compile date 2026-02-11 10:30:18 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: DB SUMMARY 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: DB Session ID: J8E4HVLK4E8NZ5Y1PHHD 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: CURRENT file: CURRENT 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: IDENTITY file: IDENTITY 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: MANIFEST file: MANIFEST-000005 size: 605 Bytes 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: SST files in db dir, Total Num: 0, files: 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: SST files in db.slow dir, Total Num: 0, files: 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Write Ahead Log file in db.wal: 000004.log size: 276 ; 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.error_if_exists: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.create_if_missing: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.paranoid_checks: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.flush_verify_memtable_count: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.track_and_verify_wals_in_manifest: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.env: 0x3f5e7e0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.fs: LegacyFileSystem 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.info_log: 0x3fa1660 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_file_opening_threads: 16 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.statistics: (nil) 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.use_fsync: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_log_file_size: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_manifest_file_size: 1073741824 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.log_file_time_to_roll: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.keep_log_file_num: 1000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.recycle_log_file_num: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.allow_fallocate: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.allow_mmap_reads: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.allow_mmap_writes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.use_direct_reads: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.create_missing_column_families: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.db_log_dir: 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.wal_dir: db.wal 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.table_cache_numshardbits: 6 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.WAL_ttl_seconds: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.WAL_size_limit_MB: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.manifest_preallocation_size: 4194304 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.is_fd_close_on_exec: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.advise_random_on_open: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.db_write_buffer_size: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.write_buffer_manager: 0x3f3d540 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.access_hint_on_compaction_start: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.use_adaptive_mutex: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.rate_limiter: (nil) 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.wal_recovery_mode: 2 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.enable_thread_tracking: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.enable_pipelined_write: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.unordered_write: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.write_thread_max_yield_usec: 100 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.row_cache: None 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.wal_filter: None 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.avoid_flush_during_recovery: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.allow_ingest_behind: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.two_write_queues: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.manual_wal_flush: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.wal_compression: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.atomic_flush: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.persist_stats_to_disk: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.write_dbid_to_manifest: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.log_readahead_size: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.file_checksum_gen_factory: Unknown 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.best_efforts_recovery: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.allow_data_in_errors: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.db_host_id: __hostname__ 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.enforce_single_del_contracts: true 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_background_jobs: 4 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_background_compactions: -1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_subcompactions: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.writable_file_max_buffer_size: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.delayed_write_rate : 16777216 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_total_wal_size: 1073741824 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.stats_dump_period_sec: 600 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.stats_persist_period_sec: 600 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.stats_history_buffer_size: 1048576 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_open_files: -1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bytes_per_sync: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.wal_bytes_per_sync: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.strict_bytes_per_sync: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_readahead_size: 2097152 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_background_flushes: -1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Compression algorithms supported: 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: kZSTD supported: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: kXpressCompression supported: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: kBZip2Compression supported: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: kZSTDNotFinalCompression supported: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: kLZ4Compression supported: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: kZlibCompression supported: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: kLZ4HCCompression supported: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: kSnappyCompression supported: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Fast CRC32 supported: Supported on x86 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: DMutex implementation: pthread_mutex_t 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: [db/db_impl/db_impl_readonly.cc:25] Opening the db in read only mode 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5527] Recovering from manifest file: db/MANIFEST-000005 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.merge_operator: .T:int64_array.b:bitwise_xor 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x3fa18c0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x3f1fe10 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [m-0]: 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x3fa18c0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x3f1fe10 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [m-1]: 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x3fa18c0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x3f1fe10 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [m-2]: 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x3fa18c0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x3f1fe10 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [p-0]: 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x3fa18c0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x3f1fe10 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [p-1]: 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x3fa18c0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x3f1fe10 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [p-2]: 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x3fa18c0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x3f1fe10 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [O-0]: 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x4020340) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x3f1f0f0 block_cache_name: BinnedLRUCache block_cache_options: capacity : 536870912 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [O-1]: 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x4020340) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x3f1f0f0 block_cache_name: BinnedLRUCache block_cache_options: capacity : 536870912 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [O-2]: 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x4020340) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x3f1f0f0 block_cache_name: BinnedLRUCache block_cache_options: capacity : 536870912 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: [db/column_family.cc:635] (skipping printing options) 2026-02-11T23:30:16.364+0000 7f7000069780 4 rocksdb: [db/column_family.cc:635] (skipping printing options) 2026-02-11T23:30:16.368+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5566] Recovered from manifest file:db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 29, last_sequence is 0, log_number is 4,prev_log_number is 0,max_column_family is 11,min_log_number_to_keep is 0 2026-02-11T23:30:16.368+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2026-02-11T23:30:16.368+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5581] Column family [m-0] (ID 1), log number is 4 2026-02-11T23:30:16.368+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5581] Column family [m-1] (ID 2), log number is 4 2026-02-11T23:30:16.368+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5581] Column family [m-2] (ID 3), log number is 4 2026-02-11T23:30:16.368+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5581] Column family [p-0] (ID 4), log number is 4 2026-02-11T23:30:16.368+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5581] Column family [p-1] (ID 5), log number is 4 2026-02-11T23:30:16.368+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5581] Column family [p-2] (ID 6), log number is 4 2026-02-11T23:30:16.368+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5581] Column family [O-0] (ID 7), log number is 4 2026-02-11T23:30:16.368+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5581] Column family [O-1] (ID 8), log number is 4 2026-02-11T23:30:16.368+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5581] Column family [O-2] (ID 9), log number is 4 2026-02-11T23:30:16.368+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5581] Column family [L] (ID 10), log number is 4 2026-02-11T23:30:16.368+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5581] Column family [P] (ID 11), log number is 4 2026-02-11T23:30:16.368+0000 7f7000069780 4 rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: d2a13b50-f500-4af9-be39-15a6598bf960 2026-02-11T23:30:16.368+0000 7f7000069780 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1770852616369065, "job": 1, "event": "recovery_started", "wal_files": [4]} 2026-02-11T23:30:16.368+0000 7f7000069780 4 rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2026-02-11T23:30:16.368+0000 7f7000069780 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1770852616369181, "job": 1, "event": "recovery_finished"} 2026-02-11T23:30:16.368+0000 7f7000069780 1 bluestore(td/osd-backfill-space/0) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-11T23:30:16.368+0000 7f7000069780 1 bluestore(td/osd-backfill-space/0) _fsck_on_open <<>> check (regular) start sb_tracker_hash_size:134217728 2026-02-11T23:30:16.417+0000 7f7000069780 1 bluefs fsck 2026-02-11T23:30:16.417+0000 7f7000069780 1 bluestore(td/osd-backfill-space/0) _fsck_on_open checking shared_blobs (phase 1) 2026-02-11T23:30:16.417+0000 7f7000069780 1 bluestore(td/osd-backfill-space/0) _fsck_on_open walking object keyspace 2026-02-11T23:30:16.419+0000 7f7000069780 1 bluestore(td/osd-backfill-space/0) _fsck_on_open checking shared_blobs (phase 2) 2026-02-11T23:30:16.429+0000 7f7000069780 1 bluestore(td/osd-backfill-space/0) _fsck_on_open checking pool_statfs 2026-02-11T23:30:16.429+0000 7f7000069780 1 bluestore(td/osd-backfill-space/0) _fsck_on_open checking for stray omap data 2026-02-11T23:30:16.429+0000 7f7000069780 1 bluestore(td/osd-backfill-space/0) _fsck_on_open checking deferred events 2026-02-11T23:30:16.429+0000 7f7000069780 1 bluestore(td/osd-backfill-space/0) _fsck_on_open <<>> with 0 errors, 0 warnings, 0 repaired, 0 remaining in 0.013304 seconds 2026-02-11T23:30:16.429+0000 7f6ff4207640 0 bluestore(td/osd-backfill-space/0) allocation stats probe 0: cnt: 0 frags: 0 size: 0 2026-02-11T23:30:16.429+0000 7f6ff4207640 0 bluestore(td/osd-backfill-space/0) probe -1: 0, 0, 0 2026-02-11T23:30:16.429+0000 7f6ff4207640 0 bluestore(td/osd-backfill-space/0) probe -2: 0, 0, 0 2026-02-11T23:30:16.429+0000 7f6ff4207640 0 bluestore(td/osd-backfill-space/0) probe -4: 0, 0, 0 2026-02-11T23:30:16.429+0000 7f6ff4207640 0 bluestore(td/osd-backfill-space/0) probe -8: 0, 0, 0 2026-02-11T23:30:16.429+0000 7f6ff4207640 0 bluestore(td/osd-backfill-space/0) probe -16: 0, 0, 0 2026-02-11T23:30:16.429+0000 7f6ff4207640 0 bluestore(td/osd-backfill-space/0) ------------ 2026-02-11T23:30:16.429+0000 7f7000069780 4 rocksdb: [db/db_impl/db_impl.cc:496] Shutdown: canceling all background work 2026-02-11T23:30:16.429+0000 7f7000069780 4 rocksdb: [db/db_impl/db_impl.cc:704] Shutdown complete 2026-02-11T23:30:16.429+0000 7f7000069780 1 bluefs umount 2026-02-11T23:30:16.429+0000 7f7000069780 1 bdev(0x3f43c00 td/osd-backfill-space/0/block) close 2026-02-11T23:30:16.449+0000 7f7000069780 1 freelist shutdown 2026-02-11T23:30:16.449+0000 7f7000069780 1 bdev(0x3f42c00 td/osd-backfill-space/0/block) close 2026-02-11T23:30:16.469+0000 7f7000069780 0 bluestore(td/osd-backfill-space/0) mkfs success 2026-02-11T23:30:16.469+0000 7f7000069780 1 bdev(0x3f42c00 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:16.469+0000 7f7000069780 0 bdev(0x3f42c00 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:16.469+0000 7f7000069780 1 bdev(0x3f42c00 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:16.469+0000 7f7000069780 1 bdev(0x3f42c00 td/osd-backfill-space/0/block) close 2026-02-11T23:30:16.489+0000 7f7000069780 1 bdev(0x3f42c00 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:16.489+0000 7f7000069780 0 bdev(0x3f42c00 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:16.489+0000 7f7000069780 1 bdev(0x3f42c00 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:16.489+0000 7f7000069780 1 bdev(0x3f42c00 td/osd-backfill-space/0/block) close 2026-02-11T23:30:16.510+0000 7f7000069780 1 bdev(0x3f42c00 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:16.510+0000 7f7000069780 0 bdev(0x3f42c00 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:16.510+0000 7f7000069780 1 bdev(0x3f42c00 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:16.510+0000 7f7000069780 1 bluestore(td/osd-backfill-space/0) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-11T23:30:16.510+0000 7f7000069780 1 bdev(0x3f43c00 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:16.510+0000 7f7000069780 0 bdev(0x3f43c00 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:16.510+0000 7f7000069780 1 bdev(0x3f43c00 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:16.510+0000 7f7000069780 1 bluefs add_block_device bdev 1 path td/osd-backfill-space/0/block size 100 GiB 2026-02-11T23:30:16.510+0000 7f7000069780 1 bluefs mount 2026-02-11T23:30:16.510+0000 7f7000069780 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-11T23:30:16.511+0000 7f7000069780 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:16.511+0000 7f7000069780 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:16.511+0000 7f7000069780 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:16.511+0000 7f7000069780 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:16.511+0000 7f7000069780 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:16.511+0000 7f7000069780 1 bluefs mount shared_bdev_used = 27197440 2026-02-11T23:30:16.511+0000 7f7000069780 1 bluestore(td/osd-backfill-space/0) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-11T23:30:16.512+0000 7f7000069780 1 rocksdb: SocketHook cannot register SocketHook 2026-02-11T23:30:16.512+0000 7f7000069780 1 rocksdb: SocketHook cannot register SocketHook 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: RocksDB version: 7.9.2 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Git sha 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Compile date 2026-02-11 10:30:18 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: DB SUMMARY 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: DB Session ID: J8E4HVLK4E8NZ5Y1PHHC 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: CURRENT file: CURRENT 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: IDENTITY file: IDENTITY 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: MANIFEST file: MANIFEST-000005 size: 605 Bytes 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: SST files in db dir, Total Num: 0, files: 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: SST files in db.slow dir, Total Num: 0, files: 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Write Ahead Log file in db.wal: 000004.log size: 276 ; 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.error_if_exists: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.create_if_missing: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.paranoid_checks: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.flush_verify_memtable_count: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.track_and_verify_wals_in_manifest: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.env: 0x402bf10 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.fs: LegacyFileSystem 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.info_log: 0x3fa0360 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_file_opening_threads: 16 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.statistics: (nil) 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.use_fsync: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_log_file_size: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_manifest_file_size: 1073741824 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.log_file_time_to_roll: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.keep_log_file_num: 1000 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.recycle_log_file_num: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.allow_fallocate: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.allow_mmap_reads: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.allow_mmap_writes: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.use_direct_reads: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.create_missing_column_families: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.db_log_dir: 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.wal_dir: db.wal 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.table_cache_numshardbits: 6 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.WAL_ttl_seconds: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.WAL_size_limit_MB: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.manifest_preallocation_size: 4194304 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.is_fd_close_on_exec: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.advise_random_on_open: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.db_write_buffer_size: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.write_buffer_manager: 0x3f3dea0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.access_hint_on_compaction_start: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.use_adaptive_mutex: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.rate_limiter: (nil) 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.wal_recovery_mode: 2 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.enable_thread_tracking: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.enable_pipelined_write: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.unordered_write: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.write_thread_max_yield_usec: 100 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.row_cache: None 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.wal_filter: None 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.avoid_flush_during_recovery: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.allow_ingest_behind: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.two_write_queues: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.manual_wal_flush: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.wal_compression: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.atomic_flush: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.persist_stats_to_disk: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.write_dbid_to_manifest: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.log_readahead_size: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.file_checksum_gen_factory: Unknown 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.best_efforts_recovery: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.allow_data_in_errors: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.db_host_id: __hostname__ 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.enforce_single_del_contracts: true 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_background_jobs: 4 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_background_compactions: -1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_subcompactions: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.writable_file_max_buffer_size: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.delayed_write_rate : 16777216 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_total_wal_size: 1073741824 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.stats_dump_period_sec: 600 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.stats_persist_period_sec: 600 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.stats_history_buffer_size: 1048576 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_open_files: -1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bytes_per_sync: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.wal_bytes_per_sync: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.strict_bytes_per_sync: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_readahead_size: 2097152 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_background_flushes: -1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Compression algorithms supported: 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: kZSTD supported: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: kXpressCompression supported: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: kBZip2Compression supported: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: kZSTDNotFinalCompression supported: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: kLZ4Compression supported: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: kZlibCompression supported: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: kLZ4HCCompression supported: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: kSnappyCompression supported: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Fast CRC32 supported: Supported on x86 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: DMutex implementation: pthread_mutex_t 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: [db/db_impl/db_impl_readonly.cc:25] Opening the db in read only mode 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5527] Recovering from manifest file: db/MANIFEST-000005 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.merge_operator: .T:int64_array.b:bitwise_xor 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x3f53ec0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x4da0d30 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [m-0]: 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x3f53ec0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x4da0d30 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [m-1]: 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x3f53ec0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x4da0d30 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [m-2]: 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x3f53ec0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x4da0d30 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [p-0]: 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x3f53ec0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x4da0d30 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:16.512+0000 7f7000069780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [p-1]: 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x3f53ec0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x4da0d30 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [p-2]: 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x3f53ec0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x4da0d30 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [O-0]: 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x3f90b20) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x4da0790 block_cache_name: BinnedLRUCache block_cache_options: capacity : 536870912 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [O-1]: 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x3f90b20) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x4da0790 block_cache_name: BinnedLRUCache block_cache_options: capacity : 536870912 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [O-2]: 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x3f90b20) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x4da0790 block_cache_name: BinnedLRUCache block_cache_options: capacity : 536870912 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: [db/column_family.cc:635] (skipping printing options) 2026-02-11T23:30:16.513+0000 7f7000069780 4 rocksdb: [db/column_family.cc:635] (skipping printing options) 2026-02-11T23:30:16.517+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5566] Recovered from manifest file:db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 29, last_sequence is 0, log_number is 4,prev_log_number is 0,max_column_family is 11,min_log_number_to_keep is 0 2026-02-11T23:30:16.517+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2026-02-11T23:30:16.517+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5581] Column family [m-0] (ID 1), log number is 4 2026-02-11T23:30:16.517+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5581] Column family [m-1] (ID 2), log number is 4 2026-02-11T23:30:16.517+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5581] Column family [m-2] (ID 3), log number is 4 2026-02-11T23:30:16.517+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5581] Column family [p-0] (ID 4), log number is 4 2026-02-11T23:30:16.517+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5581] Column family [p-1] (ID 5), log number is 4 2026-02-11T23:30:16.517+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5581] Column family [p-2] (ID 6), log number is 4 2026-02-11T23:30:16.517+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5581] Column family [O-0] (ID 7), log number is 4 2026-02-11T23:30:16.517+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5581] Column family [O-1] (ID 8), log number is 4 2026-02-11T23:30:16.517+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5581] Column family [O-2] (ID 9), log number is 4 2026-02-11T23:30:16.517+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5581] Column family [L] (ID 10), log number is 4 2026-02-11T23:30:16.517+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5581] Column family [P] (ID 11), log number is 4 2026-02-11T23:30:16.517+0000 7f7000069780 4 rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: d2a13b50-f500-4af9-be39-15a6598bf960 2026-02-11T23:30:16.517+0000 7f7000069780 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1770852616517827, "job": 1, "event": "recovery_started", "wal_files": [4]} 2026-02-11T23:30:16.517+0000 7f7000069780 4 rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2026-02-11T23:30:16.517+0000 7f7000069780 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1770852616518208, "job": 1, "event": "recovery_finished"} 2026-02-11T23:30:16.517+0000 7f7000069780 1 bluestore(td/osd-backfill-space/0) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-11T23:30:16.517+0000 7f7000069780 1 bluestore(td/osd-backfill-space/0) _open_super_meta old nid_max 0 2026-02-11T23:30:16.517+0000 7f7000069780 1 bluestore(td/osd-backfill-space/0) _open_super_meta old blobid_max 0 2026-02-11T23:30:16.517+0000 7f7000069780 1 bluestore(td/osd-backfill-space/0) _open_super_meta ondisk_format 4 compat_ondisk_format 3 2026-02-11T23:30:16.517+0000 7f7000069780 1 bluestore(td/osd-backfill-space/0) _open_super_meta min_alloc_size 0x1000 2026-02-11T23:30:16.517+0000 7f7000069780 1 freelist init 2026-02-11T23:30:16.517+0000 7f7000069780 1 freelist _read_cfg 2026-02-11T23:30:16.517+0000 7f7000069780 1 bluestore(td/osd-backfill-space/0) _open_fm effective freelist_type = null, freelist_alloc_size = 0x1000, min_alloc_size = 0x1000 2026-02-11T23:30:16.518+0000 7f7000069780 1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2026-02-11T23:30:16.518+0000 7f7000069780 1 fbmap_alloc 0x408dd40 shutdown 2026-02-11T23:30:16.518+0000 7f7000069780 0 bluestore(td/osd-backfill-space/0) _init_alloc::NCB::restore_allocator() failed! Run Full Recovery from ONodes (might take a while) ... 2026-02-11T23:30:16.518+0000 7f7000069780 1 bluestore::NCB::read_allocation_from_drive_on_startup::::Allocation Recovery was completed in 0.000187 seconds, extent_count=1 2026-02-11T23:30:16.518+0000 7f7000069780 1 bluestore(td/osd-backfill-space/0) _init_alloc loaded 0 B in 0 extents, allocator type hybrid, capacity 0x1900000000, block size 0x1000, free 0x18ffffe000, fragmentation 0 2026-02-11T23:30:16.518+0000 7f7000069780 4 rocksdb: [db/db_impl/db_impl.cc:496] Shutdown: canceling all background work 2026-02-11T23:30:16.518+0000 7f7000069780 4 rocksdb: [db/db_impl/db_impl.cc:704] Shutdown complete 2026-02-11T23:30:16.518+0000 7f7000069780 1 bluefs umount 2026-02-11T23:30:16.518+0000 7f7000069780 1 bdev(0x3f43c00 td/osd-backfill-space/0/block) close 2026-02-11T23:30:16.545+0000 7f7000069780 1 bdev(0x3f43c00 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:16.545+0000 7f7000069780 0 bdev(0x3f43c00 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:16.545+0000 7f7000069780 1 bdev(0x3f43c00 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:16.545+0000 7f7000069780 1 bluefs add_block_device bdev 1 path td/osd-backfill-space/0/block size 100 GiB 2026-02-11T23:30:16.545+0000 7f7000069780 1 bluefs mount 2026-02-11T23:30:16.545+0000 7f7000069780 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-11T23:30:16.545+0000 7f7000069780 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:16.545+0000 7f7000069780 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:16.545+0000 7f7000069780 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:16.545+0000 7f7000069780 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:16.545+0000 7f7000069780 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:16.545+0000 7f7000069780 1 bluefs mount shared_bdev_used = 27197440 2026-02-11T23:30:16.545+0000 7f7000069780 1 bluestore(td/osd-backfill-space/0) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-11T23:30:16.546+0000 7f7000069780 1 rocksdb: SocketHook cannot register SocketHook 2026-02-11T23:30:16.546+0000 7f7000069780 1 rocksdb: SocketHook cannot register SocketHook 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: RocksDB version: 7.9.2 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Git sha 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Compile date 2026-02-11 10:30:18 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: DB SUMMARY 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: DB Session ID: J8E4HVLK4E8NZ5Y1PHHJ 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: CURRENT file: CURRENT 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: IDENTITY file: IDENTITY 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: MANIFEST file: MANIFEST-000005 size: 605 Bytes 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: SST files in db dir, Total Num: 0, files: 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: SST files in db.slow dir, Total Num: 0, files: 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Write Ahead Log file in db.wal: 000004.log size: 276 ; 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.error_if_exists: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.create_if_missing: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.paranoid_checks: 1 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.flush_verify_memtable_count: 1 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.track_and_verify_wals_in_manifest: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.env: 0x3f5e9a0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.fs: LegacyFileSystem 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.info_log: 0x3fa1660 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_file_opening_threads: 16 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.statistics: (nil) 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.use_fsync: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_log_file_size: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_manifest_file_size: 1073741824 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.log_file_time_to_roll: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.keep_log_file_num: 1000 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.recycle_log_file_num: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.allow_fallocate: 1 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.allow_mmap_reads: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.allow_mmap_writes: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.use_direct_reads: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.create_missing_column_families: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.db_log_dir: 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.wal_dir: db.wal 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.table_cache_numshardbits: 6 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.WAL_ttl_seconds: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.WAL_size_limit_MB: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.manifest_preallocation_size: 4194304 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.is_fd_close_on_exec: 1 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.advise_random_on_open: 1 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.db_write_buffer_size: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.write_buffer_manager: 0x3f3dea0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.access_hint_on_compaction_start: 1 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.use_adaptive_mutex: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.rate_limiter: (nil) 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.wal_recovery_mode: 2 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.enable_thread_tracking: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.enable_pipelined_write: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.unordered_write: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.write_thread_max_yield_usec: 100 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.row_cache: None 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.wal_filter: None 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.avoid_flush_during_recovery: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.allow_ingest_behind: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.two_write_queues: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.manual_wal_flush: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.wal_compression: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.atomic_flush: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.persist_stats_to_disk: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.write_dbid_to_manifest: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.log_readahead_size: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.file_checksum_gen_factory: Unknown 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.best_efforts_recovery: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.allow_data_in_errors: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.db_host_id: __hostname__ 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.enforce_single_del_contracts: true 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_background_jobs: 4 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_background_compactions: -1 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_subcompactions: 1 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.writable_file_max_buffer_size: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.delayed_write_rate : 16777216 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_total_wal_size: 1073741824 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.stats_dump_period_sec: 600 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.stats_persist_period_sec: 600 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.stats_history_buffer_size: 1048576 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_open_files: -1 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.bytes_per_sync: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.wal_bytes_per_sync: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.strict_bytes_per_sync: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compaction_readahead_size: 2097152 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_background_flushes: -1 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Compression algorithms supported: 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: kZSTD supported: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: kXpressCompression supported: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: kBZip2Compression supported: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: kZSTDNotFinalCompression supported: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: kLZ4Compression supported: 1 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: kZlibCompression supported: 1 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: kLZ4HCCompression supported: 1 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: kSnappyCompression supported: 1 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Fast CRC32 supported: Supported on x86 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: DMutex implementation: pthread_mutex_t 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5527] Recovering from manifest file: db/MANIFEST-000005 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.merge_operator: .T:int64_array.b:bitwise_xor 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x3faec40) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x4da03d0 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [m-0]: 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x3faec40) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x4da03d0 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [m-1]: 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x3faec40) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x4da03d0 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:16.546+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [m-2]: 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x3faec40) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x4da03d0 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [p-0]: 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x3faec40) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x4da03d0 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [p-1]: 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x3faec40) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x4da03d0 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [p-2]: 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x3faec40) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x4da03d0 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [O-0]: 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x3fbd1c0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x4da0d30 block_cache_name: BinnedLRUCache block_cache_options: capacity : 536870912 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [O-1]: 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x3fbd1c0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x4da0d30 block_cache_name: BinnedLRUCache block_cache_options: capacity : 536870912 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [O-2]: 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x3fbd1c0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x4da0d30 block_cache_name: BinnedLRUCache block_cache_options: capacity : 536870912 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: [db/column_family.cc:635] (skipping printing options) 2026-02-11T23:30:16.547+0000 7f7000069780 4 rocksdb: [db/column_family.cc:635] (skipping printing options) 2026-02-11T23:30:16.551+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5566] Recovered from manifest file:db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 29, last_sequence is 0, log_number is 4,prev_log_number is 0,max_column_family is 11,min_log_number_to_keep is 0 2026-02-11T23:30:16.551+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2026-02-11T23:30:16.551+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5581] Column family [m-0] (ID 1), log number is 4 2026-02-11T23:30:16.551+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5581] Column family [m-1] (ID 2), log number is 4 2026-02-11T23:30:16.551+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5581] Column family [m-2] (ID 3), log number is 4 2026-02-11T23:30:16.551+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5581] Column family [p-0] (ID 4), log number is 4 2026-02-11T23:30:16.551+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5581] Column family [p-1] (ID 5), log number is 4 2026-02-11T23:30:16.551+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5581] Column family [p-2] (ID 6), log number is 4 2026-02-11T23:30:16.551+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5581] Column family [O-0] (ID 7), log number is 4 2026-02-11T23:30:16.551+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5581] Column family [O-1] (ID 8), log number is 4 2026-02-11T23:30:16.551+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5581] Column family [O-2] (ID 9), log number is 4 2026-02-11T23:30:16.551+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5581] Column family [L] (ID 10), log number is 4 2026-02-11T23:30:16.551+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5581] Column family [P] (ID 11), log number is 4 2026-02-11T23:30:16.551+0000 7f7000069780 4 rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: d2a13b50-f500-4af9-be39-15a6598bf960 2026-02-11T23:30:16.551+0000 7f7000069780 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1770852616551927, "job": 1, "event": "recovery_started", "wal_files": [4]} 2026-02-11T23:30:16.551+0000 7f7000069780 4 rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2026-02-11T23:30:16.552+0000 7f7000069780 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1770852616552395, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 30, "file_size": 1412, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 11, "table_properties": {"data_size": 262, "index_size": 33, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 243, "raw_average_key_size": 22, "raw_value_size": 69, "raw_average_value_size": 6, "num_data_blocks": 1, "num_entries": 11, "num_filter_entries": 11, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": ".T:int64_array.b:bitwise_xor", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "LZ4", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1770852616, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d2a13b50-f500-4af9-be39-15a6598bf960", "db_session_id": "J8E4HVLK4E8NZ5Y1PHHJ", "orig_file_number": 30, "seqno_to_time_mapping": "N/A"}} 2026-02-11T23:30:16.552+0000 7f7000069780 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1770852616552438, "job": 1, "event": "recovery_finished"} 2026-02-11T23:30:16.552+0000 7f7000069780 4 rocksdb: [db/version_set.cc:5047] Creating manifest 32 2026-02-11T23:30:16.554+0000 7f7000069780 4 rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x4ebb500 2026-02-11T23:30:16.554+0000 7f7000069780 4 rocksdb: DB pointer 0x3fd0000 2026-02-11T23:30:16.554+0000 7f7000069780 1 bluestore(td/osd-backfill-space/0) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-11T23:30:16.555+0000 7f7000069780 1 bluestore(td/osd-backfill-space/0) _upgrade_super from 4, latest 4 2026-02-11T23:30:16.555+0000 7f7000069780 1 bluestore(td/osd-backfill-space/0) _upgrade_super done 2026-02-11T23:30:16.555+0000 7f7000069780 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e0 maps [] lci=[0,0] tlb=0) 2026-02-11T23:30:16.565+0000 7f6ffaa14640 0 bluestore(td/osd-backfill-space/0) allocation stats probe 1: cnt: 0 frags: 0 size: 0 2026-02-11T23:30:16.565+0000 7f6ffaa14640 0 bluestore(td/osd-backfill-space/0) probe -1: 0, 0, 0 2026-02-11T23:30:16.565+0000 7f6ffaa14640 0 bluestore(td/osd-backfill-space/0) probe -3: 0, 0, 0 2026-02-11T23:30:16.565+0000 7f6ffaa14640 0 bluestore(td/osd-backfill-space/0) probe -5: 0, 0, 0 2026-02-11T23:30:16.565+0000 7f6ffaa14640 0 bluestore(td/osd-backfill-space/0) probe -9: 0, 0, 0 2026-02-11T23:30:16.565+0000 7f6ffaa14640 0 bluestore(td/osd-backfill-space/0) probe -17: 0, 0, 0 2026-02-11T23:30:16.565+0000 7f6ffaa14640 0 bluestore(td/osd-backfill-space/0) ------------ 2026-02-11T23:30:16.565+0000 7f7000069780 4 rocksdb: [db/db_impl/db_impl.cc:496] Shutdown: canceling all background work 2026-02-11T23:30:16.565+0000 7f7000069780 4 rocksdb: [db/db_impl/db_impl.cc:704] Shutdown complete 2026-02-11T23:30:16.568+0000 7f7000069780 1 fbmap_alloc 0x408d7a0 shutdown 2026-02-11T23:30:16.568+0000 7f7000069780 1 bluefs umount 2026-02-11T23:30:16.568+0000 7f7000069780 1 bdev(0x3f43c00 td/osd-backfill-space/0/block) close 2026-02-11T23:30:16.590+0000 7f7000069780 1 freelist shutdown 2026-02-11T23:30:16.590+0000 7f7000069780 1 bdev(0x3f42c00 td/osd-backfill-space/0/block) close 2026-02-11T23:30:16.628+0000 7f7000069780 0 created object store td/osd-backfill-space/0 for osd.0 fsid cb0a9481-482d-477a-a81c-1446b6498370 2026-02-11T23:30:16.759+0000 7f416e42d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-11T23:30:16.759+0000 7f416e42d780 10 monclient: get_monmap_and_config 2026-02-11T23:30:16.759+0000 7f416e42d780 10 monclient: build_initial_monmap 2026-02-11T23:30:16.759+0000 7f416e42d780 10 monclient: monmap: epoch 0 fsid cb0a9481-482d-477a-a81c-1446b6498370 last_changed 2026-02-11T23:30:16.760143+0000 created 2026-02-11T23:30:16.760143+0000 min_mon_release 0 (unknown) election_strategy: 1 0: v1:127.0.0.1:7180/0 mon.noname-a-legacy 1: v2:127.0.0.1:7180/0 mon.noname-a 2026-02-11T23:30:16.760+0000 7f416e42d780 1 Processor -- start 2026-02-11T23:30:16.760+0000 7f416e42d780 1 -- start start 2026-02-11T23:30:16.760+0000 7f416e42d780 10 monclient: init 2026-02-11T23:30:16.760+0000 7f416e42d780 10 monclient: _reopen_session rank -1 2026-02-11T23:30:16.760+0000 7f416e42d780 10 monclient: _add_conns ranks=[1,0] 2026-02-11T23:30:16.760+0000 7f416e42d780 1 --2- >> v2:127.0.0.1:7180/0 conn(0x3699c00 0x3856000 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-11T23:30:16.760+0000 7f416e42d780 10 monclient(hunting): picked mon.noname-a con 0x3699c00 addr v2:127.0.0.1:7180/0 2026-02-11T23:30:16.760+0000 7f416e42d780 10 monclient(hunting): picked mon.noname-a-legacy con 0x44e2000 addr v1:127.0.0.1:7180/0 2026-02-11T23:30:16.760+0000 7f416e42d780 1 -- --> v1:127.0.0.1:7180/0 -- auth(proto 0 26 bytes epoch 0) -- 0x3702f00 con 0x44e2000 2026-02-11T23:30:16.760+0000 7f416e42d780 10 monclient(hunting): start opening mon connection 2026-02-11T23:30:16.760+0000 7f416e42d780 1 -- --> v2:127.0.0.1:7180/0 -- mon_getmap magic: 0 -- 0x371d180 con 0x3699c00 2026-02-11T23:30:16.760+0000 7f416e42d780 10 monclient(hunting): _renew_subs 2026-02-11T23:30:16.760+0000 7f416e42d780 10 monclient(hunting): authenticate will time out at 1274.448380s 2026-02-11T23:30:16.760+0000 7f416bbfc640 1 --2- >> v2:127.0.0.1:7180/0 conn(0x3699c00 0x3856000 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:16.760+0000 7f416bbfc640 1 --2- >> v2:127.0.0.1:7180/0 conn(0x3699c00 0x3856000 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:127.0.0.1:7180/0 says I am v2:127.0.0.1:41140/0 (socket says 127.0.0.1:41140) 2026-02-11T23:30:16.760+0000 7f416bbfc640 1 -- 127.0.0.1:0/3017891150 learned_addr learned my addr 127.0.0.1:0/3017891150 (peer_addr_for_me v2:127.0.0.1:0/0) 2026-02-11T23:30:16.760+0000 7f416bbfc640 10 monclient(hunting): get_auth_request con 0x3699c00 auth_method 0 2026-02-11T23:30:16.760+0000 7f416bbfc640 10 monclient(hunting): get_auth_request method 1 preferred_modes [1] 2026-02-11T23:30:16.760+0000 7f416bbfc640 10 monclient(hunting): _init_auth method 1 2026-02-11T23:30:16.760+0000 7f416bbfc640 10 monclient(hunting): _init_auth creating new auth 2026-02-11T23:30:16.760+0000 7f416bbfc640 10 monclient(hunting): handle_auth_done global_id 4106 payload 0 2026-02-11T23:30:16.760+0000 7f416bbfc640 1 -- 127.0.0.1:0/3017891150 >> v1:127.0.0.1:7180/0 conn(0x44e2000 legacy=0x3823000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-11T23:30:16.760+0000 7f416bbfc640 10 monclient: _finish_hunting 0 2026-02-11T23:30:16.760+0000 7f416bbfc640 1 monclient: found mon.noname-a 2026-02-11T23:30:16.760+0000 7f416bbfc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-11T23:30:16.760+0000 7f416bbfc640 10 monclient: _send_mon_message to mon.noname-a at v2:127.0.0.1:7180/0 2026-02-11T23:30:16.760+0000 7f416bbfc640 1 -- 127.0.0.1:0/3017891150 --> v2:127.0.0.1:7180/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x36fcea0 con 0x3699c00 2026-02-11T23:30:16.760+0000 7f416bbfc640 1 --2- 127.0.0.1:0/3017891150 >> v2:127.0.0.1:7180/0 conn(0x3699c00 0x3856000 crc :-1 s=READY pgs=10 gs=2 cs=0 l=1 c_cookie=2382342441816679550 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=210fc7d421d1e47e server_cookie=0 in_seq=0 out_seq=0 2026-02-11T23:30:16.760+0000 7f416abfa640 1 -- 127.0.0.1:0/3017891150 <== mon.0 v2:127.0.0.1:7180/0 1 ==== mon_map magic: 0 ==== 178+0+0 (crc 0 0 0) 0x3700dc0 con 0x3699c00 2026-02-11T23:30:16.760+0000 7f416abfa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-11T23:30:16.760+0000 7f416abfa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-11T23:30:16.760+0000 7f416abfa640 10 monclient: dump: epoch 1 fsid cb0a9481-482d-477a-a81c-1446b6498370 last_changed 2026-02-11T23:30:15.835295+0000 created 2026-02-11T23:30:15.835295+0000 min_mon_release 20 (tentacle) election_strategy: 1 0: v2:127.0.0.1:7180/0 mon.a 2026-02-11T23:30:16.760+0000 7f416abfa640 10 monclient: _finish_auth 0 2026-02-11T23:30:16.760+0000 7f416abfa640 10 monclient: _check_auth_tickets 2026-02-11T23:30:16.760+0000 7f416abfa640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:29:46.761156+0000) 2026-02-11T23:30:16.760+0000 7f416abfa640 1 -- 127.0.0.1:0/3017891150 <== mon.0 v2:127.0.0.1:7180/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x3702f00 con 0x3699c00 2026-02-11T23:30:16.760+0000 7f416e42d780 5 monclient: authenticate success, global_id 4106 2026-02-11T23:30:16.760+0000 7f416e42d780 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-11T23:30:16.760+0000 7f416abfa640 10 monclient: handle_config config(0 keys) 2026-02-11T23:30:16.760+0000 7f416abfa640 1 -- 127.0.0.1:0/3017891150 <== mon.0 v2:127.0.0.1:7180/0 3 ==== mon_map magic: 0 ==== 178+0+0 (crc 0 0 0) 0x3701080 con 0x3699c00 2026-02-11T23:30:16.760+0000 7f416e42d780 10 monclient: get_monmap_and_config success 2026-02-11T23:30:16.760+0000 7f416e42d780 4 set_mon_vals no callback set 2026-02-11T23:30:16.760+0000 7f416abfa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-11T23:30:16.760+0000 7f416abfa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-11T23:30:16.760+0000 7f416abfa640 10 monclient: dump: epoch 1 fsid cb0a9481-482d-477a-a81c-1446b6498370 last_changed 2026-02-11T23:30:15.835295+0000 created 2026-02-11T23:30:15.835295+0000 min_mon_release 20 (tentacle) election_strategy: 1 0: v2:127.0.0.1:7180/0 mon.a 2026-02-11T23:30:16.761+0000 7f416e42d780 10 monclient: shutdown 2026-02-11T23:30:16.761+0000 7f416e42d780 20 monclient: shutdown discarding 0 pending message(s) 2026-02-11T23:30:16.761+0000 7f416e42d780 1 -- 127.0.0.1:0/3017891150 >> v2:127.0.0.1:7180/0 conn(0x3699c00 msgr2=0x3856000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-11T23:30:16.761+0000 7f416e42d780 1 --2- 127.0.0.1:0/3017891150 >> v2:127.0.0.1:7180/0 conn(0x3699c00 0x3856000 crc :-1 s=READY pgs=10 gs=2 cs=0 l=1 c_cookie=2382342441816679550 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-11T23:30:16.761+0000 7f416e42d780 1 -- 127.0.0.1:0/3017891150 shutdown_connections 2026-02-11T23:30:16.761+0000 7f416e42d780 1 --2- 127.0.0.1:0/3017891150 >> v2:127.0.0.1:7180/0 conn(0x3699c00 0x3856000 unknown :-1 s=CLOSED pgs=10 gs=2 cs=0 l=1 c_cookie=2382342441816679550 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-11T23:30:16.761+0000 7f416e42d780 1 -- 127.0.0.1:0/3017891150 >> 127.0.0.1:0/3017891150 conn(0x3699800 msgr2=0x3822800 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-11T23:30:16.761+0000 7f416e42d780 1 -- 127.0.0.1:0/3017891150 shutdown_connections 2026-02-11T23:30:16.761+0000 7f416e42d780 1 -- 127.0.0.1:0/3017891150 wait complete. 2026-02-11T23:30:16.761+0000 7f416e42d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-11T23:30:16.761+0000 7f416e42d780 0 ceph version 20.3.0-5256-gf2a98edf (f2a98edfc002a9c9c8df236a812719da23bb2dd0) tentacle (dev - RelWithDebInfo), process ceph-osd, pid 315557 2026-02-11T23:30:16.763+0000 7f416e42d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-11T23:30:16.763+0000 7f416e42d780 5 object store type is bluestore 2026-02-11T23:30:16.763+0000 7f416e42d780 1 bdev(0x3698c00 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:16.763+0000 7f416e42d780 0 bdev(0x3698c00 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:16.764+0000 7f416e42d780 1 bdev(0x3698c00 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:16.764+0000 7f416e42d780 1 bdev(0x3698c00 td/osd-backfill-space/0/block) close 2026-02-11T23:30:16.796+0000 7f416e42d780 1 bdev(0x3698c00 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:16.796+0000 7f416e42d780 0 bdev(0x3698c00 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:16.796+0000 7f416e42d780 1 bdev(0x3698c00 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:16.796+0000 7f416e42d780 1 bdev(0x3698c00 td/osd-backfill-space/0/block) close 2026-02-11T23:30:16.823+0000 7f416e42d780 1 bdev(0x3698c00 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:16.823+0000 7f416e42d780 0 bdev(0x3698c00 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:16.823+0000 7f416e42d780 1 bdev(0x3698c00 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:16.823+0000 7f416e42d780 1 bdev(0x3698c00 td/osd-backfill-space/0/block) close 2026-02-11T23:30:16.847+0000 7f416e42d780 1 bdev(0x3698c00 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:16.847+0000 7f416e42d780 0 bdev(0x3698c00 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:16.847+0000 7f416e42d780 1 bdev(0x3698c00 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:16.847+0000 7f416e42d780 1 bdev(0x3698c00 td/osd-backfill-space/0/block) close 2026-02-11T23:30:16.875+0000 7f416e42d780 1 bdev(0x3698c00 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:16.875+0000 7f416e42d780 0 bdev(0x3698c00 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:16.875+0000 7f416e42d780 1 bdev(0x3698c00 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:16.875+0000 7f416e42d780 1 bdev(0x3698c00 td/osd-backfill-space/0/block) close 2026-02-11T23:30:16.897+0000 7f416e42d780 1 bdev(0x3698c00 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:16.897+0000 7f416e42d780 0 bdev(0x3698c00 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:16.897+0000 7f416e42d780 1 bdev(0x3698c00 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:16.897+0000 7f416e42d780 1 bluestore(td/osd-backfill-space/0) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-11T23:30:16.897+0000 7f416e42d780 1 bdev(0x3699c00 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:16.897+0000 7f416e42d780 0 bdev(0x3699c00 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:16.897+0000 7f416e42d780 1 bdev(0x3699c00 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:16.897+0000 7f416e42d780 1 bluefs add_block_device bdev 1 path td/osd-backfill-space/0/block size 100 GiB 2026-02-11T23:30:16.897+0000 7f416e42d780 1 bdev(0x3699c00 td/osd-backfill-space/0/block) close 2026-02-11T23:30:16.922+0000 7f416e42d780 1 bdev(0x3698c00 td/osd-backfill-space/0/block) close 2026-02-11T23:30:16.950+0000 7f416e42d780 1 objectstore numa_node 0 2026-02-11T23:30:16.950+0000 7f416e42d780 0 starting osd.0 osd_data td/osd-backfill-space/0 td/osd-backfill-space/0/journal 2026-02-11T23:30:16.950+0000 7f416e42d780 10 picked public_addrs [v2:0.0.0.0:0/0,v1:0.0.0.0:0/0] 2026-02-11T23:30:16.951+0000 7f416e42d780 10 there is no public_bind_addrs, defaulting to public_addrs 2026-02-11T23:30:16.951+0000 7f416e42d780 -1 Falling back to public interface 2026-02-11T23:30:16.951+0000 7f416e42d780 1 -- [v2:0.0.0.0:6800/3294232400,v1:0.0.0.0:6801/3294232400] _finish_bind bind my_addrs is [v2:0.0.0.0:6800/3294232400,v1:0.0.0.0:6801/3294232400] 2026-02-11T23:30:16.951+0000 7f416e42d780 1 -- [v2:0.0.0.0:6802/3294232400,v1:0.0.0.0:6803/3294232400] _finish_bind bind my_addrs is [v2:0.0.0.0:6802/3294232400,v1:0.0.0.0:6803/3294232400] 2026-02-11T23:30:16.951+0000 7f416e42d780 1 -- [v2:0.0.0.0:6804/3294232400,v1:0.0.0.0:6805/3294232400] _finish_bind bind my_addrs is [v2:0.0.0.0:6804/3294232400,v1:0.0.0.0:6805/3294232400] 2026-02-11T23:30:16.951+0000 7f416e42d780 1 -- [v2:0.0.0.0:6806/3294232400,v1:0.0.0.0:6807/3294232400] _finish_bind bind my_addrs is [v2:0.0.0.0:6806/3294232400,v1:0.0.0.0:6807/3294232400] 2026-02-11T23:30:16.951+0000 7f416e42d780 10 monclient: build_initial_monmap 2026-02-11T23:30:16.951+0000 7f416e42d780 10 monclient: monmap: epoch 0 fsid 00000000-0000-0000-0000-000000000000 last_changed 0.000000 created 0.000000 min_mon_release 0 (unknown) election_strategy: 1 0: v2:127.0.0.1:7180/0 mon.noname-a 2026-02-11T23:30:16.953+0000 7f416e42d780 0 load: jerasure load: lrc 2026-02-11T23:30:16.953+0000 7f416e42d780 1 bdev(0x3767400 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:16.953+0000 7f416e42d780 0 bdev(0x3767400 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:16.954+0000 7f416e42d780 1 bdev(0x3767400 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:16.954+0000 7f416e42d780 1 bluestore(td/osd-backfill-space/0) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-11T23:30:16.954+0000 7f416e42d780 1 bdev(0x3767400 td/osd-backfill-space/0/block) close 2026-02-11T23:30:16.985+0000 7f416e42d780 1 bdev(0x3767400 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:16.985+0000 7f416e42d780 0 bdev(0x3767400 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:16.985+0000 7f416e42d780 1 bdev(0x3767400 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:16.985+0000 7f416e42d780 1 bluestore(td/osd-backfill-space/0) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-11T23:30:16.985+0000 7f416e42d780 1 bdev(0x3767400 td/osd-backfill-space/0/block) close 2026-02-11T23:30:17.012+0000 7f416e42d780 1 mclock_common: set_from_config: osd_bandwidth_cost_per_io: 58525.17 bytes/io, osd_bandwidth_capacity_per_shard 157286400.00 bytes/second 2026-02-11T23:30:17.012+0000 7f416e42d780 0 osd.0:0.OSDShard using op scheduler mclock_scheduler, cutoff=196 2026-02-11T23:30:17.012+0000 7f416e42d780 1 bdev(0x3767400 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:17.012+0000 7f416e42d780 0 bdev(0x3767400 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:17.012+0000 7f416e42d780 1 bdev(0x3767400 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:17.012+0000 7f416e42d780 1 bluestore(td/osd-backfill-space/0) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-11T23:30:17.012+0000 7f416e42d780 1 bdev(0x3767400 td/osd-backfill-space/0/block) close 2026-02-11T23:30:17.048+0000 7f416e42d780 1 mclock_common: set_from_config: osd_bandwidth_cost_per_io: 58525.17 bytes/io, osd_bandwidth_capacity_per_shard 157286400.00 bytes/second 2026-02-11T23:30:17.048+0000 7f416e42d780 0 osd.0:1.OSDShard using op scheduler mclock_scheduler, cutoff=196 2026-02-11T23:30:17.048+0000 7f416e42d780 1 bdev(0x3767400 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:17.048+0000 7f416e42d780 0 bdev(0x3767400 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:17.048+0000 7f416e42d780 1 bdev(0x3767400 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:17.048+0000 7f416e42d780 1 bluestore(td/osd-backfill-space/0) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-11T23:30:17.048+0000 7f416e42d780 1 bdev(0x3767400 td/osd-backfill-space/0/block) close 2026-02-11T23:30:17.080+0000 7f416e42d780 1 mclock_common: set_from_config: osd_bandwidth_cost_per_io: 58525.17 bytes/io, osd_bandwidth_capacity_per_shard 157286400.00 bytes/second 2026-02-11T23:30:17.080+0000 7f416e42d780 0 osd.0:2.OSDShard using op scheduler mclock_scheduler, cutoff=196 2026-02-11T23:30:17.080+0000 7f416e42d780 1 bdev(0x3767400 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:17.080+0000 7f416e42d780 0 bdev(0x3767400 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:17.080+0000 7f416e42d780 1 bdev(0x3767400 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:17.080+0000 7f416e42d780 1 bluestore(td/osd-backfill-space/0) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-11T23:30:17.080+0000 7f416e42d780 1 bdev(0x3767400 td/osd-backfill-space/0/block) close 2026-02-11T23:30:17.109+0000 7f416e42d780 1 mclock_common: set_from_config: osd_bandwidth_cost_per_io: 58525.17 bytes/io, osd_bandwidth_capacity_per_shard 157286400.00 bytes/second 2026-02-11T23:30:17.109+0000 7f416e42d780 0 osd.0:3.OSDShard using op scheduler mclock_scheduler, cutoff=196 2026-02-11T23:30:17.109+0000 7f416e42d780 1 bdev(0x3767400 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:17.109+0000 7f416e42d780 0 bdev(0x3767400 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:17.109+0000 7f416e42d780 1 bdev(0x3767400 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:17.109+0000 7f416e42d780 1 bluestore(td/osd-backfill-space/0) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-11T23:30:17.109+0000 7f416e42d780 1 bdev(0x3767400 td/osd-backfill-space/0/block) close 2026-02-11T23:30:17.132+0000 7f416e42d780 1 mclock_common: set_from_config: osd_bandwidth_cost_per_io: 58525.17 bytes/io, osd_bandwidth_capacity_per_shard 157286400.00 bytes/second 2026-02-11T23:30:17.132+0000 7f416e42d780 0 osd.0:4.OSDShard using op scheduler mclock_scheduler, cutoff=196 2026-02-11T23:30:17.132+0000 7f416e42d780 1 bdev(0x3767400 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:17.132+0000 7f416e42d780 0 bdev(0x3767400 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:17.132+0000 7f416e42d780 1 bdev(0x3767400 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:17.132+0000 7f416e42d780 1 bluestore(td/osd-backfill-space/0) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-11T23:30:17.132+0000 7f416e42d780 1 bdev(0x3767400 td/osd-backfill-space/0/block) close 2026-02-11T23:30:17.155+0000 7f416e42d780 1 mclock_common: set_from_config: osd_bandwidth_cost_per_io: 58525.17 bytes/io, osd_bandwidth_capacity_per_shard 157286400.00 bytes/second 2026-02-11T23:30:17.155+0000 7f416e42d780 0 osd.0:5.OSDShard using op scheduler mclock_scheduler, cutoff=196 2026-02-11T23:30:17.155+0000 7f416e42d780 1 bdev(0x3767400 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:17.155+0000 7f416e42d780 0 bdev(0x3767400 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:17.155+0000 7f416e42d780 1 bdev(0x3767400 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:17.155+0000 7f416e42d780 1 bluestore(td/osd-backfill-space/0) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-11T23:30:17.155+0000 7f416e42d780 1 bdev(0x3767400 td/osd-backfill-space/0/block) close 2026-02-11T23:30:17.184+0000 7f416e42d780 1 mclock_common: set_from_config: osd_bandwidth_cost_per_io: 58525.17 bytes/io, osd_bandwidth_capacity_per_shard 157286400.00 bytes/second 2026-02-11T23:30:17.184+0000 7f416e42d780 0 osd.0:6.OSDShard using op scheduler mclock_scheduler, cutoff=196 2026-02-11T23:30:17.184+0000 7f416e42d780 1 bdev(0x3767400 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:17.184+0000 7f416e42d780 0 bdev(0x3767400 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:17.184+0000 7f416e42d780 1 bdev(0x3767400 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:17.184+0000 7f416e42d780 1 bluestore(td/osd-backfill-space/0) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-11T23:30:17.184+0000 7f416e42d780 1 bdev(0x3767400 td/osd-backfill-space/0/block) close 2026-02-11T23:30:17.204+0000 7f416e42d780 1 mclock_common: set_from_config: osd_bandwidth_cost_per_io: 58525.17 bytes/io, osd_bandwidth_capacity_per_shard 157286400.00 bytes/second 2026-02-11T23:30:17.204+0000 7f416e42d780 0 osd.0:7.OSDShard using op scheduler mclock_scheduler, cutoff=196 2026-02-11T23:30:17.204+0000 7f416e42d780 1 -- [v2:0.0.0.0:6800/3294232400,v1:0.0.0.0:6801/3294232400] start start 2026-02-11T23:30:17.204+0000 7f416e42d780 1 -- start start 2026-02-11T23:30:17.204+0000 7f416e42d780 1 -- start start 2026-02-11T23:30:17.204+0000 7f416e42d780 1 -- [v2:0.0.0.0:6804/3294232400,v1:0.0.0.0:6805/3294232400] start start 2026-02-11T23:30:17.204+0000 7f416e42d780 1 -- [v2:0.0.0.0:6806/3294232400,v1:0.0.0.0:6807/3294232400] start start 2026-02-11T23:30:17.204+0000 7f416e42d780 1 -- [v2:0.0.0.0:6802/3294232400,v1:0.0.0.0:6803/3294232400] start start 2026-02-11T23:30:17.204+0000 7f416e42d780 1 -- start start 2026-02-11T23:30:17.204+0000 7f416e42d780 1 bdev(0x3767400 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:17.204+0000 7f416e42d780 0 bdev(0x3767400 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:17.204+0000 7f416e42d780 1 bdev(0x3767400 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:17.204+0000 7f416e42d780 1 bdev(0x3767400 td/osd-backfill-space/0/block) close 2026-02-11T23:30:17.228+0000 7f416e42d780 2 osd.0 0 init td/osd-backfill-space/0 (looks like ssd) 2026-02-11T23:30:17.228+0000 7f416e42d780 2 osd.0 0 journal td/osd-backfill-space/0/journal 2026-02-11T23:30:17.228+0000 7f416e42d780 1 bdev(0x3767400 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:17.228+0000 7f416e42d780 0 bdev(0x3767400 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:17.228+0000 7f416e42d780 1 bdev(0x3767400 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:17.228+0000 7f416e42d780 1 bdev(0x3767400 td/osd-backfill-space/0/block) close 2026-02-11T23:30:17.256+0000 7f416e42d780 1 bdev(0x3767400 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:17.256+0000 7f416e42d780 0 bdev(0x3767400 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:17.256+0000 7f416e42d780 1 bdev(0x3767400 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:17.256+0000 7f416e42d780 1 bdev(0x3767400 td/osd-backfill-space/0/block) close 2026-02-11T23:30:17.276+0000 7f416e42d780 1 bdev(0x3767400 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:17.276+0000 7f416e42d780 0 bdev(0x3767400 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:17.276+0000 7f416e42d780 1 bdev(0x3767400 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:17.276+0000 7f416e42d780 1 bluestore(td/osd-backfill-space/0) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-11T23:30:17.276+0000 7f416e42d780 1 bdev(0x4640800 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:17.276+0000 7f416e42d780 0 bdev(0x4640800 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:17.276+0000 7f416e42d780 1 bdev(0x4640800 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:17.276+0000 7f416e42d780 1 bluefs add_block_device bdev 1 path td/osd-backfill-space/0/block size 100 GiB 2026-02-11T23:30:17.276+0000 7f416e42d780 1 bluefs mount 2026-02-11T23:30:17.276+0000 7f416e42d780 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-11T23:30:17.279+0000 7f416e42d780 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:17.279+0000 7f416e42d780 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:17.279+0000 7f416e42d780 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:17.279+0000 7f416e42d780 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:17.279+0000 7f416e42d780 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:17.279+0000 7f416e42d780 1 bluefs mount shared_bdev_used = 0 2026-02-11T23:30:17.279+0000 7f416e42d780 1 bluestore(td/osd-backfill-space/0) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: RocksDB version: 7.9.2 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Git sha 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Compile date 2026-02-11 10:30:18 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: DB SUMMARY 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: DB Session ID: HFPZG9ZUDMGBEKHWSHGY 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: CURRENT file: CURRENT 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: IDENTITY file: IDENTITY 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: MANIFEST file: MANIFEST-000032 size: 1012 Bytes 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: SST files in db dir, Total Num: 1, files: 000030.sst 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: SST files in db.slow dir, Total Num: 0, files: 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Write Ahead Log file in db.wal: 000031.log size: 950 ; 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.error_if_exists: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.create_if_missing: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.paranoid_checks: 1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.flush_verify_memtable_count: 1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.track_and_verify_wals_in_manifest: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.env: 0x36b49a0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.fs: LegacyFileSystem 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.info_log: 0x37760a0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_file_opening_threads: 16 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.statistics: (nil) 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.use_fsync: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_log_file_size: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_manifest_file_size: 1073741824 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.log_file_time_to_roll: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.keep_log_file_num: 1000 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.recycle_log_file_num: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.allow_fallocate: 1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.allow_mmap_reads: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.allow_mmap_writes: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.use_direct_reads: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.create_missing_column_families: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.db_log_dir: 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.wal_dir: db.wal 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.table_cache_numshardbits: 6 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.WAL_ttl_seconds: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.WAL_size_limit_MB: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.manifest_preallocation_size: 4194304 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.is_fd_close_on_exec: 1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.advise_random_on_open: 1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.db_write_buffer_size: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.write_buffer_manager: 0x3748dc0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.access_hint_on_compaction_start: 1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.use_adaptive_mutex: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.rate_limiter: (nil) 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.wal_recovery_mode: 2 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.enable_thread_tracking: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.enable_pipelined_write: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.unordered_write: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.write_thread_max_yield_usec: 100 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.row_cache: None 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.wal_filter: None 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.avoid_flush_during_recovery: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.allow_ingest_behind: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.two_write_queues: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.manual_wal_flush: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.wal_compression: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.atomic_flush: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.persist_stats_to_disk: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.write_dbid_to_manifest: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.log_readahead_size: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.file_checksum_gen_factory: Unknown 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.best_efforts_recovery: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.allow_data_in_errors: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.db_host_id: __hostname__ 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.enforce_single_del_contracts: true 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_background_jobs: 4 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_background_compactions: -1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_subcompactions: 1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.writable_file_max_buffer_size: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.delayed_write_rate : 16777216 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_total_wal_size: 1073741824 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.stats_dump_period_sec: 600 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.stats_persist_period_sec: 600 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.stats_history_buffer_size: 1048576 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_open_files: -1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.bytes_per_sync: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.wal_bytes_per_sync: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.strict_bytes_per_sync: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compaction_readahead_size: 2097152 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_background_flushes: -1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Compression algorithms supported: 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: kZSTD supported: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: kXpressCompression supported: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: kBZip2Compression supported: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: kZSTDNotFinalCompression supported: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: kLZ4Compression supported: 1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: kZlibCompression supported: 1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: kLZ4HCCompression supported: 1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: kSnappyCompression supported: 1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Fast CRC32 supported: Supported on x86 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: DMutex implementation: pthread_mutex_t 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: [db/db_impl/db_impl_readonly.cc:25] Opening the db in read only mode 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: [db/version_set.cc:5527] Recovering from manifest file: db/MANIFEST-000032 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.merge_operator: .T:int64_array.b:bitwise_xor 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x36e6240) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x3674970 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [m-0]: 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x36e6240) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x3674970 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [m-1]: 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x36e6240) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x3674970 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:17.280+0000 7f416e42d780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [m-2]: 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x36e6240) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x3674970 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [p-0]: 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x36e6240) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x3674970 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [p-1]: 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x36e6240) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x3674970 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [p-2]: 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x36e6240) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x3674970 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [O-0]: 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x36f2c40) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x3674d30 block_cache_name: BinnedLRUCache block_cache_options: capacity : 536870912 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [O-1]: 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x36f2c40) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x3674d30 block_cache_name: BinnedLRUCache block_cache_options: capacity : 536870912 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [O-2]: 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x36f2c40) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x3674d30 block_cache_name: BinnedLRUCache block_cache_options: capacity : 536870912 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: [db/column_family.cc:635] (skipping printing options) 2026-02-11T23:30:17.281+0000 7f416e42d780 4 rocksdb: [db/column_family.cc:635] (skipping printing options) 2026-02-11T23:30:17.285+0000 7f416e42d780 4 rocksdb: [db/version_set.cc:5566] Recovered from manifest file:db/MANIFEST-000032 succeeded,manifest_file_number is 32, next_file_number is 34, last_sequence is 11, log_number is 5,prev_log_number is 0,max_column_family is 11,min_log_number_to_keep is 5 2026-02-11T23:30:17.285+0000 7f416e42d780 4 rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2026-02-11T23:30:17.285+0000 7f416e42d780 4 rocksdb: [db/version_set.cc:5581] Column family [m-0] (ID 1), log number is 5 2026-02-11T23:30:17.285+0000 7f416e42d780 4 rocksdb: [db/version_set.cc:5581] Column family [m-1] (ID 2), log number is 5 2026-02-11T23:30:17.285+0000 7f416e42d780 4 rocksdb: [db/version_set.cc:5581] Column family [m-2] (ID 3), log number is 5 2026-02-11T23:30:17.285+0000 7f416e42d780 4 rocksdb: [db/version_set.cc:5581] Column family [p-0] (ID 4), log number is 5 2026-02-11T23:30:17.285+0000 7f416e42d780 4 rocksdb: [db/version_set.cc:5581] Column family [p-1] (ID 5), log number is 5 2026-02-11T23:30:17.285+0000 7f416e42d780 4 rocksdb: [db/version_set.cc:5581] Column family [p-2] (ID 6), log number is 5 2026-02-11T23:30:17.285+0000 7f416e42d780 4 rocksdb: [db/version_set.cc:5581] Column family [O-0] (ID 7), log number is 5 2026-02-11T23:30:17.285+0000 7f416e42d780 4 rocksdb: [db/version_set.cc:5581] Column family [O-1] (ID 8), log number is 5 2026-02-11T23:30:17.285+0000 7f416e42d780 4 rocksdb: [db/version_set.cc:5581] Column family [O-2] (ID 9), log number is 5 2026-02-11T23:30:17.285+0000 7f416e42d780 4 rocksdb: [db/version_set.cc:5581] Column family [L] (ID 10), log number is 5 2026-02-11T23:30:17.285+0000 7f416e42d780 4 rocksdb: [db/version_set.cc:5581] Column family [P] (ID 11), log number is 5 2026-02-11T23:30:17.285+0000 7f416e42d780 4 rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: d2a13b50-f500-4af9-be39-15a6598bf960 2026-02-11T23:30:17.285+0000 7f416e42d780 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1770852617285929, "job": 1, "event": "recovery_started", "wal_files": [31]} 2026-02-11T23:30:17.285+0000 7f416e42d780 4 rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #31 mode 2 2026-02-11T23:30:17.285+0000 7f416e42d780 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1770852617286077, "job": 1, "event": "recovery_finished"} 2026-02-11T23:30:17.285+0000 7f416e42d780 1 bluestore(td/osd-backfill-space/0) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-11T23:30:17.285+0000 7f416e42d780 1 bluestore(td/osd-backfill-space/0) _open_super_meta old nid_max 1025 2026-02-11T23:30:17.285+0000 7f416e42d780 1 bluestore(td/osd-backfill-space/0) _open_super_meta old blobid_max 10240 2026-02-11T23:30:17.285+0000 7f416e42d780 1 bluestore(td/osd-backfill-space/0) _open_super_meta ondisk_format 4 compat_ondisk_format 3 2026-02-11T23:30:17.285+0000 7f416e42d780 1 bluestore(td/osd-backfill-space/0) _open_super_meta min_alloc_size 0x1000 2026-02-11T23:30:17.286+0000 7f416e42d780 1 freelist init 2026-02-11T23:30:17.286+0000 7f416e42d780 1 freelist _read_cfg 2026-02-11T23:30:17.286+0000 7f416e42d780 1 bluestore(td/osd-backfill-space/0) _open_fm effective freelist_type = null, freelist_alloc_size = 0x1000, min_alloc_size = 0x1000 2026-02-11T23:30:17.288+0000 7f416e42d780 1 fbmap_alloc 0x37e3200 shutdown 2026-02-11T23:30:17.288+0000 7f416e42d780 1 bluestore(td/osd-backfill-space/0) _init_alloc loaded 100 GiB in 2 extents, allocator type hybrid, capacity 0x1900000000, block size 0x1000, free 0x18ffffd000, fragmentation 3.8e-08 2026-02-11T23:30:17.288+0000 7f416e42d780 4 rocksdb: [db/db_impl/db_impl.cc:496] Shutdown: canceling all background work 2026-02-11T23:30:17.288+0000 7f416e42d780 4 rocksdb: [db/db_impl/db_impl.cc:704] Shutdown complete 2026-02-11T23:30:17.288+0000 7f416e42d780 1 bluefs umount 2026-02-11T23:30:17.288+0000 7f416e42d780 1 bdev(0x4640800 td/osd-backfill-space/0/block) close 2026-02-11T23:30:17.313+0000 7f416e42d780 1 bdev(0x4640800 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:17.313+0000 7f416e42d780 0 bdev(0x4640800 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:17.313+0000 7f416e42d780 1 bdev(0x4640800 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:17.313+0000 7f416e42d780 1 bluefs add_block_device bdev 1 path td/osd-backfill-space/0/block size 100 GiB 2026-02-11T23:30:17.313+0000 7f416e42d780 1 bluefs mount 2026-02-11T23:30:17.313+0000 7f416e42d780 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-11T23:30:17.314+0000 7f416e42d780 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:17.314+0000 7f416e42d780 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:17.314+0000 7f416e42d780 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:17.314+0000 7f416e42d780 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:17.314+0000 7f416e42d780 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:17.314+0000 7f416e42d780 1 bluefs mount shared_bdev_used = 27459584 2026-02-11T23:30:17.314+0000 7f416e42d780 1 bluestore(td/osd-backfill-space/0) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: RocksDB version: 7.9.2 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Git sha 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Compile date 2026-02-11 10:30:18 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: DB SUMMARY 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: DB Session ID: HFPZG9ZUDMGBEKHWSHGZ 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: CURRENT file: CURRENT 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: IDENTITY file: IDENTITY 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: MANIFEST file: MANIFEST-000032 size: 1012 Bytes 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: SST files in db dir, Total Num: 1, files: 000030.sst 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: SST files in db.slow dir, Total Num: 0, files: 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Write Ahead Log file in db.wal: 000031.log size: 950 ; 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.error_if_exists: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.create_if_missing: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.paranoid_checks: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.flush_verify_memtable_count: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.track_and_verify_wals_in_manifest: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.env: 0x36b5340 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.fs: LegacyFileSystem 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.info_log: 0x36cfe60 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_file_opening_threads: 16 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.statistics: (nil) 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.use_fsync: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_log_file_size: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_manifest_file_size: 1073741824 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.log_file_time_to_roll: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.keep_log_file_num: 1000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.recycle_log_file_num: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.allow_fallocate: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.allow_mmap_reads: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.allow_mmap_writes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.use_direct_reads: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.create_missing_column_families: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.db_log_dir: 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.wal_dir: db.wal 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.table_cache_numshardbits: 6 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.WAL_ttl_seconds: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.WAL_size_limit_MB: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.manifest_preallocation_size: 4194304 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.is_fd_close_on_exec: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.advise_random_on_open: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.db_write_buffer_size: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.write_buffer_manager: 0x3749900 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.access_hint_on_compaction_start: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.use_adaptive_mutex: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.rate_limiter: (nil) 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.wal_recovery_mode: 2 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.enable_thread_tracking: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.enable_pipelined_write: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.unordered_write: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.write_thread_max_yield_usec: 100 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.row_cache: None 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.wal_filter: None 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.avoid_flush_during_recovery: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.allow_ingest_behind: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.two_write_queues: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.manual_wal_flush: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.wal_compression: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.atomic_flush: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.persist_stats_to_disk: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.write_dbid_to_manifest: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.log_readahead_size: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.file_checksum_gen_factory: Unknown 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.best_efforts_recovery: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.allow_data_in_errors: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.db_host_id: __hostname__ 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.enforce_single_del_contracts: true 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_background_jobs: 4 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_background_compactions: -1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_subcompactions: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.writable_file_max_buffer_size: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.delayed_write_rate : 16777216 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_total_wal_size: 1073741824 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.stats_dump_period_sec: 600 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.stats_persist_period_sec: 600 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.stats_history_buffer_size: 1048576 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_open_files: -1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bytes_per_sync: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.wal_bytes_per_sync: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.strict_bytes_per_sync: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_readahead_size: 2097152 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_background_flushes: -1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Compression algorithms supported: 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: kZSTD supported: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: kXpressCompression supported: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: kBZip2Compression supported: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: kZSTDNotFinalCompression supported: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: kLZ4Compression supported: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: kZlibCompression supported: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: kLZ4HCCompression supported: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: kSnappyCompression supported: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Fast CRC32 supported: Supported on x86 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: DMutex implementation: pthread_mutex_t 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: [db/version_set.cc:5527] Recovering from manifest file: db/MANIFEST-000032 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.merge_operator: .T:int64_array.b:bitwise_xor 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x4a702e0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x3674d30 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [m-0]: 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x4a702e0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x3674d30 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [m-1]: 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x4a702e0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x3674d30 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [m-2]: 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x4a702e0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x3674d30 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [p-0]: 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x4a702e0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x3674d30 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [p-1]: 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x4a702e0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x3674d30 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [p-2]: 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x4a702e0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x3674d30 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [O-0]: 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x4a70420) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x36752d0 block_cache_name: BinnedLRUCache block_cache_options: capacity : 536870912 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [O-1]: 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x4a70420) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x36752d0 block_cache_name: BinnedLRUCache block_cache_options: capacity : 536870912 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [O-2]: 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x4a70420) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x36752d0 block_cache_name: BinnedLRUCache block_cache_options: capacity : 536870912 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: [db/column_family.cc:635] (skipping printing options) 2026-02-11T23:30:17.315+0000 7f416e42d780 4 rocksdb: [db/column_family.cc:635] (skipping printing options) 2026-02-11T23:30:17.320+0000 7f416e42d780 4 rocksdb: [db/version_set.cc:5566] Recovered from manifest file:db/MANIFEST-000032 succeeded,manifest_file_number is 32, next_file_number is 34, last_sequence is 11, log_number is 5,prev_log_number is 0,max_column_family is 11,min_log_number_to_keep is 5 2026-02-11T23:30:17.320+0000 7f416e42d780 4 rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2026-02-11T23:30:17.320+0000 7f416e42d780 4 rocksdb: [db/version_set.cc:5581] Column family [m-0] (ID 1), log number is 5 2026-02-11T23:30:17.320+0000 7f416e42d780 4 rocksdb: [db/version_set.cc:5581] Column family [m-1] (ID 2), log number is 5 2026-02-11T23:30:17.320+0000 7f416e42d780 4 rocksdb: [db/version_set.cc:5581] Column family [m-2] (ID 3), log number is 5 2026-02-11T23:30:17.320+0000 7f416e42d780 4 rocksdb: [db/version_set.cc:5581] Column family [p-0] (ID 4), log number is 5 2026-02-11T23:30:17.320+0000 7f416e42d780 4 rocksdb: [db/version_set.cc:5581] Column family [p-1] (ID 5), log number is 5 2026-02-11T23:30:17.320+0000 7f416e42d780 4 rocksdb: [db/version_set.cc:5581] Column family [p-2] (ID 6), log number is 5 2026-02-11T23:30:17.320+0000 7f416e42d780 4 rocksdb: [db/version_set.cc:5581] Column family [O-0] (ID 7), log number is 5 2026-02-11T23:30:17.320+0000 7f416e42d780 4 rocksdb: [db/version_set.cc:5581] Column family [O-1] (ID 8), log number is 5 2026-02-11T23:30:17.320+0000 7f416e42d780 4 rocksdb: [db/version_set.cc:5581] Column family [O-2] (ID 9), log number is 5 2026-02-11T23:30:17.320+0000 7f416e42d780 4 rocksdb: [db/version_set.cc:5581] Column family [L] (ID 10), log number is 5 2026-02-11T23:30:17.320+0000 7f416e42d780 4 rocksdb: [db/version_set.cc:5581] Column family [P] (ID 11), log number is 5 2026-02-11T23:30:17.320+0000 7f416e42d780 4 rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: d2a13b50-f500-4af9-be39-15a6598bf960 2026-02-11T23:30:17.320+0000 7f416e42d780 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1770852617320348, "job": 1, "event": "recovery_started", "wal_files": [31]} 2026-02-11T23:30:17.320+0000 7f416e42d780 4 rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #31 mode 2 2026-02-11T23:30:17.320+0000 7f416e42d780 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1770852617320817, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 35, "file_size": 1273, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12, "largest_seqno": 19, "table_properties": {"data_size": 130, "index_size": 27, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 92, "raw_average_key_size": 18, "raw_value_size": 70, "raw_average_value_size": 14, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": ".T:int64_array.b:bitwise_xor", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "LZ4", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1770852617, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d2a13b50-f500-4af9-be39-15a6598bf960", "db_session_id": "HFPZG9ZUDMGBEKHWSHGZ", "orig_file_number": 35, "seqno_to_time_mapping": "N/A"}} 2026-02-11T23:30:17.320+0000 7f416e42d780 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1770852617321069, "cf_name": "p-0", "job": 1, "event": "table_file_creation", "file_number": 36, "file_size": 1594, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 14, "largest_seqno": 15, "table_properties": {"data_size": 469, "index_size": 39, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 72, "raw_average_key_size": 36, "raw_value_size": 571, "raw_average_value_size": 285, "num_data_blocks": 1, "num_entries": 2, "num_filter_entries": 2, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "p-0", "column_family_id": 4, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "LZ4", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1770852617, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d2a13b50-f500-4af9-be39-15a6598bf960", "db_session_id": "HFPZG9ZUDMGBEKHWSHGZ", "orig_file_number": 36, "seqno_to_time_mapping": "N/A"}} 2026-02-11T23:30:17.321+0000 7f416e42d780 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1770852617321307, "cf_name": "O-2", "job": 1, "event": "table_file_creation", "file_number": 37, "file_size": 1274, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 16, "largest_seqno": 16, "table_properties": {"data_size": 121, "index_size": 64, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 55, "raw_average_key_size": 55, "raw_value_size": 50, "raw_average_value_size": 50, "num_data_blocks": 1, "num_entries": 1, "num_filter_entries": 1, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "O-2", "column_family_id": 9, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "LZ4", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1770852617, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d2a13b50-f500-4af9-be39-15a6598bf960", "db_session_id": "HFPZG9ZUDMGBEKHWSHGZ", "orig_file_number": 37, "seqno_to_time_mapping": "N/A"}} 2026-02-11T23:30:17.321+0000 7f416e42d780 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1770852617321336, "job": 1, "event": "recovery_finished"} 2026-02-11T23:30:17.321+0000 7f416e42d780 4 rocksdb: [db/version_set.cc:5047] Creating manifest 39 2026-02-11T23:30:17.323+0000 7f416e42d780 4 rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x4643500 2026-02-11T23:30:17.323+0000 7f416e42d780 4 rocksdb: DB pointer 0x49ee000 2026-02-11T23:30:17.323+0000 7f416e42d780 1 bluestore(td/osd-backfill-space/0) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-11T23:30:17.323+0000 7f41589a5640 4 rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2026-02-11T23:30:17.323+0000 7f41589a5640 4 rocksdb: [db/db_impl/db_impl.cc:1111] ** DB Stats ** Uptime(secs): 0.0 total, 0.0 interval Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.000 H:M:S, 0.0 percent ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 2/0 2.62 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.5 0.00 0.00 1 0.000 0 0 0.0 0.0 Sum 2/0 2.62 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.5 0.00 0.00 1 0.000 0 0 0.0 0.0 Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.5 0.00 0.00 1 0.000 0 0 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.5 0.00 0.00 1 0.000 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 0.0 total, 0.0 interval Flush(GB): cumulative 0.000, interval 0.000 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 0.00 GB write, 0.14 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Interval compaction: 0.00 GB write, 0.14 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count Block cache BinnedLRUCache@0x3674d30#315569 capacity: 1.35 GB usage: 0.69 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 8 last_secs: 9e-06 secs_since: 0 Block cache entry stats(count,size,portion): FilterBlock(3,0.33 KB,2.31796e-05%) IndexBlock(3,0.36 KB,2.53872e-05%) Misc(1,0.00 KB,0%) ** File Read Latency Histogram By Level [default] ** ** Compaction Stats [m-0] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 ** Compaction Stats [m-0] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 0.0 total, 0.0 interval Flush(GB): cumulative 0.000, interval 0.000 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count Block cache BinnedLRUCache@0x3674d30#315569 capacity: 1.35 GB usage: 0.69 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 8 last_secs: 9e-06 secs_since: 0 Block cache entry stats(count,size,portion): FilterBlock(3,0.33 KB,2.31796e-05%) IndexBlock(3,0.36 KB,2.53872e-05%) Misc(1,0.00 KB,0%) ** File Read Latency Histogram By Level [m-0] ** ** Compaction Stats [m-1] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 ** Compaction Stats [m-1] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 0.0 total, 0.0 interval Flush(GB): cumulative 0.000, interval 0.000 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count Block cache BinnedLRUCache@0x3674d30#315569 capacity: 1.35 GB usage: 0.69 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 8 last_secs: 9e-06 secs_since: 0 Block cache entry stats(count,size,portion): FilterBlock(3,0.33 KB,2.31796e-05%) IndexBlock(3,0.36 KB,2.53872e-05%) Misc(1,0.00 KB,0%) ** File Read Latency Histogram By Level [m-1] ** ** Compaction Stats [m-2] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 ** Compaction Stats [m-2] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 0.0 total, 0.0 interval Flush(GB): cumulative 0.000, interval 0.000 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count Block cache BinnedLRUCache@0x3674d30#315569 capacity: 1.35 GB usage: 0.69 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 8 last_secs: 9e-06 secs_since: 0 Block cache entry stats(count,size,portion): FilterBlock(3,0.33 KB,2.31796e-05%) IndexBlock(3,0.36 KB,2.53872e-05%) Misc(1,0.00 KB,0%) ** File Read Latency Histogram By Level [m-2] ** ** Compaction Stats [p-0] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 1/0 1.56 KB 0.1 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 6.7 0.00 0.00 1 0.000 0 0 0.0 0.0 Sum 1/0 1.56 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 6.7 0.00 0.00 1 0.000 0 0 0.0 0.0 Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 6.7 0.00 0.00 1 0.000 0 0 0.0 0.0 ** Compaction Stats [p-0] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 6.7 0.00 0.00 1 0.000 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 0.0 total, 0.0 interval Flush(GB): cumulative 0.000, interval 0.000 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 0.00 GB write, 0.18 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Interval compaction: 0.00 GB write, 0.18 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count Block cache BinnedLRUCache@0x3674d30#315569 capacity: 1.35 GB usage: 0.69 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 8 last_secs: 9e-06 secs_since: 0 Block cache entry stats(count,size,portion): FilterBlock(3,0.33 KB,2.31796e-05%) IndexBlock(3,0.36 KB,2.53872e-05%) Misc(1,0.00 KB,0%) ** File Read Latency Histogram By Level [p-0] ** ** Compaction Stats [p-1] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 ** Compaction Stats [p-1] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 0.0 total, 0.0 interval Flush(GB): cumulative 0.000, interval 0.000 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count Block cache BinnedLRUCache@0x3674d30#315569 capacity: 1.35 GB usage: 0.69 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 8 last_secs: 9e-06 secs_since: 0 Block cache entry stats(count,size,portion): FilterBlock(3,0.33 KB,2.31796e-05%) IndexBlock(3,0.36 KB,2.53872e-05%) Misc(1,0.00 KB,0%) ** File Read Latency Histogram By Level [p-1] ** ** Compaction Stats [p-2] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 ** Compaction Stats [p-2] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 0.0 total, 0.0 interval Flush(GB): cumulative 0.000, interval 0.000 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count Block cache BinnedLRUCache@0x3674d30#315569 capacity: 1.35 GB usage: 0.69 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 8 last_secs: 9e-06 secs_since: 0 Block cache entry stats(count,size,portion): FilterBlock(3,0.33 KB,2.31796e-05%) IndexBlock(3,0.36 KB,2.53872e-05%) Misc(1,0.00 KB,0%) ** File Read Latency Histogram By Level [p-2] ** ** Compaction Stats [O-0] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 ** Compaction Stats [O-0] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 0.0 total, 0.0 interval Flush(GB): cumulative 0.000, interval 0.000 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count Block cache BinnedLRUCache@0x36752d0#315569 capacity: 512.00 MB usage: 0.25 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 2 last_secs: 4e-06 secs_since: 0 Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.14 KB,2.68221e-05%) Misc(1,0.00 KB,0%) ** File Read Latency Histogram By Level [O-0] ** ** Compaction Stats [O-1] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 ** Compaction Stats [O-1] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 0.0 total, 0.0 interval Flush(GB): cumulative 0.000, interval 0.000 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count Block cache BinnedLRUCache@0x36752d0#315569 capacity: 512.00 MB usage: 0.25 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 2 last_secs: 4e-06 secs_since: 0 Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.14 KB,2.68221e-05%) Misc(1,0.00 KB,0%) ** File Read Latency Histogram By Level [O-1] ** ** Compaction Stats [O-2] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 1/0 1.24 KB 0.1 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.4 0.00 0.00 1 0.000 0 0 0.0 0.0 Sum 1/0 1.24 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.4 0.00 0.00 1 0.000 0 0 0.0 0.0 Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.4 0.00 0.00 1 0.000 0 0 0.0 0.0 ** Compaction Stats [O-2] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.4 0.00 0.00 1 0.000 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 0.0 total, 0.0 interval Flush(GB): cumulative 0.000, interval 0.000 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 0.00 GB write, 0.15 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Interval compaction: 0.00 GB write, 0.15 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count Block cache BinnedLRUCache@0x36752d0#315569 capacity: 512.00 MB usage: 0.25 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 2 last_secs: 4e-06 secs_since: 0 Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.14 KB,2.68221e-05%) Misc(1,0.00 KB,0%) ** File Read Latency Histogram By Level [O-2] ** ** Compaction Stats [L] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 ** Compaction Stats [L] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 0.0 total, 0.0 interval Flush(GB): cumulative 0.000, interval 0.000 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count Block cache BinnedLRUCache@0x3674d30#315569 capacity: 1.35 GB usage: 0.69 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 8 last_secs: 9e-06 secs_since: 0 Block cache entry stats(count,size,portion): FilterBlock(3,0.33 KB,2.31796e-05%) IndexBlock(3,0.36 KB,2.53872e-05%) Misc(1,0.00 KB,0%) ** File Read Latency Histogram By Level [L] ** ** Compaction Stats [P] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 ** Compaction Stats [P] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 0.0 total, 0.0 interval Flush(GB): cumulative 0.000, interval 0.000 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count Block cache BinnedLRUCache@0x3674d30#315569 capacity: 1.35 GB usage: 0.69 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 8 last_secs: 9e-06 secs_since: 0 Block cache entry stats(count,size,portion): FilterBlock(3,0.33 KB,2.31796e-05%) IndexBlock(3,0.36 KB,2.53872e-05%) Misc(1,0.00 KB,0%) ** File Read Latency Histogram By Level [P] ** 2026-02-11T23:30:17.323+0000 7f416e42d780 1 bluestore(td/osd-backfill-space/0) _upgrade_super from 4, latest 4 2026-02-11T23:30:17.323+0000 7f416e42d780 1 bluestore(td/osd-backfill-space/0) _upgrade_super done 2026-02-11T23:30:17.324+0000 7f416e42d780 2 osd.0 0 journal looks like ssd 2026-02-11T23:30:17.324+0000 7f416e42d780 2 osd.0 0 boot 2026-02-11T23:30:17.324+0000 7f416e42d780 20 osd.0 0 configured osd_max_object_name[space]_len looks ok 2026-02-11T23:30:17.324+0000 7f416e42d780 10 osd.0 0 read_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e0 maps [] lci=[0,0] tlb=0) 2026-02-11T23:30:17.324+0000 7f416e42d780 20 osd.0 0 get_map 0 - return initial 0x37b1e00 2026-02-11T23:30:17.324+0000 7f416e42d780 10 osd.0 0 init creating/touching snapmapper object 2026-02-11T23:30:17.324+0000 7f416e42d780 10 osd.0 0 init creating/touching purged_snaps object 2026-02-11T23:30:17.324+0000 7f416e42d780 10 open_all_classes 2026-02-11T23:30:17.324+0000 7f416e42d780 10 open_all_classes found journal 2026-02-11T23:30:17.324+0000 7f416e42d780 10 _get_class adding new class name journal 0x4a8d580 2026-02-11T23:30:17.324+0000 7f416e42d780 10 _load_class journal from /usr/lib64/rados-classes/libcls_journal.so 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_class journal status 3 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method journal.create flags 3 0x7f415d1a1400 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method journal.get_order flags 1 0x7f415d1a0e20 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method journal.get_splay_width flags 1 0x7f415d1a0e90 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method journal.get_pool_id flags 1 0x7f415d1a0f00 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method journal.get_minimum_set flags 1 0x7f415d1a10a0 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method journal.set_minimum_set flags 3 0x7f415d1a1720 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method journal.get_active_set flags 1 0x7f415d1a1110 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method journal.set_active_set flags 3 0x7f415d1a1850 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method journal.get_client flags 1 0x7f415d1a31e0 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method journal.client_register flags 3 0x7f415d1a3820 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method journal.client_update_data flags 3 0x7f415d1a4650 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method journal.client_update_state flags 3 0x7f415d1a4a80 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method journal.client_unregister flags 3 0x7f415d1a8210 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method journal.client_commit flags 3 0x7f415d1a4dc0 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method journal.client_list flags 1 0x7f415d1a3460 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method journal.get_next_tag_tid flags 1 0x7f415d1a1180 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method journal.get_tag flags 1 0x7f415d1a6a20 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method journal.tag_create flags 3 0x7f415d1a8430 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method journal.tag_list flags 1 0x7f415d1a53e0 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method journal.guard_append flags 3 0x7f415d1a1980 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method journal.append flags 3 0x7f415d1a1ab0 2026-02-11T23:30:17.324+0000 7f416e42d780 10 _load_class journal success 2026-02-11T23:30:17.324+0000 7f416e42d780 10 open_all_classes found lock 2026-02-11T23:30:17.324+0000 7f416e42d780 10 _get_class adding new class name lock 0x4a8d6c0 2026-02-11T23:30:17.324+0000 7f416e42d780 10 _load_class lock from /usr/lib64/rados-classes/libcls_lock.so 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_class lock status 3 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method lock.lock flags 11 0x7f415d1915b0 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method lock.unlock flags 11 0x7f415d18ec20 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method lock.break_lock flags 3 0x7f415d18eef0 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method lock.get_info flags 1 0x7f415d190af0 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method lock.list_locks flags 1 0x7f415d18c2c0 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method lock.assert_locked flags 9 0x7f415d18f1a0 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method lock.set_cookie flags 11 0x7f415d18fa60 2026-02-11T23:30:17.324+0000 7f416e42d780 10 _load_class lock success 2026-02-11T23:30:17.324+0000 7f416e42d780 10 open_all_classes found fifo 2026-02-11T23:30:17.324+0000 7f416e42d780 10 _get_class adding new class name fifo 0x4a8d800 2026-02-11T23:30:17.324+0000 7f416e42d780 10 _load_class fifo from /usr/lib64/rados-classes/libcls_fifo.so 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_class fifo status 3 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method fifo.create_meta flags 3 0x7f415d15cfd0 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method fifo.get_meta flags 1 0x7f415d15f610 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method fifo.update_meta flags 3 0x7f415d15e730 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method fifo.init_part flags 3 0x7f415d160e00 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method fifo.push_part flags 3 0x7f415d1611f0 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method fifo.trim_part flags 3 0x7f415d162160 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method fifo.part_list flags 1 0x7f415d162770 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method fifo.get_part_info flags 1 0x7f415d16fb10 2026-02-11T23:30:17.324+0000 7f416e42d780 10 _load_class fifo success 2026-02-11T23:30:17.324+0000 7f416e42d780 10 open_all_classes found hello 2026-02-11T23:30:17.324+0000 7f416e42d780 10 _get_class adding new class name hello 0x4a8d940 2026-02-11T23:30:17.324+0000 7f416e42d780 10 _load_class hello from /usr/lib64/rados-classes/libcls_hello.so 2026-02-11T23:30:17.324+0000 7f416e42d780 0 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/20.3.0-5256-gf2a98edf/rpm/el9/BUILD/ceph-20.3.0-5256-gf2a98edf/src/cls/hello/cls_hello.cc:316: loading cls_hello 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_class hello status 3 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method hello.say_hello flags 1 0x7f415d1480a0 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method hello.record_hello flags 10 0x7f415d1491c0 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method hello.write_return_data flags 2 0x7f415d148130 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method hello.writes_dont_return_data flags 2 0x7f415d148130 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method hello.write_too_much_return_data flags 2 0x7f415d148250 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method hello.replay flags 1 0x7f415d148370 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method hello.turn_it_to_11 flags 11 0x7f415d148760 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method hello.bad_reader flags 2 0x7f415d1483a0 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method hello.bad_writer flags 1 0x7f415d148360 2026-02-11T23:30:17.324+0000 7f416e42d780 10 _load_class hello success 2026-02-11T23:30:17.324+0000 7f416e42d780 10 open_all_classes found cmpomap 2026-02-11T23:30:17.324+0000 7f416e42d780 10 _get_class adding new class name cmpomap 0x4a8da80 2026-02-11T23:30:17.324+0000 7f416e42d780 10 _load_class cmpomap from /usr/lib64/rados-classes/libcls_cmpomap.so 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_class cmpomap status 3 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method cmpomap.cmp_vals flags 1 0x7f415d13c470 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method cmpomap.cmp_set_vals flags 3 0x7f415d13cee0 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method cmpomap.cmp_rm_keys flags 3 0x7f415d13e550 2026-02-11T23:30:17.324+0000 7f416e42d780 10 _load_class cmpomap success 2026-02-11T23:30:17.324+0000 7f416e42d780 10 open_all_classes found refcount 2026-02-11T23:30:17.324+0000 7f416e42d780 10 _get_class adding new class name refcount 0x4a8c040 2026-02-11T23:30:17.324+0000 7f416e42d780 10 _load_class refcount from /usr/lib64/rados-classes/libcls_refcount.so 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_class refcount status 3 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method refcount.get flags 3 0x7f415d1229f0 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method refcount.put flags 3 0x7f415d122e80 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method refcount.set flags 3 0x7f415d121160 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method refcount.read flags 1 0x7f415d1235e0 2026-02-11T23:30:17.324+0000 7f416e42d780 10 _load_class refcount success 2026-02-11T23:30:17.324+0000 7f416e42d780 10 open_all_classes found numops 2026-02-11T23:30:17.324+0000 7f416e42d780 10 _get_class adding new class name numops 0x4a8c900 2026-02-11T23:30:17.324+0000 7f416e42d780 10 _load_class numops from /usr/lib64/rados-classes/libcls_numops.so 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_class numops status 3 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method numops.add flags 3 0x7f416d62f170 2026-02-11T23:30:17.324+0000 7f416e42d780 10 register_cxx_method numops.mul flags 3 0x7f416d62fc20 2026-02-11T23:30:17.324+0000 7f416e42d780 10 _load_class numops success 2026-02-11T23:30:17.324+0000 7f416e42d780 10 open_all_classes found rbd 2026-02-11T23:30:17.324+0000 7f416e42d780 10 _get_class adding new class name rbd 0x4a8ca40 2026-02-11T23:30:17.324+0000 7f416e42d780 10 _load_class rbd from /usr/lib64/rados-classes/libcls_rbd.so 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_class rbd status 3 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.create flags 3 0x7f415d0c5850 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.get_features flags 1 0x7f415d0c3da0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.set_features flags 3 0x7f415d0c3f70 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.get_size flags 1 0x7f415d0c65f0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.set_size flags 3 0x7f415d0c42b0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.get_snapcontext flags 1 0x7f415d0cb570 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.get_object_prefix flags 1 0x7f415d0c5300 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.get_data_pool flags 1 0x7f415d0c54a0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.get_snapshot_name flags 1 0x7f415d0c73c0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.get_snapshot_timestamp flags 1 0x7f415d0c7610 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.snapshot_get flags 1 0x7f415d0c7820 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.snapshot_add flags 3 0x7f415d0cde60 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.snapshot_remove flags 3 0x7f415d0d6d40 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.snapshot_rename flags 3 0x7f415d0cee90 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.snapshot_trash_add flags 3 0x7f415d0d8e80 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.get_all_features flags 1 0x7f415d0cfa10 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.get_parent flags 1 0x7f415d0c9a10 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.set_parent flags 3 0x7f415d0c3b10 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.remove_parent flags 3 0x7f415d0cd520 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.parent_get flags 1 0x7f415d0cd560 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.parent_overlap_get flags 1 0x7f415d0c6f50 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.parent_attach flags 3 0x7f415d0cf640 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.parent_detach flags 3 0x7f415d0cd540 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.set_protection_status flags 3 0x7f415d0ca950 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.get_protection_status flags 1 0x7f415d0c69b0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.get_stripe_unit_count flags 1 0x7f415d0c46e0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.set_stripe_unit_count flags 3 0x7f415d0c4930 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.get_create_timestamp flags 1 0x7f415d0c4d00 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.get_access_timestamp flags 1 0x7f415d0c4f00 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.get_modify_timestamp flags 1 0x7f415d0c5100 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.get_flags flags 1 0x7f415d0c6c50 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.set_flags flags 3 0x7f415d0cac50 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.op_features_get flags 1 0x7f415d0c27b0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.op_features_set flags 3 0x7f415d0c84d0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.metadata_list flags 1 0x7f415d0d4f00 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.metadata_set flags 3 0x7f415d0d92a0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.metadata_remove flags 3 0x7f415d0d17d0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.metadata_get flags 1 0x7f415d0d4c90 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.snapshot_get_limit flags 1 0x7f415d0d56b0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.snapshot_set_limit flags 3 0x7f415d0d5810 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.child_attach flags 3 0x7f415d0d7510 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.child_detach flags 3 0x7f415d0d79e0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.children_list flags 1 0x7f415d0d8a20 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.migration_set flags 3 0x7f415d0d5c30 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.migration_set_state flags 3 0x7f415d0d5ea0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.migration_get flags 1 0x7f415d0d61f0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.migration_remove flags 3 0x7f415d0d6420 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.set_modify_timestamp flags 3 0x7f415d0d0420 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.set_access_timestamp flags 3 0x7f415d0d0330 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.add_child flags 3 0x7f415d0c2b50 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.remove_child flags 3 0x7f415d0c3020 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.get_children flags 1 0x7f415d0cb230 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.get_id flags 1 0x7f415d0cfed0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.set_id flags 3 0x7f415d0d0110 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.dir_get_id flags 1 0x7f415d0d0510 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.dir_get_name flags 1 0x7f415d0d0780 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.dir_list flags 1 0x7f415d0d1970 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.dir_add_image flags 3 0x7f415d0d0e60 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.dir_remove_image flags 3 0x7f415d0d1670 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.dir_rename_image flags 3 0x7f415d0d1460 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.dir_state_assert flags 1 0x7f415d0d2470 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.dir_state_set flags 3 0x7f415d0d2720 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.object_map_load flags 1 0x7f415d0d2990 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.object_map_save flags 3 0x7f415d0d2aa0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.object_map_resize flags 3 0x7f415d0d2c80 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.object_map_update flags 3 0x7f415d0d41a0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.object_map_snap_add flags 3 0x7f415d0d3610 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.object_map_snap_remove flags 3 0x7f415d0d3a80 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.snap_list flags 1 0x7f415d0d6540 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.snap_add flags 3 0x7f415d0d6880 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.snap_remove flags 3 0x7f415d0d9dd0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.snap_rename flags 3 0x7f415d0da2d0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.mirror_uuid_get flags 1 0x7f415d0dac60 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.mirror_uuid_set flags 3 0x7f415d0e2aa0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.mirror_mode_get flags 1 0x7f415d0dad40 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.mirror_mode_set flags 3 0x7f415d0df7d0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.mirror_remote_namespace_get flags 1 0x7f415d0db010 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.mirror_remote_namespace_set flags 3 0x7f415d0dada0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.mirror_peer_ping flags 3 0x7f415d0e4160 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.mirror_peer_list flags 1 0x7f415d0dfb50 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.mirror_peer_add flags 3 0x7f415d0e00b0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.mirror_peer_remove flags 3 0x7f415d0daad0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.mirror_peer_set_client flags 3 0x7f415d0db330 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.mirror_peer_set_cluster flags 3 0x7f415d0dfd00 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.mirror_peer_set_direction flags 3 0x7f415d0db620 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.mirror_image_list flags 1 0x7f415d0db890 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.mirror_image_get_image_id flags 1 0x7f415d0db0f0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.mirror_image_get flags 1 0x7f415d0dc320 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.mirror_image_set flags 3 0x7f415d0dc480 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.mirror_image_remove flags 3 0x7f415d0e1160 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.mirror_image_status_set flags 11 0x7f415d0e4ee0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.mirror_image_status_remove flags 3 0x7f415d0e1560 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.mirror_image_status_get flags 1 0x7f415d0dcb00 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.mirror_image_status_list flags 1 0x7f415d0dccb0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.mirror_image_status_get_summary flags 1 0x7f415d0e2d00 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.mirror_image_status_remove_down flags 3 0x7f415d0e61a0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.mirror_image_instance_get flags 1 0x7f415d0e1940 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.mirror_image_instance_list flags 1 0x7f415d0e1c90 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.mirror_instances_list flags 1 0x7f415d0ea990 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.mirror_instances_add flags 11 0x7f415d0e61c0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.mirror_instances_remove flags 3 0x7f415d0e63e0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.mirror_image_map_list flags 1 0x7f415d0e97a0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.mirror_image_map_update flags 2 0x7f415d0e6730 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.mirror_image_map_remove flags 2 0x7f415d0e6570 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.mirror_image_snapshot_unlink_peer flags 3 0x7f415d0ef7f0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.mirror_image_snapshot_set_copy_progress flags 3 0x7f415d0efbc0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.group_dir_list flags 1 0x7f415d0e77b0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.group_dir_add flags 3 0x7f415d0e8530 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.group_dir_remove flags 3 0x7f415d0e6e60 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.group_dir_rename flags 3 0x7f415d0e7410 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.group_image_remove flags 3 0x7f415d0e8700 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.group_image_list flags 1 0x7f415d0ee3b0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.group_image_set flags 3 0x7f415d0efef0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.image_group_add flags 3 0x7f415d0e88b0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.image_group_remove flags 3 0x7f415d0e8be0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.image_group_get flags 1 0x7f415d0e8ee0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.group_snap_set flags 3 0x7f415d0eca00 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.group_snap_remove flags 3 0x7f415d0ed6a0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.group_snap_get_by_id flags 1 0x7f415d0ec350 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.group_snap_list flags 1 0x7f415d0ebf60 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.group_snap_list_order flags 1 0x7f415d0ed950 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.trash_add flags 3 0x7f415d0e9360 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.trash_remove flags 3 0x7f415d0e90c0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.trash_list flags 1 0x7f415d0f5480 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.trash_get flags 1 0x7f415d0f0220 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.trash_state_set flags 3 0x7f415d0f0470 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.namespace_add flags 3 0x7f415d0f0860 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.namespace_remove flags 3 0x7f415d0f0a70 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.namespace_list flags 1 0x7f415d0f0c30 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.copyup flags 3 0x7f415d0cfa60 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.sparse_copyup flags 3 0x7f415d0cfad0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.assert_snapc_seq flags 3 0x7f415d0d6440 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rbd.sparsify flags 3 0x7f415d0f1480 2026-02-11T23:30:17.325+0000 7f416e42d780 10 _load_class rbd success 2026-02-11T23:30:17.325+0000 7f416e42d780 10 open_all_classes found otp 2026-02-11T23:30:17.325+0000 7f416e42d780 10 _get_class adding new class name otp 0x4a8cb80 2026-02-11T23:30:17.325+0000 7f416e42d780 10 _load_class otp from /usr/lib64/rados-classes/libcls_otp.so 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_class otp status 3 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method otp.otp_set flags 3 0x7f415d0a5070 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method otp.otp_get flags 1 0x7f415d0a6d60 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method otp.otp_check flags 3 0x7f415d0a7af0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method otp.otp_get_result flags 1 0x7f415d0a82b0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method otp.otp_remove flags 3 0x7f415d0a6190 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method otp.get_current_time flags 1 0x7f415d0a4290 2026-02-11T23:30:17.325+0000 7f416e42d780 10 _load_class otp success 2026-02-11T23:30:17.325+0000 7f416e42d780 10 open_all_classes found sem_set 2026-02-11T23:30:17.325+0000 7f416e42d780 10 _get_class adding new class name sem_set 0x4a8ccc0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 _load_class sem_set from /usr/lib64/rados-classes/libcls_sem_set.so 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_class sem_set status 3 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method sem_set.increment flags 3 0x7f415d07a800 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method sem_set.decrement flags 3 0x7f415d07af80 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method sem_set.reset flags 3 0x7f415d0769b0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method sem_set.list flags 1 0x7f415d077640 2026-02-11T23:30:17.325+0000 7f416e42d780 10 _load_class sem_set success 2026-02-11T23:30:17.325+0000 7f416e42d780 10 open_all_classes found sdk 2026-02-11T23:30:17.325+0000 7f416e42d780 0 _get_class not permitted to load sdk 2026-02-11T23:30:17.325+0000 7f416e42d780 10 open_all_classes found lua 2026-02-11T23:30:17.325+0000 7f416e42d780 0 _get_class not permitted to load lua 2026-02-11T23:30:17.325+0000 7f416e42d780 10 open_all_classes found timeindex 2026-02-11T23:30:17.325+0000 7f416e42d780 10 _get_class adding new class name timeindex 0x4a8ce00 2026-02-11T23:30:17.325+0000 7f416e42d780 10 _load_class timeindex from /usr/lib64/rados-classes/libcls_timeindex.so 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_class timeindex status 3 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method timeindex.add flags 3 0x7f415d0681a0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method timeindex.list flags 1 0x7f415d068710 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method timeindex.trim flags 3 0x7f415d069a80 2026-02-11T23:30:17.325+0000 7f416e42d780 10 _load_class timeindex success 2026-02-11T23:30:17.325+0000 7f416e42d780 10 open_all_classes found log 2026-02-11T23:30:17.325+0000 7f416e42d780 10 _get_class adding new class name log 0x4a8d080 2026-02-11T23:30:17.325+0000 7f416e42d780 10 _load_class log from /usr/lib64/rados-classes/libcls_log.so 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_class log status 3 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method log.add flags 3 0x7f415d05cd90 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method log.list flags 1 0x7f415d05b6f0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method log.trim flags 3 0x7f415d05a900 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method log.info flags 1 0x7f415d05da30 2026-02-11T23:30:17.325+0000 7f416e42d780 10 _load_class log success 2026-02-11T23:30:17.325+0000 7f416e42d780 10 open_all_classes found rgw 2026-02-11T23:30:17.325+0000 7f416e42d780 10 _get_class adding new class name rgw 0x4a8d1c0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 _load_class rgw from /usr/lib64/rados-classes/libcls_rgw.so 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_class rgw status 3 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw.bucket_init_index flags 3 0x7f415cfed5f0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw.bucket_init_index2 flags 3 0x7f415cfed5f0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw.bucket_set_tag_timeout flags 3 0x7f415cfee8d0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw.bucket_list flags 1 0x7f415cfe95f0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw.bucket_check_index flags 1 0x7f415cffeb80 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw.bucket_rebuild_index flags 3 0x7f415cffe8d0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw.bucket_update_stats flags 3 0x7f415cff4630 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw.bucket_prepare_op flags 3 0x7f415cfeeb40 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw.bucket_complete_op flags 3 0x7f415cfef860 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw.bucket_link_olh flags 3 0x7f415cff19e0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw.bucket_unlink_instance flags 3 0x7f415cff3310 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw.bucket_read_olh_log flags 1 0x7f415cff4e90 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw.bucket_trim_olh_log flags 3 0x7f415cff6400 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw.bucket_clear_olh flags 3 0x7f415cff9790 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw.obj_remove flags 3 0x7f415cff6b80 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw.obj_store_pg_ver flags 2 0x7f415cff7520 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw.obj_check_attrs_prefix flags 1 0x7f415cff7820 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw.obj_check_mtime flags 1 0x7f415cff7db0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw.bi_get flags 1 0x7f415cff8190 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw.bi_put flags 3 0x7f415cff8700 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw.bi_put_entries flags 3 0x7f415cff89d0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw.bi_list flags 1 0x7f415d001360 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw.reshard_log_trim flags 3 0x7f415cffee20 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw.bi_log_list flags 1 0x7f415d003060 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw.bi_log_trim flags 3 0x7f415d0041f0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw.dir_suggest_changes flags 3 0x7f415cff9fa0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw.bi_log_resync flags 3 0x7f415cfff240 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw.bi_log_stop flags 3 0x7f415cfff660 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw.user_usage_log_add flags 3 0x7f415d0047c0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw.user_usage_log_read flags 1 0x7f415d008350 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw.user_usage_log_trim flags 3 0x7f415d008a90 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw.usage_log_clear flags 2 0x7f415d0072f0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw.gc_set_entry flags 3 0x7f415d008df0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw.gc_defer_entry flags 3 0x7f415d009090 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw.gc_list flags 1 0x7f415d0093c0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw.gc_remove flags 3 0x7f415d00c4f0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw.lc_get_entry flags 1 0x7f415d00a380 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw.lc_set_entry flags 3 0x7f415d00a820 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw.lc_rm_entry flags 3 0x7f415d00ac20 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw.lc_get_next_entry flags 1 0x7f415d00aed0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw.lc_put_head flags 3 0x7f415d00c200 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw.lc_get_head flags 1 0x7f415d007880 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw.lc_list_entries flags 1 0x7f415d00b450 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw.mp_upload_part_info_update flags 3 0x7f415d010b80 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw.reshard_add flags 3 0x7f415d0131d0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw.reshard_list flags 1 0x7f415d013640 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw.reshard_get flags 1 0x7f415d014120 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw.reshard_remove flags 3 0x7f415d014680 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw.set_bucket_resharding flags 3 0x7f415d014b80 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw.clear_bucket_resharding flags 3 0x7f415d014e00 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw.guard_bucket_resharding flags 1 0x7f415d015060 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw.get_bucket_resharding flags 1 0x7f415d015320 2026-02-11T23:30:17.325+0000 7f416e42d780 10 _load_class rgw success 2026-02-11T23:30:17.325+0000 7f416e42d780 10 open_all_classes found 2pc_queue 2026-02-11T23:30:17.325+0000 7f416e42d780 10 _get_class adding new class name 2pc_queue 0x4a8d300 2026-02-11T23:30:17.325+0000 7f416e42d780 10 _load_class 2pc_queue from /usr/lib64/rados-classes/libcls_2pc_queue.so 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_class 2pc_queue status 3 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method 2pc_queue.2pc_queue_init flags 3 0x7f415cf99b50 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method 2pc_queue.2pc_queue_get_capacity flags 1 0x7f415cf98f50 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method 2pc_queue.2pc_queue_get_topic_stats flags 1 0x7f415cf9ad70 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method 2pc_queue.2pc_queue_reserve flags 3 0x7f415cf9bc90 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method 2pc_queue.2pc_queue_commit flags 3 0x7f415cf9c6e0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method 2pc_queue.2pc_queue_abort flags 3 0x7f415cf9e2a0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method 2pc_queue.2pc_queue_list_reservations flags 1 0x7f415cf9e8f0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method 2pc_queue.2pc_queue_list_entries flags 1 0x7f415cf9a270 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method 2pc_queue.2pc_queue_remove_entries flags 3 0x7f415cf9b010 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method 2pc_queue.2pc_queue_expire_reservations flags 3 0x7f415cf9f170 2026-02-11T23:30:17.325+0000 7f416e42d780 10 _load_class 2pc_queue success 2026-02-11T23:30:17.325+0000 7f416e42d780 10 open_all_classes found cas 2026-02-11T23:30:17.325+0000 7f416e42d780 10 _get_class adding new class name cas 0x4a8d440 2026-02-11T23:30:17.325+0000 7f416e42d780 10 _load_class cas from /usr/lib64/rados-classes/libcls_cas.so 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_class cas status 3 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method cas.chunk_create_or_get_ref flags 3 0x7f415cf862c0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method cas.chunk_get_ref flags 3 0x7f415cf868a0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method cas.chunk_put_ref flags 3 0x7f415cf86bb0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method cas.references_chunk flags 1 0x7f415cf82c40 2026-02-11T23:30:17.325+0000 7f416e42d780 10 _load_class cas success 2026-02-11T23:30:17.325+0000 7f416e42d780 10 open_all_classes found user 2026-02-11T23:30:17.325+0000 7f416e42d780 10 _get_class adding new class name user 0x4a8c540 2026-02-11T23:30:17.325+0000 7f416e42d780 10 _load_class user from /usr/lib64/rados-classes/libcls_user.so 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_class user status 3 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method user.set_buckets_info flags 3 0x7f415cf69c20 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method user.complete_stats_sync flags 3 0x7f415cf6b600 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method user.remove_bucket flags 3 0x7f415cf6ba10 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method user.list_buckets flags 1 0x7f415cf6c510 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method user.get_header flags 1 0x7f415cf6d9d0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method user.reset_user_stats flags 3 0x7f415cf6dca0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method user.reset_user_stats2 flags 3 0x7f415cf6e7e0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method user.account_resource_add flags 3 0x7f415cf6f600 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method user.account_resource_get flags 1 0x7f415cf6fd60 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method user.account_resource_rm flags 3 0x7f415cf702b0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method user.account_resource_list flags 1 0x7f415cf706b0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 _load_class user success 2026-02-11T23:30:17.325+0000 7f416e42d780 10 open_all_classes found version 2026-02-11T23:30:17.325+0000 7f416e42d780 10 _get_class adding new class name version 0x4a8c400 2026-02-11T23:30:17.325+0000 7f416e42d780 10 _load_class version from /usr/lib64/rados-classes/libcls_version.so 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_class version status 3 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method version.set flags 3 0x7f415cf588e0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method version.inc flags 3 0x7f415cf58c20 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method version.inc_conds flags 3 0x7f415cf58c20 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method version.read flags 1 0x7f415cf59320 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method version.check_conds flags 1 0x7f415cf58fa0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 _load_class version success 2026-02-11T23:30:17.325+0000 7f416e42d780 10 open_all_classes found rgw_gc 2026-02-11T23:30:17.325+0000 7f416e42d780 10 _get_class adding new class name rgw_gc 0x4a8c2c0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 _load_class rgw_gc from /usr/lib64/rados-classes/libcls_rgw_gc.so 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_class rgw_gc status 3 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw_gc.rgw_gc_queue_init flags 3 0x7f415cf3a5c0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw_gc.rgw_gc_queue_enqueue flags 3 0x7f415cf3ac70 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw_gc.rgw_gc_queue_list_entries flags 1 0x7f415cf3bcc0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw_gc.rgw_gc_queue_remove_entries flags 3 0x7f415cf3d910 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method rgw_gc.rgw_gc_queue_update_entry flags 3 0x7f415cf3ef10 2026-02-11T23:30:17.325+0000 7f416e42d780 10 _load_class rgw_gc success 2026-02-11T23:30:17.325+0000 7f416e42d780 10 open_all_classes found queue 2026-02-11T23:30:17.325+0000 7f416e42d780 10 _get_class adding new class name queue 0x4a8c180 2026-02-11T23:30:17.325+0000 7f416e42d780 10 _load_class queue from /usr/lib64/rados-classes/libcls_queue.so 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_class queue status 3 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method queue.queue_init flags 3 0x7f415cf19c70 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method queue.queue_get_capacity flags 1 0x7f415cf1a200 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method queue.queue_enqueue flags 3 0x7f415cf1cfd0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method queue.queue_list_entries flags 1 0x7f415cf1a3a0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method queue.queue_remove_entries flags 3 0x7f415cf1c8e0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 _load_class queue success 2026-02-11T23:30:17.325+0000 7f416e42d780 10 open_all_classes found cephfs 2026-02-11T23:30:17.325+0000 7f416e42d780 10 _get_class adding new class name cephfs 0x36cab80 2026-02-11T23:30:17.325+0000 7f416e42d780 10 _load_class cephfs from /usr/lib64/rados-classes/libcls_cephfs.so 2026-02-11T23:30:17.325+0000 7f416e42d780 0 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/20.3.0-5256-gf2a98edf/rpm/el9/BUILD/ceph-20.3.0-5256-gf2a98edf/src/cls/cephfs/cls_cephfs.cc:202: loading cephfs 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_class cephfs status 3 2026-02-11T23:30:17.325+0000 7f416e42d780 10 register_cxx_method cephfs.accumulate_inode_metadata flags 3 0x7f415cf08fc0 2026-02-11T23:30:17.325+0000 7f416e42d780 10 _load_class cephfs success 2026-02-11T23:30:17.325+0000 7f416e42d780 0 osd.0 0 crush map has features 288232575208783872, adjusting msgr requires for clients 2026-02-11T23:30:17.325+0000 7f416e42d780 0 osd.0 0 crush map has features 288232575208783872 was 8705, adjusting msgr requires for mons 2026-02-11T23:30:17.326+0000 7f416e42d780 0 osd.0 0 crush map has features 288232575208783872, adjusting msgr requires for osds 2026-02-11T23:30:17.326+0000 7f416e42d780 0 osd.0 0 check_osdmap_features enabling on-disk ERASURE CODES compat feature 2026-02-11T23:30:17.326+0000 7f416e42d780 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e0 maps [] lci=[0,0] tlb=0) 2026-02-11T23:30:17.326+0000 7f416e42d780 10 osd.0 0 clear_temp_objects 2026-02-11T23:30:17.326+0000 7f416e42d780 0 osd.0 0 load_pgs 2026-02-11T23:30:17.326+0000 7f416e42d780 20 osd.0 0 load_pgs pg_num_history pg_num_history(e0 pg_nums {} deleted_pools ) 2026-02-11T23:30:17.326+0000 7f416e42d780 10 osd.0 0 load_pgs ignoring unrecognized meta 2026-02-11T23:30:17.326+0000 7f416e42d780 0 osd.0 0 load_pgs opened 0 pgs 2026-02-11T23:30:17.326+0000 7f416e42d780 2 osd.0 0 superblock: I am osd.0 2026-02-11T23:30:17.326+0000 7f416e42d780 0 osd.0 0 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:17.326+0000 7f416e42d780 10 monclient: init 2026-02-11T23:30:17.326+0000 7f416e42d780 1 Processor -- start 2026-02-11T23:30:17.326+0000 7f416e42d780 20 log_channel(cluster) update_config log_to_monitors true log_to_syslog false log_channels log_prios info 2026-02-11T23:30:17.326+0000 7f416e42d780 10 log_channel(cluster) update_config to_monitors: true to_syslog: false syslog_facility: prio: info to_graylog: false graylog_host: 127.0.0.1 graylog_port: 12201) 2026-02-11T23:30:17.326+0000 7f416e42d780 -1 osd.0 0 log_to_monitors true 2026-02-11T23:30:17.326+0000 7f416e42d780 1 Processor -- start 2026-02-11T23:30:17.326+0000 7f416e42d780 1 Processor -- start 2026-02-11T23:30:17.326+0000 7f416e42d780 1 Processor -- start 2026-02-11T23:30:17.326+0000 7f416e42d780 1 Processor -- start 2026-02-11T23:30:17.326+0000 7f416e42d780 1 Processor -- start 2026-02-11T23:30:17.326+0000 7f416e42d780 1 Processor -- start 2026-02-11T23:30:17.326+0000 7f4150194640 10 osd.0 0 agent_entry start 2026-02-11T23:30:17.326+0000 7f4150194640 20 osd.0 0 agent_entry empty queue 2026-02-11T23:30:17.327+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:17.327+0000 7f414f192640 20 osd.0 op_wq(1) _process empty q, waiting 2026-02-11T23:30:17.327+0000 7f414e190640 20 osd.0 op_wq(3) _process empty q, waiting 2026-02-11T23:30:17.327+0000 7f4148184640 20 osd.0 op_wq(7) _process empty q, waiting 2026-02-11T23:30:17.327+0000 7f414d98f640 20 osd.0 op_wq(4) _process empty q, waiting 2026-02-11T23:30:17.327+0000 7f414e991640 20 osd.0 op_wq(2) _process empty q, waiting 2026-02-11T23:30:17.327+0000 7f416e42d780 10 monclient: _reopen_session rank -1 2026-02-11T23:30:17.327+0000 7f414d18e640 20 osd.0 op_wq(5) _process empty q, waiting 2026-02-11T23:30:17.327+0000 7f416e42d780 10 monclient: _add_conns ranks=[0] 2026-02-11T23:30:17.327+0000 7f414c98d640 20 osd.0 op_wq(6) _process empty q, waiting 2026-02-11T23:30:17.327+0000 7f414c18c640 20 osd.0 op_wq(7) _process empty q, waiting 2026-02-11T23:30:17.327+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:17.327+0000 7f416e42d780 1 --2- [v2:0.0.0.0:6800/3294232400,v1:0.0.0.0:6801/3294232400] >> v2:127.0.0.1:7180/0 conn(0x4acbc00 0x4b5b080 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-11T23:30:17.327+0000 7f414b18a640 20 osd.0 op_wq(1) _process empty q, waiting 2026-02-11T23:30:17.327+0000 7f414a989640 20 osd.0 op_wq(2) _process empty q, waiting 2026-02-11T23:30:17.327+0000 7f416e42d780 10 monclient(hunting): picked mon.noname-a con 0x4acbc00 addr v2:127.0.0.1:7180/0 2026-02-11T23:30:17.327+0000 7f416e42d780 10 monclient(hunting): start opening mon connection 2026-02-11T23:30:17.327+0000 7f4149987640 20 osd.0 op_wq(4) _process empty q, waiting 2026-02-11T23:30:17.327+0000 7f416e42d780 1 -- [v2:0.0.0.0:6800/3294232400,v1:0.0.0.0:6801/3294232400] --> v2:127.0.0.1:7180/0 -- mon_getmap magic: 0 -- 0x36ca780 con 0x4acbc00 2026-02-11T23:30:17.327+0000 7f414a188640 20 osd.0 op_wq(3) _process empty q, waiting 2026-02-11T23:30:17.327+0000 7f416e42d780 10 monclient(hunting): _renew_subs 2026-02-11T23:30:17.327+0000 7f4149186640 20 osd.0 op_wq(5) _process empty q, waiting 2026-02-11T23:30:17.327+0000 7f4148985640 20 osd.0 op_wq(6) _process empty q, waiting 2026-02-11T23:30:17.327+0000 7f4147983640 5 osd.0 0 heartbeat osd_stat(store_statfs(0x384000/0x0/0x384000, data 0x256/0x1000, compress 0x0/0x0/0x0, omap 0x63a, meta 0x1a4f9c6), peers [] op hist []) 2026-02-11T23:30:17.327+0000 7f4147983640 20 osd.0 0 check_full_status cur ratio 0, physical ratio 0, new state none 2026-02-11T23:30:17.327+0000 7f416cbfe640 1 --2- [v2:0.0.0.0:6800/3294232400,v1:0.0.0.0:6801/3294232400] >> v2:127.0.0.1:7180/0 conn(0x4acbc00 0x4b5b080 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:17.327+0000 7f416cbfe640 1 --2- [v2:0.0.0.0:6800/3294232400,v1:0.0.0.0:6801/3294232400] >> v2:127.0.0.1:7180/0 conn(0x4acbc00 0x4b5b080 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:127.0.0.1:7180/0 says I am v2:127.0.0.1:41198/0 (socket says 127.0.0.1:41198) 2026-02-11T23:30:17.327+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] learned_addr learned my addr [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] (peer_addr_for_me v2:127.0.0.1:0/0) 2026-02-11T23:30:17.327+0000 7f416cbfe640 10 osd.0 0 new session (outgoing) 0x4a648c0 con=0x3699400 addr=v2:127.0.0.1:6800/3294232400 2026-02-11T23:30:17.327+0000 7f416cbfe640 10 monclient(hunting): get_auth_request con 0x4acbc00 auth_method 0 2026-02-11T23:30:17.327+0000 7f416cbfe640 10 monclient(hunting): get_auth_request method 1 preferred_modes [1] 2026-02-11T23:30:17.327+0000 7f416cbfe640 10 monclient(hunting): _init_auth method 1 2026-02-11T23:30:17.327+0000 7f416cbfe640 10 monclient(hunting): _init_auth creating new auth 2026-02-11T23:30:17.327+0000 7f416cbfe640 10 monclient(hunting): handle_auth_done global_id 4111 payload 0 2026-02-11T23:30:17.327+0000 7f416cbfe640 10 monclient: _finish_hunting 0 2026-02-11T23:30:17.327+0000 7f416cbfe640 1 monclient: found mon.noname-a 2026-02-11T23:30:17.327+0000 7f416cbfe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-11T23:30:17.327+0000 7f416cbfe640 10 monclient: _send_mon_message to mon.noname-a at v2:127.0.0.1:7180/0 2026-02-11T23:30:17.327+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> v2:127.0.0.1:7180/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x49d11e0 con 0x4acbc00 2026-02-11T23:30:17.327+0000 7f416cbfe640 1 --2- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] >> v2:127.0.0.1:7180/0 conn(0x4acbc00 0x4b5b080 crc :-1 s=READY pgs=15 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-11T23:30:17.327+0000 7f415cf02640 10 osd.0 0 ms_handle_connect con 0x4acbc00 2026-02-11T23:30:17.327+0000 7f415cf02640 10 osd.0 0 ms_handle_connect on mon 2026-02-11T23:30:17.327+0000 7f415cf02640 10 osd.0 0 requeue_pg_temp 0 + 0 -> 0 2026-02-11T23:30:17.327+0000 7f415cf02640 20 osd.0 0 _send_ready_to_merge ready_to_merge_source {} not_ready_to_merge_source ready_to_merge_target {} not_ready_to_merge_target {} sent_ready_to_merge_source 2026-02-11T23:30:17.327+0000 7f415cf02640 20 osd.0 0 send_pg_created 2026-02-11T23:30:17.327+0000 7f415cf02640 10 osd.0 0 requeue_failures 0 + 0 -> 0 2026-02-11T23:30:17.327+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 1 ==== mon_map magic: 0 ==== 178+0+0 (crc 0 0 0) 0x3758420 con 0x4acbc00 2026-02-11T23:30:17.327+0000 7f415cf02640 10 monclient: handle_monmap mon_map magic: 0 2026-02-11T23:30:17.327+0000 7f415cf02640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-11T23:30:17.327+0000 7f415cf02640 10 monclient: dump: epoch 1 fsid cb0a9481-482d-477a-a81c-1446b6498370 last_changed 2026-02-11T23:30:15.835295+0000 created 2026-02-11T23:30:15.835295+0000 min_mon_release 20 (tentacle) election_strategy: 1 0: v2:127.0.0.1:7180/0 mon.a 2026-02-11T23:30:17.327+0000 7f415cf02640 10 monclient: _finish_auth 0 2026-02-11T23:30:17.327+0000 7f415cf02640 10 monclient: _check_auth_tickets 2026-02-11T23:30:17.327+0000 7f415cf02640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:29:47.327972+0000) 2026-02-11T23:30:17.327+0000 7f416e42d780 5 monclient: authenticate success, global_id 4111 2026-02-11T23:30:17.327+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x4a89b00 con 0x4acbc00 2026-02-11T23:30:17.327+0000 7f415cf02640 10 monclient: handle_config config(0 keys) 2026-02-11T23:30:17.327+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 3 ==== mon_map magic: 0 ==== 178+0+0 (crc 0 0 0) 0x3758b00 con 0x4acbc00 2026-02-11T23:30:17.327+0000 7f415cf02640 10 monclient: handle_monmap mon_map magic: 0 2026-02-11T23:30:17.327+0000 7f415cf02640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-11T23:30:17.327+0000 7f415cf02640 10 monclient: dump: epoch 1 fsid cb0a9481-482d-477a-a81c-1446b6498370 last_changed 2026-02-11T23:30:15.835295+0000 created 2026-02-11T23:30:15.835295+0000 min_mon_release 20 (tentacle) election_strategy: 1 0: v2:127.0.0.1:7180/0 mon.a 2026-02-11T23:30:17.327+0000 7f4169bf8640 4 set_mon_vals no callback set 2026-02-11T23:30:17.327+0000 7f416e42d780 1 bluestore(td/osd-backfill-space/0) collect_metadata devices span numa nodes 0 2026-02-11T23:30:17.327+0000 7f416e42d780 10 osd.0 0 mon_cmd_maybe_osd_create cmd: {"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} 2026-02-11T23:30:17.327+0000 7f416e42d780 10 start_mon_command cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}] 2026-02-11T23:30:17.328+0000 7f416e42d780 10 monclient: _send_command 1 [{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}] 2026-02-11T23:30:17.328+0000 7f416e42d780 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:30:17.328+0000 7f416e42d780 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> v2:127.0.0.1:7180/0 -- mon_command({"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} v 0) -- 0x4a89380 con 0x4acbc00 2026-02-11T23:30:17.379+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 4 ==== mon_command_ack([{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]=0 set osd(s) 0 to class 'ssd' v3) ==== 131+0+0 (crc 0 0 0) 0x49d11e0 con 0x4acbc00 2026-02-11T23:30:17.379+0000 7f415cf02640 10 monclient: handle_mon_command_ack 1 [{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}] 2026-02-11T23:30:17.379+0000 7f415cf02640 10 monclient: _finish_command 1 = system:0 set osd(s) 0 to class 'ssd' 2026-02-11T23:30:17.379+0000 7f416e42d780 10 osd.0 0 update_crush_location crush location is "host=trial048", "root=default" 2026-02-11T23:30:17.379+0000 7f416e42d780 10 osd.0 0 mon_cmd_maybe_osd_create cmd: {"prefix": "osd crush create-or-move", "id": 0, "weight":0.0977, "args": ["host=trial048", "root=default"]} 2026-02-11T23:30:17.379+0000 7f416e42d780 10 start_mon_command cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0977, "args": ["host=trial048", "root=default"]}] 2026-02-11T23:30:17.379+0000 7f416e42d780 10 monclient: _send_command 2 [{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0977, "args": ["host=trial048", "root=default"]}] 2026-02-11T23:30:17.379+0000 7f416e42d780 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:30:17.379+0000 7f416e42d780 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> v2:127.0.0.1:7180/0 -- mon_command({"prefix": "osd crush create-or-move", "id": 0, "weight":0.0977, "args": ["host=trial048", "root=default"]} v 0) -- 0x4a89200 con 0x4acbc00 2026-02-11T23:30:17.479+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 5 ==== mon_command_ack([{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0977, "args": ["host=trial048", "root=default"]}]=0 create-or-move updating item name 'osd.0' weight 0.0977 at location {host=trial048,root=default} to crush map v4) ==== 250+0+0 (crc 0 0 0) 0x49d0ea0 con 0x4acbc00 2026-02-11T23:30:17.479+0000 7f415cf02640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0977, "args": ["host=trial048", "root=default"]}] 2026-02-11T23:30:17.479+0000 7f415cf02640 10 monclient: _finish_command 2 = system:0 create-or-move updating item name 'osd.0' weight 0.0977 at location {host=trial048,root=default} to crush map 2026-02-11T23:30:17.479+0000 7f416e42d780 10 osd.0 0 ensuring pgs have consumed prior maps 2026-02-11T23:30:17.479+0000 7f416e42d780 20 osd.0 0 consume_map version 0 2026-02-11T23:30:17.479+0000 7f416e42d780 20 osd.0 0 await_reserved_maps epoch:0 2026-02-11T23:30:17.479+0000 7f416e42d780 20 osd.0 0 await_reserved_maps done 0 2026-02-11T23:30:17.479+0000 7f416e42d780 20 osd.0 0 consume_map 0 -- publish done 2026-02-11T23:30:17.479+0000 7f416e42d780 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:17.479+0000 7f416e42d780 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:17.479+0000 7f416e42d780 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:17.479+0000 7f416e42d780 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:17.479+0000 7f416e42d780 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:17.479+0000 7f416e42d780 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:17.479+0000 7f416e42d780 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:17.479+0000 7f416e42d780 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:17.479+0000 7f416e42d780 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:17.479+0000 7f416e42d780 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:17.479+0000 7f416e42d780 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:17.479+0000 7f416e42d780 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:17.479+0000 7f416e42d780 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:17.479+0000 7f416e42d780 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:17.479+0000 7f416e42d780 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:17.479+0000 7f416e42d780 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:17.479+0000 7f416e42d780 20 osd.0 0 prune_pg_created 2026-02-11T23:30:17.479+0000 7f416e42d780 10 osd.0:0.consume_map 0 (was 0) 2026-02-11T23:30:17.479+0000 7f416e42d780 10 osd.0:1.consume_map 0 (was 0) 2026-02-11T23:30:17.479+0000 7f416e42d780 10 osd.0:2.consume_map 0 (was 0) 2026-02-11T23:30:17.479+0000 7f416e42d780 10 osd.0:3.consume_map 0 (was 0) 2026-02-11T23:30:17.479+0000 7f416e42d780 10 osd.0:4.consume_map 0 (was 0) 2026-02-11T23:30:17.479+0000 7f416e42d780 10 osd.0:5.consume_map 0 (was 0) 2026-02-11T23:30:17.479+0000 7f416e42d780 10 osd.0:6.consume_map 0 (was 0) 2026-02-11T23:30:17.479+0000 7f416e42d780 10 osd.0:7.consume_map 0 (was 0) 2026-02-11T23:30:17.479+0000 7f416e42d780 10 osd.0 0 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:17.479+0000 7f416e42d780 0 osd.0 0 done with init, starting boot process 2026-02-11T23:30:17.479+0000 7f416e42d780 10 monclient: _renew_subs 2026-02-11T23:30:17.479+0000 7f416e42d780 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:30:17.479+0000 7f416e42d780 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> v2:127.0.0.1:7180/0 -- mon_subscribe({mgrmap=0+,osd_pg_creates=0+}) -- 0x49d0820 con 0x4acbc00 2026-02-11T23:30:17.479+0000 7f416e42d780 1 osd.0 0 start_boot 2026-02-11T23:30:17.479+0000 7f416e42d780 10 osd.0 0 start_boot - have maps [] 2026-02-11T23:30:17.479+0000 7f416e42d780 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:30:17.479+0000 7f416e42d780 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> v2:127.0.0.1:7180/0 -- mon_get_version(what=osdmap handle=1) -- 0x3700b00 con 0x4acbc00 2026-02-11T23:30:17.479+0000 7f416e42d780 1 osd.0 0 maybe_override_options_for_qos osd_max_backfills set to 10 2026-02-11T23:30:17.479+0000 7f416e42d780 1 osd.0 0 maybe_override_options_for_qos osd_recovery_max_active set to 0 2026-02-11T23:30:17.479+0000 7f416e42d780 1 osd.0 0 maybe_override_options_for_qos osd_recovery_max_active_hdd set to 3 2026-02-11T23:30:17.479+0000 7f416e42d780 1 osd.0 0 maybe_override_options_for_qos osd_recovery_max_active_ssd set to 10 2026-02-11T23:30:17.479+0000 7f416e42d780 0 OSDBenchTest: OSD Bench Test Params: count: 12288000 Bytes block size: 4 KiB number of objects: 100 object size: 4 MiB 2026-02-11T23:30:17.479+0000 7f416e42d780 0 OSDBenchTest: run_test: running osd bench with count: 12288000 Bytes bsize: 4 KiB onum: 100 osize: 4 MiB 2026-02-11T23:30:17.480+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 6 ==== mgrmap(e 1) ==== 939+0+0 (crc 0 0 0) 0x4c9c000 con 0x4acbc00 2026-02-11T23:30:17.480+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 7 ==== mon_get_version_reply(handle=1 version=4) ==== 24+0+0 (crc 0 0 0) 0x3700b00 con 0x4acbc00 2026-02-11T23:30:17.480+0000 7f415cf02640 10 monclient: handle_get_version_reply finishing 1 version 4 2026-02-11T23:30:17.797+0000 7f416e42d780 0 OSDBenchTest: prefill_objects: Prefill took 0.32 secs. 2026-02-11T23:30:17.844+0000 7f416e42d780 0 OSDBenchTest: perform_write_test: Test took 0.05 secs. 2026-02-11T23:30:17.848+0000 7f416e42d780 0 OSDBenchTest: cleanup: Clean-up done. 2026-02-11T23:30:17.848+0000 7f416e42d780 0 OSDBenchTest: run_test: osd bench result - bandwidth: 247 MiB/s iops: 63340.14 elapsed_sec: 0.05 2026-02-11T23:30:17.848+0000 7f416e42d780 10 osd.0 0 mon_cmd_set_config Set osd_mclock_max_capacity_iops_ssd = 63340.139654 2026-02-11T23:30:17.848+0000 7f416e42d780 10 start_mon_command cmd=[{"prefix": "config set", "who": "osd.0", "name": "osd_mclock_max_capacity_iops_ssd", "value": "63340.139654"}] 2026-02-11T23:30:17.848+0000 7f416e42d780 10 monclient: _send_command 3 [{"prefix": "config set", "who": "osd.0", "name": "osd_mclock_max_capacity_iops_ssd", "value": "63340.139654"}] 2026-02-11T23:30:17.848+0000 7f416e42d780 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:30:17.848+0000 7f416e42d780 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> v2:127.0.0.1:7180/0 -- mon_command([{prefix=config set, name=osd_mclock_max_capacity_iops_ssd}] v 0) -- 0x4a89980 con 0x4acbc00 2026-02-11T23:30:17.848+0000 7f41693f7640 10 osd.0 0 _preboot _preboot mon has osdmaps 1..4 2026-02-11T23:30:17.848+0000 7f41693f7640 5 osd.0 0 heartbeat osd_stat(store_statfs(0x384000/0x0/0x384000, data 0x256/0x1000, compress 0x0/0x0/0x0, omap 0x63a, meta 0x1a4f9c6), peers [] op hist []) 2026-02-11T23:30:17.848+0000 7f41693f7640 20 osd.0 0 check_full_status cur ratio 0, physical ratio 0, new state none 2026-02-11T23:30:17.848+0000 7f41693f7640 -1 osd.0 0 waiting for initial osdmap 2026-02-11T23:30:17.848+0000 7f41693f7640 10 monclient: _renew_subs 2026-02-11T23:30:17.848+0000 7f41693f7640 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:30:17.848+0000 7f41693f7640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> v2:127.0.0.1:7180/0 -- mon_subscribe({osdmap=1}) -- 0x49d04e0 con 0x4acbc00 2026-02-11T23:30:17.849+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 8 ==== osd_map(1..4 src has 1..4) ==== 2751+0+0 (crc 0 0 0) 0x4a656c0 con 0x4acbc00 2026-02-11T23:30:17.849+0000 7f415cf02640 20 osd.0 0 OSD::ms_dispatch: osd_map(1..4 src has 1..4) 2026-02-11T23:30:17.849+0000 7f415cf02640 20 osd.0 0 _dispatch 0x4a656c0 osd_map(1..4 src has 1..4) 2026-02-11T23:30:17.849+0000 7f415cf02640 3 osd.0 0 handle_osd_map epochs [1,4], i have 0, src has [1,4] 2026-02-11T23:30:17.849+0000 7f415cf02640 10 osd.0 0 handle_osd_map: received 0 full maps and 4 incremental maps 2026-02-11T23:30:17.849+0000 7f415cf02640 10 osd.0 0 superblock cluster_osdmap_trim_lower_bound new epoch is: 1 2026-02-11T23:30:17.849+0000 7f415cf02640 10 osd.0 0 handle_osd_map got inc map for epoch 1 2026-02-11T23:30:17.849+0000 7f415cf02640 20 osd.0 0 got_full_map 1, nothing requested 2026-02-11T23:30:17.849+0000 7f415cf02640 10 osd.0 0 handle_osd_map got inc map for epoch 2 2026-02-11T23:30:17.849+0000 7f415cf02640 20 osd.0 0 got_full_map 2, nothing requested 2026-02-11T23:30:17.849+0000 7f415cf02640 10 osd.0 0 handle_osd_map got inc map for epoch 3 2026-02-11T23:30:17.849+0000 7f415cf02640 20 osd.0 0 got_full_map 3, nothing requested 2026-02-11T23:30:17.849+0000 7f415cf02640 10 osd.0 0 handle_osd_map got inc map for epoch 4 2026-02-11T23:30:17.849+0000 7f415cf02640 20 osd.0 0 got_full_map 4, nothing requested 2026-02-11T23:30:17.849+0000 7f415cf02640 10 osd.0 0 track_pools_and_pg_num_changes no maps stored, this is probably the first start of this osd 2026-02-11T23:30:17.849+0000 7f415cf02640 20 osd.0 0 handle_osd_map pg_num_history pg_num_history(e4 pg_nums {} deleted_pools ) 2026-02-11T23:30:17.849+0000 7f415cf02640 10 snap_mapper record_purged_snaps purged_snaps {1={},2={},3={},4={}} 2026-02-11T23:30:17.849+0000 7f415cf02640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:17.849+0000 7f415cf02640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e4 maps [1~4] lci=[0,0] tlb=1) 2026-02-11T23:30:17.849+0000 7f41591a6640 10 osd.0 0 _committed_osd_maps 1..4 2026-02-11T23:30:17.849+0000 7f41591a6640 10 osd.0 0 advance to epoch 1 (<= last 4 <= newest_map 4) 2026-02-11T23:30:17.849+0000 7f41591a6640 10 osd.0 1 advance to epoch 2 (<= last 4 <= newest_map 4) 2026-02-11T23:30:17.849+0000 7f41591a6640 10 osd.0 2 advance to epoch 3 (<= last 4 <= newest_map 4) 2026-02-11T23:30:17.849+0000 7f41591a6640 10 osd.0 3 advance to epoch 4 (<= last 4 <= newest_map 4) 2026-02-11T23:30:17.849+0000 7f41591a6640 0 osd.0 4 crush map has features 288514050185494528, adjusting msgr requires for clients 2026-02-11T23:30:17.849+0000 7f41591a6640 0 osd.0 4 crush map has features 288514050185494528 was 288232575208792577, adjusting msgr requires for mons 2026-02-11T23:30:17.849+0000 7f41591a6640 0 osd.0 4 crush map has features 3314932999778484224, adjusting msgr requires for osds 2026-02-11T23:30:17.849+0000 7f41591a6640 1 osd.0 4 check_osdmap_features require_osd_release unknown -> tentacle 2026-02-11T23:30:17.850+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 9 ==== mon_command_ack([{prefix=config set, name=osd_mclock_max_capacity_iops_ssd}]=0 v3)=0 v3) ==== 143+0+0 (crc 0 0 0) 0x49d04e0 con 0x4acbc00 2026-02-11T23:30:17.850+0000 7f415cf02640 10 monclient: handle_mon_command_ack 3 [{"prefix": "config set", "who": "osd.0", "name": "osd_mclock_max_capacity_iops_ssd", "value": "63340.139654"}] 2026-02-11T23:30:17.850+0000 7f415cf02640 10 monclient: _finish_command 3 = system:0 2026-02-11T23:30:17.850+0000 7f41591a6640 20 osd.0 4 consume_map version 4 2026-02-11T23:30:17.850+0000 7f41591a6640 20 osd.0 0 await_reserved_maps epoch:4 2026-02-11T23:30:17.850+0000 7f41591a6640 20 osd.0 0 await_reserved_maps done 0 2026-02-11T23:30:17.850+0000 7f41591a6640 20 osd.0 4 consume_map 4 -- publish done 2026-02-11T23:30:17.850+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:17.850+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:17.850+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:17.850+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:17.850+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:17.850+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:17.850+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:17.850+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:17.850+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:17.850+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:17.850+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:17.850+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:17.850+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:17.850+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:17.850+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:17.850+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:17.850+0000 7f41591a6640 20 osd.0 4 prune_pg_created 2026-02-11T23:30:17.850+0000 7f41591a6640 10 osd.0:0.consume_map 4 (was 0) 2026-02-11T23:30:17.850+0000 7f41591a6640 10 osd.0:1.consume_map 4 (was 0) 2026-02-11T23:30:17.850+0000 7f41591a6640 10 osd.0:2.consume_map 4 (was 0) 2026-02-11T23:30:17.850+0000 7f41591a6640 10 osd.0:3.consume_map 4 (was 0) 2026-02-11T23:30:17.850+0000 7f41591a6640 10 osd.0:4.consume_map 4 (was 0) 2026-02-11T23:30:17.850+0000 7f41591a6640 10 osd.0:5.consume_map 4 (was 0) 2026-02-11T23:30:17.850+0000 7f41591a6640 10 osd.0:6.consume_map 4 (was 0) 2026-02-11T23:30:17.850+0000 7f41591a6640 10 osd.0:7.consume_map 4 (was 0) 2026-02-11T23:30:17.850+0000 7f41591a6640 10 osd.0 4 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:17.850+0000 7f41591a6640 10 osd.0 4 _preboot _preboot mon has osdmaps 1..4 2026-02-11T23:30:17.850+0000 7f41591a6640 5 osd.0 4 heartbeat osd_stat(store_statfs(0x384000/0x0/0x384000, data 0x256/0x1000, compress 0x0/0x0/0x0, omap 0x63a, meta 0x1a4f9c6), peers [] op hist []) 2026-02-11T23:30:17.850+0000 7f41591a6640 20 osd.0 4 check_full_status cur ratio 0, physical ratio 0, new state none 2026-02-11T23:30:17.850+0000 7f41613e7640 10 osd.0 4 operator() waiting for peering work to drain 2026-02-11T23:30:17.850+0000 7f41613e7640 10 osd.0 4 _send_boot 2026-02-11T23:30:17.850+0000 7f41613e7640 20 osd.0 4 initial client_addrs [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400], cluster_addrs [v2:0.0.0.0:6802/3294232400,v1:0.0.0.0:6803/3294232400], hb_back_addrs [v2:0.0.0.0:6806/3294232400,v1:0.0.0.0:6807/3294232400], hb_front_addrs [v2:0.0.0.0:6804/3294232400,v1:0.0.0.0:6805/3294232400] 2026-02-11T23:30:17.850+0000 7f41613e7640 1 -- [v2:0.0.0.0:6802/3294232400,v1:0.0.0.0:6803/3294232400] set_addr_unknowns [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] 2026-02-11T23:30:17.850+0000 7f41613e7640 1 -- [v2:0.0.0.0:6802/3294232400,v1:0.0.0.0:6803/3294232400] set_addr_unknowns assuming my addr v2:0.0.0.0:6802/3294232400 matches provided addr v2:127.0.0.1:6800/3294232400 2026-02-11T23:30:17.850+0000 7f41613e7640 1 -- [v2:0.0.0.0:6802/3294232400,v1:0.0.0.0:6803/3294232400] set_addr_unknowns assuming my addr v1:0.0.0.0:6803/3294232400 matches provided addr v2:127.0.0.1:6800/3294232400 2026-02-11T23:30:17.850+0000 7f41613e7640 10 osd.0 4 new session (outgoing) 0x4a65500 con=0x3699000 addr=v2:127.0.0.1:6802/3294232400 2026-02-11T23:30:17.850+0000 7f41613e7640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] set_addr_unknowns now [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] 2026-02-11T23:30:17.850+0000 7f41613e7640 10 osd.0 4 assuming cluster_addrs match client_addrs [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] 2026-02-11T23:30:17.850+0000 7f41613e7640 1 -- [v2:0.0.0.0:6806/3294232400,v1:0.0.0.0:6807/3294232400] set_addr_unknowns [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] 2026-02-11T23:30:17.850+0000 7f41613e7640 1 -- [v2:0.0.0.0:6806/3294232400,v1:0.0.0.0:6807/3294232400] set_addr_unknowns assuming my addr v2:0.0.0.0:6806/3294232400 matches provided addr v2:127.0.0.1:6802/3294232400 2026-02-11T23:30:17.850+0000 7f41613e7640 1 -- [v2:0.0.0.0:6806/3294232400,v1:0.0.0.0:6807/3294232400] set_addr_unknowns assuming my addr v1:0.0.0.0:6807/3294232400 matches provided addr v2:127.0.0.1:6802/3294232400 2026-02-11T23:30:17.850+0000 7f41613e7640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] set_addr_unknowns now [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] 2026-02-11T23:30:17.850+0000 7f41613e7640 10 osd.0 4 assuming hb_back_addrs match cluster_addrs [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] 2026-02-11T23:30:17.850+0000 7f41613e7640 1 -- [v2:0.0.0.0:6804/3294232400,v1:0.0.0.0:6805/3294232400] set_addr_unknowns [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] 2026-02-11T23:30:17.850+0000 7f41613e7640 1 -- [v2:0.0.0.0:6804/3294232400,v1:0.0.0.0:6805/3294232400] set_addr_unknowns assuming my addr v2:0.0.0.0:6804/3294232400 matches provided addr v2:127.0.0.1:6800/3294232400 2026-02-11T23:30:17.850+0000 7f41613e7640 1 -- [v2:0.0.0.0:6804/3294232400,v1:0.0.0.0:6805/3294232400] set_addr_unknowns assuming my addr v1:0.0.0.0:6805/3294232400 matches provided addr v2:127.0.0.1:6800/3294232400 2026-02-11T23:30:17.850+0000 7f41613e7640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] set_addr_unknowns now [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] 2026-02-11T23:30:17.850+0000 7f41613e7640 10 osd.0 4 assuming hb_front_addrs match client_addrs [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] 2026-02-11T23:30:17.850+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 10 ==== config(1 keys) ==== 56+0+0 (crc 0 0 0) 0x4a89380 con 0x4acbc00 2026-02-11T23:30:17.850+0000 7f415cf02640 10 monclient: handle_config config(1 keys) 2026-02-11T23:30:17.850+0000 7f41693f7640 4 set_mon_vals no callback set 2026-02-11T23:30:17.850+0000 7f41693f7640 10 set_mon_vals osd_mclock_max_capacity_iops_ssd = 63340.139654 2026-02-11T23:30:17.850+0000 7f41613e7640 1 osd.0 4 set_numa_affinity storage numa node 0 2026-02-11T23:30:17.850+0000 7f41613e7640 -1 osd.0 4 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-11T23:30:17.850+0000 7f41613e7640 1 osd.0 4 set_numa_affinity not setting numa affinity 2026-02-11T23:30:17.850+0000 7f41693f7640 1 mclock_common: set_from_config: osd_bandwidth_cost_per_io: 19865.62 bytes/io, osd_bandwidth_capacity_per_shard 157286400.00 bytes/second 2026-02-11T23:30:17.850+0000 7f41613e7640 10 osd.0 4 final client_addrs [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400], cluster_addrs [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400], hb_back_addrs [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400], hb_front_addrs [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] 2026-02-11T23:30:17.850+0000 7f41693f7640 1 mclock_common: set_from_config: osd_bandwidth_cost_per_io: 19865.62 bytes/io, osd_bandwidth_capacity_per_shard 157286400.00 bytes/second 2026-02-11T23:30:17.850+0000 7f41693f7640 1 mclock_common: set_from_config: osd_bandwidth_cost_per_io: 19865.62 bytes/io, osd_bandwidth_capacity_per_shard 157286400.00 bytes/second 2026-02-11T23:30:17.850+0000 7f41693f7640 1 mclock_common: set_from_config: osd_bandwidth_cost_per_io: 19865.62 bytes/io, osd_bandwidth_capacity_per_shard 157286400.00 bytes/second 2026-02-11T23:30:17.850+0000 7f41693f7640 1 mclock_common: set_from_config: osd_bandwidth_cost_per_io: 19865.62 bytes/io, osd_bandwidth_capacity_per_shard 157286400.00 bytes/second 2026-02-11T23:30:17.850+0000 7f41693f7640 1 mclock_common: set_from_config: osd_bandwidth_cost_per_io: 19865.62 bytes/io, osd_bandwidth_capacity_per_shard 157286400.00 bytes/second 2026-02-11T23:30:17.850+0000 7f41693f7640 1 mclock_common: set_from_config: osd_bandwidth_cost_per_io: 19865.62 bytes/io, osd_bandwidth_capacity_per_shard 157286400.00 bytes/second 2026-02-11T23:30:17.850+0000 7f41693f7640 1 mclock_common: set_from_config: osd_bandwidth_cost_per_io: 19865.62 bytes/io, osd_bandwidth_capacity_per_shard 157286400.00 bytes/second 2026-02-11T23:30:17.850+0000 7f41613e7640 1 bluestore(td/osd-backfill-space/0) collect_metadata devices span numa nodes 0 2026-02-11T23:30:17.850+0000 7f41613e7640 1 bluestore(td/osd-backfill-space/0) collect_metadata devices span numa nodes 0 2026-02-11T23:30:17.851+0000 7f41613e7640 10 osd.0 4 _collect_metadata {arch=x86_64,back_addr=[v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400],back_iface=,bluefs=1,bluefs_dedicated_db=0,bluefs_dedicated_wal=0,bluefs_single_shared_device=1,bluestore_allocation_from_file=1,bluestore_allocator=hybrid,bluestore_bdev_access_mode=file,bluestore_bdev_block_size=4096,bluestore_bdev_devices=nvme0n1,bluestore_bdev_driver=KernelDevice,bluestore_bdev_optimal_io_size=4096,bluestore_bdev_path=td/osd-backfill-space/0/block,bluestore_bdev_rotational=0,bluestore_bdev_size=107374182400,bluestore_bdev_support_discard=1,bluestore_bdev_type=ssd,bluestore_min_alloc_size=4096,bluestore_onode_segmentation=inactive,bluestore_write_mode=classic,ceph_release=tentacle,ceph_version=ceph version 20.3.0-5256-gf2a98edf (f2a98edfc002a9c9c8df236a812719da23bb2dd0) tentacle (dev - RelWithDebInfo),ceph_version_short=20.3.0-5256-gf2a98edf,ceph_version_when_created=ceph version 20.3.0-5256-gf2a98edf (f2a98edfc002a9c9c8df236a812719da23bb2dd0) tentacle (dev - RelWithDebInfo),cpu=AMD EPYC 4464P 12-Core Processor,created_at=2026-02-11T23:30:16.557108Z,default_device_class=ssd,device_ids=nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6C99,device_paths=nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1,devices=nvme0n1,distro=centos,distro_description=CentOS Stream 9,distro_version=9,front_addr=[v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400],front_iface=,hb_back_addr=[v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400],hb_front_addr=[v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400],hostname=trial048,journal_rotational=0,kernel_description=#1 SMP PREEMPT_DYNAMIC Fri Feb 6 13:57:07 UTC 2026,kernel_version=5.14.0-677.el9.x86_64,mem_swap_kb=8388604,mem_total_kb=130438944,network_numa_unknown_ifaces=back_iface,front_iface,objectstore_numa_node=0,objectstore_numa_nodes=0,os=Linux,osd_data=td/osd-backfill-space/0,osd_objectstore=bluestore,osdspec_affinity=,rotational=0} 2026-02-11T23:30:17.851+0000 7f41613e7640 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:30:17.851+0000 7f41613e7640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> v2:127.0.0.1:7180/0 -- osd_boot(osd.0 booted 0 features 4544132024016699391 v4) -- 0x4f9ea80 con 0x4acbc00 2026-02-11T23:30:17.950+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 11 ==== osd_map(4..5 src has 1..5) ==== 1489+0+0 (crc 0 0 0) 0x463fc00 con 0x4acbc00 2026-02-11T23:30:17.950+0000 7f415cf02640 20 osd.0 4 OSD::ms_dispatch: osd_map(4..5 src has 1..5) 2026-02-11T23:30:17.950+0000 7f415cf02640 20 osd.0 4 _dispatch 0x463fc00 osd_map(4..5 src has 1..5) 2026-02-11T23:30:17.950+0000 7f415cf02640 10 monclient: _renew_subs 2026-02-11T23:30:17.950+0000 7f415cf02640 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:30:17.950+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> v2:127.0.0.1:7180/0 -- mon_subscribe({osdmap=0}) -- 0x49d04e0 con 0x4acbc00 2026-02-11T23:30:17.950+0000 7f415cf02640 3 osd.0 4 handle_osd_map epochs [4,5], i have 4, src has [1,5] 2026-02-11T23:30:17.950+0000 7f415cf02640 10 osd.0 4 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:17.950+0000 7f415cf02640 10 osd.0 4 handle_osd_map got inc map for epoch 5 2026-02-11T23:30:17.950+0000 7f415cf02640 20 osd.0 4 got_full_map 5, nothing requested 2026-02-11T23:30:17.950+0000 7f415cf02640 20 osd.0 4 trim_maps: min=0 oldest_map=1 2026-02-11T23:30:17.950+0000 7f415cf02640 20 osd.0 4 handle_osd_map pg_num_history pg_num_history(e5 pg_nums {} deleted_pools ) 2026-02-11T23:30:17.950+0000 7f415cf02640 10 snap_mapper record_purged_snaps purged_snaps {5={}} 2026-02-11T23:30:17.950+0000 7f415cf02640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:17.950+0000 7f415cf02640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e5 maps [1~5] lci=[0,0] tlb=1) 2026-02-11T23:30:17.950+0000 7f41591a6640 10 osd.0 4 _committed_osd_maps 5..5 2026-02-11T23:30:17.950+0000 7f41591a6640 10 osd.0 4 advance to epoch 5 (<= last 5 <= newest_map 5) 2026-02-11T23:30:17.950+0000 7f41591a6640 10 osd.0 5 up_epoch is 5 2026-02-11T23:30:17.950+0000 7f41591a6640 10 osd.0 5 boot_epoch is 5 2026-02-11T23:30:17.950+0000 7f41591a6640 1 osd.0 5 state: booting -> active 2026-02-11T23:30:17.950+0000 7f41591a6640 20 osd.0 5 consume_map version 5 2026-02-11T23:30:17.950+0000 7f41591a6640 20 osd.0 4 await_reserved_maps epoch:5 2026-02-11T23:30:17.950+0000 7f41591a6640 20 osd.0 4 await_reserved_maps done 0 2026-02-11T23:30:17.950+0000 7f41591a6640 20 osd.0 5 consume_map 5 -- publish done 2026-02-11T23:30:17.950+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:17.950+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:17.950+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:17.950+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:17.950+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:17.950+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:17.950+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:17.950+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:17.950+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:17.950+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:17.950+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:17.950+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:17.950+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:17.950+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:17.950+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:17.950+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:17.950+0000 7f41591a6640 20 osd.0 5 prune_pg_created 2026-02-11T23:30:17.950+0000 7f41591a6640 10 osd.0:0.consume_map 5 (was 4) 2026-02-11T23:30:17.950+0000 7f41591a6640 10 osd.0:1.consume_map 5 (was 4) 2026-02-11T23:30:17.950+0000 7f41591a6640 10 osd.0:2.consume_map 5 (was 4) 2026-02-11T23:30:17.950+0000 7f41591a6640 10 osd.0:3.consume_map 5 (was 4) 2026-02-11T23:30:17.950+0000 7f41591a6640 10 osd.0:4.consume_map 5 (was 4) 2026-02-11T23:30:17.950+0000 7f41591a6640 10 osd.0:5.consume_map 5 (was 4) 2026-02-11T23:30:17.950+0000 7f41591a6640 10 osd.0:6.consume_map 5 (was 4) 2026-02-11T23:30:17.950+0000 7f41591a6640 10 osd.0:7.consume_map 5 (was 4) 2026-02-11T23:30:17.950+0000 7f41591a6640 10 osd.0 5 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:17.950+0000 7f41591a6640 10 osd.0 5 maybe_update_heartbeat_peers updating 2026-02-11T23:30:17.950+0000 7f41591a6640 10 osd.0 5 maybe_update_heartbeat_peers 0 peers, extras 2026-02-11T23:30:17.950+0000 7f41591a6640 7 osd.0 5 activate_map version 5 2026-02-11T23:30:17.950+0000 7f4150194640 20 osd.0 5 agent_entry empty queue 2026-02-11T23:30:17.951+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 12 ==== osd_map(5..5 src has 1..5) ==== 662+0+0 (crc 0 0 0) 0x4a656c0 con 0x4acbc00 2026-02-11T23:30:17.951+0000 7f415cf02640 20 osd.0 5 OSD::ms_dispatch: osd_map(5..5 src has 1..5) 2026-02-11T23:30:17.951+0000 7f415cf02640 20 osd.0 5 _dispatch 0x4a656c0 osd_map(5..5 src has 1..5) 2026-02-11T23:30:17.951+0000 7f415cf02640 10 monclient: _renew_subs 2026-02-11T23:30:17.951+0000 7f415cf02640 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:30:17.951+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> v2:127.0.0.1:7180/0 -- mon_subscribe({osdmap=0}) -- 0x49d0d00 con 0x4acbc00 2026-02-11T23:30:17.951+0000 7f415cf02640 3 osd.0 5 handle_osd_map epochs [5,5], i have 5, src has [1,5] 2026-02-11T23:30:17.951+0000 7f415cf02640 10 osd.0 5 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:30:17.951+0000 7f415cf02640 10 osd.0 5 no new maps here, dropping 2026-02-11T23:30:17.951+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 13 ==== osd_map(5..5 src has 1..5) ==== 1299+0+0 (crc 0 0 0) 0x4a65180 con 0x4acbc00 2026-02-11T23:30:17.951+0000 7f415cf02640 20 osd.0 5 OSD::ms_dispatch: osd_map(5..5 src has 1..5) 2026-02-11T23:30:17.951+0000 7f415cf02640 20 osd.0 5 _dispatch 0x4a65180 osd_map(5..5 src has 1..5) 2026-02-11T23:30:17.951+0000 7f415cf02640 3 osd.0 5 handle_osd_map epochs [5,5], i have 5, src has [1,5] 2026-02-11T23:30:17.951+0000 7f415cf02640 10 osd.0 5 handle_osd_map: received 1 full maps and 0 incremental maps 2026-02-11T23:30:17.951+0000 7f415cf02640 10 osd.0 5 no new maps here, dropping 2026-02-11T23:30:17.951+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 14 ==== osd_map(5..5 src has 1..5) ==== 1299+0+0 (crc 0 0 0) 0x463fc00 con 0x4acbc00 2026-02-11T23:30:17.951+0000 7f415cf02640 20 osd.0 5 OSD::ms_dispatch: osd_map(5..5 src has 1..5) 2026-02-11T23:30:17.951+0000 7f415cf02640 20 osd.0 5 _dispatch 0x463fc00 osd_map(5..5 src has 1..5) 2026-02-11T23:30:17.951+0000 7f415cf02640 3 osd.0 5 handle_osd_map epochs [5,5], i have 5, src has [1,5] 2026-02-11T23:30:17.951+0000 7f415cf02640 10 osd.0 5 handle_osd_map: received 1 full maps and 0 incremental maps 2026-02-11T23:30:17.951+0000 7f415cf02640 10 osd.0 5 no new maps here, dropping 2026-02-11T23:30:18.100+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 15 ==== mgrmap(e 2) ==== 53625+0+0 (crc 0 0 0) 0x499cf00 con 0x4acbc00 2026-02-11T23:30:18.147+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 16 ==== osd_map(5..6 src has 1..6) ==== 950+0+0 (crc 0 0 0) 0x4a656c0 con 0x4acbc00 2026-02-11T23:30:18.147+0000 7f415cf02640 20 osd.0 5 OSD::ms_dispatch: osd_map(5..6 src has 1..6) 2026-02-11T23:30:18.147+0000 7f415cf02640 20 osd.0 5 _dispatch 0x4a656c0 osd_map(5..6 src has 1..6) 2026-02-11T23:30:18.147+0000 7f415cf02640 3 osd.0 5 handle_osd_map epochs [5,6], i have 5, src has [1,6] 2026-02-11T23:30:18.147+0000 7f415cf02640 10 osd.0 5 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:18.147+0000 7f415cf02640 10 osd.0 5 handle_osd_map got inc map for epoch 6 2026-02-11T23:30:18.147+0000 7f415cf02640 20 osd.0 5 got_full_map 6, nothing requested 2026-02-11T23:30:18.147+0000 7f415cf02640 20 osd.0 5 trim_maps: min=0 oldest_map=1 2026-02-11T23:30:18.147+0000 7f415cf02640 20 osd.0 5 handle_osd_map pg_num_history pg_num_history(e6 pg_nums {} deleted_pools ) 2026-02-11T23:30:18.147+0000 7f415cf02640 10 snap_mapper record_purged_snaps purged_snaps {6={}} 2026-02-11T23:30:18.147+0000 7f415cf02640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:18.147+0000 7f415cf02640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e6 maps [1~6] lci=[5,6] tlb=1) 2026-02-11T23:30:18.147+0000 7f41591a6640 10 osd.0 5 _committed_osd_maps 6..6 2026-02-11T23:30:18.147+0000 7f41591a6640 10 osd.0 5 advance to epoch 6 (<= last 6 <= newest_map 6) 2026-02-11T23:30:18.147+0000 7f41591a6640 20 osd.0 6 consume_map version 6 2026-02-11T23:30:18.147+0000 7f41591a6640 20 osd.0 5 await_reserved_maps epoch:6 2026-02-11T23:30:18.147+0000 7f41591a6640 20 osd.0 5 await_reserved_maps done 0 2026-02-11T23:30:18.147+0000 7f41591a6640 20 osd.0 6 consume_map 6 -- publish done 2026-02-11T23:30:18.147+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:18.147+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:18.147+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:18.147+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:18.147+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:18.147+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:18.147+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:18.147+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:18.147+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:18.147+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:18.147+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:18.147+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:18.147+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:18.147+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:18.147+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:18.147+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:18.147+0000 7f41591a6640 20 osd.0 6 prune_pg_created 2026-02-11T23:30:18.147+0000 7f41591a6640 10 osd.0:0.consume_map 6 (was 5) 2026-02-11T23:30:18.147+0000 7f41591a6640 10 osd.0:1.consume_map 6 (was 5) 2026-02-11T23:30:18.147+0000 7f41591a6640 10 osd.0:2.consume_map 6 (was 5) 2026-02-11T23:30:18.147+0000 7f41591a6640 10 osd.0:3.consume_map 6 (was 5) 2026-02-11T23:30:18.147+0000 7f41591a6640 10 osd.0:4.consume_map 6 (was 5) 2026-02-11T23:30:18.147+0000 7f41591a6640 10 osd.0:5.consume_map 6 (was 5) 2026-02-11T23:30:18.147+0000 7f41591a6640 10 osd.0:6.consume_map 6 (was 5) 2026-02-11T23:30:18.147+0000 7f41591a6640 10 osd.0:7.consume_map 6 (was 5) 2026-02-11T23:30:18.147+0000 7f41591a6640 10 osd.0 6 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:18.147+0000 7f41591a6640 7 osd.0 6 activate_map version 6 2026-02-11T23:30:18.147+0000 7f4150194640 20 osd.0 6 agent_entry empty queue 2026-02-11T23:30:18.201+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 17 ==== mgrmap(e 3) ==== 53752+0+0 (crc 0 0 0) 0x4a34000 con 0x4acbc00 2026-02-11T23:30:18.201+0000 7f415cf02640 1 --2- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] >> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] conn(0x4aca800 0x4b5cc00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-11T23:30:18.201+0000 7f416abfa640 1 --2- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] >> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] conn(0x4aca800 0x4b5cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:18.201+0000 7f416abfa640 10 monclient: get_auth_request con 0x4aca800 auth_method 0 2026-02-11T23:30:18.202+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] -- mgropen(unknown.0) -- 0x5659200 con 0x4aca800 2026-02-11T23:30:18.202+0000 7f416abfa640 1 --2- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] >> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] conn(0x4aca800 0x4b5cc00 crc :-1 s=READY pgs=2 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-11T23:30:18.202+0000 7f415cf02640 10 osd.0 6 ms_handle_connect con 0x4aca800 2026-02-11T23:30:18.202+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mgr.4100 v2:127.0.0.1:6808/4279895178 1 ==== mgrconfigure(period=5, threshold=5) ==== 13+0+0 (crc 0 0 0) 0x4a64c40 con 0x4aca800 2026-02-11T23:30:18.203+0000 7f415cf02640 20 osd.0 6 reports for 0 queries 2026-02-11T23:30:18.203+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] -- mgrreport(unknown.0 +277-0 packed 3294 task_status=0) -- 0x4f9ee00 con 0x4aca800 2026-02-11T23:30:18.203+0000 7f415cf02640 15 osd.0 6 collect_pg_stats 2026-02-11T23:30:18.203+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] -- pg_stats(0 pgs seq 21474836481 v 0) -- 0x4a34000 con 0x4aca800 2026-02-11T23:30:18.280+0000 7f41633eb640 10 osd.0 6 tick 2026-02-11T23:30:18.280+0000 7f41633eb640 20 osd.0 6 tick last_purged_snaps_scrub 0.000000 next 115256.250000 ... now 2026-02-11T23:30:18.280+0000 7f41633eb640 10 osd.0 6 scrub_purged_snaps 2026-02-11T23:30:18.280+0000 7f41633eb640 0 log_channel(cluster) log [DBG] : purged_snaps scrub starts 2026-02-11T23:30:18.280+0000 7f41633eb640 10 snap_mapper run 2026-02-11T23:30:18.280+0000 7f41633eb640 10 snap_mapper run end, found 0 stray 2026-02-11T23:30:18.280+0000 7f41633eb640 0 log_channel(cluster) log [DBG] : purged_snaps scrub ok 2026-02-11T23:30:18.280+0000 7f41633eb640 10 osd.0 6 scrub_purged_snaps done queueing pgs, updating superblock 2026-02-11T23:30:18.280+0000 7f41633eb640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e6 maps [1~6] lci=[5,6] tlb=1) 2026-02-11T23:30:18.280+0000 7f41633eb640 20 osd.0 6 send_beacon sending 2026-02-11T23:30:18.280+0000 7f41633eb640 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:30:18.280+0000 7f41633eb640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> v2:127.0.0.1:7180/0 -- osd_beacon(pgs [] lec 6 last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 osd_beacon_report_interval 300 v6) -- 0x4c9a000 con 0x4acbc00 2026-02-11T23:30:18.280+0000 7f41633eb640 10 osd.0 6 scrub_purged_snaps done 2026-02-11T23:30:18.319+0000 7f4162bea640 10 osd.0 6 tick_without_osd_lock 2026-02-11T23:30:18.319+0000 7f4162bea640 0 osd.0 6 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:18.319+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:18.319+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:18.319+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:18.319+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.538 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:18.319+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:18.319, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:18.319+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:18.319+0000 7f4162bea640 10 osd.0 6 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:18.319+0000 7f4162bea640 20 osd.0 6 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:18.319+0000 7f4162bea640 10 osd.0 6 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:18.319+0000 7f4162bea640 20 osd.0 6 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:18.353+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 18 ==== osd_map(6..7 src has 1..7) ==== 626+0+0 (crc 0 0 0) 0x4a65180 con 0x4acbc00 2026-02-11T23:30:18.353+0000 7f415cf02640 20 osd.0 6 OSD::ms_dispatch: osd_map(6..7 src has 1..7) 2026-02-11T23:30:18.353+0000 7f415cf02640 20 osd.0 6 _dispatch 0x4a65180 osd_map(6..7 src has 1..7) 2026-02-11T23:30:18.353+0000 7f415cf02640 3 osd.0 6 handle_osd_map epochs [6,7], i have 6, src has [1,7] 2026-02-11T23:30:18.353+0000 7f415cf02640 10 osd.0 6 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:18.353+0000 7f415cf02640 10 osd.0 6 handle_osd_map got inc map for epoch 7 2026-02-11T23:30:18.353+0000 7f415cf02640 20 osd.0 6 got_full_map 7, nothing requested 2026-02-11T23:30:18.353+0000 7f415cf02640 20 osd.0 6 trim_maps: min=0 oldest_map=1 2026-02-11T23:30:18.353+0000 7f415cf02640 20 osd.0 6 handle_osd_map pg_num_history pg_num_history(e7 pg_nums {} deleted_pools ) 2026-02-11T23:30:18.353+0000 7f415cf02640 10 snap_mapper record_purged_snaps purged_snaps {7={}} 2026-02-11T23:30:18.353+0000 7f415cf02640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:18.353+0000 7f415cf02640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e7 maps [1~7] lci=[5,7] tlb=1) 2026-02-11T23:30:18.354+0000 7f41591a6640 10 osd.0 6 _committed_osd_maps 7..7 2026-02-11T23:30:18.354+0000 7f41591a6640 10 osd.0 6 advance to epoch 7 (<= last 7 <= newest_map 7) 2026-02-11T23:30:18.354+0000 7f41591a6640 20 osd.0 7 consume_map version 7 2026-02-11T23:30:18.354+0000 7f41591a6640 20 osd.0 6 await_reserved_maps epoch:7 2026-02-11T23:30:18.354+0000 7f41591a6640 20 osd.0 6 await_reserved_maps done 0 2026-02-11T23:30:18.354+0000 7f41591a6640 20 osd.0 7 consume_map 7 -- publish done 2026-02-11T23:30:18.354+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:18.354+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:18.354+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:18.354+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:18.354+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:18.354+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:18.354+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:18.354+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:18.354+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:18.354+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:18.354+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:18.354+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:18.354+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:18.354+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:18.354+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:18.354+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:18.354+0000 7f41591a6640 20 osd.0 7 prune_pg_created 2026-02-11T23:30:18.354+0000 7f41591a6640 10 osd.0:0.consume_map 7 (was 6) 2026-02-11T23:30:18.354+0000 7f41591a6640 10 osd.0:1.consume_map 7 (was 6) 2026-02-11T23:30:18.354+0000 7f41591a6640 10 osd.0:2.consume_map 7 (was 6) 2026-02-11T23:30:18.354+0000 7f41591a6640 10 osd.0:3.consume_map 7 (was 6) 2026-02-11T23:30:18.354+0000 7f41591a6640 10 osd.0:4.consume_map 7 (was 6) 2026-02-11T23:30:18.354+0000 7f41591a6640 10 osd.0:5.consume_map 7 (was 6) 2026-02-11T23:30:18.354+0000 7f41591a6640 10 osd.0:6.consume_map 7 (was 6) 2026-02-11T23:30:18.354+0000 7f41591a6640 10 osd.0:7.consume_map 7 (was 6) 2026-02-11T23:30:18.354+0000 7f41591a6640 10 osd.0 7 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:18.354+0000 7f41591a6640 7 osd.0 7 activate_map version 7 2026-02-11T23:30:18.354+0000 7f4150194640 20 osd.0 7 agent_entry empty queue 2026-02-11T23:30:19.279+0000 7f4162bea640 10 osd.0 7 tick_without_osd_lock 2026-02-11T23:30:19.279+0000 7f4162bea640 0 osd.0 7 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:19.279+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:19.279+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:19.279+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:19.279+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.538 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:19.279+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:19.280, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:19.279+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:19.279+0000 7f4162bea640 10 osd.0 7 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:19.279+0000 7f4162bea640 20 osd.0 7 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:19.279+0000 7f4162bea640 10 osd.0 7 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:19.279+0000 7f4162bea640 20 osd.0 7 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:19.282+0000 7f41633eb640 10 osd.0 7 tick 2026-02-11T23:30:19.282+0000 7f41633eb640 20 osd.0 7 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:30:19.362+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 19 ==== osd_map(7..8 src has 1..8) ==== 1186+0+0 (crc 0 0 0) 0x4a64fc0 con 0x4acbc00 2026-02-11T23:30:19.362+0000 7f415cf02640 20 osd.0 7 OSD::ms_dispatch: osd_map(7..8 src has 1..8) 2026-02-11T23:30:19.362+0000 7f415cf02640 20 osd.0 7 _dispatch 0x4a64fc0 osd_map(7..8 src has 1..8) 2026-02-11T23:30:19.362+0000 7f415cf02640 3 osd.0 7 handle_osd_map epochs [7,8], i have 7, src has [1,8] 2026-02-11T23:30:19.362+0000 7f415cf02640 10 osd.0 7 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:19.362+0000 7f415cf02640 10 osd.0 7 handle_osd_map got inc map for epoch 8 2026-02-11T23:30:19.362+0000 7f415cf02640 20 osd.0 7 got_full_map 8, nothing requested 2026-02-11T23:30:19.362+0000 7f415cf02640 20 osd.0 7 trim_maps: min=0 oldest_map=1 2026-02-11T23:30:19.362+0000 7f415cf02640 20 osd.0 7 handle_osd_map pg_num_history pg_num_history(e8 pg_nums {} deleted_pools ) 2026-02-11T23:30:19.362+0000 7f415cf02640 10 snap_mapper record_purged_snaps purged_snaps {8={}} 2026-02-11T23:30:19.362+0000 7f415cf02640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:19.362+0000 7f415cf02640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e8 maps [1~8] lci=[5,8] tlb=1) 2026-02-11T23:30:19.362+0000 7f41591a6640 10 osd.0 7 _committed_osd_maps 8..8 2026-02-11T23:30:19.362+0000 7f41591a6640 10 osd.0 7 advance to epoch 8 (<= last 8 <= newest_map 8) 2026-02-11T23:30:19.362+0000 7f41591a6640 20 osd.0 8 consume_map version 8 2026-02-11T23:30:19.362+0000 7f41591a6640 20 osd.0 7 await_reserved_maps epoch:8 2026-02-11T23:30:19.362+0000 7f41591a6640 20 osd.0 7 await_reserved_maps done 0 2026-02-11T23:30:19.362+0000 7f41591a6640 20 osd.0 8 consume_map 8 -- publish done 2026-02-11T23:30:19.362+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:19.362+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:19.362+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:19.362+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:19.362+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:19.362+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:19.362+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:19.362+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:19.362+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:19.362+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:19.362+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:19.362+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:19.362+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:19.362+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:19.362+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:19.362+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:19.362+0000 7f41591a6640 20 osd.0 8 prune_pg_created 2026-02-11T23:30:19.362+0000 7f41591a6640 10 osd.0:0.consume_map 8 (was 7) 2026-02-11T23:30:19.362+0000 7f41591a6640 10 osd.0:1.consume_map 8 (was 7) 2026-02-11T23:30:19.362+0000 7f41591a6640 10 osd.0:2.consume_map 8 (was 7) 2026-02-11T23:30:19.362+0000 7f41591a6640 10 osd.0:3.consume_map 8 (was 7) 2026-02-11T23:30:19.362+0000 7f41591a6640 10 osd.0:4.consume_map 8 (was 7) 2026-02-11T23:30:19.362+0000 7f41591a6640 10 osd.0:5.consume_map 8 (was 7) 2026-02-11T23:30:19.362+0000 7f41591a6640 10 osd.0:6.consume_map 8 (was 7) 2026-02-11T23:30:19.362+0000 7f41591a6640 10 osd.0:7.consume_map 8 (was 7) 2026-02-11T23:30:19.362+0000 7f41591a6640 10 osd.0 8 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:19.362+0000 7f41591a6640 7 osd.0 8 activate_map version 8 2026-02-11T23:30:19.362+0000 7f4150194640 20 osd.0 8 agent_entry empty queue 2026-02-11T23:30:19.462+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 20 ==== osd_map(8..9 src has 1..9) ==== 1748+0+0 (crc 0 0 0) 0x4a64e00 con 0x4acbc00 2026-02-11T23:30:19.462+0000 7f415cf02640 20 osd.0 8 OSD::ms_dispatch: osd_map(8..9 src has 1..9) 2026-02-11T23:30:19.462+0000 7f415cf02640 20 osd.0 8 _dispatch 0x4a64e00 osd_map(8..9 src has 1..9) 2026-02-11T23:30:19.462+0000 7f415cf02640 3 osd.0 8 handle_osd_map epochs [8,9], i have 8, src has [1,9] 2026-02-11T23:30:19.462+0000 7f415cf02640 10 osd.0 8 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:19.462+0000 7f415cf02640 10 osd.0 8 handle_osd_map got inc map for epoch 9 2026-02-11T23:30:19.462+0000 7f415cf02640 20 osd.0 8 got_full_map 9, nothing requested 2026-02-11T23:30:19.462+0000 7f415cf02640 20 osd.0 8 trim_maps: min=0 oldest_map=1 2026-02-11T23:30:19.462+0000 7f415cf02640 20 osd.0 8 handle_osd_map pg_num_history pg_num_history(e9 pg_nums {} deleted_pools ) 2026-02-11T23:30:19.462+0000 7f415cf02640 10 snap_mapper record_purged_snaps purged_snaps {9={}} 2026-02-11T23:30:19.462+0000 7f415cf02640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:19.462+0000 7f415cf02640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e9 maps [1~9] lci=[5,9] tlb=1) 2026-02-11T23:30:19.463+0000 7f41591a6640 10 osd.0 8 _committed_osd_maps 9..9 2026-02-11T23:30:19.463+0000 7f41591a6640 10 osd.0 8 advance to epoch 9 (<= last 9 <= newest_map 9) 2026-02-11T23:30:19.463+0000 7f41591a6640 20 osd.0 9 consume_map version 9 2026-02-11T23:30:19.463+0000 7f41591a6640 20 osd.0 8 await_reserved_maps epoch:9 2026-02-11T23:30:19.463+0000 7f41591a6640 20 osd.0 8 await_reserved_maps done 0 2026-02-11T23:30:19.463+0000 7f41591a6640 20 osd.0 9 consume_map 9 -- publish done 2026-02-11T23:30:19.463+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:19.463+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:19.463+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:19.463+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:19.463+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:19.463+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:19.463+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:19.463+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:19.463+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:19.463+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:19.463+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:19.463+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:19.463+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:19.463+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:19.463+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:19.463+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:19.463+0000 7f41591a6640 20 osd.0 9 prune_pg_created 2026-02-11T23:30:19.463+0000 7f41591a6640 10 osd.0:0.consume_map 9 (was 8) 2026-02-11T23:30:19.463+0000 7f41591a6640 10 osd.0:1.consume_map 9 (was 8) 2026-02-11T23:30:19.463+0000 7f41591a6640 10 osd.0:2.consume_map 9 (was 8) 2026-02-11T23:30:19.463+0000 7f41591a6640 10 osd.0:3.consume_map 9 (was 8) 2026-02-11T23:30:19.463+0000 7f41591a6640 10 osd.0:4.consume_map 9 (was 8) 2026-02-11T23:30:19.463+0000 7f41591a6640 10 osd.0:5.consume_map 9 (was 8) 2026-02-11T23:30:19.463+0000 7f41591a6640 10 osd.0:6.consume_map 9 (was 8) 2026-02-11T23:30:19.463+0000 7f41591a6640 10 osd.0:7.consume_map 9 (was 8) 2026-02-11T23:30:19.463+0000 7f41591a6640 10 osd.0 9 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:19.463+0000 7f41591a6640 7 osd.0 9 activate_map version 9 2026-02-11T23:30:19.463+0000 7f4150194640 20 osd.0 9 agent_entry empty queue 2026-02-11T23:30:19.869+0000 7f416c3fd640 1 --2- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] >> conn(0x4aca400 0x4b5b600 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).accept 2026-02-11T23:30:19.870+0000 7f416c3fd640 1 --2- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] >> conn(0x4aca000 0x4b5d700 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).accept 2026-02-11T23:30:19.870+0000 7f416cbfe640 1 --2- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] >> conn(0x4aca000 0x4b5d700 unknown :-1 s=BANNER_ACCEPTING pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:19.870+0000 7f416c3fd640 1 --2- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] >> conn(0x4aca400 0x4b5b600 unknown :-1 s=BANNER_ACCEPTING pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:19.870+0000 7f416cbfe640 10 osd.0 9 ms_handle_fast_authentication new session 0x4a656c0 con 0x4aca000 entity osd.1 addr 2026-02-11T23:30:19.870+0000 7f416c3fd640 10 osd.0 9 ms_handle_fast_authentication new session 0x4a64540 con 0x4aca400 entity osd.1 addr 2026-02-11T23:30:19.870+0000 7f416cbfe640 1 --2- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] >> 127.0.0.1:0/3043114013 conn(0x4aca000 0x4b5d700 crc :-1 s=READY pgs=1 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-11T23:30:19.870+0000 7f416c3fd640 1 --2- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] >> 127.0.0.1:0/3043114013 conn(0x4aca400 0x4b5b600 crc :-1 s=READY pgs=1 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-11T23:30:20.151+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 21 ==== mgrmap(e 4) ==== 53858+0+0 (crc 0 0 0) 0x4a34600 con 0x4acbc00 2026-02-11T23:30:20.253+0000 7f4162bea640 10 osd.0 9 tick_without_osd_lock 2026-02-11T23:30:20.253+0000 7f4162bea640 0 osd.0 9 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:20.253+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:20.253+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.538 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:20.253+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:20.253, recovery is active?:false restrictions:<....> 2026-02-11T23:30:20.253+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:20.253+0000 7f4162bea640 10 osd.0 9 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:20.253+0000 7f4162bea640 20 osd.0 9 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:20.253+0000 7f4162bea640 10 osd.0 9 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:20.253+0000 7f4162bea640 20 osd.0 9 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:20.293+0000 7f41633eb640 10 osd.0 9 tick 2026-02-11T23:30:20.293+0000 7f41633eb640 20 osd.0 9 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:30:20.326+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:30:20.326+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:30:20.326+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:29:50.326607+0000) 2026-02-11T23:30:20.326+0000 7f415bf00640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-11T23:30:20.326+0000 7f415bf00640 10 log_client log_queue is 2 last_log 2 sent 0 num 2 unsent 2 sending 2 2026-02-11T23:30:20.326+0000 7f415bf00640 10 log_client will send 2026-02-11T23:30:18.281173+0000 osd.0 (osd.0) 1 : cluster [DBG] purged_snaps scrub starts 2026-02-11T23:30:20.326+0000 7f415bf00640 10 log_client will send 2026-02-11T23:30:18.281204+0000 osd.0 (osd.0) 2 : cluster [DBG] purged_snaps scrub ok 2026-02-11T23:30:20.326+0000 7f415bf00640 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:30:20.326+0000 7f415bf00640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> v2:127.0.0.1:7180/0 -- log(2 entries from seq 1 at 2026-02-11T23:30:18.281173+0000) -- 0x463fc00 con 0x4acbc00 2026-02-11T23:30:20.351+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 22 ==== log(last 2) ==== 28+0+0 (crc 0 0 0) 0x4c9a000 con 0x4acbc00 2026-02-11T23:30:20.351+0000 7f415cf02640 10 log_client handle_log_ack log(last 2) 2026-02-11T23:30:20.351+0000 7f415cf02640 10 log_client logged 2026-02-11T23:30:18.281173+0000 osd.0 (osd.0) 1 : cluster [DBG] purged_snaps scrub starts 2026-02-11T23:30:20.351+0000 7f415cf02640 10 log_client logged 2026-02-11T23:30:18.281204+0000 osd.0 (osd.0) 2 : cluster [DBG] purged_snaps scrub ok 2026-02-11T23:30:20.720+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 23 ==== osd_map(10..11 src has 1..11) ==== 950+0+0 (crc 0 0 0) 0x463fc00 con 0x4acbc00 2026-02-11T23:30:20.720+0000 7f415cf02640 20 osd.0 9 OSD::ms_dispatch: osd_map(10..11 src has 1..11) 2026-02-11T23:30:20.720+0000 7f415cf02640 20 osd.0 9 _dispatch 0x463fc00 osd_map(10..11 src has 1..11) 2026-02-11T23:30:20.720+0000 7f415cf02640 3 osd.0 9 handle_osd_map epochs [10,11], i have 9, src has [1,11] 2026-02-11T23:30:20.721+0000 7f415cf02640 10 osd.0 9 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:20.721+0000 7f415cf02640 10 osd.0 9 handle_osd_map got inc map for epoch 10 2026-02-11T23:30:20.721+0000 7f415cf02640 20 osd.0 9 got_full_map 10, nothing requested 2026-02-11T23:30:20.721+0000 7f415cf02640 10 osd.0 9 handle_osd_map got inc map for epoch 11 2026-02-11T23:30:20.721+0000 7f415cf02640 20 osd.0 9 got_full_map 11, nothing requested 2026-02-11T23:30:20.721+0000 7f415cf02640 20 osd.0 9 trim_maps: min=0 oldest_map=1 2026-02-11T23:30:20.721+0000 7f415cf02640 20 osd.0 9 handle_osd_map pg_num_history pg_num_history(e11 pg_nums {} deleted_pools ) 2026-02-11T23:30:20.721+0000 7f415cf02640 10 snap_mapper record_purged_snaps purged_snaps {10={},11={}} 2026-02-11T23:30:20.721+0000 7f415cf02640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:20.721+0000 7f415cf02640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e11 maps [1~11] lci=[5,11] tlb=1) 2026-02-11T23:30:20.721+0000 7f41591a6640 10 osd.0 9 _committed_osd_maps 10..11 2026-02-11T23:30:20.721+0000 7f41591a6640 10 osd.0 9 advance to epoch 10 (<= last 11 <= newest_map 11) 2026-02-11T23:30:20.721+0000 7f41591a6640 10 osd.0 10 advance to epoch 11 (<= last 11 <= newest_map 11) 2026-02-11T23:30:20.721+0000 7f41591a6640 20 osd.0 11 consume_map version 11 2026-02-11T23:30:20.721+0000 7f41591a6640 20 osd.0 9 await_reserved_maps epoch:11 2026-02-11T23:30:20.721+0000 7f41591a6640 20 osd.0 9 await_reserved_maps done 0 2026-02-11T23:30:20.721+0000 7f41591a6640 20 osd.0 11 consume_map 11 -- publish done 2026-02-11T23:30:20.721+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:20.721+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:20.721+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:20.721+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:20.721+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:20.721+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:20.721+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:20.721+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:20.721+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:20.721+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:20.721+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:20.721+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:20.721+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:20.721+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:20.721+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:20.721+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:20.721+0000 7f41591a6640 20 osd.0 11 prune_pg_created 2026-02-11T23:30:20.721+0000 7f41591a6640 10 osd.0:0.consume_map 11 (was 9) 2026-02-11T23:30:20.721+0000 7f41591a6640 10 osd.0:1.consume_map 11 (was 9) 2026-02-11T23:30:20.721+0000 7f41591a6640 10 osd.0:2.consume_map 11 (was 9) 2026-02-11T23:30:20.721+0000 7f41591a6640 10 osd.0:3.consume_map 11 (was 9) 2026-02-11T23:30:20.721+0000 7f41591a6640 10 osd.0:4.consume_map 11 (was 9) 2026-02-11T23:30:20.721+0000 7f41591a6640 10 osd.0:5.consume_map 11 (was 9) 2026-02-11T23:30:20.721+0000 7f41591a6640 10 osd.0:6.consume_map 11 (was 9) 2026-02-11T23:30:20.721+0000 7f41591a6640 10 osd.0:7.consume_map 11 (was 9) 2026-02-11T23:30:20.721+0000 7f41591a6640 10 osd.0 11 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:20.721+0000 7f41591a6640 10 osd.0 11 maybe_update_heartbeat_peers updating 2026-02-11T23:30:20.721+0000 7f41591a6640 10 osd.0 11 adding neighbor peer osd.1 2026-02-11T23:30:20.721+0000 7f41591a6640 20 osd.0 11 get_con_osd_hb to osd.1 from_epoch 11 2026-02-11T23:30:20.721+0000 7f41591a6640 20 osd.0 11 get_nextmap_reserved map_reservations: {11=1} 2026-02-11T23:30:20.721+0000 7f41591a6640 1 --2- >> [v2:127.0.0.1:6816/3043114013,v1:127.0.0.1:6817/3043114013] conn(0x48c4400 0x4b5bb80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-11T23:30:20.721+0000 7f41591a6640 1 --2- >> [v2:127.0.0.1:6814/3043114013,v1:127.0.0.1:6815/3043114013] conn(0x4640c00 0x3810000 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-11T23:30:20.721+0000 7f41591a6640 20 osd.0 11 release_map epoch: 11 2026-02-11T23:30:20.721+0000 7f41591a6640 10 osd.0 11 _add_heartbeat_peer: new peer osd.1 v2:127.0.0.1:6816/3043114013 v2:127.0.0.1:6814/3043114013 2026-02-11T23:30:20.721+0000 7f41591a6640 10 osd.0 11 maybe_update_heartbeat_peers 1 peers, extras 1 2026-02-11T23:30:20.721+0000 7f41591a6640 7 osd.0 11 activate_map version 11 2026-02-11T23:30:20.721+0000 7f4150194640 20 osd.0 11 agent_entry empty queue 2026-02-11T23:30:20.721+0000 7f416abfa640 1 --2- >> [v2:127.0.0.1:6816/3043114013,v1:127.0.0.1:6817/3043114013] conn(0x48c4400 0x4b5bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:20.721+0000 7f416abfa640 1 --2- >> [v2:127.0.0.1:6816/3043114013,v1:127.0.0.1:6817/3043114013] conn(0x48c4400 0x4b5bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:127.0.0.1:6816/3043114013 says I am v2:127.0.0.1:42912/0 (socket says 127.0.0.1:42912) 2026-02-11T23:30:20.721+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 learned_addr learned my addr 127.0.0.1:0/3294232400 (peer_addr_for_me v2:127.0.0.1:0/0) 2026-02-11T23:30:20.721+0000 7f416abfa640 10 monclient: get_auth_request con 0x48c4400 auth_method 0 2026-02-11T23:30:20.721+0000 7f416abfa640 1 --2- 127.0.0.1:0/3294232400 >> [v2:127.0.0.1:6816/3043114013,v1:127.0.0.1:6817/3043114013] conn(0x48c4400 0x4b5bb80 crc :-1 s=READY pgs=1 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-11T23:30:20.734+0000 7f416c3fd640 1 --2- >> [v2:127.0.0.1:6814/3043114013,v1:127.0.0.1:6815/3043114013] conn(0x4640c00 0x3810000 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:20.734+0000 7f416c3fd640 1 --2- >> [v2:127.0.0.1:6814/3043114013,v1:127.0.0.1:6815/3043114013] conn(0x4640c00 0x3810000 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:127.0.0.1:6814/3043114013 says I am v2:127.0.0.1:36674/0 (socket says 127.0.0.1:36674) 2026-02-11T23:30:20.734+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 learned_addr learned my addr 127.0.0.1:0/3294232400 (peer_addr_for_me v2:127.0.0.1:0/0) 2026-02-11T23:30:20.734+0000 7f416c3fd640 10 monclient: get_auth_request con 0x4640c00 auth_method 0 2026-02-11T23:30:20.734+0000 7f416c3fd640 1 --2- 127.0.0.1:0/3294232400 >> [v2:127.0.0.1:6814/3043114013,v1:127.0.0.1:6815/3043114013] conn(0x4640c00 0x3810000 crc :-1 s=READY pgs=1 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-11T23:30:21.212+0000 7f4162bea640 10 osd.0 11 tick_without_osd_lock 2026-02-11T23:30:21.212+0000 7f4162bea640 0 osd.0 11 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:21.212+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:21.212+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.538 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:21.212+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:21.212, recovery is active?:false restrictions:<....> 2026-02-11T23:30:21.212+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:21.212+0000 7f4162bea640 10 osd.0 11 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:21.212+0000 7f4162bea640 20 osd.0 11 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:21.212+0000 7f4162bea640 10 osd.0 11 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:21.212+0000 7f4162bea640 20 osd.0 11 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:21.308+0000 7f41633eb640 10 osd.0 11 tick 2026-02-11T23:30:21.308+0000 7f41633eb640 20 osd.0 11 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:30:21.326+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:30:21.326+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:30:21.326+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:29:51.326700+0000) 2026-02-11T23:30:21.978+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 24 ==== osd_map(11..12 src has 1..12) ==== 1209+0+0 (crc 0 0 0) 0x4a65180 con 0x4acbc00 2026-02-11T23:30:21.978+0000 7f415cf02640 20 osd.0 11 OSD::ms_dispatch: osd_map(11..12 src has 1..12) 2026-02-11T23:30:21.978+0000 7f415cf02640 20 osd.0 11 _dispatch 0x4a65180 osd_map(11..12 src has 1..12) 2026-02-11T23:30:21.978+0000 7f415cf02640 3 osd.0 11 handle_osd_map epochs [11,12], i have 11, src has [1,12] 2026-02-11T23:30:21.978+0000 7f415cf02640 10 osd.0 11 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:21.978+0000 7f415cf02640 10 osd.0 11 handle_osd_map got inc map for epoch 12 2026-02-11T23:30:21.978+0000 7f415cf02640 20 osd.0 11 got_full_map 12, nothing requested 2026-02-11T23:30:21.978+0000 7f415cf02640 20 osd.0 11 trim_maps: min=0 oldest_map=1 2026-02-11T23:30:21.978+0000 7f415cf02640 20 osd.0 11 handle_osd_map pg_num_history pg_num_history(e12 pg_nums {} deleted_pools ) 2026-02-11T23:30:21.978+0000 7f415cf02640 10 snap_mapper record_purged_snaps purged_snaps {12={}} 2026-02-11T23:30:21.978+0000 7f415cf02640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:21.978+0000 7f415cf02640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e12 maps [1~12] lci=[5,12] tlb=1) 2026-02-11T23:30:21.979+0000 7f41591a6640 10 osd.0 11 _committed_osd_maps 12..12 2026-02-11T23:30:21.979+0000 7f41591a6640 10 osd.0 11 advance to epoch 12 (<= last 12 <= newest_map 12) 2026-02-11T23:30:21.979+0000 7f41591a6640 20 osd.0 12 consume_map version 12 2026-02-11T23:30:21.979+0000 7f41591a6640 20 osd.0 11 await_reserved_maps epoch:12 2026-02-11T23:30:21.979+0000 7f41591a6640 20 osd.0 11 await_reserved_maps done 0 2026-02-11T23:30:21.979+0000 7f41591a6640 20 osd.0 12 consume_map 12 -- publish done 2026-02-11T23:30:21.979+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:21.979+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:21.979+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:21.979+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:21.979+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:21.979+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:21.979+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:21.979+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:21.979+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:21.979+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:21.979+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:21.979+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:21.979+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:21.979+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:21.979+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:21.979+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:21.979+0000 7f41591a6640 20 osd.0 12 prune_pg_created 2026-02-11T23:30:21.979+0000 7f41591a6640 10 osd.0:0.consume_map 12 (was 11) 2026-02-11T23:30:21.979+0000 7f41591a6640 10 osd.0:1.consume_map 12 (was 11) 2026-02-11T23:30:21.979+0000 7f41591a6640 10 osd.0:2.consume_map 12 (was 11) 2026-02-11T23:30:21.979+0000 7f41591a6640 10 osd.0:3.consume_map 12 (was 11) 2026-02-11T23:30:21.979+0000 7f41591a6640 10 osd.0:4.consume_map 12 (was 11) 2026-02-11T23:30:21.979+0000 7f41591a6640 10 osd.0:5.consume_map 12 (was 11) 2026-02-11T23:30:21.979+0000 7f41591a6640 10 osd.0:6.consume_map 12 (was 11) 2026-02-11T23:30:21.979+0000 7f41591a6640 10 osd.0:7.consume_map 12 (was 11) 2026-02-11T23:30:21.979+0000 7f41591a6640 10 osd.0 12 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:21.979+0000 7f41591a6640 7 osd.0 12 activate_map version 12 2026-02-11T23:30:21.979+0000 7f4150194640 20 osd.0 12 agent_entry empty queue 2026-02-11T23:30:22.027+0000 7f4147983640 5 osd.0 12 heartbeat osd_stat(store_statfs(0x384000/0x0/0x384000, data 0x507e/0x18000, compress 0x0/0x0/0x0, omap 0x1d3c, meta 0x1a4e2c4), peers [1] op hist []) 2026-02-11T23:30:22.027+0000 7f4147983640 20 osd.0 12 check_full_status cur ratio 0, physical ratio 0, new state none 2026-02-11T23:30:22.027+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6816/3043114013,v1:127.0.0.1:6817/3043114013] -- osd_ping(ping e12 up_from 5 ping_stamp 2026-02-11T23:30:22.027652+0000/4.703057289s send_stamp 4.703057289s) -- 0x4c9a180 con 0x48c4400 2026-02-11T23:30:22.027+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6814/3043114013,v1:127.0.0.1:6815/3043114013] -- osd_ping(ping e12 up_from 5 ping_stamp 2026-02-11T23:30:22.027652+0000/4.703057289s send_stamp 4.703057289s) -- 0x4c9a300 con 0x4640c00 2026-02-11T23:30:22.027+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6814/3043114013 1 ==== osd_ping(ping_reply e10 up_from 10 ping_stamp 2026-02-11T23:30:22.027652+0000/4.703057289s send_stamp 2.691564798s delta_ub -2.011492729s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a300 con 0x4640c00 2026-02-11T23:30:22.027+0000 7f416c3fd640 20 osd.0 12 get_con_osd_cluster to osd.1 from_epoch 12 2026-02-11T23:30:22.027+0000 7f416c3fd640 20 osd.0 12 get_nextmap_reserved map_reservations: {12=1} 2026-02-11T23:30:22.027+0000 7f416c3fd640 1 --2- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] >> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] conn(0x4acb000 0x3812100 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-11T23:30:22.027+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6816/3043114013 1 ==== osd_ping(ping_reply e10 up_from 10 ping_stamp 2026-02-11T23:30:22.027652+0000/4.703057289s send_stamp 2.691647053s delta_ub -2.011410713s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a180 con 0x48c4400 2026-02-11T23:30:22.027+0000 7f416c3fd640 20 osd.0 12 release_map epoch: 12 2026-02-11T23:30:22.027+0000 7f416c3fd640 20 osd.0 12 handle_osd_ping new stamps hbstamp(osd.1 up_from 0 peer_clock_delta [-2.011789799s,-2.011492729s]) 2026-02-11T23:30:22.027+0000 7f416abfa640 20 osd.0 12 get_con_osd_cluster to osd.1 from_epoch 12 2026-02-11T23:30:22.027+0000 7f416abfa640 20 osd.0 12 get_nextmap_reserved map_reservations: {12=1} 2026-02-11T23:30:22.027+0000 7f416abfa640 20 osd.0 12 release_map epoch: 12 2026-02-11T23:30:22.027+0000 7f416abfa640 20 osd.0 12 handle_osd_ping new stamps hbstamp(osd.1 up_from 0 peer_clock_delta [-2.011740685s,-2.011410713s]) 2026-02-11T23:30:22.027+0000 7f416c3fd640 1 --2- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] >> conn(0x4641c00 0x3812680 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).accept 2026-02-11T23:30:22.027+0000 7f416abfa640 1 --2- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] >> conn(0x4641c00 0x3812680 unknown :-1 s=BANNER_ACCEPTING pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:22.027+0000 7f416abfa640 10 osd.0 12 ms_handle_fast_authentication new session 0x4a64000 con 0x4641c00 entity osd.1 addr 2026-02-11T23:30:22.027+0000 7f416cbfe640 1 --2- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] >> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] conn(0x4acb000 0x3812100 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:22.027+0000 7f416cbfe640 10 monclient: get_auth_request con 0x4acb000 auth_method 0 2026-02-11T23:30:22.027+0000 7f416abfa640 1 --2- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] >> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] conn(0x4641c00 0x3812680 crc :-1 s=SESSION_ACCEPTING pgs=1 gs=0 cs=0 l=0 c_cookie=1174193495719725409 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_existing_connection connection race detected, replacing existing=0x4acb000 socket by this connection's socket 2026-02-11T23:30:22.027+0000 7f416abfa640 1 --2- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] >> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] conn(0x4641c00 0x3812680 crc :-1 s=SESSION_ACCEPTING pgs=1 gs=0 cs=0 l=0 c_cookie=1174193495719725409 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-11T23:30:22.027+0000 7f416cbfe640 1 --2- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] >> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] conn(0x4acb000 0x3812100 crc :-1 s=NONE pgs=1 gs=1 cs=0 l=0 c_cookie=1174193495719725409 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-11T23:30:22.027+0000 7f415aefe640 2 osd.0 12 ms_handle_reset con 0x4641c00 session 0x4a64000 2026-02-11T23:30:22.027+0000 7f416abfa640 10 osd.0 12 new session (incoming)0x4a65880 con=0x4acb000 addr=v2:127.0.0.1:6812/3043114013 must have raced with connect 2026-02-11T23:30:22.028+0000 7f416abfa640 1 --2- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] >> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] conn(0x4acb000 0x3812100 unknown :-1 s=READY pgs=1 gs=2 cs=0 l=0 c_cookie=1174193495719725409 s_cookie=11033496277528658603 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=104b92c361f2c961 server_cookie=991eda680cb0c2ab in_seq=0 out_seq=0 2026-02-11T23:30:22.079+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 25 ==== osd_map(12..13 src has 1..13) ==== 1822+0+0 (crc 0 0 0) 0x4a64fc0 con 0x4acbc00 2026-02-11T23:30:22.079+0000 7f415cf02640 20 osd.0 12 OSD::ms_dispatch: osd_map(12..13 src has 1..13) 2026-02-11T23:30:22.079+0000 7f415cf02640 20 osd.0 12 _dispatch 0x4a64fc0 osd_map(12..13 src has 1..13) 2026-02-11T23:30:22.079+0000 7f415cf02640 3 osd.0 12 handle_osd_map epochs [12,13], i have 12, src has [1,13] 2026-02-11T23:30:22.079+0000 7f415cf02640 10 osd.0 12 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:22.079+0000 7f415cf02640 10 osd.0 12 handle_osd_map got inc map for epoch 13 2026-02-11T23:30:22.079+0000 7f415cf02640 20 osd.0 12 got_full_map 13, nothing requested 2026-02-11T23:30:22.079+0000 7f415cf02640 20 osd.0 12 trim_maps: min=0 oldest_map=1 2026-02-11T23:30:22.079+0000 7f415cf02640 20 osd.0 12 handle_osd_map pg_num_history pg_num_history(e13 pg_nums {} deleted_pools ) 2026-02-11T23:30:22.079+0000 7f415cf02640 10 snap_mapper record_purged_snaps purged_snaps {13={}} 2026-02-11T23:30:22.079+0000 7f415cf02640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:22.079+0000 7f415cf02640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e13 maps [1~13] lci=[5,13] tlb=1) 2026-02-11T23:30:22.079+0000 7f41591a6640 10 osd.0 12 _committed_osd_maps 13..13 2026-02-11T23:30:22.079+0000 7f41591a6640 10 osd.0 12 advance to epoch 13 (<= last 13 <= newest_map 13) 2026-02-11T23:30:22.079+0000 7f41591a6640 20 osd.0 13 consume_map version 13 2026-02-11T23:30:22.079+0000 7f41591a6640 20 osd.0 12 await_reserved_maps epoch:13 2026-02-11T23:30:22.079+0000 7f41591a6640 20 osd.0 12 await_reserved_maps done 0 2026-02-11T23:30:22.079+0000 7f41591a6640 20 osd.0 13 consume_map 13 -- publish done 2026-02-11T23:30:22.079+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:22.079+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:22.079+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:22.079+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:22.079+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:22.079+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:22.079+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:22.079+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:22.079+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:22.079+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:22.079+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:22.079+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:22.079+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:22.079+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:22.079+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:22.079+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:22.079+0000 7f41591a6640 20 osd.0 13 prune_pg_created 2026-02-11T23:30:22.079+0000 7f41591a6640 10 osd.0:0.consume_map 13 (was 12) 2026-02-11T23:30:22.079+0000 7f41591a6640 10 osd.0:1.consume_map 13 (was 12) 2026-02-11T23:30:22.079+0000 7f41591a6640 10 osd.0:2.consume_map 13 (was 12) 2026-02-11T23:30:22.079+0000 7f41591a6640 10 osd.0:3.consume_map 13 (was 12) 2026-02-11T23:30:22.079+0000 7f41591a6640 10 osd.0:4.consume_map 13 (was 12) 2026-02-11T23:30:22.079+0000 7f41591a6640 10 osd.0:5.consume_map 13 (was 12) 2026-02-11T23:30:22.079+0000 7f41591a6640 10 osd.0:6.consume_map 13 (was 12) 2026-02-11T23:30:22.079+0000 7f41591a6640 10 osd.0:7.consume_map 13 (was 12) 2026-02-11T23:30:22.079+0000 7f41591a6640 10 osd.0 13 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:22.079+0000 7f41591a6640 7 osd.0 13 activate_map version 13 2026-02-11T23:30:22.079+0000 7f4150194640 20 osd.0 13 agent_entry empty queue 2026-02-11T23:30:22.213+0000 7f4162bea640 10 osd.0 13 tick_without_osd_lock 2026-02-11T23:30:22.213+0000 7f4162bea640 0 osd.0 13 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:22.213+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:22.213+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:22.213+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:22.213+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.538 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:22.213+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:22.213, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:22.213+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:22.213+0000 7f4162bea640 10 osd.0 13 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:22.213+0000 7f4162bea640 20 osd.0 13 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:22.213+0000 7f4162bea640 10 osd.0 13 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:22.213+0000 7f4162bea640 20 osd.0 13 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:22.326+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:30:22.326+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:30:22.326+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:29:52.326762+0000) 2026-02-11T23:30:22.346+0000 7f41633eb640 10 osd.0 13 tick 2026-02-11T23:30:22.346+0000 7f41633eb640 20 osd.0 13 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:30:22.520+0000 7f416c3fd640 1 --2- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] >> conn(0x4641000 0x3812c00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).accept 2026-02-11T23:30:22.520+0000 7f416c3fd640 1 --2- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] >> conn(0x48cc000 0x3810b00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).accept 2026-02-11T23:30:22.520+0000 7f416cbfe640 1 --2- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] >> conn(0x4641000 0x3812c00 unknown :-1 s=BANNER_ACCEPTING pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:22.520+0000 7f416c3fd640 1 --2- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] >> conn(0x48cc000 0x3810b00 unknown :-1 s=BANNER_ACCEPTING pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:22.520+0000 7f416c3fd640 10 osd.0 13 ms_handle_fast_authentication new session 0x4a641c0 con 0x48cc000 entity osd.2 addr 2026-02-11T23:30:22.520+0000 7f416cbfe640 10 osd.0 13 ms_handle_fast_authentication new session 0x4a64a80 con 0x4641000 entity osd.2 addr 2026-02-11T23:30:22.520+0000 7f416c3fd640 1 --2- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] >> 127.0.0.1:0/2365263801 conn(0x48cc000 0x3810b00 crc :-1 s=READY pgs=1 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-11T23:30:22.520+0000 7f416cbfe640 1 --2- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] >> 127.0.0.1:0/2365263801 conn(0x4641000 0x3812c00 crc :-1 s=READY pgs=1 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-11T23:30:23.184+0000 7f4162bea640 10 osd.0 13 tick_without_osd_lock 2026-02-11T23:30:23.184+0000 7f4162bea640 0 osd.0 13 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:23.184+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:23.184+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.495 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:23.184+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:23.185, recovery is active?:false restrictions:<....> 2026-02-11T23:30:23.184+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:23.184+0000 7f4162bea640 10 osd.0 13 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:23.184+0000 7f4162bea640 20 osd.0 13 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:23.184+0000 7f4162bea640 10 osd.0 13 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:23.184+0000 7f4162bea640 20 osd.0 13 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:23.203+0000 7f415b6ff640 20 osd.0 13 reports for 0 queries 2026-02-11T23:30:23.203+0000 7f415b6ff640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] -- mgrreport(unknown.0 +0-0 packed 3294 daemon_metrics=2) -- 0x4f9f180 con 0x4aca800 2026-02-11T23:30:23.203+0000 7f415b6ff640 15 osd.0 13 collect_pg_stats 2026-02-11T23:30:23.203+0000 7f415b6ff640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] -- pg_stats(0 pgs seq 21474836482 v 0) -- 0x48ce000 con 0x4aca800 2026-02-11T23:30:23.322+0000 7f41633eb640 10 osd.0 13 tick 2026-02-11T23:30:23.322+0000 7f41633eb640 20 osd.0 13 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:30:23.326+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:30:23.326+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:30:23.326+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:29:53.326834+0000) 2026-02-11T23:30:24.037+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.1 127.0.0.1:0/3043114013 1 ==== osd_ping(ping e16 up_from 10 ping_stamp 2026-02-11T23:30:24.038853+0000/4.702598572s send_stamp 4.702598572s delta_ub -2.011410713s) ==== 2033+0+0 (crc 0 0 0) 0x4a89200 con 0x4aca000 2026-02-11T23:30:24.037+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.1 127.0.0.1:0/3043114013 1 ==== osd_ping(ping e16 up_from 10 ping_stamp 2026-02-11T23:30:24.038853+0000/4.702598572s send_stamp 4.702598572s delta_ub -2.011410713s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a300 con 0x4aca400 2026-02-11T23:30:24.037+0000 7f416cbfe640 20 osd.0 13 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011774302s,-2.011410713s]) 2026-02-11T23:30:24.037+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e13 up_from 5 ping_stamp 2026-02-11T23:30:24.038853+0000/4.702598572s send_stamp 6.714373112s delta_ub 2.011774302s) -- 0x4a89980 con 0x4aca000 2026-02-11T23:30:24.037+0000 7f416cbfe640 20 osd.0 13 get_con_osd_cluster to osd.1 from_epoch 13 2026-02-11T23:30:24.037+0000 7f416cbfe640 20 osd.0 13 get_nextmap_reserved map_reservations: {13=1} 2026-02-11T23:30:24.037+0000 7f416cbfe640 20 osd.0 13 release_map epoch: 13 2026-02-11T23:30:24.037+0000 7f416cbfe640 10 osd.0 13 maybe_share_map: con v2:127.0.0.1:6812/3043114013 updating session's projected_epoch from 0 to ping map epoch of 16 2026-02-11T23:30:24.037+0000 7f416cbfe640 10 osd.0 13 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 13 is not newer than session's projected_epoch of 16 2026-02-11T23:30:24.037+0000 7f416c3fd640 20 osd.0 13 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011797428s,-2.011410713s]) 2026-02-11T23:30:24.037+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e13 up_from 5 ping_stamp 2026-02-11T23:30:24.038853+0000/4.702598572s send_stamp 6.714396000s delta_ub 2.011797428s) -- 0x4c9a600 con 0x4aca400 2026-02-11T23:30:24.037+0000 7f416c3fd640 20 osd.0 13 get_con_osd_cluster to osd.1 from_epoch 13 2026-02-11T23:30:24.037+0000 7f416c3fd640 20 osd.0 13 get_nextmap_reserved map_reservations: {13=1} 2026-02-11T23:30:24.037+0000 7f416c3fd640 20 osd.0 13 release_map epoch: 13 2026-02-11T23:30:24.037+0000 7f416c3fd640 10 osd.0 13 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 13 is not newer than session's projected_epoch of 16 2026-02-11T23:30:24.038+0000 7f415aefe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1 ==== osd_map(14..16 src has 1..16) ==== 1252+0+0 (unknown 0 0 0) 0x4a64380 con 0x4acb000 2026-02-11T23:30:24.038+0000 7f415aefe640 20 osd.0 13 OSD::ms_dispatch: osd_map(14..16 src has 1..16) 2026-02-11T23:30:24.038+0000 7f415aefe640 20 osd.0 13 _dispatch 0x4a64380 osd_map(14..16 src has 1..16) 2026-02-11T23:30:24.038+0000 7f415aefe640 3 osd.0 13 handle_osd_map epochs [14,16], i have 13, src has [1,16] 2026-02-11T23:30:24.038+0000 7f415aefe640 10 osd.0 13 handle_osd_map: received 0 full maps and 3 incremental maps 2026-02-11T23:30:24.038+0000 7f415aefe640 10 osd.0 13 handle_osd_map got inc map for epoch 14 2026-02-11T23:30:24.038+0000 7f415aefe640 20 osd.0 13 got_full_map 14, nothing requested 2026-02-11T23:30:24.038+0000 7f415aefe640 10 osd.0 13 handle_osd_map got inc map for epoch 15 2026-02-11T23:30:24.038+0000 7f415aefe640 20 osd.0 13 got_full_map 15, nothing requested 2026-02-11T23:30:24.038+0000 7f415aefe640 10 osd.0 13 handle_osd_map got inc map for epoch 16 2026-02-11T23:30:24.038+0000 7f415aefe640 20 osd.0 13 got_full_map 16, nothing requested 2026-02-11T23:30:24.038+0000 7f415aefe640 20 osd.0 13 trim_maps: min=0 oldest_map=1 2026-02-11T23:30:24.038+0000 7f415aefe640 20 osd.0 13 handle_osd_map pg_num_history pg_num_history(e16 pg_nums {} deleted_pools ) 2026-02-11T23:30:24.038+0000 7f415aefe640 10 snap_mapper record_purged_snaps purged_snaps {14={},15={},16={}} 2026-02-11T23:30:24.038+0000 7f415aefe640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:24.038+0000 7f415aefe640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e16 maps [1~16] lci=[5,16] tlb=1) 2026-02-11T23:30:24.038+0000 7f41591a6640 10 osd.0 13 _committed_osd_maps 14..16 2026-02-11T23:30:24.038+0000 7f41591a6640 10 osd.0 13 advance to epoch 14 (<= last 16 <= newest_map 16) 2026-02-11T23:30:24.038+0000 7f41591a6640 10 osd.0 14 advance to epoch 15 (<= last 16 <= newest_map 16) 2026-02-11T23:30:24.038+0000 7f41591a6640 10 osd.0 15 advance to epoch 16 (<= last 16 <= newest_map 16) 2026-02-11T23:30:24.038+0000 7f41591a6640 20 osd.0 16 consume_map version 16 2026-02-11T23:30:24.038+0000 7f41591a6640 20 osd.0 13 await_reserved_maps epoch:16 2026-02-11T23:30:24.038+0000 7f41591a6640 20 osd.0 13 await_reserved_maps done 0 2026-02-11T23:30:24.038+0000 7f41591a6640 20 osd.0 16 consume_map 16 -- publish done 2026-02-11T23:30:24.038+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:24.038+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:24.038+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:24.038+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:24.038+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:24.038+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:24.038+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:24.038+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:24.038+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:24.038+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:24.038+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:24.038+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:24.038+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:24.038+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:24.038+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:24.038+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:24.038+0000 7f41591a6640 20 osd.0 16 prune_pg_created 2026-02-11T23:30:24.038+0000 7f41591a6640 10 osd.0:0.consume_map 16 (was 13) 2026-02-11T23:30:24.038+0000 7f41591a6640 10 osd.0:1.consume_map 16 (was 13) 2026-02-11T23:30:24.038+0000 7f41591a6640 10 osd.0:2.consume_map 16 (was 13) 2026-02-11T23:30:24.038+0000 7f41591a6640 10 osd.0:3.consume_map 16 (was 13) 2026-02-11T23:30:24.038+0000 7f41591a6640 10 osd.0:4.consume_map 16 (was 13) 2026-02-11T23:30:24.038+0000 7f41591a6640 10 osd.0:5.consume_map 16 (was 13) 2026-02-11T23:30:24.038+0000 7f41591a6640 10 osd.0:6.consume_map 16 (was 13) 2026-02-11T23:30:24.038+0000 7f41591a6640 10 osd.0:7.consume_map 16 (was 13) 2026-02-11T23:30:24.038+0000 7f41591a6640 10 osd.0 16 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:24.038+0000 7f41591a6640 10 osd.0 16 maybe_update_heartbeat_peers updating 2026-02-11T23:30:24.038+0000 7f41591a6640 10 osd.0 16 adding neighbor peer osd.1 2026-02-11T23:30:24.038+0000 7f41591a6640 10 osd.0 16 adding neighbor peer osd.2 2026-02-11T23:30:24.038+0000 7f41591a6640 20 osd.0 16 get_con_osd_hb to osd.2 from_epoch 16 2026-02-11T23:30:24.038+0000 7f41591a6640 20 osd.0 16 get_nextmap_reserved map_reservations: {16=1} 2026-02-11T23:30:24.038+0000 7f41591a6640 1 --2- 127.0.0.1:0/3294232400 >> [v2:127.0.0.1:6824/2365263801,v1:127.0.0.1:6825/2365263801] conn(0x48c4000 0x4b5c100 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-11T23:30:24.038+0000 7f41591a6640 1 --2- 127.0.0.1:0/3294232400 >> [v2:127.0.0.1:6822/2365263801,v1:127.0.0.1:6823/2365263801] conn(0x48cd000 0x3810580 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-11T23:30:24.038+0000 7f41591a6640 20 osd.0 16 release_map epoch: 16 2026-02-11T23:30:24.038+0000 7f41591a6640 10 osd.0 16 _add_heartbeat_peer: new peer osd.2 v2:127.0.0.1:6824/2365263801 v2:127.0.0.1:6822/2365263801 2026-02-11T23:30:24.038+0000 7f41591a6640 10 osd.0 16 maybe_update_heartbeat_peers 2 peers, extras 1,2 2026-02-11T23:30:24.038+0000 7f41591a6640 7 osd.0 16 activate_map version 16 2026-02-11T23:30:24.038+0000 7f4150194640 20 osd.0 16 agent_entry empty queue 2026-02-11T23:30:24.038+0000 7f416cbfe640 1 --2- 127.0.0.1:0/3294232400 >> [v2:127.0.0.1:6824/2365263801,v1:127.0.0.1:6825/2365263801] conn(0x48c4000 0x4b5c100 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:24.038+0000 7f416abfa640 1 --2- 127.0.0.1:0/3294232400 >> [v2:127.0.0.1:6822/2365263801,v1:127.0.0.1:6823/2365263801] conn(0x48cd000 0x3810580 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:24.038+0000 7f416cbfe640 10 monclient: get_auth_request con 0x48c4000 auth_method 0 2026-02-11T23:30:24.038+0000 7f416abfa640 10 monclient: get_auth_request con 0x48cd000 auth_method 0 2026-02-11T23:30:24.039+0000 7f416cbfe640 1 --2- 127.0.0.1:0/3294232400 >> [v2:127.0.0.1:6824/2365263801,v1:127.0.0.1:6825/2365263801] conn(0x48c4000 0x4b5c100 crc :-1 s=READY pgs=2 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-11T23:30:24.039+0000 7f416abfa640 1 --2- 127.0.0.1:0/3294232400 >> [v2:127.0.0.1:6822/2365263801,v1:127.0.0.1:6823/2365263801] conn(0x48cd000 0x3810580 crc :-1 s=READY pgs=2 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-11T23:30:24.179+0000 7f4162bea640 10 osd.0 16 tick_without_osd_lock 2026-02-11T23:30:24.179+0000 7f4162bea640 0 osd.0 16 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:24.179+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:24.179+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.495 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:24.179+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:24.180, recovery is active?:false restrictions:<....> 2026-02-11T23:30:24.179+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:24.179+0000 7f4162bea640 10 osd.0 16 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:24.179+0000 7f4162bea640 20 osd.0 16 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:24.179+0000 7f4162bea640 10 osd.0 16 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:24.179+0000 7f4162bea640 20 osd.0 16 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:24.279+0000 7f41633eb640 10 osd.0 16 tick 2026-02-11T23:30:24.279+0000 7f41633eb640 20 osd.0 16 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:30:24.325+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:30:24.325+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:30:24.325+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:29:54.326899+0000) 2026-02-11T23:30:25.153+0000 7f416c3fd640 1 --2- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] >> conn(0x48cc400 0x3811080 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).accept 2026-02-11T23:30:25.153+0000 7f416c3fd640 1 --2- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] >> conn(0x48cc400 0x3811080 unknown :-1 s=BANNER_ACCEPTING pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:25.153+0000 7f416c3fd640 1 --2- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] >> conn(0x48cc800 0x3857080 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).accept 2026-02-11T23:30:25.153+0000 7f416cbfe640 1 --2- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] >> conn(0x48cc800 0x3857080 unknown :-1 s=BANNER_ACCEPTING pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:25.159+0000 7f416cbfe640 10 osd.0 16 ms_handle_fast_authentication new session 0x4a64700 con 0x48cc800 entity osd.3 addr 2026-02-11T23:30:25.159+0000 7f416c3fd640 10 osd.0 16 ms_handle_fast_authentication new session 0x463fc00 con 0x48cc400 entity osd.3 addr 2026-02-11T23:30:25.159+0000 7f416cbfe640 1 --2- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] >> 127.0.0.1:0/2699377081 conn(0x48cc800 0x3857080 crc :-1 s=READY pgs=1 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-11T23:30:25.159+0000 7f416c3fd640 1 --2- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] >> 127.0.0.1:0/2699377081 conn(0x48cc400 0x3811080 crc :-1 s=READY pgs=1 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-11T23:30:25.187+0000 7f4162bea640 10 osd.0 16 tick_without_osd_lock 2026-02-11T23:30:25.187+0000 7f4162bea640 0 osd.0 16 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:25.187+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:25.187+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:25.187+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:25.187+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.495 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:25.187+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:25.188, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:25.187+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:25.187+0000 7f4162bea640 10 osd.0 16 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:25.187+0000 7f4162bea640 20 osd.0 16 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:25.187+0000 7f4162bea640 10 osd.0 16 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:25.187+0000 7f4162bea640 20 osd.0 16 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:25.294+0000 7f41633eb640 10 osd.0 16 tick 2026-02-11T23:30:25.294+0000 7f41633eb640 20 osd.0 16 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:30:25.325+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:30:25.325+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:30:25.325+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:29:55.326956+0000) 2026-02-11T23:30:26.154+0000 7f4162bea640 10 osd.0 16 tick_without_osd_lock 2026-02-11T23:30:26.154+0000 7f4162bea640 0 osd.0 16 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:26.154+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:26.154+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:26.154+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:26.154+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.495 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:26.154+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:26.155, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:26.154+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:26.154+0000 7f4162bea640 10 osd.0 16 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:26.154+0000 7f4162bea640 20 osd.0 16 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:26.154+0000 7f4162bea640 10 osd.0 16 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:26.154+0000 7f4162bea640 20 osd.0 16 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:26.324+0000 7f41633eb640 10 osd.0 16 tick 2026-02-11T23:30:26.324+0000 7f41633eb640 20 osd.0 16 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:30:26.325+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:30:26.325+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:30:26.325+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:29:56.327027+0000) 2026-02-11T23:30:27.194+0000 7f4162bea640 10 osd.0 16 tick_without_osd_lock 2026-02-11T23:30:27.194+0000 7f4162bea640 0 osd.0 16 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:27.194+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:27.194+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:27.194+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:27.194+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.495 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:27.194+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:27.195, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:27.194+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:27.194+0000 7f4162bea640 10 osd.0 16 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:27.194+0000 7f4162bea640 20 osd.0 16 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:27.194+0000 7f4162bea640 10 osd.0 16 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:27.194+0000 7f4162bea640 20 osd.0 16 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:27.222+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 26 ==== osd_map(20..21 src has 1..21) ==== 1283+0+0 (crc 0 0 0) 0x4a30000 con 0x4acbc00 2026-02-11T23:30:27.222+0000 7f415cf02640 20 osd.0 16 OSD::ms_dispatch: osd_map(20..21 src has 1..21) 2026-02-11T23:30:27.222+0000 7f415cf02640 20 osd.0 16 _dispatch 0x4a30000 osd_map(20..21 src has 1..21) 2026-02-11T23:30:27.222+0000 7f415cf02640 10 monclient: _renew_subs 2026-02-11T23:30:27.222+0000 7f415cf02640 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:30:27.222+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> v2:127.0.0.1:7180/0 -- mon_subscribe({osdmap=17}) -- 0x3814000 con 0x4acbc00 2026-02-11T23:30:27.222+0000 7f415cf02640 3 osd.0 16 handle_osd_map epochs [20,21], i have 16, src has [1,21] 2026-02-11T23:30:27.222+0000 7f415cf02640 10 osd.0 16 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:27.222+0000 7f415cf02640 10 osd.0 16 handle_osd_map message skips epochs 17..19 2026-02-11T23:30:27.225+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 27 ==== osd_map(17..21 src has 1..21) ==== 3769+0+0 (crc 0 0 0) 0x4a301c0 con 0x4acbc00 2026-02-11T23:30:27.225+0000 7f415cf02640 20 osd.0 16 OSD::ms_dispatch: osd_map(17..21 src has 1..21) 2026-02-11T23:30:27.225+0000 7f415cf02640 20 osd.0 16 _dispatch 0x4a301c0 osd_map(17..21 src has 1..21) 2026-02-11T23:30:27.225+0000 7f415cf02640 3 osd.0 16 handle_osd_map epochs [17,21], i have 16, src has [1,21] 2026-02-11T23:30:27.225+0000 7f415cf02640 10 osd.0 16 handle_osd_map: received 0 full maps and 5 incremental maps 2026-02-11T23:30:27.225+0000 7f415cf02640 10 osd.0 16 handle_osd_map got inc map for epoch 17 2026-02-11T23:30:27.225+0000 7f415cf02640 20 osd.0 16 got_full_map 17, nothing requested 2026-02-11T23:30:27.225+0000 7f415cf02640 10 osd.0 16 handle_osd_map got inc map for epoch 18 2026-02-11T23:30:27.225+0000 7f415cf02640 20 osd.0 16 got_full_map 18, nothing requested 2026-02-11T23:30:27.225+0000 7f415cf02640 10 osd.0 16 handle_osd_map got inc map for epoch 19 2026-02-11T23:30:27.225+0000 7f415cf02640 20 osd.0 16 got_full_map 19, nothing requested 2026-02-11T23:30:27.225+0000 7f415cf02640 10 osd.0 16 handle_osd_map got inc map for epoch 20 2026-02-11T23:30:27.225+0000 7f415cf02640 20 osd.0 16 got_full_map 20, nothing requested 2026-02-11T23:30:27.225+0000 7f415cf02640 10 osd.0 16 handle_osd_map got inc map for epoch 21 2026-02-11T23:30:27.225+0000 7f415cf02640 20 osd.0 16 got_full_map 21, nothing requested 2026-02-11T23:30:27.225+0000 7f415cf02640 20 osd.0 16 trim_maps: min=0 oldest_map=1 2026-02-11T23:30:27.225+0000 7f415cf02640 20 osd.0 16 handle_osd_map pg_num_history pg_num_history(e21 pg_nums {} deleted_pools ) 2026-02-11T23:30:27.225+0000 7f415cf02640 10 snap_mapper record_purged_snaps purged_snaps {17={},18={},19={},20={},21={}} 2026-02-11T23:30:27.225+0000 7f415cf02640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:27.225+0000 7f415cf02640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e21 maps [1~21] lci=[5,21] tlb=1) 2026-02-11T23:30:27.225+0000 7f41591a6640 10 osd.0 16 _committed_osd_maps 17..21 2026-02-11T23:30:27.225+0000 7f41591a6640 10 osd.0 16 advance to epoch 17 (<= last 21 <= newest_map 21) 2026-02-11T23:30:27.225+0000 7f41591a6640 10 osd.0 17 advance to epoch 18 (<= last 21 <= newest_map 21) 2026-02-11T23:30:27.225+0000 7f41591a6640 10 osd.0 18 advance to epoch 19 (<= last 21 <= newest_map 21) 2026-02-11T23:30:27.225+0000 7f41591a6640 10 osd.0 19 advance to epoch 20 (<= last 21 <= newest_map 21) 2026-02-11T23:30:27.225+0000 7f41591a6640 10 osd.0 20 advance to epoch 21 (<= last 21 <= newest_map 21) 2026-02-11T23:30:27.225+0000 7f41591a6640 20 osd.0 21 consume_map version 21 2026-02-11T23:30:27.225+0000 7f41591a6640 20 osd.0 16 await_reserved_maps epoch:21 2026-02-11T23:30:27.225+0000 7f41591a6640 20 osd.0 16 await_reserved_maps done 0 2026-02-11T23:30:27.225+0000 7f41591a6640 20 osd.0 21 consume_map 21 -- publish done 2026-02-11T23:30:27.225+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:27.225+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:27.225+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:27.225+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:27.225+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:27.225+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:27.225+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:27.225+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:27.225+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:27.225+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:27.225+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:27.225+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:27.225+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:27.225+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:27.225+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:27.225+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:27.225+0000 7f41591a6640 20 osd.0 21 prune_pg_created 2026-02-11T23:30:27.225+0000 7f41591a6640 10 osd.0:0.consume_map 21 (was 16) 2026-02-11T23:30:27.225+0000 7f41591a6640 10 osd.0:1.consume_map 21 (was 16) 2026-02-11T23:30:27.225+0000 7f41591a6640 10 osd.0:2.consume_map 21 (was 16) 2026-02-11T23:30:27.225+0000 7f41591a6640 10 osd.0:3.consume_map 21 (was 16) 2026-02-11T23:30:27.225+0000 7f41591a6640 10 osd.0:4.consume_map 21 (was 16) 2026-02-11T23:30:27.225+0000 7f41591a6640 10 osd.0:5.consume_map 21 (was 16) 2026-02-11T23:30:27.225+0000 7f41591a6640 10 osd.0:6.consume_map 21 (was 16) 2026-02-11T23:30:27.225+0000 7f41591a6640 10 osd.0:7.consume_map 21 (was 16) 2026-02-11T23:30:27.225+0000 7f41591a6640 10 osd.0 21 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:27.225+0000 7f41591a6640 10 osd.0 21 maybe_update_heartbeat_peers updating 2026-02-11T23:30:27.225+0000 7f41591a6640 10 osd.0 21 adding neighbor peer osd.1 2026-02-11T23:30:27.225+0000 7f41591a6640 10 osd.0 21 adding neighbor peer osd.3 2026-02-11T23:30:27.225+0000 7f41591a6640 20 osd.0 21 get_con_osd_hb to osd.3 from_epoch 21 2026-02-11T23:30:27.225+0000 7f41591a6640 20 osd.0 21 get_nextmap_reserved map_reservations: {21=1} 2026-02-11T23:30:27.225+0000 7f41591a6640 1 --2- 127.0.0.1:0/3294232400 >> [v2:127.0.0.1:6832/2699377081,v1:127.0.0.1:6833/2699377081] conn(0x48cd400 0x3813180 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-11T23:30:27.225+0000 7f41591a6640 1 --2- 127.0.0.1:0/3294232400 >> [v2:127.0.0.1:6830/2699377081,v1:127.0.0.1:6831/2699377081] conn(0x48cd800 0x4a2d080 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-11T23:30:27.225+0000 7f41591a6640 20 osd.0 21 release_map epoch: 21 2026-02-11T23:30:27.225+0000 7f41591a6640 10 osd.0 21 _add_heartbeat_peer: new peer osd.3 v2:127.0.0.1:6832/2699377081 v2:127.0.0.1:6830/2699377081 2026-02-11T23:30:27.225+0000 7f41591a6640 10 osd.0 21 maybe_update_heartbeat_peers 3 peers, extras 1,2,3 2026-02-11T23:30:27.225+0000 7f41591a6640 7 osd.0 21 activate_map version 21 2026-02-11T23:30:27.225+0000 7f4150194640 20 osd.0 21 agent_entry empty queue 2026-02-11T23:30:27.225+0000 7f416abfa640 1 --2- 127.0.0.1:0/3294232400 >> [v2:127.0.0.1:6832/2699377081,v1:127.0.0.1:6833/2699377081] conn(0x48cd400 0x3813180 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:27.225+0000 7f416abfa640 10 monclient: get_auth_request con 0x48cd400 auth_method 0 2026-02-11T23:30:27.225+0000 7f416c3fd640 1 --2- 127.0.0.1:0/3294232400 >> [v2:127.0.0.1:6830/2699377081,v1:127.0.0.1:6831/2699377081] conn(0x48cd800 0x4a2d080 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:27.225+0000 7f416c3fd640 10 monclient: get_auth_request con 0x48cd800 auth_method 0 2026-02-11T23:30:27.226+0000 7f416abfa640 1 --2- 127.0.0.1:0/3294232400 >> [v2:127.0.0.1:6832/2699377081,v1:127.0.0.1:6833/2699377081] conn(0x48cd400 0x3813180 crc :-1 s=READY pgs=2 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-11T23:30:27.226+0000 7f416c3fd640 1 --2- 127.0.0.1:0/3294232400 >> [v2:127.0.0.1:6830/2699377081,v1:127.0.0.1:6831/2699377081] conn(0x48cd800 0x4a2d080 crc :-1 s=READY pgs=2 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-11T23:30:27.311+0000 7f41633eb640 10 osd.0 21 tick 2026-02-11T23:30:27.311+0000 7f41633eb640 20 osd.0 21 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:30:27.325+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:30:27.325+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:30:27.325+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:29:57.327106+0000) 2026-02-11T23:30:27.326+0000 7f4147983640 5 osd.0 21 heartbeat osd_stat(store_statfs(0x384000/0x0/0x384000, data 0x83a6/0x22000, compress 0x0/0x0/0x0, omap 0x24dd, meta 0x1a4db23), peers [1,2,3] op hist []) 2026-02-11T23:30:27.326+0000 7f4147983640 20 osd.0 21 check_full_status cur ratio 0, physical ratio 0, new state none 2026-02-11T23:30:27.326+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6816/3043114013,v1:127.0.0.1:6817/3043114013] -- osd_ping(ping e21 up_from 5 ping_stamp 2026-02-11T23:30:27.327832+0000/10.003236771s send_stamp 10.003236771s delta_ub 2.011797428s) -- 0x4c9a480 con 0x48c4400 2026-02-11T23:30:27.326+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6814/3043114013,v1:127.0.0.1:6815/3043114013] -- osd_ping(ping e21 up_from 5 ping_stamp 2026-02-11T23:30:27.327832+0000/10.003236771s send_stamp 10.003236771s delta_ub 2.011797428s) -- 0x4c9a780 con 0x4640c00 2026-02-11T23:30:27.326+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6824/2365263801,v1:127.0.0.1:6825/2365263801] -- osd_ping(ping e21 up_from 5 ping_stamp 2026-02-11T23:30:27.327832+0000/10.003236771s send_stamp 10.003236771s) -- 0x4c9a900 con 0x48c4000 2026-02-11T23:30:27.326+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6822/2365263801,v1:127.0.0.1:6823/2365263801] -- osd_ping(ping e21 up_from 5 ping_stamp 2026-02-11T23:30:27.327832+0000/10.003236771s send_stamp 10.003236771s) -- 0x4c9aa80 con 0x48cd000 2026-02-11T23:30:27.326+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6832/2699377081,v1:127.0.0.1:6833/2699377081] -- osd_ping(ping e21 up_from 5 ping_stamp 2026-02-11T23:30:27.327832+0000/10.003236771s send_stamp 10.003236771s) -- 0x4c9ac00 con 0x48cd400 2026-02-11T23:30:27.326+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6830/2699377081,v1:127.0.0.1:6831/2699377081] -- osd_ping(ping e21 up_from 5 ping_stamp 2026-02-11T23:30:27.327832+0000/10.003236771s send_stamp 10.003236771s) -- 0x4c9ad80 con 0x48cd800 2026-02-11T23:30:27.326+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6830/2699377081 1 ==== osd_ping(ping_reply e19 up_from 19 ping_stamp 2026-02-11T23:30:27.327832+0000/10.003236771s send_stamp 2.783875465s delta_ub -7.219361782s) ==== 2033+0+0 (crc 0 0 0) 0x4c9ad80 con 0x48cd800 2026-02-11T23:30:27.326+0000 7f416c3fd640 20 osd.0 21 get_con_osd_cluster to osd.3 from_epoch 21 2026-02-11T23:30:27.326+0000 7f416c3fd640 20 osd.0 21 get_nextmap_reserved map_reservations: {21=1} 2026-02-11T23:30:27.326+0000 7f416c3fd640 1 --2- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] >> [v2:127.0.0.1:6828/2699377081,v1:127.0.0.1:6829/2699377081] conn(0x48ccc00 0x4b5c680 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-11T23:30:27.326+0000 7f416c3fd640 20 osd.0 21 release_map epoch: 21 2026-02-11T23:30:27.326+0000 7f416c3fd640 20 osd.0 21 handle_osd_ping new stamps hbstamp(osd.3 up_from 0 peer_clock_delta [-7.219554424s,-7.219361782s]) 2026-02-11T23:30:27.326+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6832/2699377081 1 ==== osd_ping(ping_reply e19 up_from 19 ping_stamp 2026-02-11T23:30:27.327832+0000/10.003236771s send_stamp 2.783927441s delta_ub -7.219309330s) ==== 2033+0+0 (crc 0 0 0) 0x4c9ac00 con 0x48cd400 2026-02-11T23:30:27.326+0000 7f416abfa640 20 osd.0 21 get_con_osd_cluster to osd.3 from_epoch 21 2026-02-11T23:30:27.326+0000 7f416abfa640 20 osd.0 21 get_nextmap_reserved map_reservations: {21=1} 2026-02-11T23:30:27.326+0000 7f416abfa640 20 osd.0 21 release_map epoch: 21 2026-02-11T23:30:27.326+0000 7f416abfa640 20 osd.0 21 handle_osd_ping new stamps hbstamp(osd.3 up_from 0 peer_clock_delta [-7.219548225s,-7.219309330s]) 2026-02-11T23:30:27.326+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6814/3043114013 2 ==== osd_ping(ping_reply e22 up_from 10 ping_stamp 2026-02-11T23:30:27.327832+0000/10.003236771s send_stamp 7.991650581s delta_ub -2.011586428s) ==== 2033+0+0 (crc 0 0 0) 0x4c9ad80 con 0x4640c00 2026-02-11T23:30:27.326+0000 7f416c3fd640 20 osd.0 21 get_con_osd_cluster to osd.1 from_epoch 21 2026-02-11T23:30:27.326+0000 7f416c3fd640 20 osd.0 21 get_nextmap_reserved map_reservations: {21=1} 2026-02-11T23:30:27.326+0000 7f416c3fd640 20 osd.0 21 release_map epoch: 21 2026-02-11T23:30:27.326+0000 7f416c3fd640 10 osd.0 21 maybe_share_map: con v2:127.0.0.1:6812/3043114013 updating session's projected_epoch from 16 to ping map epoch of 22 2026-02-11T23:30:27.326+0000 7f416c3fd640 10 osd.0 21 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 21 is not newer than session's projected_epoch of 22 2026-02-11T23:30:27.326+0000 7f416c3fd640 20 osd.0 21 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011847258s,-2.011586428s]) 2026-02-11T23:30:27.326+0000 7f416c3fd640 1 --2- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] >> conn(0x48cdc00 0x4a2e100 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).accept 2026-02-11T23:30:27.326+0000 7f416c3fd640 1 --2- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] >> conn(0x507a000 0x4a2e680 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).accept 2026-02-11T23:30:27.326+0000 7f416cbfe640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6824/2365263801 1 ==== osd_ping(ping_reply e20 up_from 14 ping_stamp 2026-02-11T23:30:27.327832+0000/10.003236771s send_stamp 5.436564445s delta_ub -4.566672802s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a900 con 0x48c4000 2026-02-11T23:30:27.326+0000 7f416cbfe640 20 osd.0 21 get_con_osd_cluster to osd.2 from_epoch 21 2026-02-11T23:30:27.326+0000 7f416cbfe640 20 osd.0 21 get_nextmap_reserved map_reservations: {21=1} 2026-02-11T23:30:27.326+0000 7f416cbfe640 1 --2- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] >> [v2:127.0.0.1:6820/2365263801,v1:127.0.0.1:6821/2365263801] conn(0x507b800 0x4a2c580 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-11T23:30:27.326+0000 7f416cbfe640 20 osd.0 21 release_map epoch: 21 2026-02-11T23:30:27.326+0000 7f416cbfe640 20 osd.0 21 handle_osd_ping new stamps hbstamp(osd.2 up_from 0 peer_clock_delta [-4.567018032s,-4.566672802s]) 2026-02-11T23:30:27.326+0000 7f416c3fd640 1 --2- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] >> conn(0x507a000 0x4a2e680 unknown :-1 s=BANNER_ACCEPTING pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:27.326+0000 7f415aefe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 2 ==== osd_map(22..22 src has 1..22) ==== 1011+0+0 (unknown 0 0 0) 0x4a65180 con 0x4acb000 2026-02-11T23:30:27.326+0000 7f415aefe640 20 osd.0 21 OSD::ms_dispatch: osd_map(22..22 src has 1..22) 2026-02-11T23:30:27.326+0000 7f415aefe640 20 osd.0 21 _dispatch 0x4a65180 osd_map(22..22 src has 1..22) 2026-02-11T23:30:27.326+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6816/3043114013 2 ==== osd_ping(ping_reply e22 up_from 10 ping_stamp 2026-02-11T23:30:27.327832+0000/10.003236771s send_stamp 7.991786003s delta_ub -2.011451483s) ==== 2033+0+0 (crc 0 0 0) 0x4c9ac00 con 0x48c4400 2026-02-11T23:30:27.326+0000 7f416abfa640 20 osd.0 21 get_con_osd_cluster to osd.1 from_epoch 21 2026-02-11T23:30:27.326+0000 7f416abfa640 20 osd.0 21 get_nextmap_reserved map_reservations: {21=1} 2026-02-11T23:30:27.326+0000 7f416abfa640 20 osd.0 21 release_map epoch: 21 2026-02-11T23:30:27.326+0000 7f416abfa640 10 osd.0 21 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 21 is not newer than session's projected_epoch of 22 2026-02-11T23:30:27.326+0000 7f416abfa640 20 osd.0 21 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011842728s,-2.011451483s]) 2026-02-11T23:30:27.326+0000 7f416c3fd640 10 osd.0 21 ms_handle_fast_authentication new session 0x4a30a80 con 0x507a000 entity osd.2 addr 2026-02-11T23:30:27.326+0000 7f415aefe640 3 osd.0 21 handle_osd_map epochs [22,22], i have 21, src has [1,22] 2026-02-11T23:30:27.326+0000 7f415aefe640 10 osd.0 21 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:30:27.326+0000 7f415aefe640 10 osd.0 21 handle_osd_map got inc map for epoch 22 2026-02-11T23:30:27.326+0000 7f416cbfe640 1 --2- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] >> [v2:127.0.0.1:6828/2699377081,v1:127.0.0.1:6829/2699377081] conn(0x48ccc00 0x4b5c680 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:27.326+0000 7f416cbfe640 10 monclient: get_auth_request con 0x48ccc00 auth_method 0 2026-02-11T23:30:27.326+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6822/2365263801 1 ==== osd_ping(ping_reply e20 up_from 14 ping_stamp 2026-02-11T23:30:27.327832+0000/10.003236771s send_stamp 5.436636448s delta_ub -4.566600800s) ==== 2033+0+0 (crc 0 0 0) 0x4c9ac00 con 0x48cd000 2026-02-11T23:30:27.326+0000 7f416abfa640 20 osd.0 21 get_con_osd_cluster to osd.2 from_epoch 21 2026-02-11T23:30:27.326+0000 7f416abfa640 20 osd.0 21 get_nextmap_reserved map_reservations: {21=1} 2026-02-11T23:30:27.326+0000 7f416abfa640 20 osd.0 21 release_map epoch: 21 2026-02-11T23:30:27.326+0000 7f416abfa640 20 osd.0 21 handle_osd_ping new stamps hbstamp(osd.2 up_from 0 peer_clock_delta [-4.567054749s,-4.566600800s]) 2026-02-11T23:30:27.326+0000 7f416cbfe640 1 --2- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] >> [v2:127.0.0.1:6820/2365263801,v1:127.0.0.1:6821/2365263801] conn(0x507b800 0x4a2c580 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:27.327+0000 7f415aefe640 20 osd.0 21 got_full_map 22, nothing requested 2026-02-11T23:30:27.327+0000 7f416cbfe640 10 monclient: get_auth_request con 0x507b800 auth_method 0 2026-02-11T23:30:27.327+0000 7f416abfa640 1 --2- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] >> conn(0x48cdc00 0x4a2e100 unknown :-1 s=BANNER_ACCEPTING pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:27.327+0000 7f416c3fd640 1 --2- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] >> [v2:127.0.0.1:6820/2365263801,v1:127.0.0.1:6821/2365263801] conn(0x507a000 0x4a2e680 crc :-1 s=SESSION_ACCEPTING pgs=4 gs=0 cs=0 l=0 c_cookie=4789331044101304509 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_existing_connection connection race detected, this connection loses to existing=0x507b800 2026-02-11T23:30:27.327+0000 7f415aefe640 20 osd.0 21 trim_maps: min=0 oldest_map=1 2026-02-11T23:30:27.327+0000 7f415aefe640 20 osd.0 21 handle_osd_map pg_num_history pg_num_history(e22 pg_nums {} deleted_pools ) 2026-02-11T23:30:27.327+0000 7f415aefe640 10 snap_mapper record_purged_snaps purged_snaps {22={}} 2026-02-11T23:30:27.327+0000 7f415aefe640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:27.327+0000 7f415aefe640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e22 maps [1~22] lci=[5,22] tlb=1) 2026-02-11T23:30:27.327+0000 7f416abfa640 10 osd.0 21 ms_handle_fast_authentication new session 0x4a64fc0 con 0x48cdc00 entity osd.3 addr 2026-02-11T23:30:27.327+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] >> [v2:127.0.0.1:6820/2365263801,v1:127.0.0.1:6821/2365263801] conn(0x507a000 msgr2=0x4a2e680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 76 2026-02-11T23:30:27.327+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] >> [v2:127.0.0.1:6820/2365263801,v1:127.0.0.1:6821/2365263801] conn(0x507a000 msgr2=0x4a2e680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-11T23:30:27.327+0000 7f416c3fd640 1 --2- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] >> [v2:127.0.0.1:6820/2365263801,v1:127.0.0.1:6821/2365263801] conn(0x507a000 0x4a2e680 crc :-1 s=SESSION_ACCEPTING pgs=4 gs=0 cs=0 l=0 c_cookie=4789331044101304509 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-11T23:30:27.327+0000 7f416c3fd640 1 --2- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] >> [v2:127.0.0.1:6820/2365263801,v1:127.0.0.1:6821/2365263801] conn(0x507a000 0x4a2e680 crc :-1 s=SESSION_ACCEPTING pgs=4 gs=0 cs=0 l=0 c_cookie=4789331044101304509 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-11T23:30:27.327+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] >> conn(0x48cdc00 msgr2=0x4a2e100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 75 2026-02-11T23:30:27.327+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] >> conn(0x48cdc00 msgr2=0x4a2e100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-11T23:30:27.327+0000 7f416abfa640 1 --2- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] >> conn(0x48cdc00 0x4a2e100 crc :-1 s=AUTH_ACCEPTING_SIGN pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-11T23:30:27.327+0000 7f415aefe640 2 osd.0 21 ms_handle_reset con 0x507a000 session 0x4a30a80 2026-02-11T23:30:27.327+0000 7f416abfa640 1 --2- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] >> conn(0x48cdc00 0x4a2e100 crc :-1 s=AUTH_ACCEPTING_SIGN pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-11T23:30:27.327+0000 7f415aefe640 2 osd.0 21 ms_handle_reset con 0x48cdc00 session 0x4a64fc0 2026-02-11T23:30:27.327+0000 7f415aefe640 10 osd.0 21 ms_handle_connect con 0x507b800 2026-02-11T23:30:27.327+0000 7f416cbfe640 10 osd.0 21 new session (outgoing) 0x4a30380 con=0x507b800 addr=v2:127.0.0.1:6820/2365263801 2026-02-11T23:30:27.327+0000 7f416cbfe640 1 --2- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] >> [v2:127.0.0.1:6820/2365263801,v1:127.0.0.1:6821/2365263801] conn(0x507b800 0x4a2c580 crc :-1 s=READY pgs=6 gs=4 cs=0 l=0 c_cookie=16404714220505508977 s_cookie=2446182072557096178 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=e3a940aaeb07ac71 server_cookie=21f295a272b1ccf2 in_seq=0 out_seq=0 2026-02-11T23:30:27.327+0000 7f416cbfe640 10 osd.0 21 new session (outgoing) 0x4a30540 con=0x48ccc00 addr=v2:127.0.0.1:6828/2699377081 2026-02-11T23:30:27.327+0000 7f416cbfe640 1 --2- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] >> [v2:127.0.0.1:6828/2699377081,v1:127.0.0.1:6829/2699377081] conn(0x48ccc00 0x4b5c680 crc :-1 s=READY pgs=2 gs=3 cs=0 l=0 c_cookie=16772831927719883901 s_cookie=5065999579134511511 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=e8c511c5a112f47d server_cookie=464e0bc6c2974997 in_seq=0 out_seq=0 2026-02-11T23:30:27.327+0000 7f415aefe640 10 osd.0 21 ms_handle_connect con 0x48ccc00 2026-02-11T23:30:27.336+0000 7f41591a6640 10 osd.0 21 _committed_osd_maps 22..22 2026-02-11T23:30:27.336+0000 7f41591a6640 10 osd.0 21 advance to epoch 22 (<= last 22 <= newest_map 22) 2026-02-11T23:30:27.336+0000 7f41591a6640 20 osd.0 22 consume_map version 22 2026-02-11T23:30:27.336+0000 7f41591a6640 20 osd.0 21 await_reserved_maps epoch:22 2026-02-11T23:30:27.336+0000 7f41591a6640 20 osd.0 21 await_reserved_maps done 0 2026-02-11T23:30:27.336+0000 7f41591a6640 20 osd.0 22 consume_map 22 -- publish done 2026-02-11T23:30:27.336+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:27.336+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:27.336+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:27.336+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:27.336+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:27.336+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:27.336+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:27.336+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:27.336+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:27.336+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:27.336+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:27.336+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:27.336+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:27.336+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:27.336+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:27.336+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:27.336+0000 7f41591a6640 20 osd.0 22 prune_pg_created 2026-02-11T23:30:27.336+0000 7f41591a6640 10 osd.0:0.consume_map 22 (was 21) 2026-02-11T23:30:27.336+0000 7f41591a6640 10 osd.0:1.consume_map 22 (was 21) 2026-02-11T23:30:27.336+0000 7f41591a6640 10 osd.0:2.consume_map 22 (was 21) 2026-02-11T23:30:27.336+0000 7f41591a6640 10 osd.0:3.consume_map 22 (was 21) 2026-02-11T23:30:27.336+0000 7f41591a6640 10 osd.0:4.consume_map 22 (was 21) 2026-02-11T23:30:27.336+0000 7f41591a6640 10 osd.0:5.consume_map 22 (was 21) 2026-02-11T23:30:27.336+0000 7f41591a6640 10 osd.0:6.consume_map 22 (was 21) 2026-02-11T23:30:27.336+0000 7f41591a6640 10 osd.0:7.consume_map 22 (was 21) 2026-02-11T23:30:27.336+0000 7f41591a6640 10 osd.0 22 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:27.336+0000 7f41591a6640 7 osd.0 22 activate_map version 22 2026-02-11T23:30:27.336+0000 7f4150194640 20 osd.0 22 agent_entry empty queue 2026-02-11T23:30:27.537+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.1 127.0.0.1:0/3043114013 2 ==== osd_ping(ping e22 up_from 10 ping_stamp 2026-02-11T23:30:27.539018+0000/8.202763557s send_stamp 8.202763557s delta_ub -2.011451483s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a900 con 0x4aca000 2026-02-11T23:30:27.537+0000 7f416cbfe640 20 osd.0 22 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011736870s,-2.011451483s]) 2026-02-11T23:30:27.537+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.1 127.0.0.1:0/3043114013 2 ==== osd_ping(ping e22 up_from 10 ping_stamp 2026-02-11T23:30:27.539018+0000/8.202763557s send_stamp 8.202763557s delta_ub -2.011451483s) ==== 2033+0+0 (crc 0 0 0) 0x4c9ad80 con 0x4aca400 2026-02-11T23:30:27.537+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e22 up_from 5 ping_stamp 2026-02-11T23:30:27.539018+0000/8.202763557s send_stamp 10.214500427s delta_ub 2.011736870s) -- 0x4a89980 con 0x4aca000 2026-02-11T23:30:27.537+0000 7f416cbfe640 20 osd.0 22 get_con_osd_cluster to osd.1 from_epoch 22 2026-02-11T23:30:27.537+0000 7f416cbfe640 20 osd.0 22 get_nextmap_reserved map_reservations: {22=1} 2026-02-11T23:30:27.537+0000 7f416cbfe640 20 osd.0 22 release_map epoch: 22 2026-02-11T23:30:27.537+0000 7f416cbfe640 10 osd.0 22 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 22 is not newer than session's projected_epoch of 22 2026-02-11T23:30:27.537+0000 7f416c3fd640 20 osd.0 22 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011754751s,-2.011451483s]) 2026-02-11T23:30:27.537+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e22 up_from 5 ping_stamp 2026-02-11T23:30:27.539018+0000/8.202763557s send_stamp 10.214517593s delta_ub 2.011754751s) -- 0x4c9a180 con 0x4aca400 2026-02-11T23:30:27.537+0000 7f416c3fd640 20 osd.0 22 get_con_osd_cluster to osd.1 from_epoch 22 2026-02-11T23:30:27.537+0000 7f416c3fd640 20 osd.0 22 get_nextmap_reserved map_reservations: {22=1} 2026-02-11T23:30:27.537+0000 7f416c3fd640 20 osd.0 22 release_map epoch: 22 2026-02-11T23:30:27.537+0000 7f416c3fd640 10 osd.0 22 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 22 is not newer than session's projected_epoch of 22 2026-02-11T23:30:27.773+0000 7f416c3fd640 1 --2- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] >> conn(0x507a400 0x4a2ec00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).accept 2026-02-11T23:30:27.773+0000 7f416c3fd640 1 --2- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] >> conn(0x507a800 0x4a2f180 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).accept 2026-02-11T23:30:27.773+0000 7f416abfa640 1 --2- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] >> conn(0x507a400 0x4a2ec00 unknown :-1 s=BANNER_ACCEPTING pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:27.773+0000 7f416c3fd640 1 --2- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] >> conn(0x507a800 0x4a2f180 unknown :-1 s=BANNER_ACCEPTING pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:27.782+0000 7f416abfa640 10 osd.0 22 ms_handle_fast_authentication new session 0x4a30a80 con 0x507a400 entity osd.4 addr 2026-02-11T23:30:27.782+0000 7f416c3fd640 10 osd.0 22 ms_handle_fast_authentication new session 0x4a30c40 con 0x507a800 entity osd.4 addr 2026-02-11T23:30:27.782+0000 7f416abfa640 1 --2- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] >> 127.0.0.1:0/3331151400 conn(0x507a400 0x4a2ec00 crc :-1 s=READY pgs=1 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-11T23:30:27.782+0000 7f416c3fd640 1 --2- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] >> 127.0.0.1:0/3331151400 conn(0x507a800 0x4a2f180 crc :-1 s=READY pgs=1 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-11T23:30:27.792+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.2 127.0.0.1:0/2365263801 1 ==== osd_ping(ping e23 up_from 14 ping_stamp 2026-02-11T23:30:27.793961+0000/5.902557373s send_stamp 5.902557373s delta_ub -4.566600800s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a180 con 0x48cc000 2026-02-11T23:30:27.792+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.2 127.0.0.1:0/2365263801 1 ==== osd_ping(ping e23 up_from 14 ping_stamp 2026-02-11T23:30:27.793961+0000/5.902557373s send_stamp 5.902557373s delta_ub -4.566600800s) ==== 2033+0+0 (crc 0 0 0) 0x4a89980 con 0x4641000 2026-02-11T23:30:27.792+0000 7f416c3fd640 20 osd.0 22 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566872120s,-4.566600800s]) 2026-02-11T23:30:27.792+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e22 up_from 5 ping_stamp 2026-02-11T23:30:27.793961+0000/5.902557373s send_stamp 10.469429016s delta_ub 4.566872120s) -- 0x4c9ad80 con 0x48cc000 2026-02-11T23:30:27.792+0000 7f416c3fd640 20 osd.0 22 get_con_osd_cluster to osd.2 from_epoch 22 2026-02-11T23:30:27.792+0000 7f416c3fd640 20 osd.0 22 get_nextmap_reserved map_reservations: {22=1} 2026-02-11T23:30:27.792+0000 7f416c3fd640 20 osd.0 22 release_map epoch: 22 2026-02-11T23:30:27.792+0000 7f416c3fd640 10 osd.0 22 maybe_share_map: con v2:127.0.0.1:6820/2365263801 updating session's projected_epoch from 0 to ping map epoch of 23 2026-02-11T23:30:27.792+0000 7f416c3fd640 10 osd.0 22 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 22 is not newer than session's projected_epoch of 23 2026-02-11T23:30:27.792+0000 7f416cbfe640 20 osd.0 22 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566889286s,-4.566600800s]) 2026-02-11T23:30:27.792+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e22 up_from 5 ping_stamp 2026-02-11T23:30:27.793961+0000/5.902557373s send_stamp 10.469446182s delta_ub 4.566889286s) -- 0x4c9a900 con 0x4641000 2026-02-11T23:30:27.792+0000 7f416cbfe640 20 osd.0 22 get_con_osd_cluster to osd.2 from_epoch 22 2026-02-11T23:30:27.792+0000 7f416cbfe640 20 osd.0 22 get_nextmap_reserved map_reservations: {22=1} 2026-02-11T23:30:27.792+0000 7f416cbfe640 20 osd.0 22 release_map epoch: 22 2026-02-11T23:30:27.792+0000 7f416cbfe640 10 osd.0 22 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 22 is not newer than session's projected_epoch of 23 2026-02-11T23:30:27.793+0000 7f415aefe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.2 v2:127.0.0.1:6820/2365263801 1 ==== osd_map(23..23 src has 1..23) ==== 662+0+0 (crc 0 0 0) 0x4a30700 con 0x507b800 2026-02-11T23:30:27.793+0000 7f415aefe640 20 osd.0 22 OSD::ms_dispatch: osd_map(23..23 src has 1..23) 2026-02-11T23:30:27.793+0000 7f415aefe640 20 osd.0 22 _dispatch 0x4a30700 osd_map(23..23 src has 1..23) 2026-02-11T23:30:27.793+0000 7f415aefe640 3 osd.0 22 handle_osd_map epochs [23,23], i have 22, src has [1,23] 2026-02-11T23:30:27.793+0000 7f415aefe640 10 osd.0 22 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:30:27.793+0000 7f415aefe640 10 osd.0 22 handle_osd_map got inc map for epoch 23 2026-02-11T23:30:27.793+0000 7f415aefe640 20 osd.0 22 got_full_map 23, nothing requested 2026-02-11T23:30:27.793+0000 7f415aefe640 20 osd.0 22 trim_maps: min=0 oldest_map=1 2026-02-11T23:30:27.793+0000 7f415aefe640 20 osd.0 22 handle_osd_map pg_num_history pg_num_history(e23 pg_nums {} deleted_pools ) 2026-02-11T23:30:27.793+0000 7f415aefe640 10 snap_mapper record_purged_snaps purged_snaps {23={}} 2026-02-11T23:30:27.793+0000 7f415aefe640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:27.793+0000 7f415aefe640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e23 maps [1~23] lci=[5,23] tlb=1) 2026-02-11T23:30:27.793+0000 7f41591a6640 10 osd.0 22 _committed_osd_maps 23..23 2026-02-11T23:30:27.793+0000 7f41591a6640 10 osd.0 22 advance to epoch 23 (<= last 23 <= newest_map 23) 2026-02-11T23:30:27.793+0000 7f41591a6640 20 osd.0 23 consume_map version 23 2026-02-11T23:30:27.793+0000 7f41591a6640 20 osd.0 22 await_reserved_maps epoch:23 2026-02-11T23:30:27.793+0000 7f41591a6640 20 osd.0 22 await_reserved_maps done 0 2026-02-11T23:30:27.793+0000 7f41591a6640 20 osd.0 23 consume_map 23 -- publish done 2026-02-11T23:30:27.793+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:27.793+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:27.793+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:27.793+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:27.793+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:27.793+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:27.793+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:27.793+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:27.793+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:27.793+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:27.793+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:27.793+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:27.793+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:27.793+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:27.793+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:27.793+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:27.793+0000 7f41591a6640 20 osd.0 23 prune_pg_created 2026-02-11T23:30:27.793+0000 7f41591a6640 10 osd.0:0.consume_map 23 (was 22) 2026-02-11T23:30:27.793+0000 7f41591a6640 10 osd.0:1.consume_map 23 (was 22) 2026-02-11T23:30:27.793+0000 7f41591a6640 10 osd.0:2.consume_map 23 (was 22) 2026-02-11T23:30:27.793+0000 7f41591a6640 10 osd.0:3.consume_map 23 (was 22) 2026-02-11T23:30:27.793+0000 7f41591a6640 10 osd.0:4.consume_map 23 (was 22) 2026-02-11T23:30:27.793+0000 7f41591a6640 10 osd.0:5.consume_map 23 (was 22) 2026-02-11T23:30:27.793+0000 7f41591a6640 10 osd.0:6.consume_map 23 (was 22) 2026-02-11T23:30:27.793+0000 7f41591a6640 10 osd.0:7.consume_map 23 (was 22) 2026-02-11T23:30:27.793+0000 7f41591a6640 10 osd.0 23 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:27.793+0000 7f41591a6640 10 osd.0 23 maybe_update_heartbeat_peers updating 2026-02-11T23:30:27.793+0000 7f41591a6640 10 osd.0 23 adding neighbor peer osd.1 2026-02-11T23:30:27.793+0000 7f41591a6640 10 osd.0 23 adding neighbor peer osd.4 2026-02-11T23:30:27.793+0000 7f41591a6640 20 osd.0 23 get_con_osd_hb to osd.4 from_epoch 23 2026-02-11T23:30:27.793+0000 7f41591a6640 20 osd.0 23 get_nextmap_reserved map_reservations: {23=1} 2026-02-11T23:30:27.793+0000 7f41591a6640 1 --2- 127.0.0.1:0/3294232400 >> [v2:127.0.0.1:6840/3331151400,v1:127.0.0.1:6841/3331151400] conn(0x507bc00 0x4a2d600 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-11T23:30:27.793+0000 7f41591a6640 1 --2- 127.0.0.1:0/3294232400 >> [v2:127.0.0.1:6838/3331151400,v1:127.0.0.1:6839/3331151400] conn(0x5282000 0x4a2db80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-11T23:30:27.793+0000 7f41591a6640 20 osd.0 23 release_map epoch: 23 2026-02-11T23:30:27.793+0000 7f41591a6640 10 osd.0 23 _add_heartbeat_peer: new peer osd.4 v2:127.0.0.1:6840/3331151400 v2:127.0.0.1:6838/3331151400 2026-02-11T23:30:27.793+0000 7f41591a6640 10 osd.0 23 maybe_update_heartbeat_peers 4 peers, extras 1,2,3,4 2026-02-11T23:30:27.793+0000 7f41591a6640 7 osd.0 23 activate_map version 23 2026-02-11T23:30:27.793+0000 7f4150194640 20 osd.0 23 agent_entry empty queue 2026-02-11T23:30:27.793+0000 7f416abfa640 1 --2- 127.0.0.1:0/3294232400 >> [v2:127.0.0.1:6840/3331151400,v1:127.0.0.1:6841/3331151400] conn(0x507bc00 0x4a2d600 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:27.793+0000 7f416abfa640 10 monclient: get_auth_request con 0x507bc00 auth_method 0 2026-02-11T23:30:27.793+0000 7f416c3fd640 1 --2- 127.0.0.1:0/3294232400 >> [v2:127.0.0.1:6838/3331151400,v1:127.0.0.1:6839/3331151400] conn(0x5282000 0x4a2db80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:27.793+0000 7f416c3fd640 10 monclient: get_auth_request con 0x5282000 auth_method 0 2026-02-11T23:30:27.793+0000 7f416abfa640 1 --2- 127.0.0.1:0/3294232400 >> [v2:127.0.0.1:6840/3331151400,v1:127.0.0.1:6841/3331151400] conn(0x507bc00 0x4a2d600 crc :-1 s=READY pgs=2 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-11T23:30:27.793+0000 7f416c3fd640 1 --2- 127.0.0.1:0/3294232400 >> [v2:127.0.0.1:6838/3331151400,v1:127.0.0.1:6839/3331151400] conn(0x5282000 0x4a2db80 crc :-1 s=READY pgs=2 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-11T23:30:28.038+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.1 127.0.0.1:0/3043114013 3 ==== osd_ping(ping e23 up_from 10 ping_stamp 2026-02-11T23:30:28.039211+0000/8.702956200s send_stamp 8.702956200s delta_ub -2.011620760s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a900 con 0x4aca000 2026-02-11T23:30:28.038+0000 7f416cbfe640 20 osd.0 23 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011730909s,-2.011620760s]) 2026-02-11T23:30:28.038+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e23 up_from 5 ping_stamp 2026-02-11T23:30:28.039211+0000/8.702956200s send_stamp 10.714687347s delta_ub 2.011730909s) -- 0x4a89980 con 0x4aca000 2026-02-11T23:30:28.038+0000 7f416cbfe640 20 osd.0 23 get_con_osd_cluster to osd.1 from_epoch 23 2026-02-11T23:30:28.038+0000 7f416cbfe640 20 osd.0 23 get_nextmap_reserved map_reservations: {23=1} 2026-02-11T23:30:28.038+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.1 127.0.0.1:0/3043114013 3 ==== osd_ping(ping e23 up_from 10 ping_stamp 2026-02-11T23:30:28.039211+0000/8.702956200s send_stamp 8.702956200s delta_ub -2.011620760s) ==== 2033+0+0 (crc 0 0 0) 0x4c9ad80 con 0x4aca400 2026-02-11T23:30:28.038+0000 7f416cbfe640 20 osd.0 23 release_map epoch: 23 2026-02-11T23:30:28.038+0000 7f416cbfe640 10 osd.0 23 maybe_share_map: con v2:127.0.0.1:6812/3043114013 updating session's projected_epoch from 22 to ping map epoch of 23 2026-02-11T23:30:28.038+0000 7f416cbfe640 10 osd.0 23 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 23 is not newer than session's projected_epoch of 23 2026-02-11T23:30:28.038+0000 7f416c3fd640 20 osd.0 23 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011774063s,-2.011620760s]) 2026-02-11T23:30:28.038+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e23 up_from 5 ping_stamp 2026-02-11T23:30:28.039211+0000/8.702956200s send_stamp 10.714730263s delta_ub 2.011774063s) -- 0x4c9a180 con 0x4aca400 2026-02-11T23:30:28.038+0000 7f416c3fd640 20 osd.0 23 get_con_osd_cluster to osd.1 from_epoch 23 2026-02-11T23:30:28.038+0000 7f416c3fd640 20 osd.0 23 get_nextmap_reserved map_reservations: {23=1} 2026-02-11T23:30:28.038+0000 7f416c3fd640 20 osd.0 23 release_map epoch: 23 2026-02-11T23:30:28.038+0000 7f416c3fd640 10 osd.0 23 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 23 is not newer than session's projected_epoch of 23 2026-02-11T23:30:28.045+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.3 127.0.0.1:0/2699377081 1 ==== osd_ping(ping e23 up_from 19 ping_stamp 2026-02-11T23:30:28.046747+0000/3.502686739s send_stamp 3.502686739s delta_ub -7.219309330s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a180 con 0x48cc400 2026-02-11T23:30:28.045+0000 7f416c3fd640 20 osd.0 23 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219509602s,-7.219309330s]) 2026-02-11T23:30:28.045+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e23 up_from 5 ping_stamp 2026-02-11T23:30:28.046747+0000/3.502686739s send_stamp 10.722196579s delta_ub 7.219509602s) -- 0x4c9ad80 con 0x48cc400 2026-02-11T23:30:28.045+0000 7f416c3fd640 20 osd.0 23 get_con_osd_cluster to osd.3 from_epoch 23 2026-02-11T23:30:28.045+0000 7f416c3fd640 20 osd.0 23 get_nextmap_reserved map_reservations: {23=1} 2026-02-11T23:30:28.045+0000 7f416c3fd640 20 osd.0 23 release_map epoch: 23 2026-02-11T23:30:28.045+0000 7f416c3fd640 10 osd.0 23 maybe_share_map: con v2:127.0.0.1:6828/2699377081 updating session's projected_epoch from 0 to ping map epoch of 23 2026-02-11T23:30:28.045+0000 7f416c3fd640 10 osd.0 23 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 23 is not newer than session's projected_epoch of 23 2026-02-11T23:30:28.045+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.3 127.0.0.1:0/2699377081 1 ==== osd_ping(ping e23 up_from 19 ping_stamp 2026-02-11T23:30:28.046747+0000/3.502686739s send_stamp 3.502686739s delta_ub -7.219309330s) ==== 2033+0+0 (crc 0 0 0) 0x4a89980 con 0x48cc800 2026-02-11T23:30:28.045+0000 7f416cbfe640 20 osd.0 23 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219521523s,-7.219309330s]) 2026-02-11T23:30:28.045+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e23 up_from 5 ping_stamp 2026-02-11T23:30:28.046747+0000/3.502686739s send_stamp 10.722208977s delta_ub 7.219521523s) -- 0x4c9a900 con 0x48cc800 2026-02-11T23:30:28.045+0000 7f416cbfe640 20 osd.0 23 get_con_osd_cluster to osd.3 from_epoch 23 2026-02-11T23:30:28.045+0000 7f416cbfe640 20 osd.0 23 get_nextmap_reserved map_reservations: {23=1} 2026-02-11T23:30:28.045+0000 7f416cbfe640 20 osd.0 23 release_map epoch: 23 2026-02-11T23:30:28.045+0000 7f416cbfe640 10 osd.0 23 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 23 is not newer than session's projected_epoch of 23 2026-02-11T23:30:28.172+0000 7f4162bea640 10 osd.0 23 tick_without_osd_lock 2026-02-11T23:30:28.172+0000 7f4162bea640 0 osd.0 23 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:28.172+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:28.172+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:28.172+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:28.172+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.549 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:28.172+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:28.173, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:28.172+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:28.172+0000 7f4162bea640 10 osd.0 23 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:28.172+0000 7f4162bea640 20 osd.0 23 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:28.172+0000 7f4162bea640 10 osd.0 23 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:28.172+0000 7f4162bea640 20 osd.0 23 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:28.202+0000 7f415b6ff640 20 osd.0 23 reports for 0 queries 2026-02-11T23:30:28.202+0000 7f415b6ff640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] -- mgrreport(unknown.0 +0-0 packed 3294 daemon_metrics=2) -- 0x4f9ee00 con 0x4aca800 2026-02-11T23:30:28.202+0000 7f415b6ff640 15 osd.0 23 collect_pg_stats 2026-02-11T23:30:28.202+0000 7f415b6ff640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] -- pg_stats(0 pgs seq 21474836483 v 0) -- 0x507cf00 con 0x4aca800 2026-02-11T23:30:28.325+0000 7f41633eb640 10 osd.0 23 tick 2026-02-11T23:30:28.325+0000 7f41633eb640 20 osd.0 23 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:30:28.325+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:30:28.325+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:30:28.325+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:29:58.327171+0000) 2026-02-11T23:30:28.426+0000 7f4147983640 5 osd.0 23 heartbeat osd_stat(store_statfs(0x384000/0x0/0x384000, data 0xdf08/0x30000, compress 0x0/0x0/0x0, omap 0x2c7e, meta 0x1a4d382), peers [1,2,3,4] op hist []) 2026-02-11T23:30:28.426+0000 7f4147983640 20 osd.0 23 check_full_status cur ratio 0, physical ratio 0, new state none 2026-02-11T23:30:28.426+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6816/3043114013,v1:127.0.0.1:6817/3043114013] -- osd_ping(ping e23 up_from 5 ping_stamp 2026-02-11T23:30:28.428001+0000/11.103406906s send_stamp 11.103406906s delta_ub 2.011774063s) -- 0x4c9af00 con 0x48c4400 2026-02-11T23:30:28.426+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6814/3043114013,v1:127.0.0.1:6815/3043114013] -- osd_ping(ping e23 up_from 5 ping_stamp 2026-02-11T23:30:28.428001+0000/11.103406906s send_stamp 11.103406906s delta_ub 2.011774063s) -- 0x4c9b080 con 0x4640c00 2026-02-11T23:30:28.426+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6824/2365263801,v1:127.0.0.1:6825/2365263801] -- osd_ping(ping e23 up_from 5 ping_stamp 2026-02-11T23:30:28.428001+0000/11.103406906s send_stamp 11.103406906s delta_ub 4.566889286s) -- 0x4c9a600 con 0x48c4000 2026-02-11T23:30:28.426+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6822/2365263801,v1:127.0.0.1:6823/2365263801] -- osd_ping(ping e23 up_from 5 ping_stamp 2026-02-11T23:30:28.428001+0000/11.103406906s send_stamp 11.103406906s delta_ub 4.566889286s) -- 0x4c9a780 con 0x48cd000 2026-02-11T23:30:28.426+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6832/2699377081,v1:127.0.0.1:6833/2699377081] -- osd_ping(ping e23 up_from 5 ping_stamp 2026-02-11T23:30:28.428001+0000/11.103406906s send_stamp 11.103406906s delta_ub 7.219521523s) -- 0x4c9b200 con 0x48cd400 2026-02-11T23:30:28.426+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6830/2699377081,v1:127.0.0.1:6831/2699377081] -- osd_ping(ping e23 up_from 5 ping_stamp 2026-02-11T23:30:28.428001+0000/11.103406906s send_stamp 11.103406906s delta_ub 7.219521523s) -- 0x4c9b380 con 0x48cd800 2026-02-11T23:30:28.426+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6840/3331151400,v1:127.0.0.1:6841/3331151400] -- osd_ping(ping e23 up_from 5 ping_stamp 2026-02-11T23:30:28.428001+0000/11.103406906s send_stamp 11.103406906s) -- 0x4c9b500 con 0x507bc00 2026-02-11T23:30:28.426+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6838/3331151400,v1:127.0.0.1:6839/3331151400] -- osd_ping(ping e23 up_from 5 ping_stamp 2026-02-11T23:30:28.428001+0000/11.103406906s send_stamp 11.103406906s) -- 0x4c9a300 con 0x5282000 2026-02-11T23:30:28.426+0000 7f416cbfe640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6824/2365263801 2 ==== osd_ping(ping_reply e23 up_from 14 ping_stamp 2026-02-11T23:30:28.428001+0000/11.103406906s send_stamp 6.536710739s delta_ub -4.566696167s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a600 con 0x48c4000 2026-02-11T23:30:28.426+0000 7f416cbfe640 20 osd.0 23 get_con_osd_cluster to osd.2 from_epoch 23 2026-02-11T23:30:28.426+0000 7f416cbfe640 20 osd.0 23 get_nextmap_reserved map_reservations: {23=1} 2026-02-11T23:30:28.426+0000 7f416cbfe640 20 osd.0 23 release_map epoch: 23 2026-02-11T23:30:28.426+0000 7f416cbfe640 10 osd.0 23 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 23 is not newer than session's projected_epoch of 23 2026-02-11T23:30:28.426+0000 7f416cbfe640 20 osd.0 23 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566864967s,-4.566696167s]) 2026-02-11T23:30:28.426+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6832/2699377081 2 ==== osd_ping(ping_reply e23 up_from 19 ping_stamp 2026-02-11T23:30:28.428001+0000/11.103406906s send_stamp 3.884070158s delta_ub -7.219336510s) ==== 2033+0+0 (crc 0 0 0) 0x4c9b500 con 0x48cd400 2026-02-11T23:30:28.426+0000 7f416abfa640 20 osd.0 23 get_con_osd_cluster to osd.3 from_epoch 23 2026-02-11T23:30:28.426+0000 7f416abfa640 20 osd.0 23 get_nextmap_reserved map_reservations: {23=1} 2026-02-11T23:30:28.426+0000 7f416abfa640 20 osd.0 23 release_map epoch: 23 2026-02-11T23:30:28.426+0000 7f416abfa640 10 osd.0 23 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 23 is not newer than session's projected_epoch of 23 2026-02-11T23:30:28.426+0000 7f416abfa640 20 osd.0 23 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219521046s,-7.219336510s]) 2026-02-11T23:30:28.426+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6822/2365263801 2 ==== osd_ping(ping_reply e23 up_from 14 ping_stamp 2026-02-11T23:30:28.428001+0000/11.103406906s send_stamp 6.536726952s delta_ub -4.566679955s) ==== 2033+0+0 (crc 0 0 0) 0x4c9b500 con 0x48cd000 2026-02-11T23:30:28.426+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6838/3331151400 1 ==== osd_ping(ping_reply e23 up_from 23 ping_stamp 2026-02-11T23:30:28.428001+0000/11.103406906s send_stamp 1.266897798s delta_ub -9.836508751s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a300 con 0x5282000 2026-02-11T23:30:28.426+0000 7f416abfa640 20 osd.0 23 get_con_osd_cluster to osd.2 from_epoch 23 2026-02-11T23:30:28.426+0000 7f416abfa640 20 osd.0 23 get_nextmap_reserved map_reservations: {23=1} 2026-02-11T23:30:28.426+0000 7f416abfa640 20 osd.0 23 release_map epoch: 23 2026-02-11T23:30:28.426+0000 7f416abfa640 10 osd.0 23 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 23 is not newer than session's projected_epoch of 23 2026-02-11T23:30:28.426+0000 7f416abfa640 20 osd.0 23 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566884995s,-4.566679955s]) 2026-02-11T23:30:28.426+0000 7f416c3fd640 20 osd.0 23 get_con_osd_cluster to osd.4 from_epoch 23 2026-02-11T23:30:28.426+0000 7f416c3fd640 20 osd.0 23 get_nextmap_reserved map_reservations: {23=1} 2026-02-11T23:30:28.426+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6840/3331151400 1 ==== osd_ping(ping_reply e23 up_from 23 ping_stamp 2026-02-11T23:30:28.428001+0000/11.103406906s send_stamp 1.266939998s delta_ub -9.836466789s) ==== 2033+0+0 (crc 0 0 0) 0x4c9b500 con 0x507bc00 2026-02-11T23:30:28.426+0000 7f416c3fd640 1 --2- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] >> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] conn(0x507ac00 0x4a2f700 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-11T23:30:28.426+0000 7f416c3fd640 20 osd.0 23 release_map epoch: 23 2026-02-11T23:30:28.426+0000 7f416c3fd640 20 osd.0 23 handle_osd_ping new stamps hbstamp(osd.4 up_from 0 peer_clock_delta [-9.836724281s,-9.836508751s]) 2026-02-11T23:30:28.426+0000 7f416abfa640 20 osd.0 23 get_con_osd_cluster to osd.4 from_epoch 23 2026-02-11T23:30:28.426+0000 7f416abfa640 20 osd.0 23 get_nextmap_reserved map_reservations: {23=1} 2026-02-11T23:30:28.426+0000 7f416abfa640 20 osd.0 23 release_map epoch: 23 2026-02-11T23:30:28.426+0000 7f416abfa640 20 osd.0 23 handle_osd_ping new stamps hbstamp(osd.4 up_from 0 peer_clock_delta [-9.836702347s,-9.836466789s]) 2026-02-11T23:30:28.426+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6830/2699377081 2 ==== osd_ping(ping_reply e23 up_from 19 ping_stamp 2026-02-11T23:30:28.428001+0000/11.103406906s send_stamp 3.884092331s delta_ub -7.219314575s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a300 con 0x48cd800 2026-02-11T23:30:28.426+0000 7f416c3fd640 20 osd.0 23 get_con_osd_cluster to osd.3 from_epoch 23 2026-02-11T23:30:28.426+0000 7f416c3fd640 20 osd.0 23 get_nextmap_reserved map_reservations: {23=1} 2026-02-11T23:30:28.426+0000 7f416c3fd640 20 osd.0 23 release_map epoch: 23 2026-02-11T23:30:28.426+0000 7f416c3fd640 10 osd.0 23 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 23 is not newer than session's projected_epoch of 23 2026-02-11T23:30:28.426+0000 7f416c3fd640 20 osd.0 23 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219560146s,-7.219314575s]) 2026-02-11T23:30:28.426+0000 7f416c3fd640 1 --2- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] >> conn(0x507b000 0x527eb00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).accept 2026-02-11T23:30:28.426+0000 7f416abfa640 1 --2- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] >> conn(0x507b000 0x527eb00 unknown :-1 s=BANNER_ACCEPTING pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:28.427+0000 7f416cbfe640 1 --2- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] >> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] conn(0x507ac00 0x4a2f700 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=5 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:28.427+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6814/3043114013 3 ==== osd_ping(ping_reply e24 up_from 10 ping_stamp 2026-02-11T23:30:28.428001+0000/11.103406906s send_stamp 9.091855049s delta_ub -2.011551857s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a300 con 0x4640c00 2026-02-11T23:30:28.427+0000 7f416c3fd640 20 osd.0 23 get_con_osd_cluster to osd.1 from_epoch 23 2026-02-11T23:30:28.427+0000 7f416c3fd640 20 osd.0 23 get_nextmap_reserved map_reservations: {23=1} 2026-02-11T23:30:28.427+0000 7f416c3fd640 20 osd.0 23 release_map epoch: 23 2026-02-11T23:30:28.427+0000 7f416c3fd640 10 osd.0 23 maybe_share_map: con v2:127.0.0.1:6812/3043114013 updating session's projected_epoch from 23 to ping map epoch of 24 2026-02-11T23:30:28.427+0000 7f416c3fd640 10 osd.0 23 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 23 is not newer than session's projected_epoch of 24 2026-02-11T23:30:28.427+0000 7f416c3fd640 20 osd.0 23 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011885881s,-2.011551857s]) 2026-02-11T23:30:28.427+0000 7f416cbfe640 10 monclient: get_auth_request con 0x507ac00 auth_method 0 2026-02-11T23:30:28.427+0000 7f415aefe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 3 ==== osd_map(24..24 src has 1..24) ==== 338+0+0 (unknown 0 0 0) 0x4a31180 con 0x4acb000 2026-02-11T23:30:28.427+0000 7f415aefe640 20 osd.0 23 OSD::ms_dispatch: osd_map(24..24 src has 1..24) 2026-02-11T23:30:28.427+0000 7f415aefe640 20 osd.0 23 _dispatch 0x4a31180 osd_map(24..24 src has 1..24) 2026-02-11T23:30:28.427+0000 7f416abfa640 10 osd.0 23 ms_handle_fast_authentication new session 0x4a64000 con 0x507b000 entity osd.4 addr 2026-02-11T23:30:28.427+0000 7f415aefe640 3 osd.0 23 handle_osd_map epochs [24,24], i have 23, src has [1,24] 2026-02-11T23:30:28.427+0000 7f415aefe640 10 osd.0 23 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:30:28.427+0000 7f415aefe640 10 osd.0 23 handle_osd_map got inc map for epoch 24 2026-02-11T23:30:28.427+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6816/3043114013 3 ==== osd_ping(ping_reply e24 up_from 10 ping_stamp 2026-02-11T23:30:28.428001+0000/11.103406906s send_stamp 9.092042923s delta_ub -2.011363983s) ==== 2033+0+0 (crc 0 0 0) 0x4c9b500 con 0x48c4400 2026-02-11T23:30:28.427+0000 7f416abfa640 20 osd.0 23 get_con_osd_cluster to osd.1 from_epoch 23 2026-02-11T23:30:28.427+0000 7f416abfa640 20 osd.0 23 get_nextmap_reserved map_reservations: {23=1} 2026-02-11T23:30:28.427+0000 7f416abfa640 20 osd.0 23 release_map epoch: 23 2026-02-11T23:30:28.427+0000 7f416abfa640 10 osd.0 23 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 23 is not newer than session's projected_epoch of 24 2026-02-11T23:30:28.427+0000 7f416abfa640 20 osd.0 23 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011758327s,-2.011363983s]) 2026-02-11T23:30:28.427+0000 7f415aefe640 20 osd.0 23 got_full_map 24, nothing requested 2026-02-11T23:30:28.427+0000 7f415aefe640 20 osd.0 23 trim_maps: min=0 oldest_map=1 2026-02-11T23:30:28.427+0000 7f415aefe640 20 osd.0 23 handle_osd_map pg_num_history pg_num_history(e24 pg_nums {} deleted_pools ) 2026-02-11T23:30:28.427+0000 7f415aefe640 10 snap_mapper record_purged_snaps purged_snaps {24={}} 2026-02-11T23:30:28.427+0000 7f415aefe640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:28.427+0000 7f415aefe640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e24 maps [1~24] lci=[5,24] tlb=1) 2026-02-11T23:30:28.427+0000 7f416abfa640 1 --2- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] >> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] conn(0x507b000 0x527eb00 crc :-1 s=SESSION_ACCEPTING pgs=6 gs=0 cs=0 l=0 c_cookie=16176904088909127360 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_existing_connection connection race detected, replacing existing=0x507ac00 socket by this connection's socket 2026-02-11T23:30:28.427+0000 7f416abfa640 1 --2- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] >> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] conn(0x507b000 0x527eb00 crc :-1 s=SESSION_ACCEPTING pgs=6 gs=0 cs=0 l=0 c_cookie=16176904088909127360 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-11T23:30:28.427+0000 7f416abfa640 10 osd.0 23 new session (incoming)0x4a31340 con=0x507ac00 addr=v2:127.0.0.1:6836/3331151400 must have raced with connect 2026-02-11T23:30:28.427+0000 7f416abfa640 1 --2- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] >> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] conn(0x507ac00 0x4a2f700 unknown :-1 s=READY pgs=6 gs=6 cs=0 l=0 c_cookie=16176904088909127360 s_cookie=3202828791592709710 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=e07fe88f58fe1ec0 server_cookie=2c72bbd10f46ee4e in_seq=0 out_seq=0 2026-02-11T23:30:28.427+0000 7f415aefe640 2 osd.0 23 ms_handle_reset con 0x507b000 session 0x4a64000 2026-02-11T23:30:28.427+0000 7f41591a6640 10 osd.0 23 _committed_osd_maps 24..24 2026-02-11T23:30:28.427+0000 7f41591a6640 10 osd.0 23 advance to epoch 24 (<= last 24 <= newest_map 24) 2026-02-11T23:30:28.427+0000 7f41591a6640 20 osd.0 24 consume_map version 24 2026-02-11T23:30:28.427+0000 7f41591a6640 20 osd.0 23 await_reserved_maps epoch:24 2026-02-11T23:30:28.427+0000 7f41591a6640 20 osd.0 23 await_reserved_maps done 0 2026-02-11T23:30:28.427+0000 7f41591a6640 20 osd.0 24 consume_map 24 -- publish done 2026-02-11T23:30:28.427+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:28.427+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:28.427+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:28.427+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:28.427+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:28.427+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:28.427+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:28.427+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:28.427+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:28.427+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:28.427+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:28.427+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:28.427+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:28.427+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:28.427+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:28.427+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:28.427+0000 7f41591a6640 20 osd.0 24 prune_pg_created 2026-02-11T23:30:28.427+0000 7f41591a6640 10 osd.0:0.consume_map 24 (was 23) 2026-02-11T23:30:28.427+0000 7f41591a6640 10 osd.0:1.consume_map 24 (was 23) 2026-02-11T23:30:28.427+0000 7f41591a6640 10 osd.0:2.consume_map 24 (was 23) 2026-02-11T23:30:28.427+0000 7f41591a6640 10 osd.0:3.consume_map 24 (was 23) 2026-02-11T23:30:28.427+0000 7f41591a6640 10 osd.0:4.consume_map 24 (was 23) 2026-02-11T23:30:28.427+0000 7f41591a6640 10 osd.0:5.consume_map 24 (was 23) 2026-02-11T23:30:28.427+0000 7f41591a6640 10 osd.0:6.consume_map 24 (was 23) 2026-02-11T23:30:28.427+0000 7f41591a6640 10 osd.0:7.consume_map 24 (was 23) 2026-02-11T23:30:28.427+0000 7f41591a6640 10 osd.0 24 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:28.427+0000 7f41591a6640 7 osd.0 24 activate_map version 24 2026-02-11T23:30:28.427+0000 7f4150194640 20 osd.0 24 agent_entry empty queue 2026-02-11T23:30:29.138+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.1 127.0.0.1:0/3043114013 4 ==== osd_ping(ping e25 up_from 10 ping_stamp 2026-02-11T23:30:29.139491+0000/9.803236008s send_stamp 9.803236008s delta_ub -2.011363983s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a300 con 0x4aca400 2026-02-11T23:30:29.138+0000 7f416c3fd640 20 osd.0 24 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011781931s,-2.011363983s]) 2026-02-11T23:30:29.138+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e24 up_from 5 ping_stamp 2026-02-11T23:30:29.139491+0000/9.803236008s send_stamp 11.815017700s delta_ub 2.011781931s) -- 0x4c9b380 con 0x4aca400 2026-02-11T23:30:29.138+0000 7f416c3fd640 20 osd.0 24 get_con_osd_cluster to osd.1 from_epoch 24 2026-02-11T23:30:29.138+0000 7f416c3fd640 20 osd.0 24 get_nextmap_reserved map_reservations: {24=1} 2026-02-11T23:30:29.138+0000 7f416c3fd640 20 osd.0 24 release_map epoch: 24 2026-02-11T23:30:29.138+0000 7f416c3fd640 10 osd.0 24 maybe_share_map: con v2:127.0.0.1:6812/3043114013 updating session's projected_epoch from 24 to ping map epoch of 25 2026-02-11T23:30:29.138+0000 7f416c3fd640 10 osd.0 24 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 24 is not newer than session's projected_epoch of 25 2026-02-11T23:30:29.138+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.1 127.0.0.1:0/3043114013 4 ==== osd_ping(ping e25 up_from 10 ping_stamp 2026-02-11T23:30:29.139491+0000/9.803236008s send_stamp 9.803236008s delta_ub -2.011363983s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a600 con 0x4aca000 2026-02-11T23:30:29.138+0000 7f416cbfe640 20 osd.0 24 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011821747s,-2.011363983s]) 2026-02-11T23:30:29.138+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e24 up_from 5 ping_stamp 2026-02-11T23:30:29.139491+0000/9.803236008s send_stamp 11.815057755s delta_ub 2.011821747s) -- 0x4c9a900 con 0x4aca000 2026-02-11T23:30:29.138+0000 7f416cbfe640 20 osd.0 24 get_con_osd_cluster to osd.1 from_epoch 24 2026-02-11T23:30:29.138+0000 7f416cbfe640 20 osd.0 24 get_nextmap_reserved map_reservations: {24=1} 2026-02-11T23:30:29.138+0000 7f416cbfe640 20 osd.0 24 release_map epoch: 24 2026-02-11T23:30:29.138+0000 7f416cbfe640 10 osd.0 24 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 24 is not newer than session's projected_epoch of 25 2026-02-11T23:30:29.138+0000 7f415aefe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 4 ==== osd_map(25..25 src has 1..25) ==== 288+0+0 (unknown 0 0 0) 0x4a31500 con 0x4acb000 2026-02-11T23:30:29.138+0000 7f415aefe640 20 osd.0 24 OSD::ms_dispatch: osd_map(25..25 src has 1..25) 2026-02-11T23:30:29.138+0000 7f415aefe640 20 osd.0 24 _dispatch 0x4a31500 osd_map(25..25 src has 1..25) 2026-02-11T23:30:29.138+0000 7f415aefe640 3 osd.0 24 handle_osd_map epochs [25,25], i have 24, src has [1,25] 2026-02-11T23:30:29.138+0000 7f415aefe640 10 osd.0 24 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:30:29.138+0000 7f415aefe640 10 osd.0 24 handle_osd_map got inc map for epoch 25 2026-02-11T23:30:29.138+0000 7f415aefe640 20 osd.0 24 got_full_map 25, nothing requested 2026-02-11T23:30:29.138+0000 7f415aefe640 20 osd.0 24 trim_maps: min=0 oldest_map=1 2026-02-11T23:30:29.138+0000 7f415aefe640 20 osd.0 24 handle_osd_map pg_num_history pg_num_history(e25 pg_nums {} deleted_pools ) 2026-02-11T23:30:29.138+0000 7f415aefe640 10 snap_mapper record_purged_snaps purged_snaps {25={}} 2026-02-11T23:30:29.138+0000 7f415aefe640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:29.138+0000 7f415aefe640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e25 maps [1~25] lci=[5,25] tlb=1) 2026-02-11T23:30:29.139+0000 7f41591a6640 10 osd.0 24 _committed_osd_maps 25..25 2026-02-11T23:30:29.139+0000 7f41591a6640 10 osd.0 24 advance to epoch 25 (<= last 25 <= newest_map 25) 2026-02-11T23:30:29.139+0000 7f41591a6640 20 osd.0 25 consume_map version 25 2026-02-11T23:30:29.139+0000 7f41591a6640 20 osd.0 24 await_reserved_maps epoch:25 2026-02-11T23:30:29.139+0000 7f41591a6640 20 osd.0 24 await_reserved_maps done 0 2026-02-11T23:30:29.139+0000 7f41591a6640 20 osd.0 25 consume_map 25 -- publish done 2026-02-11T23:30:29.139+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:29.139+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:29.139+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:29.139+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:29.139+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:29.139+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:29.139+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:29.139+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:29.139+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:29.139+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:29.139+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:29.139+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:29.139+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:29.139+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:29.139+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:29.139+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:29.139+0000 7f41591a6640 20 osd.0 25 prune_pg_created 2026-02-11T23:30:29.139+0000 7f41591a6640 10 osd.0:0.consume_map 25 (was 24) 2026-02-11T23:30:29.139+0000 7f41591a6640 10 osd.0:1.consume_map 25 (was 24) 2026-02-11T23:30:29.139+0000 7f41591a6640 10 osd.0:2.consume_map 25 (was 24) 2026-02-11T23:30:29.139+0000 7f41591a6640 10 osd.0:3.consume_map 25 (was 24) 2026-02-11T23:30:29.139+0000 7f41591a6640 10 osd.0:4.consume_map 25 (was 24) 2026-02-11T23:30:29.139+0000 7f41591a6640 10 osd.0:5.consume_map 25 (was 24) 2026-02-11T23:30:29.139+0000 7f41591a6640 10 osd.0:6.consume_map 25 (was 24) 2026-02-11T23:30:29.139+0000 7f41591a6640 10 osd.0:7.consume_map 25 (was 24) 2026-02-11T23:30:29.139+0000 7f41591a6640 10 osd.0 25 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:29.139+0000 7f41591a6640 7 osd.0 25 activate_map version 25 2026-02-11T23:30:29.139+0000 7f4150194640 20 osd.0 25 agent_entry empty queue 2026-02-11T23:30:29.139+0000 7f415aefe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 5 ==== osd_map(26..27 src has 1..27) ==== 887+0+0 (unknown 0 0 0) 0x4a316c0 con 0x4acb000 2026-02-11T23:30:29.139+0000 7f415aefe640 20 osd.0 25 OSD::ms_dispatch: osd_map(26..27 src has 1..27) 2026-02-11T23:30:29.139+0000 7f415aefe640 20 osd.0 25 _dispatch 0x4a316c0 osd_map(26..27 src has 1..27) 2026-02-11T23:30:29.139+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 6 ==== pg_query2(1.0 1.0 query(info 0'0 epoch_sent 27) e27/27) ==== 158+0+0 (unknown 0 0 0) 0x4fca960 con 0x4acb000 2026-02-11T23:30:29.139+0000 7f415aefe640 3 osd.0 25 handle_osd_map epochs [26,27], i have 25, src has [1,27] 2026-02-11T23:30:29.139+0000 7f415aefe640 10 osd.0 25 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:29.139+0000 7f415aefe640 10 osd.0 25 handle_osd_map got inc map for epoch 26 2026-02-11T23:30:29.139+0000 7f416abfa640 15 osd.0 25 enqueue_peering_evt 1.0 epoch_sent: 27 epoch_requested: 27 MQuery 1.0 from 1 query_epoch 27 query: query(info 0'0 epoch_sent 27) 2026-02-11T23:30:29.139+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 27 epoch_requested: 27 MQuery 1.0 from 1 query_epoch 27 query: query(info 0'0 epoch_sent 27)) class_id 2 prio 255 cost 10 e27) 2026-02-11T23:30:29.139+0000 7f415aefe640 20 osd.0 25 got_full_map 26, nothing requested 2026-02-11T23:30:29.139+0000 7f415aefe640 10 osd.0 25 handle_osd_map got inc map for epoch 27 2026-02-11T23:30:29.139+0000 7f415aefe640 20 osd.0 25 got_full_map 27, nothing requested 2026-02-11T23:30:29.139+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 (new) to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:29.139+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 27 epoch_requested: 27 MQuery 1.0 from 1 query_epoch 27 query: query(info 0'0 epoch_sent 27)) class_id 2immediate prio 255 cost 10 e27) queued 2026-02-11T23:30:29.139+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:29.139+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 27 epoch_requested: 27 MQuery 1.0 from 1 query_epoch 27 query: query(info 0'0 epoch_sent 27)) class_id 2immediate prio 255 cost 10 e27) pg 0 2026-02-11T23:30:29.139+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 map 27 > 25 2026-02-11T23:30:29.139+0000 7f414f993640 20 osd.0 op_wq _add_slot_waiter 1.0 peering, item epoch is 27, will wait on OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 27 epoch_requested: 27 MQuery 1.0 from 1 query_epoch 27 query: query(info 0'0 epoch_sent 27)) class_id 2immediate prio 255 cost 10 e27) 2026-02-11T23:30:29.139+0000 7f415aefe640 10 osd.0 25 _track_pools_and_pg_num_changes recording new pool 1 pg_num 1 2026-02-11T23:30:29.139+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:29.139+0000 7f415aefe640 20 osd.0 25 trim_maps: min=0 oldest_map=1 2026-02-11T23:30:29.139+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:29.139+0000 7f415aefe640 20 osd.0 25 handle_osd_map pg_num_history pg_num_history(e27 pg_nums {1={27=1}} deleted_pools ) 2026-02-11T23:30:29.139+0000 7f415aefe640 10 snap_mapper record_purged_snaps purged_snaps {26={},27={}} 2026-02-11T23:30:29.139+0000 7f415aefe640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:29.139+0000 7f415aefe640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e27 maps [1~27] lci=[5,27] tlb=1) 2026-02-11T23:30:29.140+0000 7f41591a6640 10 osd.0 25 _committed_osd_maps 26..27 2026-02-11T23:30:29.140+0000 7f41591a6640 10 osd.0 25 advance to epoch 26 (<= last 27 <= newest_map 27) 2026-02-11T23:30:29.140+0000 7f41591a6640 10 osd.0 26 advance to epoch 27 (<= last 27 <= newest_map 27) 2026-02-11T23:30:29.140+0000 7f41591a6640 0 osd.0 27 crush map has features 288514051259236352, adjusting msgr requires for clients 2026-02-11T23:30:29.140+0000 7f41591a6640 0 osd.0 27 crush map has features 288514051259236352 was 288514050185503233, adjusting msgr requires for mons 2026-02-11T23:30:29.140+0000 7f41591a6640 0 osd.0 27 crush map has features 3314933000852226048, adjusting msgr requires for osds 2026-02-11T23:30:29.140+0000 7f41591a6640 20 osd.0 27 consume_map version 27 2026-02-11T23:30:29.140+0000 7f41591a6640 20 osd.0 25 await_reserved_maps epoch:27 2026-02-11T23:30:29.140+0000 7f41591a6640 20 osd.0 25 await_reserved_maps done 0 2026-02-11T23:30:29.140+0000 7f41591a6640 20 osd.0 27 consume_map 27 -- publish done 2026-02-11T23:30:29.140+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 1 slots 2026-02-11T23:30:29.140+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3700dc0 2026-02-11T23:30:29.140+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot 1.0 has no pg and waiting_for_split 2026-02-11T23:30:29.140+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:29.140+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:29.140+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:29.140+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:29.140+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:29.140+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:29.140+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:29.140+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:29.140+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:29.140+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:29.140+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:29.140+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:29.140+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:29.140+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:29.140+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:29.140+0000 7f41591a6640 20 osd.0 27 prune_pg_created 2026-02-11T23:30:29.140+0000 7f41591a6640 10 osd.0:0.consume_map 27 (was 25) 2026-02-11T23:30:29.140+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:29.140+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 pending_peering first epoch 27 <= 27, requeueing 2026-02-11T23:30:29.140+0000 7f41591a6640 20 osd.0:0._wake_pg_slot _wake_pg_slot 1.0 to_process <> waiting <> waiting_peering {27=} 2026-02-11T23:30:29.140+0000 7f41591a6640 10 osd.0:1.consume_map 27 (was 25) 2026-02-11T23:30:29.140+0000 7f41591a6640 10 osd.0:2.consume_map 27 (was 25) 2026-02-11T23:30:29.140+0000 7f41591a6640 10 osd.0:3.consume_map 27 (was 25) 2026-02-11T23:30:29.140+0000 7f41591a6640 10 osd.0:4.consume_map 27 (was 25) 2026-02-11T23:30:29.140+0000 7f41591a6640 10 osd.0:5.consume_map 27 (was 25) 2026-02-11T23:30:29.140+0000 7f41591a6640 10 osd.0:6.consume_map 27 (was 25) 2026-02-11T23:30:29.140+0000 7f41591a6640 10 osd.0:7.consume_map 27 (was 25) 2026-02-11T23:30:29.140+0000 7f41591a6640 10 osd.0 27 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:29.140+0000 7f41591a6640 7 osd.0 27 activate_map version 27 2026-02-11T23:30:29.140+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:29.140+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 27 epoch_requested: 27 MQuery 1.0 from 1 query_epoch 27 query: query(info 0'0 epoch_sent 27)) class_id 2immediate prio 255 cost 10 e27) queued 2026-02-11T23:30:29.140+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:29.140+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 27 epoch_requested: 27 MQuery 1.0 from 1 query_epoch 27 query: query(info 0'0 epoch_sent 27)) class_id 2immediate prio 255 cost 10 e27) pg 0 2026-02-11T23:30:29.140+0000 7f4150194640 20 osd.0 27 agent_entry empty queue 2026-02-11T23:30:29.140+0000 7f414f993640 10 osd.0 27 handle_pg_query_nopg 1.0 2026-02-11T23:30:29.140+0000 7f414f993640 10 osd.0 27 pg 1.0 dne 2026-02-11T23:30:29.140+0000 7f414f993640 20 osd.0 27 get_con_osd_cluster to osd.1 from_epoch 27 2026-02-11T23:30:29.140+0000 7f414f993640 20 osd.0 27 get_nextmap_reserved map_reservations: {27=1} 2026-02-11T23:30:29.140+0000 7f414f993640 20 osd.0 27 release_map epoch: 27 2026-02-11T23:30:29.140+0000 7f414f993640 10 osd.0 27 maybe_share_map: con v2:127.0.0.1:6812/3043114013 map epoch 25 -> 27 (shared) 2026-02-11T23:30:29.140+0000 7f414f993640 10 osd.0 27 send_incremental_map epoch range: (26, 27] to v2:127.0.0.1:6812/3043114013 2026-02-11T23:30:29.140+0000 7f414f993640 10 osd.0 27 add_map_inc_bl 26 244 bytes 2026-02-11T23:30:29.140+0000 7f414f993640 10 osd.0 27 add_map_inc_bl 27 591 bytes 2026-02-11T23:30:29.140+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- osd_map(26..27 src has 1..27) -- 0x4a31c00 con 0x4acb000 2026-02-11T23:30:29.140+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_notify2(1.0 1.0 (query:27 sent:27 1.0( DNE empty local-lis/les=0/0 n=0 ec=0/0 lis/c=0/0 les/c/f=0/0/0 sis=0) ([0,0] all_participants={} intervals=[])) e27/27) -- 0x3739500 con 0x4acb000 2026-02-11T23:30:29.140+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:29.159+0000 7f4162bea640 10 osd.0 27 tick_without_osd_lock 2026-02-11T23:30:29.159+0000 7f4162bea640 0 osd.0 27 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:29.159+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:29.159+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:29.159+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:29.159+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.549 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:29.159+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:29.160, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:29.159+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:29.159+0000 7f4162bea640 10 osd.0 27 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:29.159+0000 7f4162bea640 20 osd.0 27 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:29.159+0000 7f4162bea640 10 osd.0 27 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:29.159+0000 7f4162bea640 20 osd.0 27 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:29.191+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 7 ==== PGlog(1.0 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 25.856254578s int 16.000000000s) e28/27) ==== 1172+0+0 (unknown 0 0 0) 0x4a1c800 con 0x4acb000 2026-02-11T23:30:29.191+0000 7f416abfa640 15 osd.0 27 enqueue_peering_evt 1.0 epoch_sent: 28 epoch_requested: 27 MLogRec from 1 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 25.856254578s int 16.000000000s) +create_info 2026-02-11T23:30:29.191+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 28 epoch_requested: 27 MLogRec from 1 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 25.856254578s int 16.000000000s) +create_info) class_id 2 prio 255 cost 10 e28) 2026-02-11T23:30:29.191+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:29.191+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 28 epoch_requested: 27 MLogRec from 1 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 25.856254578s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e28) queued 2026-02-11T23:30:29.191+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:29.191+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 28 epoch_requested: 27 MLogRec from 1 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 25.856254578s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e28) pg 0 2026-02-11T23:30:29.191+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 map 28 > 27 2026-02-11T23:30:29.191+0000 7f414b98b640 20 osd.0 op_wq _add_slot_waiter 1.0 peering, item epoch is 28, will wait on OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 28 epoch_requested: 27 MLogRec from 1 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 25.856254578s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e28) 2026-02-11T23:30:29.191+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:29.191+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:29.325+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:30:29.325+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:30:29.325+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:29:59.327234+0000) 2026-02-11T23:30:29.326+0000 7f41633eb640 10 osd.0 27 tick 2026-02-11T23:30:29.326+0000 7f41633eb640 20 osd.0 27 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:30:30.170+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 9 ==== pg_query2(1.0 1.0 query(info 0'0 epoch_sent 29) e29/29) ==== 158+0+0 (unknown 0 0 0) 0x4fca960 con 0x4acb000 2026-02-11T23:30:30.170+0000 7f415aefe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 8 ==== osd_map(28..29 src has 1..29) ==== 878+0+0 (unknown 0 0 0) 0x4a31c00 con 0x4acb000 2026-02-11T23:30:30.170+0000 7f415aefe640 20 osd.0 27 OSD::ms_dispatch: osd_map(28..29 src has 1..29) 2026-02-11T23:30:30.170+0000 7f415aefe640 20 osd.0 27 _dispatch 0x4a31c00 osd_map(28..29 src has 1..29) 2026-02-11T23:30:30.170+0000 7f416abfa640 15 osd.0 27 enqueue_peering_evt 1.0 epoch_sent: 29 epoch_requested: 29 MQuery 1.0 from 1 query_epoch 29 query: query(info 0'0 epoch_sent 29) 2026-02-11T23:30:30.170+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 29 epoch_requested: 29 MQuery 1.0 from 1 query_epoch 29 query: query(info 0'0 epoch_sent 29)) class_id 2 prio 255 cost 10 e29) 2026-02-11T23:30:30.170+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {28=} 2026-02-11T23:30:30.170+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 29 epoch_requested: 29 MQuery 1.0 from 1 query_epoch 29 query: query(info 0'0 epoch_sent 29)) class_id 2immediate prio 255 cost 10 e29) queued 2026-02-11T23:30:30.170+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {28=} 2026-02-11T23:30:30.170+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 29 epoch_requested: 29 MQuery 1.0 from 1 query_epoch 29 query: query(info 0'0 epoch_sent 29)) class_id 2immediate prio 255 cost 10 e29) pg 0 2026-02-11T23:30:30.170+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 map 29 > 27 2026-02-11T23:30:30.170+0000 7f414b98b640 20 osd.0 op_wq _add_slot_waiter 1.0 peering, item epoch is 29, will wait on OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 29 epoch_requested: 29 MQuery 1.0 from 1 query_epoch 29 query: query(info 0'0 epoch_sent 29)) class_id 2immediate prio 255 cost 10 e29) 2026-02-11T23:30:30.170+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:30.170+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:30.170+0000 7f415aefe640 3 osd.0 27 handle_osd_map epochs [28,29], i have 27, src has [1,29] 2026-02-11T23:30:30.170+0000 7f415aefe640 10 osd.0 27 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:30.170+0000 7f415aefe640 10 osd.0 27 handle_osd_map got inc map for epoch 28 2026-02-11T23:30:30.170+0000 7f415aefe640 20 osd.0 27 got_full_map 28, nothing requested 2026-02-11T23:30:30.170+0000 7f415aefe640 10 osd.0 27 handle_osd_map got inc map for epoch 29 2026-02-11T23:30:30.170+0000 7f415aefe640 20 osd.0 27 got_full_map 29, nothing requested 2026-02-11T23:30:30.170+0000 7f415aefe640 20 osd.0 27 trim_maps: min=0 oldest_map=1 2026-02-11T23:30:30.170+0000 7f415aefe640 20 osd.0 27 handle_osd_map pg_num_history pg_num_history(e29 pg_nums {1={27=1}} deleted_pools ) 2026-02-11T23:30:30.170+0000 7f415aefe640 10 snap_mapper record_purged_snaps purged_snaps {28={},29={}} 2026-02-11T23:30:30.170+0000 7f415aefe640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:30.170+0000 7f415aefe640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e29 maps [1~29] lci=[5,29] tlb=1) 2026-02-11T23:30:30.170+0000 7f41591a6640 10 osd.0 27 _committed_osd_maps 28..29 2026-02-11T23:30:30.170+0000 7f41591a6640 10 osd.0 27 advance to epoch 28 (<= last 29 <= newest_map 29) 2026-02-11T23:30:30.170+0000 7f41591a6640 10 osd.0 28 advance to epoch 29 (<= last 29 <= newest_map 29) 2026-02-11T23:30:30.170+0000 7f41591a6640 20 osd.0 29 consume_map version 29 2026-02-11T23:30:30.170+0000 7f41591a6640 20 osd.0 27 await_reserved_maps epoch:29 2026-02-11T23:30:30.170+0000 7f41591a6640 20 osd.0 27 await_reserved_maps done 0 2026-02-11T23:30:30.170+0000 7f41591a6640 20 osd.0 29 consume_map 29 -- publish done 2026-02-11T23:30:30.170+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 1 slots 2026-02-11T23:30:30.170+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3700dc0 2026-02-11T23:30:30.170+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot 1.0 has no pg and waiting_for_split 2026-02-11T23:30:30.170+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:30.170+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:30.170+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:30.170+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:30.170+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:30.170+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:30.170+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:30.170+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:30.170+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:30.170+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:30.170+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:30.170+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:30.170+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:30.170+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:30.170+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:30.170+0000 7f41591a6640 20 osd.0 29 prune_pg_created 2026-02-11T23:30:30.170+0000 7f41591a6640 10 osd.0:0.consume_map 29 (was 27) 2026-02-11T23:30:30.170+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:30.170+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 pending_peering first epoch 28 <= 29, requeueing 2026-02-11T23:30:30.170+0000 7f41591a6640 20 osd.0:0._wake_pg_slot _wake_pg_slot 1.0 to_process <> waiting <> waiting_peering {28=,29=} 2026-02-11T23:30:30.170+0000 7f41591a6640 10 osd.0:1.consume_map 29 (was 27) 2026-02-11T23:30:30.170+0000 7f41591a6640 10 osd.0:2.consume_map 29 (was 27) 2026-02-11T23:30:30.170+0000 7f41591a6640 10 osd.0:3.consume_map 29 (was 27) 2026-02-11T23:30:30.170+0000 7f41591a6640 10 osd.0:4.consume_map 29 (was 27) 2026-02-11T23:30:30.170+0000 7f41591a6640 10 osd.0:5.consume_map 29 (was 27) 2026-02-11T23:30:30.170+0000 7f41591a6640 10 osd.0:6.consume_map 29 (was 27) 2026-02-11T23:30:30.170+0000 7f41591a6640 10 osd.0:7.consume_map 29 (was 27) 2026-02-11T23:30:30.170+0000 7f41591a6640 10 osd.0 29 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:30.170+0000 7f41591a6640 7 osd.0 29 activate_map version 29 2026-02-11T23:30:30.170+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:30.170+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 28 epoch_requested: 27 MLogRec from 1 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 25.856254578s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e28) queued 2026-02-11T23:30:30.170+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:30.170+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 28 epoch_requested: 27 MLogRec from 1 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 25.856254578s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e28) pg 0 2026-02-11T23:30:30.170+0000 7f4150194640 20 osd.0 29 agent_entry empty queue 2026-02-11T23:30:30.170+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 no pg, should create on OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 28 epoch_requested: 27 MLogRec from 1 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 25.856254578s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e28) 2026-02-11T23:30:30.170+0000 7f414f993640 10 osd.0 29 _make_pg 1.0 2026-02-11T23:30:30.170+0000 7f414f993640 10 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:1, shard:-1, shard_prefix: , prefixes: {}] SnapMapper 2026-02-11T23:30:30.170+0000 7f414f993640 20 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:1, shard:-1, shard_prefix: , prefixes: {}] update_bits new_bits: 0 2026-02-11T23:30:30.170+0000 7f414f993640 20 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:1, shard:-1, shard_prefix: , prefixes: {"0000000000000001."}] update_bits prefix updated 2026-02-11T23:30:30.170+0000 7f414f993640 10 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:1, shard:-1, shard_prefix: , prefixes: {"0000000000000001."}] reset_prefix_itr::from <0> to ::update_bits 2026-02-11T23:30:30.170+0000 7f414f993640 5 osd.0 pg_epoch: 27 pg[1.0(unlocked)] enter Initial 2026-02-11T23:30:30.170+0000 7f414f993640 20 osd.0 pg_epoch: 27 pg[1.0(unlocked)] enter NotTrimming 2026-02-11T23:30:30.170+0000 7f414f993640 10 osd.0 pg_epoch: 27 pg[1.0(unlocked)] scrubber: FSM: -- state -->> NotActive 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 pg_epoch: 27 pg[1.0( DNE empty local-lis/les=0/0 n=0 ec=0/0 lis/c=0/0 les/c/f=0/0/0 sis=0) [] r=-1 lpr=0 crt=0'0 unknown mbc={}] PeeringState::init init role 1 up [1,0,2] acting [1,0,2] history ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=27 past_intervals ([0,0] all_participants={} intervals=[]) 2026-02-11T23:30:30.171+0000 7f414f993640 20 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=27) [1,0,2] r=1 lpr=0 crt=0'0 unknown mbc={}] PeeringState::on_new_interval 2026-02-11T23:30:30.171+0000 7f414f993640 20 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=27) [1,0,2] r=1 lpr=0 crt=0'0 unknown mbc={}] PeeringState::on_new_interval upacting_features 0x3f0ffffffffdffff from [1,0,2]+[1,0,2] 2026-02-11T23:30:30.171+0000 7f414f993640 20 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=27) [1,0,2] r=1 lpr=0 crt=0'0 unknown mbc={}] PeeringState::on_new_interval checking missing set deletes flag. missing = missing(0 may_include_deletes = 1) 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=27) [1,0,2] r=1 lpr=0 crt=0'0 unknown mbc={}] PeeringState::init_hb_stamps now [0x56849a0] 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=27) [1,0,2] r=1 lpr=0 crt=0'0 unknown mbc={}] PeeringState::on_new_interval prior_readable_until_ub 0.000000000s (mnow 12.847728729s + 0.000000000s) 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=27) [1,0,2] r=1 lpr=0 crt=0'0 unknown mbc={}] cancel_recovery 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=27) [1,0,2] r=1 lpr=0 crt=0'0 unknown mbc={}] clear_recovery_state 2026-02-11T23:30:30.171+0000 7f414f993640 15 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=27) [1,0,2] r=1 lpr=0 crt=0'0 unknown mbc={}] _clear_recovery_state 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=27) [1,0,2] r=1 lpr=0 crt=0'0 unknown mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:30.171+0000 7f414f993640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:30.171+0000 7f414f993640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:30.171+0000 7f414f993640 6 write_log_and_missing with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, writeout_from: 4294967295'18446744073709551615, trimmed: , trimmed_dups: , clear_divergent_priors: 0 2026-02-11T23:30:30.171+0000 7f414f993640 10 _write_log_and_missing clearing up to 0'0 dirty_to_dups=0'0 dirty_from_dups=4294967295'18446744073709551615 write_from_dups=4294967295'18446744073709551615 trimmed_dups.size()=0 2026-02-11T23:30:30.171+0000 7f414f993640 10 _write_log_and_missing going to encode log.dups.size()=0 2026-02-11T23:30:30.171+0000 7f414f993640 10 _write_log_and_missing 1st round encoded log.dups.size()=0 2026-02-11T23:30:30.171+0000 7f414f993640 10 _write_log_and_missing 2st round encoded log.dups.size()=0 2026-02-11T23:30:30.171+0000 7f414f993640 10 end of _write_log_and_missing 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=27) [1,0,2] r=1 lpr=0 crt=0'0 unknown mbc={}] handle_initialize 2026-02-11T23:30:30.171+0000 7f414f993640 5 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=27) [1,0,2] r=1 lpr=0 crt=0'0 unknown mbc={}] exit Initial 0.000086 0 0.000000 2026-02-11T23:30:30.171+0000 7f414f993640 5 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=27) [1,0,2] r=1 lpr=0 crt=0'0 unknown mbc={}] enter Reset 2026-02-11T23:30:30.171+0000 7f414f993640 20 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=27) [1,0,2] r=1 lpr=0 crt=0'0 unknown mbc={}] PeeringState::set_last_peering_reset set_last_peering_reset 27 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=27) [1,0,2] r=1 lpr=27 crt=0'0 unknown mbc={}] PeeringState::set_last_peering_reset Clearing blocked outgoing recovery messages 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=27) [1,0,2] r=1 lpr=27 crt=0'0 unknown mbc={}] PeeringState::set_last_peering_reset Not blocking outgoing recovery messages 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=27) [1,0,2] r=1 lpr=27 crt=0'0 unknown mbc={}] handle_activate_map: epoch range: 27..27 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=27) [1,0,2] r=1 lpr=27 crt=0'0 unknown mbc={}] PeeringState::activate_map 2026-02-11T23:30:30.171+0000 7f414f993640 5 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=27) [1,0,2] r=1 lpr=27 crt=0'0 unknown mbc={}] exit Reset 0.000010 1 0.000012 2026-02-11T23:30:30.171+0000 7f414f993640 5 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=27) [1,0,2] r=1 lpr=27 crt=0'0 unknown mbc={}] enter Started 2026-02-11T23:30:30.171+0000 7f414f993640 5 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=27) [1,0,2] r=1 lpr=27 crt=0'0 unknown mbc={}] enter Start 2026-02-11T23:30:30.171+0000 7f414f993640 1 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=27) [1,0,2] r=1 lpr=27 crt=0'0 unknown mbc={}] state: transitioning to Stray 2026-02-11T23:30:30.171+0000 7f414f993640 5 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=27) [1,0,2] r=1 lpr=27 crt=0'0 unknown mbc={}] exit Start 0.000002 0 0.000000 2026-02-11T23:30:30.171+0000 7f414f993640 5 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=27) [1,0,2] r=1 lpr=27 crt=0'0 unknown mbc={}] enter Started/Stray 2026-02-11T23:30:30.171+0000 7f414f993640 20 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=27) [1,0,2] r=1 lpr=27 crt=0'0 unknown mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 27 while current is 27 2026-02-11T23:30:30.171+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=27) [1,0,2] r=1 lpr=27 crt=0'0 unknown mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 29 handle_pg_create_info new pg pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=27) [1,0,2] r=1 lpr=27 crt=0'0 unknown mbc={}] 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0:0._attach_pg 1.0 0x5946000 2026-02-11T23:30:30.171+0000 7f414f993640 20 osd.0:0._wake_pg_slot _wake_pg_slot 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:30.171+0000 7f414f993640 20 osd.0 29 identify_splits_and_merges 1.0 e27 to e29 2026-02-11T23:30:30.171+0000 7f414f993640 20 osd.0 29 identify_splits_and_merges 1.0 e27 to e29 pg_nums {27=1} 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0:0._prime_splits 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=27) [1,0,2] r=1 lpr=27 crt=0'0 unknown mbc={}] handle_advance_map: 28 2026-02-11T23:30:30.171+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:30.171+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 29 epoch_requested: 29 MQuery 1.0 from 1 query_epoch 29 query: query(info 0'0 epoch_sent 29)) class_id 2immediate prio 255 cost 10 e29) queued 2026-02-11T23:30:30.171+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 27 -> 28 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=27) [1,0,2] r=1 lpr=27 crt=0'0 unknown mbc={}] PeeringState::advance_map handle_advance_map [1,0,2]/[1,0,2] -- 1/1 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 pg_epoch: 28 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=27) [1,0,2] r=1 lpr=27 crt=0'0 unknown mbc={}] state: Started advmap 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 pg_epoch: 28 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=27) [1,0,2] r=1 lpr=27 crt=0'0 unknown mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:30.171+0000 7f414f993640 20 osd.0 29 advance_pg new pool opts old pool opts 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 pg_epoch: 28 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=27) [1,0,2] r=1 lpr=27 crt=0'0 unknown mbc={}] handle_advance_map: 29 2026-02-11T23:30:30.171+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 28 -> 29 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 pg_epoch: 28 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=27) [1,0,2] r=1 lpr=27 crt=0'0 unknown mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=27) [1,0,2] r=1 lpr=27 crt=0'0 unknown mbc={}] state: Started advmap 2026-02-11T23:30:30.171+0000 7f414f993640 20 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=27) [1,0,2] r=1 lpr=27 crt=0'0 unknown mbc={}] PeeringState::should_restart_peering new interval newup [1,0] newacting [1,0] 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=27) [1,0,2] r=1 lpr=27 crt=0'0 unknown mbc={}] state: should_restart_peering, transitioning to Reset 2026-02-11T23:30:30.171+0000 7f414f993640 5 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=27) [1,0,2] r=1 lpr=27 crt=0'0 unknown mbc={}] exit Started/Stray 0.000069 2 0.000011 2026-02-11T23:30:30.171+0000 7f414f993640 5 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=27) [1,0,2] r=1 lpr=27 crt=0'0 unknown mbc={}] exit Started 0.000075 0 0.000000 2026-02-11T23:30:30.171+0000 7f414f993640 5 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=27) [1,0,2] r=1 lpr=27 crt=0'0 unknown mbc={}] enter Reset 2026-02-11T23:30:30.171+0000 7f414f993640 20 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=27) [1,0,2] r=1 lpr=27 crt=0'0 unknown mbc={}] PeeringState::set_last_peering_reset set_last_peering_reset 29 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=27) [1,0,2] r=1 lpr=29 crt=0'0 unknown mbc={}] PeeringState::set_last_peering_reset Clearing blocked outgoing recovery messages 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=27) [1,0,2] r=1 lpr=29 crt=0'0 unknown mbc={}] PeeringState::set_last_peering_reset Beginning to block outgoing recovery messages 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=27) [1,0,2] r=1 lpr=29 crt=0'0 unknown mbc={}] state: Reset advmap 2026-02-11T23:30:30.171+0000 7f414f993640 20 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=27) [1,0,2] r=1 lpr=29 crt=0'0 unknown mbc={}] PeeringState::should_restart_peering new interval newup [1,0] newacting [1,0] 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=27) [1,0,2] r=1 lpr=29 crt=0'0 unknown mbc={}] state: should restart peering, calling start_peering_interval again 2026-02-11T23:30:30.171+0000 7f414f993640 20 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=27) [1,0,2] r=1 lpr=29 crt=0'0 unknown mbc={}] PeeringState::set_last_peering_reset set_last_peering_reset 29 2026-02-11T23:30:30.171+0000 7f414f993640 15 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=27) [1,0] r=1 lpr=29 crt=0'0 unknown mbc={}] clear_stats 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=27) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown mbc={}] PeeringState::start_peering_interval : check_new_interval output: check_new_interval interval(27-28 up [1, 0, 2](1) acting [1, 0, 2](1)) up_thru 27 up_from 10 last_epoch_clean 0 interval(27-28 up [1, 0, 2](1) acting [1, 0, 2](1) maybe_went_rw) : primary up 10-27 includes interval 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=27) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown mbc={}] PeeringState::start_peering_interval noting past ([27,28] all_participants={0, 1, 2} intervals=[([27,28] acting={0, 1, 2})]) 2026-02-11T23:30:30.171+0000 7f414f993640 20 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown mbc={}] PeeringState::on_new_interval 2026-02-11T23:30:30.171+0000 7f414f993640 20 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown mbc={}] PeeringState::on_new_interval upacting_features 0x3f0ffffffffdffff from [1,0]+[1,0] 2026-02-11T23:30:30.171+0000 7f414f993640 20 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown mbc={}] PeeringState::on_new_interval checking missing set deletes flag. missing = missing(0 may_include_deletes = 1) 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown mbc={}] PeeringState::init_hb_stamps now [0x56849a0] 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown mbc={}] PeeringState::on_new_interval prior_readable_until_ub 0.000000000s (mnow 12.847879410s + 0.000000000s) 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown mbc={}] cancel_recovery 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown mbc={}] clear_recovery_state 2026-02-11T23:30:30.171+0000 7f414f993640 15 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown mbc={}] _clear_recovery_state 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:30.171+0000 7f414f993640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:30.171+0000 7f414f993640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:30.171+0000 7f414f993640 1 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown mbc={}] PeeringState::start_peering_interval up [1,0,2] -> [1,0], acting [1,0,2] -> [1,0], acting_primary 1 -> 1, up_primary 1 -> 1, role 1 -> 1, features acting 4544132024016699391 upacting 4544132024016699391 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown mbc={}] PeeringState::clear_primary_state clear_primary_state 2026-02-11T23:30:30.171+0000 7f414f993640 20 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown mbc={}] clear_primary_state 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown mbc={}] release_backoffs [1:00000000::::0,MAX) 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:30.171+0000 7f414f993640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:30.171+0000 7f414f993640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:30.171+0000 7f414f993640 20 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown mbc={}] agent_stop 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown mbc={}] on_change 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown mbc={}] cancel_copy_ops 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown mbc={}] cancel_flush_ops 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown mbc={}] cancel_proxy_ops 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown mbc={}] cancel_manifest_ops 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown mbc={}] cancel_cls_gather_ops 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown mbc={}] on_change_cleanup 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown mbc={}] on_change 2026-02-11T23:30:30.171+0000 7f414f993640 20 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown mbc={}] exit NotTrimming 2026-02-11T23:30:30.171+0000 7f414f993640 20 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown mbc={}] enter NotTrimming 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown NOTIFY mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown NOTIFY mbc={}] plpg_on_pool_change 2026-02-11T23:30:30.171+0000 7f414f993640 20 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown NOTIFY mbc={}] hit_set_clear 2026-02-11T23:30:30.171+0000 7f414f993640 20 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown NOTIFY mbc={}] agent_stop 2026-02-11T23:30:30.171+0000 7f414f993640 20 osd.0 29 advance_pg new pool opts old pool opts 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown NOTIFY mbc={}] handle_activate_map: epoch range: 28..29 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown NOTIFY mbc={}] PeeringState::activate_map 2026-02-11T23:30:30.171+0000 7f414f993640 5 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown NOTIFY mbc={}] exit Reset 0.000067 1 2026-02-11T23:30:30.172508+0000 2026-02-11T23:30:30.171+0000 7f414f993640 5 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown NOTIFY mbc={}] enter Started 2026-02-11T23:30:30.171+0000 7f414f993640 5 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown NOTIFY mbc={}] enter Start 2026-02-11T23:30:30.171+0000 7f414f993640 1 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown NOTIFY mbc={}] state: transitioning to Stray 2026-02-11T23:30:30.171+0000 7f414f993640 5 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown NOTIFY mbc={}] exit Start 0.000001 0 0.000000 2026-02-11T23:30:30.171+0000 7f414f993640 5 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown NOTIFY mbc={}] enter Started/Stray 2026-02-11T23:30:30.171+0000 7f414f993640 20 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown NOTIFY mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 27 while current is 29 2026-02-11T23:30:30.171+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown NOTIFY mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown NOTIFY mbc={}] do_peering_event: epoch_sent: 28 epoch_requested: 27 MLogRec from 1 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 25.856254578s int 16.000000000s) +create_info 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown NOTIFY mbc={}] old_peering_msg reply_epoch 28 query_epoch 27 last_peering_reset 29 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown NOTIFY mbc={}] discard old epoch_sent: 28 epoch_requested: 27 MLogRec from 1 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 25.856254578s int 16.000000000s) +create_info 2026-02-11T23:30:30.171+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:30.171+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:30.171+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 29 epoch_requested: 29 MQuery 1.0 from 1 query_epoch 29 query: query(info 0'0 epoch_sent 29)) class_id 2immediate prio 255 cost 10 e29) pg 0x5946000 2026-02-11T23:30:30.171+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:30.171+0000 7f414b98b640 10 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown NOTIFY mbc={}] do_peering_event: epoch_sent: 29 epoch_requested: 29 MQuery 1.0 from 1 query_epoch 29 query: query(info 0'0 epoch_sent 29) 2026-02-11T23:30:30.171+0000 7f414b98b640 10 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown NOTIFY mbc={}] PeeringState::fulfill_info sending info 2026-02-11T23:30:30.171+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:30.171+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:30.171+0000 7f414f993640 15 osd.0 29 enqueue_peering_evt 1.0 epoch_sent: 29 epoch_requested: 29 IntervalFlush 2026-02-11T23:30:30.171+0000 7f414f993640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 29 epoch_requested: 29 IntervalFlush) class_id 2 prio 255 cost 10 e29) 2026-02-11T23:30:30.171+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:30.171+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:30.171+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 29 epoch_requested: 29 IntervalFlush) class_id 2immediate prio 255 cost 10 e29) queued 2026-02-11T23:30:30.171+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:30.171+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 29 epoch_requested: 29 IntervalFlush) class_id 2immediate prio 255 cost 10 e29) pg 0x5946000 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown NOTIFY mbc={}] do_peering_event: epoch_sent: 29 epoch_requested: 29 IntervalFlush 2026-02-11T23:30:30.171+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown NOTIFY mbc={}] state: Ending blocked outgoing recovery messages 2026-02-11T23:30:30.171+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:30.171+0000 7f414f993640 20 osd.0 29 get_con_osd_cluster to osd.1 from_epoch 29 2026-02-11T23:30:30.171+0000 7f414f993640 20 osd.0 29 get_nextmap_reserved map_reservations: {29=1} 2026-02-11T23:30:30.171+0000 7f414f993640 20 osd.0 29 release_map epoch: 29 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 29 maybe_share_map: con v2:127.0.0.1:6812/3043114013 map epoch 27 -> 29 (shared) 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 29 send_incremental_map epoch range: (28, 29] to v2:127.0.0.1:6812/3043114013 2026-02-11T23:30:30.171+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 29 add_map_inc_bl 28 252 bytes 2026-02-11T23:30:30.171+0000 7f414f993640 10 osd.0 29 add_map_inc_bl 29 574 bytes 2026-02-11T23:30:30.171+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- osd_map(28..29 src has 1..29) -- 0x5954000 con 0x4acb000 2026-02-11T23:30:30.171+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_notify2(1.0 1.0 (query:29 sent:29 1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) ([27,28] all_participants={0, 1, 2} intervals=[([27,28] acting={0, 1, 2})])) e29/29) -- 0x3739c00 con 0x4acb000 2026-02-11T23:30:30.171+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_notify2(1.0 1.0 (query:29 sent:29 1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) ([27,28] all_participants={0, 1, 2} intervals=[([27,28] acting={0, 1, 2})])) e29/29) -- 0x373aa00 con 0x4acb000 2026-02-11T23:30:30.171+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:30.208+0000 7f4162bea640 10 osd.0 29 tick_without_osd_lock 2026-02-11T23:30:30.208+0000 7f4162bea640 0 osd.0 29 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:30.208+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:30.208+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:30.208+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:30.208+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.549 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:30.208+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:30.209, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:30.208+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:30.208+0000 7f4162bea640 10 osd.0 29 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:30.208+0000 7f4162bea640 20 osd.0 29 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:30.208+0000 7f4162bea640 10 osd.0 29 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:30.208+0000 7f4162bea640 20 osd.0 29 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:30.270+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 28 ==== osd_map(29..30 src has 1..30) ==== 1208+0+0 (crc 0 0 0) 0x4a308c0 con 0x4acbc00 2026-02-11T23:30:30.270+0000 7f415cf02640 20 osd.0 29 OSD::ms_dispatch: osd_map(29..30 src has 1..30) 2026-02-11T23:30:30.270+0000 7f415cf02640 20 osd.0 29 _dispatch 0x4a308c0 osd_map(29..30 src has 1..30) 2026-02-11T23:30:30.270+0000 7f415cf02640 3 osd.0 29 handle_osd_map epochs [29,30], i have 29, src has [1,30] 2026-02-11T23:30:30.270+0000 7f415cf02640 10 osd.0 29 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:30.270+0000 7f415cf02640 10 osd.0 29 handle_osd_map got inc map for epoch 30 2026-02-11T23:30:30.270+0000 7f415cf02640 20 osd.0 29 got_full_map 30, nothing requested 2026-02-11T23:30:30.270+0000 7f415cf02640 20 osd.0 29 trim_maps: min=0 oldest_map=1 2026-02-11T23:30:30.270+0000 7f415cf02640 20 osd.0 29 handle_osd_map pg_num_history pg_num_history(e30 pg_nums {1={27=1}} deleted_pools ) 2026-02-11T23:30:30.270+0000 7f415cf02640 10 snap_mapper record_purged_snaps purged_snaps {30={}} 2026-02-11T23:30:30.270+0000 7f415cf02640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:30.270+0000 7f415cf02640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e30 maps [1~30] lci=[5,30] tlb=1) 2026-02-11T23:30:30.270+0000 7f41591a6640 10 osd.0 29 _committed_osd_maps 30..30 2026-02-11T23:30:30.270+0000 7f41591a6640 10 osd.0 29 advance to epoch 30 (<= last 30 <= newest_map 30) 2026-02-11T23:30:30.270+0000 7f41591a6640 20 osd.0 30 consume_map version 30 2026-02-11T23:30:30.270+0000 7f41591a6640 20 osd.0 29 await_reserved_maps epoch:30 2026-02-11T23:30:30.270+0000 7f41591a6640 20 osd.0 29 await_reserved_maps done 0 2026-02-11T23:30:30.270+0000 7f41591a6640 20 osd.0 30 consume_map 30 -- publish done 2026-02-11T23:30:30.270+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 1 slots 2026-02-11T23:30:30.270+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3700dc0 2026-02-11T23:30:30.270+0000 7f41591a6640 20 osd.0 30 identify_splits_and_merges 1.0 e29 to e30 2026-02-11T23:30:30.270+0000 7f41591a6640 20 osd.0 30 identify_splits_and_merges 1.0 e29 to e30 pg_nums {27=1} 2026-02-11T23:30:30.270+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:30.270+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:30.270+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:30.270+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:30.270+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:30.270+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:30.270+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:30.270+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:30.270+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:30.270+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:30.270+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:30.270+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:30.270+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:30.270+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:30.270+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:30.270+0000 7f41591a6640 20 osd.0 30 prune_pg_created 2026-02-11T23:30:30.270+0000 7f41591a6640 10 osd.0:0.consume_map 30 (was 29) 2026-02-11T23:30:30.270+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:30.270+0000 7f41591a6640 10 osd.0:1.consume_map 30 (was 29) 2026-02-11T23:30:30.270+0000 7f41591a6640 10 osd.0:2.consume_map 30 (was 29) 2026-02-11T23:30:30.270+0000 7f41591a6640 10 osd.0:3.consume_map 30 (was 29) 2026-02-11T23:30:30.270+0000 7f41591a6640 10 osd.0:4.consume_map 30 (was 29) 2026-02-11T23:30:30.270+0000 7f41591a6640 10 osd.0:5.consume_map 30 (was 29) 2026-02-11T23:30:30.270+0000 7f41591a6640 10 osd.0:6.consume_map 30 (was 29) 2026-02-11T23:30:30.270+0000 7f41591a6640 10 osd.0:7.consume_map 30 (was 29) 2026-02-11T23:30:30.270+0000 7f41591a6640 10 osd.0 30 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:30.270+0000 7f41591a6640 15 osd.0 30 enqueue_peering_evt 1.0 epoch_sent: 30 epoch_requested: 30 NullEvt 2026-02-11T23:30:30.270+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 30 epoch_requested: 30 NullEvt) class_id 2 prio 255 cost 10 e30) 2026-02-11T23:30:30.270+0000 7f41591a6640 7 osd.0 30 activate_map version 30 2026-02-11T23:30:30.270+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:30.270+0000 7f4150194640 20 osd.0 30 agent_entry empty queue 2026-02-11T23:30:30.270+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 30 epoch_requested: 30 NullEvt) class_id 2immediate prio 255 cost 10 e30) queued 2026-02-11T23:30:30.270+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:30.270+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 30 epoch_requested: 30 NullEvt) class_id 2immediate prio 255 cost 10 e30) pg 0x5946000 2026-02-11T23:30:30.270+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:30.270+0000 7f414b98b640 10 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown NOTIFY mbc={}] handle_advance_map: 30 2026-02-11T23:30:30.270+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 1.0 29 -> 30 2026-02-11T23:30:30.270+0000 7f414b98b640 10 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown NOTIFY mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:30.270+0000 7f414b98b640 10 osd.0 pg_epoch: 30 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown NOTIFY mbc={}] state: Started advmap 2026-02-11T23:30:30.270+0000 7f414b98b640 10 osd.0 pg_epoch: 30 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown NOTIFY mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:30.270+0000 7f414b98b640 10 osd.0 pg_epoch: 30 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown NOTIFY mbc={}] plpg_on_pool_change 2026-02-11T23:30:30.270+0000 7f414b98b640 20 osd.0 pg_epoch: 30 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown NOTIFY mbc={}] hit_set_clear 2026-02-11T23:30:30.270+0000 7f414b98b640 20 osd.0 pg_epoch: 30 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown NOTIFY mbc={}] agent_stop 2026-02-11T23:30:30.270+0000 7f414b98b640 20 osd.0 30 advance_pg new pool opts old pool opts 2026-02-11T23:30:30.270+0000 7f414b98b640 10 osd.0 pg_epoch: 30 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown NOTIFY mbc={}] handle_activate_map: epoch range: 30..30 2026-02-11T23:30:30.270+0000 7f414b98b640 10 osd.0 pg_epoch: 30 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown NOTIFY mbc={}] PeeringState::activate_map 2026-02-11T23:30:30.270+0000 7f414b98b640 20 osd.0 pg_epoch: 30 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown NOTIFY mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 29 while current is 30 2026-02-11T23:30:30.270+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:30.270+0000 7f414b98b640 10 osd.0 pg_epoch: 30 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown NOTIFY mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:30:30.270+0000 7f414b98b640 10 osd.0 pg_epoch: 30 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown NOTIFY mbc={}] do_peering_event: epoch_sent: 30 epoch_requested: 30 NullEvt 2026-02-11T23:30:30.270+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:30.270+0000 7f414b98b640 20 osd.0 30 get_con_osd_cluster to osd.1 from_epoch 30 2026-02-11T23:30:30.270+0000 7f414b98b640 20 osd.0 30 get_nextmap_reserved map_reservations: {30=1} 2026-02-11T23:30:30.270+0000 7f414b98b640 20 osd.0 30 release_map epoch: 30 2026-02-11T23:30:30.270+0000 7f414b98b640 10 osd.0 30 maybe_share_map: con v2:127.0.0.1:6812/3043114013 map epoch 29 -> 30 (shared) 2026-02-11T23:30:30.270+0000 7f414b98b640 10 osd.0 30 send_incremental_map epoch range: (30, 30] to v2:127.0.0.1:6812/3043114013 2026-02-11T23:30:30.271+0000 7f414b98b640 10 osd.0 30 add_map_inc_bl 30 582 bytes 2026-02-11T23:30:30.271+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- osd_map(30..30 src has 1..30) -- 0x4a30700 con 0x4acb000 2026-02-11T23:30:30.271+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_notify2(1.0 1.0 (query:30 sent:30 1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) ([27,28] all_participants={0, 1, 2} intervals=[([27,28] acting={0, 1, 2})])) e30/30) -- 0x3739500 con 0x4acb000 2026-02-11T23:30:30.271+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:30.271+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 10 ==== PGlog(1.0 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 26.936565399s int 16.000000000s) e30/29) ==== 1164+0+0 (unknown 0 0 0) 0x4a1b800 con 0x4acb000 2026-02-11T23:30:30.271+0000 7f416abfa640 15 osd.0 30 enqueue_peering_evt 1.0 epoch_sent: 30 epoch_requested: 29 MLogRec from 1 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 26.936565399s int 16.000000000s) +create_info 2026-02-11T23:30:30.271+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 30 epoch_requested: 29 MLogRec from 1 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 26.936565399s int 16.000000000s) +create_info) class_id 2 prio 255 cost 10 e30) 2026-02-11T23:30:30.271+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:30.271+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 30 epoch_requested: 29 MLogRec from 1 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 26.936565399s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e30) queued 2026-02-11T23:30:30.271+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:30.271+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 30 epoch_requested: 29 MLogRec from 1 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 26.936565399s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e30) pg 0x5946000 2026-02-11T23:30:30.271+0000 7f414f993640 10 osd.0 pg_epoch: 30 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown NOTIFY mbc={}] do_peering_event: epoch_sent: 30 epoch_requested: 29 MLogRec from 1 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 26.936565399s int 16.000000000s) +create_info 2026-02-11T23:30:30.271+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:30.271+0000 7f414f993640 10 osd.0 pg_epoch: 30 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown NOTIFY mbc={}] state: got info+log from osd.1 1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) log((0'0,0'0], crt=0'0) 2026-02-11T23:30:30.271+0000 7f414f993640 10 merge_log log((0'0,0'0], crt=0'0) from osd.1 into log((0'0,0'0], crt=0'0) 2026-02-11T23:30:30.271+0000 7f414f993640 5 merge_log_dups log.dups.size()=0olog.dups.size()=0 2026-02-11T23:30:30.271+0000 7f414f993640 5 end of merge_log_dups changed=0 log.dups.size()=0 olog.dups.size()=0 2026-02-11T23:30:30.271+0000 7f414f993640 10 merge_log result log((0'0,0'0], crt=0'0) missing(0 may_include_deletes = 1) changed=0 2026-02-11T23:30:30.271+0000 7f414f993640 10 osd.0 pg_epoch: 30 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown NOTIFY mbc={}] PeeringState::proc_lease pg_lease(ru 0.000000000s ub 26.936565399s int 16.000000000s) 2026-02-11T23:30:30.271+0000 7f414f993640 20 osd.0 pg_epoch: 30 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown NOTIFY mbc={}] PeeringState::proc_lease peer_clock_delta_lb -2.011821747s -> ruub 28.948387146s 2026-02-11T23:30:30.271+0000 7f414f993640 20 osd.0 pg_epoch: 30 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown NOTIFY mbc={}] PeeringState::proc_lease readable_until_ub now 28.948387146s 2026-02-11T23:30:30.271+0000 7f414f993640 5 osd.0 pg_epoch: 30 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown NOTIFY mbc={}] exit Started/Stray 0.100397 6 2026-02-11T23:30:30.172719+0000 2026-02-11T23:30:30.271+0000 7f414f993640 5 osd.0 pg_epoch: 30 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown NOTIFY mbc={}] enter Started/ReplicaActive 2026-02-11T23:30:30.271+0000 7f414f993640 5 osd.0 pg_epoch: 30 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown NOTIFY mbc={}] enter Started/ReplicaActive/RepNotRecovering 2026-02-11T23:30:30.271+0000 7f414f993640 10 osd.0 pg_epoch: 30 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown NOTIFY mbc={}] state: In ReplicaActive, about to call activate 2026-02-11T23:30:30.271+0000 7f414f993640 10 osd.0 pg_epoch: 30 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown mbc={}] PeeringState::activate activate - no missing, moving last_complete 0'0 -> 0'0 2026-02-11T23:30:30.271+0000 7f414f993640 10 osd.0 pg_epoch: 30 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown mbc={}] scrubber: on_replica_activate 2026-02-11T23:30:30.271+0000 7f414f993640 20 scrubberFSM event: --vvvv---- ReplicaActivate 2026-02-11T23:30:30.271+0000 7f414f993640 10 osd.0 pg_epoch: 30 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown mbc={}] scrubber: FSM: -- state -->> ReplicaActive 2026-02-11T23:30:30.271+0000 7f414f993640 10 osd.0 pg_epoch: 30 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown mbc={}] scrubber: FSM: -- state -->> ReplicaActive/ReplicaIdle 2026-02-11T23:30:30.271+0000 7f414f993640 20 scrubberFSM event: --^^^^---- ReplicaActivate 2026-02-11T23:30:30.271+0000 7f414f993640 10 osd.0 pg_epoch: 30 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown mbc={}] state: Activate Finished 2026-02-11T23:30:30.271+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:30.271+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:30.271+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:30.271+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:30.271+0000 7f414f993640 15 osd.0 30 enqueue_peering_evt 1.0 epoch_sent: 30 epoch_requested: 30 ActivateCommitted from 30 processed at 30 2026-02-11T23:30:30.271+0000 7f414f993640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 30 epoch_requested: 30 ActivateCommitted from 30 processed at 30) class_id 2 prio 255 cost 10 e30) 2026-02-11T23:30:30.271+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:30.271+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 30 epoch_requested: 30 ActivateCommitted from 30 processed at 30) class_id 2immediate prio 255 cost 10 e30) queued 2026-02-11T23:30:30.271+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:30.271+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 30 epoch_requested: 30 ActivateCommitted from 30 processed at 30) class_id 2immediate prio 255 cost 10 e30) pg 0x5946000 2026-02-11T23:30:30.271+0000 7f414f993640 10 osd.0 pg_epoch: 30 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown mbc={}] do_peering_event: epoch_sent: 30 epoch_requested: 30 ActivateCommitted from 30 processed at 30 2026-02-11T23:30:30.271+0000 7f414f993640 10 osd.0 pg_epoch: 30 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 unknown mbc={}] state: react 30 telling primary 2026-02-11T23:30:30.271+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:30.271+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:30.271+0000 7f414f993640 20 osd.0 30 get_con_osd_cluster to osd.1 from_epoch 30 2026-02-11T23:30:30.271+0000 7f414f993640 20 osd.0 30 get_nextmap_reserved map_reservations: {30=1} 2026-02-11T23:30:30.271+0000 7f414f993640 20 osd.0 30 release_map epoch: 30 2026-02-11T23:30:30.271+0000 7f414f993640 10 osd.0 30 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 30 is not newer than session's projected_epoch of 30 2026-02-11T23:30:30.271+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_info2(1.0 1.0 1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/0 les/c/f=30/0/0 sis=29) e30/30) -- 0x373a300 con 0x4acb000 2026-02-11T23:30:30.271+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:30.272+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 11 ==== pg_lease(1.0 pg_lease(ru 26.936565399s ub 26.937025070s int 16.000000000s) e30/30) ==== 58+0+0 (unknown 0 0 0) 0x4c9b200 con 0x4acb000 2026-02-11T23:30:30.272+0000 7f416abfa640 15 osd.0 30 enqueue_peering_evt 1.0 epoch_sent: 30 epoch_requested: 30 MLease epoch 30 from osd.1 pg_lease(ru 26.936565399s ub 26.937025070s int 16.000000000s) 2026-02-11T23:30:30.272+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 30 epoch_requested: 30 MLease epoch 30 from osd.1 pg_lease(ru 26.936565399s ub 26.937025070s int 16.000000000s)) class_id 2 prio 255 cost 10 e30) 2026-02-11T23:30:30.272+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:30.272+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 30 epoch_requested: 30 MLease epoch 30 from osd.1 pg_lease(ru 26.936565399s ub 26.937025070s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e30) queued 2026-02-11T23:30:30.272+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:30.272+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:30.272+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 30 epoch_requested: 30 MLease epoch 30 from osd.1 pg_lease(ru 26.936565399s ub 26.937025070s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e30) pg 0x5946000 2026-02-11T23:30:30.272+0000 7f414b98b640 10 osd.0 pg_epoch: 30 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 30 epoch_requested: 30 MLease epoch 30 from osd.1 pg_lease(ru 26.936565399s ub 26.937025070s int 16.000000000s) 2026-02-11T23:30:30.272+0000 7f414b98b640 10 osd.0 pg_epoch: 30 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 active mbc={}] PeeringState::proc_lease pg_lease(ru 26.936565399s ub 26.937025070s int 16.000000000s) 2026-02-11T23:30:30.272+0000 7f414b98b640 20 osd.0 pg_epoch: 30 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 active mbc={}] PeeringState::proc_lease peer_clock_delta_ub -2.011363983s -> ru 28.947931290s 2026-02-11T23:30:30.272+0000 7f414b98b640 20 osd.0 pg_epoch: 30 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 active mbc={}] PeeringState::proc_lease readable_until now 28.947931290s 2026-02-11T23:30:30.272+0000 7f414b98b640 20 osd.0 pg_epoch: 30 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 active mbc={}] PeeringState::proc_lease peer_clock_delta_lb -2.011821747s -> ruub 28.948846817s 2026-02-11T23:30:30.272+0000 7f414b98b640 20 osd.0 pg_epoch: 30 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 active mbc={}] PeeringState::proc_lease readable_until_ub now 28.948846817s 2026-02-11T23:30:30.272+0000 7f414b98b640 20 osd.0 30 get_con_osd_cluster to osd.1 from_epoch 30 2026-02-11T23:30:30.272+0000 7f414b98b640 20 osd.0 30 get_nextmap_reserved map_reservations: {30=1} 2026-02-11T23:30:30.272+0000 7f414b98b640 20 osd.0 30 release_map epoch: 30 2026-02-11T23:30:30.272+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_lease_ack(1.0 pg_lease_ack(ruub 26.937025070s) e30/30) -- 0x3700000 con 0x4acb000 2026-02-11T23:30:30.272+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 12 ==== pg_info2(1.0 1.0 1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/0 les/c/f=30/0/0 sis=29) e30/30) ==== 1118+0+0 (unknown 0 0 0) 0x373aa00 con 0x4acb000 2026-02-11T23:30:30.272+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:30.272+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:30.272+0000 7f416abfa640 15 osd.0 30 enqueue_peering_evt 1.0 epoch_sent: 30 epoch_requested: 30 MInfoRec from 1 info: 1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/0 les/c/f=30/0/0 sis=29) pg_lease(ru 26.936565399s ub 26.937025070s int 16.000000000s) 2026-02-11T23:30:30.272+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 30 epoch_requested: 30 MInfoRec from 1 info: 1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/0 les/c/f=30/0/0 sis=29) pg_lease(ru 26.936565399s ub 26.937025070s int 16.000000000s)) class_id 2 prio 255 cost 10 e30) 2026-02-11T23:30:30.272+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:30.272+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 30 epoch_requested: 30 MInfoRec from 1 info: 1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/0 les/c/f=30/0/0 sis=29) pg_lease(ru 26.936565399s ub 26.937025070s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e30) queued 2026-02-11T23:30:30.272+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:30.272+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 30 epoch_requested: 30 MInfoRec from 1 info: 1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/0 les/c/f=30/0/0 sis=29) pg_lease(ru 26.936565399s ub 26.937025070s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e30) pg 0x5946000 2026-02-11T23:30:30.272+0000 7f414f993640 10 osd.0 pg_epoch: 30 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=0/0 les/c/f=0/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 30 epoch_requested: 30 MInfoRec from 1 info: 1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/0 les/c/f=30/0/0 sis=29) pg_lease(ru 26.936565399s ub 26.937025070s int 16.000000000s) 2026-02-11T23:30:30.272+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:30.272+0000 7f414f993640 20 osd.0 pg_epoch: 30 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/0 les/c/f=30/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 active mbc={}] PeeringState::update_history advanced history from ec=27/27 lis/c=29/0 les/c/f=30/0/0 sis=29 2026-02-11T23:30:30.272+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:30.272+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:30.272+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 13 ==== pg_info2(1.0 1.0 1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) e30/30) ==== 1118+0+0 (unknown 0 0 0) 0x373aa00 con 0x4acb000 2026-02-11T23:30:30.272+0000 7f416abfa640 15 osd.0 30 enqueue_peering_evt 1.0 epoch_sent: 30 epoch_requested: 30 MInfoRec from 1 info: 1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) pg_lease(ru 26.936565399s ub 26.937025070s int 16.000000000s) 2026-02-11T23:30:30.272+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:30.272+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:30.272+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 30 epoch_requested: 30 MInfoRec from 1 info: 1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) pg_lease(ru 26.936565399s ub 26.937025070s int 16.000000000s)) class_id 2 prio 255 cost 10 e30) 2026-02-11T23:30:30.272+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:30.272+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 30 epoch_requested: 30 MInfoRec from 1 info: 1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) pg_lease(ru 26.936565399s ub 26.937025070s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e30) queued 2026-02-11T23:30:30.272+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:30.272+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 30 epoch_requested: 30 MInfoRec from 1 info: 1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) pg_lease(ru 26.936565399s ub 26.937025070s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e30) pg 0x5946000 2026-02-11T23:30:30.272+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:30.272+0000 7f414f993640 10 osd.0 pg_epoch: 30 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/0 les/c/f=30/0/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 30 epoch_requested: 30 MInfoRec from 1 info: 1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) pg_lease(ru 26.936565399s ub 26.937025070s int 16.000000000s) 2026-02-11T23:30:30.272+0000 7f414f993640 20 osd.0 pg_epoch: 30 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 active mbc={}] PeeringState::update_history advanced history from ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29 2026-02-11T23:30:30.272+0000 7f414f993640 20 osd.0 pg_epoch: 30 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 pi=[27,29)/1 crt=0'0 active mbc={}] PeeringState::update_history clearing past_intervals 2026-02-11T23:30:30.272+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:30.272+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:30.272+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:30.272+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:30.325+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:30:30.326+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:30:30.326+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:00.327310+0000) 2026-02-11T23:30:30.326+0000 7f415bf00640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-11T23:30:30.336+0000 7f41633eb640 10 osd.0 30 tick 2026-02-11T23:30:30.336+0000 7f41633eb640 20 osd.0 30 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:30:30.779+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 29 ==== osd_map(33..34 src has 1..34) ==== 1217+0+0 (crc 0 0 0) 0x5954540 con 0x4acbc00 2026-02-11T23:30:30.779+0000 7f415cf02640 20 osd.0 30 OSD::ms_dispatch: osd_map(33..34 src has 1..34) 2026-02-11T23:30:30.779+0000 7f415cf02640 20 osd.0 30 _dispatch 0x5954540 osd_map(33..34 src has 1..34) 2026-02-11T23:30:30.779+0000 7f415cf02640 10 monclient: _renew_subs 2026-02-11T23:30:30.779+0000 7f415cf02640 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:30:30.779+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> v2:127.0.0.1:7180/0 -- mon_subscribe({osdmap=31}) -- 0x56b64e0 con 0x4acbc00 2026-02-11T23:30:30.779+0000 7f415cf02640 3 osd.0 30 handle_osd_map epochs [33,34], i have 30, src has [1,34] 2026-02-11T23:30:30.779+0000 7f415cf02640 10 osd.0 30 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:30.779+0000 7f415cf02640 10 osd.0 30 handle_osd_map message skips epochs 31..32 2026-02-11T23:30:30.780+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 30 ==== osd_map(31..34 src has 1..34) ==== 2381+0+0 (crc 0 0 0) 0x4a30fc0 con 0x4acbc00 2026-02-11T23:30:30.780+0000 7f415cf02640 20 osd.0 30 OSD::ms_dispatch: osd_map(31..34 src has 1..34) 2026-02-11T23:30:30.780+0000 7f415cf02640 20 osd.0 30 _dispatch 0x4a30fc0 osd_map(31..34 src has 1..34) 2026-02-11T23:30:30.780+0000 7f415cf02640 3 osd.0 30 handle_osd_map epochs [31,34], i have 30, src has [1,34] 2026-02-11T23:30:30.780+0000 7f415cf02640 10 osd.0 30 handle_osd_map: received 0 full maps and 4 incremental maps 2026-02-11T23:30:30.780+0000 7f415cf02640 10 osd.0 30 handle_osd_map got inc map for epoch 31 2026-02-11T23:30:30.780+0000 7f415cf02640 20 osd.0 30 got_full_map 31, nothing requested 2026-02-11T23:30:30.780+0000 7f415cf02640 10 osd.0 30 handle_osd_map got inc map for epoch 32 2026-02-11T23:30:30.780+0000 7f415cf02640 20 osd.0 30 got_full_map 32, nothing requested 2026-02-11T23:30:30.780+0000 7f415cf02640 10 osd.0 30 handle_osd_map got inc map for epoch 33 2026-02-11T23:30:30.780+0000 7f415cf02640 20 osd.0 30 got_full_map 33, nothing requested 2026-02-11T23:30:30.780+0000 7f415cf02640 10 osd.0 30 handle_osd_map got inc map for epoch 34 2026-02-11T23:30:30.780+0000 7f415cf02640 20 osd.0 30 got_full_map 34, nothing requested 2026-02-11T23:30:30.780+0000 7f415cf02640 10 osd.0 30 _track_pools_and_pg_num_changes recording new pool 2 pg_num 1 2026-02-11T23:30:30.780+0000 7f415cf02640 20 osd.0 30 trim_maps: min=0 oldest_map=1 2026-02-11T23:30:30.780+0000 7f415cf02640 20 osd.0 30 handle_osd_map pg_num_history pg_num_history(e34 pg_nums {1={27=1},2={34=1}} deleted_pools ) 2026-02-11T23:30:30.780+0000 7f415cf02640 10 snap_mapper record_purged_snaps purged_snaps {31={},32={},33={},34={}} 2026-02-11T23:30:30.780+0000 7f415cf02640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:30.780+0000 7f415cf02640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e34 maps [1~34] lci=[5,34] tlb=1) 2026-02-11T23:30:30.780+0000 7f41591a6640 10 osd.0 30 _committed_osd_maps 31..34 2026-02-11T23:30:30.780+0000 7f41591a6640 10 osd.0 30 advance to epoch 31 (<= last 34 <= newest_map 34) 2026-02-11T23:30:30.780+0000 7f41591a6640 10 osd.0 31 advance to epoch 32 (<= last 34 <= newest_map 34) 2026-02-11T23:30:30.780+0000 7f41591a6640 10 osd.0 32 advance to epoch 33 (<= last 34 <= newest_map 34) 2026-02-11T23:30:30.780+0000 7f41591a6640 10 osd.0 33 advance to epoch 34 (<= last 34 <= newest_map 34) 2026-02-11T23:30:30.780+0000 7f41591a6640 20 osd.0 34 consume_map version 34 2026-02-11T23:30:30.780+0000 7f41591a6640 20 osd.0 30 await_reserved_maps epoch:34 2026-02-11T23:30:30.780+0000 7f41591a6640 20 osd.0 30 await_reserved_maps done 0 2026-02-11T23:30:30.780+0000 7f41591a6640 20 osd.0 34 consume_map 34 -- publish done 2026-02-11T23:30:30.780+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 1 slots 2026-02-11T23:30:30.780+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3700dc0 2026-02-11T23:30:30.780+0000 7f41591a6640 20 osd.0 34 identify_splits_and_merges 1.0 e30 to e34 2026-02-11T23:30:30.780+0000 7f41591a6640 20 osd.0 34 identify_splits_and_merges 1.0 e30 to e34 pg_nums {27=1} 2026-02-11T23:30:30.780+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:30.780+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:30.780+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:30.780+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:30.780+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:30.780+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:30.780+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:30.780+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:30.780+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:30.780+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:30.780+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:30.780+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:30.780+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:30.780+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:30.780+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:30.780+0000 7f41591a6640 20 osd.0 34 prune_pg_created 2026-02-11T23:30:30.780+0000 7f41591a6640 10 osd.0:0.consume_map 34 (was 30) 2026-02-11T23:30:30.780+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:30.780+0000 7f41591a6640 10 osd.0:1.consume_map 34 (was 30) 2026-02-11T23:30:30.780+0000 7f41591a6640 10 osd.0:2.consume_map 34 (was 30) 2026-02-11T23:30:30.780+0000 7f41591a6640 10 osd.0:3.consume_map 34 (was 30) 2026-02-11T23:30:30.780+0000 7f41591a6640 10 osd.0:4.consume_map 34 (was 30) 2026-02-11T23:30:30.780+0000 7f41591a6640 10 osd.0:5.consume_map 34 (was 30) 2026-02-11T23:30:30.780+0000 7f41591a6640 10 osd.0:6.consume_map 34 (was 30) 2026-02-11T23:30:30.780+0000 7f41591a6640 10 osd.0:7.consume_map 34 (was 30) 2026-02-11T23:30:30.780+0000 7f41591a6640 10 osd.0 34 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:30.780+0000 7f41591a6640 15 osd.0 34 enqueue_peering_evt 1.0 epoch_sent: 34 epoch_requested: 34 NullEvt 2026-02-11T23:30:30.780+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 34 epoch_requested: 34 NullEvt) class_id 2 prio 255 cost 10 e34) 2026-02-11T23:30:30.780+0000 7f41591a6640 7 osd.0 34 activate_map version 34 2026-02-11T23:30:30.780+0000 7f4150194640 20 osd.0 34 agent_entry empty queue 2026-02-11T23:30:30.780+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:30.780+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 34 epoch_requested: 34 NullEvt) class_id 2immediate prio 255 cost 10 e34) queued 2026-02-11T23:30:30.780+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:30.780+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 34 epoch_requested: 34 NullEvt) class_id 2immediate prio 255 cost 10 e34) pg 0x5946000 2026-02-11T23:30:30.780+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:30.780+0000 7f414f993640 10 osd.0 pg_epoch: 30 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 31 2026-02-11T23:30:30.780+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 30 -> 31 2026-02-11T23:30:30.780+0000 7f414f993640 10 osd.0 pg_epoch: 30 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:30.780+0000 7f414f993640 10 osd.0 pg_epoch: 31 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:30.780+0000 7f414f993640 10 osd.0 pg_epoch: 31 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:30.780+0000 7f414f993640 10 osd.0 pg_epoch: 31 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] plpg_on_pool_change 2026-02-11T23:30:30.780+0000 7f414f993640 20 osd.0 pg_epoch: 31 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] hit_set_clear 2026-02-11T23:30:30.780+0000 7f414f993640 20 osd.0 pg_epoch: 31 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] agent_stop 2026-02-11T23:30:30.780+0000 7f414f993640 20 osd.0 34 advance_pg new pool opts old pool opts 2026-02-11T23:30:30.780+0000 7f414f993640 10 osd.0 pg_epoch: 31 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 32 2026-02-11T23:30:30.780+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 31 -> 32 2026-02-11T23:30:30.780+0000 7f414f993640 10 osd.0 pg_epoch: 31 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:30.780+0000 7f414f993640 10 osd.0 pg_epoch: 32 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:30.780+0000 7f414f993640 10 osd.0 pg_epoch: 32 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:30.780+0000 7f414f993640 10 osd.0 pg_epoch: 32 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] plpg_on_pool_change 2026-02-11T23:30:30.780+0000 7f414f993640 20 osd.0 pg_epoch: 32 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] hit_set_clear 2026-02-11T23:30:30.780+0000 7f414f993640 20 osd.0 pg_epoch: 32 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] agent_stop 2026-02-11T23:30:30.780+0000 7f414f993640 20 osd.0 34 advance_pg new pool opts old pool opts 2026-02-11T23:30:30.780+0000 7f414f993640 10 osd.0 pg_epoch: 32 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 33 2026-02-11T23:30:30.780+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 32 -> 33 2026-02-11T23:30:30.780+0000 7f414f993640 10 osd.0 pg_epoch: 32 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:30.780+0000 7f414f993640 10 osd.0 pg_epoch: 33 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:30.780+0000 7f414f993640 10 osd.0 pg_epoch: 33 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:30.780+0000 7f414f993640 10 osd.0 pg_epoch: 33 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] plpg_on_pool_change 2026-02-11T23:30:30.780+0000 7f414f993640 20 osd.0 pg_epoch: 33 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] hit_set_clear 2026-02-11T23:30:30.780+0000 7f414f993640 20 osd.0 pg_epoch: 33 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] agent_stop 2026-02-11T23:30:30.780+0000 7f414f993640 20 osd.0 34 advance_pg new pool opts old pool opts 2026-02-11T23:30:30.780+0000 7f414f993640 10 osd.0 pg_epoch: 33 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 34 2026-02-11T23:30:30.780+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 33 -> 34 2026-02-11T23:30:30.780+0000 7f414f993640 10 osd.0 pg_epoch: 33 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:30.780+0000 7f414f993640 10 osd.0 pg_epoch: 34 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:30.780+0000 7f414f993640 10 osd.0 pg_epoch: 34 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:30.780+0000 7f414f993640 20 osd.0 34 advance_pg new pool opts old pool opts 2026-02-11T23:30:30.780+0000 7f414f993640 10 osd.0 pg_epoch: 34 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_activate_map: epoch range: 31..34 2026-02-11T23:30:30.780+0000 7f414f993640 10 osd.0 pg_epoch: 34 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:30.780+0000 7f414f993640 20 osd.0 pg_epoch: 34 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 30 while current is 34 2026-02-11T23:30:30.780+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:30.780+0000 7f414f993640 10 osd.0 pg_epoch: 34 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:30:30.780+0000 7f414f993640 10 osd.0 pg_epoch: 34 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 34 epoch_requested: 34 NullEvt 2026-02-11T23:30:30.780+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:30.780+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:30.838+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.1 127.0.0.1:0/3043114013 5 ==== osd_ping(ping e31 up_from 10 ping_stamp 2026-02-11T23:30:30.839742+0000/11.503487587s send_stamp 11.503487587s delta_ub -2.011442900s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a900 con 0x4aca000 2026-02-11T23:30:30.838+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.1 127.0.0.1:0/3043114013 5 ==== osd_ping(ping e31 up_from 10 ping_stamp 2026-02-11T23:30:30.839742+0000/11.503487587s send_stamp 11.503487587s delta_ub -2.011442900s) ==== 2033+0+0 (crc 0 0 0) 0x4c9b380 con 0x4aca400 2026-02-11T23:30:30.838+0000 7f416c3fd640 20 osd.0 34 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011729002s,-2.011442900s]) 2026-02-11T23:30:30.838+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e34 up_from 5 ping_stamp 2026-02-11T23:30:30.839742+0000/11.503487587s send_stamp 13.515216827s delta_ub 2.011729002s) -- 0x4c9a300 con 0x4aca400 2026-02-11T23:30:30.838+0000 7f416c3fd640 20 osd.0 34 get_con_osd_cluster to osd.1 from_epoch 34 2026-02-11T23:30:30.838+0000 7f416c3fd640 20 osd.0 34 get_nextmap_reserved map_reservations: {34=1} 2026-02-11T23:30:30.838+0000 7f416c3fd640 20 osd.0 34 release_map epoch: 34 2026-02-11T23:30:30.838+0000 7f416c3fd640 10 osd.0 34 maybe_share_map: con v2:127.0.0.1:6812/3043114013 updating session's projected_epoch from 30 to ping map epoch of 31 2026-02-11T23:30:30.838+0000 7f416c3fd640 10 osd.0 34 maybe_share_map: con v2:127.0.0.1:6812/3043114013 map epoch 31 -> 34 (shared) 2026-02-11T23:30:30.838+0000 7f416c3fd640 10 osd.0 34 send_incremental_map epoch range: (32, 34] to v2:127.0.0.1:6812/3043114013 2026-02-11T23:30:30.838+0000 7f416c3fd640 10 osd.0 34 add_map_inc_bl 32 574 bytes 2026-02-11T23:30:30.838+0000 7f416c3fd640 10 osd.0 34 add_map_inc_bl 33 574 bytes 2026-02-11T23:30:30.838+0000 7f416c3fd640 10 osd.0 34 add_map_inc_bl 34 591 bytes 2026-02-11T23:30:30.838+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- osd_map(32..34 src has 1..34) -- 0x4a30e00 con 0x4acb000 2026-02-11T23:30:30.838+0000 7f416cbfe640 20 osd.0 34 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.012033463s,-2.011442900s]) 2026-02-11T23:30:30.838+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e34 up_from 5 ping_stamp 2026-02-11T23:30:30.839742+0000/11.503487587s send_stamp 13.515521049s delta_ub 2.012033463s) -- 0x4c9a600 con 0x4aca000 2026-02-11T23:30:30.838+0000 7f416cbfe640 20 osd.0 34 get_con_osd_cluster to osd.1 from_epoch 34 2026-02-11T23:30:30.838+0000 7f416cbfe640 20 osd.0 34 get_nextmap_reserved map_reservations: {34=1} 2026-02-11T23:30:30.838+0000 7f416cbfe640 20 osd.0 34 release_map epoch: 34 2026-02-11T23:30:30.838+0000 7f416cbfe640 10 osd.0 34 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 34 is not newer than session's projected_epoch of 34 2026-02-11T23:30:31.257+0000 7f4162bea640 10 osd.0 34 tick_without_osd_lock 2026-02-11T23:30:31.257+0000 7f4162bea640 0 osd.0 34 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:31.257+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:31.257+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.549 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:31.257+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:31.258, recovery is active?:false restrictions:<....> 2026-02-11T23:30:31.257+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:31.257+0000 7f4162bea640 10 osd.0 34 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:31.257+0000 7f4162bea640 20 osd.0 34 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:31.257+0000 7f4162bea640 10 osd.0 34 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:31.257+0000 7f4162bea640 20 osd.0 34 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:31.262+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.4 127.0.0.1:0/3331151400 1 ==== osd_ping(ping e31 up_from 23 ping_stamp 2026-02-11T23:30:31.263823+0000/4.102591038s send_stamp 4.102591038s delta_ub -9.836466789s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a300 con 0x507a800 2026-02-11T23:30:31.262+0000 7f416c3fd640 20 osd.0 34 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836747169s,-9.836466789s]) 2026-02-11T23:30:31.262+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e34 up_from 5 ping_stamp 2026-02-11T23:30:31.263823+0000/4.102591038s send_stamp 13.939338684s delta_ub 9.836747169s) -- 0x4c9b380 con 0x507a800 2026-02-11T23:30:31.262+0000 7f416c3fd640 20 osd.0 34 get_con_osd_cluster to osd.4 from_epoch 34 2026-02-11T23:30:31.262+0000 7f416c3fd640 20 osd.0 34 get_nextmap_reserved map_reservations: {34=1} 2026-02-11T23:30:31.262+0000 7f416c3fd640 20 osd.0 34 release_map epoch: 34 2026-02-11T23:30:31.262+0000 7f416c3fd640 10 osd.0 34 maybe_share_map: con v2:127.0.0.1:6836/3331151400 updating session's projected_epoch from 0 to ping map epoch of 31 2026-02-11T23:30:31.262+0000 7f416c3fd640 10 osd.0 34 maybe_share_map: con v2:127.0.0.1:6836/3331151400 map epoch 31 -> 34 (shared) 2026-02-11T23:30:31.262+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.4 127.0.0.1:0/3331151400 1 ==== osd_ping(ping e31 up_from 23 ping_stamp 2026-02-11T23:30:31.263823+0000/4.102591038s send_stamp 4.102591038s delta_ub -9.836466789s) ==== 2033+0+0 (crc 0 0 0) 0x4c9b200 con 0x507a400 2026-02-11T23:30:31.262+0000 7f416c3fd640 10 osd.0 34 send_incremental_map epoch range: (32, 34] to v2:127.0.0.1:6836/3331151400 2026-02-11T23:30:31.262+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- osd_map(32..34 src has 1..34) -- 0x5954700 con 0x507ac00 2026-02-11T23:30:31.262+0000 7f416abfa640 20 osd.0 34 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836804390s,-9.836466789s]) 2026-02-11T23:30:31.262+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e34 up_from 5 ping_stamp 2026-02-11T23:30:31.263823+0000/4.102591038s send_stamp 13.939395905s delta_ub 9.836804390s) -- 0x4c9a780 con 0x507a400 2026-02-11T23:30:31.262+0000 7f416abfa640 20 osd.0 34 get_con_osd_cluster to osd.4 from_epoch 34 2026-02-11T23:30:31.262+0000 7f416abfa640 20 osd.0 34 get_nextmap_reserved map_reservations: {34=1} 2026-02-11T23:30:31.262+0000 7f416abfa640 20 osd.0 34 release_map epoch: 34 2026-02-11T23:30:31.262+0000 7f416abfa640 10 osd.0 34 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 34 is not newer than session's projected_epoch of 34 2026-02-11T23:30:31.307+0000 7f41633eb640 10 osd.0 34 tick 2026-02-11T23:30:31.307+0000 7f41633eb640 20 osd.0 34 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:30:31.326+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:30:31.326+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:30:31.326+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:01.327404+0000) 2026-02-11T23:30:31.438+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.3 v2:127.0.0.1:6828/2699377081 2 ==== pg_query2(2.0 2.0 query(info 0'0 epoch_sent 34) e34/34) ==== 158+0+0 (crc 0 0 0) 0x5968000 con 0x48ccc00 2026-02-11T23:30:31.438+0000 7f415aefe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.3 v2:127.0.0.1:6828/2699377081 1 ==== osd_map(24..34 src has 1..34) ==== 5218+0+0 (crc 0 0 0) 0x4a31180 con 0x48ccc00 2026-02-11T23:30:31.438+0000 7f415aefe640 20 osd.0 34 OSD::ms_dispatch: osd_map(24..34 src has 1..34) 2026-02-11T23:30:31.438+0000 7f415aefe640 20 osd.0 34 _dispatch 0x4a31180 osd_map(24..34 src has 1..34) 2026-02-11T23:30:31.438+0000 7f416cbfe640 15 osd.0 34 enqueue_peering_evt 2.0 epoch_sent: 34 epoch_requested: 34 MQuery 2.0 from 3 query_epoch 34 query: query(info 0'0 epoch_sent 34) 2026-02-11T23:30:31.438+0000 7f415aefe640 3 osd.0 34 handle_osd_map epochs [24,34], i have 34, src has [1,34] 2026-02-11T23:30:31.438+0000 7f415aefe640 10 osd.0 34 handle_osd_map: received 0 full maps and 11 incremental maps 2026-02-11T23:30:31.438+0000 7f416cbfe640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 34 epoch_requested: 34 MQuery 2.0 from 3 query_epoch 34 query: query(info 0'0 epoch_sent 34)) class_id 2 prio 255 cost 10 e34) 2026-02-11T23:30:31.438+0000 7f415aefe640 10 osd.0 34 no new maps here, dropping 2026-02-11T23:30:31.438+0000 7f414b98b640 20 osd.0 op_wq(0) _process 2.0 (new) to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:31.438+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 34 epoch_requested: 34 MQuery 2.0 from 3 query_epoch 34 query: query(info 0'0 epoch_sent 34)) class_id 2immediate prio 255 cost 10 e34) queued 2026-02-11T23:30:31.438+0000 7f414b98b640 20 osd.0 op_wq(0) _process 2.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:31.438+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 34 epoch_requested: 34 MQuery 2.0 from 3 query_epoch 34 query: query(info 0'0 epoch_sent 34)) class_id 2immediate prio 255 cost 10 e34) pg 0 2026-02-11T23:30:31.438+0000 7f414b98b640 10 osd.0 34 handle_pg_query_nopg 2.0 2026-02-11T23:30:31.438+0000 7f414b98b640 10 osd.0 34 pg 2.0 dne 2026-02-11T23:30:31.438+0000 7f414b98b640 20 osd.0 34 get_con_osd_cluster to osd.3 from_epoch 34 2026-02-11T23:30:31.438+0000 7f414b98b640 20 osd.0 34 get_nextmap_reserved map_reservations: {34=1} 2026-02-11T23:30:31.438+0000 7f414b98b640 20 osd.0 34 release_map epoch: 34 2026-02-11T23:30:31.438+0000 7f414b98b640 10 osd.0 34 maybe_share_map: con v2:127.0.0.1:6828/2699377081 map epoch 23 -> 34 (shared) 2026-02-11T23:30:31.438+0000 7f414b98b640 10 osd.0 34 send_incremental_map epoch range: (24, 34] to v2:127.0.0.1:6828/2699377081 2026-02-11T23:30:31.438+0000 7f414b98b640 10 osd.0 34 add_map_inc_bl 24 294 bytes 2026-02-11T23:30:31.438+0000 7f414b98b640 10 osd.0 34 add_map_inc_bl 25 244 bytes 2026-02-11T23:30:31.439+0000 7f414b98b640 10 osd.0 34 add_map_inc_bl 31 574 bytes 2026-02-11T23:30:31.439+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6828/2699377081,v1:127.0.0.1:6829/2699377081] -- osd_map(24..34 src has 1..34) -- 0x4a64fc0 con 0x48ccc00 2026-02-11T23:30:31.439+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6828/2699377081,v1:127.0.0.1:6829/2699377081] -- pg_notify2(2.0 2.0 (query:34 sent:34 2.0( DNE empty local-lis/les=0/0 n=0 ec=0/0 lis/c=0/0 les/c/f=0/0/0 sis=0) ([0,0] all_participants={} intervals=[])) e34/34) -- 0x3739c00 con 0x48ccc00 2026-02-11T23:30:31.439+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:31.439+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:31.490+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.3 v2:127.0.0.1:6828/2699377081 3 ==== PGlog(2.0 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 22.947696686s int 16.000000000s) e35/34) ==== 1172+0+0 (crc 0 0 0) 0x4a1c000 con 0x48ccc00 2026-02-11T23:30:31.490+0000 7f416cbfe640 15 osd.0 34 enqueue_peering_evt 2.0 epoch_sent: 35 epoch_requested: 34 MLogRec from 3 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 22.947696686s int 16.000000000s) +create_info 2026-02-11T23:30:31.490+0000 7f416cbfe640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 35 epoch_requested: 34 MLogRec from 3 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 22.947696686s int 16.000000000s) +create_info) class_id 2 prio 255 cost 10 e35) 2026-02-11T23:30:31.490+0000 7f414b98b640 20 osd.0 op_wq(0) _process 2.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:31.490+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 35 epoch_requested: 34 MLogRec from 3 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 22.947696686s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e35) queued 2026-02-11T23:30:31.490+0000 7f414b98b640 20 osd.0 op_wq(0) _process 2.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:31.490+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 35 epoch_requested: 34 MLogRec from 3 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 22.947696686s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e35) pg 0 2026-02-11T23:30:31.490+0000 7f414b98b640 20 osd.0 op_wq(0) _process 2.0 map 35 > 34 2026-02-11T23:30:31.490+0000 7f414b98b640 20 osd.0 op_wq _add_slot_waiter 2.0 peering, item epoch is 35, will wait on OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 35 epoch_requested: 34 MLogRec from 3 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 22.947696686s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e35) 2026-02-11T23:30:31.490+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:31.490+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:31.926+0000 7f4147983640 5 osd.0 34 heartbeat osd_stat(store_statfs(0x384000/0x0/0x384000, data 0x17f76/0x46000, compress 0x0/0x0/0x0, omap 0x3bc0, meta 0x1a4c440), peers [1,2,3,4] op hist []) 2026-02-11T23:30:31.926+0000 7f4147983640 20 osd.0 34 check_full_status cur ratio 0, physical ratio 0, new state none 2026-02-11T23:30:31.926+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6816/3043114013,v1:127.0.0.1:6817/3043114013] -- osd_ping(ping e34 up_from 5 ping_stamp 2026-02-11T23:30:31.928227+0000/14.603632927s send_stamp 14.603632927s delta_ub 2.012033463s) -- 0x4c9a480 con 0x48c4400 2026-02-11T23:30:31.926+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6814/3043114013,v1:127.0.0.1:6815/3043114013] -- osd_ping(ping e34 up_from 5 ping_stamp 2026-02-11T23:30:31.928227+0000/14.603632927s send_stamp 14.603632927s delta_ub 2.012033463s) -- 0x4c9a180 con 0x4640c00 2026-02-11T23:30:31.926+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6824/2365263801,v1:127.0.0.1:6825/2365263801] -- osd_ping(ping e34 up_from 5 ping_stamp 2026-02-11T23:30:31.928227+0000/14.603632927s send_stamp 14.603632927s delta_ub 4.566884995s) -- 0x4c9ad80 con 0x48c4000 2026-02-11T23:30:31.926+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6822/2365263801,v1:127.0.0.1:6823/2365263801] -- osd_ping(ping e34 up_from 5 ping_stamp 2026-02-11T23:30:31.928227+0000/14.603632927s send_stamp 14.603632927s delta_ub 4.566884995s) -- 0x4c9b080 con 0x48cd000 2026-02-11T23:30:31.926+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6832/2699377081,v1:127.0.0.1:6833/2699377081] -- osd_ping(ping e34 up_from 5 ping_stamp 2026-02-11T23:30:31.928227+0000/14.603632927s send_stamp 14.603632927s delta_ub 7.219560146s) -- 0x4c9aa80 con 0x48cd400 2026-02-11T23:30:31.927+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6830/2699377081,v1:127.0.0.1:6831/2699377081] -- osd_ping(ping e34 up_from 5 ping_stamp 2026-02-11T23:30:31.928227+0000/14.603632927s send_stamp 14.603632927s delta_ub 7.219560146s) -- 0x4c9b200 con 0x48cd800 2026-02-11T23:30:31.927+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6840/3331151400,v1:127.0.0.1:6841/3331151400] -- osd_ping(ping e34 up_from 5 ping_stamp 2026-02-11T23:30:31.928227+0000/14.603632927s send_stamp 14.603632927s delta_ub 9.836804390s) -- 0x4c9a780 con 0x507bc00 2026-02-11T23:30:31.927+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6838/3331151400,v1:127.0.0.1:6839/3331151400] -- osd_ping(ping e34 up_from 5 ping_stamp 2026-02-11T23:30:31.928227+0000/14.603632927s send_stamp 14.603632927s delta_ub 9.836804390s) -- 0x4c9a480 con 0x5282000 2026-02-11T23:30:31.927+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6816/3043114013 4 ==== osd_ping(ping_reply e34 up_from 10 ping_stamp 2026-02-11T23:30:31.928227+0000/14.603632927s send_stamp 12.592060089s delta_ub -2.011571884s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a780 con 0x48c4400 2026-02-11T23:30:31.927+0000 7f416abfa640 20 osd.0 34 get_con_osd_cluster to osd.1 from_epoch 34 2026-02-11T23:30:31.927+0000 7f416abfa640 20 osd.0 34 get_nextmap_reserved map_reservations: {34=1} 2026-02-11T23:30:31.927+0000 7f416abfa640 20 osd.0 34 release_map epoch: 34 2026-02-11T23:30:31.927+0000 7f416abfa640 10 osd.0 34 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 34 is not newer than session's projected_epoch of 34 2026-02-11T23:30:31.927+0000 7f416abfa640 20 osd.0 34 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011722803s,-2.011571884s]) 2026-02-11T23:30:31.927+0000 7f416cbfe640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6824/2365263801 3 ==== osd_ping(ping_reply e33 up_from 14 ping_stamp 2026-02-11T23:30:31.928227+0000/14.603632927s send_stamp 10.036931992s delta_ub -4.566700459s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a600 con 0x48c4000 2026-02-11T23:30:31.927+0000 7f416cbfe640 20 osd.0 34 get_con_osd_cluster to osd.2 from_epoch 34 2026-02-11T23:30:31.927+0000 7f416cbfe640 20 osd.0 34 get_nextmap_reserved map_reservations: {34=1} 2026-02-11T23:30:31.927+0000 7f416cbfe640 20 osd.0 34 release_map epoch: 34 2026-02-11T23:30:31.927+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6814/3043114013 4 ==== osd_ping(ping_reply e34 up_from 10 ping_stamp 2026-02-11T23:30:31.928227+0000/14.603632927s send_stamp 12.592083931s delta_ub -2.011548996s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a480 con 0x4640c00 2026-02-11T23:30:31.927+0000 7f416cbfe640 10 osd.0 34 maybe_share_map: con v2:127.0.0.1:6820/2365263801 updating session's projected_epoch from 23 to ping map epoch of 33 2026-02-11T23:30:31.927+0000 7f416cbfe640 10 osd.0 34 maybe_share_map: con v2:127.0.0.1:6820/2365263801 map epoch 33 -> 34 (shared) 2026-02-11T23:30:31.927+0000 7f416cbfe640 10 osd.0 34 send_incremental_map epoch range: (34, 34] to v2:127.0.0.1:6820/2365263801 2026-02-11T23:30:31.927+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6822/2365263801 3 ==== osd_ping(ping_reply e33 up_from 14 ping_stamp 2026-02-11T23:30:31.928227+0000/14.603632927s send_stamp 10.036949158s delta_ub -4.566683292s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a780 con 0x48cd000 2026-02-11T23:30:31.927+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6820/2365263801,v1:127.0.0.1:6821/2365263801] -- osd_map(34..34 src has 1..34) -- 0x4a64fc0 con 0x507b800 2026-02-11T23:30:31.927+0000 7f416cbfe640 20 osd.0 34 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566862106s,-4.566700459s]) 2026-02-11T23:30:31.927+0000 7f416c3fd640 20 osd.0 34 get_con_osd_cluster to osd.1 from_epoch 34 2026-02-11T23:30:31.927+0000 7f416c3fd640 20 osd.0 34 get_nextmap_reserved map_reservations: {34=1} 2026-02-11T23:30:31.927+0000 7f416c3fd640 20 osd.0 34 release_map epoch: 34 2026-02-11T23:30:31.927+0000 7f416c3fd640 10 osd.0 34 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 34 is not newer than session's projected_epoch of 34 2026-02-11T23:30:31.927+0000 7f416c3fd640 20 osd.0 34 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011737823s,-2.011548996s]) 2026-02-11T23:30:31.927+0000 7f416abfa640 20 osd.0 34 get_con_osd_cluster to osd.2 from_epoch 34 2026-02-11T23:30:31.927+0000 7f416abfa640 20 osd.0 34 get_nextmap_reserved map_reservations: {34=1} 2026-02-11T23:30:31.927+0000 7f416abfa640 20 osd.0 34 release_map epoch: 34 2026-02-11T23:30:31.927+0000 7f416abfa640 10 osd.0 34 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 34 is not newer than session's projected_epoch of 34 2026-02-11T23:30:31.927+0000 7f416abfa640 20 osd.0 34 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566888332s,-4.566683292s]) 2026-02-11T23:30:31.927+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6832/2699377081 3 ==== osd_ping(ping_reply e35 up_from 19 ping_stamp 2026-02-11T23:30:31.928227+0000/14.603632927s send_stamp 7.384299755s delta_ub -7.219333172s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a780 con 0x48cd400 2026-02-11T23:30:31.927+0000 7f416abfa640 20 osd.0 34 get_con_osd_cluster to osd.3 from_epoch 34 2026-02-11T23:30:31.927+0000 7f416abfa640 20 osd.0 34 get_nextmap_reserved map_reservations: {34=1} 2026-02-11T23:30:31.927+0000 7f416abfa640 20 osd.0 34 release_map epoch: 34 2026-02-11T23:30:31.927+0000 7f416abfa640 10 osd.0 34 maybe_share_map: con v2:127.0.0.1:6828/2699377081 updating session's projected_epoch from 34 to ping map epoch of 35 2026-02-11T23:30:31.927+0000 7f416abfa640 10 osd.0 34 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 34 is not newer than session's projected_epoch of 35 2026-02-11T23:30:31.927+0000 7f416abfa640 20 osd.0 34 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219668388s,-7.219333172s]) 2026-02-11T23:30:31.927+0000 7f415aefe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.3 v2:127.0.0.1:6828/2699377081 4 ==== osd_map(35..35 src has 1..35) ==== 296+0+0 (crc 0 0 0) 0x4a64fc0 con 0x48ccc00 2026-02-11T23:30:31.927+0000 7f415aefe640 20 osd.0 34 OSD::ms_dispatch: osd_map(35..35 src has 1..35) 2026-02-11T23:30:31.927+0000 7f415aefe640 20 osd.0 34 _dispatch 0x4a64fc0 osd_map(35..35 src has 1..35) 2026-02-11T23:30:31.927+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6830/2699377081 3 ==== osd_ping(ping_reply e35 up_from 19 ping_stamp 2026-02-11T23:30:31.928227+0000/14.603632927s send_stamp 7.384482861s delta_ub -7.219149590s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a480 con 0x48cd800 2026-02-11T23:30:31.927+0000 7f416c3fd640 20 osd.0 34 get_con_osd_cluster to osd.3 from_epoch 34 2026-02-11T23:30:31.927+0000 7f416c3fd640 20 osd.0 34 get_nextmap_reserved map_reservations: {34=1} 2026-02-11T23:30:31.927+0000 7f416c3fd640 20 osd.0 34 release_map epoch: 34 2026-02-11T23:30:31.927+0000 7f416c3fd640 10 osd.0 34 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 34 is not newer than session's projected_epoch of 35 2026-02-11T23:30:31.927+0000 7f415aefe640 3 osd.0 34 handle_osd_map epochs [35,35], i have 34, src has [1,35] 2026-02-11T23:30:31.927+0000 7f415aefe640 10 osd.0 34 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:30:31.927+0000 7f416c3fd640 20 osd.0 34 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219509125s,-7.219149590s]) 2026-02-11T23:30:31.927+0000 7f415aefe640 10 osd.0 34 handle_osd_map got inc map for epoch 35 2026-02-11T23:30:31.927+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6838/3331151400 2 ==== osd_ping(ping_reply e35 up_from 23 ping_stamp 2026-02-11T23:30:31.928227+0000/14.603632927s send_stamp 4.767150402s delta_ub -9.836481094s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a480 con 0x5282000 2026-02-11T23:30:31.927+0000 7f416c3fd640 20 osd.0 34 get_con_osd_cluster to osd.4 from_epoch 34 2026-02-11T23:30:31.927+0000 7f416c3fd640 20 osd.0 34 get_nextmap_reserved map_reservations: {34=1} 2026-02-11T23:30:31.927+0000 7f416c3fd640 20 osd.0 34 release_map epoch: 34 2026-02-11T23:30:31.927+0000 7f416c3fd640 10 osd.0 34 maybe_share_map: con v2:127.0.0.1:6836/3331151400 updating session's projected_epoch from 34 to ping map epoch of 35 2026-02-11T23:30:31.927+0000 7f416c3fd640 10 osd.0 34 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 34 is not newer than session's projected_epoch of 35 2026-02-11T23:30:31.927+0000 7f416c3fd640 20 osd.0 34 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836860657s,-9.836481094s]) 2026-02-11T23:30:31.927+0000 7f415aefe640 20 osd.0 34 got_full_map 35, nothing requested 2026-02-11T23:30:31.927+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6840/3331151400 2 ==== osd_ping(ping_reply e35 up_from 23 ping_stamp 2026-02-11T23:30:31.928227+0000/14.603632927s send_stamp 4.767350674s delta_ub -9.836281776s) ==== 2033+0+0 (crc 0 0 0) 0x4c9aa80 con 0x507bc00 2026-02-11T23:30:31.927+0000 7f416abfa640 20 osd.0 34 get_con_osd_cluster to osd.4 from_epoch 34 2026-02-11T23:30:31.927+0000 7f416abfa640 20 osd.0 34 get_nextmap_reserved map_reservations: {34=1} 2026-02-11T23:30:31.927+0000 7f416abfa640 20 osd.0 34 release_map epoch: 34 2026-02-11T23:30:31.927+0000 7f416abfa640 10 osd.0 34 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 34 is not newer than session's projected_epoch of 35 2026-02-11T23:30:31.927+0000 7f415aefe640 20 osd.0 34 trim_maps: min=0 oldest_map=1 2026-02-11T23:30:31.927+0000 7f416abfa640 20 osd.0 34 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836686134s,-9.836281776s]) 2026-02-11T23:30:31.927+0000 7f415aefe640 20 osd.0 34 handle_osd_map pg_num_history pg_num_history(e35 pg_nums {1={27=1},2={34=1}} deleted_pools ) 2026-02-11T23:30:31.927+0000 7f415aefe640 10 snap_mapper record_purged_snaps purged_snaps {35={}} 2026-02-11T23:30:31.927+0000 7f415aefe640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:31.927+0000 7f415aefe640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e35 maps [1~35] lci=[5,35] tlb=1) 2026-02-11T23:30:31.927+0000 7f415aefe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1 ==== osd_map(35..35 src has 1..35) ==== 296+0+0 (unknown 0 0 0) 0x5954700 con 0x507ac00 2026-02-11T23:30:31.927+0000 7f415aefe640 20 osd.0 34 OSD::ms_dispatch: osd_map(35..35 src has 1..35) 2026-02-11T23:30:31.927+0000 7f415aefe640 20 osd.0 34 _dispatch 0x5954700 osd_map(35..35 src has 1..35) 2026-02-11T23:30:31.927+0000 7f415aefe640 3 osd.0 34 handle_osd_map epochs [35,35], i have 35, src has [1,35] 2026-02-11T23:30:31.927+0000 7f415aefe640 10 osd.0 34 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:30:31.927+0000 7f415aefe640 10 osd.0 34 no new maps here, dropping 2026-02-11T23:30:31.927+0000 7f41591a6640 10 osd.0 34 _committed_osd_maps 35..35 2026-02-11T23:30:31.927+0000 7f41591a6640 10 osd.0 34 advance to epoch 35 (<= last 35 <= newest_map 35) 2026-02-11T23:30:31.927+0000 7f41591a6640 20 osd.0 35 consume_map version 35 2026-02-11T23:30:31.927+0000 7f41591a6640 20 osd.0 34 await_reserved_maps epoch:35 2026-02-11T23:30:31.927+0000 7f41591a6640 20 osd.0 34 await_reserved_maps done 0 2026-02-11T23:30:31.927+0000 7f41591a6640 20 osd.0 35 consume_map 35 -- publish done 2026-02-11T23:30:31.927+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:30:31.927+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:2.0slot:0x37002c0 2026-02-11T23:30:31.927+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot 2.0 has no pg and waiting_for_split 2026-02-11T23:30:31.927+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3700dc0 2026-02-11T23:30:31.927+0000 7f41591a6640 20 osd.0 35 identify_splits_and_merges 1.0 e34 to e35 2026-02-11T23:30:31.927+0000 7f41591a6640 20 osd.0 35 identify_splits_and_merges 1.0 e34 to e35 pg_nums {27=1} 2026-02-11T23:30:31.927+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:31.927+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:31.927+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:31.927+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:31.927+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:31.927+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:31.927+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:31.927+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:31.927+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:31.927+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:31.927+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:31.927+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:31.927+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:31.927+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:31.927+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:31.927+0000 7f41591a6640 20 osd.0 35 prune_pg_created 2026-02-11T23:30:31.927+0000 7f41591a6640 10 osd.0:0.consume_map 35 (was 34) 2026-02-11T23:30:31.927+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 2.0 2026-02-11T23:30:31.927+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 2.0 pending_peering first epoch 35 <= 35, requeueing 2026-02-11T23:30:31.927+0000 7f41591a6640 20 osd.0:0._wake_pg_slot _wake_pg_slot 2.0 to_process <> waiting <> waiting_peering {35=} 2026-02-11T23:30:31.927+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:31.927+0000 7f41591a6640 10 osd.0:1.consume_map 35 (was 34) 2026-02-11T23:30:31.927+0000 7f41591a6640 10 osd.0:2.consume_map 35 (was 34) 2026-02-11T23:30:31.927+0000 7f41591a6640 10 osd.0:3.consume_map 35 (was 34) 2026-02-11T23:30:31.927+0000 7f41591a6640 10 osd.0:4.consume_map 35 (was 34) 2026-02-11T23:30:31.927+0000 7f41591a6640 10 osd.0:5.consume_map 35 (was 34) 2026-02-11T23:30:31.927+0000 7f41591a6640 10 osd.0:6.consume_map 35 (was 34) 2026-02-11T23:30:31.927+0000 7f41591a6640 10 osd.0:7.consume_map 35 (was 34) 2026-02-11T23:30:31.927+0000 7f41591a6640 10 osd.0 35 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:31.927+0000 7f41591a6640 15 osd.0 35 enqueue_peering_evt 1.0 epoch_sent: 35 epoch_requested: 35 NullEvt 2026-02-11T23:30:31.927+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 35 epoch_requested: 35 NullEvt) class_id 2 prio 255 cost 10 e35) 2026-02-11T23:30:31.927+0000 7f414b98b640 20 osd.0 op_wq(0) _process 2.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:31.927+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 35 epoch_requested: 34 MLogRec from 3 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 22.947696686s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e35) queued 2026-02-11T23:30:31.927+0000 7f414b98b640 20 osd.0 op_wq(0) _process 2.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:31.927+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 35 epoch_requested: 34 MLogRec from 3 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 22.947696686s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e35) pg 0 2026-02-11T23:30:31.927+0000 7f414b98b640 20 osd.0 op_wq(0) _process 2.0 no pg, should create on OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 35 epoch_requested: 34 MLogRec from 3 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 22.947696686s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e35) 2026-02-11T23:30:31.927+0000 7f414b98b640 10 osd.0 35 _make_pg 2.0 2026-02-11T23:30:31.927+0000 7f414b98b640 10 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:2, shard:-1, shard_prefix: , prefixes: {}] SnapMapper 2026-02-11T23:30:31.927+0000 7f414b98b640 20 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:2, shard:-1, shard_prefix: , prefixes: {}] update_bits new_bits: 0 2026-02-11T23:30:31.927+0000 7f414b98b640 20 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:2, shard:-1, shard_prefix: , prefixes: {"0000000000000002."}] update_bits prefix updated 2026-02-11T23:30:31.927+0000 7f414b98b640 10 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:2, shard:-1, shard_prefix: , prefixes: {"0000000000000002."}] reset_prefix_itr::from <0> to ::update_bits 2026-02-11T23:30:31.927+0000 7f414b98b640 5 osd.0 pg_epoch: 34 pg[2.0(unlocked)] enter Initial 2026-02-11T23:30:31.927+0000 7f414b98b640 20 osd.0 pg_epoch: 34 pg[2.0(unlocked)] enter NotTrimming 2026-02-11T23:30:31.927+0000 7f414b98b640 10 osd.0 pg_epoch: 34 pg[2.0(unlocked)] scrubber: FSM: -- state -->> NotActive 2026-02-11T23:30:31.927+0000 7f414b98b640 10 osd.0 pg_epoch: 34 pg[2.0( DNE empty local-lis/les=0/0 n=0 ec=0/0 lis/c=0/0 les/c/f=0/0/0 sis=0) [] r=-1 lpr=0 crt=0'0 unknown mbc={}] PeeringState::init init role 2 up [3,1,0] acting [3,1,0] history ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34 past_intervals ([0,0] all_participants={} intervals=[]) 2026-02-11T23:30:31.927+0000 7f414b98b640 20 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=0/0 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=0 crt=0'0 unknown mbc={}] PeeringState::on_new_interval 2026-02-11T23:30:31.927+0000 7f414b98b640 20 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=0/0 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=0 crt=0'0 unknown mbc={}] PeeringState::on_new_interval upacting_features 0x3f0ffffffffdffff from [3,1,0]+[3,1,0] 2026-02-11T23:30:31.927+0000 7f414b98b640 20 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=0/0 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=0 crt=0'0 unknown mbc={}] PeeringState::on_new_interval checking missing set deletes flag. missing = missing(0 may_include_deletes = 1) 2026-02-11T23:30:31.927+0000 7f414b98b640 10 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=0/0 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=0 crt=0'0 unknown mbc={}] PeeringState::init_hb_stamps now [0x56858f0] 2026-02-11T23:30:31.927+0000 7f414b98b640 10 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=0/0 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=0 crt=0'0 unknown mbc={}] PeeringState::on_new_interval prior_readable_until_ub 0.000000000s (mnow 14.604547501s + 0.000000000s) 2026-02-11T23:30:31.927+0000 7f414b98b640 10 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=0/0 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=0 crt=0'0 unknown mbc={}] cancel_recovery 2026-02-11T23:30:31.927+0000 7f414b98b640 10 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=0/0 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=0 crt=0'0 unknown mbc={}] clear_recovery_state 2026-02-11T23:30:31.927+0000 7f414b98b640 15 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=0/0 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=0 crt=0'0 unknown mbc={}] _clear_recovery_state 2026-02-11T23:30:31.927+0000 7f414b98b640 10 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=0/0 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=0 crt=0'0 unknown mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:31.927+0000 7f414b98b640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:31.927+0000 7f414b98b640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:31.927+0000 7f414b98b640 6 write_log_and_missing with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, writeout_from: 4294967295'18446744073709551615, trimmed: , trimmed_dups: , clear_divergent_priors: 0 2026-02-11T23:30:31.927+0000 7f414b98b640 10 _write_log_and_missing clearing up to 0'0 dirty_to_dups=0'0 dirty_from_dups=4294967295'18446744073709551615 write_from_dups=4294967295'18446744073709551615 trimmed_dups.size()=0 2026-02-11T23:30:31.927+0000 7f414b98b640 10 _write_log_and_missing going to encode log.dups.size()=0 2026-02-11T23:30:31.927+0000 7f414b98b640 10 _write_log_and_missing 1st round encoded log.dups.size()=0 2026-02-11T23:30:31.927+0000 7f414b98b640 10 _write_log_and_missing 2st round encoded log.dups.size()=0 2026-02-11T23:30:31.927+0000 7f414b98b640 10 end of _write_log_and_missing 2026-02-11T23:30:31.927+0000 7f414b98b640 10 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=0/0 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=0 crt=0'0 unknown mbc={}] handle_initialize 2026-02-11T23:30:31.927+0000 7f414b98b640 5 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=0/0 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=0 crt=0'0 unknown mbc={}] exit Initial 0.000086 0 0.000000 2026-02-11T23:30:31.927+0000 7f414b98b640 5 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=0/0 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=0 crt=0'0 unknown mbc={}] enter Reset 2026-02-11T23:30:31.927+0000 7f414b98b640 20 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=0/0 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=0 crt=0'0 unknown mbc={}] PeeringState::set_last_peering_reset set_last_peering_reset 34 2026-02-11T23:30:31.927+0000 7f414b98b640 10 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=0/0 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 unknown mbc={}] PeeringState::set_last_peering_reset Clearing blocked outgoing recovery messages 2026-02-11T23:30:31.927+0000 7f414b98b640 10 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=0/0 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 unknown mbc={}] PeeringState::set_last_peering_reset Not blocking outgoing recovery messages 2026-02-11T23:30:31.927+0000 7f414b98b640 10 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=0/0 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 unknown mbc={}] handle_activate_map: epoch range: 34..34 2026-02-11T23:30:31.927+0000 7f414b98b640 10 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=0/0 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 unknown mbc={}] PeeringState::activate_map 2026-02-11T23:30:31.927+0000 7f414b98b640 5 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=0/0 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 unknown mbc={}] exit Reset 0.000017 1 0.000016 2026-02-11T23:30:31.927+0000 7f414b98b640 5 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=0/0 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 unknown mbc={}] enter Started 2026-02-11T23:30:31.927+0000 7f414b98b640 5 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=0/0 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 unknown mbc={}] enter Start 2026-02-11T23:30:31.927+0000 7f414b98b640 1 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=0/0 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 unknown mbc={}] state: transitioning to Stray 2026-02-11T23:30:31.927+0000 7f414b98b640 5 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=0/0 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 unknown mbc={}] exit Start 0.000003 0 0.000000 2026-02-11T23:30:31.927+0000 7f414b98b640 5 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=0/0 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 unknown mbc={}] enter Started/Stray 2026-02-11T23:30:31.927+0000 7f414b98b640 20 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=0/0 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 unknown mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 34 while current is 34 2026-02-11T23:30:31.927+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:31.927+0000 7f414b98b640 10 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=0/0 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 unknown mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:30:31.927+0000 7f414b98b640 10 osd.0 35 handle_pg_create_info new pg pg[2.0( empty local-lis/les=0/0 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 unknown mbc={}] 2026-02-11T23:30:31.927+0000 7f414b98b640 10 osd.0:0._attach_pg 2.0 0x5978000 2026-02-11T23:30:31.927+0000 7f414b98b640 20 osd.0:0._wake_pg_slot _wake_pg_slot 2.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:31.927+0000 7f414b98b640 20 osd.0 35 identify_splits_and_merges 2.0 e34 to e35 2026-02-11T23:30:31.927+0000 7f414b98b640 20 osd.0 35 identify_splits_and_merges 2.0 e34 to e35 pg_nums {34=1} 2026-02-11T23:30:31.927+0000 7f414b98b640 10 osd.0:0._prime_splits 2026-02-11T23:30:31.927+0000 7f414b98b640 10 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=0/0 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 unknown mbc={}] handle_advance_map: 35 2026-02-11T23:30:31.927+0000 7f41591a6640 7 osd.0 35 activate_map version 35 2026-02-11T23:30:31.927+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 2.0 34 -> 35 2026-02-11T23:30:31.927+0000 7f414b98b640 10 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=0/0 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 unknown mbc={}] PeeringState::advance_map handle_advance_map [3,1,0]/[3,1,0] -- 3/3 2026-02-11T23:30:31.927+0000 7f414b98b640 10 osd.0 pg_epoch: 35 pg[2.0( empty local-lis/les=0/0 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 unknown mbc={}] state: Started advmap 2026-02-11T23:30:31.927+0000 7f4150194640 20 osd.0 35 agent_entry empty queue 2026-02-11T23:30:31.927+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:31.927+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 35 epoch_requested: 35 NullEvt) class_id 2immediate prio 255 cost 10 e35) queued 2026-02-11T23:30:31.927+0000 7f414b98b640 10 osd.0 pg_epoch: 35 pg[2.0( empty local-lis/les=0/0 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 unknown mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:31.927+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:31.927+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 35 epoch_requested: 35 NullEvt) class_id 2immediate prio 255 cost 10 e35) pg 0x5946000 2026-02-11T23:30:31.927+0000 7f414b98b640 20 osd.0 35 advance_pg new pool opts old pool opts 2026-02-11T23:30:31.927+0000 7f414b98b640 10 osd.0 pg_epoch: 35 pg[2.0( empty local-lis/les=0/0 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 unknown mbc={}] handle_activate_map: epoch range: 35..35 2026-02-11T23:30:31.927+0000 7f414b98b640 10 osd.0 pg_epoch: 35 pg[2.0( empty local-lis/les=0/0 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 unknown mbc={}] PeeringState::activate_map 2026-02-11T23:30:31.927+0000 7f414b98b640 20 osd.0 pg_epoch: 35 pg[2.0( empty local-lis/les=0/0 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 unknown mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 34 while current is 35 2026-02-11T23:30:31.927+0000 7f414f993640 10 osd.0 pg_epoch: 34 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 35 2026-02-11T23:30:31.927+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 34 -> 35 2026-02-11T23:30:31.927+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:31.927+0000 7f414f993640 10 osd.0 pg_epoch: 34 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:31.927+0000 7f414b98b640 10 osd.0 pg_epoch: 35 pg[2.0( empty local-lis/les=0/0 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 unknown mbc={}] do_peering_event: epoch_sent: 35 epoch_requested: 34 MLogRec from 3 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 22.947696686s int 16.000000000s) +create_info 2026-02-11T23:30:31.927+0000 7f414f993640 10 osd.0 pg_epoch: 35 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:31.927+0000 7f414b98b640 10 osd.0 pg_epoch: 35 pg[2.0( empty local-lis/les=0/0 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 unknown mbc={}] state: got info+log from osd.3 2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) log((0'0,0'0], crt=0'0) 2026-02-11T23:30:31.927+0000 7f414f993640 10 osd.0 pg_epoch: 35 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:31.927+0000 7f414b98b640 10 merge_log log((0'0,0'0], crt=0'0) from osd.3 into log((0'0,0'0], crt=0'0) 2026-02-11T23:30:31.927+0000 7f414b98b640 5 merge_log_dups log.dups.size()=0olog.dups.size()=0 2026-02-11T23:30:31.927+0000 7f414f993640 20 osd.0 35 advance_pg new pool opts old pool opts 2026-02-11T23:30:31.927+0000 7f414b98b640 5 end of merge_log_dups changed=0 log.dups.size()=0 olog.dups.size()=0 2026-02-11T23:30:31.927+0000 7f414b98b640 10 merge_log result log((0'0,0'0], crt=0'0) missing(0 may_include_deletes = 1) changed=0 2026-02-11T23:30:31.927+0000 7f414f993640 10 osd.0 pg_epoch: 35 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_activate_map: epoch range: 35..35 2026-02-11T23:30:31.927+0000 7f414f993640 10 osd.0 pg_epoch: 35 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:31.927+0000 7f414b98b640 10 osd.0 pg_epoch: 35 pg[2.0( empty local-lis/les=0/0 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 unknown mbc={}] PeeringState::proc_lease pg_lease(ru 0.000000000s ub 22.947696686s int 16.000000000s) 2026-02-11T23:30:31.927+0000 7f414f993640 20 osd.0 pg_epoch: 35 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 30 while current is 35 2026-02-11T23:30:31.927+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:31.927+0000 7f414b98b640 20 osd.0 pg_epoch: 35 pg[2.0( empty local-lis/les=0/0 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 unknown mbc={}] PeeringState::proc_lease peer_clock_delta_lb -7.219509125s -> ruub 30.167205811s 2026-02-11T23:30:31.927+0000 7f414f993640 10 osd.0 pg_epoch: 35 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 35 epoch_requested: 35 NullEvt 2026-02-11T23:30:31.927+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:31.927+0000 7f414b98b640 20 osd.0 pg_epoch: 35 pg[2.0( empty local-lis/les=0/0 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 unknown mbc={}] PeeringState::proc_lease readable_until_ub now 30.167205811s 2026-02-11T23:30:31.927+0000 7f414b98b640 5 osd.0 pg_epoch: 35 pg[2.0( empty local-lis/les=0/0 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 unknown mbc={}] exit Started/Stray 0.000088 3 0.000018 2026-02-11T23:30:31.927+0000 7f414b98b640 5 osd.0 pg_epoch: 35 pg[2.0( empty local-lis/les=0/0 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 unknown mbc={}] enter Started/ReplicaActive 2026-02-11T23:30:31.927+0000 7f414b98b640 5 osd.0 pg_epoch: 35 pg[2.0( empty local-lis/les=0/0 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 unknown mbc={}] enter Started/ReplicaActive/RepNotRecovering 2026-02-11T23:30:31.927+0000 7f414b98b640 10 osd.0 pg_epoch: 35 pg[2.0( empty local-lis/les=0/0 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 unknown mbc={}] state: In ReplicaActive, about to call activate 2026-02-11T23:30:31.927+0000 7f414b98b640 10 osd.0 pg_epoch: 35 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 unknown mbc={}] PeeringState::activate activate - no missing, moving last_complete 0'0 -> 0'0 2026-02-11T23:30:31.928+0000 7f414b98b640 10 osd.0 pg_epoch: 35 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 unknown mbc={}] scrubber: on_replica_activate 2026-02-11T23:30:31.928+0000 7f414b98b640 20 scrubberFSM event: --vvvv---- ReplicaActivate 2026-02-11T23:30:31.928+0000 7f414b98b640 10 osd.0 pg_epoch: 35 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 unknown mbc={}] scrubber: FSM: -- state -->> ReplicaActive 2026-02-11T23:30:31.928+0000 7f414b98b640 10 osd.0 pg_epoch: 35 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 unknown mbc={}] scrubber: FSM: -- state -->> ReplicaActive/ReplicaIdle 2026-02-11T23:30:31.928+0000 7f414b98b640 20 scrubberFSM event: --^^^^---- ReplicaActivate 2026-02-11T23:30:31.928+0000 7f414b98b640 10 osd.0 pg_epoch: 35 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 unknown mbc={}] state: Activate Finished 2026-02-11T23:30:31.928+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:31.928+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:31.928+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:31.928+0000 7f414f993640 15 osd.0 35 enqueue_peering_evt 2.0 epoch_sent: 35 epoch_requested: 35 ActivateCommitted from 35 processed at 35 2026-02-11T23:30:31.928+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:31.928+0000 7f414f993640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 35 epoch_requested: 35 ActivateCommitted from 35 processed at 35) class_id 2 prio 255 cost 10 e35) 2026-02-11T23:30:31.928+0000 7f414f993640 20 osd.0 op_wq(0) _process 2.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:31.928+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 35 epoch_requested: 35 ActivateCommitted from 35 processed at 35) class_id 2immediate prio 255 cost 10 e35) queued 2026-02-11T23:30:31.928+0000 7f414f993640 20 osd.0 op_wq(0) _process 2.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:31.928+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 35 epoch_requested: 35 ActivateCommitted from 35 processed at 35) class_id 2immediate prio 255 cost 10 e35) pg 0x5978000 2026-02-11T23:30:31.928+0000 7f414f993640 10 osd.0 pg_epoch: 35 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 unknown mbc={}] do_peering_event: epoch_sent: 35 epoch_requested: 35 ActivateCommitted from 35 processed at 35 2026-02-11T23:30:31.928+0000 7f414f993640 10 osd.0 pg_epoch: 35 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 unknown mbc={}] state: react 35 telling primary 2026-02-11T23:30:31.928+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:31.928+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:31.928+0000 7f414f993640 20 osd.0 35 get_con_osd_cluster to osd.3 from_epoch 35 2026-02-11T23:30:31.928+0000 7f414f993640 20 osd.0 35 get_nextmap_reserved map_reservations: {35=1} 2026-02-11T23:30:31.928+0000 7f414f993640 20 osd.0 35 release_map epoch: 35 2026-02-11T23:30:31.928+0000 7f414f993640 10 osd.0 35 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 35 is not newer than session's projected_epoch of 35 2026-02-11T23:30:31.928+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6828/2699377081,v1:127.0.0.1:6829/2699377081] -- pg_info2(2.0 2.0 2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/0 les/c/f=35/0/0 sis=34) e35/35) -- 0x373b100 con 0x48ccc00 2026-02-11T23:30:31.928+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:31.938+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.1 127.0.0.1:0/3043114013 6 ==== osd_ping(ping e34 up_from 10 ping_stamp 2026-02-11T23:30:31.939941+0000/12.603687286s send_stamp 12.603687286s delta_ub -2.011548996s) ==== 2033+0+0 (crc 0 0 0) 0x4c9ad80 con 0x4aca000 2026-02-11T23:30:31.938+0000 7f416cbfe640 20 osd.0 35 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011709929s,-2.011548996s]) 2026-02-11T23:30:31.938+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.1 127.0.0.1:0/3043114013 6 ==== osd_ping(ping e34 up_from 10 ping_stamp 2026-02-11T23:30:31.939941+0000/12.603687286s send_stamp 12.603687286s delta_ub -2.011548996s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a480 con 0x4aca400 2026-02-11T23:30:31.938+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e35 up_from 5 ping_stamp 2026-02-11T23:30:31.939941+0000/12.603687286s send_stamp 14.615396500s delta_ub 2.011709929s) -- 0x4c9ac00 con 0x4aca000 2026-02-11T23:30:31.938+0000 7f416cbfe640 20 osd.0 35 get_con_osd_cluster to osd.1 from_epoch 35 2026-02-11T23:30:31.938+0000 7f416cbfe640 20 osd.0 35 get_nextmap_reserved map_reservations: {35=1} 2026-02-11T23:30:31.938+0000 7f416cbfe640 20 osd.0 35 release_map epoch: 35 2026-02-11T23:30:31.938+0000 7f416cbfe640 10 osd.0 35 maybe_share_map: con v2:127.0.0.1:6812/3043114013 map epoch 34 -> 35 (shared) 2026-02-11T23:30:31.938+0000 7f416cbfe640 10 osd.0 35 send_incremental_map epoch range: (35, 35] to v2:127.0.0.1:6812/3043114013 2026-02-11T23:30:31.938+0000 7f416cbfe640 10 osd.0 35 add_map_inc_bl 35 252 bytes 2026-02-11T23:30:31.938+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- osd_map(35..35 src has 1..35) -- 0x4a31500 con 0x4acb000 2026-02-11T23:30:31.938+0000 7f416c3fd640 20 osd.0 35 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011801004s,-2.011548996s]) 2026-02-11T23:30:31.938+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e35 up_from 5 ping_stamp 2026-02-11T23:30:31.939941+0000/12.603687286s send_stamp 14.615487099s delta_ub 2.011801004s) -- 0x4c9b200 con 0x4aca400 2026-02-11T23:30:31.938+0000 7f416c3fd640 20 osd.0 35 get_con_osd_cluster to osd.1 from_epoch 35 2026-02-11T23:30:31.938+0000 7f416c3fd640 20 osd.0 35 get_nextmap_reserved map_reservations: {35=1} 2026-02-11T23:30:31.938+0000 7f416c3fd640 20 osd.0 35 release_map epoch: 35 2026-02-11T23:30:31.938+0000 7f416c3fd640 10 osd.0 35 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 35 is not newer than session's projected_epoch of 35 2026-02-11T23:30:31.939+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.3 v2:127.0.0.1:6828/2699377081 5 ==== pg_lease(2.0 pg_lease(ru 22.947696686s ub 23.397006989s int 16.000000000s) e35/35) ==== 58+0+0 (crc 0 0 0) 0x4c9ac00 con 0x48ccc00 2026-02-11T23:30:31.939+0000 7f416cbfe640 15 osd.0 35 enqueue_peering_evt 2.0 epoch_sent: 35 epoch_requested: 35 MLease epoch 35 from osd.3 pg_lease(ru 22.947696686s ub 23.397006989s int 16.000000000s) 2026-02-11T23:30:31.939+0000 7f416cbfe640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 35 epoch_requested: 35 MLease epoch 35 from osd.3 pg_lease(ru 22.947696686s ub 23.397006989s int 16.000000000s)) class_id 2 prio 255 cost 10 e35) 2026-02-11T23:30:31.939+0000 7f414b98b640 20 osd.0 op_wq(0) _process 2.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:31.939+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 35 epoch_requested: 35 MLease epoch 35 from osd.3 pg_lease(ru 22.947696686s ub 23.397006989s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e35) queued 2026-02-11T23:30:31.939+0000 7f414b98b640 20 osd.0 op_wq(0) _process 2.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:31.939+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 35 epoch_requested: 35 MLease epoch 35 from osd.3 pg_lease(ru 22.947696686s ub 23.397006989s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e35) pg 0x5978000 2026-02-11T23:30:31.939+0000 7f414b98b640 10 osd.0 pg_epoch: 35 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 35 epoch_requested: 35 MLease epoch 35 from osd.3 pg_lease(ru 22.947696686s ub 23.397006989s int 16.000000000s) 2026-02-11T23:30:31.939+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:31.939+0000 7f414b98b640 10 osd.0 pg_epoch: 35 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 active mbc={}] PeeringState::proc_lease pg_lease(ru 22.947696686s ub 23.397006989s int 16.000000000s) 2026-02-11T23:30:31.939+0000 7f414b98b640 20 osd.0 pg_epoch: 35 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 active mbc={}] PeeringState::proc_lease peer_clock_delta_ub -7.219149590s -> ru 30.166847229s 2026-02-11T23:30:31.939+0000 7f414b98b640 20 osd.0 pg_epoch: 35 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 active mbc={}] PeeringState::proc_lease readable_until now 30.166847229s 2026-02-11T23:30:31.939+0000 7f414b98b640 20 osd.0 pg_epoch: 35 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 active mbc={}] PeeringState::proc_lease peer_clock_delta_lb -7.219509125s -> ruub 30.616516113s 2026-02-11T23:30:31.939+0000 7f414b98b640 20 osd.0 pg_epoch: 35 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 active mbc={}] PeeringState::proc_lease readable_until_ub now 30.616516113s 2026-02-11T23:30:31.939+0000 7f414b98b640 20 osd.0 35 get_con_osd_cluster to osd.3 from_epoch 35 2026-02-11T23:30:31.939+0000 7f414b98b640 20 osd.0 35 get_nextmap_reserved map_reservations: {35=1} 2026-02-11T23:30:31.939+0000 7f414b98b640 20 osd.0 35 release_map epoch: 35 2026-02-11T23:30:31.939+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.3 v2:127.0.0.1:6828/2699377081 6 ==== pg_info2(2.0 2.0 2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/0 les/c/f=35/0/0 sis=34) e35/35) ==== 1126+0+0 (crc 0 0 0) 0x373b100 con 0x48ccc00 2026-02-11T23:30:31.939+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6828/2699377081,v1:127.0.0.1:6829/2699377081] -- pg_lease_ack(2.0 pg_lease_ack(ruub 23.397006989s) e35/35) -- 0x3758c60 con 0x48ccc00 2026-02-11T23:30:31.939+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:31.939+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:31.939+0000 7f416cbfe640 15 osd.0 35 enqueue_peering_evt 2.0 epoch_sent: 35 epoch_requested: 35 MInfoRec from 3 info: 2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/0 les/c/f=35/0/0 sis=34) pg_lease(ru 22.947696686s ub 23.397006989s int 16.000000000s) 2026-02-11T23:30:31.939+0000 7f416cbfe640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 35 epoch_requested: 35 MInfoRec from 3 info: 2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/0 les/c/f=35/0/0 sis=34) pg_lease(ru 22.947696686s ub 23.397006989s int 16.000000000s)) class_id 2 prio 255 cost 10 e35) 2026-02-11T23:30:31.939+0000 7f414f993640 20 osd.0 op_wq(0) _process 2.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:31.939+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 35 epoch_requested: 35 MInfoRec from 3 info: 2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/0 les/c/f=35/0/0 sis=34) pg_lease(ru 22.947696686s ub 23.397006989s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e35) queued 2026-02-11T23:30:31.939+0000 7f414f993640 20 osd.0 op_wq(0) _process 2.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:31.939+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 35 epoch_requested: 35 MInfoRec from 3 info: 2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/0 les/c/f=35/0/0 sis=34) pg_lease(ru 22.947696686s ub 23.397006989s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e35) pg 0x5978000 2026-02-11T23:30:31.939+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:31.939+0000 7f414f993640 10 osd.0 pg_epoch: 35 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=0/0 les/c/f=0/0/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 35 epoch_requested: 35 MInfoRec from 3 info: 2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/0 les/c/f=35/0/0 sis=34) pg_lease(ru 22.947696686s ub 23.397006989s int 16.000000000s) 2026-02-11T23:30:31.939+0000 7f414f993640 20 osd.0 pg_epoch: 35 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/0 les/c/f=35/0/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 active mbc={}] PeeringState::update_history advanced history from ec=34/34 lis/c=34/0 les/c/f=35/0/0 sis=34 2026-02-11T23:30:31.939+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:31.939+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:31.939+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.3 v2:127.0.0.1:6828/2699377081 7 ==== pg_info2(2.0 2.0 2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=34) e35/35) ==== 1126+0+0 (crc 0 0 0) 0x373b100 con 0x48ccc00 2026-02-11T23:30:31.939+0000 7f416cbfe640 15 osd.0 35 enqueue_peering_evt 2.0 epoch_sent: 35 epoch_requested: 35 MInfoRec from 3 info: 2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=34) pg_lease(ru 22.947696686s ub 23.397006989s int 16.000000000s) 2026-02-11T23:30:31.939+0000 7f416cbfe640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 35 epoch_requested: 35 MInfoRec from 3 info: 2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=34) pg_lease(ru 22.947696686s ub 23.397006989s int 16.000000000s)) class_id 2 prio 255 cost 10 e35) 2026-02-11T23:30:31.939+0000 7f414b98b640 20 osd.0 op_wq(0) _process 2.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:31.939+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 35 epoch_requested: 35 MInfoRec from 3 info: 2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=34) pg_lease(ru 22.947696686s ub 23.397006989s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e35) queued 2026-02-11T23:30:31.939+0000 7f414b98b640 20 osd.0 op_wq(0) _process 2.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:31.939+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 35 epoch_requested: 35 MInfoRec from 3 info: 2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=34) pg_lease(ru 22.947696686s ub 23.397006989s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e35) pg 0x5978000 2026-02-11T23:30:31.940+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:31.939+0000 7f414b98b640 10 osd.0 pg_epoch: 35 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/0 les/c/f=35/0/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 35 epoch_requested: 35 MInfoRec from 3 info: 2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=34) pg_lease(ru 22.947696686s ub 23.397006989s int 16.000000000s) 2026-02-11T23:30:31.940+0000 7f414b98b640 20 osd.0 pg_epoch: 35 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 active mbc={}] PeeringState::update_history advanced history from ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=34 2026-02-11T23:30:31.940+0000 7f414b98b640 20 osd.0 pg_epoch: 35 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 active mbc={}] PeeringState::update_history clearing past_intervals 2026-02-11T23:30:31.940+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:31.940+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:31.940+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:31.940+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:31.940+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:31.940+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:32.195+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.3 v2:127.0.0.1:6828/2699377081 8 ==== pg_query2(2.0 2.0 query(info 0'0 epoch_sent 38) e38/38) ==== 158+0+0 (crc 0 0 0) 0x5968000 con 0x48ccc00 2026-02-11T23:30:32.195+0000 7f416cbfe640 15 osd.0 35 enqueue_peering_evt 2.0 epoch_sent: 38 epoch_requested: 38 MQuery 2.0 from 3 query_epoch 38 query: query(info 0'0 epoch_sent 38) 2026-02-11T23:30:32.195+0000 7f416cbfe640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 38 epoch_requested: 38 MQuery 2.0 from 3 query_epoch 38 query: query(info 0'0 epoch_sent 38)) class_id 2 prio 255 cost 10 e38) 2026-02-11T23:30:32.195+0000 7f414f993640 20 osd.0 op_wq(0) _process 2.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:32.195+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 38 epoch_requested: 38 MQuery 2.0 from 3 query_epoch 38 query: query(info 0'0 epoch_sent 38)) class_id 2immediate prio 255 cost 10 e38) queued 2026-02-11T23:30:32.195+0000 7f414f993640 20 osd.0 op_wq(0) _process 2.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:32.195+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 38 epoch_requested: 38 MQuery 2.0 from 3 query_epoch 38 query: query(info 0'0 epoch_sent 38)) class_id 2immediate prio 255 cost 10 e38) pg 0x5978000 2026-02-11T23:30:32.195+0000 7f414f993640 20 osd.0 op_wq _add_slot_waiter 2.0 peering, item epoch is 38, will wait on OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 38 epoch_requested: 38 MQuery 2.0 from 3 query_epoch 38 query: query(info 0'0 epoch_sent 38)) class_id 2immediate prio 255 cost 10 e38) 2026-02-11T23:30:32.195+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:32.195+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:32.195+0000 7f415aefe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.3 v2:127.0.0.1:6828/2699377081 9 ==== osd_map(36..38 src has 1..38) ==== 1782+0+0 (crc 0 0 0) 0x4a316c0 con 0x48ccc00 2026-02-11T23:30:32.195+0000 7f415aefe640 20 osd.0 35 OSD::ms_dispatch: osd_map(36..38 src has 1..38) 2026-02-11T23:30:32.195+0000 7f415aefe640 20 osd.0 35 _dispatch 0x4a316c0 osd_map(36..38 src has 1..38) 2026-02-11T23:30:32.195+0000 7f415aefe640 3 osd.0 35 handle_osd_map epochs [36,38], i have 35, src has [1,38] 2026-02-11T23:30:32.195+0000 7f415aefe640 10 osd.0 35 handle_osd_map: received 0 full maps and 3 incremental maps 2026-02-11T23:30:32.195+0000 7f415aefe640 10 osd.0 35 handle_osd_map got inc map for epoch 36 2026-02-11T23:30:32.195+0000 7f415aefe640 20 osd.0 35 got_full_map 36, nothing requested 2026-02-11T23:30:32.195+0000 7f415aefe640 10 osd.0 35 handle_osd_map got inc map for epoch 37 2026-02-11T23:30:32.195+0000 7f415aefe640 20 osd.0 35 got_full_map 37, nothing requested 2026-02-11T23:30:32.195+0000 7f415aefe640 10 osd.0 35 handle_osd_map got inc map for epoch 38 2026-02-11T23:30:32.195+0000 7f415aefe640 20 osd.0 35 got_full_map 38, nothing requested 2026-02-11T23:30:32.195+0000 7f415aefe640 20 osd.0 35 trim_maps: min=0 oldest_map=1 2026-02-11T23:30:32.195+0000 7f415aefe640 20 osd.0 35 handle_osd_map pg_num_history pg_num_history(e38 pg_nums {1={27=1},2={34=1}} deleted_pools ) 2026-02-11T23:30:32.195+0000 7f415aefe640 10 snap_mapper record_purged_snaps purged_snaps {36={},37={},38={}} 2026-02-11T23:30:32.195+0000 7f415aefe640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:32.195+0000 7f415aefe640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e38 maps [1~38] lci=[5,38] tlb=1) 2026-02-11T23:30:32.196+0000 7f41591a6640 10 osd.0 35 _committed_osd_maps 36..38 2026-02-11T23:30:32.196+0000 7f41591a6640 10 osd.0 35 advance to epoch 36 (<= last 38 <= newest_map 38) 2026-02-11T23:30:32.196+0000 7f41591a6640 10 osd.0 36 advance to epoch 37 (<= last 38 <= newest_map 38) 2026-02-11T23:30:32.196+0000 7f41591a6640 10 osd.0 37 advance to epoch 38 (<= last 38 <= newest_map 38) 2026-02-11T23:30:32.196+0000 7f41591a6640 20 osd.0 38 consume_map version 38 2026-02-11T23:30:32.196+0000 7f41591a6640 20 osd.0 35 await_reserved_maps epoch:38 2026-02-11T23:30:32.196+0000 7f41591a6640 20 osd.0 35 await_reserved_maps done 0 2026-02-11T23:30:32.196+0000 7f41591a6640 20 osd.0 38 consume_map 38 -- publish done 2026-02-11T23:30:32.196+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:30:32.196+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:2.0slot:0x37002c0 2026-02-11T23:30:32.196+0000 7f41591a6640 20 osd.0 38 identify_splits_and_merges 2.0 e35 to e38 2026-02-11T23:30:32.196+0000 7f41591a6640 20 osd.0 38 identify_splits_and_merges 2.0 e35 to e38 pg_nums {34=1} 2026-02-11T23:30:32.196+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3700dc0 2026-02-11T23:30:32.196+0000 7f41591a6640 20 osd.0 38 identify_splits_and_merges 1.0 e35 to e38 2026-02-11T23:30:32.196+0000 7f41591a6640 20 osd.0 38 identify_splits_and_merges 1.0 e35 to e38 pg_nums {27=1} 2026-02-11T23:30:32.196+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:32.196+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:32.196+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:32.196+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:32.196+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:32.196+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:32.196+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:32.196+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:32.196+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:32.196+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:32.196+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:32.196+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:32.196+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:32.196+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:32.196+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:32.196+0000 7f41591a6640 20 osd.0 38 prune_pg_created 2026-02-11T23:30:32.196+0000 7f41591a6640 10 osd.0:0.consume_map 38 (was 35) 2026-02-11T23:30:32.196+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 2.0 2026-02-11T23:30:32.196+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 2.0 pending_peering first epoch 38 <= 38, requeueing 2026-02-11T23:30:32.196+0000 7f41591a6640 20 osd.0:0._wake_pg_slot _wake_pg_slot 2.0 to_process <> waiting <> waiting_peering {38=} 2026-02-11T23:30:32.196+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:32.196+0000 7f41591a6640 10 osd.0:1.consume_map 38 (was 35) 2026-02-11T23:30:32.196+0000 7f41591a6640 10 osd.0:2.consume_map 38 (was 35) 2026-02-11T23:30:32.196+0000 7f41591a6640 10 osd.0:3.consume_map 38 (was 35) 2026-02-11T23:30:32.196+0000 7f41591a6640 10 osd.0:4.consume_map 38 (was 35) 2026-02-11T23:30:32.196+0000 7f41591a6640 10 osd.0:5.consume_map 38 (was 35) 2026-02-11T23:30:32.196+0000 7f41591a6640 10 osd.0:6.consume_map 38 (was 35) 2026-02-11T23:30:32.196+0000 7f41591a6640 10 osd.0:7.consume_map 38 (was 35) 2026-02-11T23:30:32.196+0000 7f414f993640 20 osd.0 op_wq(0) _process 2.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:32.196+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 38 epoch_requested: 38 MQuery 2.0 from 3 query_epoch 38 query: query(info 0'0 epoch_sent 38)) class_id 2immediate prio 255 cost 10 e38) queued 2026-02-11T23:30:32.196+0000 7f414f993640 20 osd.0 op_wq(0) _process 2.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:32.196+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 38 epoch_requested: 38 MQuery 2.0 from 3 query_epoch 38 query: query(info 0'0 epoch_sent 38)) class_id 2immediate prio 255 cost 10 e38) pg 0x5978000 2026-02-11T23:30:32.196+0000 7f41591a6640 10 osd.0 38 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:32.196+0000 7f41591a6640 15 osd.0 38 enqueue_peering_evt 2.0 epoch_sent: 38 epoch_requested: 38 NullEvt 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 35 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 active mbc={}] handle_advance_map: 36 2026-02-11T23:30:32.196+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 38 epoch_requested: 38 NullEvt) class_id 2 prio 255 cost 10 e38) 2026-02-11T23:30:32.196+0000 7f414f993640 20 osd.0:0.update_pg_epoch 2.0 35 -> 36 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 35 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [3,1,0]/[3,1,0] -- 3/3 2026-02-11T23:30:32.196+0000 7f41591a6640 15 osd.0 38 enqueue_peering_evt 1.0 epoch_sent: 38 epoch_requested: 38 NullEvt 2026-02-11T23:30:32.196+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 38 epoch_requested: 38 NullEvt) class_id 2 prio 255 cost 10 e38) 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 36 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:32.196+0000 7f41591a6640 7 osd.0 38 activate_map version 38 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 36 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 36 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 active mbc={}] plpg_on_pool_change 2026-02-11T23:30:32.196+0000 7f414f993640 20 osd.0 pg_epoch: 36 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 active mbc={}] hit_set_clear 2026-02-11T23:30:32.196+0000 7f414f993640 20 osd.0 pg_epoch: 36 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 active mbc={}] agent_stop 2026-02-11T23:30:32.196+0000 7f414b98b640 20 osd.0 op_wq(0) _process 2.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:32.196+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 38 epoch_requested: 38 NullEvt) class_id 2immediate prio 255 cost 10 e38) queued 2026-02-11T23:30:32.196+0000 7f414f993640 20 osd.0 38 advance_pg new pool opts old pool opts 2026-02-11T23:30:32.196+0000 7f4150194640 20 osd.0 38 agent_entry empty queue 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 36 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 active mbc={}] handle_advance_map: 37 2026-02-11T23:30:32.196+0000 7f414f993640 20 osd.0:0.update_pg_epoch 2.0 36 -> 37 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 36 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [3,1]/[3,1] -- 3/3 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 37 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:32.196+0000 7f414f993640 20 osd.0 pg_epoch: 37 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 active mbc={}] PeeringState::should_restart_peering new interval newup [3,1] newacting [3,1] 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 37 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 active mbc={}] state: should_restart_peering, transitioning to Reset 2026-02-11T23:30:32.196+0000 7f414f993640 5 osd.0 pg_epoch: 37 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 active mbc={}] exit Started/ReplicaActive/RepNotRecovering 0.268321 6 0.000075 2026-02-11T23:30:32.196+0000 7f414f993640 5 osd.0 pg_epoch: 37 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 active mbc={}] exit Started/ReplicaActive 0.268325 0 0.000000 2026-02-11T23:30:32.196+0000 7f414f993640 5 osd.0 pg_epoch: 37 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 active mbc={}] exit Started 0.268425 0 0.000000 2026-02-11T23:30:32.196+0000 7f414f993640 5 osd.0 pg_epoch: 37 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 active mbc={}] enter Reset 2026-02-11T23:30:32.196+0000 7f414f993640 20 osd.0 pg_epoch: 37 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=34) [3,1,0] r=2 lpr=34 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset set_last_peering_reset 37 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 37 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=34) [3,1,0] r=2 lpr=37 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset Clearing blocked outgoing recovery messages 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 37 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=34) [3,1,0] r=2 lpr=37 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset Not blocking outgoing recovery messages 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 37 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=34) [3,1,0] r=2 lpr=37 crt=0'0 active mbc={}] state: Reset advmap 2026-02-11T23:30:32.196+0000 7f414f993640 20 osd.0 pg_epoch: 37 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=34) [3,1,0] r=2 lpr=37 crt=0'0 active mbc={}] PeeringState::should_restart_peering new interval newup [3,1] newacting [3,1] 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 37 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=34) [3,1,0] r=2 lpr=37 crt=0'0 active mbc={}] state: should restart peering, calling start_peering_interval again 2026-02-11T23:30:32.196+0000 7f414f993640 20 osd.0 pg_epoch: 37 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=34) [3,1,0] r=2 lpr=37 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset set_last_peering_reset 37 2026-02-11T23:30:32.196+0000 7f414f993640 15 osd.0 pg_epoch: 37 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=34) [3,1] r=2 lpr=37 crt=0'0 active mbc={}] clear_stats 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 37 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=34) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 active mbc={}] PeeringState::start_peering_interval : check_new_interval output: check_new_interval interval(34-36 up [3, 1, 0](3) acting [3, 1, 0](3)) up_thru 34 up_from 19 last_epoch_clean 35 interval(34-36 up [3, 1, 0](3) acting [3, 1, 0](3) maybe_went_rw) : primary up 19-34 includes interval 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 37 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=34) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 active mbc={}] PeeringState::start_peering_interval noting past ([34,36] all_participants={0, 1, 3} intervals=[([34,36] acting={0, 1, 3})]) 2026-02-11T23:30:32.196+0000 7f414f993640 20 osd.0 pg_epoch: 37 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 active mbc={}] PeeringState::on_new_interval 2026-02-11T23:30:32.196+0000 7f414f993640 20 osd.0 pg_epoch: 37 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 active mbc={}] PeeringState::on_new_interval upacting_features 0x3f0ffffffffdffff from [3,1]+[3,1] 2026-02-11T23:30:32.196+0000 7f414f993640 20 osd.0 pg_epoch: 37 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 active mbc={}] PeeringState::on_new_interval checking missing set deletes flag. missing = missing(0 may_include_deletes = 1) 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 37 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 active mbc={}] PeeringState::init_hb_stamps now [] 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 37 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743453026s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 active pruub 30.616516113s@ mbc={}] PeeringState::on_new_interval prior_readable_until_ub 30.616516113s (mnow 14.873064041s + 15.743453026s) 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 37 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743453026s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 active pruub 30.616516113s@ mbc={}] cancel_recovery 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 37 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743453026s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 active pruub 30.616516113s@ mbc={}] clear_recovery_state 2026-02-11T23:30:32.196+0000 7f414f993640 15 osd.0 pg_epoch: 37 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743453026s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 active pruub 30.616516113s@ mbc={}] _clear_recovery_state 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 37 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743453026s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 active pruub 30.616516113s@ mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:32.196+0000 7f414f993640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:32.196+0000 7f414f993640 20 osd.0 pg_epoch: 37 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743453026s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 active pruub 30.616516113s@ mbc={}] scrubber: FSM: ReplicaActive::exit() 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 37 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743453026s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 active pruub 30.616516113s@ mbc={}] scrubber: FSM: ReplicaActive::clear_remote_reservation(): pending_reservation_nonce 0, reservation_granted false 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 37 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743453026s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 active pruub 30.616516113s@ mbc={}] scrubber: FSM: -- state -->> NotActive 2026-02-11T23:30:32.196+0000 7f414f993640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:32.196+0000 7f414f993640 1 osd.0 pg_epoch: 37 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743453026s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 active pruub 30.616516113s@ mbc={}] PeeringState::start_peering_interval up [3,1,0] -> [3,1], acting [3,1,0] -> [3,1], acting_primary 3 -> 3, up_primary 3 -> 3, role 2 -> -1, features acting 4544132024016699391 upacting 4544132024016699391 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 37 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743453026s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown pruub 30.616516113s@ mbc={}] PeeringState::clear_primary_state clear_primary_state 2026-02-11T23:30:32.196+0000 7f414f993640 20 osd.0 pg_epoch: 37 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743453026s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown pruub 30.616516113s@ mbc={}] clear_primary_state 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 37 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743453026s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown pruub 30.616516113s@ mbc={}] release_backoffs [2:00000000::::0,MAX) 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 37 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743453026s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown pruub 30.616516113s@ mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:32.196+0000 7f414f993640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:32.196+0000 7f414f993640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:32.196+0000 7f414f993640 20 osd.0 pg_epoch: 37 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743453026s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown pruub 30.616516113s@ mbc={}] agent_stop 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 37 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743453026s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown pruub 30.616516113s@ mbc={}] on_change 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 37 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743453026s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown pruub 30.616516113s@ mbc={}] cancel_copy_ops 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 37 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743453026s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown pruub 30.616516113s@ mbc={}] cancel_flush_ops 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 37 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743453026s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown pruub 30.616516113s@ mbc={}] cancel_proxy_ops 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 37 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743453026s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown pruub 30.616516113s@ mbc={}] cancel_manifest_ops 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 37 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743453026s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown pruub 30.616516113s@ mbc={}] cancel_cls_gather_ops 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 37 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743453026s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown pruub 30.616516113s@ mbc={}] on_change_cleanup 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 37 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743453026s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown pruub 30.616516113s@ mbc={}] on_change 2026-02-11T23:30:32.196+0000 7f414f993640 20 osd.0 pg_epoch: 37 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743453026s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown pruub 30.616516113s@ mbc={}] exit NotTrimming 2026-02-11T23:30:32.196+0000 7f414f993640 20 osd.0 pg_epoch: 37 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743453026s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown pruub 30.616516113s@ mbc={}] enter NotTrimming 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 37 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743453026s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] plpg_on_role_change 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 37 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743453026s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 37 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743453026s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] plpg_on_pool_change 2026-02-11T23:30:32.196+0000 7f414f993640 20 osd.0 pg_epoch: 37 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743453026s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] hit_set_clear 2026-02-11T23:30:32.196+0000 7f414f993640 20 osd.0 pg_epoch: 37 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743453026s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] agent_stop 2026-02-11T23:30:32.196+0000 7f414f993640 20 osd.0 38 advance_pg new pool opts old pool opts 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 37 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743453026s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] handle_advance_map: 38 2026-02-11T23:30:32.196+0000 7f414f993640 20 osd.0:0.update_pg_epoch 2.0 37 -> 38 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 37 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743453026s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] PeeringState::advance_map handle_advance_map [3,1]/[3,1] -- 3/3 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 38 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743453026s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] state: Reset advmap 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 38 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743453026s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 38 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743453026s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] plpg_on_pool_change 2026-02-11T23:30:32.196+0000 7f414f993640 20 osd.0 pg_epoch: 38 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743453026s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] hit_set_clear 2026-02-11T23:30:32.196+0000 7f414f993640 20 osd.0 pg_epoch: 38 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743453026s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] agent_stop 2026-02-11T23:30:32.196+0000 7f414f993640 20 osd.0 38 advance_pg new pool opts old pool opts 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 38 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743453026s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] handle_activate_map: epoch range: 36..38 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 38 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743453026s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] PeeringState::activate_map 2026-02-11T23:30:32.196+0000 7f414f993640 5 osd.0 pg_epoch: 38 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743384361s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] exit Reset 0.000105 2 0.000096 2026-02-11T23:30:32.196+0000 7f414f993640 5 osd.0 pg_epoch: 38 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743384361s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] enter Started 2026-02-11T23:30:32.196+0000 7f414f993640 5 osd.0 pg_epoch: 38 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743384361s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] enter Start 2026-02-11T23:30:32.196+0000 7f414f993640 1 osd.0 pg_epoch: 38 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743384361s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] state: transitioning to Stray 2026-02-11T23:30:32.196+0000 7f414f993640 5 osd.0 pg_epoch: 38 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743384361s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] exit Start 0.000002 0 0.000000 2026-02-11T23:30:32.196+0000 7f414f993640 5 osd.0 pg_epoch: 38 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743384361s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] enter Started/Stray 2026-02-11T23:30:32.196+0000 7f414f993640 20 osd.0 pg_epoch: 38 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743384361s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 35 while current is 38 2026-02-11T23:30:32.196+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 38 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743384361s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 38 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743384361s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] do_peering_event: epoch_sent: 38 epoch_requested: 38 MQuery 2.0 from 3 query_epoch 38 query: query(info 0'0 epoch_sent 38) 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 38 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743355751s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] PeeringState::fulfill_info sending info 2026-02-11T23:30:32.196+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:32.196+0000 7f414f993640 20 osd.0 38 get_con_osd_cluster to osd.3 from_epoch 38 2026-02-11T23:30:32.196+0000 7f414f993640 20 osd.0 38 get_nextmap_reserved map_reservations: {38=1} 2026-02-11T23:30:32.196+0000 7f414f993640 20 osd.0 38 release_map epoch: 38 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 38 maybe_share_map: con v2:127.0.0.1:6828/2699377081 map epoch 35 -> 38 (shared) 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 38 send_incremental_map epoch range: (36, 38] to v2:127.0.0.1:6828/2699377081 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 38 add_map_inc_bl 36 574 bytes 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 38 add_map_inc_bl 37 574 bytes 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 38 add_map_inc_bl 38 574 bytes 2026-02-11T23:30:32.196+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6828/2699377081,v1:127.0.0.1:6829/2699377081] -- osd_map(36..38 src has 1..38) -- 0x59541c0 con 0x48ccc00 2026-02-11T23:30:32.196+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6828/2699377081,v1:127.0.0.1:6829/2699377081] -- pg_notify2(2.0 2.0 (query:38 sent:38 2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743384361s) ([34,36] all_participants={0, 1, 3} intervals=[([34,36] acting={0, 1, 3})])) e38/38) -- 0x373b800 con 0x48ccc00 2026-02-11T23:30:32.196+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6828/2699377081,v1:127.0.0.1:6829/2699377081] -- pg_notify2(2.0 2.0 (query:38 sent:38 2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743355751s) ([34,36] all_participants={0, 1, 3} intervals=[([34,36] acting={0, 1, 3})])) e38/38) -- 0x5a8c000 con 0x48ccc00 2026-02-11T23:30:32.196+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:32.196+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 38 epoch_requested: 38 NullEvt) class_id 2immediate prio 255 cost 10 e38) queued 2026-02-11T23:30:32.196+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:32.196+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 38 epoch_requested: 38 NullEvt) class_id 2immediate prio 255 cost 10 e38) pg 0x5946000 2026-02-11T23:30:32.196+0000 7f414b98b640 20 osd.0 op_wq(0) _process 2.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:32.196+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 38 epoch_requested: 38 NullEvt) class_id 2immediate prio 255 cost 10 e38) pg 0x5978000 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 35 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 36 2026-02-11T23:30:32.196+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 35 -> 36 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 35 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:32.196+0000 7f414b98b640 10 osd.0 pg_epoch: 38 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743355751s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] do_peering_event: epoch_sent: 38 epoch_requested: 38 NullEvt 2026-02-11T23:30:32.196+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 36 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:32.196+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 36 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:32.196+0000 7f414f993640 20 osd.0 38 advance_pg new pool opts old pool opts 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 36 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 37 2026-02-11T23:30:32.196+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 36 -> 37 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 36 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 37 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 37 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:32.196+0000 7f414f993640 20 osd.0 38 advance_pg new pool opts old pool opts 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 37 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 38 2026-02-11T23:30:32.196+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 37 -> 38 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 37 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 38 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 38 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:32.196+0000 7f414f993640 20 osd.0 38 advance_pg new pool opts old pool opts 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 38 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_activate_map: epoch range: 36..38 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 38 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:32.196+0000 7f414f993640 20 osd.0 pg_epoch: 38 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 30 while current is 38 2026-02-11T23:30:32.196+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:32.196+0000 7f414f993640 10 osd.0 pg_epoch: 38 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 38 epoch_requested: 38 NullEvt 2026-02-11T23:30:32.196+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:32.196+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:32.196+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:32.196+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:32.295+0000 7f4162bea640 10 osd.0 38 tick_without_osd_lock 2026-02-11T23:30:32.295+0000 7f4162bea640 0 osd.0 38 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:32.295+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:32.295+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.549 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:32.295+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:32.296, recovery is active?:false restrictions:<....> 2026-02-11T23:30:32.295+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:32.295+0000 7f4162bea640 10 osd.0 38 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:32.295+0000 7f4162bea640 20 osd.0 38 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:32.295+0000 7f4162bea640 10 osd.0 38 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:32.295+0000 7f4162bea640 20 osd.0 38 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:32.326+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:30:32.326+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:30:32.326+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:02.327463+0000) 2026-02-11T23:30:32.345+0000 7f41633eb640 10 osd.0 38 tick 2026-02-11T23:30:32.346+0000 7f41633eb640 20 osd.0 38 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:30:32.705+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.3 v2:127.0.0.1:6828/2699377081 10 ==== osd pg remove(epoch 42; pg2.0; ) ==== 32+0+0 (crc 0 0 0) 0x3758c60 con 0x48ccc00 2026-02-11T23:30:32.705+0000 7f416cbfe640 7 osd.0 38 handle_fast_pg_remove osd pg remove(epoch 42; pg2.0; ) from osd.3 2026-02-11T23:30:32.705+0000 7f416cbfe640 15 osd.0 38 enqueue_peering_evt 2.0 epoch_sent: 42 epoch_requested: 42 DeleteStart 2026-02-11T23:30:32.705+0000 7f416cbfe640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 42 epoch_requested: 42 DeleteStart) class_id 2 prio 255 cost 10 e42) 2026-02-11T23:30:32.705+0000 7f414b98b640 20 osd.0 op_wq(0) _process 2.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:32.705+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 42 epoch_requested: 42 DeleteStart) class_id 2immediate prio 255 cost 10 e42) queued 2026-02-11T23:30:32.705+0000 7f414b98b640 20 osd.0 op_wq(0) _process 2.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:32.705+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 42 epoch_requested: 42 DeleteStart) class_id 2immediate prio 255 cost 10 e42) pg 0x5978000 2026-02-11T23:30:32.705+0000 7f414b98b640 20 osd.0 op_wq _add_slot_waiter 2.0 peering, item epoch is 42, will wait on OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 42 epoch_requested: 42 DeleteStart) class_id 2immediate prio 255 cost 10 e42) 2026-02-11T23:30:32.705+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:32.705+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:33.039+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.1 127.0.0.1:0/3043114013 7 ==== osd_ping(ping e43 up_from 10 ping_stamp 2026-02-11T23:30:33.040108+0000/13.703852654s send_stamp 13.703852654s delta_ub -2.011489630s) ==== 2033+0+0 (crc 0 0 0) 0x4c9ac00 con 0x4aca000 2026-02-11T23:30:33.039+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.1 127.0.0.1:0/3043114013 7 ==== osd_ping(ping e43 up_from 10 ping_stamp 2026-02-11T23:30:33.040108+0000/13.703852654s send_stamp 13.703852654s delta_ub -2.011489630s) ==== 2033+0+0 (crc 0 0 0) 0x4c9b200 con 0x4aca400 2026-02-11T23:30:33.039+0000 7f416cbfe640 20 osd.0 38 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011766195s,-2.011489630s]) 2026-02-11T23:30:33.039+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e38 up_from 5 ping_stamp 2026-02-11T23:30:33.040108+0000/13.703852654s send_stamp 15.715620041s delta_ub 2.011766195s) -- 0x4c9ad80 con 0x4aca000 2026-02-11T23:30:33.039+0000 7f416cbfe640 20 osd.0 38 get_con_osd_cluster to osd.1 from_epoch 38 2026-02-11T23:30:33.039+0000 7f416cbfe640 20 osd.0 38 get_nextmap_reserved map_reservations: {38=1} 2026-02-11T23:30:33.039+0000 7f416cbfe640 20 osd.0 38 release_map epoch: 38 2026-02-11T23:30:33.039+0000 7f416cbfe640 10 osd.0 38 maybe_share_map: con v2:127.0.0.1:6812/3043114013 updating session's projected_epoch from 35 to ping map epoch of 43 2026-02-11T23:30:33.039+0000 7f416cbfe640 10 osd.0 38 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 38 is not newer than session's projected_epoch of 43 2026-02-11T23:30:33.039+0000 7f416c3fd640 20 osd.0 38 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011815310s,-2.011489630s]) 2026-02-11T23:30:33.039+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e38 up_from 5 ping_stamp 2026-02-11T23:30:33.040108+0000/13.703852654s send_stamp 15.715668678s delta_ub 2.011815310s) -- 0x4c9a480 con 0x4aca400 2026-02-11T23:30:33.039+0000 7f416c3fd640 20 osd.0 38 get_con_osd_cluster to osd.1 from_epoch 38 2026-02-11T23:30:33.039+0000 7f416c3fd640 20 osd.0 38 get_nextmap_reserved map_reservations: {38=1} 2026-02-11T23:30:33.039+0000 7f416c3fd640 20 osd.0 38 release_map epoch: 38 2026-02-11T23:30:33.039+0000 7f416c3fd640 10 osd.0 38 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 38 is not newer than session's projected_epoch of 43 2026-02-11T23:30:33.039+0000 7f415aefe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 14 ==== osd_map(39..43 src has 1..43) ==== 2319+0+0 (unknown 0 0 0) 0x4a31500 con 0x4acb000 2026-02-11T23:30:33.039+0000 7f415aefe640 20 osd.0 38 OSD::ms_dispatch: osd_map(39..43 src has 1..43) 2026-02-11T23:30:33.039+0000 7f415aefe640 20 osd.0 38 _dispatch 0x4a31500 osd_map(39..43 src has 1..43) 2026-02-11T23:30:33.039+0000 7f415aefe640 3 osd.0 38 handle_osd_map epochs [39,43], i have 38, src has [1,43] 2026-02-11T23:30:33.039+0000 7f415aefe640 10 osd.0 38 handle_osd_map: received 0 full maps and 5 incremental maps 2026-02-11T23:30:33.039+0000 7f415aefe640 10 osd.0 38 handle_osd_map got inc map for epoch 39 2026-02-11T23:30:33.039+0000 7f415aefe640 20 osd.0 38 got_full_map 39, nothing requested 2026-02-11T23:30:33.039+0000 7f415aefe640 10 osd.0 38 handle_osd_map got inc map for epoch 40 2026-02-11T23:30:33.039+0000 7f415aefe640 20 osd.0 38 got_full_map 40, nothing requested 2026-02-11T23:30:33.039+0000 7f415aefe640 10 osd.0 38 handle_osd_map got inc map for epoch 41 2026-02-11T23:30:33.039+0000 7f415aefe640 20 osd.0 38 got_full_map 41, nothing requested 2026-02-11T23:30:33.039+0000 7f415aefe640 10 osd.0 38 handle_osd_map got inc map for epoch 42 2026-02-11T23:30:33.039+0000 7f415aefe640 20 osd.0 38 got_full_map 42, nothing requested 2026-02-11T23:30:33.039+0000 7f415aefe640 10 osd.0 38 handle_osd_map got inc map for epoch 43 2026-02-11T23:30:33.039+0000 7f415aefe640 20 osd.0 38 got_full_map 43, nothing requested 2026-02-11T23:30:33.039+0000 7f415aefe640 10 osd.0 38 _track_pools_and_pg_num_changes recording new pool 3 pg_num 1 2026-02-11T23:30:33.039+0000 7f415aefe640 20 osd.0 38 trim_maps: min=0 oldest_map=1 2026-02-11T23:30:33.039+0000 7f415aefe640 20 osd.0 38 handle_osd_map pg_num_history pg_num_history(e43 pg_nums {1={27=1},2={34=1},3={42=1}} deleted_pools ) 2026-02-11T23:30:33.039+0000 7f415aefe640 10 snap_mapper record_purged_snaps purged_snaps {39={},40={},41={},42={},43={}} 2026-02-11T23:30:33.039+0000 7f415aefe640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:33.039+0000 7f415aefe640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e43 maps [1~43] lci=[5,43] tlb=1) 2026-02-11T23:30:33.040+0000 7f41591a6640 10 osd.0 38 _committed_osd_maps 39..43 2026-02-11T23:30:33.040+0000 7f41591a6640 10 osd.0 38 advance to epoch 39 (<= last 43 <= newest_map 43) 2026-02-11T23:30:33.040+0000 7f41591a6640 10 osd.0 39 advance to epoch 40 (<= last 43 <= newest_map 43) 2026-02-11T23:30:33.040+0000 7f41591a6640 10 osd.0 40 advance to epoch 41 (<= last 43 <= newest_map 43) 2026-02-11T23:30:33.040+0000 7f41591a6640 10 osd.0 41 advance to epoch 42 (<= last 43 <= newest_map 43) 2026-02-11T23:30:33.040+0000 7f41591a6640 10 osd.0 42 advance to epoch 43 (<= last 43 <= newest_map 43) 2026-02-11T23:30:33.040+0000 7f41591a6640 20 osd.0 43 consume_map version 43 2026-02-11T23:30:33.040+0000 7f41591a6640 20 osd.0 38 await_reserved_maps epoch:43 2026-02-11T23:30:33.040+0000 7f41591a6640 20 osd.0 38 await_reserved_maps done 0 2026-02-11T23:30:33.040+0000 7f41591a6640 20 osd.0 43 consume_map 43 -- publish done 2026-02-11T23:30:33.040+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:30:33.040+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:2.0slot:0x37002c0 2026-02-11T23:30:33.040+0000 7f41591a6640 20 osd.0 43 identify_splits_and_merges 2.0 e38 to e43 2026-02-11T23:30:33.040+0000 7f41591a6640 20 osd.0 43 identify_splits_and_merges 2.0 e38 to e43 pg_nums {34=1} 2026-02-11T23:30:33.040+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3700dc0 2026-02-11T23:30:33.040+0000 7f41591a6640 20 osd.0 43 identify_splits_and_merges 1.0 e38 to e43 2026-02-11T23:30:33.040+0000 7f41591a6640 20 osd.0 43 identify_splits_and_merges 1.0 e38 to e43 pg_nums {27=1} 2026-02-11T23:30:33.040+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:33.040+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:33.040+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:33.040+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:33.040+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:33.040+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:33.040+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:33.040+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:33.040+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:33.040+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:33.040+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:33.040+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:33.040+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:33.040+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:33.040+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:33.040+0000 7f41591a6640 20 osd.0 43 prune_pg_created 2026-02-11T23:30:33.040+0000 7f41591a6640 10 osd.0:0.consume_map 43 (was 38) 2026-02-11T23:30:33.040+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 2.0 2026-02-11T23:30:33.040+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 2.0 pending_peering first epoch 42 <= 43, requeueing 2026-02-11T23:30:33.040+0000 7f41591a6640 20 osd.0:0._wake_pg_slot _wake_pg_slot 2.0 to_process <> waiting <> waiting_peering {42=} 2026-02-11T23:30:33.040+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:33.040+0000 7f41591a6640 10 osd.0:1.consume_map 43 (was 38) 2026-02-11T23:30:33.040+0000 7f41591a6640 10 osd.0:2.consume_map 43 (was 38) 2026-02-11T23:30:33.040+0000 7f41591a6640 10 osd.0:3.consume_map 43 (was 38) 2026-02-11T23:30:33.040+0000 7f41591a6640 10 osd.0:4.consume_map 43 (was 38) 2026-02-11T23:30:33.040+0000 7f41591a6640 10 osd.0:5.consume_map 43 (was 38) 2026-02-11T23:30:33.040+0000 7f41591a6640 10 osd.0:6.consume_map 43 (was 38) 2026-02-11T23:30:33.040+0000 7f41591a6640 10 osd.0:7.consume_map 43 (was 38) 2026-02-11T23:30:33.040+0000 7f414b98b640 20 osd.0 op_wq(0) _process 2.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:33.040+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 42 epoch_requested: 42 DeleteStart) class_id 2immediate prio 255 cost 10 e42) queued 2026-02-11T23:30:33.040+0000 7f414b98b640 20 osd.0 op_wq(0) _process 2.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:33.040+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 42 epoch_requested: 42 DeleteStart) class_id 2immediate prio 255 cost 10 e42) pg 0x5978000 2026-02-11T23:30:33.040+0000 7f41591a6640 10 osd.0 43 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:33.040+0000 7f41591a6640 15 osd.0 43 enqueue_peering_evt 2.0 epoch_sent: 43 epoch_requested: 43 NullEvt 2026-02-11T23:30:33.040+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 43 epoch_requested: 43 NullEvt) class_id 2 prio 255 cost 10 e43) 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 38 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743355751s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] handle_advance_map: 39 2026-02-11T23:30:33.040+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 2.0 38 -> 39 2026-02-11T23:30:33.040+0000 7f41591a6640 15 osd.0 43 enqueue_peering_evt 1.0 epoch_sent: 43 epoch_requested: 43 NullEvt 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 38 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743355751s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] PeeringState::advance_map handle_advance_map [3,1]/[3,1] -- 3/3 2026-02-11T23:30:33.040+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 43 epoch_requested: 43 NullEvt) class_id 2 prio 255 cost 10 e43) 2026-02-11T23:30:33.040+0000 7f41591a6640 7 osd.0 43 activate_map version 43 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 39 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743355751s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] state: Started advmap 2026-02-11T23:30:33.040+0000 7f414f993640 20 osd.0 op_wq(0) _process 2.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:33.040+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 43 epoch_requested: 43 NullEvt) class_id 2immediate prio 255 cost 10 e43) queued 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 39 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743355751s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:33.040+0000 7f4150194640 20 osd.0 43 agent_entry empty queue 2026-02-11T23:30:33.040+0000 7f414b98b640 20 osd.0 43 advance_pg new pool opts old pool opts 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 39 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743355751s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] handle_advance_map: 40 2026-02-11T23:30:33.040+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 2.0 39 -> 40 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 39 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743355751s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] PeeringState::advance_map handle_advance_map [3,1]/[3,1] -- 3/3 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 40 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743355751s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] state: Started advmap 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 40 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743355751s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 40 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743355751s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] plpg_on_pool_change 2026-02-11T23:30:33.040+0000 7f414b98b640 20 osd.0 pg_epoch: 40 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743355751s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] hit_set_clear 2026-02-11T23:30:33.040+0000 7f414b98b640 20 osd.0 pg_epoch: 40 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743355751s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] agent_stop 2026-02-11T23:30:33.040+0000 7f414b98b640 20 osd.0 43 advance_pg new pool opts old pool opts 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 40 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743355751s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] handle_advance_map: 41 2026-02-11T23:30:33.040+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 2.0 40 -> 41 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 40 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743355751s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] PeeringState::advance_map handle_advance_map [3,1]/[3,1] -- 3/3 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 41 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743355751s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] state: Started advmap 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 41 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743355751s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 41 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743355751s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] plpg_on_pool_change 2026-02-11T23:30:33.040+0000 7f414b98b640 20 osd.0 pg_epoch: 41 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743355751s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] hit_set_clear 2026-02-11T23:30:33.040+0000 7f414b98b640 20 osd.0 pg_epoch: 41 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743355751s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] agent_stop 2026-02-11T23:30:33.040+0000 7f414b98b640 20 osd.0 43 advance_pg new pool opts old pool opts 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 41 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743355751s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] handle_advance_map: 42 2026-02-11T23:30:33.040+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 2.0 41 -> 42 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 41 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743355751s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] PeeringState::advance_map handle_advance_map [3,1]/[3,1] -- 3/3 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 42 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743355751s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] state: Started advmap 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 42 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743355751s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:33.040+0000 7f414b98b640 20 osd.0 43 advance_pg new pool opts old pool opts 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 42 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743355751s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] handle_advance_map: 43 2026-02-11T23:30:33.040+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 2.0 42 -> 43 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 42 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743355751s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] PeeringState::advance_map handle_advance_map [3,1]/[3,1] -- 3/3 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 43 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743355751s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] state: Started advmap 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 43 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743355751s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:33.040+0000 7f414b98b640 20 osd.0 43 advance_pg new pool opts old pool opts 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 43 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743355751s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] handle_activate_map: epoch range: 39..43 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 43 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=15.743355751s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] PeeringState::activate_map 2026-02-11T23:30:33.040+0000 7f414b98b640 20 osd.0 pg_epoch: 43 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=14.899634361s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 38 while current is 43 2026-02-11T23:30:33.040+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 43 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=14.899634361s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 43 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=14.899634361s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] do_peering_event: epoch_sent: 42 epoch_requested: 42 DeleteStart 2026-02-11T23:30:33.040+0000 7f414b98b640 5 osd.0 pg_epoch: 43 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=14.899634361s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] exit Started/Stray 0.843753 9 0.000053 2026-02-11T23:30:33.040+0000 7f414b98b640 5 osd.0 pg_epoch: 43 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=14.899634361s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] enter Started/ToDelete 2026-02-11T23:30:33.040+0000 7f414b98b640 5 osd.0 pg_epoch: 43 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=14.899634361s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] enter Started/ToDelete/WaitDeleteReserved 2026-02-11T23:30:33.040+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:33.040+0000 7f414b98b640 20 osd.0 43 get_con_osd_cluster to osd.3 from_epoch 43 2026-02-11T23:30:33.040+0000 7f414b98b640 20 osd.0 43 get_nextmap_reserved map_reservations: {43=1} 2026-02-11T23:30:33.040+0000 7f414b98b640 20 osd.0 43 release_map epoch: 43 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 43 maybe_share_map: con v2:127.0.0.1:6828/2699377081 map epoch 38 -> 43 (shared) 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 43 send_incremental_map epoch range: (39, 43] to v2:127.0.0.1:6828/2699377081 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 43 add_map_inc_bl 39 252 bytes 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 43 add_map_inc_bl 40 574 bytes 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 43 add_map_inc_bl 41 574 bytes 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 43 add_map_inc_bl 42 591 bytes 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 43 add_map_inc_bl 43 252 bytes 2026-02-11T23:30:33.040+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6828/2699377081,v1:127.0.0.1:6829/2699377081] -- osd_map(39..43 src has 1..43) -- 0x5954e00 con 0x48ccc00 2026-02-11T23:30:33.040+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6828/2699377081,v1:127.0.0.1:6829/2699377081] -- pg_notify2(2.0 2.0 (query:43 sent:43 2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=14.899634361s) ([34,36] all_participants={0, 1, 3} intervals=[([34,36] acting={0, 1, 3})])) e43/43) -- 0x3739c00 con 0x48ccc00 2026-02-11T23:30:33.040+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:33.040+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 43 epoch_requested: 43 NullEvt) class_id 2immediate prio 255 cost 10 e43) queued 2026-02-11T23:30:33.040+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:33.040+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 43 epoch_requested: 43 NullEvt) class_id 2immediate prio 255 cost 10 e43) pg 0x5946000 2026-02-11T23:30:33.040+0000 7f414f993640 20 osd.0 op_wq(0) _process 2.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:33.040+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 43 epoch_requested: 43 NullEvt) class_id 2immediate prio 255 cost 10 e43) pg 0x5978000 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 38 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 39 2026-02-11T23:30:33.040+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 1.0 38 -> 39 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 38 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:33.040+0000 7f414f993640 10 osd.0 pg_epoch: 43 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=14.899634361s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] do_peering_event: epoch_sent: 43 epoch_requested: 43 NullEvt 2026-02-11T23:30:33.040+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 39 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 39 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:33.040+0000 7f414b98b640 20 osd.0 43 advance_pg new pool opts old pool opts 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 39 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 40 2026-02-11T23:30:33.040+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 1.0 39 -> 40 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 39 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 40 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:33.040+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 40 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:33.040+0000 7f414b98b640 20 osd.0 43 advance_pg new pool opts old pool opts 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 40 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 41 2026-02-11T23:30:33.040+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 1.0 40 -> 41 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 40 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 41 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 41 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:33.040+0000 7f414b98b640 20 osd.0 43 advance_pg new pool opts old pool opts 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 41 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 42 2026-02-11T23:30:33.040+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 1.0 41 -> 42 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 41 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 42 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 42 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:33.040+0000 7f414b98b640 20 osd.0 43 advance_pg new pool opts old pool opts 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 42 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 43 2026-02-11T23:30:33.040+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 1.0 42 -> 43 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 42 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 43 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 43 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:33.040+0000 7f414b98b640 20 osd.0 43 advance_pg new pool opts old pool opts 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 43 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_activate_map: epoch range: 39..43 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 43 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:33.040+0000 7f414b98b640 20 osd.0 pg_epoch: 43 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 30 while current is 43 2026-02-11T23:30:33.040+0000 7f4152999640 15 osd.0 43 enqueue_peering_evt 2.0 epoch_sent: 43 epoch_requested: 43 DeleteReserved 2026-02-11T23:30:33.040+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 43 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 43 epoch_requested: 43 NullEvt 2026-02-11T23:30:33.040+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:33.040+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:33.040+0000 7f4152999640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 43 epoch_requested: 43 DeleteReserved) class_id 2 prio 255 cost 10 e43) 2026-02-11T23:30:33.040+0000 7f414b98b640 20 osd.0 op_wq(0) _process 2.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:33.040+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 43 epoch_requested: 43 DeleteReserved) class_id 2immediate prio 255 cost 10 e43) queued 2026-02-11T23:30:33.040+0000 7f414b98b640 20 osd.0 op_wq(0) _process 2.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:33.040+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 43 epoch_requested: 43 DeleteReserved) class_id 2immediate prio 255 cost 10 e43) pg 0x5978000 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 43 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=14.899634361s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] do_peering_event: epoch_sent: 43 epoch_requested: 43 DeleteReserved 2026-02-11T23:30:33.040+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:33.040+0000 7f414b98b640 5 osd.0 pg_epoch: 43 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=14.899634361s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] exit Started/ToDelete/WaitDeleteReserved 0.000311 2 0.000017 2026-02-11T23:30:33.040+0000 7f414b98b640 5 osd.0 pg_epoch: 43 pg[2.0( empty local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=14.899634361s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] enter Started/ToDelete/Deleting 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 43 pg[2.0( empty lb MIN local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=14.899634361s) [3,1] r=-1 lpr=37 DELETING pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] on_removal 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 43 pg[2.0( empty lb MIN local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=14.899634361s) [3,1] r=-1 lpr=37 DELETING pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] on_shutdown 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 43 pg[2.0( empty lb MIN local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=14.899634361s) [3,1] r=-1 lpr=37 DELETING pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:33.040+0000 7f414b98b640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:33.040+0000 7f414b98b640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 43 pg[2.0( empty lb MIN local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=14.899634361s) [3,1] r=-1 lpr=37 DELETING pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] cancel_copy_ops 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 43 pg[2.0( empty lb MIN local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=14.899634361s) [3,1] r=-1 lpr=37 DELETING pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] cancel_flush_ops 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 43 pg[2.0( empty lb MIN local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=14.899634361s) [3,1] r=-1 lpr=37 DELETING pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] cancel_proxy_ops 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 43 pg[2.0( empty lb MIN local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=14.899634361s) [3,1] r=-1 lpr=37 DELETING pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] cancel_manifest_ops 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 43 pg[2.0( empty lb MIN local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=14.899634361s) [3,1] r=-1 lpr=37 DELETING pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] cancel_cls_gather_ops 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 43 pg[2.0( empty lb MIN local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=14.899634361s) [3,1] r=-1 lpr=37 DELETING pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] clear_backoffs 2026-02-11T23:30:33.040+0000 7f414b98b640 20 osd.0 pg_epoch: 43 pg[2.0( empty lb MIN local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=14.899634361s) [3,1] r=-1 lpr=37 DELETING pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] exit NotTrimming 2026-02-11T23:30:33.040+0000 7f414b98b640 20 osd.0 pg_epoch: 43 pg[2.0( empty lb MIN local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=14.899634361s) [3,1] r=-1 lpr=37 DELETING pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] enter NotTrimming 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 43 pg[2.0( empty lb MIN local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=14.899634361s) [3,1] r=-1 lpr=37 DELETING pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] on_change 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 43 pg[2.0( empty lb MIN local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=14.899634361s) [3,1] r=-1 lpr=37 DELETING pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] clear_async_reads 2026-02-11T23:30:33.040+0000 7f414b98b640 20 osd.0 pg_epoch: 43 pg[2.0( empty lb MIN local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=14.899634361s) [3,1] r=-1 lpr=37 DELETING pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] clear_primary_state 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 43 pg[2.0( empty lb MIN local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=14.899634361s) [3,1] r=-1 lpr=37 DELETING pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] release_backoffs [2:00000000::::0,MAX) 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 43 pg[2.0( empty lb MIN local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=14.899634361s) [3,1] r=-1 lpr=37 DELETING pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:33.040+0000 7f414b98b640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:33.040+0000 7f414b98b640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:33.040+0000 7f414b98b640 20 osd.0 pg_epoch: 43 pg[2.0( empty lb MIN local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=14.899634361s) [3,1] r=-1 lpr=37 DELETING pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] agent_stop 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 43 pg[2.0( empty lb MIN local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=14.899634361s) [3,1] r=-1 lpr=37 DELETING pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] cancel_recovery 2026-02-11T23:30:33.040+0000 7f414b98b640 10 osd.0 pg_epoch: 43 pg[2.0( empty lb MIN local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=14.899634361s) [3,1] r=-1 lpr=37 DELETING pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] clear_recovery_state 2026-02-11T23:30:33.040+0000 7f414b98b640 15 osd.0 pg_epoch: 43 pg[2.0( empty lb MIN local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=14.899634361s) [3,1] r=-1 lpr=37 DELETING pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] _clear_recovery_state 2026-02-11T23:30:33.040+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:33.040+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:33.040+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:33.040+0000 7f414f993640 10 osd.0 43 queue_for_pg_delete on 2.0 e 43 2026-02-11T23:30:33.040+0000 7f414f993640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(2.0 PGDelete(pgid=2.0 epoch_queued=43) class_id 1 prio 5 cost 1048576 e43) 2026-02-11T23:30:33.040+0000 7f414f993640 20 osd.0 op_wq(0) _process 2.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:33.040+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGDelete(2.0 e43) class_id 1background_best_effort prio 5 qos_cost 1048576 cost 1048576 e43) queued 2026-02-11T23:30:33.040+0000 7f414f993640 20 osd.0 op_wq(0) _process 2.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:33.040+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGDelete(2.0 e43) class_id 1background_best_effort prio 5 qos_cost 1048576 cost 1048576 e43) pg 0x5978000 2026-02-11T23:30:33.040+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:33.040+0000 7f414f993640 10 osd.0 pg_epoch: 43 pg[2.0( empty lb MIN local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=14.899634361s) [3,1] r=-1 lpr=37 DELETING pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] do_peering_event: epoch_sent: 43 epoch_requested: 43 DeleteSome 2026-02-11T23:30:33.040+0000 7f414f993640 10 osd.0 pg_epoch: 43 pg[2.0( empty lb MIN local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=14.899634361s) [3,1] r=-1 lpr=37 DELETING pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] do_delete_work 2026-02-11T23:30:33.040+0000 7f414f993640 20 osd.0 pg_epoch: 43 pg[2.0( empty lb MIN local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=14.899634361s) [3,1] r=-1 lpr=37 DELETING pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] do_delete_work [#2:00000000::::head#] 2026-02-11T23:30:33.040+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:33.040+0000 7f414f993640 20 osd.0 43 try_finish_pg_delete 2.0 0x5978000 2026-02-11T23:30:33.040+0000 7f414f993640 10 osd.0:0._detach_pg 2.0 0x5978000 2026-02-11T23:30:33.040+0000 7f414f993640 5 osd.0 pg_epoch: 43 pg[2.0( empty lb MIN local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=14.899634361s) [3,1] r=-1 lpr=37 DELETING pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] exit Started/ToDelete/Deleting 0.000303 1 0.000057 2026-02-11T23:30:33.040+0000 7f414f993640 5 osd.0 pg_epoch: 43 pg[2.0( empty lb MIN local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=14.899634361s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] exit Started/ToDelete 0.000623 0 0.000000 2026-02-11T23:30:33.040+0000 7f414f993640 5 osd.0 pg_epoch: 43 pg[2.0( empty lb MIN local-lis/les=34/35 n=0 ec=34/34 lis/c=34/34 les/c/f=35/35/0 sis=37 pruub=14.899634361s) [3,1] r=-1 lpr=37 pi=[34,37)/1 crt=0'0 unknown NOTIFY pruub 30.616516113s@ mbc={}] exit Started 0.844390 0 0.000000 2026-02-11T23:30:33.040+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:33.040+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:33.040+0000 7f414f993640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:33.040+0000 7f414f993640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:33.040+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:33.040+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:33.203+0000 7f415b6ff640 20 osd.0 43 reports for 0 queries 2026-02-11T23:30:33.203+0000 7f415b6ff640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] -- mgrreport(unknown.0 +0-0 packed 3294 daemon_metrics=2) -- 0x4f9f180 con 0x4aca800 2026-02-11T23:30:33.203+0000 7f415b6ff640 15 osd.0 43 collect_pg_stats 2026-02-11T23:30:33.203+0000 7f415b6ff640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] -- pg_stats(0 pgs seq 21474836484 v 0) -- 0x4a34300 con 0x4aca800 2026-02-11T23:30:33.248+0000 7f4162bea640 10 osd.0 43 tick_without_osd_lock 2026-02-11T23:30:33.248+0000 7f4162bea640 0 osd.0 43 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:33.248+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:33.248+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:33.248+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:33.248+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.508 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:33.248+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:33.249, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:33.248+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:33.248+0000 7f4162bea640 10 osd.0 43 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:33.248+0000 7f4162bea640 20 osd.0 43 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:33.248+0000 7f4162bea640 10 osd.0 43 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:33.248+0000 7f4162bea640 20 osd.0 43 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:33.326+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:30:33.326+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:30:33.326+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:03.327481+0000) 2026-02-11T23:30:33.346+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.3 127.0.0.1:0/2699377081 2 ==== osd_ping(ping e43 up_from 19 ping_stamp 2026-02-11T23:30:33.346953+0000/8.802892685s send_stamp 8.802892685s delta_ub -7.219149590s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a480 con 0x48cc400 2026-02-11T23:30:33.346+0000 7f416c3fd640 20 osd.0 43 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219539165s,-7.219149590s]) 2026-02-11T23:30:33.346+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e43 up_from 5 ping_stamp 2026-02-11T23:30:33.346953+0000/8.802892685s send_stamp 16.022432327s delta_ub 7.219539165s) -- 0x4c9b200 con 0x48cc400 2026-02-11T23:30:33.346+0000 7f416c3fd640 20 osd.0 43 get_con_osd_cluster to osd.3 from_epoch 43 2026-02-11T23:30:33.346+0000 7f416c3fd640 20 osd.0 43 get_nextmap_reserved map_reservations: {43=1} 2026-02-11T23:30:33.346+0000 7f416c3fd640 20 osd.0 43 release_map epoch: 43 2026-02-11T23:30:33.346+0000 7f416c3fd640 10 osd.0 43 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 43 is not newer than session's projected_epoch of 43 2026-02-11T23:30:33.346+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.3 127.0.0.1:0/2699377081 2 ==== osd_ping(ping e43 up_from 19 ping_stamp 2026-02-11T23:30:33.346953+0000/8.802892685s send_stamp 8.802892685s delta_ub -7.219149590s) ==== 2033+0+0 (crc 0 0 0) 0x4c9ad80 con 0x48cc800 2026-02-11T23:30:33.346+0000 7f416cbfe640 20 osd.0 43 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219564915s,-7.219149590s]) 2026-02-11T23:30:33.346+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e43 up_from 5 ping_stamp 2026-02-11T23:30:33.346953+0000/8.802892685s send_stamp 16.022457123s delta_ub 7.219564915s) -- 0x4c9ac00 con 0x48cc800 2026-02-11T23:30:33.346+0000 7f416cbfe640 20 osd.0 43 get_con_osd_cluster to osd.3 from_epoch 43 2026-02-11T23:30:33.346+0000 7f416cbfe640 20 osd.0 43 get_nextmap_reserved map_reservations: {43=1} 2026-02-11T23:30:33.346+0000 7f416cbfe640 20 osd.0 43 release_map epoch: 43 2026-02-11T23:30:33.346+0000 7f416cbfe640 10 osd.0 43 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 43 is not newer than session's projected_epoch of 43 2026-02-11T23:30:33.392+0000 7f41633eb640 10 osd.0 43 tick 2026-02-11T23:30:33.392+0000 7f41633eb640 20 osd.0 43 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:30:33.693+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.2 127.0.0.1:0/2365263801 2 ==== osd_ping(ping e44 up_from 14 ping_stamp 2026-02-11T23:30:33.694182+0000/11.802778244s send_stamp 11.802778244s delta_ub -4.566683292s) ==== 2033+0+0 (crc 0 0 0) 0x4c9ac00 con 0x4641000 2026-02-11T23:30:33.693+0000 7f416cbfe640 20 osd.0 43 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566880703s,-4.566683292s]) 2026-02-11T23:30:33.693+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e43 up_from 5 ping_stamp 2026-02-11T23:30:33.694182+0000/11.802778244s send_stamp 16.369659424s delta_ub 4.566880703s) -- 0x4c9ad80 con 0x4641000 2026-02-11T23:30:33.693+0000 7f416cbfe640 20 osd.0 43 get_con_osd_cluster to osd.2 from_epoch 43 2026-02-11T23:30:33.693+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.2 127.0.0.1:0/2365263801 2 ==== osd_ping(ping e44 up_from 14 ping_stamp 2026-02-11T23:30:33.694182+0000/11.802778244s send_stamp 11.802778244s delta_ub -4.566683292s) ==== 2033+0+0 (crc 0 0 0) 0x4c9b200 con 0x48cc000 2026-02-11T23:30:33.693+0000 7f416cbfe640 20 osd.0 43 get_nextmap_reserved map_reservations: {43=1} 2026-02-11T23:30:33.693+0000 7f416cbfe640 20 osd.0 43 release_map epoch: 43 2026-02-11T23:30:33.693+0000 7f416cbfe640 10 osd.0 43 maybe_share_map: con v2:127.0.0.1:6820/2365263801 updating session's projected_epoch from 34 to ping map epoch of 44 2026-02-11T23:30:33.693+0000 7f416cbfe640 10 osd.0 43 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 43 is not newer than session's projected_epoch of 44 2026-02-11T23:30:33.693+0000 7f416c3fd640 20 osd.0 43 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566905499s,-4.566683292s]) 2026-02-11T23:30:33.693+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e43 up_from 5 ping_stamp 2026-02-11T23:30:33.694182+0000/11.802778244s send_stamp 16.369684219s delta_ub 4.566905499s) -- 0x4c9a480 con 0x48cc000 2026-02-11T23:30:33.693+0000 7f416c3fd640 20 osd.0 43 get_con_osd_cluster to osd.2 from_epoch 43 2026-02-11T23:30:33.693+0000 7f416c3fd640 20 osd.0 43 get_nextmap_reserved map_reservations: {43=1} 2026-02-11T23:30:33.693+0000 7f416c3fd640 20 osd.0 43 release_map epoch: 43 2026-02-11T23:30:33.693+0000 7f416c3fd640 10 osd.0 43 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 43 is not newer than session's projected_epoch of 44 2026-02-11T23:30:33.693+0000 7f415aefe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.2 v2:127.0.0.1:6820/2365263801 2 ==== osd_map(44..44 src has 1..44) ==== 618+0+0 (crc 0 0 0) 0x5954e00 con 0x507b800 2026-02-11T23:30:33.693+0000 7f415aefe640 20 osd.0 43 OSD::ms_dispatch: osd_map(44..44 src has 1..44) 2026-02-11T23:30:33.693+0000 7f415aefe640 20 osd.0 43 _dispatch 0x5954e00 osd_map(44..44 src has 1..44) 2026-02-11T23:30:33.693+0000 7f415aefe640 3 osd.0 43 handle_osd_map epochs [44,44], i have 43, src has [1,44] 2026-02-11T23:30:33.693+0000 7f415aefe640 10 osd.0 43 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:30:33.693+0000 7f415aefe640 10 osd.0 43 handle_osd_map got inc map for epoch 44 2026-02-11T23:30:33.693+0000 7f415aefe640 20 osd.0 43 got_full_map 44, nothing requested 2026-02-11T23:30:33.693+0000 7f415aefe640 20 osd.0 43 trim_maps: min=0 oldest_map=1 2026-02-11T23:30:33.693+0000 7f415aefe640 20 osd.0 43 handle_osd_map pg_num_history pg_num_history(e44 pg_nums {1={27=1},2={34=1},3={42=1}} deleted_pools ) 2026-02-11T23:30:33.693+0000 7f415aefe640 10 snap_mapper record_purged_snaps purged_snaps {44={}} 2026-02-11T23:30:33.693+0000 7f415aefe640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:33.693+0000 7f415aefe640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e44 maps [1~44] lci=[5,44] tlb=1) 2026-02-11T23:30:33.693+0000 7f41591a6640 10 osd.0 43 _committed_osd_maps 44..44 2026-02-11T23:30:33.693+0000 7f41591a6640 10 osd.0 43 advance to epoch 44 (<= last 44 <= newest_map 44) 2026-02-11T23:30:33.693+0000 7f41591a6640 20 osd.0 44 consume_map version 44 2026-02-11T23:30:33.693+0000 7f41591a6640 20 osd.0 43 await_reserved_maps epoch:44 2026-02-11T23:30:33.693+0000 7f41591a6640 20 osd.0 43 await_reserved_maps done 0 2026-02-11T23:30:33.693+0000 7f41591a6640 20 osd.0 44 consume_map 44 -- publish done 2026-02-11T23:30:33.693+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:30:33.693+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:2.0slot:0x37002c0 2026-02-11T23:30:33.693+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot 2.0 has no pg and waiting_for_split 2026-02-11T23:30:33.693+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3700dc0 2026-02-11T23:30:33.693+0000 7f41591a6640 20 osd.0 44 identify_splits_and_merges 1.0 e43 to e44 2026-02-11T23:30:33.693+0000 7f41591a6640 20 osd.0 44 identify_splits_and_merges 1.0 e43 to e44 pg_nums {27=1} 2026-02-11T23:30:33.693+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:33.693+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:33.693+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:33.693+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:33.693+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:33.693+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:33.693+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:33.693+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:33.693+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:33.693+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:33.693+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:33.693+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:33.693+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:33.693+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:33.693+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:33.693+0000 7f41591a6640 20 osd.0 44 prune_pg_created 2026-02-11T23:30:33.693+0000 7f41591a6640 10 osd.0:0.consume_map 44 (was 43) 2026-02-11T23:30:33.693+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 2.0 2026-02-11T23:30:33.693+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 2.0 empty, pruning 2026-02-11T23:30:33.693+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:33.693+0000 7f41591a6640 10 osd.0:1.consume_map 44 (was 43) 2026-02-11T23:30:33.693+0000 7f41591a6640 10 osd.0:2.consume_map 44 (was 43) 2026-02-11T23:30:33.693+0000 7f41591a6640 10 osd.0:3.consume_map 44 (was 43) 2026-02-11T23:30:33.693+0000 7f41591a6640 10 osd.0:4.consume_map 44 (was 43) 2026-02-11T23:30:33.693+0000 7f41591a6640 10 osd.0:5.consume_map 44 (was 43) 2026-02-11T23:30:33.693+0000 7f41591a6640 10 osd.0:6.consume_map 44 (was 43) 2026-02-11T23:30:33.693+0000 7f41591a6640 10 osd.0:7.consume_map 44 (was 43) 2026-02-11T23:30:33.693+0000 7f41591a6640 10 osd.0 44 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:33.693+0000 7f41591a6640 15 osd.0 44 enqueue_peering_evt 1.0 epoch_sent: 44 epoch_requested: 44 NullEvt 2026-02-11T23:30:33.693+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 44 epoch_requested: 44 NullEvt) class_id 2 prio 255 cost 10 e44) 2026-02-11T23:30:33.693+0000 7f41591a6640 7 osd.0 44 activate_map version 44 2026-02-11T23:30:33.693+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:33.693+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 44 epoch_requested: 44 NullEvt) class_id 2immediate prio 255 cost 10 e44) queued 2026-02-11T23:30:33.693+0000 7f4150194640 20 osd.0 44 agent_entry empty queue 2026-02-11T23:30:33.693+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:33.693+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 44 epoch_requested: 44 NullEvt) class_id 2immediate prio 255 cost 10 e44) pg 0x5946000 2026-02-11T23:30:33.693+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:33.693+0000 7f414b98b640 10 osd.0 pg_epoch: 43 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 44 2026-02-11T23:30:33.693+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 1.0 43 -> 44 2026-02-11T23:30:33.693+0000 7f414b98b640 10 osd.0 pg_epoch: 43 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:33.693+0000 7f414b98b640 10 osd.0 pg_epoch: 44 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:33.693+0000 7f414b98b640 10 osd.0 pg_epoch: 44 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:33.693+0000 7f414b98b640 20 osd.0 44 advance_pg new pool opts old pool opts 2026-02-11T23:30:33.693+0000 7f414b98b640 10 osd.0 pg_epoch: 44 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_activate_map: epoch range: 44..44 2026-02-11T23:30:33.693+0000 7f414b98b640 10 osd.0 pg_epoch: 44 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:33.693+0000 7f414b98b640 20 osd.0 pg_epoch: 44 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 30 while current is 44 2026-02-11T23:30:33.693+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:33.693+0000 7f414b98b640 10 osd.0 pg_epoch: 44 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 44 epoch_requested: 44 NullEvt 2026-02-11T23:30:33.693+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:33.693+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:34.139+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.1 127.0.0.1:0/3043114013 8 ==== osd_ping(ping e46 up_from 10 ping_stamp 2026-02-11T23:30:34.140342+0000/14.804087639s send_stamp 14.804087639s delta_ub -2.011302948s) ==== 2033+0+0 (crc 0 0 0) 0x4c9ad80 con 0x4aca000 2026-02-11T23:30:34.139+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.1 127.0.0.1:0/3043114013 8 ==== osd_ping(ping e46 up_from 10 ping_stamp 2026-02-11T23:30:34.140342+0000/14.804087639s send_stamp 14.804087639s delta_ub -2.011302948s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a480 con 0x4aca400 2026-02-11T23:30:34.139+0000 7f416cbfe640 20 osd.0 44 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011759281s,-2.011302948s]) 2026-02-11T23:30:34.139+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e44 up_from 5 ping_stamp 2026-02-11T23:30:34.140342+0000/14.804087639s send_stamp 16.815845490s delta_ub 2.011759281s) -- 0x4c9ac00 con 0x4aca000 2026-02-11T23:30:34.139+0000 7f416cbfe640 20 osd.0 44 get_con_osd_cluster to osd.1 from_epoch 44 2026-02-11T23:30:34.139+0000 7f416cbfe640 20 osd.0 44 get_nextmap_reserved map_reservations: {44=1} 2026-02-11T23:30:34.139+0000 7f416cbfe640 20 osd.0 44 release_map epoch: 44 2026-02-11T23:30:34.139+0000 7f416cbfe640 10 osd.0 44 maybe_share_map: con v2:127.0.0.1:6812/3043114013 updating session's projected_epoch from 43 to ping map epoch of 46 2026-02-11T23:30:34.139+0000 7f416cbfe640 10 osd.0 44 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 44 is not newer than session's projected_epoch of 46 2026-02-11T23:30:34.139+0000 7f416c3fd640 20 osd.0 44 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011784554s,-2.011302948s]) 2026-02-11T23:30:34.139+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e44 up_from 5 ping_stamp 2026-02-11T23:30:34.140342+0000/14.804087639s send_stamp 16.815872192s delta_ub 2.011784554s) -- 0x4c9b200 con 0x4aca400 2026-02-11T23:30:34.139+0000 7f416c3fd640 20 osd.0 44 get_con_osd_cluster to osd.1 from_epoch 44 2026-02-11T23:30:34.139+0000 7f416c3fd640 20 osd.0 44 get_nextmap_reserved map_reservations: {44=1} 2026-02-11T23:30:34.139+0000 7f416c3fd640 20 osd.0 44 release_map epoch: 44 2026-02-11T23:30:34.139+0000 7f416c3fd640 10 osd.0 44 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 44 is not newer than session's projected_epoch of 46 2026-02-11T23:30:34.139+0000 7f415aefe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 15 ==== osd_map(45..46 src has 1..46) ==== 1208+0+0 (unknown 0 0 0) 0x4a30e00 con 0x4acb000 2026-02-11T23:30:34.139+0000 7f415aefe640 20 osd.0 44 OSD::ms_dispatch: osd_map(45..46 src has 1..46) 2026-02-11T23:30:34.139+0000 7f415aefe640 20 osd.0 44 _dispatch 0x4a30e00 osd_map(45..46 src has 1..46) 2026-02-11T23:30:34.139+0000 7f415aefe640 3 osd.0 44 handle_osd_map epochs [45,46], i have 44, src has [1,46] 2026-02-11T23:30:34.139+0000 7f415aefe640 10 osd.0 44 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:34.139+0000 7f415aefe640 10 osd.0 44 handle_osd_map got inc map for epoch 45 2026-02-11T23:30:34.139+0000 7f415aefe640 20 osd.0 44 got_full_map 45, nothing requested 2026-02-11T23:30:34.139+0000 7f415aefe640 10 osd.0 44 handle_osd_map got inc map for epoch 46 2026-02-11T23:30:34.139+0000 7f415aefe640 20 osd.0 44 got_full_map 46, nothing requested 2026-02-11T23:30:34.139+0000 7f415aefe640 20 osd.0 44 trim_maps: min=0 oldest_map=1 2026-02-11T23:30:34.139+0000 7f415aefe640 20 osd.0 44 handle_osd_map pg_num_history pg_num_history(e46 pg_nums {1={27=1},2={34=1},3={42=1}} deleted_pools ) 2026-02-11T23:30:34.139+0000 7f415aefe640 10 snap_mapper record_purged_snaps purged_snaps {45={},46={}} 2026-02-11T23:30:34.139+0000 7f415aefe640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:34.139+0000 7f415aefe640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e46 maps [1~46] lci=[5,46] tlb=1) 2026-02-11T23:30:34.139+0000 7f41591a6640 10 osd.0 44 _committed_osd_maps 45..46 2026-02-11T23:30:34.139+0000 7f41591a6640 10 osd.0 44 advance to epoch 45 (<= last 46 <= newest_map 46) 2026-02-11T23:30:34.139+0000 7f41591a6640 10 osd.0 45 advance to epoch 46 (<= last 46 <= newest_map 46) 2026-02-11T23:30:34.139+0000 7f41591a6640 20 osd.0 46 consume_map version 46 2026-02-11T23:30:34.139+0000 7f41591a6640 20 osd.0 44 await_reserved_maps epoch:46 2026-02-11T23:30:34.139+0000 7f41591a6640 20 osd.0 44 await_reserved_maps done 0 2026-02-11T23:30:34.139+0000 7f41591a6640 20 osd.0 46 consume_map 46 -- publish done 2026-02-11T23:30:34.139+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 1 slots 2026-02-11T23:30:34.139+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3700dc0 2026-02-11T23:30:34.139+0000 7f41591a6640 20 osd.0 46 identify_splits_and_merges 1.0 e44 to e46 2026-02-11T23:30:34.139+0000 7f41591a6640 20 osd.0 46 identify_splits_and_merges 1.0 e44 to e46 pg_nums {27=1} 2026-02-11T23:30:34.139+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:34.139+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:34.139+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:34.139+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:34.139+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:34.139+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:34.139+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:34.139+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:34.139+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:34.139+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:34.139+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:34.139+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:34.139+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:34.139+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:34.139+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:34.139+0000 7f41591a6640 20 osd.0 46 prune_pg_created 2026-02-11T23:30:34.139+0000 7f41591a6640 10 osd.0:0.consume_map 46 (was 44) 2026-02-11T23:30:34.139+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:34.139+0000 7f41591a6640 10 osd.0:1.consume_map 46 (was 44) 2026-02-11T23:30:34.139+0000 7f41591a6640 10 osd.0:2.consume_map 46 (was 44) 2026-02-11T23:30:34.139+0000 7f41591a6640 10 osd.0:3.consume_map 46 (was 44) 2026-02-11T23:30:34.139+0000 7f41591a6640 10 osd.0:4.consume_map 46 (was 44) 2026-02-11T23:30:34.139+0000 7f41591a6640 10 osd.0:5.consume_map 46 (was 44) 2026-02-11T23:30:34.139+0000 7f41591a6640 10 osd.0:6.consume_map 46 (was 44) 2026-02-11T23:30:34.139+0000 7f41591a6640 10 osd.0:7.consume_map 46 (was 44) 2026-02-11T23:30:34.139+0000 7f41591a6640 10 osd.0 46 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:34.139+0000 7f41591a6640 15 osd.0 46 enqueue_peering_evt 1.0 epoch_sent: 46 epoch_requested: 46 NullEvt 2026-02-11T23:30:34.139+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 46 epoch_requested: 46 NullEvt) class_id 2 prio 255 cost 10 e46) 2026-02-11T23:30:34.139+0000 7f41591a6640 7 osd.0 46 activate_map version 46 2026-02-11T23:30:34.139+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:34.139+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 46 epoch_requested: 46 NullEvt) class_id 2immediate prio 255 cost 10 e46) queued 2026-02-11T23:30:34.139+0000 7f4150194640 20 osd.0 46 agent_entry empty queue 2026-02-11T23:30:34.139+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:34.139+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 46 epoch_requested: 46 NullEvt) class_id 2immediate prio 255 cost 10 e46) pg 0x5946000 2026-02-11T23:30:34.139+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:34.139+0000 7f414f993640 10 osd.0 pg_epoch: 44 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 45 2026-02-11T23:30:34.139+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 44 -> 45 2026-02-11T23:30:34.139+0000 7f414f993640 10 osd.0 pg_epoch: 44 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:34.139+0000 7f414f993640 10 osd.0 pg_epoch: 45 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:34.139+0000 7f414f993640 10 osd.0 pg_epoch: 45 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:34.139+0000 7f414f993640 20 osd.0 46 advance_pg new pool opts old pool opts 2026-02-11T23:30:34.139+0000 7f414f993640 10 osd.0 pg_epoch: 45 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 46 2026-02-11T23:30:34.139+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 45 -> 46 2026-02-11T23:30:34.139+0000 7f414f993640 10 osd.0 pg_epoch: 45 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:34.139+0000 7f414f993640 10 osd.0 pg_epoch: 46 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:34.139+0000 7f414f993640 10 osd.0 pg_epoch: 46 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:34.139+0000 7f414f993640 20 osd.0 46 advance_pg new pool opts old pool opts 2026-02-11T23:30:34.139+0000 7f414f993640 10 osd.0 pg_epoch: 46 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_activate_map: epoch range: 45..46 2026-02-11T23:30:34.139+0000 7f414f993640 10 osd.0 pg_epoch: 46 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:34.139+0000 7f414f993640 20 osd.0 pg_epoch: 46 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 30 while current is 46 2026-02-11T23:30:34.139+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:34.139+0000 7f414f993640 10 osd.0 pg_epoch: 46 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 46 epoch_requested: 46 NullEvt 2026-02-11T23:30:34.139+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:34.139+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:34.162+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.4 127.0.0.1:0/3331151400 2 ==== osd_ping(ping e46 up_from 23 ping_stamp 2026-02-11T23:30:34.164090+0000/7.002857685s send_stamp 7.002857685s delta_ub -9.836281776s) ==== 2033+0+0 (crc 0 0 0) 0x4c9b200 con 0x507a800 2026-02-11T23:30:34.162+0000 7f416c3fd640 20 osd.0 46 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836699486s,-9.836281776s]) 2026-02-11T23:30:34.162+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.4 127.0.0.1:0/3331151400 2 ==== osd_ping(ping e46 up_from 23 ping_stamp 2026-02-11T23:30:34.164090+0000/7.002857685s send_stamp 7.002857685s delta_ub -9.836281776s) ==== 2033+0+0 (crc 0 0 0) 0x4c9aa80 con 0x507a400 2026-02-11T23:30:34.162+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e46 up_from 5 ping_stamp 2026-02-11T23:30:34.164090+0000/7.002857685s send_stamp 16.839557648s delta_ub 9.836699486s) -- 0x4c9a480 con 0x507a800 2026-02-11T23:30:34.162+0000 7f416c3fd640 20 osd.0 46 get_con_osd_cluster to osd.4 from_epoch 46 2026-02-11T23:30:34.162+0000 7f416c3fd640 20 osd.0 46 get_nextmap_reserved map_reservations: {46=1} 2026-02-11T23:30:34.162+0000 7f416c3fd640 20 osd.0 46 release_map epoch: 46 2026-02-11T23:30:34.162+0000 7f416c3fd640 10 osd.0 46 maybe_share_map: con v2:127.0.0.1:6836/3331151400 updating session's projected_epoch from 35 to ping map epoch of 46 2026-02-11T23:30:34.162+0000 7f416c3fd640 10 osd.0 46 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 46 is not newer than session's projected_epoch of 46 2026-02-11T23:30:34.162+0000 7f416abfa640 20 osd.0 46 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836717606s,-9.836281776s]) 2026-02-11T23:30:34.162+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e46 up_from 5 ping_stamp 2026-02-11T23:30:34.164090+0000/7.002857685s send_stamp 16.839574814s delta_ub 9.836717606s) -- 0x4c9b500 con 0x507a400 2026-02-11T23:30:34.162+0000 7f416abfa640 20 osd.0 46 get_con_osd_cluster to osd.4 from_epoch 46 2026-02-11T23:30:34.162+0000 7f416abfa640 20 osd.0 46 get_nextmap_reserved map_reservations: {46=1} 2026-02-11T23:30:34.162+0000 7f416abfa640 20 osd.0 46 release_map epoch: 46 2026-02-11T23:30:34.162+0000 7f416abfa640 10 osd.0 46 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 46 is not newer than session's projected_epoch of 46 2026-02-11T23:30:34.218+0000 7f4162bea640 10 osd.0 46 tick_without_osd_lock 2026-02-11T23:30:34.218+0000 7f4162bea640 0 osd.0 46 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:34.218+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:34.218+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.508 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:34.218+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:34.219, recovery is active?:false restrictions:<....> 2026-02-11T23:30:34.218+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:34.218+0000 7f4162bea640 10 osd.0 46 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:34.218+0000 7f4162bea640 20 osd.0 46 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:34.218+0000 7f4162bea640 10 osd.0 46 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:34.218+0000 7f4162bea640 20 osd.0 46 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:34.326+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:30:34.326+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:30:34.326+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:04.327533+0000) 2026-02-11T23:30:34.380+0000 7f41633eb640 10 osd.0 46 tick 2026-02-11T23:30:34.380+0000 7f41633eb640 20 osd.0 46 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:30:34.598+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 3 ==== pg_query2(4.0 4.0 query(info 0'0 epoch_sent 49) e49/49) ==== 158+0+0 (unknown 0 0 0) 0x4fca960 con 0x507ac00 2026-02-11T23:30:34.598+0000 7f415aefe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 2 ==== osd_map(47..49 src has 1..49) ==== 1799+0+0 (unknown 0 0 0) 0x4a30700 con 0x507ac00 2026-02-11T23:30:34.598+0000 7f415aefe640 20 osd.0 46 OSD::ms_dispatch: osd_map(47..49 src has 1..49) 2026-02-11T23:30:34.598+0000 7f415aefe640 20 osd.0 46 _dispatch 0x4a30700 osd_map(47..49 src has 1..49) 2026-02-11T23:30:34.598+0000 7f416abfa640 15 osd.0 46 enqueue_peering_evt 4.0 epoch_sent: 49 epoch_requested: 49 MQuery 4.0 from 4 query_epoch 49 query: query(info 0'0 epoch_sent 49) 2026-02-11T23:30:34.598+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 49 epoch_requested: 49 MQuery 4.0 from 4 query_epoch 49 query: query(info 0'0 epoch_sent 49)) class_id 2 prio 255 cost 10 e49) 2026-02-11T23:30:34.598+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 (new) to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:34.598+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 49 epoch_requested: 49 MQuery 4.0 from 4 query_epoch 49 query: query(info 0'0 epoch_sent 49)) class_id 2immediate prio 255 cost 10 e49) queued 2026-02-11T23:30:34.598+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:34.598+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 49 epoch_requested: 49 MQuery 4.0 from 4 query_epoch 49 query: query(info 0'0 epoch_sent 49)) class_id 2immediate prio 255 cost 10 e49) pg 0 2026-02-11T23:30:34.598+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 map 49 > 46 2026-02-11T23:30:34.598+0000 7f414b98b640 20 osd.0 op_wq _add_slot_waiter 4.0 peering, item epoch is 49, will wait on OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 49 epoch_requested: 49 MQuery 4.0 from 4 query_epoch 49 query: query(info 0'0 epoch_sent 49)) class_id 2immediate prio 255 cost 10 e49) 2026-02-11T23:30:34.598+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:34.598+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:34.598+0000 7f415aefe640 3 osd.0 46 handle_osd_map epochs [47,49], i have 46, src has [1,49] 2026-02-11T23:30:34.598+0000 7f415aefe640 10 osd.0 46 handle_osd_map: received 0 full maps and 3 incremental maps 2026-02-11T23:30:34.598+0000 7f415aefe640 10 osd.0 46 handle_osd_map got inc map for epoch 47 2026-02-11T23:30:34.598+0000 7f415aefe640 20 osd.0 46 got_full_map 47, nothing requested 2026-02-11T23:30:34.598+0000 7f415aefe640 10 osd.0 46 handle_osd_map got inc map for epoch 48 2026-02-11T23:30:34.598+0000 7f415aefe640 20 osd.0 46 got_full_map 48, nothing requested 2026-02-11T23:30:34.598+0000 7f415aefe640 10 osd.0 46 handle_osd_map got inc map for epoch 49 2026-02-11T23:30:34.598+0000 7f415aefe640 20 osd.0 46 got_full_map 49, nothing requested 2026-02-11T23:30:34.598+0000 7f415aefe640 10 osd.0 46 _track_pools_and_pg_num_changes recording new pool 4 pg_num 1 2026-02-11T23:30:34.598+0000 7f415aefe640 20 osd.0 46 trim_maps: min=0 oldest_map=1 2026-02-11T23:30:34.598+0000 7f415aefe640 20 osd.0 46 handle_osd_map pg_num_history pg_num_history(e49 pg_nums {1={27=1},2={34=1},3={42=1},4={49=1}} deleted_pools ) 2026-02-11T23:30:34.598+0000 7f415aefe640 10 snap_mapper record_purged_snaps purged_snaps {47={},48={},49={}} 2026-02-11T23:30:34.598+0000 7f415aefe640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:34.598+0000 7f415aefe640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e49 maps [1~49] lci=[5,49] tlb=1) 2026-02-11T23:30:34.598+0000 7f41591a6640 10 osd.0 46 _committed_osd_maps 47..49 2026-02-11T23:30:34.598+0000 7f41591a6640 10 osd.0 46 advance to epoch 47 (<= last 49 <= newest_map 49) 2026-02-11T23:30:34.598+0000 7f41591a6640 10 osd.0 47 advance to epoch 48 (<= last 49 <= newest_map 49) 2026-02-11T23:30:34.598+0000 7f41591a6640 10 osd.0 48 advance to epoch 49 (<= last 49 <= newest_map 49) 2026-02-11T23:30:34.598+0000 7f41591a6640 20 osd.0 49 consume_map version 49 2026-02-11T23:30:34.598+0000 7f41591a6640 20 osd.0 46 await_reserved_maps epoch:49 2026-02-11T23:30:34.598+0000 7f41591a6640 20 osd.0 46 await_reserved_maps done 0 2026-02-11T23:30:34.598+0000 7f41591a6640 20 osd.0 49 consume_map 49 -- publish done 2026-02-11T23:30:34.598+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:30:34.598+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x37582c0 2026-02-11T23:30:34.598+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot 4.0 has no pg and waiting_for_split 2026-02-11T23:30:34.598+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3700dc0 2026-02-11T23:30:34.598+0000 7f41591a6640 20 osd.0 49 identify_splits_and_merges 1.0 e46 to e49 2026-02-11T23:30:34.598+0000 7f41591a6640 20 osd.0 49 identify_splits_and_merges 1.0 e46 to e49 pg_nums {27=1} 2026-02-11T23:30:34.598+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:34.598+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:34.598+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:34.598+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:34.598+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:34.598+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:34.598+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:34.598+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:34.598+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:34.598+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:34.598+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:34.598+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:34.598+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:34.598+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:34.598+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:34.598+0000 7f41591a6640 20 osd.0 49 prune_pg_created 2026-02-11T23:30:34.598+0000 7f41591a6640 10 osd.0:0.consume_map 49 (was 46) 2026-02-11T23:30:34.598+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:34.598+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 pending_peering first epoch 49 <= 49, requeueing 2026-02-11T23:30:34.598+0000 7f41591a6640 20 osd.0:0._wake_pg_slot _wake_pg_slot 4.0 to_process <> waiting <> waiting_peering {49=} 2026-02-11T23:30:34.598+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:34.598+0000 7f41591a6640 10 osd.0:1.consume_map 49 (was 46) 2026-02-11T23:30:34.598+0000 7f41591a6640 10 osd.0:2.consume_map 49 (was 46) 2026-02-11T23:30:34.598+0000 7f41591a6640 10 osd.0:3.consume_map 49 (was 46) 2026-02-11T23:30:34.598+0000 7f41591a6640 10 osd.0:4.consume_map 49 (was 46) 2026-02-11T23:30:34.598+0000 7f41591a6640 10 osd.0:5.consume_map 49 (was 46) 2026-02-11T23:30:34.598+0000 7f41591a6640 10 osd.0:6.consume_map 49 (was 46) 2026-02-11T23:30:34.598+0000 7f41591a6640 10 osd.0:7.consume_map 49 (was 46) 2026-02-11T23:30:34.598+0000 7f41591a6640 10 osd.0 49 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:34.598+0000 7f41591a6640 15 osd.0 49 enqueue_peering_evt 1.0 epoch_sent: 49 epoch_requested: 49 NullEvt 2026-02-11T23:30:34.598+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:34.598+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 49 epoch_requested: 49 MQuery 4.0 from 4 query_epoch 49 query: query(info 0'0 epoch_sent 49)) class_id 2immediate prio 255 cost 10 e49) queued 2026-02-11T23:30:34.598+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 49 epoch_requested: 49 NullEvt) class_id 2 prio 255 cost 10 e49) 2026-02-11T23:30:34.598+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:34.598+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 49 epoch_requested: 49 MQuery 4.0 from 4 query_epoch 49 query: query(info 0'0 epoch_sent 49)) class_id 2immediate prio 255 cost 10 e49) pg 0 2026-02-11T23:30:34.598+0000 7f414b98b640 10 osd.0 49 handle_pg_query_nopg 4.0 2026-02-11T23:30:34.598+0000 7f414b98b640 10 osd.0 49 pg 4.0 dne 2026-02-11T23:30:34.598+0000 7f414b98b640 20 osd.0 49 get_con_osd_cluster to osd.4 from_epoch 49 2026-02-11T23:30:34.598+0000 7f414b98b640 20 osd.0 49 get_nextmap_reserved map_reservations: {49=1} 2026-02-11T23:30:34.598+0000 7f414b98b640 20 osd.0 49 release_map epoch: 49 2026-02-11T23:30:34.598+0000 7f414b98b640 10 osd.0 49 maybe_share_map: con v2:127.0.0.1:6836/3331151400 map epoch 46 -> 49 (shared) 2026-02-11T23:30:34.598+0000 7f414b98b640 10 osd.0 49 send_incremental_map epoch range: (47, 49] to v2:127.0.0.1:6836/3331151400 2026-02-11T23:30:34.598+0000 7f414b98b640 10 osd.0 49 add_map_inc_bl 47 574 bytes 2026-02-11T23:30:34.598+0000 7f414b98b640 10 osd.0 49 add_map_inc_bl 48 574 bytes 2026-02-11T23:30:34.598+0000 7f414b98b640 10 osd.0 49 add_map_inc_bl 49 591 bytes 2026-02-11T23:30:34.598+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- osd_map(47..49 src has 1..49) -- 0x5954fc0 con 0x507ac00 2026-02-11T23:30:34.598+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_notify2(4.0 4.0 (query:49 sent:49 4.0( DNE empty local-lis/les=0/0 n=0 ec=0/0 lis/c=0/0 les/c/f=0/0/0 sis=0) ([0,0] all_participants={} intervals=[])) e49/49) -- 0x373b100 con 0x507ac00 2026-02-11T23:30:34.598+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:34.598+0000 7f41591a6640 7 osd.0 49 activate_map version 49 2026-02-11T23:30:34.598+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:34.598+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 49 epoch_requested: 49 NullEvt) class_id 2immediate prio 255 cost 10 e49) queued 2026-02-11T23:30:34.598+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:34.598+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 49 epoch_requested: 49 NullEvt) class_id 2immediate prio 255 cost 10 e49) pg 0x5946000 2026-02-11T23:30:34.598+0000 7f4150194640 20 osd.0 49 agent_entry empty queue 2026-02-11T23:30:34.598+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:34.598+0000 7f414f993640 10 osd.0 pg_epoch: 46 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 47 2026-02-11T23:30:34.598+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 46 -> 47 2026-02-11T23:30:34.598+0000 7f414f993640 10 osd.0 pg_epoch: 46 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:34.598+0000 7f414f993640 10 osd.0 pg_epoch: 47 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:34.598+0000 7f414f993640 10 osd.0 pg_epoch: 47 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:34.598+0000 7f414f993640 20 osd.0 49 advance_pg new pool opts old pool opts 2026-02-11T23:30:34.598+0000 7f414f993640 10 osd.0 pg_epoch: 47 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 48 2026-02-11T23:30:34.598+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 47 -> 48 2026-02-11T23:30:34.598+0000 7f414f993640 10 osd.0 pg_epoch: 47 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:34.598+0000 7f414f993640 10 osd.0 pg_epoch: 48 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:34.598+0000 7f414f993640 10 osd.0 pg_epoch: 48 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:34.598+0000 7f414f993640 20 osd.0 49 advance_pg new pool opts old pool opts 2026-02-11T23:30:34.598+0000 7f414f993640 10 osd.0 pg_epoch: 48 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 49 2026-02-11T23:30:34.598+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 48 -> 49 2026-02-11T23:30:34.598+0000 7f414f993640 10 osd.0 pg_epoch: 48 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:34.598+0000 7f414f993640 10 osd.0 pg_epoch: 49 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:34.598+0000 7f414f993640 10 osd.0 pg_epoch: 49 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:34.598+0000 7f414f993640 20 osd.0 49 advance_pg new pool opts old pool opts 2026-02-11T23:30:34.598+0000 7f414f993640 10 osd.0 pg_epoch: 49 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_activate_map: epoch range: 47..49 2026-02-11T23:30:34.598+0000 7f414f993640 10 osd.0 pg_epoch: 49 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:34.598+0000 7f414f993640 20 osd.0 pg_epoch: 49 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 30 while current is 49 2026-02-11T23:30:34.598+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:34.598+0000 7f414f993640 10 osd.0 pg_epoch: 49 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 49 epoch_requested: 49 NullEvt 2026-02-11T23:30:34.598+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:34.598+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:34.699+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 4 ==== PGlog(4.0 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 23.539068222s int 16.000000000s) e50/49) ==== 1172+0+0 (unknown 0 0 0) 0x5a94800 con 0x507ac00 2026-02-11T23:30:34.699+0000 7f416abfa640 15 osd.0 49 enqueue_peering_evt 4.0 epoch_sent: 50 epoch_requested: 49 MLogRec from 4 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 23.539068222s int 16.000000000s) +create_info 2026-02-11T23:30:34.699+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 50 epoch_requested: 49 MLogRec from 4 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 23.539068222s int 16.000000000s) +create_info) class_id 2 prio 255 cost 10 e50) 2026-02-11T23:30:34.699+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:34.699+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 50 epoch_requested: 49 MLogRec from 4 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 23.539068222s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e50) queued 2026-02-11T23:30:34.699+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:34.699+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 50 epoch_requested: 49 MLogRec from 4 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 23.539068222s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e50) pg 0 2026-02-11T23:30:34.699+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 map 50 > 49 2026-02-11T23:30:34.699+0000 7f414f993640 20 osd.0 op_wq _add_slot_waiter 4.0 peering, item epoch is 50, will wait on OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 50 epoch_requested: 49 MLogRec from 4 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 23.539068222s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e50) 2026-02-11T23:30:34.699+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:34.699+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:35.196+0000 7f4162bea640 10 osd.0 49 tick_without_osd_lock 2026-02-11T23:30:35.196+0000 7f4162bea640 0 osd.0 49 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:35.196+0000 7f4162bea640 20 osd.0 49 tick_without_osd_lock max_waiting_epoch 50, requesting new map 2026-02-11T23:30:35.196+0000 7f4162bea640 10 monclient: _renew_subs 2026-02-11T23:30:35.196+0000 7f4162bea640 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:30:35.196+0000 7f4162bea640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> v2:127.0.0.1:7180/0 -- mon_subscribe({osdmap=50}) -- 0x3814340 con 0x4acbc00 2026-02-11T23:30:35.196+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:35.196+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.508 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:35.196+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:35.197, recovery is active?:false restrictions:<....> 2026-02-11T23:30:35.196+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:35.196+0000 7f4162bea640 10 osd.0 49 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:35.196+0000 7f4162bea640 20 osd.0 49 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:35.196+0000 7f4162bea640 10 osd.0 49 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:35.196+0000 7f4162bea640 20 osd.0 49 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:35.196+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 31 ==== osd_map(50..50 src has 1..50) ==== 296+0+0 (crc 0 0 0) 0x59541c0 con 0x4acbc00 2026-02-11T23:30:35.196+0000 7f415cf02640 20 osd.0 49 OSD::ms_dispatch: osd_map(50..50 src has 1..50) 2026-02-11T23:30:35.196+0000 7f415cf02640 20 osd.0 49 _dispatch 0x59541c0 osd_map(50..50 src has 1..50) 2026-02-11T23:30:35.196+0000 7f415cf02640 3 osd.0 49 handle_osd_map epochs [50,50], i have 49, src has [1,50] 2026-02-11T23:30:35.196+0000 7f415cf02640 10 osd.0 49 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:30:35.196+0000 7f415cf02640 10 osd.0 49 handle_osd_map got inc map for epoch 50 2026-02-11T23:30:35.196+0000 7f415cf02640 20 osd.0 49 got_full_map 50, nothing requested 2026-02-11T23:30:35.196+0000 7f415cf02640 20 osd.0 49 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:35.196+0000 7f415cf02640 20 osd.0 49 handle_osd_map pg_num_history pg_num_history(e50 pg_nums {1={27=1},2={34=1},3={42=1},4={49=1}} deleted_pools ) 2026-02-11T23:30:35.196+0000 7f415cf02640 10 snap_mapper record_purged_snaps purged_snaps {50={}} 2026-02-11T23:30:35.196+0000 7f415cf02640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:35.196+0000 7f415cf02640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e50 maps [1~50] lci=[5,50] tlb=1) 2026-02-11T23:30:35.196+0000 7f41591a6640 10 osd.0 49 _committed_osd_maps 50..50 2026-02-11T23:30:35.196+0000 7f41591a6640 10 osd.0 49 advance to epoch 50 (<= last 50 <= newest_map 50) 2026-02-11T23:30:35.196+0000 7f41591a6640 20 osd.0 50 consume_map version 50 2026-02-11T23:30:35.196+0000 7f41591a6640 20 osd.0 49 await_reserved_maps epoch:50 2026-02-11T23:30:35.196+0000 7f41591a6640 20 osd.0 49 await_reserved_maps done 0 2026-02-11T23:30:35.196+0000 7f41591a6640 20 osd.0 50 consume_map 50 -- publish done 2026-02-11T23:30:35.196+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:30:35.197+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x37582c0 2026-02-11T23:30:35.197+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot 4.0 has no pg and waiting_for_split 2026-02-11T23:30:35.197+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3700dc0 2026-02-11T23:30:35.197+0000 7f41591a6640 20 osd.0 50 identify_splits_and_merges 1.0 e49 to e50 2026-02-11T23:30:35.197+0000 7f41591a6640 20 osd.0 50 identify_splits_and_merges 1.0 e49 to e50 pg_nums {27=1} 2026-02-11T23:30:35.197+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:35.197+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:35.197+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:35.197+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:35.197+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:35.197+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:35.197+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:35.197+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:35.197+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:35.197+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:35.197+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:35.197+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:35.197+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:35.197+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:35.197+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:35.197+0000 7f41591a6640 20 osd.0 50 prune_pg_created 2026-02-11T23:30:35.197+0000 7f41591a6640 10 osd.0:0.consume_map 50 (was 49) 2026-02-11T23:30:35.197+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:35.197+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 pending_peering first epoch 50 <= 50, requeueing 2026-02-11T23:30:35.197+0000 7f41591a6640 20 osd.0:0._wake_pg_slot _wake_pg_slot 4.0 to_process <> waiting <> waiting_peering {50=} 2026-02-11T23:30:35.197+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:35.197+0000 7f41591a6640 10 osd.0:1.consume_map 50 (was 49) 2026-02-11T23:30:35.197+0000 7f41591a6640 10 osd.0:2.consume_map 50 (was 49) 2026-02-11T23:30:35.197+0000 7f41591a6640 10 osd.0:3.consume_map 50 (was 49) 2026-02-11T23:30:35.197+0000 7f41591a6640 10 osd.0:4.consume_map 50 (was 49) 2026-02-11T23:30:35.197+0000 7f41591a6640 10 osd.0:5.consume_map 50 (was 49) 2026-02-11T23:30:35.197+0000 7f41591a6640 10 osd.0:6.consume_map 50 (was 49) 2026-02-11T23:30:35.197+0000 7f41591a6640 10 osd.0:7.consume_map 50 (was 49) 2026-02-11T23:30:35.197+0000 7f41591a6640 10 osd.0 50 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:35.197+0000 7f41591a6640 15 osd.0 50 enqueue_peering_evt 1.0 epoch_sent: 50 epoch_requested: 50 NullEvt 2026-02-11T23:30:35.197+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:35.197+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 50 epoch_requested: 50 NullEvt) class_id 2 prio 255 cost 10 e50) 2026-02-11T23:30:35.197+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 50 epoch_requested: 49 MLogRec from 4 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 23.539068222s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e50) queued 2026-02-11T23:30:35.197+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:35.197+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 50 epoch_requested: 49 MLogRec from 4 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 23.539068222s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e50) pg 0 2026-02-11T23:30:35.197+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 no pg, should create on OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 50 epoch_requested: 49 MLogRec from 4 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 23.539068222s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e50) 2026-02-11T23:30:35.197+0000 7f414f993640 10 osd.0 50 _make_pg 4.0 2026-02-11T23:30:35.197+0000 7f414f993640 10 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:4, shard:-1, shard_prefix: , prefixes: {}] SnapMapper 2026-02-11T23:30:35.197+0000 7f414f993640 20 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:4, shard:-1, shard_prefix: , prefixes: {}] update_bits new_bits: 0 2026-02-11T23:30:35.197+0000 7f414f993640 20 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:4, shard:-1, shard_prefix: , prefixes: {"0000000000000004."}] update_bits prefix updated 2026-02-11T23:30:35.197+0000 7f414f993640 10 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:4, shard:-1, shard_prefix: , prefixes: {"0000000000000004."}] reset_prefix_itr::from <0> to ::update_bits 2026-02-11T23:30:35.197+0000 7f414f993640 5 osd.0 pg_epoch: 49 pg[4.0(unlocked)] enter Initial 2026-02-11T23:30:35.197+0000 7f414f993640 20 osd.0 pg_epoch: 49 pg[4.0(unlocked)] enter NotTrimming 2026-02-11T23:30:35.197+0000 7f414f993640 10 osd.0 pg_epoch: 49 pg[4.0(unlocked)] scrubber: FSM: -- state -->> NotActive 2026-02-11T23:30:35.197+0000 7f414f993640 10 osd.0 pg_epoch: 49 pg[4.0( DNE empty local-lis/les=0/0 n=0 ec=0/0 lis/c=0/0 les/c/f=0/0/0 sis=0) [] r=-1 lpr=0 crt=0'0 unknown mbc={}] PeeringState::init init role 1 up [4,0,3] acting [4,0,3] history ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49 past_intervals ([0,0] all_participants={} intervals=[]) 2026-02-11T23:30:35.197+0000 7f414f993640 20 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=0 crt=0'0 unknown mbc={}] PeeringState::on_new_interval 2026-02-11T23:30:35.197+0000 7f414f993640 20 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=0 crt=0'0 unknown mbc={}] PeeringState::on_new_interval upacting_features 0x3f0ffffffffdffff from [4,0,3]+[4,0,3] 2026-02-11T23:30:35.197+0000 7f414f993640 20 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=0 crt=0'0 unknown mbc={}] PeeringState::on_new_interval checking missing set deletes flag. missing = missing(0 may_include_deletes = 1) 2026-02-11T23:30:35.197+0000 7f414f993640 10 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=0 crt=0'0 unknown mbc={}] PeeringState::init_hb_stamps now [0x56859d0] 2026-02-11T23:30:35.197+0000 7f414f993640 10 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=0 crt=0'0 unknown mbc={}] PeeringState::on_new_interval prior_readable_until_ub 0.000000000s (mnow 17.873868942s + 0.000000000s) 2026-02-11T23:30:35.197+0000 7f414f993640 10 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=0 crt=0'0 unknown mbc={}] cancel_recovery 2026-02-11T23:30:35.197+0000 7f414f993640 10 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=0 crt=0'0 unknown mbc={}] clear_recovery_state 2026-02-11T23:30:35.197+0000 7f414f993640 15 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=0 crt=0'0 unknown mbc={}] _clear_recovery_state 2026-02-11T23:30:35.197+0000 7f414f993640 10 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=0 crt=0'0 unknown mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:35.197+0000 7f414f993640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:35.197+0000 7f414f993640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:35.197+0000 7f414f993640 6 write_log_and_missing with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, writeout_from: 4294967295'18446744073709551615, trimmed: , trimmed_dups: , clear_divergent_priors: 0 2026-02-11T23:30:35.197+0000 7f414f993640 10 _write_log_and_missing clearing up to 0'0 dirty_to_dups=0'0 dirty_from_dups=4294967295'18446744073709551615 write_from_dups=4294967295'18446744073709551615 trimmed_dups.size()=0 2026-02-11T23:30:35.197+0000 7f414f993640 10 _write_log_and_missing going to encode log.dups.size()=0 2026-02-11T23:30:35.197+0000 7f414f993640 10 _write_log_and_missing 1st round encoded log.dups.size()=0 2026-02-11T23:30:35.197+0000 7f414f993640 10 _write_log_and_missing 2st round encoded log.dups.size()=0 2026-02-11T23:30:35.197+0000 7f414f993640 10 end of _write_log_and_missing 2026-02-11T23:30:35.197+0000 7f414f993640 10 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=0 crt=0'0 unknown mbc={}] handle_initialize 2026-02-11T23:30:35.197+0000 7f414f993640 5 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=0 crt=0'0 unknown mbc={}] exit Initial 0.000089 0 0.000000 2026-02-11T23:30:35.197+0000 7f414f993640 5 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=0 crt=0'0 unknown mbc={}] enter Reset 2026-02-11T23:30:35.197+0000 7f414f993640 20 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=0 crt=0'0 unknown mbc={}] PeeringState::set_last_peering_reset set_last_peering_reset 49 2026-02-11T23:30:35.197+0000 7f414f993640 10 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 unknown mbc={}] PeeringState::set_last_peering_reset Clearing blocked outgoing recovery messages 2026-02-11T23:30:35.197+0000 7f414f993640 10 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 unknown mbc={}] PeeringState::set_last_peering_reset Not blocking outgoing recovery messages 2026-02-11T23:30:35.197+0000 7f414f993640 10 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 unknown mbc={}] handle_activate_map: epoch range: 49..49 2026-02-11T23:30:35.197+0000 7f414f993640 10 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 unknown mbc={}] PeeringState::activate_map 2026-02-11T23:30:35.197+0000 7f414f993640 5 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 unknown mbc={}] exit Reset 0.000018 1 0.000017 2026-02-11T23:30:35.197+0000 7f414f993640 5 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 unknown mbc={}] enter Started 2026-02-11T23:30:35.197+0000 7f414f993640 5 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 unknown mbc={}] enter Start 2026-02-11T23:30:35.197+0000 7f414f993640 1 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 unknown mbc={}] state: transitioning to Stray 2026-02-11T23:30:35.197+0000 7f414f993640 5 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 unknown mbc={}] exit Start 0.000003 0 0.000000 2026-02-11T23:30:35.197+0000 7f414f993640 5 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 unknown mbc={}] enter Started/Stray 2026-02-11T23:30:35.197+0000 7f414f993640 20 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 unknown mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 49 while current is 49 2026-02-11T23:30:35.197+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:35.197+0000 7f414f993640 10 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 unknown mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:30:35.197+0000 7f414f993640 10 osd.0 50 handle_pg_create_info new pg pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 unknown mbc={}] 2026-02-11T23:30:35.197+0000 7f414f993640 10 osd.0:0._attach_pg 4.0 0x5978000 2026-02-11T23:30:35.197+0000 7f414f993640 20 osd.0:0._wake_pg_slot _wake_pg_slot 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:35.197+0000 7f414f993640 20 osd.0 50 identify_splits_and_merges 4.0 e49 to e50 2026-02-11T23:30:35.197+0000 7f414f993640 20 osd.0 50 identify_splits_and_merges 4.0 e49 to e50 pg_nums {49=1} 2026-02-11T23:30:35.197+0000 7f414f993640 10 osd.0:0._prime_splits 2026-02-11T23:30:35.197+0000 7f414f993640 10 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 unknown mbc={}] handle_advance_map: 50 2026-02-11T23:30:35.197+0000 7f414f993640 20 osd.0:0.update_pg_epoch 4.0 49 -> 50 2026-02-11T23:30:35.197+0000 7f41591a6640 7 osd.0 50 activate_map version 50 2026-02-11T23:30:35.197+0000 7f414f993640 10 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 unknown mbc={}] PeeringState::advance_map handle_advance_map [4,0,3]/[4,0,3] -- 4/4 2026-02-11T23:30:35.197+0000 7f414f993640 10 osd.0 pg_epoch: 50 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 unknown mbc={}] state: Started advmap 2026-02-11T23:30:35.197+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:35.197+0000 7f414f993640 10 osd.0 pg_epoch: 50 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 unknown mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:35.197+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 50 epoch_requested: 50 NullEvt) class_id 2immediate prio 255 cost 10 e50) queued 2026-02-11T23:30:35.197+0000 7f414f993640 20 osd.0 50 advance_pg new pool opts old pool opts 2026-02-11T23:30:35.197+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:35.197+0000 7f4150194640 20 osd.0 50 agent_entry empty queue 2026-02-11T23:30:35.197+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 50 epoch_requested: 50 NullEvt) class_id 2immediate prio 255 cost 10 e50) pg 0x5946000 2026-02-11T23:30:35.197+0000 7f414f993640 10 osd.0 pg_epoch: 50 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 unknown mbc={}] handle_activate_map: epoch range: 50..50 2026-02-11T23:30:35.197+0000 7f414f993640 10 osd.0 pg_epoch: 50 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 unknown mbc={}] PeeringState::activate_map 2026-02-11T23:30:35.197+0000 7f414b98b640 10 osd.0 pg_epoch: 49 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 50 2026-02-11T23:30:35.197+0000 7f414f993640 20 osd.0 pg_epoch: 50 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 unknown mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 49 while current is 50 2026-02-11T23:30:35.197+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 1.0 49 -> 50 2026-02-11T23:30:35.197+0000 7f414b98b640 10 osd.0 pg_epoch: 49 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:35.197+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:35.197+0000 7f414b98b640 10 osd.0 pg_epoch: 50 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:35.197+0000 7f414f993640 10 osd.0 pg_epoch: 50 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 unknown mbc={}] do_peering_event: epoch_sent: 50 epoch_requested: 49 MLogRec from 4 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 23.539068222s int 16.000000000s) +create_info 2026-02-11T23:30:35.197+0000 7f414b98b640 10 osd.0 pg_epoch: 50 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:35.197+0000 7f414f993640 10 osd.0 pg_epoch: 50 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 unknown mbc={}] state: got info+log from osd.4 4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) log((0'0,0'0], crt=0'0) 2026-02-11T23:30:35.197+0000 7f414b98b640 20 osd.0 50 advance_pg new pool opts old pool opts 2026-02-11T23:30:35.197+0000 7f414f993640 10 merge_log log((0'0,0'0], crt=0'0) from osd.4 into log((0'0,0'0], crt=0'0) 2026-02-11T23:30:35.197+0000 7f414f993640 5 merge_log_dups log.dups.size()=0olog.dups.size()=0 2026-02-11T23:30:35.197+0000 7f414f993640 5 end of merge_log_dups changed=0 log.dups.size()=0 olog.dups.size()=0 2026-02-11T23:30:35.197+0000 7f414b98b640 10 osd.0 pg_epoch: 50 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_activate_map: epoch range: 50..50 2026-02-11T23:30:35.197+0000 7f414f993640 10 merge_log result log((0'0,0'0], crt=0'0) missing(0 may_include_deletes = 1) changed=0 2026-02-11T23:30:35.197+0000 7f414f993640 10 osd.0 pg_epoch: 50 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 unknown mbc={}] PeeringState::proc_lease pg_lease(ru 0.000000000s ub 23.539068222s int 16.000000000s) 2026-02-11T23:30:35.197+0000 7f414f993640 20 osd.0 pg_epoch: 50 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 unknown mbc={}] PeeringState::proc_lease peer_clock_delta_lb -9.836717606s -> ruub 33.375785828s 2026-02-11T23:30:35.197+0000 7f414b98b640 10 osd.0 pg_epoch: 50 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:35.197+0000 7f414f993640 20 osd.0 pg_epoch: 50 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 unknown mbc={}] PeeringState::proc_lease readable_until_ub now 33.375785828s 2026-02-11T23:30:35.197+0000 7f414b98b640 20 osd.0 pg_epoch: 50 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 30 while current is 50 2026-02-11T23:30:35.197+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:35.197+0000 7f414f993640 5 osd.0 pg_epoch: 50 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 unknown mbc={}] exit Started/Stray 0.000112 3 0.000019 2026-02-11T23:30:35.197+0000 7f414f993640 5 osd.0 pg_epoch: 50 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 unknown mbc={}] enter Started/ReplicaActive 2026-02-11T23:30:35.197+0000 7f414b98b640 10 osd.0 pg_epoch: 50 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 50 epoch_requested: 50 NullEvt 2026-02-11T23:30:35.197+0000 7f414f993640 5 osd.0 pg_epoch: 50 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 unknown mbc={}] enter Started/ReplicaActive/RepNotRecovering 2026-02-11T23:30:35.197+0000 7f414f993640 10 osd.0 pg_epoch: 50 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 unknown mbc={}] state: In ReplicaActive, about to call activate 2026-02-11T23:30:35.197+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:35.197+0000 7f414f993640 10 osd.0 pg_epoch: 50 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 unknown mbc={}] PeeringState::activate activate - no missing, moving last_complete 0'0 -> 0'0 2026-02-11T23:30:35.197+0000 7f414f993640 10 osd.0 pg_epoch: 50 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 unknown mbc={}] scrubber: on_replica_activate 2026-02-11T23:30:35.197+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:35.197+0000 7f414f993640 20 scrubberFSM event: --vvvv---- ReplicaActivate 2026-02-11T23:30:35.197+0000 7f414f993640 10 osd.0 pg_epoch: 50 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 unknown mbc={}] scrubber: FSM: -- state -->> ReplicaActive 2026-02-11T23:30:35.197+0000 7f414f993640 10 osd.0 pg_epoch: 50 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 unknown mbc={}] scrubber: FSM: -- state -->> ReplicaActive/ReplicaIdle 2026-02-11T23:30:35.197+0000 7f414f993640 20 scrubberFSM event: --^^^^---- ReplicaActivate 2026-02-11T23:30:35.197+0000 7f414f993640 10 osd.0 pg_epoch: 50 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 unknown mbc={}] state: Activate Finished 2026-02-11T23:30:35.197+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:35.197+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:35.197+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:35.197+0000 7f414f993640 15 osd.0 50 enqueue_peering_evt 4.0 epoch_sent: 50 epoch_requested: 50 ActivateCommitted from 50 processed at 50 2026-02-11T23:30:35.197+0000 7f414f993640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 50 epoch_requested: 50 ActivateCommitted from 50 processed at 50) class_id 2 prio 255 cost 10 e50) 2026-02-11T23:30:35.197+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:35.197+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 50 epoch_requested: 50 ActivateCommitted from 50 processed at 50) class_id 2immediate prio 255 cost 10 e50) queued 2026-02-11T23:30:35.197+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:35.197+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 50 epoch_requested: 50 ActivateCommitted from 50 processed at 50) class_id 2immediate prio 255 cost 10 e50) pg 0x5978000 2026-02-11T23:30:35.197+0000 7f414f993640 10 osd.0 pg_epoch: 50 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 unknown mbc={}] do_peering_event: epoch_sent: 50 epoch_requested: 50 ActivateCommitted from 50 processed at 50 2026-02-11T23:30:35.197+0000 7f414f993640 10 osd.0 pg_epoch: 50 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 unknown mbc={}] state: react 50 telling primary 2026-02-11T23:30:35.197+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:35.197+0000 7f414f993640 20 osd.0 50 get_con_osd_cluster to osd.4 from_epoch 50 2026-02-11T23:30:35.197+0000 7f414f993640 20 osd.0 50 get_nextmap_reserved map_reservations: {50=1} 2026-02-11T23:30:35.197+0000 7f414f993640 20 osd.0 50 release_map epoch: 50 2026-02-11T23:30:35.197+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:35.197+0000 7f414f993640 10 osd.0 50 maybe_share_map: con v2:127.0.0.1:6836/3331151400 map epoch 49 -> 50 (shared) 2026-02-11T23:30:35.197+0000 7f414f993640 10 osd.0 50 send_incremental_map epoch range: (50, 50] to v2:127.0.0.1:6836/3331151400 2026-02-11T23:30:35.197+0000 7f414f993640 10 osd.0 50 add_map_inc_bl 50 252 bytes 2026-02-11T23:30:35.197+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- osd_map(50..50 src has 1..50) -- 0x5954380 con 0x507ac00 2026-02-11T23:30:35.197+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_info2(4.0 4.0 4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/0 les/c/f=50/0/0 sis=49) e50/50) -- 0x5a8c700 con 0x507ac00 2026-02-11T23:30:35.197+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:35.326+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:30:35.326+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:30:35.326+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:05.327553+0000) 2026-02-11T23:30:35.369+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 5 ==== pg_lease(4.0 pg_lease(ru 23.539068222s ub 24.209175110s int 16.000000000s) e50/50) ==== 58+0+0 (unknown 0 0 0) 0x4c9b500 con 0x507ac00 2026-02-11T23:30:35.369+0000 7f416abfa640 15 osd.0 50 enqueue_peering_evt 4.0 epoch_sent: 50 epoch_requested: 50 MLease epoch 50 from osd.4 pg_lease(ru 23.539068222s ub 24.209175110s int 16.000000000s) 2026-02-11T23:30:35.369+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 50 epoch_requested: 50 MLease epoch 50 from osd.4 pg_lease(ru 23.539068222s ub 24.209175110s int 16.000000000s)) class_id 2 prio 255 cost 10 e50) 2026-02-11T23:30:35.369+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:35.369+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 50 epoch_requested: 50 MLease epoch 50 from osd.4 pg_lease(ru 23.539068222s ub 24.209175110s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e50) queued 2026-02-11T23:30:35.369+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:35.369+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 50 epoch_requested: 50 MLease epoch 50 from osd.4 pg_lease(ru 23.539068222s ub 24.209175110s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e50) pg 0x5978000 2026-02-11T23:30:35.369+0000 7f414b98b640 10 osd.0 pg_epoch: 50 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 50 epoch_requested: 50 MLease epoch 50 from osd.4 pg_lease(ru 23.539068222s ub 24.209175110s int 16.000000000s) 2026-02-11T23:30:35.369+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:35.369+0000 7f414b98b640 10 osd.0 pg_epoch: 50 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 active mbc={}] PeeringState::proc_lease pg_lease(ru 23.539068222s ub 24.209175110s int 16.000000000s) 2026-02-11T23:30:35.369+0000 7f414b98b640 20 osd.0 pg_epoch: 50 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 active mbc={}] PeeringState::proc_lease peer_clock_delta_ub -9.836281776s -> ru 33.375350952s 2026-02-11T23:30:35.369+0000 7f414b98b640 20 osd.0 pg_epoch: 50 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 active mbc={}] PeeringState::proc_lease readable_until now 33.375350952s 2026-02-11T23:30:35.369+0000 7f414b98b640 20 osd.0 pg_epoch: 50 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 active mbc={}] PeeringState::proc_lease peer_clock_delta_lb -9.836717606s -> ruub 34.045890808s 2026-02-11T23:30:35.369+0000 7f414b98b640 20 osd.0 pg_epoch: 50 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 active mbc={}] PeeringState::proc_lease readable_until_ub now 34.045890808s 2026-02-11T23:30:35.369+0000 7f414b98b640 20 osd.0 50 get_con_osd_cluster to osd.4 from_epoch 50 2026-02-11T23:30:35.369+0000 7f414b98b640 20 osd.0 50 get_nextmap_reserved map_reservations: {50=1} 2026-02-11T23:30:35.369+0000 7f414b98b640 20 osd.0 50 release_map epoch: 50 2026-02-11T23:30:35.369+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_lease_ack(4.0 pg_lease_ack(ruub 24.209175110s) e50/50) -- 0x37009a0 con 0x507ac00 2026-02-11T23:30:35.369+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:35.369+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:35.369+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 6 ==== pg_info2(4.0 4.0 4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/0 les/c/f=50/0/0 sis=49) e50/50) ==== 1126+0+0 (unknown 0 0 0) 0x5a8c700 con 0x507ac00 2026-02-11T23:30:35.369+0000 7f416abfa640 15 osd.0 50 enqueue_peering_evt 4.0 epoch_sent: 50 epoch_requested: 50 MInfoRec from 4 info: 4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/0 les/c/f=50/0/0 sis=49) pg_lease(ru 23.539068222s ub 24.209175110s int 16.000000000s) 2026-02-11T23:30:35.369+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 50 epoch_requested: 50 MInfoRec from 4 info: 4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/0 les/c/f=50/0/0 sis=49) pg_lease(ru 23.539068222s ub 24.209175110s int 16.000000000s)) class_id 2 prio 255 cost 10 e50) 2026-02-11T23:30:35.369+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:35.369+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 50 epoch_requested: 50 MInfoRec from 4 info: 4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/0 les/c/f=50/0/0 sis=49) pg_lease(ru 23.539068222s ub 24.209175110s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e50) queued 2026-02-11T23:30:35.369+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:35.369+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 50 epoch_requested: 50 MInfoRec from 4 info: 4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/0 les/c/f=50/0/0 sis=49) pg_lease(ru 23.539068222s ub 24.209175110s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e50) pg 0x5978000 2026-02-11T23:30:35.369+0000 7f414b98b640 10 osd.0 pg_epoch: 50 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=0/0 les/c/f=0/0/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 50 epoch_requested: 50 MInfoRec from 4 info: 4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/0 les/c/f=50/0/0 sis=49) pg_lease(ru 23.539068222s ub 24.209175110s int 16.000000000s) 2026-02-11T23:30:35.369+0000 7f414b98b640 20 osd.0 pg_epoch: 50 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/0 les/c/f=50/0/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 active mbc={}] PeeringState::update_history advanced history from ec=49/49 lis/c=49/0 les/c/f=50/0/0 sis=49 2026-02-11T23:30:35.369+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:35.369+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:35.369+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:35.369+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 7 ==== pg_info2(4.0 4.0 4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=49) e50/50) ==== 1126+0+0 (unknown 0 0 0) 0x5a8c700 con 0x507ac00 2026-02-11T23:30:35.369+0000 7f416abfa640 15 osd.0 50 enqueue_peering_evt 4.0 epoch_sent: 50 epoch_requested: 50 MInfoRec from 4 info: 4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=49) pg_lease(ru 23.539068222s ub 24.209175110s int 16.000000000s) 2026-02-11T23:30:35.369+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 50 epoch_requested: 50 MInfoRec from 4 info: 4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=49) pg_lease(ru 23.539068222s ub 24.209175110s int 16.000000000s)) class_id 2 prio 255 cost 10 e50) 2026-02-11T23:30:35.369+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:35.369+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 50 epoch_requested: 50 MInfoRec from 4 info: 4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=49) pg_lease(ru 23.539068222s ub 24.209175110s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e50) queued 2026-02-11T23:30:35.369+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:35.369+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 50 epoch_requested: 50 MInfoRec from 4 info: 4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=49) pg_lease(ru 23.539068222s ub 24.209175110s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e50) pg 0x5978000 2026-02-11T23:30:35.369+0000 7f414b98b640 10 osd.0 pg_epoch: 50 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/0 les/c/f=50/0/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 50 epoch_requested: 50 MInfoRec from 4 info: 4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=49) pg_lease(ru 23.539068222s ub 24.209175110s int 16.000000000s) 2026-02-11T23:30:35.369+0000 7f414b98b640 20 osd.0 pg_epoch: 50 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 active mbc={}] PeeringState::update_history advanced history from ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=49 2026-02-11T23:30:35.369+0000 7f414b98b640 20 osd.0 pg_epoch: 50 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 active mbc={}] PeeringState::update_history clearing past_intervals 2026-02-11T23:30:35.369+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:35.369+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:35.369+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:35.369+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:35.369+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:35.369+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:35.410+0000 7f41633eb640 10 osd.0 50 tick 2026-02-11T23:30:35.410+0000 7f41633eb640 20 osd.0 50 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:30:35.887+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 9 ==== pg_query2(4.0 4.0 query(info 0'0 epoch_sent 52) e52/52) ==== 158+0+0 (unknown 0 0 0) 0x4fca960 con 0x507ac00 2026-02-11T23:30:35.887+0000 7f415aefe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 8 ==== osd_map(50..52 src has 1..52) ==== 1460+0+0 (unknown 0 0 0) 0x5954380 con 0x507ac00 2026-02-11T23:30:35.887+0000 7f415aefe640 20 osd.0 50 OSD::ms_dispatch: osd_map(50..52 src has 1..52) 2026-02-11T23:30:35.887+0000 7f415aefe640 20 osd.0 50 _dispatch 0x5954380 osd_map(50..52 src has 1..52) 2026-02-11T23:30:35.887+0000 7f416abfa640 15 osd.0 50 enqueue_peering_evt 4.0 epoch_sent: 52 epoch_requested: 52 MQuery 4.0 from 4 query_epoch 52 query: query(info 0'0 epoch_sent 52) 2026-02-11T23:30:35.887+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 52 epoch_requested: 52 MQuery 4.0 from 4 query_epoch 52 query: query(info 0'0 epoch_sent 52)) class_id 2 prio 255 cost 10 e52) 2026-02-11T23:30:35.887+0000 7f415aefe640 3 osd.0 50 handle_osd_map epochs [50,52], i have 50, src has [1,52] 2026-02-11T23:30:35.887+0000 7f415aefe640 10 osd.0 50 handle_osd_map: received 0 full maps and 3 incremental maps 2026-02-11T23:30:35.887+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:35.887+0000 7f415aefe640 10 osd.0 50 handle_osd_map got inc map for epoch 51 2026-02-11T23:30:35.887+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 52 epoch_requested: 52 MQuery 4.0 from 4 query_epoch 52 query: query(info 0'0 epoch_sent 52)) class_id 2immediate prio 255 cost 10 e52) queued 2026-02-11T23:30:35.887+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:35.887+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 52 epoch_requested: 52 MQuery 4.0 from 4 query_epoch 52 query: query(info 0'0 epoch_sent 52)) class_id 2immediate prio 255 cost 10 e52) pg 0x5978000 2026-02-11T23:30:35.887+0000 7f414f993640 20 osd.0 op_wq _add_slot_waiter 4.0 peering, item epoch is 52, will wait on OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 52 epoch_requested: 52 MQuery 4.0 from 4 query_epoch 52 query: query(info 0'0 epoch_sent 52)) class_id 2immediate prio 255 cost 10 e52) 2026-02-11T23:30:35.887+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:35.887+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:35.887+0000 7f415aefe640 20 osd.0 50 got_full_map 51, nothing requested 2026-02-11T23:30:35.887+0000 7f415aefe640 10 osd.0 50 handle_osd_map got inc map for epoch 52 2026-02-11T23:30:35.887+0000 7f415aefe640 20 osd.0 50 got_full_map 52, nothing requested 2026-02-11T23:30:35.887+0000 7f415aefe640 20 osd.0 50 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:35.887+0000 7f415aefe640 20 osd.0 50 handle_osd_map pg_num_history pg_num_history(e52 pg_nums {1={27=1},2={34=1},3={42=1},4={49=1}} deleted_pools ) 2026-02-11T23:30:35.887+0000 7f415aefe640 10 snap_mapper record_purged_snaps purged_snaps {51={},52={}} 2026-02-11T23:30:35.887+0000 7f415aefe640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:35.887+0000 7f415aefe640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e52 maps [1~52] lci=[5,52] tlb=1) 2026-02-11T23:30:35.887+0000 7f41591a6640 10 osd.0 50 _committed_osd_maps 51..52 2026-02-11T23:30:35.887+0000 7f41591a6640 10 osd.0 50 advance to epoch 51 (<= last 52 <= newest_map 52) 2026-02-11T23:30:35.887+0000 7f41591a6640 10 osd.0 51 advance to epoch 52 (<= last 52 <= newest_map 52) 2026-02-11T23:30:35.887+0000 7f41591a6640 20 osd.0 52 consume_map version 52 2026-02-11T23:30:35.887+0000 7f41591a6640 20 osd.0 50 await_reserved_maps epoch:52 2026-02-11T23:30:35.887+0000 7f41591a6640 20 osd.0 50 await_reserved_maps done 0 2026-02-11T23:30:35.887+0000 7f41591a6640 20 osd.0 52 consume_map 52 -- publish done 2026-02-11T23:30:35.887+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:30:35.887+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x37582c0 2026-02-11T23:30:35.887+0000 7f41591a6640 20 osd.0 52 identify_splits_and_merges 4.0 e50 to e52 2026-02-11T23:30:35.887+0000 7f41591a6640 20 osd.0 52 identify_splits_and_merges 4.0 e50 to e52 pg_nums {49=1} 2026-02-11T23:30:35.887+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3700dc0 2026-02-11T23:30:35.887+0000 7f41591a6640 20 osd.0 52 identify_splits_and_merges 1.0 e50 to e52 2026-02-11T23:30:35.887+0000 7f41591a6640 20 osd.0 52 identify_splits_and_merges 1.0 e50 to e52 pg_nums {27=1} 2026-02-11T23:30:35.887+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:35.887+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:35.887+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:35.887+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:35.887+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:35.887+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:35.887+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:35.887+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:35.887+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:35.887+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:35.887+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:35.887+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:35.887+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:35.887+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:35.887+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:35.887+0000 7f41591a6640 20 osd.0 52 prune_pg_created 2026-02-11T23:30:35.887+0000 7f41591a6640 10 osd.0:0.consume_map 52 (was 50) 2026-02-11T23:30:35.887+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:35.887+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 pending_peering first epoch 52 <= 52, requeueing 2026-02-11T23:30:35.887+0000 7f41591a6640 20 osd.0:0._wake_pg_slot _wake_pg_slot 4.0 to_process <> waiting <> waiting_peering {52=} 2026-02-11T23:30:35.887+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:35.887+0000 7f41591a6640 10 osd.0:1.consume_map 52 (was 50) 2026-02-11T23:30:35.887+0000 7f41591a6640 10 osd.0:2.consume_map 52 (was 50) 2026-02-11T23:30:35.887+0000 7f41591a6640 10 osd.0:3.consume_map 52 (was 50) 2026-02-11T23:30:35.887+0000 7f41591a6640 10 osd.0:4.consume_map 52 (was 50) 2026-02-11T23:30:35.887+0000 7f41591a6640 10 osd.0:5.consume_map 52 (was 50) 2026-02-11T23:30:35.887+0000 7f41591a6640 10 osd.0:6.consume_map 52 (was 50) 2026-02-11T23:30:35.887+0000 7f41591a6640 10 osd.0:7.consume_map 52 (was 50) 2026-02-11T23:30:35.887+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:35.887+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 52 epoch_requested: 52 MQuery 4.0 from 4 query_epoch 52 query: query(info 0'0 epoch_sent 52)) class_id 2immediate prio 255 cost 10 e52) queued 2026-02-11T23:30:35.887+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:35.887+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 52 epoch_requested: 52 MQuery 4.0 from 4 query_epoch 52 query: query(info 0'0 epoch_sent 52)) class_id 2immediate prio 255 cost 10 e52) pg 0x5978000 2026-02-11T23:30:35.887+0000 7f41591a6640 10 osd.0 52 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:35.887+0000 7f41591a6640 15 osd.0 52 enqueue_peering_evt 4.0 epoch_sent: 52 epoch_requested: 52 NullEvt 2026-02-11T23:30:35.887+0000 7f414b98b640 10 osd.0 pg_epoch: 50 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 active mbc={}] handle_advance_map: 51 2026-02-11T23:30:35.887+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 4.0 50 -> 51 2026-02-11T23:30:35.887+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 52 epoch_requested: 52 NullEvt) class_id 2 prio 255 cost 10 e52) 2026-02-11T23:30:35.887+0000 7f414b98b640 10 osd.0 pg_epoch: 50 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0,3]/[4,0,3] -- 4/4 2026-02-11T23:30:35.887+0000 7f414b98b640 10 osd.0 pg_epoch: 51 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:35.887+0000 7f41591a6640 15 osd.0 52 enqueue_peering_evt 1.0 epoch_sent: 52 epoch_requested: 52 NullEvt 2026-02-11T23:30:35.887+0000 7f414b98b640 10 osd.0 pg_epoch: 51 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:35.887+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:35.887+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 52 epoch_requested: 52 NullEvt) class_id 2immediate prio 255 cost 10 e52) queued 2026-02-11T23:30:35.887+0000 7f414b98b640 10 osd.0 pg_epoch: 51 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 active mbc={}] plpg_on_pool_change 2026-02-11T23:30:35.887+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 52 epoch_requested: 52 NullEvt) class_id 2 prio 255 cost 10 e52) 2026-02-11T23:30:35.887+0000 7f414b98b640 20 osd.0 pg_epoch: 51 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 active mbc={}] hit_set_clear 2026-02-11T23:30:35.887+0000 7f41591a6640 7 osd.0 52 activate_map version 52 2026-02-11T23:30:35.887+0000 7f414b98b640 20 osd.0 pg_epoch: 51 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 active mbc={}] agent_stop 2026-02-11T23:30:35.887+0000 7f414b98b640 20 osd.0 52 advance_pg new pool opts old pool opts 2026-02-11T23:30:35.887+0000 7f414b98b640 10 osd.0 pg_epoch: 51 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 active mbc={}] handle_advance_map: 52 2026-02-11T23:30:35.887+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 4.0 51 -> 52 2026-02-11T23:30:35.887+0000 7f414b98b640 10 osd.0 pg_epoch: 51 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:35.887+0000 7f414b98b640 10 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:35.887+0000 7f414b98b640 20 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 active mbc={}] PeeringState::should_restart_peering new interval newup [4,0] newacting [4,0] 2026-02-11T23:30:35.887+0000 7f414b98b640 10 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 active mbc={}] state: should_restart_peering, transitioning to Reset 2026-02-11T23:30:35.887+0000 7f4150194640 20 osd.0 52 agent_entry empty queue 2026-02-11T23:30:35.887+0000 7f414b98b640 5 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 active mbc={}] exit Started/ReplicaActive/RepNotRecovering 0.690454 6 0.000093 2026-02-11T23:30:35.887+0000 7f414b98b640 5 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 active mbc={}] exit Started/ReplicaActive 0.690459 0 0.000000 2026-02-11T23:30:35.887+0000 7f414b98b640 5 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 active mbc={}] exit Started 0.690594 0 0.000000 2026-02-11T23:30:35.887+0000 7f414b98b640 5 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 active mbc={}] enter Reset 2026-02-11T23:30:35.887+0000 7f414b98b640 20 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=49) [4,0,3] r=1 lpr=49 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset set_last_peering_reset 52 2026-02-11T23:30:35.887+0000 7f414b98b640 10 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=49) [4,0,3] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset Clearing blocked outgoing recovery messages 2026-02-11T23:30:35.887+0000 7f414b98b640 10 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=49) [4,0,3] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset Not blocking outgoing recovery messages 2026-02-11T23:30:35.887+0000 7f414b98b640 10 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=49) [4,0,3] r=1 lpr=52 crt=0'0 active mbc={}] state: Reset advmap 2026-02-11T23:30:35.887+0000 7f414b98b640 20 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=49) [4,0,3] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::should_restart_peering new interval newup [4,0] newacting [4,0] 2026-02-11T23:30:35.887+0000 7f414b98b640 10 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=49) [4,0,3] r=1 lpr=52 crt=0'0 active mbc={}] state: should restart peering, calling start_peering_interval again 2026-02-11T23:30:35.887+0000 7f414b98b640 20 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=49) [4,0,3] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset set_last_peering_reset 52 2026-02-11T23:30:35.887+0000 7f414b98b640 15 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=49) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] clear_stats 2026-02-11T23:30:35.887+0000 7f414b98b640 10 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=49) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 active mbc={}] PeeringState::start_peering_interval : check_new_interval output: check_new_interval interval(49-51 up [4, 0, 3](4) acting [4, 0, 3](4)) up_thru 49 up_from 23 last_epoch_clean 50 interval(49-51 up [4, 0, 3](4) acting [4, 0, 3](4) maybe_went_rw) : primary up 23-49 includes interval 2026-02-11T23:30:35.887+0000 7f414b98b640 10 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=49) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 active mbc={}] PeeringState::start_peering_interval noting past ([49,51] all_participants={0, 3, 4} intervals=[([49,51] acting={0, 3, 4})]) 2026-02-11T23:30:35.887+0000 7f414b98b640 20 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 active mbc={}] PeeringState::on_new_interval 2026-02-11T23:30:35.887+0000 7f414b98b640 20 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 active mbc={}] PeeringState::on_new_interval upacting_features 0x3f0ffffffffdffff from [4,0]+[4,0] 2026-02-11T23:30:35.887+0000 7f414b98b640 20 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 active mbc={}] PeeringState::on_new_interval checking missing set deletes flag. missing = missing(0 may_include_deletes = 1) 2026-02-11T23:30:35.887+0000 7f414b98b640 10 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 active mbc={}] PeeringState::init_hb_stamps now [0x56859d0] 2026-02-11T23:30:35.887+0000 7f414b98b640 10 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.481337547s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 active pruub 34.045890808s@ mbc={}] PeeringState::on_new_interval prior_readable_until_ub 34.045890808s (mnow 18.564554214s + 15.481337547s) 2026-02-11T23:30:35.887+0000 7f414b98b640 10 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.481337547s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 active pruub 34.045890808s@ mbc={}] cancel_recovery 2026-02-11T23:30:35.887+0000 7f414b98b640 10 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.481337547s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 active pruub 34.045890808s@ mbc={}] clear_recovery_state 2026-02-11T23:30:35.887+0000 7f414b98b640 15 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.481337547s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 active pruub 34.045890808s@ mbc={}] _clear_recovery_state 2026-02-11T23:30:35.887+0000 7f414b98b640 10 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.481337547s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 active pruub 34.045890808s@ mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:35.887+0000 7f414b98b640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:35.887+0000 7f414b98b640 20 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.481337547s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 active pruub 34.045890808s@ mbc={}] scrubber: FSM: ReplicaActive::exit() 2026-02-11T23:30:35.887+0000 7f414b98b640 10 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.481337547s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 active pruub 34.045890808s@ mbc={}] scrubber: FSM: ReplicaActive::clear_remote_reservation(): pending_reservation_nonce 0, reservation_granted false 2026-02-11T23:30:35.887+0000 7f414b98b640 10 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.481337547s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 active pruub 34.045890808s@ mbc={}] scrubber: FSM: -- state -->> NotActive 2026-02-11T23:30:35.887+0000 7f414b98b640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:35.887+0000 7f414b98b640 1 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.481337547s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 active pruub 34.045890808s@ mbc={}] PeeringState::start_peering_interval up [4,0,3] -> [4,0], acting [4,0,3] -> [4,0], acting_primary 4 -> 4, up_primary 4 -> 4, role 1 -> 1, features acting 4544132024016699391 upacting 4544132024016699391 2026-02-11T23:30:35.887+0000 7f414b98b640 10 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.481337547s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown pruub 34.045890808s@ mbc={}] PeeringState::clear_primary_state clear_primary_state 2026-02-11T23:30:35.887+0000 7f414b98b640 20 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.481337547s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown pruub 34.045890808s@ mbc={}] clear_primary_state 2026-02-11T23:30:35.887+0000 7f414b98b640 10 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.481337547s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown pruub 34.045890808s@ mbc={}] release_backoffs [4:00000000::::0,MAX) 2026-02-11T23:30:35.887+0000 7f414b98b640 10 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.481337547s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown pruub 34.045890808s@ mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:35.887+0000 7f414b98b640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:35.887+0000 7f414b98b640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:35.887+0000 7f414b98b640 20 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.481337547s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown pruub 34.045890808s@ mbc={}] agent_stop 2026-02-11T23:30:35.887+0000 7f414b98b640 10 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.481337547s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown pruub 34.045890808s@ mbc={}] on_change 2026-02-11T23:30:35.887+0000 7f414b98b640 10 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.481337547s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown pruub 34.045890808s@ mbc={}] cancel_copy_ops 2026-02-11T23:30:35.887+0000 7f414b98b640 10 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.481337547s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown pruub 34.045890808s@ mbc={}] cancel_flush_ops 2026-02-11T23:30:35.887+0000 7f414b98b640 10 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.481337547s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown pruub 34.045890808s@ mbc={}] cancel_proxy_ops 2026-02-11T23:30:35.887+0000 7f414b98b640 10 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.481337547s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown pruub 34.045890808s@ mbc={}] cancel_manifest_ops 2026-02-11T23:30:35.887+0000 7f414b98b640 10 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.481337547s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown pruub 34.045890808s@ mbc={}] cancel_cls_gather_ops 2026-02-11T23:30:35.887+0000 7f414b98b640 10 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.481337547s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown pruub 34.045890808s@ mbc={}] on_change_cleanup 2026-02-11T23:30:35.887+0000 7f414b98b640 10 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.481337547s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown pruub 34.045890808s@ mbc={}] on_change 2026-02-11T23:30:35.887+0000 7f414b98b640 20 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.481337547s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown pruub 34.045890808s@ mbc={}] exit NotTrimming 2026-02-11T23:30:35.887+0000 7f414b98b640 20 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.481337547s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown pruub 34.045890808s@ mbc={}] enter NotTrimming 2026-02-11T23:30:35.887+0000 7f414b98b640 10 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.481337547s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown NOTIFY pruub 34.045890808s@ mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:35.887+0000 7f414b98b640 10 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.481337547s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown NOTIFY pruub 34.045890808s@ mbc={}] plpg_on_pool_change 2026-02-11T23:30:35.887+0000 7f414b98b640 20 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.481337547s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown NOTIFY pruub 34.045890808s@ mbc={}] hit_set_clear 2026-02-11T23:30:35.887+0000 7f414b98b640 20 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.481337547s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown NOTIFY pruub 34.045890808s@ mbc={}] agent_stop 2026-02-11T23:30:35.887+0000 7f414b98b640 20 osd.0 52 advance_pg new pool opts old pool opts 2026-02-11T23:30:35.887+0000 7f414b98b640 10 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.481337547s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown NOTIFY pruub 34.045890808s@ mbc={}] handle_activate_map: epoch range: 51..52 2026-02-11T23:30:35.887+0000 7f414b98b640 10 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.481337547s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown NOTIFY pruub 34.045890808s@ mbc={}] PeeringState::activate_map 2026-02-11T23:30:35.887+0000 7f414b98b640 5 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.481280327s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown NOTIFY pruub 34.045890808s@ mbc={}] exit Reset 0.000093 1 0.000095 2026-02-11T23:30:35.887+0000 7f414b98b640 5 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.481280327s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown NOTIFY pruub 34.045890808s@ mbc={}] enter Started 2026-02-11T23:30:35.887+0000 7f414b98b640 5 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.481280327s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown NOTIFY pruub 34.045890808s@ mbc={}] enter Start 2026-02-11T23:30:35.887+0000 7f414b98b640 1 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.481280327s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown NOTIFY pruub 34.045890808s@ mbc={}] state: transitioning to Stray 2026-02-11T23:30:35.887+0000 7f414b98b640 5 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.481280327s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown NOTIFY pruub 34.045890808s@ mbc={}] exit Start 0.000002 0 0.000000 2026-02-11T23:30:35.887+0000 7f414b98b640 5 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.481280327s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown NOTIFY pruub 34.045890808s@ mbc={}] enter Started/Stray 2026-02-11T23:30:35.887+0000 7f414b98b640 20 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.481280327s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown NOTIFY pruub 34.045890808s@ mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 50 while current is 52 2026-02-11T23:30:35.887+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:35.887+0000 7f414b98b640 10 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.481280327s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown NOTIFY pruub 34.045890808s@ mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:30:35.887+0000 7f414b98b640 10 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.481280327s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown NOTIFY pruub 34.045890808s@ mbc={}] do_peering_event: epoch_sent: 52 epoch_requested: 52 MQuery 4.0 from 4 query_epoch 52 query: query(info 0'0 epoch_sent 52) 2026-02-11T23:30:35.887+0000 7f414b98b640 10 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.481263161s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown NOTIFY pruub 34.045890808s@ mbc={}] PeeringState::fulfill_info sending info 2026-02-11T23:30:35.887+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:35.887+0000 7f414b98b640 20 osd.0 52 get_con_osd_cluster to osd.4 from_epoch 52 2026-02-11T23:30:35.887+0000 7f414b98b640 20 osd.0 52 get_nextmap_reserved map_reservations: {52=1} 2026-02-11T23:30:35.887+0000 7f414b98b640 20 osd.0 52 release_map epoch: 52 2026-02-11T23:30:35.887+0000 7f414b98b640 10 osd.0 52 maybe_share_map: con v2:127.0.0.1:6836/3331151400 map epoch 50 -> 52 (shared) 2026-02-11T23:30:35.887+0000 7f414b98b640 10 osd.0 52 send_incremental_map epoch range: (51, 52] to v2:127.0.0.1:6836/3331151400 2026-02-11T23:30:35.887+0000 7f414b98b640 10 osd.0 52 add_map_inc_bl 51 574 bytes 2026-02-11T23:30:35.887+0000 7f414b98b640 10 osd.0 52 add_map_inc_bl 52 574 bytes 2026-02-11T23:30:35.887+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- osd_map(51..52 src has 1..52) -- 0x4a31180 con 0x507ac00 2026-02-11T23:30:35.887+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_notify2(4.0 4.0 (query:52 sent:52 4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.481280327s) ([49,51] all_participants={0, 3, 4} intervals=[([49,51] acting={0, 3, 4})])) e52/52) -- 0x373b800 con 0x507ac00 2026-02-11T23:30:35.887+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_notify2(4.0 4.0 (query:52 sent:52 4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.481263161s) ([49,51] all_participants={0, 3, 4} intervals=[([49,51] acting={0, 3, 4})])) e52/52) -- 0x5a8c700 con 0x507ac00 2026-02-11T23:30:35.888+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:35.888+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 52 epoch_requested: 52 NullEvt) class_id 2immediate prio 255 cost 10 e52) queued 2026-02-11T23:30:35.888+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:35.888+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 52 epoch_requested: 52 NullEvt) class_id 2immediate prio 255 cost 10 e52) pg 0x5946000 2026-02-11T23:30:35.888+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:35.888+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 52 epoch_requested: 52 NullEvt) class_id 2immediate prio 255 cost 10 e52) pg 0x5978000 2026-02-11T23:30:35.888+0000 7f414b98b640 10 osd.0 pg_epoch: 50 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 51 2026-02-11T23:30:35.888+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 1.0 50 -> 51 2026-02-11T23:30:35.888+0000 7f414b98b640 10 osd.0 pg_epoch: 50 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:35.888+0000 7f414f993640 10 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.481263161s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown NOTIFY pruub 34.045890808s@ mbc={}] do_peering_event: epoch_sent: 52 epoch_requested: 52 NullEvt 2026-02-11T23:30:35.888+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:35.888+0000 7f414b98b640 10 osd.0 pg_epoch: 51 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:35.888+0000 7f414b98b640 10 osd.0 pg_epoch: 51 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:35.888+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:35.888+0000 7f414b98b640 20 osd.0 52 advance_pg new pool opts old pool opts 2026-02-11T23:30:35.888+0000 7f414b98b640 10 osd.0 pg_epoch: 51 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 52 2026-02-11T23:30:35.888+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 1.0 51 -> 52 2026-02-11T23:30:35.888+0000 7f414b98b640 10 osd.0 pg_epoch: 51 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:35.888+0000 7f414b98b640 10 osd.0 pg_epoch: 52 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:35.888+0000 7f414b98b640 10 osd.0 pg_epoch: 52 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:35.888+0000 7f414b98b640 20 osd.0 52 advance_pg new pool opts old pool opts 2026-02-11T23:30:35.888+0000 7f414b98b640 10 osd.0 pg_epoch: 52 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_activate_map: epoch range: 51..52 2026-02-11T23:30:35.888+0000 7f414b98b640 10 osd.0 pg_epoch: 52 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:35.888+0000 7f414b98b640 20 osd.0 pg_epoch: 52 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 30 while current is 52 2026-02-11T23:30:35.888+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:35.888+0000 7f414b98b640 10 osd.0 pg_epoch: 52 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 52 epoch_requested: 52 NullEvt 2026-02-11T23:30:35.888+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:35.888+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:35.888+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:35.888+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:35.986+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 32 ==== osd_map(52..53 src has 1..53) ==== 1208+0+0 (crc 0 0 0) 0x4a31c00 con 0x4acbc00 2026-02-11T23:30:35.986+0000 7f415cf02640 20 osd.0 52 OSD::ms_dispatch: osd_map(52..53 src has 1..53) 2026-02-11T23:30:35.986+0000 7f415cf02640 20 osd.0 52 _dispatch 0x4a31c00 osd_map(52..53 src has 1..53) 2026-02-11T23:30:35.986+0000 7f415cf02640 3 osd.0 52 handle_osd_map epochs [52,53], i have 52, src has [1,53] 2026-02-11T23:30:35.986+0000 7f415cf02640 10 osd.0 52 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:35.986+0000 7f415cf02640 10 osd.0 52 handle_osd_map got inc map for epoch 53 2026-02-11T23:30:35.986+0000 7f415cf02640 20 osd.0 52 got_full_map 53, nothing requested 2026-02-11T23:30:35.986+0000 7f415cf02640 20 osd.0 52 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:35.986+0000 7f415cf02640 20 osd.0 52 handle_osd_map pg_num_history pg_num_history(e53 pg_nums {1={27=1},2={34=1},3={42=1},4={49=1}} deleted_pools ) 2026-02-11T23:30:35.986+0000 7f415cf02640 10 snap_mapper record_purged_snaps purged_snaps {53={}} 2026-02-11T23:30:35.986+0000 7f415cf02640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:35.986+0000 7f415cf02640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e53 maps [1~53] lci=[5,53] tlb=1) 2026-02-11T23:30:35.987+0000 7f41591a6640 10 osd.0 52 _committed_osd_maps 53..53 2026-02-11T23:30:35.987+0000 7f41591a6640 10 osd.0 52 advance to epoch 53 (<= last 53 <= newest_map 53) 2026-02-11T23:30:35.987+0000 7f41591a6640 20 osd.0 53 consume_map version 53 2026-02-11T23:30:35.987+0000 7f41591a6640 20 osd.0 52 await_reserved_maps epoch:53 2026-02-11T23:30:35.987+0000 7f41591a6640 20 osd.0 52 await_reserved_maps done 0 2026-02-11T23:30:35.987+0000 7f41591a6640 20 osd.0 53 consume_map 53 -- publish done 2026-02-11T23:30:35.987+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:30:35.987+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x37582c0 2026-02-11T23:30:35.987+0000 7f41591a6640 20 osd.0 53 identify_splits_and_merges 4.0 e52 to e53 2026-02-11T23:30:35.987+0000 7f41591a6640 20 osd.0 53 identify_splits_and_merges 4.0 e52 to e53 pg_nums {49=1} 2026-02-11T23:30:35.987+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3700dc0 2026-02-11T23:30:35.987+0000 7f41591a6640 20 osd.0 53 identify_splits_and_merges 1.0 e52 to e53 2026-02-11T23:30:35.987+0000 7f41591a6640 20 osd.0 53 identify_splits_and_merges 1.0 e52 to e53 pg_nums {27=1} 2026-02-11T23:30:35.987+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:35.987+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:35.987+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:35.987+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:35.987+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:35.987+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:35.987+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:35.987+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:35.987+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:35.987+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:35.987+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:35.987+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:35.987+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:35.987+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:35.987+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:35.987+0000 7f41591a6640 20 osd.0 53 prune_pg_created 2026-02-11T23:30:35.987+0000 7f41591a6640 10 osd.0:0.consume_map 53 (was 52) 2026-02-11T23:30:35.987+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:35.987+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:35.987+0000 7f41591a6640 10 osd.0:1.consume_map 53 (was 52) 2026-02-11T23:30:35.987+0000 7f41591a6640 10 osd.0:2.consume_map 53 (was 52) 2026-02-11T23:30:35.987+0000 7f41591a6640 10 osd.0:3.consume_map 53 (was 52) 2026-02-11T23:30:35.987+0000 7f41591a6640 10 osd.0:4.consume_map 53 (was 52) 2026-02-11T23:30:35.987+0000 7f41591a6640 10 osd.0:5.consume_map 53 (was 52) 2026-02-11T23:30:35.987+0000 7f41591a6640 10 osd.0:6.consume_map 53 (was 52) 2026-02-11T23:30:35.987+0000 7f41591a6640 10 osd.0:7.consume_map 53 (was 52) 2026-02-11T23:30:35.987+0000 7f41591a6640 10 osd.0 53 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:35.987+0000 7f41591a6640 15 osd.0 53 enqueue_peering_evt 4.0 epoch_sent: 53 epoch_requested: 53 NullEvt 2026-02-11T23:30:35.987+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 53 epoch_requested: 53 NullEvt) class_id 2 prio 255 cost 10 e53) 2026-02-11T23:30:35.987+0000 7f41591a6640 15 osd.0 53 enqueue_peering_evt 1.0 epoch_sent: 53 epoch_requested: 53 NullEvt 2026-02-11T23:30:35.987+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 53 epoch_requested: 53 NullEvt) class_id 2 prio 255 cost 10 e53) 2026-02-11T23:30:35.987+0000 7f41591a6640 7 osd.0 53 activate_map version 53 2026-02-11T23:30:35.987+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:35.987+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 53 epoch_requested: 53 NullEvt) class_id 2immediate prio 255 cost 10 e53) queued 2026-02-11T23:30:35.987+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:35.987+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 53 epoch_requested: 53 NullEvt) class_id 2immediate prio 255 cost 10 e53) pg 0x5978000 2026-02-11T23:30:35.987+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:35.987+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 53 epoch_requested: 53 NullEvt) class_id 2immediate prio 255 cost 10 e53) queued 2026-02-11T23:30:35.987+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:35.987+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 53 epoch_requested: 53 NullEvt) class_id 2immediate prio 255 cost 10 e53) pg 0x5946000 2026-02-11T23:30:35.987+0000 7f414f993640 10 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.481263161s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown NOTIFY pruub 34.045890808s@ mbc={}] handle_advance_map: 53 2026-02-11T23:30:35.987+0000 7f414f993640 20 osd.0:0.update_pg_epoch 4.0 52 -> 53 2026-02-11T23:30:35.987+0000 7f4150194640 20 osd.0 53 agent_entry empty queue 2026-02-11T23:30:35.987+0000 7f414b98b640 10 osd.0 pg_epoch: 52 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 53 2026-02-11T23:30:35.987+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 1.0 52 -> 53 2026-02-11T23:30:35.987+0000 7f414b98b640 10 osd.0 pg_epoch: 52 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:35.987+0000 7f414f993640 10 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.481263161s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown NOTIFY pruub 34.045890808s@ mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:35.987+0000 7f414b98b640 10 osd.0 pg_epoch: 53 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:35.987+0000 7f414b98b640 10 osd.0 pg_epoch: 53 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:35.987+0000 7f414f993640 10 osd.0 pg_epoch: 53 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.481263161s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown NOTIFY pruub 34.045890808s@ mbc={}] state: Started advmap 2026-02-11T23:30:35.987+0000 7f414b98b640 20 osd.0 53 advance_pg new pool opts old pool opts 2026-02-11T23:30:35.987+0000 7f414f993640 10 osd.0 pg_epoch: 53 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.481263161s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown NOTIFY pruub 34.045890808s@ mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:35.987+0000 7f414b98b640 10 osd.0 pg_epoch: 53 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_activate_map: epoch range: 53..53 2026-02-11T23:30:35.987+0000 7f414f993640 10 osd.0 pg_epoch: 53 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.481263161s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown NOTIFY pruub 34.045890808s@ mbc={}] plpg_on_pool_change 2026-02-11T23:30:35.987+0000 7f414b98b640 10 osd.0 pg_epoch: 53 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:35.987+0000 7f414f993640 20 osd.0 pg_epoch: 53 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.481263161s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown NOTIFY pruub 34.045890808s@ mbc={}] hit_set_clear 2026-02-11T23:30:35.987+0000 7f414b98b640 20 osd.0 pg_epoch: 53 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 30 while current is 53 2026-02-11T23:30:35.987+0000 7f414f993640 20 osd.0 pg_epoch: 53 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.481263161s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown NOTIFY pruub 34.045890808s@ mbc={}] agent_stop 2026-02-11T23:30:35.987+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:35.987+0000 7f414f993640 20 osd.0 53 advance_pg new pool opts old pool opts 2026-02-11T23:30:35.987+0000 7f414b98b640 10 osd.0 pg_epoch: 53 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 53 epoch_requested: 53 NullEvt 2026-02-11T23:30:35.987+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:35.987+0000 7f414f993640 10 osd.0 pg_epoch: 53 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.481263161s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown NOTIFY pruub 34.045890808s@ mbc={}] handle_activate_map: epoch range: 53..53 2026-02-11T23:30:35.987+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:35.987+0000 7f414f993640 10 osd.0 pg_epoch: 53 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.481263161s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown NOTIFY pruub 34.045890808s@ mbc={}] PeeringState::activate_map 2026-02-11T23:30:35.987+0000 7f414f993640 20 osd.0 pg_epoch: 53 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.381997108s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown NOTIFY pruub 34.045890808s@ mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 52 while current is 53 2026-02-11T23:30:35.987+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:35.987+0000 7f414f993640 10 osd.0 pg_epoch: 53 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.381997108s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown NOTIFY pruub 34.045890808s@ mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:30:35.987+0000 7f414f993640 10 osd.0 pg_epoch: 53 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.381997108s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown NOTIFY pruub 34.045890808s@ mbc={}] do_peering_event: epoch_sent: 53 epoch_requested: 53 NullEvt 2026-02-11T23:30:35.987+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:35.987+0000 7f414f993640 20 osd.0 53 get_con_osd_cluster to osd.4 from_epoch 53 2026-02-11T23:30:35.987+0000 7f414f993640 20 osd.0 53 get_nextmap_reserved map_reservations: {53=1} 2026-02-11T23:30:35.987+0000 7f414f993640 20 osd.0 53 release_map epoch: 53 2026-02-11T23:30:35.987+0000 7f414f993640 10 osd.0 53 maybe_share_map: con v2:127.0.0.1:6836/3331151400 map epoch 52 -> 53 (shared) 2026-02-11T23:30:35.987+0000 7f414f993640 10 osd.0 53 send_incremental_map epoch range: (53, 53] to v2:127.0.0.1:6836/3331151400 2026-02-11T23:30:35.987+0000 7f414f993640 10 osd.0 53 add_map_inc_bl 53 582 bytes 2026-02-11T23:30:35.987+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- osd_map(53..53 src has 1..53) -- 0x4a30fc0 con 0x507ac00 2026-02-11T23:30:35.987+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_notify2(4.0 4.0 (query:53 sent:53 4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.381997108s) ([49,51] all_participants={0, 3, 4} intervals=[([49,51] acting={0, 3, 4})])) e53/53) -- 0x5a8ce00 con 0x507ac00 2026-02-11T23:30:35.987+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:35.987+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 10 ==== PGlog(4.0 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 24.827859879s int 16.000000000s) e53/52) ==== 1164+0+0 (unknown 0 0 0) 0x5a95000 con 0x507ac00 2026-02-11T23:30:35.987+0000 7f416abfa640 15 osd.0 53 enqueue_peering_evt 4.0 epoch_sent: 53 epoch_requested: 52 MLogRec from 4 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 24.827859879s int 16.000000000s) +create_info 2026-02-11T23:30:35.987+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 53 epoch_requested: 52 MLogRec from 4 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 24.827859879s int 16.000000000s) +create_info) class_id 2 prio 255 cost 10 e53) 2026-02-11T23:30:35.987+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:35.987+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 53 epoch_requested: 52 MLogRec from 4 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 24.827859879s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e53) queued 2026-02-11T23:30:35.987+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:35.987+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 53 epoch_requested: 52 MLogRec from 4 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 24.827859879s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e53) pg 0x5978000 2026-02-11T23:30:35.987+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:35.987+0000 7f414f993640 10 osd.0 pg_epoch: 53 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.381997108s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown NOTIFY pruub 34.045890808s@ mbc={}] do_peering_event: epoch_sent: 53 epoch_requested: 52 MLogRec from 4 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 24.827859879s int 16.000000000s) +create_info 2026-02-11T23:30:35.987+0000 7f414f993640 10 osd.0 pg_epoch: 53 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.381997108s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown NOTIFY pruub 34.045890808s@ mbc={}] state: got info+log from osd.4 4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52) log((0'0,0'0], crt=0'0) 2026-02-11T23:30:35.987+0000 7f414f993640 10 merge_log log((0'0,0'0], crt=0'0) from osd.4 into log((0'0,0'0], crt=0'0) 2026-02-11T23:30:35.987+0000 7f414f993640 5 merge_log_dups log.dups.size()=0olog.dups.size()=0 2026-02-11T23:30:35.987+0000 7f414f993640 5 end of merge_log_dups changed=0 log.dups.size()=0 olog.dups.size()=0 2026-02-11T23:30:35.987+0000 7f414f993640 10 merge_log result log((0'0,0'0], crt=0'0) missing(0 may_include_deletes = 1) changed=0 2026-02-11T23:30:35.987+0000 7f414f993640 10 osd.0 pg_epoch: 53 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.381997108s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown NOTIFY pruub 34.045890808s@ mbc={}] PeeringState::proc_lease pg_lease(ru 0.000000000s ub 24.827859879s int 16.000000000s) 2026-02-11T23:30:35.987+0000 7f414f993640 20 osd.0 pg_epoch: 53 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.381997108s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown NOTIFY pruub 34.045890808s@ mbc={}] PeeringState::proc_lease peer_clock_delta_lb -9.836717606s -> ruub 34.664573669s 2026-02-11T23:30:35.987+0000 7f414f993640 20 osd.0 pg_epoch: 53 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.381997108s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown NOTIFY pruub 34.045890808s@ mbc={}] PeeringState::proc_lease readable_until_ub now 34.664573669s 2026-02-11T23:30:35.987+0000 7f414f993640 5 osd.0 pg_epoch: 53 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.381997108s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown NOTIFY pruub 34.045890808s@ mbc={}] exit Started/Stray 0.099971 6 0.000018 2026-02-11T23:30:35.987+0000 7f414f993640 5 osd.0 pg_epoch: 53 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.381997108s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown NOTIFY pruub 34.045890808s@ mbc={}] enter Started/ReplicaActive 2026-02-11T23:30:35.987+0000 7f414f993640 5 osd.0 pg_epoch: 53 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.381997108s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown NOTIFY pruub 34.045890808s@ mbc={}] enter Started/ReplicaActive/RepNotRecovering 2026-02-11T23:30:35.987+0000 7f414f993640 10 osd.0 pg_epoch: 53 pg[4.0( empty local-lis/les=49/50 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.381997108s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown NOTIFY pruub 34.045890808s@ mbc={}] state: In ReplicaActive, about to call activate 2026-02-11T23:30:35.987+0000 7f414f993640 10 osd.0 pg_epoch: 53 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.381997108s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown pruub 34.045890808s@ mbc={}] PeeringState::activate activate - no missing, moving last_complete 0'0 -> 0'0 2026-02-11T23:30:35.987+0000 7f414f993640 10 osd.0 pg_epoch: 53 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.381997108s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown pruub 34.045890808s@ mbc={}] scrubber: on_replica_activate 2026-02-11T23:30:35.987+0000 7f414f993640 20 scrubberFSM event: --vvvv---- ReplicaActivate 2026-02-11T23:30:35.987+0000 7f414f993640 10 osd.0 pg_epoch: 53 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.381997108s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown pruub 34.045890808s@ mbc={}] scrubber: FSM: -- state -->> ReplicaActive 2026-02-11T23:30:35.987+0000 7f414f993640 10 osd.0 pg_epoch: 53 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.381997108s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown pruub 34.045890808s@ mbc={}] scrubber: FSM: -- state -->> ReplicaActive/ReplicaIdle 2026-02-11T23:30:35.987+0000 7f414f993640 20 scrubberFSM event: --^^^^---- ReplicaActivate 2026-02-11T23:30:35.987+0000 7f414f993640 10 osd.0 pg_epoch: 53 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.381997108s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown pruub 34.045890808s@ mbc={}] state: Activate Finished 2026-02-11T23:30:35.987+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:35.987+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:35.987+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:35.987+0000 7f414f993640 15 osd.0 53 enqueue_peering_evt 4.0 epoch_sent: 53 epoch_requested: 53 ActivateCommitted from 53 processed at 53 2026-02-11T23:30:35.987+0000 7f414f993640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 53 epoch_requested: 53 ActivateCommitted from 53 processed at 53) class_id 2 prio 255 cost 10 e53) 2026-02-11T23:30:35.987+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:35.987+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:35.987+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 53 epoch_requested: 53 ActivateCommitted from 53 processed at 53) class_id 2immediate prio 255 cost 10 e53) queued 2026-02-11T23:30:35.987+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:35.987+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 53 epoch_requested: 53 ActivateCommitted from 53 processed at 53) class_id 2immediate prio 255 cost 10 e53) pg 0x5978000 2026-02-11T23:30:35.987+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:35.987+0000 7f414f993640 10 osd.0 pg_epoch: 53 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.381997108s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown pruub 34.045890808s@ mbc={}] do_peering_event: epoch_sent: 53 epoch_requested: 53 ActivateCommitted from 53 processed at 53 2026-02-11T23:30:35.987+0000 7f414f993640 10 osd.0 pg_epoch: 53 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.381997108s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 unknown pruub 34.045890808s@ mbc={}] state: react 53 telling primary 2026-02-11T23:30:35.987+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:35.987+0000 7f414f993640 20 osd.0 53 get_con_osd_cluster to osd.4 from_epoch 53 2026-02-11T23:30:35.987+0000 7f414f993640 20 osd.0 53 get_nextmap_reserved map_reservations: {53=1} 2026-02-11T23:30:35.987+0000 7f414f993640 20 osd.0 53 release_map epoch: 53 2026-02-11T23:30:35.987+0000 7f414f993640 10 osd.0 53 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 53 is not newer than session's projected_epoch of 53 2026-02-11T23:30:35.987+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_info2(4.0 4.0 4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/49 les/c/f=53/50/0 sis=52 pruub=15.381997108s) e53/53) -- 0x5a8d500 con 0x507ac00 2026-02-11T23:30:35.988+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:35.988+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 11 ==== pg_lease(4.0 pg_lease(ru 24.827859879s ub 24.828193665s int 16.000000000s) e53/53) ==== 58+0+0 (unknown 0 0 0) 0x4c9b500 con 0x507ac00 2026-02-11T23:30:35.988+0000 7f416abfa640 15 osd.0 53 enqueue_peering_evt 4.0 epoch_sent: 53 epoch_requested: 53 MLease epoch 53 from osd.4 pg_lease(ru 24.827859879s ub 24.828193665s int 16.000000000s) 2026-02-11T23:30:35.988+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 53 epoch_requested: 53 MLease epoch 53 from osd.4 pg_lease(ru 24.827859879s ub 24.828193665s int 16.000000000s)) class_id 2 prio 255 cost 10 e53) 2026-02-11T23:30:35.988+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:35.988+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 53 epoch_requested: 53 MLease epoch 53 from osd.4 pg_lease(ru 24.827859879s ub 24.828193665s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e53) queued 2026-02-11T23:30:35.988+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:35.988+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 53 epoch_requested: 53 MLease epoch 53 from osd.4 pg_lease(ru 24.827859879s ub 24.828193665s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e53) pg 0x5978000 2026-02-11T23:30:35.988+0000 7f414b98b640 10 osd.0 pg_epoch: 53 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.381997108s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 active pruub 34.045890808s@ mbc={}] do_peering_event: epoch_sent: 53 epoch_requested: 53 MLease epoch 53 from osd.4 pg_lease(ru 24.827859879s ub 24.828193665s int 16.000000000s) 2026-02-11T23:30:35.988+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:35.988+0000 7f414b98b640 10 osd.0 pg_epoch: 53 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.381997108s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 active pruub 34.045890808s@ mbc={}] PeeringState::proc_lease pg_lease(ru 24.827859879s ub 24.828193665s int 16.000000000s) 2026-02-11T23:30:35.988+0000 7f414b98b640 20 osd.0 pg_epoch: 53 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.381997108s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 active pruub 34.045890808s@ mbc={}] PeeringState::proc_lease peer_clock_delta_ub -9.836281776s -> ru 34.664142609s 2026-02-11T23:30:35.988+0000 7f414b98b640 20 osd.0 pg_epoch: 53 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.381997108s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 active pruub 34.045890808s@ mbc={}] PeeringState::proc_lease readable_until now 34.664142609s 2026-02-11T23:30:35.988+0000 7f414b98b640 20 osd.0 pg_epoch: 53 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.381997108s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 active pruub 34.045890808s@ mbc={}] PeeringState::proc_lease peer_clock_delta_lb -9.836717606s -> ruub 34.664909363s 2026-02-11T23:30:35.988+0000 7f414b98b640 20 osd.0 pg_epoch: 53 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.381997108s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 active pruub 34.045890808s@ mbc={}] PeeringState::proc_lease readable_until_ub now 34.664909363s 2026-02-11T23:30:35.988+0000 7f414b98b640 20 osd.0 53 get_con_osd_cluster to osd.4 from_epoch 53 2026-02-11T23:30:35.988+0000 7f414b98b640 20 osd.0 53 get_nextmap_reserved map_reservations: {53=1} 2026-02-11T23:30:35.988+0000 7f414b98b640 20 osd.0 53 release_map epoch: 53 2026-02-11T23:30:35.988+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_lease_ack(4.0 pg_lease_ack(ruub 24.828193665s) e53/53) -- 0x3758f20 con 0x507ac00 2026-02-11T23:30:35.988+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:35.988+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:35.988+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 12 ==== pg_info2(4.0 4.0 4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/49 les/c/f=53/50/0 sis=52) e53/53) ==== 1118+0+0 (unknown 0 0 0) 0x5a8d500 con 0x507ac00 2026-02-11T23:30:35.988+0000 7f416abfa640 15 osd.0 53 enqueue_peering_evt 4.0 epoch_sent: 53 epoch_requested: 53 MInfoRec from 4 info: 4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/49 les/c/f=53/50/0 sis=52) pg_lease(ru 24.827859879s ub 24.828193665s int 16.000000000s) 2026-02-11T23:30:35.988+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 53 epoch_requested: 53 MInfoRec from 4 info: 4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/49 les/c/f=53/50/0 sis=52) pg_lease(ru 24.827859879s ub 24.828193665s int 16.000000000s)) class_id 2 prio 255 cost 10 e53) 2026-02-11T23:30:35.988+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:35.988+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 53 epoch_requested: 53 MInfoRec from 4 info: 4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/49 les/c/f=53/50/0 sis=52) pg_lease(ru 24.827859879s ub 24.828193665s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e53) queued 2026-02-11T23:30:35.988+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:35.988+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 53 epoch_requested: 53 MInfoRec from 4 info: 4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/49 les/c/f=53/50/0 sis=52) pg_lease(ru 24.827859879s ub 24.828193665s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e53) pg 0x5978000 2026-02-11T23:30:35.988+0000 7f414b98b640 10 osd.0 pg_epoch: 53 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=49/49 les/c/f=50/50/0 sis=52 pruub=15.381997108s) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 active pruub 34.045890808s@ mbc={}] do_peering_event: epoch_sent: 53 epoch_requested: 53 MInfoRec from 4 info: 4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/49 les/c/f=53/50/0 sis=52) pg_lease(ru 24.827859879s ub 24.828193665s int 16.000000000s) 2026-02-11T23:30:35.988+0000 7f414b98b640 20 osd.0 pg_epoch: 53 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/49 les/c/f=53/50/0 sis=52) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 active pruub 34.045890808s@ mbc={}] PeeringState::update_history advanced history from ec=49/49 lis/c=52/49 les/c/f=53/50/0 sis=52 2026-02-11T23:30:35.988+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:35.988+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:35.988+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:35.988+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:35.988+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:35.988+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 13 ==== pg_info2(4.0 4.0 4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) e53/53) ==== 1118+0+0 (unknown 0 0 0) 0x5a8d500 con 0x507ac00 2026-02-11T23:30:35.988+0000 7f416abfa640 15 osd.0 53 enqueue_peering_evt 4.0 epoch_sent: 53 epoch_requested: 53 MInfoRec from 4 info: 4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) pg_lease(ru 24.827859879s ub 24.828193665s int 16.000000000s) 2026-02-11T23:30:35.988+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 53 epoch_requested: 53 MInfoRec from 4 info: 4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) pg_lease(ru 24.827859879s ub 24.828193665s int 16.000000000s)) class_id 2 prio 255 cost 10 e53) 2026-02-11T23:30:35.988+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:35.988+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 53 epoch_requested: 53 MInfoRec from 4 info: 4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) pg_lease(ru 24.827859879s ub 24.828193665s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e53) queued 2026-02-11T23:30:35.988+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:35.988+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 53 epoch_requested: 53 MInfoRec from 4 info: 4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) pg_lease(ru 24.827859879s ub 24.828193665s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e53) pg 0x5978000 2026-02-11T23:30:35.988+0000 7f414b98b640 10 osd.0 pg_epoch: 53 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/49 les/c/f=53/50/0 sis=52) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 53 epoch_requested: 53 MInfoRec from 4 info: 4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) pg_lease(ru 24.827859879s ub 24.828193665s int 16.000000000s) 2026-02-11T23:30:35.988+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:35.988+0000 7f414b98b640 20 osd.0 pg_epoch: 53 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 active mbc={}] PeeringState::update_history advanced history from ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52 2026-02-11T23:30:35.988+0000 7f414b98b640 20 osd.0 pg_epoch: 53 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 pi=[49,52)/1 crt=0'0 active mbc={}] PeeringState::update_history clearing past_intervals 2026-02-11T23:30:35.988+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:35.988+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:35.988+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:35.988+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:36.216+0000 7f4162bea640 10 osd.0 53 tick_without_osd_lock 2026-02-11T23:30:36.216+0000 7f4162bea640 0 osd.0 53 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:36.216+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:36.216+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:36.216+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:36.216+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.508 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:36.216+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:36.217, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:36.216+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:36.216+0000 7f4162bea640 10 osd.0 53 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:36.216+0000 7f4162bea640 20 osd.0 53 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:36.216+0000 7f4162bea640 10 osd.0 53 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:36.216+0000 7f4162bea640 20 osd.0 53 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:36.246+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.3 127.0.0.1:0/2699377081 3 ==== osd_ping(ping e52 up_from 19 ping_stamp 2026-02-11T23:30:36.247202+0000/11.703141212s send_stamp 11.703141212s delta_ub -7.219387531s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a480 con 0x48cc400 2026-02-11T23:30:36.246+0000 7f416c3fd640 20 osd.0 53 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219543457s,-7.219387531s]) 2026-02-11T23:30:36.246+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e53 up_from 5 ping_stamp 2026-02-11T23:30:36.247202+0000/11.703141212s send_stamp 18.922683716s delta_ub 7.219543457s) -- 0x4c9b200 con 0x48cc400 2026-02-11T23:30:36.246+0000 7f416c3fd640 20 osd.0 53 get_con_osd_cluster to osd.3 from_epoch 53 2026-02-11T23:30:36.246+0000 7f416c3fd640 20 osd.0 53 get_nextmap_reserved map_reservations: {53=1} 2026-02-11T23:30:36.246+0000 7f416c3fd640 20 osd.0 53 release_map epoch: 53 2026-02-11T23:30:36.246+0000 7f416c3fd640 10 osd.0 53 maybe_share_map: con v2:127.0.0.1:6828/2699377081 updating session's projected_epoch from 43 to ping map epoch of 52 2026-02-11T23:30:36.246+0000 7f416c3fd640 10 osd.0 53 maybe_share_map: con v2:127.0.0.1:6828/2699377081 map epoch 52 -> 53 (shared) 2026-02-11T23:30:36.246+0000 7f416c3fd640 10 osd.0 53 send_incremental_map epoch range: (53, 53] to v2:127.0.0.1:6828/2699377081 2026-02-11T23:30:36.246+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6828/2699377081,v1:127.0.0.1:6829/2699377081] -- osd_map(53..53 src has 1..53) -- 0x59548c0 con 0x48ccc00 2026-02-11T23:30:36.246+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.3 127.0.0.1:0/2699377081 3 ==== osd_ping(ping e52 up_from 19 ping_stamp 2026-02-11T23:30:36.247202+0000/11.703141212s send_stamp 11.703141212s delta_ub -7.219387531s) ==== 2033+0+0 (crc 0 0 0) 0x4c9ad80 con 0x48cc800 2026-02-11T23:30:36.246+0000 7f416cbfe640 20 osd.0 53 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219582558s,-7.219387531s]) 2026-02-11T23:30:36.246+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e53 up_from 5 ping_stamp 2026-02-11T23:30:36.247202+0000/11.703141212s send_stamp 18.922725677s delta_ub 7.219582558s) -- 0x4c9af00 con 0x48cc800 2026-02-11T23:30:36.246+0000 7f416cbfe640 20 osd.0 53 get_con_osd_cluster to osd.3 from_epoch 53 2026-02-11T23:30:36.246+0000 7f416cbfe640 20 osd.0 53 get_nextmap_reserved map_reservations: {53=1} 2026-02-11T23:30:36.246+0000 7f416cbfe640 20 osd.0 53 release_map epoch: 53 2026-02-11T23:30:36.246+0000 7f416cbfe640 10 osd.0 53 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 53 is not newer than session's projected_epoch of 53 2026-02-11T23:30:36.255+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 33 ==== osd_map(53..54 src has 1..54) ==== 1208+0+0 (crc 0 0 0) 0x59548c0 con 0x4acbc00 2026-02-11T23:30:36.255+0000 7f415cf02640 20 osd.0 53 OSD::ms_dispatch: osd_map(53..54 src has 1..54) 2026-02-11T23:30:36.255+0000 7f415cf02640 20 osd.0 53 _dispatch 0x59548c0 osd_map(53..54 src has 1..54) 2026-02-11T23:30:36.255+0000 7f415cf02640 3 osd.0 53 handle_osd_map epochs [53,54], i have 53, src has [1,54] 2026-02-11T23:30:36.255+0000 7f415cf02640 10 osd.0 53 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:36.255+0000 7f415cf02640 10 osd.0 53 handle_osd_map got inc map for epoch 54 2026-02-11T23:30:36.255+0000 7f415cf02640 20 osd.0 53 got_full_map 54, nothing requested 2026-02-11T23:30:36.255+0000 7f415cf02640 20 osd.0 53 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:36.255+0000 7f415cf02640 20 osd.0 53 handle_osd_map pg_num_history pg_num_history(e54 pg_nums {1={27=1},2={34=1},3={42=1},4={49=1}} deleted_pools ) 2026-02-11T23:30:36.255+0000 7f415cf02640 10 snap_mapper record_purged_snaps purged_snaps {54={}} 2026-02-11T23:30:36.255+0000 7f415cf02640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:36.255+0000 7f415cf02640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e54 maps [1~54] lci=[5,54] tlb=1) 2026-02-11T23:30:36.255+0000 7f41591a6640 10 osd.0 53 _committed_osd_maps 54..54 2026-02-11T23:30:36.255+0000 7f41591a6640 10 osd.0 53 advance to epoch 54 (<= last 54 <= newest_map 54) 2026-02-11T23:30:36.255+0000 7f41591a6640 20 osd.0 54 consume_map version 54 2026-02-11T23:30:36.255+0000 7f41591a6640 20 osd.0 53 await_reserved_maps epoch:54 2026-02-11T23:30:36.255+0000 7f41591a6640 20 osd.0 53 await_reserved_maps done 0 2026-02-11T23:30:36.255+0000 7f41591a6640 20 osd.0 54 consume_map 54 -- publish done 2026-02-11T23:30:36.255+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:30:36.255+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x37582c0 2026-02-11T23:30:36.255+0000 7f41591a6640 20 osd.0 54 identify_splits_and_merges 4.0 e53 to e54 2026-02-11T23:30:36.255+0000 7f41591a6640 20 osd.0 54 identify_splits_and_merges 4.0 e53 to e54 pg_nums {49=1} 2026-02-11T23:30:36.255+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3700dc0 2026-02-11T23:30:36.255+0000 7f41591a6640 20 osd.0 54 identify_splits_and_merges 1.0 e53 to e54 2026-02-11T23:30:36.255+0000 7f41591a6640 20 osd.0 54 identify_splits_and_merges 1.0 e53 to e54 pg_nums {27=1} 2026-02-11T23:30:36.255+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:36.255+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:36.255+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:36.255+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:36.255+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:36.255+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:36.255+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:36.255+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:36.255+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:36.255+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:36.255+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:36.255+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:36.255+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:36.255+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:36.255+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:36.255+0000 7f41591a6640 20 osd.0 54 prune_pg_created 2026-02-11T23:30:36.255+0000 7f41591a6640 10 osd.0:0.consume_map 54 (was 53) 2026-02-11T23:30:36.255+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:36.255+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:36.255+0000 7f41591a6640 10 osd.0:1.consume_map 54 (was 53) 2026-02-11T23:30:36.255+0000 7f41591a6640 10 osd.0:2.consume_map 54 (was 53) 2026-02-11T23:30:36.255+0000 7f41591a6640 10 osd.0:3.consume_map 54 (was 53) 2026-02-11T23:30:36.255+0000 7f41591a6640 10 osd.0:4.consume_map 54 (was 53) 2026-02-11T23:30:36.255+0000 7f41591a6640 10 osd.0:5.consume_map 54 (was 53) 2026-02-11T23:30:36.255+0000 7f41591a6640 10 osd.0:6.consume_map 54 (was 53) 2026-02-11T23:30:36.255+0000 7f41591a6640 10 osd.0:7.consume_map 54 (was 53) 2026-02-11T23:30:36.255+0000 7f41591a6640 10 osd.0 54 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:36.255+0000 7f41591a6640 15 osd.0 54 enqueue_peering_evt 4.0 epoch_sent: 54 epoch_requested: 54 NullEvt 2026-02-11T23:30:36.255+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 54 epoch_requested: 54 NullEvt) class_id 2 prio 255 cost 10 e54) 2026-02-11T23:30:36.255+0000 7f41591a6640 15 osd.0 54 enqueue_peering_evt 1.0 epoch_sent: 54 epoch_requested: 54 NullEvt 2026-02-11T23:30:36.255+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 54 epoch_requested: 54 NullEvt) class_id 2 prio 255 cost 10 e54) 2026-02-11T23:30:36.255+0000 7f41591a6640 7 osd.0 54 activate_map version 54 2026-02-11T23:30:36.255+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:36.255+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 54 epoch_requested: 54 NullEvt) class_id 2immediate prio 255 cost 10 e54) queued 2026-02-11T23:30:36.255+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:36.255+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 54 epoch_requested: 54 NullEvt) class_id 2immediate prio 255 cost 10 e54) pg 0x5978000 2026-02-11T23:30:36.255+0000 7f4150194640 20 osd.0 54 agent_entry empty queue 2026-02-11T23:30:36.255+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:36.255+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 54 epoch_requested: 54 NullEvt) class_id 2immediate prio 255 cost 10 e54) queued 2026-02-11T23:30:36.255+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:36.255+0000 7f414b98b640 10 osd.0 pg_epoch: 53 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 54 2026-02-11T23:30:36.255+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 54 epoch_requested: 54 NullEvt) class_id 2immediate prio 255 cost 10 e54) pg 0x5946000 2026-02-11T23:30:36.255+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 4.0 53 -> 54 2026-02-11T23:30:36.255+0000 7f414f993640 10 osd.0 pg_epoch: 53 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 54 2026-02-11T23:30:36.255+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 53 -> 54 2026-02-11T23:30:36.255+0000 7f414b98b640 10 osd.0 pg_epoch: 53 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:36.255+0000 7f414f993640 10 osd.0 pg_epoch: 53 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:36.255+0000 7f414b98b640 10 osd.0 pg_epoch: 54 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:36.255+0000 7f414f993640 10 osd.0 pg_epoch: 54 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:36.255+0000 7f414b98b640 10 osd.0 pg_epoch: 54 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:36.255+0000 7f414f993640 10 osd.0 pg_epoch: 54 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:36.255+0000 7f414b98b640 10 osd.0 pg_epoch: 54 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] plpg_on_pool_change 2026-02-11T23:30:36.255+0000 7f414f993640 20 osd.0 54 advance_pg new pool opts old pool opts 2026-02-11T23:30:36.255+0000 7f414b98b640 20 osd.0 pg_epoch: 54 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] hit_set_clear 2026-02-11T23:30:36.255+0000 7f414b98b640 20 osd.0 pg_epoch: 54 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] agent_stop 2026-02-11T23:30:36.255+0000 7f414f993640 10 osd.0 pg_epoch: 54 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_activate_map: epoch range: 54..54 2026-02-11T23:30:36.255+0000 7f414b98b640 20 osd.0 54 advance_pg new pool opts old pool opts 2026-02-11T23:30:36.255+0000 7f414f993640 10 osd.0 pg_epoch: 54 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:36.255+0000 7f414b98b640 10 osd.0 pg_epoch: 54 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_activate_map: epoch range: 54..54 2026-02-11T23:30:36.255+0000 7f414f993640 20 osd.0 pg_epoch: 54 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 30 while current is 54 2026-02-11T23:30:36.255+0000 7f414b98b640 10 osd.0 pg_epoch: 54 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:36.255+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:36.255+0000 7f414b98b640 20 osd.0 pg_epoch: 54 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 53 while current is 54 2026-02-11T23:30:36.255+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:36.255+0000 7f414f993640 10 osd.0 pg_epoch: 54 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 54 epoch_requested: 54 NullEvt 2026-02-11T23:30:36.255+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:36.255+0000 7f414b98b640 10 osd.0 pg_epoch: 54 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:30:36.255+0000 7f414b98b640 10 osd.0 pg_epoch: 54 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 54 epoch_requested: 54 NullEvt 2026-02-11T23:30:36.255+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:36.255+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:36.255+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:36.327+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:30:36.327+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:30:36.327+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:06.327598+0000) 2026-02-11T23:30:36.430+0000 7f41633eb640 10 osd.0 54 tick 2026-02-11T23:30:36.430+0000 7f41633eb640 20 osd.0 54 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:30:36.627+0000 7f4147983640 5 osd.0 54 heartbeat osd_stat(store_statfs(0x384000/0x0/0x384000, data 0x2cd54/0x71000, compress 0x0/0x0/0x0, omap 0x52a3, meta 0x1a4ad5d), peers [1,2,3,4] op hist []) 2026-02-11T23:30:36.627+0000 7f4147983640 20 osd.0 54 check_full_status cur ratio 0, physical ratio 0, new state none 2026-02-11T23:30:36.627+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6816/3043114013,v1:127.0.0.1:6817/3043114013] -- osd_ping(ping e54 up_from 5 ping_stamp 2026-02-11T23:30:36.628476+0000/19.303880692s send_stamp 19.303880692s delta_ub 2.011784554s) -- 0x4c9b080 con 0x48c4400 2026-02-11T23:30:36.627+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6814/3043114013,v1:127.0.0.1:6815/3043114013] -- osd_ping(ping e54 up_from 5 ping_stamp 2026-02-11T23:30:36.628476+0000/19.303880692s send_stamp 19.303880692s delta_ub 2.011784554s) -- 0x4c9a900 con 0x4640c00 2026-02-11T23:30:36.627+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6824/2365263801,v1:127.0.0.1:6825/2365263801] -- osd_ping(ping e54 up_from 5 ping_stamp 2026-02-11T23:30:36.628476+0000/19.303880692s send_stamp 19.303880692s delta_ub 4.566905499s) -- 0x4a89200 con 0x48c4000 2026-02-11T23:30:36.627+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6822/2365263801,v1:127.0.0.1:6823/2365263801] -- osd_ping(ping e54 up_from 5 ping_stamp 2026-02-11T23:30:36.628476+0000/19.303880692s send_stamp 19.303880692s delta_ub 4.566905499s) -- 0x4a89980 con 0x48cd000 2026-02-11T23:30:36.627+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6832/2699377081,v1:127.0.0.1:6833/2699377081] -- osd_ping(ping e54 up_from 5 ping_stamp 2026-02-11T23:30:36.628476+0000/19.303880692s send_stamp 19.303880692s delta_ub 7.219582558s) -- 0x4c9b980 con 0x48cd400 2026-02-11T23:30:36.627+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6830/2699377081,v1:127.0.0.1:6831/2699377081] -- osd_ping(ping e54 up_from 5 ping_stamp 2026-02-11T23:30:36.628476+0000/19.303880692s send_stamp 19.303880692s delta_ub 7.219582558s) -- 0x4c9b800 con 0x48cd800 2026-02-11T23:30:36.627+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6840/3331151400,v1:127.0.0.1:6841/3331151400] -- osd_ping(ping e54 up_from 5 ping_stamp 2026-02-11T23:30:36.628476+0000/19.303880692s send_stamp 19.303880692s delta_ub 9.836717606s) -- 0x4c9a600 con 0x507bc00 2026-02-11T23:30:36.627+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6838/3331151400,v1:127.0.0.1:6839/3331151400] -- osd_ping(ping e54 up_from 5 ping_stamp 2026-02-11T23:30:36.628476+0000/19.303880692s send_stamp 19.303880692s delta_ub 9.836717606s) -- 0x4c9aa80 con 0x5282000 2026-02-11T23:30:36.627+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6822/2365263801 4 ==== osd_ping(ping_reply e52 up_from 14 ping_stamp 2026-02-11T23:30:36.628476+0000/19.303880692s send_stamp 14.737177849s delta_ub -4.566704273s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a600 con 0x48cd000 2026-02-11T23:30:36.627+0000 7f416abfa640 20 osd.0 54 get_con_osd_cluster to osd.2 from_epoch 54 2026-02-11T23:30:36.627+0000 7f416abfa640 20 osd.0 54 get_nextmap_reserved map_reservations: {54=1} 2026-02-11T23:30:36.627+0000 7f416abfa640 20 osd.0 54 release_map epoch: 54 2026-02-11T23:30:36.627+0000 7f416abfa640 10 osd.0 54 maybe_share_map: con v2:127.0.0.1:6820/2365263801 updating session's projected_epoch from 44 to ping map epoch of 52 2026-02-11T23:30:36.627+0000 7f416abfa640 10 osd.0 54 maybe_share_map: con v2:127.0.0.1:6820/2365263801 map epoch 52 -> 54 (shared) 2026-02-11T23:30:36.627+0000 7f416abfa640 10 osd.0 54 send_incremental_map epoch range: (53, 54] to v2:127.0.0.1:6820/2365263801 2026-02-11T23:30:36.627+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6838/3331151400 3 ==== osd_ping(ping_reply e53 up_from 23 ping_stamp 2026-02-11T23:30:36.628476+0000/19.303880692s send_stamp 9.467367172s delta_ub -9.836514473s) ==== 2033+0+0 (crc 0 0 0) 0x4c9aa80 con 0x5282000 2026-02-11T23:30:36.627+0000 7f416cbfe640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6824/2365263801 4 ==== osd_ping(ping_reply e52 up_from 14 ping_stamp 2026-02-11T23:30:36.628476+0000/19.303880692s send_stamp 14.737196922s delta_ub -4.566684246s) ==== 2033+0+0 (crc 0 0 0) 0x4a89200 con 0x48c4000 2026-02-11T23:30:36.627+0000 7f416abfa640 10 osd.0 54 add_map_inc_bl 54 574 bytes 2026-02-11T23:30:36.627+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6820/2365263801,v1:127.0.0.1:6821/2365263801] -- osd_map(53..54 src has 1..54) -- 0x4a30fc0 con 0x507b800 2026-02-11T23:30:36.627+0000 7f416abfa640 20 osd.0 54 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566863537s,-4.566704273s]) 2026-02-11T23:30:36.627+0000 7f416c3fd640 20 osd.0 54 get_con_osd_cluster to osd.4 from_epoch 54 2026-02-11T23:30:36.627+0000 7f416c3fd640 20 osd.0 54 get_nextmap_reserved map_reservations: {54=1} 2026-02-11T23:30:36.627+0000 7f416c3fd640 20 osd.0 54 release_map epoch: 54 2026-02-11T23:30:36.627+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6840/3331151400 3 ==== osd_ping(ping_reply e53 up_from 23 ping_stamp 2026-02-11T23:30:36.628476+0000/19.303880692s send_stamp 9.467392921s delta_ub -9.836487770s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a600 con 0x507bc00 2026-02-11T23:30:36.627+0000 7f416c3fd640 10 osd.0 54 maybe_share_map: con v2:127.0.0.1:6836/3331151400 map epoch 53 -> 54 (shared) 2026-02-11T23:30:36.627+0000 7f416c3fd640 10 osd.0 54 send_incremental_map epoch range: (54, 54] to v2:127.0.0.1:6836/3331151400 2026-02-11T23:30:36.627+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- osd_map(54..54 src has 1..54) -- 0x5954a80 con 0x507ac00 2026-02-11T23:30:36.627+0000 7f416c3fd640 20 osd.0 54 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836812973s,-9.836514473s]) 2026-02-11T23:30:36.627+0000 7f416cbfe640 20 osd.0 54 get_con_osd_cluster to osd.2 from_epoch 54 2026-02-11T23:30:36.627+0000 7f416cbfe640 20 osd.0 54 get_nextmap_reserved map_reservations: {54=1} 2026-02-11T23:30:36.627+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6814/3043114013 5 ==== osd_ping(ping_reply e56 up_from 10 ping_stamp 2026-02-11T23:30:36.628476+0000/19.303880692s send_stamp 17.292480469s delta_ub -2.011401892s) ==== 2033+0+0 (crc 0 0 0) 0x4c9aa80 con 0x4640c00 2026-02-11T23:30:36.627+0000 7f416cbfe640 20 osd.0 54 release_map epoch: 54 2026-02-11T23:30:36.627+0000 7f416cbfe640 10 osd.0 54 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 54 is not newer than session's projected_epoch of 54 2026-02-11T23:30:36.627+0000 7f416cbfe640 20 osd.0 54 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.567018509s,-4.566684246s]) 2026-02-11T23:30:36.627+0000 7f416abfa640 20 osd.0 54 get_con_osd_cluster to osd.4 from_epoch 54 2026-02-11T23:30:36.627+0000 7f416abfa640 20 osd.0 54 get_nextmap_reserved map_reservations: {54=1} 2026-02-11T23:30:36.627+0000 7f416abfa640 20 osd.0 54 release_map epoch: 54 2026-02-11T23:30:36.627+0000 7f416abfa640 10 osd.0 54 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 54 is not newer than session's projected_epoch of 54 2026-02-11T23:30:36.627+0000 7f416abfa640 20 osd.0 54 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836840630s,-9.836487770s]) 2026-02-11T23:30:36.627+0000 7f416c3fd640 20 osd.0 54 get_con_osd_cluster to osd.1 from_epoch 54 2026-02-11T23:30:36.627+0000 7f416c3fd640 20 osd.0 54 get_nextmap_reserved map_reservations: {54=1} 2026-02-11T23:30:36.627+0000 7f416c3fd640 20 osd.0 54 release_map epoch: 54 2026-02-11T23:30:36.627+0000 7f416c3fd640 10 osd.0 54 maybe_share_map: con v2:127.0.0.1:6812/3043114013 updating session's projected_epoch from 46 to ping map epoch of 56 2026-02-11T23:30:36.627+0000 7f416c3fd640 10 osd.0 54 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 54 is not newer than session's projected_epoch of 56 2026-02-11T23:30:36.627+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6816/3043114013 5 ==== osd_ping(ping_reply e56 up_from 10 ping_stamp 2026-02-11T23:30:36.628476+0000/19.303880692s send_stamp 17.292312622s delta_ub -2.011568069s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a600 con 0x48c4400 2026-02-11T23:30:36.627+0000 7f416c3fd640 20 osd.0 54 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011767864s,-2.011401892s]) 2026-02-11T23:30:36.627+0000 7f416abfa640 20 osd.0 54 get_con_osd_cluster to osd.1 from_epoch 54 2026-02-11T23:30:36.627+0000 7f416abfa640 20 osd.0 54 get_nextmap_reserved map_reservations: {54=1} 2026-02-11T23:30:36.627+0000 7f416abfa640 20 osd.0 54 release_map epoch: 54 2026-02-11T23:30:36.627+0000 7f416abfa640 10 osd.0 54 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 54 is not newer than session's projected_epoch of 56 2026-02-11T23:30:36.627+0000 7f416abfa640 20 osd.0 54 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011944771s,-2.011568069s]) 2026-02-11T23:30:36.627+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6830/2699377081 4 ==== osd_ping(ping_reply e55 up_from 19 ping_stamp 2026-02-11T23:30:36.628476+0000/19.303880692s send_stamp 12.084701538s delta_ub -7.219180584s) ==== 2033+0+0 (crc 0 0 0) 0x4c9aa80 con 0x48cd800 2026-02-11T23:30:36.627+0000 7f416c3fd640 20 osd.0 54 get_con_osd_cluster to osd.3 from_epoch 54 2026-02-11T23:30:36.627+0000 7f416c3fd640 20 osd.0 54 get_nextmap_reserved map_reservations: {54=1} 2026-02-11T23:30:36.627+0000 7f416c3fd640 20 osd.0 54 release_map epoch: 54 2026-02-11T23:30:36.627+0000 7f416c3fd640 10 osd.0 54 maybe_share_map: con v2:127.0.0.1:6828/2699377081 updating session's projected_epoch from 53 to ping map epoch of 55 2026-02-11T23:30:36.627+0000 7f416c3fd640 10 osd.0 54 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 54 is not newer than session's projected_epoch of 55 2026-02-11T23:30:36.627+0000 7f416c3fd640 20 osd.0 54 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219563961s,-7.219180584s]) 2026-02-11T23:30:36.627+0000 7f415aefe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.3 v2:127.0.0.1:6828/2699377081 11 ==== osd_map(55..55 src has 1..55) ==== 618+0+0 (crc 0 0 0) 0x4a308c0 con 0x48ccc00 2026-02-11T23:30:36.627+0000 7f415aefe640 20 osd.0 54 OSD::ms_dispatch: osd_map(55..55 src has 1..55) 2026-02-11T23:30:36.627+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6832/2699377081 4 ==== osd_ping(ping_reply e55 up_from 19 ping_stamp 2026-02-11T23:30:36.628476+0000/19.303880692s send_stamp 12.084529877s delta_ub -7.219351768s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a600 con 0x48cd400 2026-02-11T23:30:36.627+0000 7f415aefe640 20 osd.0 54 _dispatch 0x4a308c0 osd_map(55..55 src has 1..55) 2026-02-11T23:30:36.627+0000 7f416abfa640 20 osd.0 54 get_con_osd_cluster to osd.3 from_epoch 54 2026-02-11T23:30:36.627+0000 7f416abfa640 20 osd.0 54 get_nextmap_reserved map_reservations: {54=1} 2026-02-11T23:30:36.627+0000 7f416abfa640 20 osd.0 54 release_map epoch: 54 2026-02-11T23:30:36.627+0000 7f416abfa640 10 osd.0 54 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 54 is not newer than session's projected_epoch of 55 2026-02-11T23:30:36.627+0000 7f416abfa640 20 osd.0 54 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219748974s,-7.219351768s]) 2026-02-11T23:30:36.627+0000 7f415aefe640 3 osd.0 54 handle_osd_map epochs [55,55], i have 54, src has [1,55] 2026-02-11T23:30:36.627+0000 7f415aefe640 10 osd.0 54 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:30:36.627+0000 7f415aefe640 10 osd.0 54 handle_osd_map got inc map for epoch 55 2026-02-11T23:30:36.627+0000 7f415aefe640 20 osd.0 54 got_full_map 55, nothing requested 2026-02-11T23:30:36.627+0000 7f415aefe640 20 osd.0 54 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:36.627+0000 7f415aefe640 20 osd.0 54 handle_osd_map pg_num_history pg_num_history(e55 pg_nums {1={27=1},2={34=1},3={42=1},4={49=1}} deleted_pools ) 2026-02-11T23:30:36.627+0000 7f415aefe640 10 snap_mapper record_purged_snaps purged_snaps {55={}} 2026-02-11T23:30:36.627+0000 7f415aefe640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:36.627+0000 7f415aefe640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e55 maps [1~55] lci=[5,55] tlb=1) 2026-02-11T23:30:36.627+0000 7f415aefe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 16 ==== osd_map(55..56 src has 1..56) ==== 1217+0+0 (unknown 0 0 0) 0x4a31180 con 0x4acb000 2026-02-11T23:30:36.627+0000 7f415aefe640 20 osd.0 54 OSD::ms_dispatch: osd_map(55..56 src has 1..56) 2026-02-11T23:30:36.627+0000 7f415aefe640 20 osd.0 54 _dispatch 0x4a31180 osd_map(55..56 src has 1..56) 2026-02-11T23:30:36.627+0000 7f415aefe640 3 osd.0 54 handle_osd_map epochs [55,56], i have 55, src has [1,56] 2026-02-11T23:30:36.627+0000 7f415aefe640 10 osd.0 54 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:36.627+0000 7f415aefe640 10 osd.0 54 handle_osd_map got inc map for epoch 56 2026-02-11T23:30:36.627+0000 7f415aefe640 20 osd.0 54 got_full_map 56, nothing requested 2026-02-11T23:30:36.627+0000 7f415aefe640 10 osd.0 54 _track_pools_and_pg_num_changes recording new pool 5 pg_num 1 2026-02-11T23:30:36.627+0000 7f415aefe640 20 osd.0 54 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:36.627+0000 7f415aefe640 20 osd.0 54 handle_osd_map pg_num_history pg_num_history(e56 pg_nums {1={27=1},2={34=1},3={42=1},4={49=1},5={56=1}} deleted_pools ) 2026-02-11T23:30:36.627+0000 7f415aefe640 10 snap_mapper record_purged_snaps purged_snaps {56={}} 2026-02-11T23:30:36.627+0000 7f415aefe640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:36.627+0000 7f415aefe640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e56 maps [1~56] lci=[5,56] tlb=1) 2026-02-11T23:30:36.627+0000 7f41591a6640 10 osd.0 54 _committed_osd_maps 55..55 2026-02-11T23:30:36.627+0000 7f41591a6640 10 osd.0 54 advance to epoch 55 (<= last 55 <= newest_map 56) 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0 55 consume_map version 55 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0 54 await_reserved_maps epoch:55 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0 54 await_reserved_maps done 0 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0 55 consume_map 55 -- publish done 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x37582c0 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0 55 identify_splits_and_merges 4.0 e54 to e55 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0 55 identify_splits_and_merges 4.0 e54 to e55 pg_nums {49=1} 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3700dc0 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0 55 identify_splits_and_merges 1.0 e54 to e55 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0 55 identify_splits_and_merges 1.0 e54 to e55 pg_nums {27=1} 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0 55 prune_pg_created 2026-02-11T23:30:36.627+0000 7f41591a6640 10 osd.0:0.consume_map 55 (was 54) 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:36.627+0000 7f41591a6640 10 osd.0:1.consume_map 55 (was 54) 2026-02-11T23:30:36.627+0000 7f41591a6640 10 osd.0:2.consume_map 55 (was 54) 2026-02-11T23:30:36.627+0000 7f41591a6640 10 osd.0:3.consume_map 55 (was 54) 2026-02-11T23:30:36.627+0000 7f41591a6640 10 osd.0:4.consume_map 55 (was 54) 2026-02-11T23:30:36.627+0000 7f41591a6640 10 osd.0:5.consume_map 55 (was 54) 2026-02-11T23:30:36.627+0000 7f41591a6640 10 osd.0:6.consume_map 55 (was 54) 2026-02-11T23:30:36.627+0000 7f41591a6640 10 osd.0:7.consume_map 55 (was 54) 2026-02-11T23:30:36.627+0000 7f41591a6640 10 osd.0 55 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:36.627+0000 7f41591a6640 15 osd.0 55 enqueue_peering_evt 4.0 epoch_sent: 55 epoch_requested: 55 NullEvt 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 55 epoch_requested: 55 NullEvt) class_id 2 prio 255 cost 10 e55) 2026-02-11T23:30:36.627+0000 7f41591a6640 15 osd.0 55 enqueue_peering_evt 1.0 epoch_sent: 55 epoch_requested: 55 NullEvt 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 55 epoch_requested: 55 NullEvt) class_id 2 prio 255 cost 10 e55) 2026-02-11T23:30:36.627+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:36.627+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 55 epoch_requested: 55 NullEvt) class_id 2immediate prio 255 cost 10 e55) queued 2026-02-11T23:30:36.627+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:36.627+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 55 epoch_requested: 55 NullEvt) class_id 2immediate prio 255 cost 10 e55) pg 0x5978000 2026-02-11T23:30:36.627+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:36.627+0000 7f414b98b640 10 osd.0 pg_epoch: 54 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 55 2026-02-11T23:30:36.627+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 4.0 54 -> 55 2026-02-11T23:30:36.627+0000 7f414b98b640 10 osd.0 pg_epoch: 54 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:36.627+0000 7f41591a6640 7 osd.0 55 activate_map version 55 2026-02-11T23:30:36.627+0000 7f414b98b640 10 osd.0 pg_epoch: 55 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:36.627+0000 7f414b98b640 10 osd.0 pg_epoch: 55 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:36.627+0000 7f414b98b640 10 osd.0 pg_epoch: 55 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] plpg_on_pool_change 2026-02-11T23:30:36.627+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:36.627+0000 7f414b98b640 20 osd.0 pg_epoch: 55 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] hit_set_clear 2026-02-11T23:30:36.627+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 55 epoch_requested: 55 NullEvt) class_id 2immediate prio 255 cost 10 e55) queued 2026-02-11T23:30:36.627+0000 7f414b98b640 20 osd.0 pg_epoch: 55 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] agent_stop 2026-02-11T23:30:36.627+0000 7f41591a6640 10 osd.0 55 _committed_osd_maps 56..56 2026-02-11T23:30:36.627+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:36.627+0000 7f41591a6640 10 osd.0 55 advance to epoch 56 (<= last 56 <= newest_map 56) 2026-02-11T23:30:36.627+0000 7f414b98b640 20 osd.0 55 advance_pg new pool opts old pool opts 2026-02-11T23:30:36.627+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 55 epoch_requested: 55 NullEvt) class_id 2immediate prio 255 cost 10 e55) pg 0x5946000 2026-02-11T23:30:36.627+0000 7f414b98b640 10 osd.0 pg_epoch: 55 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_activate_map: epoch range: 55..55 2026-02-11T23:30:36.627+0000 7f414b98b640 10 osd.0 pg_epoch: 55 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:36.627+0000 7f4150194640 20 osd.0 55 agent_entry empty queue 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0 56 consume_map version 56 2026-02-11T23:30:36.627+0000 7f414b98b640 20 osd.0 pg_epoch: 55 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 53 while current is 55 2026-02-11T23:30:36.627+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0 55 await_reserved_maps epoch:56 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0 55 await_reserved_maps done 0 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0 56 consume_map 56 -- publish done 2026-02-11T23:30:36.627+0000 7f414b98b640 10 osd.0 pg_epoch: 55 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:30:36.627+0000 7f414f993640 10 osd.0 pg_epoch: 54 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 55 2026-02-11T23:30:36.627+0000 7f414b98b640 10 osd.0 pg_epoch: 55 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 55 epoch_requested: 55 NullEvt 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x37582c0 2026-02-11T23:30:36.627+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0 56 identify_splits_and_merges 4.0 e55 to e56 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0 56 identify_splits_and_merges 4.0 e55 to e56 pg_nums {49=1} 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3700dc0 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0 56 identify_splits_and_merges 1.0 e55 to e56 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0 56 identify_splits_and_merges 1.0 e55 to e56 pg_nums {27=1} 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:36.627+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 54 -> 55 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:36.627+0000 7f414f993640 10 osd.0 pg_epoch: 54 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:36.627+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0 56 prune_pg_created 2026-02-11T23:30:36.627+0000 7f414f993640 10 osd.0 pg_epoch: 55 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:36.627+0000 7f41591a6640 10 osd.0:0.consume_map 56 (was 55) 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:36.627+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:36.627+0000 7f414f993640 10 osd.0 pg_epoch: 55 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:36.627+0000 7f41591a6640 10 osd.0:1.consume_map 56 (was 55) 2026-02-11T23:30:36.628+0000 7f41591a6640 10 osd.0:2.consume_map 56 (was 55) 2026-02-11T23:30:36.628+0000 7f41591a6640 10 osd.0:3.consume_map 56 (was 55) 2026-02-11T23:30:36.628+0000 7f414f993640 20 osd.0 56 advance_pg new pool opts old pool opts 2026-02-11T23:30:36.628+0000 7f41591a6640 10 osd.0:4.consume_map 56 (was 55) 2026-02-11T23:30:36.628+0000 7f41591a6640 10 osd.0:5.consume_map 56 (was 55) 2026-02-11T23:30:36.628+0000 7f414f993640 10 osd.0 pg_epoch: 55 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_activate_map: epoch range: 55..55 2026-02-11T23:30:36.628+0000 7f41591a6640 10 osd.0:6.consume_map 56 (was 55) 2026-02-11T23:30:36.628+0000 7f414f993640 10 osd.0 pg_epoch: 55 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:36.628+0000 7f41591a6640 10 osd.0:7.consume_map 56 (was 55) 2026-02-11T23:30:36.628+0000 7f414f993640 20 osd.0 pg_epoch: 55 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 30 while current is 55 2026-02-11T23:30:36.628+0000 7f41591a6640 10 osd.0 56 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:36.628+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:36.628+0000 7f414f993640 10 osd.0 pg_epoch: 55 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 55 epoch_requested: 55 NullEvt 2026-02-11T23:30:36.628+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:36.628+0000 7f41591a6640 15 osd.0 56 enqueue_peering_evt 4.0 epoch_sent: 56 epoch_requested: 56 NullEvt 2026-02-11T23:30:36.628+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 56 epoch_requested: 56 NullEvt) class_id 2 prio 255 cost 10 e56) 2026-02-11T23:30:36.628+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:36.628+0000 7f41591a6640 15 osd.0 56 enqueue_peering_evt 1.0 epoch_sent: 56 epoch_requested: 56 NullEvt 2026-02-11T23:30:36.628+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:36.628+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 56 epoch_requested: 56 NullEvt) class_id 2 prio 255 cost 10 e56) 2026-02-11T23:30:36.628+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 56 epoch_requested: 56 NullEvt) class_id 2immediate prio 255 cost 10 e56) queued 2026-02-11T23:30:36.628+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:36.628+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 56 epoch_requested: 56 NullEvt) class_id 2immediate prio 255 cost 10 e56) pg 0x5978000 2026-02-11T23:30:36.628+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:36.628+0000 7f414b98b640 10 osd.0 pg_epoch: 55 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 56 2026-02-11T23:30:36.628+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 4.0 55 -> 56 2026-02-11T23:30:36.628+0000 7f414b98b640 10 osd.0 pg_epoch: 55 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:36.628+0000 7f414b98b640 10 osd.0 pg_epoch: 56 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:36.628+0000 7f414b98b640 10 osd.0 pg_epoch: 56 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:36.628+0000 7f414b98b640 20 osd.0 56 advance_pg new pool opts old pool opts 2026-02-11T23:30:36.628+0000 7f414b98b640 10 osd.0 pg_epoch: 56 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_activate_map: epoch range: 56..56 2026-02-11T23:30:36.628+0000 7f41591a6640 7 osd.0 56 activate_map version 56 2026-02-11T23:30:36.628+0000 7f414b98b640 10 osd.0 pg_epoch: 56 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:36.628+0000 7f414b98b640 20 osd.0 pg_epoch: 56 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 53 while current is 56 2026-02-11T23:30:36.628+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:36.628+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:36.628+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 56 epoch_requested: 56 NullEvt) class_id 2immediate prio 255 cost 10 e56) queued 2026-02-11T23:30:36.628+0000 7f414b98b640 10 osd.0 pg_epoch: 56 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 56 epoch_requested: 56 NullEvt 2026-02-11T23:30:36.628+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:36.628+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:36.628+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 56 epoch_requested: 56 NullEvt) class_id 2immediate prio 255 cost 10 e56) pg 0x5946000 2026-02-11T23:30:36.628+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:36.628+0000 7f414f993640 10 osd.0 pg_epoch: 55 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 56 2026-02-11T23:30:36.628+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 55 -> 56 2026-02-11T23:30:36.628+0000 7f4150194640 20 osd.0 56 agent_entry empty queue 2026-02-11T23:30:36.628+0000 7f414f993640 10 osd.0 pg_epoch: 55 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:36.628+0000 7f414f993640 10 osd.0 pg_epoch: 56 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:36.628+0000 7f414f993640 10 osd.0 pg_epoch: 56 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:36.628+0000 7f414f993640 20 osd.0 56 advance_pg new pool opts old pool opts 2026-02-11T23:30:36.628+0000 7f414f993640 10 osd.0 pg_epoch: 56 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_activate_map: epoch range: 56..56 2026-02-11T23:30:36.628+0000 7f414f993640 10 osd.0 pg_epoch: 56 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:36.628+0000 7f414f993640 20 osd.0 pg_epoch: 56 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 30 while current is 56 2026-02-11T23:30:36.628+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:36.628+0000 7f414f993640 10 osd.0 pg_epoch: 56 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 56 epoch_requested: 56 NullEvt 2026-02-11T23:30:36.628+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:36.628+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:37.039+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.1 127.0.0.1:0/3043114013 9 ==== osd_ping(ping e56 up_from 10 ping_stamp 2026-02-11T23:30:37.040556+0000/17.704301834s send_stamp 17.704301834s delta_ub -2.011401892s) ==== 2033+0+0 (crc 0 0 0) 0x4a89200 con 0x4aca000 2026-02-11T23:30:37.039+0000 7f416cbfe640 20 osd.0 56 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011734962s,-2.011401892s]) 2026-02-11T23:30:37.039+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e56 up_from 5 ping_stamp 2026-02-11T23:30:37.040556+0000/17.704301834s send_stamp 19.716035843s delta_ub 2.011734962s) -- 0x4c9af00 con 0x4aca000 2026-02-11T23:30:37.039+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.1 127.0.0.1:0/3043114013 9 ==== osd_ping(ping e56 up_from 10 ping_stamp 2026-02-11T23:30:37.040556+0000/17.704301834s send_stamp 17.704301834s delta_ub -2.011401892s) ==== 2033+0+0 (crc 0 0 0) 0x4c9aa80 con 0x4aca400 2026-02-11T23:30:37.039+0000 7f416cbfe640 20 osd.0 56 get_con_osd_cluster to osd.1 from_epoch 56 2026-02-11T23:30:37.039+0000 7f416cbfe640 20 osd.0 56 get_nextmap_reserved map_reservations: {56=1} 2026-02-11T23:30:37.039+0000 7f416cbfe640 20 osd.0 56 release_map epoch: 56 2026-02-11T23:30:37.039+0000 7f416cbfe640 10 osd.0 56 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 56 is not newer than session's projected_epoch of 56 2026-02-11T23:30:37.039+0000 7f416c3fd640 20 osd.0 56 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011763096s,-2.011401892s]) 2026-02-11T23:30:37.039+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e56 up_from 5 ping_stamp 2026-02-11T23:30:37.040556+0000/17.704301834s send_stamp 19.716064453s delta_ub 2.011763096s) -- 0x4c9b800 con 0x4aca400 2026-02-11T23:30:37.039+0000 7f416c3fd640 20 osd.0 56 get_con_osd_cluster to osd.1 from_epoch 56 2026-02-11T23:30:37.039+0000 7f416c3fd640 20 osd.0 56 get_nextmap_reserved map_reservations: {56=1} 2026-02-11T23:30:37.039+0000 7f416c3fd640 20 osd.0 56 release_map epoch: 56 2026-02-11T23:30:37.039+0000 7f416c3fd640 10 osd.0 56 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 56 is not newer than session's projected_epoch of 56 2026-02-11T23:30:37.040+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.3 v2:127.0.0.1:6828/2699377081 13 ==== pg_query2(5.0 5.0 query(info 0'0 epoch_sent 56) e56/56) ==== 158+0+0 (crc 0 0 0) 0x5968000 con 0x48ccc00 2026-02-11T23:30:37.040+0000 7f415aefe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.3 v2:127.0.0.1:6828/2699377081 12 ==== osd_map(56..56 src has 1..56) ==== 635+0+0 (crc 0 0 0) 0x4a30fc0 con 0x48ccc00 2026-02-11T23:30:37.040+0000 7f415aefe640 20 osd.0 56 OSD::ms_dispatch: osd_map(56..56 src has 1..56) 2026-02-11T23:30:37.040+0000 7f415aefe640 20 osd.0 56 _dispatch 0x4a30fc0 osd_map(56..56 src has 1..56) 2026-02-11T23:30:37.040+0000 7f416cbfe640 15 osd.0 56 enqueue_peering_evt 5.0 epoch_sent: 56 epoch_requested: 56 MQuery 5.0 from 3 query_epoch 56 query: query(info 0'0 epoch_sent 56) 2026-02-11T23:30:37.040+0000 7f415aefe640 3 osd.0 56 handle_osd_map epochs [56,56], i have 56, src has [1,56] 2026-02-11T23:30:37.040+0000 7f415aefe640 10 osd.0 56 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:30:37.040+0000 7f415aefe640 10 osd.0 56 no new maps here, dropping 2026-02-11T23:30:37.040+0000 7f416cbfe640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 56 epoch_requested: 56 MQuery 5.0 from 3 query_epoch 56 query: query(info 0'0 epoch_sent 56)) class_id 2 prio 255 cost 10 e56) 2026-02-11T23:30:37.040+0000 7f414b98b640 20 osd.0 op_wq(0) _process 5.0 (new) to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:37.040+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 56 epoch_requested: 56 MQuery 5.0 from 3 query_epoch 56 query: query(info 0'0 epoch_sent 56)) class_id 2immediate prio 255 cost 10 e56) queued 2026-02-11T23:30:37.040+0000 7f414b98b640 20 osd.0 op_wq(0) _process 5.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:37.040+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 56 epoch_requested: 56 MQuery 5.0 from 3 query_epoch 56 query: query(info 0'0 epoch_sent 56)) class_id 2immediate prio 255 cost 10 e56) pg 0 2026-02-11T23:30:37.040+0000 7f414b98b640 10 osd.0 56 handle_pg_query_nopg 5.0 2026-02-11T23:30:37.040+0000 7f414b98b640 10 osd.0 56 pg 5.0 dne 2026-02-11T23:30:37.040+0000 7f414b98b640 20 osd.0 56 get_con_osd_cluster to osd.3 from_epoch 56 2026-02-11T23:30:37.040+0000 7f414b98b640 20 osd.0 56 get_nextmap_reserved map_reservations: {56=1} 2026-02-11T23:30:37.040+0000 7f414b98b640 20 osd.0 56 release_map epoch: 56 2026-02-11T23:30:37.040+0000 7f414b98b640 10 osd.0 56 maybe_share_map: con v2:127.0.0.1:6828/2699377081 map epoch 55 -> 56 (shared) 2026-02-11T23:30:37.040+0000 7f414b98b640 10 osd.0 56 send_incremental_map epoch range: (56, 56] to v2:127.0.0.1:6828/2699377081 2026-02-11T23:30:37.040+0000 7f414b98b640 10 osd.0 56 add_map_inc_bl 56 591 bytes 2026-02-11T23:30:37.040+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6828/2699377081,v1:127.0.0.1:6829/2699377081] -- osd_map(56..56 src has 1..56) -- 0x5954e00 con 0x48ccc00 2026-02-11T23:30:37.040+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6828/2699377081,v1:127.0.0.1:6829/2699377081] -- pg_notify2(5.0 5.0 (query:56 sent:56 5.0( DNE empty local-lis/les=0/0 n=0 ec=0/0 lis/c=0/0 les/c/f=0/0/0 sis=0) ([0,0] all_participants={} intervals=[])) e56/56) -- 0x5a8dc00 con 0x48ccc00 2026-02-11T23:30:37.040+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:37.040+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:37.063+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.4 127.0.0.1:0/3331151400 3 ==== osd_ping(ping e56 up_from 23 ping_stamp 2026-02-11T23:30:37.064310+0000/9.903078079s send_stamp 9.903078079s delta_ub -9.836487770s) ==== 2033+0+0 (crc 0 0 0) 0x4c9b800 con 0x507a800 2026-02-11T23:30:37.063+0000 7f416c3fd640 20 osd.0 56 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836727142s,-9.836487770s]) 2026-02-11T23:30:37.063+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e56 up_from 5 ping_stamp 2026-02-11T23:30:37.064310+0000/9.903078079s send_stamp 19.739805222s delta_ub 9.836727142s) -- 0x4c9aa80 con 0x507a800 2026-02-11T23:30:37.063+0000 7f416c3fd640 20 osd.0 56 get_con_osd_cluster to osd.4 from_epoch 56 2026-02-11T23:30:37.063+0000 7f416c3fd640 20 osd.0 56 get_nextmap_reserved map_reservations: {56=1} 2026-02-11T23:30:37.063+0000 7f416c3fd640 20 osd.0 56 release_map epoch: 56 2026-02-11T23:30:37.063+0000 7f416c3fd640 10 osd.0 56 maybe_share_map: con v2:127.0.0.1:6836/3331151400 updating session's projected_epoch from 54 to ping map epoch of 56 2026-02-11T23:30:37.063+0000 7f416c3fd640 10 osd.0 56 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 56 is not newer than session's projected_epoch of 56 2026-02-11T23:30:37.063+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.4 127.0.0.1:0/3331151400 3 ==== osd_ping(ping e56 up_from 23 ping_stamp 2026-02-11T23:30:37.064310+0000/9.903078079s send_stamp 9.903078079s delta_ub -9.836487770s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a600 con 0x507a400 2026-02-11T23:30:37.063+0000 7f416abfa640 20 osd.0 56 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836817741s,-9.836487770s]) 2026-02-11T23:30:37.063+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e56 up_from 5 ping_stamp 2026-02-11T23:30:37.064310+0000/9.903078079s send_stamp 19.739894867s delta_ub 9.836817741s) -- 0x4c9b980 con 0x507a400 2026-02-11T23:30:37.063+0000 7f416abfa640 20 osd.0 56 get_con_osd_cluster to osd.4 from_epoch 56 2026-02-11T23:30:37.063+0000 7f416abfa640 20 osd.0 56 get_nextmap_reserved map_reservations: {56=1} 2026-02-11T23:30:37.063+0000 7f416abfa640 20 osd.0 56 release_map epoch: 56 2026-02-11T23:30:37.063+0000 7f416abfa640 10 osd.0 56 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 56 is not newer than session's projected_epoch of 56 2026-02-11T23:30:37.092+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.3 v2:127.0.0.1:6828/2699377081 14 ==== PGlog(5.0 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 28.550058365s int 16.000000000s) e57/56) ==== 1172+0+0 (crc 0 0 0) 0x4a1b800 con 0x48ccc00 2026-02-11T23:30:37.092+0000 7f416cbfe640 15 osd.0 56 enqueue_peering_evt 5.0 epoch_sent: 57 epoch_requested: 56 MLogRec from 3 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 28.550058365s int 16.000000000s) +create_info 2026-02-11T23:30:37.092+0000 7f416cbfe640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 57 epoch_requested: 56 MLogRec from 3 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 28.550058365s int 16.000000000s) +create_info) class_id 2 prio 255 cost 10 e57) 2026-02-11T23:30:37.092+0000 7f414b98b640 20 osd.0 op_wq(0) _process 5.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:37.092+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 57 epoch_requested: 56 MLogRec from 3 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 28.550058365s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e57) queued 2026-02-11T23:30:37.092+0000 7f414b98b640 20 osd.0 op_wq(0) _process 5.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:37.092+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 57 epoch_requested: 56 MLogRec from 3 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 28.550058365s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e57) pg 0 2026-02-11T23:30:37.092+0000 7f414b98b640 20 osd.0 op_wq(0) _process 5.0 map 57 > 56 2026-02-11T23:30:37.092+0000 7f414b98b640 20 osd.0 op_wq _add_slot_waiter 5.0 peering, item epoch is 57, will wait on OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 57 epoch_requested: 56 MLogRec from 3 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 28.550058365s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e57) 2026-02-11T23:30:37.092+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:37.092+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:37.183+0000 7f4162bea640 10 osd.0 56 tick_without_osd_lock 2026-02-11T23:30:37.183+0000 7f4162bea640 0 osd.0 56 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:37.183+0000 7f4162bea640 20 osd.0 56 tick_without_osd_lock max_waiting_epoch 57, requesting new map 2026-02-11T23:30:37.183+0000 7f4162bea640 10 monclient: _renew_subs 2026-02-11T23:30:37.183+0000 7f4162bea640 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:30:37.183+0000 7f4162bea640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> v2:127.0.0.1:7180/0 -- mon_subscribe({osdmap=57}) -- 0x38144e0 con 0x4acbc00 2026-02-11T23:30:37.183+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:37.183+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.508 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:37.183+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:37.183, recovery is active?:false restrictions:<....> 2026-02-11T23:30:37.183+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:37.183+0000 7f4162bea640 10 osd.0 56 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:37.183+0000 7f4162bea640 20 osd.0 56 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:37.183+0000 7f4162bea640 10 osd.0 56 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:37.183+0000 7f4162bea640 20 osd.0 56 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:37.183+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 34 ==== osd_map(57..57 src has 1..57) ==== 296+0+0 (crc 0 0 0) 0x5954e00 con 0x4acbc00 2026-02-11T23:30:37.183+0000 7f415cf02640 20 osd.0 56 OSD::ms_dispatch: osd_map(57..57 src has 1..57) 2026-02-11T23:30:37.183+0000 7f415cf02640 20 osd.0 56 _dispatch 0x5954e00 osd_map(57..57 src has 1..57) 2026-02-11T23:30:37.183+0000 7f415cf02640 3 osd.0 56 handle_osd_map epochs [57,57], i have 56, src has [1,57] 2026-02-11T23:30:37.183+0000 7f415cf02640 10 osd.0 56 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:30:37.183+0000 7f415cf02640 10 osd.0 56 handle_osd_map got inc map for epoch 57 2026-02-11T23:30:37.183+0000 7f415cf02640 20 osd.0 56 got_full_map 57, nothing requested 2026-02-11T23:30:37.183+0000 7f415cf02640 20 osd.0 56 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:37.183+0000 7f415cf02640 20 osd.0 56 handle_osd_map pg_num_history pg_num_history(e57 pg_nums {1={27=1},2={34=1},3={42=1},4={49=1},5={56=1}} deleted_pools ) 2026-02-11T23:30:37.183+0000 7f415cf02640 10 snap_mapper record_purged_snaps purged_snaps {57={}} 2026-02-11T23:30:37.183+0000 7f415cf02640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:37.183+0000 7f415cf02640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e57 maps [1~57] lci=[5,57] tlb=1) 2026-02-11T23:30:37.183+0000 7f41591a6640 10 osd.0 56 _committed_osd_maps 57..57 2026-02-11T23:30:37.183+0000 7f41591a6640 10 osd.0 56 advance to epoch 57 (<= last 57 <= newest_map 57) 2026-02-11T23:30:37.183+0000 7f41591a6640 20 osd.0 57 consume_map version 57 2026-02-11T23:30:37.183+0000 7f41591a6640 20 osd.0 56 await_reserved_maps epoch:57 2026-02-11T23:30:37.183+0000 7f41591a6640 20 osd.0 56 await_reserved_maps done 0 2026-02-11T23:30:37.183+0000 7f41591a6640 20 osd.0 57 consume_map 57 -- publish done 2026-02-11T23:30:37.183+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 3 slots 2026-02-11T23:30:37.183+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:5.0slot:0x3759080 2026-02-11T23:30:37.183+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot 5.0 has no pg and waiting_for_split 2026-02-11T23:30:37.183+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x37582c0 2026-02-11T23:30:37.183+0000 7f41591a6640 20 osd.0 57 identify_splits_and_merges 4.0 e56 to e57 2026-02-11T23:30:37.183+0000 7f41591a6640 20 osd.0 57 identify_splits_and_merges 4.0 e56 to e57 pg_nums {49=1} 2026-02-11T23:30:37.183+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3700dc0 2026-02-11T23:30:37.183+0000 7f41591a6640 20 osd.0 57 identify_splits_and_merges 1.0 e56 to e57 2026-02-11T23:30:37.183+0000 7f41591a6640 20 osd.0 57 identify_splits_and_merges 1.0 e56 to e57 pg_nums {27=1} 2026-02-11T23:30:37.183+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:37.183+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:37.183+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:37.183+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:37.183+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:37.183+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:37.183+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:37.183+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:37.183+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:37.183+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:37.183+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:37.183+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:37.183+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:37.183+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:37.183+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:37.183+0000 7f41591a6640 20 osd.0 57 prune_pg_created 2026-02-11T23:30:37.183+0000 7f41591a6640 10 osd.0:0.consume_map 57 (was 56) 2026-02-11T23:30:37.183+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 5.0 2026-02-11T23:30:37.183+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 5.0 pending_peering first epoch 57 <= 57, requeueing 2026-02-11T23:30:37.183+0000 7f41591a6640 20 osd.0:0._wake_pg_slot _wake_pg_slot 5.0 to_process <> waiting <> waiting_peering {57=} 2026-02-11T23:30:37.183+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:37.183+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:37.183+0000 7f41591a6640 10 osd.0:1.consume_map 57 (was 56) 2026-02-11T23:30:37.183+0000 7f41591a6640 10 osd.0:2.consume_map 57 (was 56) 2026-02-11T23:30:37.183+0000 7f41591a6640 10 osd.0:3.consume_map 57 (was 56) 2026-02-11T23:30:37.183+0000 7f41591a6640 10 osd.0:4.consume_map 57 (was 56) 2026-02-11T23:30:37.183+0000 7f41591a6640 10 osd.0:5.consume_map 57 (was 56) 2026-02-11T23:30:37.183+0000 7f41591a6640 10 osd.0:6.consume_map 57 (was 56) 2026-02-11T23:30:37.183+0000 7f41591a6640 10 osd.0:7.consume_map 57 (was 56) 2026-02-11T23:30:37.183+0000 7f41591a6640 10 osd.0 57 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:37.183+0000 7f414b98b640 20 osd.0 op_wq(0) _process 5.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:37.183+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 57 epoch_requested: 56 MLogRec from 3 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 28.550058365s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e57) queued 2026-02-11T23:30:37.183+0000 7f41591a6640 15 osd.0 57 enqueue_peering_evt 4.0 epoch_sent: 57 epoch_requested: 57 NullEvt 2026-02-11T23:30:37.183+0000 7f414b98b640 20 osd.0 op_wq(0) _process 5.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:37.183+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 57 epoch_requested: 56 MLogRec from 3 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 28.550058365s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e57) pg 0 2026-02-11T23:30:37.183+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 57 epoch_requested: 57 NullEvt) class_id 2 prio 255 cost 10 e57) 2026-02-11T23:30:37.183+0000 7f414b98b640 20 osd.0 op_wq(0) _process 5.0 no pg, should create on OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 57 epoch_requested: 56 MLogRec from 3 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 28.550058365s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e57) 2026-02-11T23:30:37.183+0000 7f414b98b640 10 osd.0 57 _make_pg 5.0 2026-02-11T23:30:37.183+0000 7f414b98b640 10 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:5, shard:-1, shard_prefix: , prefixes: {}] SnapMapper 2026-02-11T23:30:37.183+0000 7f414b98b640 20 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:5, shard:-1, shard_prefix: , prefixes: {}] update_bits new_bits: 0 2026-02-11T23:30:37.183+0000 7f414b98b640 20 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:5, shard:-1, shard_prefix: , prefixes: {"0000000000000005."}] update_bits prefix updated 2026-02-11T23:30:37.183+0000 7f414b98b640 10 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:5, shard:-1, shard_prefix: , prefixes: {"0000000000000005."}] reset_prefix_itr::from <0> to ::update_bits 2026-02-11T23:30:37.183+0000 7f414b98b640 5 osd.0 pg_epoch: 56 pg[5.0(unlocked)] enter Initial 2026-02-11T23:30:37.183+0000 7f414b98b640 20 osd.0 pg_epoch: 56 pg[5.0(unlocked)] enter NotTrimming 2026-02-11T23:30:37.183+0000 7f414b98b640 10 osd.0 pg_epoch: 56 pg[5.0(unlocked)] scrubber: FSM: -- state -->> NotActive 2026-02-11T23:30:37.183+0000 7f414b98b640 10 osd.0 pg_epoch: 56 pg[5.0( DNE empty local-lis/les=0/0 n=0 ec=0/0 lis/c=0/0 les/c/f=0/0/0 sis=0) [] r=-1 lpr=0 crt=0'0 unknown mbc={}] PeeringState::init init role 2 up [3,4,0] acting [3,4,0] history ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56 past_intervals ([0,0] all_participants={} intervals=[]) 2026-02-11T23:30:37.183+0000 7f414b98b640 20 osd.0 pg_epoch: 56 pg[5.0( empty local-lis/les=0/0 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=0 crt=0'0 unknown mbc={}] PeeringState::on_new_interval 2026-02-11T23:30:37.183+0000 7f414b98b640 20 osd.0 pg_epoch: 56 pg[5.0( empty local-lis/les=0/0 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=0 crt=0'0 unknown mbc={}] PeeringState::on_new_interval upacting_features 0x3f0ffffffffdffff from [3,4,0]+[3,4,0] 2026-02-11T23:30:37.183+0000 7f414b98b640 20 osd.0 pg_epoch: 56 pg[5.0( empty local-lis/les=0/0 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=0 crt=0'0 unknown mbc={}] PeeringState::on_new_interval checking missing set deletes flag. missing = missing(0 may_include_deletes = 1) 2026-02-11T23:30:37.183+0000 7f414b98b640 10 osd.0 pg_epoch: 56 pg[5.0( empty local-lis/les=0/0 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=0 crt=0'0 unknown mbc={}] PeeringState::init_hb_stamps now [0x56858f0] 2026-02-11T23:30:37.183+0000 7f414b98b640 10 osd.0 pg_epoch: 56 pg[5.0( empty local-lis/les=0/0 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=0 crt=0'0 unknown mbc={}] PeeringState::on_new_interval prior_readable_until_ub 0.000000000s (mnow 19.859844208s + 0.000000000s) 2026-02-11T23:30:37.183+0000 7f414b98b640 10 osd.0 pg_epoch: 56 pg[5.0( empty local-lis/les=0/0 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=0 crt=0'0 unknown mbc={}] cancel_recovery 2026-02-11T23:30:37.183+0000 7f414b98b640 10 osd.0 pg_epoch: 56 pg[5.0( empty local-lis/les=0/0 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=0 crt=0'0 unknown mbc={}] clear_recovery_state 2026-02-11T23:30:37.183+0000 7f414b98b640 15 osd.0 pg_epoch: 56 pg[5.0( empty local-lis/les=0/0 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=0 crt=0'0 unknown mbc={}] _clear_recovery_state 2026-02-11T23:30:37.183+0000 7f414b98b640 10 osd.0 pg_epoch: 56 pg[5.0( empty local-lis/les=0/0 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=0 crt=0'0 unknown mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:37.183+0000 7f414b98b640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:37.183+0000 7f414b98b640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:37.183+0000 7f414b98b640 6 write_log_and_missing with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, writeout_from: 4294967295'18446744073709551615, trimmed: , trimmed_dups: , clear_divergent_priors: 0 2026-02-11T23:30:37.183+0000 7f414b98b640 10 _write_log_and_missing clearing up to 0'0 dirty_to_dups=0'0 dirty_from_dups=4294967295'18446744073709551615 write_from_dups=4294967295'18446744073709551615 trimmed_dups.size()=0 2026-02-11T23:30:37.183+0000 7f414b98b640 10 _write_log_and_missing going to encode log.dups.size()=0 2026-02-11T23:30:37.183+0000 7f414b98b640 10 _write_log_and_missing 1st round encoded log.dups.size()=0 2026-02-11T23:30:37.183+0000 7f414b98b640 10 _write_log_and_missing 2st round encoded log.dups.size()=0 2026-02-11T23:30:37.183+0000 7f414b98b640 10 end of _write_log_and_missing 2026-02-11T23:30:37.183+0000 7f414b98b640 10 osd.0 pg_epoch: 56 pg[5.0( empty local-lis/les=0/0 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=0 crt=0'0 unknown mbc={}] handle_initialize 2026-02-11T23:30:37.183+0000 7f414b98b640 5 osd.0 pg_epoch: 56 pg[5.0( empty local-lis/les=0/0 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=0 crt=0'0 unknown mbc={}] exit Initial 0.000145 0 0.000000 2026-02-11T23:30:37.183+0000 7f414b98b640 5 osd.0 pg_epoch: 56 pg[5.0( empty local-lis/les=0/0 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=0 crt=0'0 unknown mbc={}] enter Reset 2026-02-11T23:30:37.183+0000 7f414b98b640 20 osd.0 pg_epoch: 56 pg[5.0( empty local-lis/les=0/0 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=0 crt=0'0 unknown mbc={}] PeeringState::set_last_peering_reset set_last_peering_reset 56 2026-02-11T23:30:37.183+0000 7f414b98b640 10 osd.0 pg_epoch: 56 pg[5.0( empty local-lis/les=0/0 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 unknown mbc={}] PeeringState::set_last_peering_reset Clearing blocked outgoing recovery messages 2026-02-11T23:30:37.183+0000 7f414b98b640 10 osd.0 pg_epoch: 56 pg[5.0( empty local-lis/les=0/0 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 unknown mbc={}] PeeringState::set_last_peering_reset Not blocking outgoing recovery messages 2026-02-11T23:30:37.183+0000 7f414b98b640 10 osd.0 pg_epoch: 56 pg[5.0( empty local-lis/les=0/0 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 unknown mbc={}] handle_activate_map: epoch range: 56..56 2026-02-11T23:30:37.183+0000 7f414b98b640 10 osd.0 pg_epoch: 56 pg[5.0( empty local-lis/les=0/0 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 unknown mbc={}] PeeringState::activate_map 2026-02-11T23:30:37.183+0000 7f414b98b640 5 osd.0 pg_epoch: 56 pg[5.0( empty local-lis/les=0/0 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 unknown mbc={}] exit Reset 0.000022 1 0.000024 2026-02-11T23:30:37.183+0000 7f414b98b640 5 osd.0 pg_epoch: 56 pg[5.0( empty local-lis/les=0/0 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 unknown mbc={}] enter Started 2026-02-11T23:30:37.183+0000 7f414b98b640 5 osd.0 pg_epoch: 56 pg[5.0( empty local-lis/les=0/0 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 unknown mbc={}] enter Start 2026-02-11T23:30:37.183+0000 7f414b98b640 1 osd.0 pg_epoch: 56 pg[5.0( empty local-lis/les=0/0 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 unknown mbc={}] state: transitioning to Stray 2026-02-11T23:30:37.183+0000 7f414b98b640 5 osd.0 pg_epoch: 56 pg[5.0( empty local-lis/les=0/0 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 unknown mbc={}] exit Start 0.000004 0 0.000000 2026-02-11T23:30:37.183+0000 7f414b98b640 5 osd.0 pg_epoch: 56 pg[5.0( empty local-lis/les=0/0 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 unknown mbc={}] enter Started/Stray 2026-02-11T23:30:37.183+0000 7f414b98b640 20 osd.0 pg_epoch: 56 pg[5.0( empty local-lis/les=0/0 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 unknown mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 56 while current is 56 2026-02-11T23:30:37.183+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:37.183+0000 7f414b98b640 10 osd.0 pg_epoch: 56 pg[5.0( empty local-lis/les=0/0 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 unknown mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:30:37.183+0000 7f414b98b640 10 osd.0 57 handle_pg_create_info new pg pg[5.0( empty local-lis/les=0/0 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 unknown mbc={}] 2026-02-11T23:30:37.183+0000 7f414b98b640 10 osd.0:0._attach_pg 5.0 0x4aa0000 2026-02-11T23:30:37.183+0000 7f414b98b640 20 osd.0:0._wake_pg_slot _wake_pg_slot 5.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:37.183+0000 7f414b98b640 20 osd.0 57 identify_splits_and_merges 5.0 e56 to e57 2026-02-11T23:30:37.183+0000 7f414b98b640 20 osd.0 57 identify_splits_and_merges 5.0 e56 to e57 pg_nums {56=1} 2026-02-11T23:30:37.183+0000 7f414b98b640 10 osd.0:0._prime_splits 2026-02-11T23:30:37.183+0000 7f41591a6640 15 osd.0 57 enqueue_peering_evt 1.0 epoch_sent: 57 epoch_requested: 57 NullEvt 2026-02-11T23:30:37.183+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 57 epoch_requested: 57 NullEvt) class_id 2 prio 255 cost 10 e57) 2026-02-11T23:30:37.183+0000 7f414b98b640 10 osd.0 pg_epoch: 56 pg[5.0( empty local-lis/les=0/0 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 unknown mbc={}] handle_advance_map: 57 2026-02-11T23:30:37.183+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:37.183+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 57 epoch_requested: 57 NullEvt) class_id 2immediate prio 255 cost 10 e57) queued 2026-02-11T23:30:37.183+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:37.183+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 57 epoch_requested: 57 NullEvt) class_id 2immediate prio 255 cost 10 e57) pg 0x5978000 2026-02-11T23:30:37.183+0000 7f41591a6640 7 osd.0 57 activate_map version 57 2026-02-11T23:30:37.183+0000 7f414f993640 10 osd.0 pg_epoch: 56 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 57 2026-02-11T23:30:37.183+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 5.0 56 -> 57 2026-02-11T23:30:37.183+0000 7f414b98b640 10 osd.0 pg_epoch: 56 pg[5.0( empty local-lis/les=0/0 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 unknown mbc={}] PeeringState::advance_map handle_advance_map [3,4,0]/[3,4,0] -- 3/3 2026-02-11T23:30:37.183+0000 7f4150194640 20 osd.0 57 agent_entry empty queue 2026-02-11T23:30:37.183+0000 7f414f993640 20 osd.0:0.update_pg_epoch 4.0 56 -> 57 2026-02-11T23:30:37.183+0000 7f414b98b640 10 osd.0 pg_epoch: 57 pg[5.0( empty local-lis/les=0/0 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 unknown mbc={}] state: Started advmap 2026-02-11T23:30:37.183+0000 7f414f993640 10 osd.0 pg_epoch: 56 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:37.183+0000 7f414b98b640 10 osd.0 pg_epoch: 57 pg[5.0( empty local-lis/les=0/0 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 unknown mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:37.183+0000 7f414f993640 10 osd.0 pg_epoch: 57 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:37.183+0000 7f414f993640 10 osd.0 pg_epoch: 57 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:37.183+0000 7f414f993640 20 osd.0 57 advance_pg new pool opts old pool opts 2026-02-11T23:30:37.183+0000 7f414f993640 10 osd.0 pg_epoch: 57 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_activate_map: epoch range: 57..57 2026-02-11T23:30:37.183+0000 7f414f993640 10 osd.0 pg_epoch: 57 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:37.183+0000 7f414b98b640 20 osd.0 57 advance_pg new pool opts old pool opts 2026-02-11T23:30:37.183+0000 7f414f993640 20 osd.0 pg_epoch: 57 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 53 while current is 57 2026-02-11T23:30:37.183+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:37.183+0000 7f414b98b640 10 osd.0 pg_epoch: 57 pg[5.0( empty local-lis/les=0/0 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 unknown mbc={}] handle_activate_map: epoch range: 57..57 2026-02-11T23:30:37.183+0000 7f414f993640 10 osd.0 pg_epoch: 57 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 57 epoch_requested: 57 NullEvt 2026-02-11T23:30:37.183+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:37.183+0000 7f414b98b640 10 osd.0 pg_epoch: 57 pg[5.0( empty local-lis/les=0/0 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 unknown mbc={}] PeeringState::activate_map 2026-02-11T23:30:37.183+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:37.183+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 57 epoch_requested: 57 NullEvt) class_id 2immediate prio 255 cost 10 e57) queued 2026-02-11T23:30:37.183+0000 7f414b98b640 20 osd.0 pg_epoch: 57 pg[5.0( empty local-lis/les=0/0 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 unknown mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 56 while current is 57 2026-02-11T23:30:37.183+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:37.183+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:37.183+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 57 epoch_requested: 57 NullEvt) class_id 2immediate prio 255 cost 10 e57) pg 0x5946000 2026-02-11T23:30:37.183+0000 7f414b98b640 10 osd.0 pg_epoch: 57 pg[5.0( empty local-lis/les=0/0 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 unknown mbc={}] do_peering_event: epoch_sent: 57 epoch_requested: 56 MLogRec from 3 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 28.550058365s int 16.000000000s) +create_info 2026-02-11T23:30:37.183+0000 7f414f993640 10 osd.0 pg_epoch: 56 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 57 2026-02-11T23:30:37.183+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 56 -> 57 2026-02-11T23:30:37.183+0000 7f414b98b640 10 osd.0 pg_epoch: 57 pg[5.0( empty local-lis/les=0/0 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 unknown mbc={}] state: got info+log from osd.3 5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) log((0'0,0'0], crt=0'0) 2026-02-11T23:30:37.183+0000 7f414f993640 10 osd.0 pg_epoch: 56 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:37.183+0000 7f414b98b640 10 merge_log log((0'0,0'0], crt=0'0) from osd.3 into log((0'0,0'0], crt=0'0) 2026-02-11T23:30:37.183+0000 7f414b98b640 5 merge_log_dups log.dups.size()=0olog.dups.size()=0 2026-02-11T23:30:37.183+0000 7f414b98b640 5 end of merge_log_dups changed=0 log.dups.size()=0 olog.dups.size()=0 2026-02-11T23:30:37.183+0000 7f414f993640 10 osd.0 pg_epoch: 57 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:37.183+0000 7f414b98b640 10 merge_log result log((0'0,0'0], crt=0'0) missing(0 may_include_deletes = 1) changed=0 2026-02-11T23:30:37.183+0000 7f414f993640 10 osd.0 pg_epoch: 57 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:37.183+0000 7f414f993640 20 osd.0 57 advance_pg new pool opts old pool opts 2026-02-11T23:30:37.183+0000 7f414b98b640 10 osd.0 pg_epoch: 57 pg[5.0( empty local-lis/les=0/0 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 unknown mbc={}] PeeringState::proc_lease pg_lease(ru 0.000000000s ub 28.550058365s int 16.000000000s) 2026-02-11T23:30:37.183+0000 7f414f993640 10 osd.0 pg_epoch: 57 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_activate_map: epoch range: 57..57 2026-02-11T23:30:37.183+0000 7f414f993640 10 osd.0 pg_epoch: 57 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:37.183+0000 7f414b98b640 20 osd.0 pg_epoch: 57 pg[5.0( empty local-lis/les=0/0 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 unknown mbc={}] PeeringState::proc_lease peer_clock_delta_lb -7.219748974s -> ruub 35.769805908s 2026-02-11T23:30:37.183+0000 7f414f993640 20 osd.0 pg_epoch: 57 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 30 while current is 57 2026-02-11T23:30:37.183+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:37.183+0000 7f414b98b640 20 osd.0 pg_epoch: 57 pg[5.0( empty local-lis/les=0/0 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 unknown mbc={}] PeeringState::proc_lease readable_until_ub now 35.769805908s 2026-02-11T23:30:37.183+0000 7f414f993640 10 osd.0 pg_epoch: 57 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 57 epoch_requested: 57 NullEvt 2026-02-11T23:30:37.183+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:37.183+0000 7f414b98b640 5 osd.0 pg_epoch: 57 pg[5.0( empty local-lis/les=0/0 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 unknown mbc={}] exit Started/Stray 0.000188 3 0.000025 2026-02-11T23:30:37.183+0000 7f414b98b640 5 osd.0 pg_epoch: 57 pg[5.0( empty local-lis/les=0/0 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 unknown mbc={}] enter Started/ReplicaActive 2026-02-11T23:30:37.183+0000 7f414b98b640 5 osd.0 pg_epoch: 57 pg[5.0( empty local-lis/les=0/0 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 unknown mbc={}] enter Started/ReplicaActive/RepNotRecovering 2026-02-11T23:30:37.183+0000 7f414b98b640 10 osd.0 pg_epoch: 57 pg[5.0( empty local-lis/les=0/0 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 unknown mbc={}] state: In ReplicaActive, about to call activate 2026-02-11T23:30:37.183+0000 7f414b98b640 10 osd.0 pg_epoch: 57 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 unknown mbc={}] PeeringState::activate activate - no missing, moving last_complete 0'0 -> 0'0 2026-02-11T23:30:37.183+0000 7f414b98b640 10 osd.0 pg_epoch: 57 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 unknown mbc={}] scrubber: on_replica_activate 2026-02-11T23:30:37.183+0000 7f414b98b640 20 scrubberFSM event: --vvvv---- ReplicaActivate 2026-02-11T23:30:37.183+0000 7f414b98b640 10 osd.0 pg_epoch: 57 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 unknown mbc={}] scrubber: FSM: -- state -->> ReplicaActive 2026-02-11T23:30:37.183+0000 7f414b98b640 10 osd.0 pg_epoch: 57 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 unknown mbc={}] scrubber: FSM: -- state -->> ReplicaActive/ReplicaIdle 2026-02-11T23:30:37.183+0000 7f414b98b640 20 scrubberFSM event: --^^^^---- ReplicaActivate 2026-02-11T23:30:37.183+0000 7f414b98b640 10 osd.0 pg_epoch: 57 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 unknown mbc={}] state: Activate Finished 2026-02-11T23:30:37.183+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:37.183+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:37.183+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:37.183+0000 7f414f993640 15 osd.0 57 enqueue_peering_evt 5.0 epoch_sent: 57 epoch_requested: 57 ActivateCommitted from 57 processed at 57 2026-02-11T23:30:37.183+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:37.183+0000 7f414f993640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 57 epoch_requested: 57 ActivateCommitted from 57 processed at 57) class_id 2 prio 255 cost 10 e57) 2026-02-11T23:30:37.183+0000 7f414f993640 20 osd.0 op_wq(0) _process 5.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:37.183+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 57 epoch_requested: 57 ActivateCommitted from 57 processed at 57) class_id 2immediate prio 255 cost 10 e57) queued 2026-02-11T23:30:37.183+0000 7f414f993640 20 osd.0 op_wq(0) _process 5.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:37.183+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 57 epoch_requested: 57 ActivateCommitted from 57 processed at 57) class_id 2immediate prio 255 cost 10 e57) pg 0x4aa0000 2026-02-11T23:30:37.183+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:37.183+0000 7f414f993640 10 osd.0 pg_epoch: 57 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 unknown mbc={}] do_peering_event: epoch_sent: 57 epoch_requested: 57 ActivateCommitted from 57 processed at 57 2026-02-11T23:30:37.183+0000 7f414f993640 10 osd.0 pg_epoch: 57 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 unknown mbc={}] state: react 57 telling primary 2026-02-11T23:30:37.183+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:37.183+0000 7f414f993640 20 osd.0 57 get_con_osd_cluster to osd.3 from_epoch 57 2026-02-11T23:30:37.183+0000 7f414f993640 20 osd.0 57 get_nextmap_reserved map_reservations: {57=1} 2026-02-11T23:30:37.183+0000 7f414f993640 20 osd.0 57 release_map epoch: 57 2026-02-11T23:30:37.183+0000 7f414f993640 10 osd.0 57 maybe_share_map: con v2:127.0.0.1:6828/2699377081 map epoch 56 -> 57 (shared) 2026-02-11T23:30:37.183+0000 7f414f993640 10 osd.0 57 send_incremental_map epoch range: (57, 57] to v2:127.0.0.1:6828/2699377081 2026-02-11T23:30:37.183+0000 7f414f993640 10 osd.0 57 add_map_inc_bl 57 252 bytes 2026-02-11T23:30:37.183+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6828/2699377081,v1:127.0.0.1:6829/2699377081] -- osd_map(57..57 src has 1..57) -- 0x4a316c0 con 0x48ccc00 2026-02-11T23:30:37.183+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6828/2699377081,v1:127.0.0.1:6829/2699377081] -- pg_info2(5.0 5.0 5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/0 les/c/f=57/0/0 sis=56) e57/57) -- 0x373b100 con 0x48ccc00 2026-02-11T23:30:37.183+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:37.259+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.3 v2:127.0.0.1:6828/2699377081 15 ==== pg_lease(5.0 pg_lease(ru 28.550058365s ub 28.717052460s int 16.000000000s) e57/57) ==== 58+0+0 (crc 0 0 0) 0x4a89200 con 0x48ccc00 2026-02-11T23:30:37.259+0000 7f416cbfe640 15 osd.0 57 enqueue_peering_evt 5.0 epoch_sent: 57 epoch_requested: 57 MLease epoch 57 from osd.3 pg_lease(ru 28.550058365s ub 28.717052460s int 16.000000000s) 2026-02-11T23:30:37.259+0000 7f416cbfe640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 57 epoch_requested: 57 MLease epoch 57 from osd.3 pg_lease(ru 28.550058365s ub 28.717052460s int 16.000000000s)) class_id 2 prio 255 cost 10 e57) 2026-02-11T23:30:37.259+0000 7f414b98b640 20 osd.0 op_wq(0) _process 5.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:37.259+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 57 epoch_requested: 57 MLease epoch 57 from osd.3 pg_lease(ru 28.550058365s ub 28.717052460s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e57) queued 2026-02-11T23:30:37.259+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.3 v2:127.0.0.1:6828/2699377081 16 ==== pg_info2(5.0 5.0 5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/0 les/c/f=57/0/0 sis=56) e57/57) ==== 1126+0+0 (crc 0 0 0) 0x5a8c000 con 0x48ccc00 2026-02-11T23:30:37.259+0000 7f414b98b640 20 osd.0 op_wq(0) _process 5.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:37.259+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 57 epoch_requested: 57 MLease epoch 57 from osd.3 pg_lease(ru 28.550058365s ub 28.717052460s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e57) pg 0x4aa0000 2026-02-11T23:30:37.259+0000 7f414b98b640 10 osd.0 pg_epoch: 57 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 57 epoch_requested: 57 MLease epoch 57 from osd.3 pg_lease(ru 28.550058365s ub 28.717052460s int 16.000000000s) 2026-02-11T23:30:37.259+0000 7f416cbfe640 15 osd.0 57 enqueue_peering_evt 5.0 epoch_sent: 57 epoch_requested: 57 MInfoRec from 3 info: 5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/0 les/c/f=57/0/0 sis=56) pg_lease(ru 28.550058365s ub 28.717052460s int 16.000000000s) 2026-02-11T23:30:37.259+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:37.259+0000 7f414b98b640 10 osd.0 pg_epoch: 57 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 active mbc={}] PeeringState::proc_lease pg_lease(ru 28.550058365s ub 28.717052460s int 16.000000000s) 2026-02-11T23:30:37.259+0000 7f416cbfe640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 57 epoch_requested: 57 MInfoRec from 3 info: 5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/0 les/c/f=57/0/0 sis=56) pg_lease(ru 28.550058365s ub 28.717052460s int 16.000000000s)) class_id 2 prio 255 cost 10 e57) 2026-02-11T23:30:37.259+0000 7f414b98b640 20 osd.0 pg_epoch: 57 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 active mbc={}] PeeringState::proc_lease peer_clock_delta_ub -7.219351768s -> ru 35.769409180s 2026-02-11T23:30:37.259+0000 7f414b98b640 20 osd.0 pg_epoch: 57 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 active mbc={}] PeeringState::proc_lease readable_until now 35.769409180s 2026-02-11T23:30:37.259+0000 7f414b98b640 20 osd.0 pg_epoch: 57 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 active mbc={}] PeeringState::proc_lease peer_clock_delta_lb -7.219748974s -> ruub 35.936801910s 2026-02-11T23:30:37.259+0000 7f414b98b640 20 osd.0 pg_epoch: 57 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 active mbc={}] PeeringState::proc_lease readable_until_ub now 35.936801910s 2026-02-11T23:30:37.259+0000 7f414b98b640 20 osd.0 57 get_con_osd_cluster to osd.3 from_epoch 57 2026-02-11T23:30:37.259+0000 7f414b98b640 20 osd.0 57 get_nextmap_reserved map_reservations: {57=1} 2026-02-11T23:30:37.259+0000 7f414b98b640 20 osd.0 57 release_map epoch: 57 2026-02-11T23:30:37.259+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6828/2699377081,v1:127.0.0.1:6829/2699377081] -- pg_lease_ack(5.0 pg_lease_ack(ruub 28.717052460s) e57/57) -- 0x37009a0 con 0x48ccc00 2026-02-11T23:30:37.259+0000 7f414f993640 20 osd.0 op_wq(0) _process 5.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:37.259+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 57 epoch_requested: 57 MInfoRec from 3 info: 5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/0 les/c/f=57/0/0 sis=56) pg_lease(ru 28.550058365s ub 28.717052460s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e57) queued 2026-02-11T23:30:37.259+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:37.259+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:37.259+0000 7f414f993640 20 osd.0 op_wq(0) _process 5.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:37.259+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 57 epoch_requested: 57 MInfoRec from 3 info: 5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/0 les/c/f=57/0/0 sis=56) pg_lease(ru 28.550058365s ub 28.717052460s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e57) pg 0x4aa0000 2026-02-11T23:30:37.259+0000 7f414f993640 10 osd.0 pg_epoch: 57 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=0/0 les/c/f=0/0/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 57 epoch_requested: 57 MInfoRec from 3 info: 5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/0 les/c/f=57/0/0 sis=56) pg_lease(ru 28.550058365s ub 28.717052460s int 16.000000000s) 2026-02-11T23:30:37.259+0000 7f414f993640 20 osd.0 pg_epoch: 57 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/0 les/c/f=57/0/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 active mbc={}] PeeringState::update_history advanced history from ec=56/56 lis/c=56/0 les/c/f=57/0/0 sis=56 2026-02-11T23:30:37.259+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:37.259+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:37.259+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.3 v2:127.0.0.1:6828/2699377081 17 ==== pg_info2(5.0 5.0 5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=56) e57/57) ==== 1126+0+0 (crc 0 0 0) 0x5a8c000 con 0x48ccc00 2026-02-11T23:30:37.259+0000 7f416cbfe640 15 osd.0 57 enqueue_peering_evt 5.0 epoch_sent: 57 epoch_requested: 57 MInfoRec from 3 info: 5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=56) pg_lease(ru 28.550058365s ub 28.717052460s int 16.000000000s) 2026-02-11T23:30:37.259+0000 7f416cbfe640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 57 epoch_requested: 57 MInfoRec from 3 info: 5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=56) pg_lease(ru 28.550058365s ub 28.717052460s int 16.000000000s)) class_id 2 prio 255 cost 10 e57) 2026-02-11T23:30:37.260+0000 7f414b98b640 20 osd.0 op_wq(0) _process 5.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:37.260+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 57 epoch_requested: 57 MInfoRec from 3 info: 5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=56) pg_lease(ru 28.550058365s ub 28.717052460s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e57) queued 2026-02-11T23:30:37.260+0000 7f414b98b640 20 osd.0 op_wq(0) _process 5.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:37.260+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 57 epoch_requested: 57 MInfoRec from 3 info: 5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=56) pg_lease(ru 28.550058365s ub 28.717052460s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e57) pg 0x4aa0000 2026-02-11T23:30:37.260+0000 7f414b98b640 10 osd.0 pg_epoch: 57 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/0 les/c/f=57/0/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 57 epoch_requested: 57 MInfoRec from 3 info: 5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=56) pg_lease(ru 28.550058365s ub 28.717052460s int 16.000000000s) 2026-02-11T23:30:37.260+0000 7f414b98b640 20 osd.0 pg_epoch: 57 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 active mbc={}] PeeringState::update_history advanced history from ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=56 2026-02-11T23:30:37.260+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:37.260+0000 7f414b98b640 20 osd.0 pg_epoch: 57 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 active mbc={}] PeeringState::update_history clearing past_intervals 2026-02-11T23:30:37.260+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:37.260+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:37.260+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:37.260+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:37.260+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:37.260+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:37.310+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 35 ==== osd_map(57..58 src has 1..58) ==== 878+0+0 (crc 0 0 0) 0x4a316c0 con 0x4acbc00 2026-02-11T23:30:37.310+0000 7f415cf02640 20 osd.0 57 OSD::ms_dispatch: osd_map(57..58 src has 1..58) 2026-02-11T23:30:37.310+0000 7f415cf02640 20 osd.0 57 _dispatch 0x4a316c0 osd_map(57..58 src has 1..58) 2026-02-11T23:30:37.310+0000 7f415cf02640 3 osd.0 57 handle_osd_map epochs [57,58], i have 57, src has [1,58] 2026-02-11T23:30:37.310+0000 7f415cf02640 10 osd.0 57 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:37.310+0000 7f415cf02640 10 osd.0 57 handle_osd_map got inc map for epoch 58 2026-02-11T23:30:37.310+0000 7f415cf02640 20 osd.0 57 got_full_map 58, nothing requested 2026-02-11T23:30:37.310+0000 7f415cf02640 20 osd.0 57 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:37.310+0000 7f415cf02640 20 osd.0 57 handle_osd_map pg_num_history pg_num_history(e58 pg_nums {1={27=1},2={34=1},3={42=1},4={49=1},5={56=1}} deleted_pools ) 2026-02-11T23:30:37.310+0000 7f415cf02640 10 snap_mapper record_purged_snaps purged_snaps {58={}} 2026-02-11T23:30:37.310+0000 7f415cf02640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:37.310+0000 7f415cf02640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e58 maps [1~58] lci=[5,58] tlb=1) 2026-02-11T23:30:37.310+0000 7f41591a6640 10 osd.0 57 _committed_osd_maps 58..58 2026-02-11T23:30:37.310+0000 7f41591a6640 10 osd.0 57 advance to epoch 58 (<= last 58 <= newest_map 58) 2026-02-11T23:30:37.310+0000 7f41591a6640 20 osd.0 58 consume_map version 58 2026-02-11T23:30:37.310+0000 7f41591a6640 20 osd.0 57 await_reserved_maps epoch:58 2026-02-11T23:30:37.310+0000 7f41591a6640 20 osd.0 57 await_reserved_maps done 0 2026-02-11T23:30:37.310+0000 7f41591a6640 20 osd.0 58 consume_map 58 -- publish done 2026-02-11T23:30:37.310+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 3 slots 2026-02-11T23:30:37.310+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:5.0slot:0x3759080 2026-02-11T23:30:37.310+0000 7f41591a6640 20 osd.0 58 identify_splits_and_merges 5.0 e57 to e58 2026-02-11T23:30:37.310+0000 7f41591a6640 20 osd.0 58 identify_splits_and_merges 5.0 e57 to e58 pg_nums {56=1} 2026-02-11T23:30:37.310+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x37582c0 2026-02-11T23:30:37.310+0000 7f41591a6640 20 osd.0 58 identify_splits_and_merges 4.0 e57 to e58 2026-02-11T23:30:37.310+0000 7f41591a6640 20 osd.0 58 identify_splits_and_merges 4.0 e57 to e58 pg_nums {49=1} 2026-02-11T23:30:37.310+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3700dc0 2026-02-11T23:30:37.310+0000 7f41591a6640 20 osd.0 58 identify_splits_and_merges 1.0 e57 to e58 2026-02-11T23:30:37.310+0000 7f41591a6640 20 osd.0 58 identify_splits_and_merges 1.0 e57 to e58 pg_nums {27=1} 2026-02-11T23:30:37.310+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:37.310+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:37.310+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:37.310+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:37.310+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:37.310+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:37.310+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:37.310+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:37.310+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:37.310+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:37.310+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:37.310+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:37.310+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:37.310+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:37.310+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:37.310+0000 7f41591a6640 20 osd.0 58 prune_pg_created 2026-02-11T23:30:37.310+0000 7f41591a6640 10 osd.0:0.consume_map 58 (was 57) 2026-02-11T23:30:37.310+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 5.0 2026-02-11T23:30:37.310+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:37.310+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:37.310+0000 7f41591a6640 10 osd.0:1.consume_map 58 (was 57) 2026-02-11T23:30:37.310+0000 7f41591a6640 10 osd.0:2.consume_map 58 (was 57) 2026-02-11T23:30:37.310+0000 7f41591a6640 10 osd.0:3.consume_map 58 (was 57) 2026-02-11T23:30:37.310+0000 7f41591a6640 10 osd.0:4.consume_map 58 (was 57) 2026-02-11T23:30:37.310+0000 7f41591a6640 10 osd.0:5.consume_map 58 (was 57) 2026-02-11T23:30:37.310+0000 7f41591a6640 10 osd.0:6.consume_map 58 (was 57) 2026-02-11T23:30:37.310+0000 7f41591a6640 10 osd.0:7.consume_map 58 (was 57) 2026-02-11T23:30:37.310+0000 7f41591a6640 10 osd.0 58 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:37.310+0000 7f41591a6640 15 osd.0 58 enqueue_peering_evt 5.0 epoch_sent: 58 epoch_requested: 58 NullEvt 2026-02-11T23:30:37.310+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 58 epoch_requested: 58 NullEvt) class_id 2 prio 255 cost 10 e58) 2026-02-11T23:30:37.310+0000 7f41591a6640 15 osd.0 58 enqueue_peering_evt 4.0 epoch_sent: 58 epoch_requested: 58 NullEvt 2026-02-11T23:30:37.310+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 58 epoch_requested: 58 NullEvt) class_id 2 prio 255 cost 10 e58) 2026-02-11T23:30:37.310+0000 7f414f993640 20 osd.0 op_wq(0) _process 5.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:37.310+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 58 epoch_requested: 58 NullEvt) class_id 2immediate prio 255 cost 10 e58) queued 2026-02-11T23:30:37.310+0000 7f414f993640 20 osd.0 op_wq(0) _process 5.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:37.310+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 58 epoch_requested: 58 NullEvt) class_id 2immediate prio 255 cost 10 e58) pg 0x4aa0000 2026-02-11T23:30:37.310+0000 7f41591a6640 15 osd.0 58 enqueue_peering_evt 1.0 epoch_sent: 58 epoch_requested: 58 NullEvt 2026-02-11T23:30:37.310+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 58 epoch_requested: 58 NullEvt) class_id 2 prio 255 cost 10 e58) 2026-02-11T23:30:37.310+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:37.310+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 58 epoch_requested: 58 NullEvt) class_id 2immediate prio 255 cost 10 e58) queued 2026-02-11T23:30:37.310+0000 7f414f993640 10 osd.0 pg_epoch: 57 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 active mbc={}] handle_advance_map: 58 2026-02-11T23:30:37.310+0000 7f414f993640 20 osd.0:0.update_pg_epoch 5.0 57 -> 58 2026-02-11T23:30:37.310+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:37.310+0000 7f414f993640 10 osd.0 pg_epoch: 57 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [3,4,0]/[3,4,0] -- 3/3 2026-02-11T23:30:37.310+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 58 epoch_requested: 58 NullEvt) class_id 2immediate prio 255 cost 10 e58) pg 0x5978000 2026-02-11T23:30:37.310+0000 7f41591a6640 7 osd.0 58 activate_map version 58 2026-02-11T23:30:37.310+0000 7f414f993640 10 osd.0 pg_epoch: 58 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:37.310+0000 7f414f993640 10 osd.0 pg_epoch: 58 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:37.310+0000 7f414f993640 10 osd.0 pg_epoch: 58 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 active mbc={}] plpg_on_pool_change 2026-02-11T23:30:37.310+0000 7f414b98b640 10 osd.0 pg_epoch: 57 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 58 2026-02-11T23:30:37.310+0000 7f414f993640 20 osd.0 pg_epoch: 58 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 active mbc={}] hit_set_clear 2026-02-11T23:30:37.310+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 4.0 57 -> 58 2026-02-11T23:30:37.310+0000 7f414f993640 20 osd.0 pg_epoch: 58 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 active mbc={}] agent_stop 2026-02-11T23:30:37.310+0000 7f414b98b640 10 osd.0 pg_epoch: 57 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:37.310+0000 7f414f993640 20 osd.0 58 advance_pg new pool opts old pool opts 2026-02-11T23:30:37.310+0000 7f4150194640 20 osd.0 58 agent_entry empty queue 2026-02-11T23:30:37.310+0000 7f414f993640 10 osd.0 pg_epoch: 58 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 active mbc={}] handle_activate_map: epoch range: 58..58 2026-02-11T23:30:37.310+0000 7f414b98b640 10 osd.0 pg_epoch: 58 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:37.310+0000 7f414f993640 10 osd.0 pg_epoch: 58 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:37.310+0000 7f414f993640 20 osd.0 pg_epoch: 58 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 57 while current is 58 2026-02-11T23:30:37.310+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:37.310+0000 7f414b98b640 10 osd.0 pg_epoch: 58 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:37.310+0000 7f414f993640 10 osd.0 pg_epoch: 58 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 active mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:30:37.310+0000 7f414b98b640 20 osd.0 58 advance_pg new pool opts old pool opts 2026-02-11T23:30:37.310+0000 7f414f993640 10 osd.0 pg_epoch: 58 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 58 epoch_requested: 58 NullEvt 2026-02-11T23:30:37.310+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:37.310+0000 7f414b98b640 10 osd.0 pg_epoch: 58 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_activate_map: epoch range: 58..58 2026-02-11T23:30:37.310+0000 7f414b98b640 10 osd.0 pg_epoch: 58 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:37.310+0000 7f414b98b640 20 osd.0 pg_epoch: 58 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 53 while current is 58 2026-02-11T23:30:37.310+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:37.310+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:37.310+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 58 epoch_requested: 58 NullEvt) class_id 2immediate prio 255 cost 10 e58) queued 2026-02-11T23:30:37.310+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:37.310+0000 7f414b98b640 10 osd.0 pg_epoch: 58 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 58 epoch_requested: 58 NullEvt 2026-02-11T23:30:37.310+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 58 epoch_requested: 58 NullEvt) class_id 2immediate prio 255 cost 10 e58) pg 0x5946000 2026-02-11T23:30:37.310+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:37.310+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:37.310+0000 7f414f993640 10 osd.0 pg_epoch: 57 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 58 2026-02-11T23:30:37.310+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 57 -> 58 2026-02-11T23:30:37.310+0000 7f414f993640 10 osd.0 pg_epoch: 57 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:37.310+0000 7f414f993640 10 osd.0 pg_epoch: 58 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:37.310+0000 7f414f993640 10 osd.0 pg_epoch: 58 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:37.310+0000 7f414f993640 20 osd.0 58 advance_pg new pool opts old pool opts 2026-02-11T23:30:37.310+0000 7f414f993640 10 osd.0 pg_epoch: 58 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_activate_map: epoch range: 58..58 2026-02-11T23:30:37.310+0000 7f414f993640 10 osd.0 pg_epoch: 58 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:37.311+0000 7f414f993640 20 osd.0 pg_epoch: 58 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 30 while current is 58 2026-02-11T23:30:37.311+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:37.311+0000 7f414f993640 10 osd.0 pg_epoch: 58 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 58 epoch_requested: 58 NullEvt 2026-02-11T23:30:37.311+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:37.311+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:37.327+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:30:37.327+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:30:37.327+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:07.327656+0000) 2026-02-11T23:30:37.445+0000 7f41633eb640 10 osd.0 58 tick 2026-02-11T23:30:37.445+0000 7f41633eb640 20 osd.0 58 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:30:37.947+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.3 v2:127.0.0.1:6828/2699377081 18 ==== pg_query2(5.0 5.0 query(info 0'0 epoch_sent 60) e60/60) ==== 158+0+0 (crc 0 0 0) 0x5968000 con 0x48ccc00 2026-02-11T23:30:37.947+0000 7f416cbfe640 15 osd.0 58 enqueue_peering_evt 5.0 epoch_sent: 60 epoch_requested: 60 MQuery 5.0 from 3 query_epoch 60 query: query(info 0'0 epoch_sent 60) 2026-02-11T23:30:37.947+0000 7f416cbfe640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 60 epoch_requested: 60 MQuery 5.0 from 3 query_epoch 60 query: query(info 0'0 epoch_sent 60)) class_id 2 prio 255 cost 10 e60) 2026-02-11T23:30:37.947+0000 7f414b98b640 20 osd.0 op_wq(0) _process 5.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:37.947+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 60 epoch_requested: 60 MQuery 5.0 from 3 query_epoch 60 query: query(info 0'0 epoch_sent 60)) class_id 2immediate prio 255 cost 10 e60) queued 2026-02-11T23:30:37.947+0000 7f414b98b640 20 osd.0 op_wq(0) _process 5.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:37.947+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 60 epoch_requested: 60 MQuery 5.0 from 3 query_epoch 60 query: query(info 0'0 epoch_sent 60)) class_id 2immediate prio 255 cost 10 e60) pg 0x4aa0000 2026-02-11T23:30:37.947+0000 7f414b98b640 20 osd.0 op_wq _add_slot_waiter 5.0 peering, item epoch is 60, will wait on OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 60 epoch_requested: 60 MQuery 5.0 from 3 query_epoch 60 query: query(info 0'0 epoch_sent 60)) class_id 2immediate prio 255 cost 10 e60) 2026-02-11T23:30:37.947+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:37.947+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:37.947+0000 7f415aefe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.3 v2:127.0.0.1:6828/2699377081 19 ==== osd_map(57..60 src has 1..60) ==== 2042+0+0 (crc 0 0 0) 0x5955180 con 0x48ccc00 2026-02-11T23:30:37.947+0000 7f415aefe640 20 osd.0 58 OSD::ms_dispatch: osd_map(57..60 src has 1..60) 2026-02-11T23:30:37.947+0000 7f415aefe640 20 osd.0 58 _dispatch 0x5955180 osd_map(57..60 src has 1..60) 2026-02-11T23:30:37.947+0000 7f415aefe640 3 osd.0 58 handle_osd_map epochs [57,60], i have 58, src has [1,60] 2026-02-11T23:30:37.947+0000 7f415aefe640 10 osd.0 58 handle_osd_map: received 0 full maps and 4 incremental maps 2026-02-11T23:30:37.947+0000 7f415aefe640 10 osd.0 58 handle_osd_map got inc map for epoch 59 2026-02-11T23:30:37.947+0000 7f415aefe640 20 osd.0 58 got_full_map 59, nothing requested 2026-02-11T23:30:37.947+0000 7f415aefe640 10 osd.0 58 handle_osd_map got inc map for epoch 60 2026-02-11T23:30:37.947+0000 7f415aefe640 20 osd.0 58 got_full_map 60, nothing requested 2026-02-11T23:30:37.947+0000 7f415aefe640 20 osd.0 58 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:37.947+0000 7f415aefe640 20 osd.0 58 handle_osd_map pg_num_history pg_num_history(e60 pg_nums {1={27=1},2={34=1},3={42=1},4={49=1},5={56=1}} deleted_pools ) 2026-02-11T23:30:37.947+0000 7f415aefe640 10 snap_mapper record_purged_snaps purged_snaps {59={},60={}} 2026-02-11T23:30:37.947+0000 7f415aefe640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:37.947+0000 7f415aefe640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e60 maps [1~60] lci=[5,60] tlb=1) 2026-02-11T23:30:37.948+0000 7f41591a6640 10 osd.0 58 _committed_osd_maps 59..60 2026-02-11T23:30:37.948+0000 7f41591a6640 10 osd.0 58 advance to epoch 59 (<= last 60 <= newest_map 60) 2026-02-11T23:30:37.948+0000 7f41591a6640 10 osd.0 59 advance to epoch 60 (<= last 60 <= newest_map 60) 2026-02-11T23:30:37.948+0000 7f41591a6640 20 osd.0 60 consume_map version 60 2026-02-11T23:30:37.948+0000 7f41591a6640 20 osd.0 58 await_reserved_maps epoch:60 2026-02-11T23:30:37.948+0000 7f41591a6640 20 osd.0 58 await_reserved_maps done 0 2026-02-11T23:30:37.948+0000 7f41591a6640 20 osd.0 60 consume_map 60 -- publish done 2026-02-11T23:30:37.948+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 3 slots 2026-02-11T23:30:37.948+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:5.0slot:0x3759080 2026-02-11T23:30:37.948+0000 7f41591a6640 20 osd.0 60 identify_splits_and_merges 5.0 e58 to e60 2026-02-11T23:30:37.948+0000 7f41591a6640 20 osd.0 60 identify_splits_and_merges 5.0 e58 to e60 pg_nums {56=1} 2026-02-11T23:30:37.948+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x37582c0 2026-02-11T23:30:37.948+0000 7f41591a6640 20 osd.0 60 identify_splits_and_merges 4.0 e58 to e60 2026-02-11T23:30:37.948+0000 7f41591a6640 20 osd.0 60 identify_splits_and_merges 4.0 e58 to e60 pg_nums {49=1} 2026-02-11T23:30:37.948+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3700dc0 2026-02-11T23:30:37.948+0000 7f41591a6640 20 osd.0 60 identify_splits_and_merges 1.0 e58 to e60 2026-02-11T23:30:37.948+0000 7f41591a6640 20 osd.0 60 identify_splits_and_merges 1.0 e58 to e60 pg_nums {27=1} 2026-02-11T23:30:37.948+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:37.948+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:37.948+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:37.948+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:37.948+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:37.948+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:37.948+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:37.948+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:37.948+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:37.948+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:37.948+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:37.948+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:37.948+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:37.948+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:37.948+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:37.948+0000 7f41591a6640 20 osd.0 60 prune_pg_created 2026-02-11T23:30:37.948+0000 7f41591a6640 10 osd.0:0.consume_map 60 (was 58) 2026-02-11T23:30:37.948+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 5.0 2026-02-11T23:30:37.948+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 5.0 pending_peering first epoch 60 <= 60, requeueing 2026-02-11T23:30:37.948+0000 7f41591a6640 20 osd.0:0._wake_pg_slot _wake_pg_slot 5.0 to_process <> waiting <> waiting_peering {60=} 2026-02-11T23:30:37.948+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:37.948+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:37.948+0000 7f41591a6640 10 osd.0:1.consume_map 60 (was 58) 2026-02-11T23:30:37.948+0000 7f41591a6640 10 osd.0:2.consume_map 60 (was 58) 2026-02-11T23:30:37.948+0000 7f41591a6640 10 osd.0:3.consume_map 60 (was 58) 2026-02-11T23:30:37.948+0000 7f41591a6640 10 osd.0:4.consume_map 60 (was 58) 2026-02-11T23:30:37.948+0000 7f41591a6640 10 osd.0:5.consume_map 60 (was 58) 2026-02-11T23:30:37.948+0000 7f41591a6640 10 osd.0:6.consume_map 60 (was 58) 2026-02-11T23:30:37.948+0000 7f41591a6640 10 osd.0:7.consume_map 60 (was 58) 2026-02-11T23:30:37.948+0000 7f41591a6640 10 osd.0 60 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:37.948+0000 7f414b98b640 20 osd.0 op_wq(0) _process 5.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:37.948+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 60 epoch_requested: 60 MQuery 5.0 from 3 query_epoch 60 query: query(info 0'0 epoch_sent 60)) class_id 2immediate prio 255 cost 10 e60) queued 2026-02-11T23:30:37.948+0000 7f414b98b640 20 osd.0 op_wq(0) _process 5.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:37.948+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 60 epoch_requested: 60 MQuery 5.0 from 3 query_epoch 60 query: query(info 0'0 epoch_sent 60)) class_id 2immediate prio 255 cost 10 e60) pg 0x4aa0000 2026-02-11T23:30:37.948+0000 7f41591a6640 15 osd.0 60 enqueue_peering_evt 5.0 epoch_sent: 60 epoch_requested: 60 NullEvt 2026-02-11T23:30:37.948+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 60 epoch_requested: 60 NullEvt) class_id 2 prio 255 cost 10 e60) 2026-02-11T23:30:37.948+0000 7f41591a6640 15 osd.0 60 enqueue_peering_evt 4.0 epoch_sent: 60 epoch_requested: 60 NullEvt 2026-02-11T23:30:37.948+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 60 epoch_requested: 60 NullEvt) class_id 2 prio 255 cost 10 e60) 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 pg_epoch: 58 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 active mbc={}] handle_advance_map: 59 2026-02-11T23:30:37.948+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 5.0 58 -> 59 2026-02-11T23:30:37.948+0000 7f41591a6640 15 osd.0 60 enqueue_peering_evt 1.0 epoch_sent: 60 epoch_requested: 60 NullEvt 2026-02-11T23:30:37.948+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 60 epoch_requested: 60 NullEvt) class_id 2 prio 255 cost 10 e60) 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 pg_epoch: 58 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [3,4]/[3,4] -- 3/3 2026-02-11T23:30:37.948+0000 7f414f993640 20 osd.0 op_wq(0) _process 5.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:37.948+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 60 epoch_requested: 60 NullEvt) class_id 2immediate prio 255 cost 10 e60) queued 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 pg_epoch: 59 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:37.948+0000 7f414b98b640 20 osd.0 pg_epoch: 59 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 active mbc={}] PeeringState::should_restart_peering new interval newup [3,4] newacting [3,4] 2026-02-11T23:30:37.948+0000 7f41591a6640 7 osd.0 60 activate_map version 60 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 pg_epoch: 59 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 active mbc={}] state: should_restart_peering, transitioning to Reset 2026-02-11T23:30:37.948+0000 7f414b98b640 5 osd.0 pg_epoch: 59 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 active mbc={}] exit Started/ReplicaActive/RepNotRecovering 0.764943 8 0.000093 2026-02-11T23:30:37.948+0000 7f414b98b640 5 osd.0 pg_epoch: 59 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 active mbc={}] exit Started/ReplicaActive 0.764949 0 0.000000 2026-02-11T23:30:37.948+0000 7f414b98b640 5 osd.0 pg_epoch: 59 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 active mbc={}] exit Started 0.765152 0 0.000000 2026-02-11T23:30:37.948+0000 7f4150194640 20 osd.0 60 agent_entry empty queue 2026-02-11T23:30:37.948+0000 7f414b98b640 5 osd.0 pg_epoch: 59 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 active mbc={}] enter Reset 2026-02-11T23:30:37.948+0000 7f414b98b640 20 osd.0 pg_epoch: 59 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=56) [3,4,0] r=2 lpr=56 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset set_last_peering_reset 59 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 pg_epoch: 59 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=56) [3,4,0] r=2 lpr=59 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset Clearing blocked outgoing recovery messages 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 pg_epoch: 59 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=56) [3,4,0] r=2 lpr=59 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset Not blocking outgoing recovery messages 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 pg_epoch: 59 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=56) [3,4,0] r=2 lpr=59 crt=0'0 active mbc={}] state: Reset advmap 2026-02-11T23:30:37.948+0000 7f414b98b640 20 osd.0 pg_epoch: 59 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=56) [3,4,0] r=2 lpr=59 crt=0'0 active mbc={}] PeeringState::should_restart_peering new interval newup [3,4] newacting [3,4] 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 pg_epoch: 59 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=56) [3,4,0] r=2 lpr=59 crt=0'0 active mbc={}] state: should restart peering, calling start_peering_interval again 2026-02-11T23:30:37.948+0000 7f414b98b640 20 osd.0 pg_epoch: 59 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=56) [3,4,0] r=2 lpr=59 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset set_last_peering_reset 59 2026-02-11T23:30:37.948+0000 7f414b98b640 15 osd.0 pg_epoch: 59 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=56) [3,4] r=2 lpr=59 crt=0'0 active mbc={}] clear_stats 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 pg_epoch: 59 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=56) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 active mbc={}] PeeringState::start_peering_interval : check_new_interval output: check_new_interval interval(56-58 up [3, 4, 0](3) acting [3, 4, 0](3)) up_thru 56 up_from 19 last_epoch_clean 57 interval(56-58 up [3, 4, 0](3) acting [3, 4, 0](3) maybe_went_rw) : primary up 19-56 includes interval 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 pg_epoch: 59 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=56) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 active mbc={}] PeeringState::start_peering_interval noting past ([56,58] all_participants={0, 3, 4} intervals=[([56,58] acting={0, 3, 4})]) 2026-02-11T23:30:37.948+0000 7f414b98b640 20 osd.0 pg_epoch: 59 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 active mbc={}] PeeringState::on_new_interval 2026-02-11T23:30:37.948+0000 7f414b98b640 20 osd.0 pg_epoch: 59 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 active mbc={}] PeeringState::on_new_interval upacting_features 0x3f0ffffffffdffff from [3,4]+[3,4] 2026-02-11T23:30:37.948+0000 7f414b98b640 20 osd.0 pg_epoch: 59 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 active mbc={}] PeeringState::on_new_interval checking missing set deletes flag. missing = missing(0 may_include_deletes = 1) 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 pg_epoch: 59 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 active mbc={}] PeeringState::init_hb_stamps now [] 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 pg_epoch: 59 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311687469s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 active pruub 35.936801910s@ mbc={}] PeeringState::on_new_interval prior_readable_until_ub 35.936801910s (mnow 20.625112534s + 15.311687469s) 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 pg_epoch: 59 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311687469s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 active pruub 35.936801910s@ mbc={}] cancel_recovery 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 pg_epoch: 59 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311687469s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 active pruub 35.936801910s@ mbc={}] clear_recovery_state 2026-02-11T23:30:37.948+0000 7f414b98b640 15 osd.0 pg_epoch: 59 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311687469s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 active pruub 35.936801910s@ mbc={}] _clear_recovery_state 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 pg_epoch: 59 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311687469s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 active pruub 35.936801910s@ mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:37.948+0000 7f414b98b640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:37.948+0000 7f414b98b640 20 osd.0 pg_epoch: 59 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311687469s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 active pruub 35.936801910s@ mbc={}] scrubber: FSM: ReplicaActive::exit() 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 pg_epoch: 59 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311687469s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 active pruub 35.936801910s@ mbc={}] scrubber: FSM: ReplicaActive::clear_remote_reservation(): pending_reservation_nonce 0, reservation_granted false 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 pg_epoch: 59 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311687469s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 active pruub 35.936801910s@ mbc={}] scrubber: FSM: -- state -->> NotActive 2026-02-11T23:30:37.948+0000 7f414b98b640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:37.948+0000 7f414b98b640 1 osd.0 pg_epoch: 59 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311687469s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 active pruub 35.936801910s@ mbc={}] PeeringState::start_peering_interval up [3,4,0] -> [3,4], acting [3,4,0] -> [3,4], acting_primary 3 -> 3, up_primary 3 -> 3, role 2 -> -1, features acting 4544132024016699391 upacting 4544132024016699391 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 pg_epoch: 59 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311687469s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown pruub 35.936801910s@ mbc={}] PeeringState::clear_primary_state clear_primary_state 2026-02-11T23:30:37.948+0000 7f414b98b640 20 osd.0 pg_epoch: 59 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311687469s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown pruub 35.936801910s@ mbc={}] clear_primary_state 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 pg_epoch: 59 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311687469s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown pruub 35.936801910s@ mbc={}] release_backoffs [5:00000000::::0,MAX) 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 pg_epoch: 59 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311687469s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown pruub 35.936801910s@ mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:37.948+0000 7f414b98b640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:37.948+0000 7f414b98b640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:37.948+0000 7f414b98b640 20 osd.0 pg_epoch: 59 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311687469s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown pruub 35.936801910s@ mbc={}] agent_stop 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 pg_epoch: 59 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311687469s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown pruub 35.936801910s@ mbc={}] on_change 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 pg_epoch: 59 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311687469s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown pruub 35.936801910s@ mbc={}] cancel_copy_ops 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 pg_epoch: 59 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311687469s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown pruub 35.936801910s@ mbc={}] cancel_flush_ops 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 pg_epoch: 59 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311687469s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown pruub 35.936801910s@ mbc={}] cancel_proxy_ops 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 pg_epoch: 59 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311687469s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown pruub 35.936801910s@ mbc={}] cancel_manifest_ops 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 pg_epoch: 59 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311687469s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown pruub 35.936801910s@ mbc={}] cancel_cls_gather_ops 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 pg_epoch: 59 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311687469s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown pruub 35.936801910s@ mbc={}] on_change_cleanup 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 pg_epoch: 59 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311687469s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown pruub 35.936801910s@ mbc={}] on_change 2026-02-11T23:30:37.948+0000 7f414b98b640 20 osd.0 pg_epoch: 59 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311687469s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown pruub 35.936801910s@ mbc={}] exit NotTrimming 2026-02-11T23:30:37.948+0000 7f414b98b640 20 osd.0 pg_epoch: 59 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311687469s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown pruub 35.936801910s@ mbc={}] enter NotTrimming 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 pg_epoch: 59 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311687469s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] plpg_on_role_change 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 pg_epoch: 59 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311687469s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 pg_epoch: 59 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311687469s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] plpg_on_pool_change 2026-02-11T23:30:37.948+0000 7f414b98b640 20 osd.0 pg_epoch: 59 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311687469s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] hit_set_clear 2026-02-11T23:30:37.948+0000 7f414b98b640 20 osd.0 pg_epoch: 59 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311687469s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] agent_stop 2026-02-11T23:30:37.948+0000 7f414b98b640 20 osd.0 60 advance_pg new pool opts old pool opts 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 pg_epoch: 59 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311687469s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] handle_advance_map: 60 2026-02-11T23:30:37.948+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 5.0 59 -> 60 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 pg_epoch: 59 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311687469s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] PeeringState::advance_map handle_advance_map [3,4]/[3,4] -- 3/3 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 pg_epoch: 60 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311687469s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] state: Reset advmap 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 pg_epoch: 60 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311687469s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 pg_epoch: 60 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311687469s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] plpg_on_pool_change 2026-02-11T23:30:37.948+0000 7f414b98b640 20 osd.0 pg_epoch: 60 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311687469s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] hit_set_clear 2026-02-11T23:30:37.948+0000 7f414b98b640 20 osd.0 pg_epoch: 60 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311687469s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] agent_stop 2026-02-11T23:30:37.948+0000 7f414b98b640 20 osd.0 60 advance_pg new pool opts old pool opts 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 pg_epoch: 60 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311687469s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] handle_activate_map: epoch range: 59..60 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 pg_epoch: 60 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311687469s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] PeeringState::activate_map 2026-02-11T23:30:37.948+0000 7f414b98b640 5 osd.0 pg_epoch: 60 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311628342s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] exit Reset 0.000092 2 0.000095 2026-02-11T23:30:37.948+0000 7f414b98b640 5 osd.0 pg_epoch: 60 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311628342s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] enter Started 2026-02-11T23:30:37.948+0000 7f414b98b640 5 osd.0 pg_epoch: 60 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311628342s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] enter Start 2026-02-11T23:30:37.948+0000 7f414b98b640 1 osd.0 pg_epoch: 60 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311628342s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] state: transitioning to Stray 2026-02-11T23:30:37.948+0000 7f414b98b640 5 osd.0 pg_epoch: 60 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311628342s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] exit Start 0.000002 0 0.000000 2026-02-11T23:30:37.948+0000 7f414b98b640 5 osd.0 pg_epoch: 60 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311628342s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] enter Started/Stray 2026-02-11T23:30:37.948+0000 7f414b98b640 20 osd.0 pg_epoch: 60 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311628342s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 57 while current is 60 2026-02-11T23:30:37.948+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 pg_epoch: 60 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311628342s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 pg_epoch: 60 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311628342s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] do_peering_event: epoch_sent: 60 epoch_requested: 60 MQuery 5.0 from 3 query_epoch 60 query: query(info 0'0 epoch_sent 60) 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 pg_epoch: 60 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311600685s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] PeeringState::fulfill_info sending info 2026-02-11T23:30:37.948+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:37.948+0000 7f414b98b640 20 osd.0 60 get_con_osd_cluster to osd.3 from_epoch 60 2026-02-11T23:30:37.948+0000 7f414b98b640 20 osd.0 60 get_nextmap_reserved map_reservations: {60=1} 2026-02-11T23:30:37.948+0000 7f414b98b640 20 osd.0 60 release_map epoch: 60 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 60 maybe_share_map: con v2:127.0.0.1:6828/2699377081 map epoch 57 -> 60 (shared) 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 60 send_incremental_map epoch range: (58, 60] to v2:127.0.0.1:6828/2699377081 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 60 add_map_inc_bl 58 574 bytes 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 60 add_map_inc_bl 59 574 bytes 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 60 add_map_inc_bl 60 574 bytes 2026-02-11T23:30:37.948+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6828/2699377081,v1:127.0.0.1:6829/2699377081] -- osd_map(58..60 src has 1..60) -- 0x4a30e00 con 0x48ccc00 2026-02-11T23:30:37.948+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6828/2699377081,v1:127.0.0.1:6829/2699377081] -- pg_notify2(5.0 5.0 (query:60 sent:60 5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311628342s) ([56,58] all_participants={0, 3, 4} intervals=[([56,58] acting={0, 3, 4})])) e60/60) -- 0x5a8e300 con 0x48ccc00 2026-02-11T23:30:37.948+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6828/2699377081,v1:127.0.0.1:6829/2699377081] -- pg_notify2(5.0 5.0 (query:60 sent:60 5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311600685s) ([56,58] all_participants={0, 3, 4} intervals=[([56,58] acting={0, 3, 4})])) e60/60) -- 0x3738e00 con 0x48ccc00 2026-02-11T23:30:37.948+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:37.948+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 60 epoch_requested: 60 NullEvt) class_id 2immediate prio 255 cost 10 e60) queued 2026-02-11T23:30:37.948+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:37.948+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 60 epoch_requested: 60 NullEvt) class_id 2immediate prio 255 cost 10 e60) pg 0x5978000 2026-02-11T23:30:37.948+0000 7f414f993640 20 osd.0 op_wq(0) _process 5.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:37.948+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 60 epoch_requested: 60 NullEvt) class_id 2immediate prio 255 cost 10 e60) pg 0x4aa0000 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 pg_epoch: 58 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 59 2026-02-11T23:30:37.948+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 4.0 58 -> 59 2026-02-11T23:30:37.948+0000 7f414f993640 10 osd.0 pg_epoch: 60 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311600685s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] do_peering_event: epoch_sent: 60 epoch_requested: 60 NullEvt 2026-02-11T23:30:37.948+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 pg_epoch: 58 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:37.948+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 pg_epoch: 59 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:37.948+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 60 epoch_requested: 60 NullEvt) class_id 2immediate prio 255 cost 10 e60) queued 2026-02-11T23:30:37.948+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:37.948+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 60 epoch_requested: 60 NullEvt) class_id 2immediate prio 255 cost 10 e60) pg 0x5946000 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 pg_epoch: 59 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:37.948+0000 7f414b98b640 20 osd.0 60 advance_pg new pool opts old pool opts 2026-02-11T23:30:37.948+0000 7f414f993640 10 osd.0 pg_epoch: 58 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 59 2026-02-11T23:30:37.948+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 58 -> 59 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 pg_epoch: 59 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 60 2026-02-11T23:30:37.948+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 4.0 59 -> 60 2026-02-11T23:30:37.948+0000 7f414f993640 10 osd.0 pg_epoch: 58 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 pg_epoch: 59 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:37.948+0000 7f414f993640 10 osd.0 pg_epoch: 59 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 pg_epoch: 60 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:37.948+0000 7f414f993640 10 osd.0 pg_epoch: 59 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 pg_epoch: 60 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:37.948+0000 7f414b98b640 20 osd.0 60 advance_pg new pool opts old pool opts 2026-02-11T23:30:37.948+0000 7f414f993640 20 osd.0 60 advance_pg new pool opts old pool opts 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 pg_epoch: 60 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_activate_map: epoch range: 59..60 2026-02-11T23:30:37.948+0000 7f414f993640 10 osd.0 pg_epoch: 59 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 60 2026-02-11T23:30:37.948+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 59 -> 60 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 pg_epoch: 60 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:37.948+0000 7f414f993640 10 osd.0 pg_epoch: 59 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:37.948+0000 7f414b98b640 20 osd.0 pg_epoch: 60 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 53 while current is 60 2026-02-11T23:30:37.948+0000 7f414f993640 10 osd.0 pg_epoch: 60 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:37.948+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:37.948+0000 7f414f993640 10 osd.0 pg_epoch: 60 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:37.948+0000 7f414f993640 20 osd.0 60 advance_pg new pool opts old pool opts 2026-02-11T23:30:37.948+0000 7f414b98b640 10 osd.0 pg_epoch: 60 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 60 epoch_requested: 60 NullEvt 2026-02-11T23:30:37.948+0000 7f414f993640 10 osd.0 pg_epoch: 60 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_activate_map: epoch range: 59..60 2026-02-11T23:30:37.948+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:37.948+0000 7f414f993640 10 osd.0 pg_epoch: 60 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:37.948+0000 7f414f993640 20 osd.0 pg_epoch: 60 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 30 while current is 60 2026-02-11T23:30:37.948+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:37.948+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:37.948+0000 7f414f993640 10 osd.0 pg_epoch: 60 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 60 epoch_requested: 60 NullEvt 2026-02-11T23:30:37.948+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:37.948+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:37.948+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:37.948+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:38.150+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 36 ==== osd_map(61..62 src has 1..62) ==== 878+0+0 (crc 0 0 0) 0x4a30e00 con 0x4acbc00 2026-02-11T23:30:38.150+0000 7f415cf02640 20 osd.0 60 OSD::ms_dispatch: osd_map(61..62 src has 1..62) 2026-02-11T23:30:38.150+0000 7f415cf02640 20 osd.0 60 _dispatch 0x4a30e00 osd_map(61..62 src has 1..62) 2026-02-11T23:30:38.150+0000 7f415cf02640 3 osd.0 60 handle_osd_map epochs [61,62], i have 60, src has [1,62] 2026-02-11T23:30:38.150+0000 7f415cf02640 10 osd.0 60 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:38.150+0000 7f415cf02640 10 osd.0 60 handle_osd_map got inc map for epoch 61 2026-02-11T23:30:38.150+0000 7f415cf02640 20 osd.0 60 got_full_map 61, nothing requested 2026-02-11T23:30:38.150+0000 7f415cf02640 10 osd.0 60 handle_osd_map got inc map for epoch 62 2026-02-11T23:30:38.150+0000 7f415cf02640 20 osd.0 60 got_full_map 62, nothing requested 2026-02-11T23:30:38.150+0000 7f415cf02640 20 osd.0 60 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:38.150+0000 7f415cf02640 20 osd.0 60 handle_osd_map pg_num_history pg_num_history(e62 pg_nums {1={27=1},2={34=1},3={42=1},4={49=1},5={56=1}} deleted_pools ) 2026-02-11T23:30:38.150+0000 7f415cf02640 10 snap_mapper record_purged_snaps purged_snaps {61={},62={}} 2026-02-11T23:30:38.150+0000 7f415cf02640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:38.150+0000 7f415cf02640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e62 maps [1~62] lci=[5,62] tlb=1) 2026-02-11T23:30:38.151+0000 7f41591a6640 10 osd.0 60 _committed_osd_maps 61..62 2026-02-11T23:30:38.151+0000 7f41591a6640 10 osd.0 60 advance to epoch 61 (<= last 62 <= newest_map 62) 2026-02-11T23:30:38.151+0000 7f41591a6640 10 osd.0 61 advance to epoch 62 (<= last 62 <= newest_map 62) 2026-02-11T23:30:38.151+0000 7f41591a6640 20 osd.0 62 consume_map version 62 2026-02-11T23:30:38.151+0000 7f41591a6640 20 osd.0 60 await_reserved_maps epoch:62 2026-02-11T23:30:38.151+0000 7f41591a6640 20 osd.0 60 await_reserved_maps done 0 2026-02-11T23:30:38.151+0000 7f41591a6640 20 osd.0 62 consume_map 62 -- publish done 2026-02-11T23:30:38.151+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 3 slots 2026-02-11T23:30:38.151+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:5.0slot:0x3759080 2026-02-11T23:30:38.151+0000 7f41591a6640 20 osd.0 62 identify_splits_and_merges 5.0 e60 to e62 2026-02-11T23:30:38.151+0000 7f41591a6640 20 osd.0 62 identify_splits_and_merges 5.0 e60 to e62 pg_nums {56=1} 2026-02-11T23:30:38.151+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x37582c0 2026-02-11T23:30:38.151+0000 7f41591a6640 20 osd.0 62 identify_splits_and_merges 4.0 e60 to e62 2026-02-11T23:30:38.151+0000 7f41591a6640 20 osd.0 62 identify_splits_and_merges 4.0 e60 to e62 pg_nums {49=1} 2026-02-11T23:30:38.151+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3700dc0 2026-02-11T23:30:38.151+0000 7f41591a6640 20 osd.0 62 identify_splits_and_merges 1.0 e60 to e62 2026-02-11T23:30:38.151+0000 7f41591a6640 20 osd.0 62 identify_splits_and_merges 1.0 e60 to e62 pg_nums {27=1} 2026-02-11T23:30:38.151+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:38.151+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:38.151+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:38.151+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:38.151+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:38.151+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:38.151+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:38.151+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:38.151+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:38.151+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:38.151+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:38.151+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:38.151+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:38.151+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:38.151+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:38.151+0000 7f41591a6640 20 osd.0 62 prune_pg_created 2026-02-11T23:30:38.151+0000 7f41591a6640 10 osd.0:0.consume_map 62 (was 60) 2026-02-11T23:30:38.151+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 5.0 2026-02-11T23:30:38.151+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:38.151+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:38.151+0000 7f41591a6640 10 osd.0:1.consume_map 62 (was 60) 2026-02-11T23:30:38.151+0000 7f41591a6640 10 osd.0:2.consume_map 62 (was 60) 2026-02-11T23:30:38.151+0000 7f41591a6640 10 osd.0:3.consume_map 62 (was 60) 2026-02-11T23:30:38.151+0000 7f41591a6640 10 osd.0:4.consume_map 62 (was 60) 2026-02-11T23:30:38.151+0000 7f41591a6640 10 osd.0:5.consume_map 62 (was 60) 2026-02-11T23:30:38.151+0000 7f41591a6640 10 osd.0:6.consume_map 62 (was 60) 2026-02-11T23:30:38.151+0000 7f41591a6640 10 osd.0:7.consume_map 62 (was 60) 2026-02-11T23:30:38.151+0000 7f41591a6640 10 osd.0 62 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:38.151+0000 7f41591a6640 15 osd.0 62 enqueue_peering_evt 5.0 epoch_sent: 62 epoch_requested: 62 NullEvt 2026-02-11T23:30:38.151+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 62 epoch_requested: 62 NullEvt) class_id 2 prio 255 cost 10 e62) 2026-02-11T23:30:38.151+0000 7f41591a6640 15 osd.0 62 enqueue_peering_evt 4.0 epoch_sent: 62 epoch_requested: 62 NullEvt 2026-02-11T23:30:38.151+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 62 epoch_requested: 62 NullEvt) class_id 2 prio 255 cost 10 e62) 2026-02-11T23:30:38.151+0000 7f41591a6640 15 osd.0 62 enqueue_peering_evt 1.0 epoch_sent: 62 epoch_requested: 62 NullEvt 2026-02-11T23:30:38.151+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 62 epoch_requested: 62 NullEvt) class_id 2 prio 255 cost 10 e62) 2026-02-11T23:30:38.151+0000 7f414b98b640 20 osd.0 op_wq(0) _process 5.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:38.151+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 62 epoch_requested: 62 NullEvt) class_id 2immediate prio 255 cost 10 e62) queued 2026-02-11T23:30:38.151+0000 7f414b98b640 20 osd.0 op_wq(0) _process 5.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:38.151+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 62 epoch_requested: 62 NullEvt) class_id 2immediate prio 255 cost 10 e62) pg 0x4aa0000 2026-02-11T23:30:38.151+0000 7f41591a6640 10 osd.0 62 maybe_update_heartbeat_peers forcing update after 20.201375 seconds 2026-02-11T23:30:38.151+0000 7f41591a6640 10 osd.0 62 reset_heartbeat_peers 2026-02-11T23:30:38.151+0000 7f414b98b640 10 osd.0 pg_epoch: 60 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311600685s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] handle_advance_map: 61 2026-02-11T23:30:38.151+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:38.151+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 62 epoch_requested: 62 NullEvt) class_id 2immediate prio 255 cost 10 e62) queued 2026-02-11T23:30:38.151+0000 7f41591a6640 10 osd.0 62 maybe_update_heartbeat_peers updating 2026-02-11T23:30:38.151+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:38.151+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 62 epoch_requested: 62 NullEvt) class_id 2immediate prio 255 cost 10 e62) pg 0x5978000 2026-02-11T23:30:38.151+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 5.0 60 -> 61 2026-02-11T23:30:38.151+0000 7f414b98b640 10 osd.0 pg_epoch: 60 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311600685s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] PeeringState::advance_map handle_advance_map [3,4]/[3,4] -- 3/3 2026-02-11T23:30:38.151+0000 7f414f993640 10 osd.0 pg_epoch: 60 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 61 2026-02-11T23:30:38.151+0000 7f414f993640 20 osd.0:0.update_pg_epoch 4.0 60 -> 61 2026-02-11T23:30:38.151+0000 7f414b98b640 10 osd.0 pg_epoch: 61 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311600685s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] state: Started advmap 2026-02-11T23:30:38.151+0000 7f414f993640 10 osd.0 pg_epoch: 60 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:38.151+0000 7f414b98b640 10 osd.0 pg_epoch: 61 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311600685s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:38.151+0000 7f414b98b640 20 osd.0 62 advance_pg new pool opts old pool opts 2026-02-11T23:30:38.151+0000 7f414f993640 10 osd.0 pg_epoch: 61 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:38.151+0000 7f414f993640 10 osd.0 pg_epoch: 61 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:38.151+0000 7f414b98b640 10 osd.0 pg_epoch: 61 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311600685s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] handle_advance_map: 62 2026-02-11T23:30:38.151+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 5.0 61 -> 62 2026-02-11T23:30:38.151+0000 7f414f993640 20 osd.0 62 advance_pg new pool opts old pool opts 2026-02-11T23:30:38.151+0000 7f414b98b640 10 osd.0 pg_epoch: 61 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311600685s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] PeeringState::advance_map handle_advance_map [3,4]/[3,4] -- 3/3 2026-02-11T23:30:38.151+0000 7f414f993640 10 osd.0 pg_epoch: 61 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 62 2026-02-11T23:30:38.151+0000 7f414f993640 20 osd.0:0.update_pg_epoch 4.0 61 -> 62 2026-02-11T23:30:38.151+0000 7f414b98b640 10 osd.0 pg_epoch: 62 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311600685s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] state: Started advmap 2026-02-11T23:30:38.151+0000 7f414f993640 10 osd.0 pg_epoch: 61 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:38.151+0000 7f414b98b640 10 osd.0 pg_epoch: 62 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311600685s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:38.151+0000 7f414f993640 10 osd.0 pg_epoch: 62 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:38.151+0000 7f414b98b640 10 osd.0 pg_epoch: 62 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311600685s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] plpg_on_pool_change 2026-02-11T23:30:38.151+0000 7f414f993640 10 osd.0 pg_epoch: 62 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:38.151+0000 7f414f993640 20 osd.0 62 advance_pg new pool opts old pool opts 2026-02-11T23:30:38.151+0000 7f414b98b640 20 osd.0 pg_epoch: 62 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311600685s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] hit_set_clear 2026-02-11T23:30:38.151+0000 7f414f993640 10 osd.0 pg_epoch: 62 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_activate_map: epoch range: 61..62 2026-02-11T23:30:38.151+0000 7f414b98b640 20 osd.0 pg_epoch: 62 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311600685s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] agent_stop 2026-02-11T23:30:38.151+0000 7f414f993640 10 osd.0 pg_epoch: 62 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:38.151+0000 7f414b98b640 20 osd.0 62 advance_pg new pool opts old pool opts 2026-02-11T23:30:38.151+0000 7f414f993640 20 osd.0 pg_epoch: 62 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 53 while current is 62 2026-02-11T23:30:38.151+0000 7f41591a6640 10 osd.0 62 adding neighbor peer osd.1 2026-02-11T23:30:38.151+0000 7f414b98b640 10 osd.0 pg_epoch: 62 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311600685s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] handle_activate_map: epoch range: 61..62 2026-02-11T23:30:38.151+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:38.151+0000 7f41591a6640 10 osd.0 62 adding neighbor peer osd.4 2026-02-11T23:30:38.151+0000 7f414b98b640 10 osd.0 pg_epoch: 62 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.311600685s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] PeeringState::activate_map 2026-02-11T23:30:38.151+0000 7f414f993640 10 osd.0 pg_epoch: 62 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 62 epoch_requested: 62 NullEvt 2026-02-11T23:30:38.151+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:38.151+0000 7f41591a6640 10 osd.0 62 maybe_update_heartbeat_peers 4 peers, extras 1,2,3,4 2026-02-11T23:30:38.151+0000 7f41591a6640 7 osd.0 62 activate_map version 62 2026-02-11T23:30:38.151+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:38.151+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 62 epoch_requested: 62 NullEvt) class_id 2immediate prio 255 cost 10 e62) queued 2026-02-11T23:30:38.151+0000 7f414b98b640 20 osd.0 pg_epoch: 62 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.108879089s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 60 while current is 62 2026-02-11T23:30:38.151+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:38.151+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:38.151+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 62 epoch_requested: 62 NullEvt) class_id 2immediate prio 255 cost 10 e62) pg 0x5946000 2026-02-11T23:30:38.151+0000 7f414b98b640 10 osd.0 pg_epoch: 62 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.108879089s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:30:38.151+0000 7f414f993640 10 osd.0 pg_epoch: 60 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 61 2026-02-11T23:30:38.151+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 60 -> 61 2026-02-11T23:30:38.151+0000 7f414b98b640 10 osd.0 pg_epoch: 62 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.108879089s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] do_peering_event: epoch_sent: 62 epoch_requested: 62 NullEvt 2026-02-11T23:30:38.151+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:38.151+0000 7f414f993640 10 osd.0 pg_epoch: 60 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:38.151+0000 7f414b98b640 20 osd.0 62 get_con_osd_cluster to osd.3 from_epoch 62 2026-02-11T23:30:38.151+0000 7f414b98b640 20 osd.0 62 get_nextmap_reserved map_reservations: {62=1} 2026-02-11T23:30:38.151+0000 7f414f993640 10 osd.0 pg_epoch: 61 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:38.151+0000 7f414b98b640 20 osd.0 62 release_map epoch: 62 2026-02-11T23:30:38.151+0000 7f414f993640 10 osd.0 pg_epoch: 61 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:38.151+0000 7f414f993640 20 osd.0 62 advance_pg new pool opts old pool opts 2026-02-11T23:30:38.151+0000 7f414b98b640 10 osd.0 62 maybe_share_map: con v2:127.0.0.1:6828/2699377081 map epoch 60 -> 62 (shared) 2026-02-11T23:30:38.151+0000 7f414f993640 10 osd.0 pg_epoch: 61 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 62 2026-02-11T23:30:38.151+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 61 -> 62 2026-02-11T23:30:38.151+0000 7f414f993640 10 osd.0 pg_epoch: 61 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:38.151+0000 7f414b98b640 10 osd.0 62 send_incremental_map epoch range: (61, 62] to v2:127.0.0.1:6828/2699377081 2026-02-11T23:30:38.151+0000 7f414f993640 10 osd.0 pg_epoch: 62 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:38.151+0000 7f414f993640 10 osd.0 pg_epoch: 62 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:38.151+0000 7f414f993640 20 osd.0 62 advance_pg new pool opts old pool opts 2026-02-11T23:30:38.151+0000 7f414f993640 10 osd.0 pg_epoch: 62 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_activate_map: epoch range: 61..62 2026-02-11T23:30:38.151+0000 7f414f993640 10 osd.0 pg_epoch: 62 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:38.151+0000 7f414f993640 20 osd.0 pg_epoch: 62 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 30 while current is 62 2026-02-11T23:30:38.151+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:38.151+0000 7f414f993640 10 osd.0 pg_epoch: 62 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 62 epoch_requested: 62 NullEvt 2026-02-11T23:30:38.151+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:38.151+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:38.151+0000 7f4150194640 20 osd.0 62 agent_entry empty queue 2026-02-11T23:30:38.151+0000 7f414b98b640 10 osd.0 62 add_map_inc_bl 61 252 bytes 2026-02-11T23:30:38.151+0000 7f414b98b640 10 osd.0 62 add_map_inc_bl 62 574 bytes 2026-02-11T23:30:38.151+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6828/2699377081,v1:127.0.0.1:6829/2699377081] -- osd_map(61..62 src has 1..62) -- 0x4a30700 con 0x48ccc00 2026-02-11T23:30:38.151+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6828/2699377081,v1:127.0.0.1:6829/2699377081] -- pg_notify2(5.0 5.0 (query:62 sent:62 5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.108879089s) ([56,58] all_participants={0, 3, 4} intervals=[([56,58] acting={0, 3, 4})])) e62/62) -- 0x5a8dc00 con 0x48ccc00 2026-02-11T23:30:38.151+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:38.203+0000 7f415b6ff640 20 osd.0 62 reports for 0 queries 2026-02-11T23:30:38.203+0000 7f415b6ff640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] -- mgrreport(unknown.0 +0-0 packed 3294 daemon_metrics=2) -- 0x4f9e700 con 0x4aca800 2026-02-11T23:30:38.203+0000 7f415b6ff640 15 osd.0 62 collect_pg_stats 2026-02-11T23:30:38.203+0000 7f415b6ff640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] -- pg_stats(0 pgs seq 21474836485 v 0) -- 0x5ab2000 con 0x4aca800 2026-02-11T23:30:38.227+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.3 v2:127.0.0.1:6828/2699377081 20 ==== osd pg remove(epoch 62; pg5.0; ) ==== 32+0+0 (crc 0 0 0) 0x37009a0 con 0x48ccc00 2026-02-11T23:30:38.227+0000 7f416cbfe640 7 osd.0 62 handle_fast_pg_remove osd pg remove(epoch 62; pg5.0; ) from osd.3 2026-02-11T23:30:38.227+0000 7f416cbfe640 15 osd.0 62 enqueue_peering_evt 5.0 epoch_sent: 62 epoch_requested: 62 DeleteStart 2026-02-11T23:30:38.227+0000 7f416cbfe640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 62 epoch_requested: 62 DeleteStart) class_id 2 prio 255 cost 10 e62) 2026-02-11T23:30:38.227+0000 7f414f993640 20 osd.0 op_wq(0) _process 5.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:38.227+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 62 epoch_requested: 62 DeleteStart) class_id 2immediate prio 255 cost 10 e62) queued 2026-02-11T23:30:38.227+0000 7f414f993640 20 osd.0 op_wq(0) _process 5.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:38.227+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 62 epoch_requested: 62 DeleteStart) class_id 2immediate prio 255 cost 10 e62) pg 0x4aa0000 2026-02-11T23:30:38.227+0000 7f414f993640 10 osd.0 pg_epoch: 62 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.108879089s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] do_peering_event: epoch_sent: 62 epoch_requested: 62 DeleteStart 2026-02-11T23:30:38.227+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:38.227+0000 7f414f993640 5 osd.0 pg_epoch: 62 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.108879089s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] exit Started/Stray 0.278614 7 0.000025 2026-02-11T23:30:38.227+0000 7f414f993640 5 osd.0 pg_epoch: 62 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.108879089s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] enter Started/ToDelete 2026-02-11T23:30:38.227+0000 7f414f993640 5 osd.0 pg_epoch: 62 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.108879089s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] enter Started/ToDelete/WaitDeleteReserved 2026-02-11T23:30:38.227+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:38.227+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:38.227+0000 7f4152999640 15 osd.0 62 enqueue_peering_evt 5.0 epoch_sent: 62 epoch_requested: 62 DeleteReserved 2026-02-11T23:30:38.227+0000 7f4152999640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 62 epoch_requested: 62 DeleteReserved) class_id 2 prio 255 cost 10 e62) 2026-02-11T23:30:38.227+0000 7f414f993640 20 osd.0 op_wq(0) _process 5.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:38.227+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 62 epoch_requested: 62 DeleteReserved) class_id 2immediate prio 255 cost 10 e62) queued 2026-02-11T23:30:38.227+0000 7f414f993640 20 osd.0 op_wq(0) _process 5.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:38.227+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 62 epoch_requested: 62 DeleteReserved) class_id 2immediate prio 255 cost 10 e62) pg 0x4aa0000 2026-02-11T23:30:38.227+0000 7f414f993640 10 osd.0 pg_epoch: 62 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.108879089s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] do_peering_event: epoch_sent: 62 epoch_requested: 62 DeleteReserved 2026-02-11T23:30:38.227+0000 7f414f993640 5 osd.0 pg_epoch: 62 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.108879089s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] exit Started/ToDelete/WaitDeleteReserved 0.000032 1 0.000015 2026-02-11T23:30:38.227+0000 7f414f993640 5 osd.0 pg_epoch: 62 pg[5.0( empty local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.108879089s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] enter Started/ToDelete/Deleting 2026-02-11T23:30:38.227+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:38.227+0000 7f414f993640 10 osd.0 pg_epoch: 62 pg[5.0( empty lb MIN local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.108879089s) [3,4] r=-1 lpr=59 DELETING pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] on_removal 2026-02-11T23:30:38.227+0000 7f414f993640 10 osd.0 pg_epoch: 62 pg[5.0( empty lb MIN local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.108879089s) [3,4] r=-1 lpr=59 DELETING pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] on_shutdown 2026-02-11T23:30:38.227+0000 7f414f993640 10 osd.0 pg_epoch: 62 pg[5.0( empty lb MIN local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.108879089s) [3,4] r=-1 lpr=59 DELETING pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:38.227+0000 7f414f993640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:38.227+0000 7f414f993640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:38.227+0000 7f414f993640 10 osd.0 pg_epoch: 62 pg[5.0( empty lb MIN local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.108879089s) [3,4] r=-1 lpr=59 DELETING pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] cancel_copy_ops 2026-02-11T23:30:38.227+0000 7f414f993640 10 osd.0 pg_epoch: 62 pg[5.0( empty lb MIN local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.108879089s) [3,4] r=-1 lpr=59 DELETING pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] cancel_flush_ops 2026-02-11T23:30:38.227+0000 7f414f993640 10 osd.0 pg_epoch: 62 pg[5.0( empty lb MIN local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.108879089s) [3,4] r=-1 lpr=59 DELETING pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] cancel_proxy_ops 2026-02-11T23:30:38.227+0000 7f414f993640 10 osd.0 pg_epoch: 62 pg[5.0( empty lb MIN local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.108879089s) [3,4] r=-1 lpr=59 DELETING pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] cancel_manifest_ops 2026-02-11T23:30:38.227+0000 7f414f993640 10 osd.0 pg_epoch: 62 pg[5.0( empty lb MIN local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.108879089s) [3,4] r=-1 lpr=59 DELETING pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] cancel_cls_gather_ops 2026-02-11T23:30:38.227+0000 7f414f993640 10 osd.0 pg_epoch: 62 pg[5.0( empty lb MIN local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.108879089s) [3,4] r=-1 lpr=59 DELETING pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] clear_backoffs 2026-02-11T23:30:38.227+0000 7f414f993640 20 osd.0 pg_epoch: 62 pg[5.0( empty lb MIN local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.108879089s) [3,4] r=-1 lpr=59 DELETING pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] exit NotTrimming 2026-02-11T23:30:38.227+0000 7f414f993640 20 osd.0 pg_epoch: 62 pg[5.0( empty lb MIN local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.108879089s) [3,4] r=-1 lpr=59 DELETING pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] enter NotTrimming 2026-02-11T23:30:38.227+0000 7f414f993640 10 osd.0 pg_epoch: 62 pg[5.0( empty lb MIN local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.108879089s) [3,4] r=-1 lpr=59 DELETING pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] on_change 2026-02-11T23:30:38.227+0000 7f414f993640 10 osd.0 pg_epoch: 62 pg[5.0( empty lb MIN local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.108879089s) [3,4] r=-1 lpr=59 DELETING pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] clear_async_reads 2026-02-11T23:30:38.227+0000 7f414f993640 20 osd.0 pg_epoch: 62 pg[5.0( empty lb MIN local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.108879089s) [3,4] r=-1 lpr=59 DELETING pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] clear_primary_state 2026-02-11T23:30:38.227+0000 7f414f993640 10 osd.0 pg_epoch: 62 pg[5.0( empty lb MIN local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.108879089s) [3,4] r=-1 lpr=59 DELETING pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] release_backoffs [5:00000000::::0,MAX) 2026-02-11T23:30:38.227+0000 7f414f993640 10 osd.0 pg_epoch: 62 pg[5.0( empty lb MIN local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.108879089s) [3,4] r=-1 lpr=59 DELETING pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:38.227+0000 7f414f993640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:38.227+0000 7f414f993640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:38.227+0000 7f414f993640 20 osd.0 pg_epoch: 62 pg[5.0( empty lb MIN local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.108879089s) [3,4] r=-1 lpr=59 DELETING pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] agent_stop 2026-02-11T23:30:38.227+0000 7f414f993640 10 osd.0 pg_epoch: 62 pg[5.0( empty lb MIN local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.108879089s) [3,4] r=-1 lpr=59 DELETING pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] cancel_recovery 2026-02-11T23:30:38.227+0000 7f414f993640 10 osd.0 pg_epoch: 62 pg[5.0( empty lb MIN local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.108879089s) [3,4] r=-1 lpr=59 DELETING pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] clear_recovery_state 2026-02-11T23:30:38.227+0000 7f414f993640 15 osd.0 pg_epoch: 62 pg[5.0( empty lb MIN local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.108879089s) [3,4] r=-1 lpr=59 DELETING pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] _clear_recovery_state 2026-02-11T23:30:38.227+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:38.227+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:38.227+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:38.227+0000 7f414f993640 10 osd.0 62 queue_for_pg_delete on 5.0 e 62 2026-02-11T23:30:38.227+0000 7f414f993640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(5.0 PGDelete(pgid=5.0 epoch_queued=62) class_id 1 prio 5 cost 1048576 e62) 2026-02-11T23:30:38.227+0000 7f414f993640 20 osd.0 op_wq(0) _process 5.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:38.227+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGDelete(5.0 e62) class_id 1background_best_effort prio 5 qos_cost 1048576 cost 1048576 e62) queued 2026-02-11T23:30:38.227+0000 7f414f993640 20 osd.0 op_wq(0) _process 5.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:38.227+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGDelete(5.0 e62) class_id 1background_best_effort prio 5 qos_cost 1048576 cost 1048576 e62) pg 0x4aa0000 2026-02-11T23:30:38.227+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:38.227+0000 7f414f993640 10 osd.0 pg_epoch: 62 pg[5.0( empty lb MIN local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.108879089s) [3,4] r=-1 lpr=59 DELETING pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] do_peering_event: epoch_sent: 62 epoch_requested: 62 DeleteSome 2026-02-11T23:30:38.227+0000 7f414f993640 10 osd.0 pg_epoch: 62 pg[5.0( empty lb MIN local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.108879089s) [3,4] r=-1 lpr=59 DELETING pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] do_delete_work 2026-02-11T23:30:38.227+0000 7f414f993640 20 osd.0 pg_epoch: 62 pg[5.0( empty lb MIN local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.108879089s) [3,4] r=-1 lpr=59 DELETING pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] do_delete_work [#5:00000000::::head#] 2026-02-11T23:30:38.227+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:38.227+0000 7f414f993640 20 osd.0 62 try_finish_pg_delete 5.0 0x4aa0000 2026-02-11T23:30:38.227+0000 7f414f993640 10 osd.0:0._detach_pg 5.0 0x4aa0000 2026-02-11T23:30:38.227+0000 7f414f993640 5 osd.0 pg_epoch: 62 pg[5.0( empty lb MIN local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.108879089s) [3,4] r=-1 lpr=59 DELETING pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] exit Started/ToDelete/Deleting 0.000260 1 0.000031 2026-02-11T23:30:38.227+0000 7f414f993640 5 osd.0 pg_epoch: 62 pg[5.0( empty lb MIN local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.108879089s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] exit Started/ToDelete 0.000301 0 0.000000 2026-02-11T23:30:38.227+0000 7f414f993640 5 osd.0 pg_epoch: 62 pg[5.0( empty lb MIN local-lis/les=56/57 n=0 ec=56/56 lis/c=56/56 les/c/f=57/57/0 sis=59 pruub=15.108879089s) [3,4] r=-1 lpr=59 pi=[56,59)/1 crt=0'0 unknown NOTIFY pruub 35.936801910s@ mbc={}] exit Started 0.278927 0 0.000000 2026-02-11T23:30:38.227+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:38.227+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:38.227+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:38.227+0000 7f414f993640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:38.227+0000 7f414f993640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:38.227+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:38.229+0000 7f4162bea640 10 osd.0 62 tick_without_osd_lock 2026-02-11T23:30:38.229+0000 7f4162bea640 0 osd.0 62 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:38.229+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:38.229+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:38.229+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:38.229+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.468 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:38.229+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:38.231, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:38.230+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:38.230+0000 7f4162bea640 10 osd.0 62 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:38.230+0000 7f4162bea640 20 osd.0 62 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:38.230+0000 7f4162bea640 10 osd.0 62 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:38.230+0000 7f4162bea640 20 osd.0 62 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:38.272+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 17 ==== pg_lease(1.0 pg_lease(ru 26.937025070s ub 34.937133789s int 16.000000000s) e63/63) ==== 58+0+0 (unknown 0 0 0) 0x4c9b980 con 0x4acb000 2026-02-11T23:30:38.272+0000 7f416abfa640 15 osd.0 62 enqueue_peering_evt 1.0 epoch_sent: 63 epoch_requested: 63 MLease epoch 63 from osd.1 pg_lease(ru 26.937025070s ub 34.937133789s int 16.000000000s) 2026-02-11T23:30:38.272+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 63 epoch_requested: 63 MLease epoch 63 from osd.1 pg_lease(ru 26.937025070s ub 34.937133789s int 16.000000000s)) class_id 2 prio 255 cost 10 e63) 2026-02-11T23:30:38.272+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:38.272+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 63 epoch_requested: 63 MLease epoch 63 from osd.1 pg_lease(ru 26.937025070s ub 34.937133789s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e63) queued 2026-02-11T23:30:38.272+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:38.272+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 63 epoch_requested: 63 MLease epoch 63 from osd.1 pg_lease(ru 26.937025070s ub 34.937133789s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e63) pg 0x5946000 2026-02-11T23:30:38.272+0000 7f414b98b640 20 osd.0 op_wq _add_slot_waiter 1.0 peering, item epoch is 63, will wait on OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 63 epoch_requested: 63 MLease epoch 63 from osd.1 pg_lease(ru 26.937025070s ub 34.937133789s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e63) 2026-02-11T23:30:38.272+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:38.272+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:38.326+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:30:38.326+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:30:38.326+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:08.327720+0000) 2026-02-11T23:30:38.393+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.2 127.0.0.1:0/2365263801 3 ==== osd_ping(ping e61 up_from 14 ping_stamp 2026-02-11T23:30:38.394434+0000/16.503030777s send_stamp 16.503030777s delta_ub -4.566684246s) ==== 2033+0+0 (crc 0 0 0) 0x4a89200 con 0x4641000 2026-02-11T23:30:38.393+0000 7f416cbfe640 20 osd.0 62 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566867352s,-4.566684246s]) 2026-02-11T23:30:38.393+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e62 up_from 5 ping_stamp 2026-02-11T23:30:38.394434+0000/16.503030777s send_stamp 21.069898605s delta_ub 4.566867352s) -- 0x4c9ad80 con 0x4641000 2026-02-11T23:30:38.393+0000 7f416cbfe640 20 osd.0 62 get_con_osd_cluster to osd.2 from_epoch 62 2026-02-11T23:30:38.393+0000 7f416cbfe640 20 osd.0 62 get_nextmap_reserved map_reservations: {62=1} 2026-02-11T23:30:38.393+0000 7f416cbfe640 20 osd.0 62 release_map epoch: 62 2026-02-11T23:30:38.393+0000 7f416cbfe640 10 osd.0 62 maybe_share_map: con v2:127.0.0.1:6820/2365263801 updating session's projected_epoch from 54 to ping map epoch of 61 2026-02-11T23:30:38.393+0000 7f416cbfe640 10 osd.0 62 maybe_share_map: con v2:127.0.0.1:6820/2365263801 map epoch 61 -> 62 (shared) 2026-02-11T23:30:38.393+0000 7f416cbfe640 10 osd.0 62 send_incremental_map epoch range: (62, 62] to v2:127.0.0.1:6820/2365263801 2026-02-11T23:30:38.393+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6820/2365263801,v1:127.0.0.1:6821/2365263801] -- osd_map(62..62 src has 1..62) -- 0x4a30700 con 0x507b800 2026-02-11T23:30:38.393+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.2 127.0.0.1:0/2365263801 3 ==== osd_ping(ping e61 up_from 14 ping_stamp 2026-02-11T23:30:38.394434+0000/16.503030777s send_stamp 16.503030777s delta_ub -4.566684246s) ==== 2033+0+0 (crc 0 0 0) 0x4c9aa80 con 0x48cc000 2026-02-11T23:30:38.393+0000 7f416c3fd640 20 osd.0 62 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566909790s,-4.566684246s]) 2026-02-11T23:30:38.393+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e62 up_from 5 ping_stamp 2026-02-11T23:30:38.394434+0000/16.503030777s send_stamp 21.069940567s delta_ub 4.566909790s) -- 0x4c9b800 con 0x48cc000 2026-02-11T23:30:38.393+0000 7f416c3fd640 20 osd.0 62 get_con_osd_cluster to osd.2 from_epoch 62 2026-02-11T23:30:38.393+0000 7f416c3fd640 20 osd.0 62 get_nextmap_reserved map_reservations: {62=1} 2026-02-11T23:30:38.393+0000 7f416c3fd640 20 osd.0 62 release_map epoch: 62 2026-02-11T23:30:38.393+0000 7f416c3fd640 10 osd.0 62 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 62 is not newer than session's projected_epoch of 62 2026-02-11T23:30:38.398+0000 7f41633eb640 10 osd.0 62 tick 2026-02-11T23:30:38.398+0000 7f41633eb640 20 osd.0 62 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:30:39.261+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 14 ==== pg_query2(6.0 6.0 query(info 0'0 epoch_sent 64) e64/64) ==== 158+0+0 (unknown 0 0 0) 0x4fca960 con 0x507ac00 2026-02-11T23:30:39.261+0000 7f416abfa640 15 osd.0 62 enqueue_peering_evt 6.0 epoch_sent: 64 epoch_requested: 64 MQuery 6.0 from 4 query_epoch 64 query: query(info 0'0 epoch_sent 64) 2026-02-11T23:30:39.261+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 64 epoch_requested: 64 MQuery 6.0 from 4 query_epoch 64 query: query(info 0'0 epoch_sent 64)) class_id 2 prio 255 cost 10 e64) 2026-02-11T23:30:39.261+0000 7f414b98b640 20 osd.0 op_wq(0) _process 6.0 (new) to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:39.261+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 64 epoch_requested: 64 MQuery 6.0 from 4 query_epoch 64 query: query(info 0'0 epoch_sent 64)) class_id 2immediate prio 255 cost 10 e64) queued 2026-02-11T23:30:39.261+0000 7f414b98b640 20 osd.0 op_wq(0) _process 6.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:39.261+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 64 epoch_requested: 64 MQuery 6.0 from 4 query_epoch 64 query: query(info 0'0 epoch_sent 64)) class_id 2immediate prio 255 cost 10 e64) pg 0 2026-02-11T23:30:39.261+0000 7f414b98b640 20 osd.0 op_wq(0) _process 6.0 map 64 > 62 2026-02-11T23:30:39.261+0000 7f414b98b640 20 osd.0 op_wq _add_slot_waiter 6.0 peering, item epoch is 64, will wait on OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 64 epoch_requested: 64 MQuery 6.0 from 4 query_epoch 64 query: query(info 0'0 epoch_sent 64)) class_id 2immediate prio 255 cost 10 e64) 2026-02-11T23:30:39.261+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:39.261+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:39.261+0000 7f415aefe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 15 ==== osd_map(57..64 src has 1..64) ==== 4065+0+0 (unknown 0 0 0) 0x5954a80 con 0x507ac00 2026-02-11T23:30:39.261+0000 7f415aefe640 20 osd.0 62 OSD::ms_dispatch: osd_map(57..64 src has 1..64) 2026-02-11T23:30:39.261+0000 7f415aefe640 20 osd.0 62 _dispatch 0x5954a80 osd_map(57..64 src has 1..64) 2026-02-11T23:30:39.261+0000 7f415aefe640 3 osd.0 62 handle_osd_map epochs [57,64], i have 62, src has [1,64] 2026-02-11T23:30:39.261+0000 7f415aefe640 10 osd.0 62 handle_osd_map: received 0 full maps and 8 incremental maps 2026-02-11T23:30:39.261+0000 7f415aefe640 10 osd.0 62 handle_osd_map got inc map for epoch 63 2026-02-11T23:30:39.261+0000 7f415aefe640 20 osd.0 62 got_full_map 63, nothing requested 2026-02-11T23:30:39.261+0000 7f415aefe640 10 osd.0 62 handle_osd_map got inc map for epoch 64 2026-02-11T23:30:39.261+0000 7f415aefe640 20 osd.0 62 got_full_map 64, nothing requested 2026-02-11T23:30:39.261+0000 7f415aefe640 10 osd.0 62 _track_pools_and_pg_num_changes recording new pool 6 pg_num 1 2026-02-11T23:30:39.261+0000 7f415aefe640 20 osd.0 62 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:39.261+0000 7f415aefe640 20 osd.0 62 handle_osd_map pg_num_history pg_num_history(e64 pg_nums {1={27=1},2={34=1},3={42=1},4={49=1},5={56=1},6={64=1}} deleted_pools ) 2026-02-11T23:30:39.261+0000 7f415aefe640 10 snap_mapper record_purged_snaps purged_snaps {63={},64={}} 2026-02-11T23:30:39.261+0000 7f415aefe640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:39.261+0000 7f415aefe640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e64 maps [1~64] lci=[5,64] tlb=1) 2026-02-11T23:30:39.261+0000 7f41591a6640 10 osd.0 62 _committed_osd_maps 63..64 2026-02-11T23:30:39.261+0000 7f41591a6640 10 osd.0 62 advance to epoch 63 (<= last 64 <= newest_map 64) 2026-02-11T23:30:39.261+0000 7f41591a6640 10 osd.0 63 advance to epoch 64 (<= last 64 <= newest_map 64) 2026-02-11T23:30:39.261+0000 7f41591a6640 20 osd.0 64 consume_map version 64 2026-02-11T23:30:39.261+0000 7f41591a6640 20 osd.0 62 await_reserved_maps epoch:64 2026-02-11T23:30:39.261+0000 7f41591a6640 20 osd.0 62 await_reserved_maps done 0 2026-02-11T23:30:39.261+0000 7f41591a6640 20 osd.0 64 consume_map 64 -- publish done 2026-02-11T23:30:39.261+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 4 slots 2026-02-11T23:30:39.261+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:6.0slot:0x37591e0 2026-02-11T23:30:39.261+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot 6.0 has no pg and waiting_for_split 2026-02-11T23:30:39.261+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:5.0slot:0x3759080 2026-02-11T23:30:39.261+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot 5.0 has no pg and waiting_for_split 2026-02-11T23:30:39.261+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x37582c0 2026-02-11T23:30:39.261+0000 7f41591a6640 20 osd.0 64 identify_splits_and_merges 4.0 e62 to e64 2026-02-11T23:30:39.261+0000 7f41591a6640 20 osd.0 64 identify_splits_and_merges 4.0 e62 to e64 pg_nums {49=1} 2026-02-11T23:30:39.261+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3700dc0 2026-02-11T23:30:39.261+0000 7f41591a6640 20 osd.0 64 identify_splits_and_merges 1.0 e62 to e64 2026-02-11T23:30:39.261+0000 7f41591a6640 20 osd.0 64 identify_splits_and_merges 1.0 e62 to e64 pg_nums {27=1} 2026-02-11T23:30:39.261+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:39.261+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:39.261+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:39.261+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:39.261+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:39.261+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:39.261+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:39.261+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:39.261+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:39.261+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:39.261+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:39.261+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:39.261+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:39.261+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:39.261+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:39.261+0000 7f41591a6640 20 osd.0 64 prune_pg_created 2026-02-11T23:30:39.261+0000 7f41591a6640 10 osd.0:0.consume_map 64 (was 62) 2026-02-11T23:30:39.261+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 6.0 2026-02-11T23:30:39.261+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 6.0 pending_peering first epoch 64 <= 64, requeueing 2026-02-11T23:30:39.261+0000 7f41591a6640 20 osd.0:0._wake_pg_slot _wake_pg_slot 6.0 to_process <> waiting <> waiting_peering {64=} 2026-02-11T23:30:39.261+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 5.0 2026-02-11T23:30:39.261+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 5.0 empty, pruning 2026-02-11T23:30:39.261+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:39.261+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:39.261+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 pending_peering first epoch 63 <= 64, requeueing 2026-02-11T23:30:39.261+0000 7f41591a6640 20 osd.0:0._wake_pg_slot _wake_pg_slot 1.0 to_process <> waiting <> waiting_peering {63=} 2026-02-11T23:30:39.261+0000 7f41591a6640 10 osd.0:1.consume_map 64 (was 62) 2026-02-11T23:30:39.261+0000 7f41591a6640 10 osd.0:2.consume_map 64 (was 62) 2026-02-11T23:30:39.261+0000 7f41591a6640 10 osd.0:3.consume_map 64 (was 62) 2026-02-11T23:30:39.261+0000 7f41591a6640 10 osd.0:4.consume_map 64 (was 62) 2026-02-11T23:30:39.261+0000 7f41591a6640 10 osd.0:5.consume_map 64 (was 62) 2026-02-11T23:30:39.261+0000 7f41591a6640 10 osd.0:6.consume_map 64 (was 62) 2026-02-11T23:30:39.261+0000 7f41591a6640 10 osd.0:7.consume_map 64 (was 62) 2026-02-11T23:30:39.261+0000 7f41591a6640 10 osd.0 64 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:39.261+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:39.261+0000 7f41591a6640 15 osd.0 64 enqueue_peering_evt 4.0 epoch_sent: 64 epoch_requested: 64 NullEvt 2026-02-11T23:30:39.261+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 63 epoch_requested: 63 MLease epoch 63 from osd.1 pg_lease(ru 26.937025070s ub 34.937133789s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e63) queued 2026-02-11T23:30:39.261+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 64 epoch_requested: 64 NullEvt) class_id 2 prio 255 cost 10 e64) 2026-02-11T23:30:39.261+0000 7f41591a6640 15 osd.0 64 enqueue_peering_evt 1.0 epoch_sent: 64 epoch_requested: 64 NullEvt 2026-02-11T23:30:39.261+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:39.261+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 64 epoch_requested: 64 NullEvt) class_id 2 prio 255 cost 10 e64) 2026-02-11T23:30:39.261+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 63 epoch_requested: 63 MLease epoch 63 from osd.1 pg_lease(ru 26.937025070s ub 34.937133789s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e63) pg 0x5946000 2026-02-11T23:30:39.261+0000 7f414b98b640 20 osd.0 op_wq(0) _process 6.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:39.261+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 64 epoch_requested: 64 MQuery 6.0 from 4 query_epoch 64 query: query(info 0'0 epoch_sent 64)) class_id 2immediate prio 255 cost 10 e64) queued 2026-02-11T23:30:39.261+0000 7f414b98b640 20 osd.0 op_wq(0) _process 6.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:39.261+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 64 epoch_requested: 64 MQuery 6.0 from 4 query_epoch 64 query: query(info 0'0 epoch_sent 64)) class_id 2immediate prio 255 cost 10 e64) pg 0 2026-02-11T23:30:39.261+0000 7f414f993640 10 osd.0 pg_epoch: 62 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 63 2026-02-11T23:30:39.261+0000 7f414b98b640 10 osd.0 64 handle_pg_query_nopg 6.0 2026-02-11T23:30:39.261+0000 7f414b98b640 10 osd.0 64 pg 6.0 dne 2026-02-11T23:30:39.261+0000 7f414b98b640 20 osd.0 64 get_con_osd_cluster to osd.4 from_epoch 64 2026-02-11T23:30:39.261+0000 7f414b98b640 20 osd.0 64 get_nextmap_reserved map_reservations: {64=1} 2026-02-11T23:30:39.261+0000 7f414b98b640 20 osd.0 64 release_map epoch: 64 2026-02-11T23:30:39.261+0000 7f414b98b640 10 osd.0 64 maybe_share_map: con v2:127.0.0.1:6836/3331151400 map epoch 56 -> 64 (shared) 2026-02-11T23:30:39.261+0000 7f414b98b640 10 osd.0 64 send_incremental_map epoch range: (57, 64] to v2:127.0.0.1:6836/3331151400 2026-02-11T23:30:39.261+0000 7f414b98b640 10 osd.0 64 add_map_inc_bl 63 574 bytes 2026-02-11T23:30:39.261+0000 7f414b98b640 10 osd.0 64 add_map_inc_bl 64 591 bytes 2026-02-11T23:30:39.261+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- osd_map(57..64 src has 1..64) -- 0x4a31c00 con 0x507ac00 2026-02-11T23:30:39.261+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_notify2(6.0 6.0 (query:64 sent:64 6.0( DNE empty local-lis/les=0/0 n=0 ec=0/0 lis/c=0/0 les/c/f=0/0/0 sis=0) ([0,0] all_participants={} intervals=[])) e64/64) -- 0x5a8c000 con 0x507ac00 2026-02-11T23:30:39.261+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:39.261+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 64 epoch_requested: 64 NullEvt) class_id 2immediate prio 255 cost 10 e64) queued 2026-02-11T23:30:39.261+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:39.261+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 64 epoch_requested: 64 NullEvt) class_id 2immediate prio 255 cost 10 e64) pg 0x5978000 2026-02-11T23:30:39.261+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 62 -> 63 2026-02-11T23:30:39.261+0000 7f414f993640 10 osd.0 pg_epoch: 62 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:39.261+0000 7f41591a6640 7 osd.0 64 activate_map version 64 2026-02-11T23:30:39.261+0000 7f414b98b640 10 osd.0 pg_epoch: 62 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 63 2026-02-11T23:30:39.261+0000 7f414f993640 10 osd.0 pg_epoch: 63 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:39.261+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 4.0 62 -> 63 2026-02-11T23:30:39.261+0000 7f414f993640 10 osd.0 pg_epoch: 63 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:39.261+0000 7f414b98b640 10 osd.0 pg_epoch: 62 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:39.261+0000 7f414f993640 20 osd.0 64 advance_pg new pool opts old pool opts 2026-02-11T23:30:39.261+0000 7f414f993640 10 osd.0 pg_epoch: 63 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 64 2026-02-11T23:30:39.261+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 63 -> 64 2026-02-11T23:30:39.261+0000 7f414b98b640 10 osd.0 pg_epoch: 63 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:39.261+0000 7f414f993640 10 osd.0 pg_epoch: 63 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:39.261+0000 7f414b98b640 10 osd.0 pg_epoch: 63 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:39.261+0000 7f414f993640 10 osd.0 pg_epoch: 64 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:39.261+0000 7f414f993640 10 osd.0 pg_epoch: 64 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:39.261+0000 7f414b98b640 20 osd.0 64 advance_pg new pool opts old pool opts 2026-02-11T23:30:39.261+0000 7f414f993640 20 osd.0 64 advance_pg new pool opts old pool opts 2026-02-11T23:30:39.261+0000 7f414f993640 10 osd.0 pg_epoch: 64 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_activate_map: epoch range: 63..64 2026-02-11T23:30:39.261+0000 7f414f993640 10 osd.0 pg_epoch: 64 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:39.261+0000 7f414b98b640 10 osd.0 pg_epoch: 63 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 64 2026-02-11T23:30:39.261+0000 7f4150194640 20 osd.0 64 agent_entry empty queue 2026-02-11T23:30:39.261+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 4.0 63 -> 64 2026-02-11T23:30:39.261+0000 7f414f993640 20 osd.0 pg_epoch: 64 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 30 while current is 64 2026-02-11T23:30:39.261+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:39.261+0000 7f414b98b640 10 osd.0 pg_epoch: 63 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:39.261+0000 7f414f993640 10 osd.0 pg_epoch: 64 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 63 epoch_requested: 63 MLease epoch 63 from osd.1 pg_lease(ru 26.937025070s ub 34.937133789s int 16.000000000s) 2026-02-11T23:30:39.261+0000 7f414b98b640 10 osd.0 pg_epoch: 64 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:39.261+0000 7f414b98b640 10 osd.0 pg_epoch: 64 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:39.261+0000 7f414b98b640 20 osd.0 64 advance_pg new pool opts old pool opts 2026-02-11T23:30:39.261+0000 7f414f993640 10 osd.0 pg_epoch: 64 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::proc_lease pg_lease(ru 26.937025070s ub 34.937133789s int 16.000000000s) 2026-02-11T23:30:39.261+0000 7f414b98b640 10 osd.0 pg_epoch: 64 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_activate_map: epoch range: 63..64 2026-02-11T23:30:39.261+0000 7f414f993640 20 osd.0 pg_epoch: 64 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::proc_lease peer_clock_delta_ub -2.011401892s -> ru 28.948427200s 2026-02-11T23:30:39.261+0000 7f414b98b640 10 osd.0 pg_epoch: 64 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:39.261+0000 7f414f993640 20 osd.0 pg_epoch: 64 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::proc_lease readable_until now 28.948427200s 2026-02-11T23:30:39.261+0000 7f414b98b640 20 osd.0 pg_epoch: 64 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 53 while current is 64 2026-02-11T23:30:39.261+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:39.261+0000 7f414f993640 20 osd.0 pg_epoch: 64 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::proc_lease peer_clock_delta_lb -2.011763096s -> ruub 36.948898315s 2026-02-11T23:30:39.261+0000 7f414b98b640 10 osd.0 pg_epoch: 64 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 64 epoch_requested: 64 NullEvt 2026-02-11T23:30:39.261+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:39.261+0000 7f414f993640 20 osd.0 pg_epoch: 64 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::proc_lease readable_until_ub now 36.948898315s 2026-02-11T23:30:39.261+0000 7f414f993640 20 osd.0 64 get_con_osd_cluster to osd.1 from_epoch 64 2026-02-11T23:30:39.261+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:39.261+0000 7f414f993640 20 osd.0 64 get_nextmap_reserved map_reservations: {64=1} 2026-02-11T23:30:39.261+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 64 epoch_requested: 64 NullEvt) class_id 2immediate prio 255 cost 10 e64) queued 2026-02-11T23:30:39.261+0000 7f414f993640 20 osd.0 64 release_map epoch: 64 2026-02-11T23:30:39.261+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_lease_ack(1.0 pg_lease_ack(ruub 34.937133789s) e64/64) -- 0x3700000 con 0x4acb000 2026-02-11T23:30:39.261+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:39.261+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:39.261+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:39.261+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 64 epoch_requested: 64 NullEvt) class_id 2immediate prio 255 cost 10 e64) pg 0x5946000 2026-02-11T23:30:39.261+0000 7f414b98b640 10 osd.0 pg_epoch: 64 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 64 epoch_requested: 64 NullEvt 2026-02-11T23:30:39.261+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:39.261+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:39.271+0000 7f4162bea640 10 osd.0 64 tick_without_osd_lock 2026-02-11T23:30:39.271+0000 7f4162bea640 0 osd.0 64 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:39.271+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:39.271+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:39.271+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:39.271+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.468 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:39.271+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:39.271, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:39.271+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:39.271+0000 7f4162bea640 10 osd.0 64 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:39.271+0000 7f4162bea640 20 osd.0 64 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:39.271+0000 7f4162bea640 10 osd.0 64 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:39.271+0000 7f4162bea640 20 osd.0 64 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:39.312+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 37 ==== osd_map(64..65 src has 1..65) ==== 895+0+0 (crc 0 0 0) 0x4a30700 con 0x4acbc00 2026-02-11T23:30:39.312+0000 7f415cf02640 20 osd.0 64 OSD::ms_dispatch: osd_map(64..65 src has 1..65) 2026-02-11T23:30:39.312+0000 7f415cf02640 20 osd.0 64 _dispatch 0x4a30700 osd_map(64..65 src has 1..65) 2026-02-11T23:30:39.312+0000 7f415cf02640 3 osd.0 64 handle_osd_map epochs [64,65], i have 64, src has [1,65] 2026-02-11T23:30:39.312+0000 7f415cf02640 10 osd.0 64 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:39.312+0000 7f415cf02640 10 osd.0 64 handle_osd_map got inc map for epoch 65 2026-02-11T23:30:39.312+0000 7f415cf02640 20 osd.0 64 got_full_map 65, nothing requested 2026-02-11T23:30:39.312+0000 7f415cf02640 20 osd.0 64 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:39.312+0000 7f415cf02640 20 osd.0 64 handle_osd_map pg_num_history pg_num_history(e65 pg_nums {1={27=1},2={34=1},3={42=1},4={49=1},5={56=1},6={64=1}} deleted_pools ) 2026-02-11T23:30:39.312+0000 7f415cf02640 10 snap_mapper record_purged_snaps purged_snaps {65={}} 2026-02-11T23:30:39.312+0000 7f415cf02640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:39.312+0000 7f415cf02640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e65 maps [1~65] lci=[5,65] tlb=1) 2026-02-11T23:30:39.312+0000 7f41591a6640 10 osd.0 64 _committed_osd_maps 65..65 2026-02-11T23:30:39.312+0000 7f41591a6640 10 osd.0 64 advance to epoch 65 (<= last 65 <= newest_map 65) 2026-02-11T23:30:39.312+0000 7f41591a6640 20 osd.0 65 consume_map version 65 2026-02-11T23:30:39.312+0000 7f41591a6640 20 osd.0 64 await_reserved_maps epoch:65 2026-02-11T23:30:39.312+0000 7f41591a6640 20 osd.0 64 await_reserved_maps done 0 2026-02-11T23:30:39.312+0000 7f41591a6640 20 osd.0 65 consume_map 65 -- publish done 2026-02-11T23:30:39.312+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 3 slots 2026-02-11T23:30:39.312+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:6.0slot:0x37591e0 2026-02-11T23:30:39.312+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot 6.0 has no pg and waiting_for_split 2026-02-11T23:30:39.312+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x37582c0 2026-02-11T23:30:39.312+0000 7f41591a6640 20 osd.0 65 identify_splits_and_merges 4.0 e64 to e65 2026-02-11T23:30:39.312+0000 7f41591a6640 20 osd.0 65 identify_splits_and_merges 4.0 e64 to e65 pg_nums {49=1} 2026-02-11T23:30:39.312+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3700dc0 2026-02-11T23:30:39.312+0000 7f41591a6640 20 osd.0 65 identify_splits_and_merges 1.0 e64 to e65 2026-02-11T23:30:39.312+0000 7f41591a6640 20 osd.0 65 identify_splits_and_merges 1.0 e64 to e65 pg_nums {27=1} 2026-02-11T23:30:39.312+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:39.312+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:39.312+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:39.312+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:39.312+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:39.312+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:39.312+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:39.312+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:39.312+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:39.312+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:39.312+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:39.312+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:39.312+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:39.312+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:39.312+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:39.312+0000 7f41591a6640 20 osd.0 65 prune_pg_created 2026-02-11T23:30:39.312+0000 7f41591a6640 10 osd.0:0.consume_map 65 (was 64) 2026-02-11T23:30:39.312+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 6.0 2026-02-11T23:30:39.312+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 6.0 empty, pruning 2026-02-11T23:30:39.312+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:39.312+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:39.312+0000 7f41591a6640 10 osd.0:1.consume_map 65 (was 64) 2026-02-11T23:30:39.312+0000 7f41591a6640 10 osd.0:2.consume_map 65 (was 64) 2026-02-11T23:30:39.312+0000 7f41591a6640 10 osd.0:3.consume_map 65 (was 64) 2026-02-11T23:30:39.312+0000 7f41591a6640 10 osd.0:4.consume_map 65 (was 64) 2026-02-11T23:30:39.312+0000 7f41591a6640 10 osd.0:5.consume_map 65 (was 64) 2026-02-11T23:30:39.312+0000 7f41591a6640 10 osd.0:6.consume_map 65 (was 64) 2026-02-11T23:30:39.312+0000 7f41591a6640 10 osd.0:7.consume_map 65 (was 64) 2026-02-11T23:30:39.312+0000 7f41591a6640 10 osd.0 65 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:39.312+0000 7f41591a6640 15 osd.0 65 enqueue_peering_evt 4.0 epoch_sent: 65 epoch_requested: 65 NullEvt 2026-02-11T23:30:39.312+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 65 epoch_requested: 65 NullEvt) class_id 2 prio 255 cost 10 e65) 2026-02-11T23:30:39.312+0000 7f41591a6640 15 osd.0 65 enqueue_peering_evt 1.0 epoch_sent: 65 epoch_requested: 65 NullEvt 2026-02-11T23:30:39.312+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 65 epoch_requested: 65 NullEvt) class_id 2 prio 255 cost 10 e65) 2026-02-11T23:30:39.312+0000 7f41591a6640 7 osd.0 65 activate_map version 65 2026-02-11T23:30:39.312+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:39.312+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 65 epoch_requested: 65 NullEvt) class_id 2immediate prio 255 cost 10 e65) queued 2026-02-11T23:30:39.312+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:39.312+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 65 epoch_requested: 65 NullEvt) class_id 2immediate prio 255 cost 10 e65) pg 0x5978000 2026-02-11T23:30:39.312+0000 7f4150194640 20 osd.0 65 agent_entry empty queue 2026-02-11T23:30:39.312+0000 7f414f993640 10 osd.0 pg_epoch: 64 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 65 2026-02-11T23:30:39.312+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:39.312+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 65 epoch_requested: 65 NullEvt) class_id 2immediate prio 255 cost 10 e65) queued 2026-02-11T23:30:39.312+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:39.312+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 65 epoch_requested: 65 NullEvt) class_id 2immediate prio 255 cost 10 e65) pg 0x5946000 2026-02-11T23:30:39.312+0000 7f414f993640 20 osd.0:0.update_pg_epoch 4.0 64 -> 65 2026-02-11T23:30:39.312+0000 7f414f993640 10 osd.0 pg_epoch: 64 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:39.312+0000 7f414b98b640 10 osd.0 pg_epoch: 64 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 65 2026-02-11T23:30:39.312+0000 7f414f993640 10 osd.0 pg_epoch: 65 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:39.312+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 1.0 64 -> 65 2026-02-11T23:30:39.312+0000 7f414f993640 10 osd.0 pg_epoch: 65 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:39.312+0000 7f414b98b640 10 osd.0 pg_epoch: 64 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:39.312+0000 7f414f993640 20 osd.0 65 advance_pg new pool opts old pool opts 2026-02-11T23:30:39.312+0000 7f414f993640 10 osd.0 pg_epoch: 65 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_activate_map: epoch range: 65..65 2026-02-11T23:30:39.312+0000 7f414b98b640 10 osd.0 pg_epoch: 65 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:39.312+0000 7f414f993640 10 osd.0 pg_epoch: 65 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:39.312+0000 7f414b98b640 10 osd.0 pg_epoch: 65 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:39.312+0000 7f414f993640 20 osd.0 pg_epoch: 65 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 53 while current is 65 2026-02-11T23:30:39.312+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:39.312+0000 7f414b98b640 20 osd.0 65 advance_pg new pool opts old pool opts 2026-02-11T23:30:39.312+0000 7f414f993640 10 osd.0 pg_epoch: 65 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 65 epoch_requested: 65 NullEvt 2026-02-11T23:30:39.312+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:39.312+0000 7f414b98b640 10 osd.0 pg_epoch: 65 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_activate_map: epoch range: 65..65 2026-02-11T23:30:39.312+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:39.312+0000 7f414b98b640 10 osd.0 pg_epoch: 65 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:39.312+0000 7f414b98b640 20 osd.0 pg_epoch: 65 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 30 while current is 65 2026-02-11T23:30:39.312+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:39.312+0000 7f414b98b640 10 osd.0 pg_epoch: 65 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 65 epoch_requested: 65 NullEvt 2026-02-11T23:30:39.312+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:39.312+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:39.313+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 16 ==== PGlog(6.0 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 28.153278351s int 16.000000000s) e65/64) ==== 1172+0+0 (unknown 0 0 0) 0x5a95800 con 0x507ac00 2026-02-11T23:30:39.313+0000 7f416abfa640 15 osd.0 65 enqueue_peering_evt 6.0 epoch_sent: 65 epoch_requested: 64 MLogRec from 4 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 28.153278351s int 16.000000000s) +create_info 2026-02-11T23:30:39.313+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 65 epoch_requested: 64 MLogRec from 4 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 28.153278351s int 16.000000000s) +create_info) class_id 2 prio 255 cost 10 e65) 2026-02-11T23:30:39.313+0000 7f414f993640 20 osd.0 op_wq(0) _process 6.0 (new) to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:39.313+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 65 epoch_requested: 64 MLogRec from 4 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 28.153278351s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e65) queued 2026-02-11T23:30:39.313+0000 7f414f993640 20 osd.0 op_wq(0) _process 6.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:39.313+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 65 epoch_requested: 64 MLogRec from 4 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 28.153278351s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e65) pg 0 2026-02-11T23:30:39.313+0000 7f414f993640 20 osd.0 op_wq(0) _process 6.0 no pg, should create on OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 65 epoch_requested: 64 MLogRec from 4 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 28.153278351s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e65) 2026-02-11T23:30:39.313+0000 7f414f993640 10 osd.0 65 _make_pg 6.0 2026-02-11T23:30:39.313+0000 7f414f993640 10 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:6, shard:-1, shard_prefix: , prefixes: {}] SnapMapper 2026-02-11T23:30:39.313+0000 7f414f993640 20 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:6, shard:-1, shard_prefix: , prefixes: {}] update_bits new_bits: 0 2026-02-11T23:30:39.313+0000 7f414f993640 20 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:6, shard:-1, shard_prefix: , prefixes: {"0000000000000006."}] update_bits prefix updated 2026-02-11T23:30:39.313+0000 7f414f993640 10 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:6, shard:-1, shard_prefix: , prefixes: {"0000000000000006."}] reset_prefix_itr::from <0> to ::update_bits 2026-02-11T23:30:39.313+0000 7f414f993640 5 osd.0 pg_epoch: 64 pg[6.0(unlocked)] enter Initial 2026-02-11T23:30:39.313+0000 7f414f993640 20 osd.0 pg_epoch: 64 pg[6.0(unlocked)] enter NotTrimming 2026-02-11T23:30:39.313+0000 7f414f993640 10 osd.0 pg_epoch: 64 pg[6.0(unlocked)] scrubber: FSM: -- state -->> NotActive 2026-02-11T23:30:39.313+0000 7f414f993640 10 osd.0 pg_epoch: 64 pg[6.0( DNE empty local-lis/les=0/0 n=0 ec=0/0 lis/c=0/0 les/c/f=0/0/0 sis=0) [] r=-1 lpr=0 crt=0'0 unknown mbc={}] PeeringState::init init role 2 up [4,2,0] acting [4,2,0] history ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64 past_intervals ([0,0] all_participants={} intervals=[]) 2026-02-11T23:30:39.313+0000 7f414f993640 20 osd.0 pg_epoch: 64 pg[6.0( empty local-lis/les=0/0 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=0 crt=0'0 unknown mbc={}] PeeringState::on_new_interval 2026-02-11T23:30:39.313+0000 7f414f993640 20 osd.0 pg_epoch: 64 pg[6.0( empty local-lis/les=0/0 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=0 crt=0'0 unknown mbc={}] PeeringState::on_new_interval upacting_features 0x3f0ffffffffdffff from [4,2,0]+[4,2,0] 2026-02-11T23:30:39.313+0000 7f414f993640 20 osd.0 pg_epoch: 64 pg[6.0( empty local-lis/les=0/0 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=0 crt=0'0 unknown mbc={}] PeeringState::on_new_interval checking missing set deletes flag. missing = missing(0 may_include_deletes = 1) 2026-02-11T23:30:39.313+0000 7f414f993640 10 osd.0 pg_epoch: 64 pg[6.0( empty local-lis/les=0/0 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=0 crt=0'0 unknown mbc={}] PeeringState::init_hb_stamps now [0x56859d0] 2026-02-11T23:30:39.313+0000 7f414f993640 10 osd.0 pg_epoch: 64 pg[6.0( empty local-lis/les=0/0 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=0 crt=0'0 unknown mbc={}] PeeringState::on_new_interval prior_readable_until_ub 0.000000000s (mnow 21.990110397s + 0.000000000s) 2026-02-11T23:30:39.313+0000 7f414f993640 10 osd.0 pg_epoch: 64 pg[6.0( empty local-lis/les=0/0 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=0 crt=0'0 unknown mbc={}] cancel_recovery 2026-02-11T23:30:39.313+0000 7f414f993640 10 osd.0 pg_epoch: 64 pg[6.0( empty local-lis/les=0/0 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=0 crt=0'0 unknown mbc={}] clear_recovery_state 2026-02-11T23:30:39.313+0000 7f414f993640 15 osd.0 pg_epoch: 64 pg[6.0( empty local-lis/les=0/0 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=0 crt=0'0 unknown mbc={}] _clear_recovery_state 2026-02-11T23:30:39.313+0000 7f414f993640 10 osd.0 pg_epoch: 64 pg[6.0( empty local-lis/les=0/0 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=0 crt=0'0 unknown mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:39.313+0000 7f414f993640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:39.313+0000 7f414f993640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:39.313+0000 7f414f993640 6 write_log_and_missing with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, writeout_from: 4294967295'18446744073709551615, trimmed: , trimmed_dups: , clear_divergent_priors: 0 2026-02-11T23:30:39.313+0000 7f414f993640 10 _write_log_and_missing clearing up to 0'0 dirty_to_dups=0'0 dirty_from_dups=4294967295'18446744073709551615 write_from_dups=4294967295'18446744073709551615 trimmed_dups.size()=0 2026-02-11T23:30:39.313+0000 7f414f993640 10 _write_log_and_missing going to encode log.dups.size()=0 2026-02-11T23:30:39.313+0000 7f414f993640 10 _write_log_and_missing 1st round encoded log.dups.size()=0 2026-02-11T23:30:39.313+0000 7f414f993640 10 _write_log_and_missing 2st round encoded log.dups.size()=0 2026-02-11T23:30:39.313+0000 7f414f993640 10 end of _write_log_and_missing 2026-02-11T23:30:39.313+0000 7f414f993640 10 osd.0 pg_epoch: 64 pg[6.0( empty local-lis/les=0/0 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=0 crt=0'0 unknown mbc={}] handle_initialize 2026-02-11T23:30:39.313+0000 7f414f993640 5 osd.0 pg_epoch: 64 pg[6.0( empty local-lis/les=0/0 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=0 crt=0'0 unknown mbc={}] exit Initial 0.000054 0 0.000000 2026-02-11T23:30:39.313+0000 7f414f993640 5 osd.0 pg_epoch: 64 pg[6.0( empty local-lis/les=0/0 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=0 crt=0'0 unknown mbc={}] enter Reset 2026-02-11T23:30:39.313+0000 7f414f993640 20 osd.0 pg_epoch: 64 pg[6.0( empty local-lis/les=0/0 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=0 crt=0'0 unknown mbc={}] PeeringState::set_last_peering_reset set_last_peering_reset 64 2026-02-11T23:30:39.313+0000 7f414f993640 10 osd.0 pg_epoch: 64 pg[6.0( empty local-lis/les=0/0 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=64 crt=0'0 unknown mbc={}] PeeringState::set_last_peering_reset Clearing blocked outgoing recovery messages 2026-02-11T23:30:39.313+0000 7f414f993640 10 osd.0 pg_epoch: 64 pg[6.0( empty local-lis/les=0/0 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=64 crt=0'0 unknown mbc={}] PeeringState::set_last_peering_reset Not blocking outgoing recovery messages 2026-02-11T23:30:39.313+0000 7f414f993640 10 osd.0 pg_epoch: 64 pg[6.0( empty local-lis/les=0/0 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=64 crt=0'0 unknown mbc={}] handle_activate_map: epoch range: 64..64 2026-02-11T23:30:39.313+0000 7f414f993640 10 osd.0 pg_epoch: 64 pg[6.0( empty local-lis/les=0/0 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=64 crt=0'0 unknown mbc={}] PeeringState::activate_map 2026-02-11T23:30:39.313+0000 7f414f993640 5 osd.0 pg_epoch: 64 pg[6.0( empty local-lis/les=0/0 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=64 crt=0'0 unknown mbc={}] exit Reset 0.000008 1 0.000008 2026-02-11T23:30:39.313+0000 7f414f993640 5 osd.0 pg_epoch: 64 pg[6.0( empty local-lis/les=0/0 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=64 crt=0'0 unknown mbc={}] enter Started 2026-02-11T23:30:39.313+0000 7f414f993640 5 osd.0 pg_epoch: 64 pg[6.0( empty local-lis/les=0/0 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=64 crt=0'0 unknown mbc={}] enter Start 2026-02-11T23:30:39.313+0000 7f414f993640 1 osd.0 pg_epoch: 64 pg[6.0( empty local-lis/les=0/0 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=64 crt=0'0 unknown mbc={}] state: transitioning to Stray 2026-02-11T23:30:39.313+0000 7f414f993640 5 osd.0 pg_epoch: 64 pg[6.0( empty local-lis/les=0/0 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=64 crt=0'0 unknown mbc={}] exit Start 0.000002 0 0.000000 2026-02-11T23:30:39.313+0000 7f414f993640 5 osd.0 pg_epoch: 64 pg[6.0( empty local-lis/les=0/0 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=64 crt=0'0 unknown mbc={}] enter Started/Stray 2026-02-11T23:30:39.313+0000 7f414f993640 20 osd.0 pg_epoch: 64 pg[6.0( empty local-lis/les=0/0 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=64 crt=0'0 unknown mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 64 while current is 64 2026-02-11T23:30:39.313+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:39.313+0000 7f414f993640 10 osd.0 pg_epoch: 64 pg[6.0( empty local-lis/les=0/0 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=64 crt=0'0 unknown mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:30:39.313+0000 7f414f993640 10 osd.0 65 handle_pg_create_info new pg pg[6.0( empty local-lis/les=0/0 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=64 crt=0'0 unknown mbc={}] 2026-02-11T23:30:39.313+0000 7f414f993640 10 osd.0:0._attach_pg 6.0 0x4aa0000 2026-02-11T23:30:39.313+0000 7f414f993640 20 osd.0:0._wake_pg_slot _wake_pg_slot 6.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:39.313+0000 7f414f993640 20 osd.0 65 identify_splits_and_merges 6.0 e64 to e65 2026-02-11T23:30:39.313+0000 7f414f993640 20 osd.0 65 identify_splits_and_merges 6.0 e64 to e65 pg_nums {64=1} 2026-02-11T23:30:39.313+0000 7f414f993640 10 osd.0:0._prime_splits 2026-02-11T23:30:39.313+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:39.313+0000 7f414f993640 10 osd.0 pg_epoch: 64 pg[6.0( empty local-lis/les=0/0 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=64 crt=0'0 unknown mbc={}] handle_advance_map: 65 2026-02-11T23:30:39.313+0000 7f414f993640 20 osd.0:0.update_pg_epoch 6.0 64 -> 65 2026-02-11T23:30:39.313+0000 7f414f993640 10 osd.0 pg_epoch: 64 pg[6.0( empty local-lis/les=0/0 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=64 crt=0'0 unknown mbc={}] PeeringState::advance_map handle_advance_map [4,2,0]/[4,2,0] -- 4/4 2026-02-11T23:30:39.313+0000 7f414f993640 10 osd.0 pg_epoch: 65 pg[6.0( empty local-lis/les=0/0 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=64 crt=0'0 unknown mbc={}] state: Started advmap 2026-02-11T23:30:39.313+0000 7f414f993640 10 osd.0 pg_epoch: 65 pg[6.0( empty local-lis/les=0/0 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=64 crt=0'0 unknown mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:39.313+0000 7f414f993640 20 osd.0 65 advance_pg new pool opts old pool opts 2026-02-11T23:30:39.313+0000 7f414f993640 10 osd.0 pg_epoch: 65 pg[6.0( empty local-lis/les=0/0 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=64 crt=0'0 unknown mbc={}] handle_activate_map: epoch range: 65..65 2026-02-11T23:30:39.313+0000 7f414f993640 10 osd.0 pg_epoch: 65 pg[6.0( empty local-lis/les=0/0 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=64 crt=0'0 unknown mbc={}] PeeringState::activate_map 2026-02-11T23:30:39.313+0000 7f414f993640 20 osd.0 pg_epoch: 65 pg[6.0( empty local-lis/les=0/0 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=64 crt=0'0 unknown mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 64 while current is 65 2026-02-11T23:30:39.313+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:39.313+0000 7f414f993640 10 osd.0 pg_epoch: 65 pg[6.0( empty local-lis/les=0/0 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=64 crt=0'0 unknown mbc={}] do_peering_event: epoch_sent: 65 epoch_requested: 64 MLogRec from 4 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 28.153278351s int 16.000000000s) +create_info 2026-02-11T23:30:39.313+0000 7f414f993640 10 osd.0 pg_epoch: 65 pg[6.0( empty local-lis/les=0/0 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=64 crt=0'0 unknown mbc={}] state: got info+log from osd.4 6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) log((0'0,0'0], crt=0'0) 2026-02-11T23:30:39.313+0000 7f414f993640 10 merge_log log((0'0,0'0], crt=0'0) from osd.4 into log((0'0,0'0], crt=0'0) 2026-02-11T23:30:39.313+0000 7f414f993640 5 merge_log_dups log.dups.size()=0olog.dups.size()=0 2026-02-11T23:30:39.313+0000 7f414f993640 5 end of merge_log_dups changed=0 log.dups.size()=0 olog.dups.size()=0 2026-02-11T23:30:39.313+0000 7f414f993640 10 merge_log result log((0'0,0'0], crt=0'0) missing(0 may_include_deletes = 1) changed=0 2026-02-11T23:30:39.313+0000 7f414f993640 10 osd.0 pg_epoch: 65 pg[6.0( empty local-lis/les=0/0 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=64 crt=0'0 unknown mbc={}] PeeringState::proc_lease pg_lease(ru 0.000000000s ub 28.153278351s int 16.000000000s) 2026-02-11T23:30:39.313+0000 7f414f993640 20 osd.0 pg_epoch: 65 pg[6.0( empty local-lis/les=0/0 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=64 crt=0'0 unknown mbc={}] PeeringState::proc_lease peer_clock_delta_lb -9.836817741s -> ruub 37.990097046s 2026-02-11T23:30:39.313+0000 7f414f993640 20 osd.0 pg_epoch: 65 pg[6.0( empty local-lis/les=0/0 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=64 crt=0'0 unknown mbc={}] PeeringState::proc_lease readable_until_ub now 37.990097046s 2026-02-11T23:30:39.313+0000 7f414f993640 5 osd.0 pg_epoch: 65 pg[6.0( empty local-lis/les=0/0 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=64 crt=0'0 unknown mbc={}] exit Started/Stray 0.000050 3 0.000009 2026-02-11T23:30:39.313+0000 7f414f993640 5 osd.0 pg_epoch: 65 pg[6.0( empty local-lis/les=0/0 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=64 crt=0'0 unknown mbc={}] enter Started/ReplicaActive 2026-02-11T23:30:39.313+0000 7f414f993640 5 osd.0 pg_epoch: 65 pg[6.0( empty local-lis/les=0/0 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=64 crt=0'0 unknown mbc={}] enter Started/ReplicaActive/RepNotRecovering 2026-02-11T23:30:39.313+0000 7f414f993640 10 osd.0 pg_epoch: 65 pg[6.0( empty local-lis/les=0/0 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=64 crt=0'0 unknown mbc={}] state: In ReplicaActive, about to call activate 2026-02-11T23:30:39.313+0000 7f414f993640 10 osd.0 pg_epoch: 65 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=64 crt=0'0 unknown mbc={}] PeeringState::activate activate - no missing, moving last_complete 0'0 -> 0'0 2026-02-11T23:30:39.313+0000 7f414f993640 10 osd.0 pg_epoch: 65 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=64 crt=0'0 unknown mbc={}] scrubber: on_replica_activate 2026-02-11T23:30:39.313+0000 7f414f993640 20 scrubberFSM event: --vvvv---- ReplicaActivate 2026-02-11T23:30:39.313+0000 7f414f993640 10 osd.0 pg_epoch: 65 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=64 crt=0'0 unknown mbc={}] scrubber: FSM: -- state -->> ReplicaActive 2026-02-11T23:30:39.313+0000 7f414f993640 10 osd.0 pg_epoch: 65 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=64 crt=0'0 unknown mbc={}] scrubber: FSM: -- state -->> ReplicaActive/ReplicaIdle 2026-02-11T23:30:39.313+0000 7f414f993640 20 scrubberFSM event: --^^^^---- ReplicaActivate 2026-02-11T23:30:39.313+0000 7f414f993640 10 osd.0 pg_epoch: 65 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=64 crt=0'0 unknown mbc={}] state: Activate Finished 2026-02-11T23:30:39.313+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:39.313+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:39.313+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:39.313+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:39.313+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:39.313+0000 7f414f993640 15 osd.0 65 enqueue_peering_evt 6.0 epoch_sent: 65 epoch_requested: 65 ActivateCommitted from 65 processed at 65 2026-02-11T23:30:39.313+0000 7f414f993640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 65 epoch_requested: 65 ActivateCommitted from 65 processed at 65) class_id 2 prio 255 cost 10 e65) 2026-02-11T23:30:39.313+0000 7f414f993640 20 osd.0 op_wq(0) _process 6.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:39.313+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 65 epoch_requested: 65 ActivateCommitted from 65 processed at 65) class_id 2immediate prio 255 cost 10 e65) queued 2026-02-11T23:30:39.313+0000 7f414f993640 20 osd.0 op_wq(0) _process 6.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:39.313+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 65 epoch_requested: 65 ActivateCommitted from 65 processed at 65) class_id 2immediate prio 255 cost 10 e65) pg 0x4aa0000 2026-02-11T23:30:39.313+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:39.313+0000 7f414f993640 10 osd.0 pg_epoch: 65 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=64 crt=0'0 unknown mbc={}] do_peering_event: epoch_sent: 65 epoch_requested: 65 ActivateCommitted from 65 processed at 65 2026-02-11T23:30:39.313+0000 7f414f993640 10 osd.0 pg_epoch: 65 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=64 crt=0'0 unknown mbc={}] state: react 65 telling primary 2026-02-11T23:30:39.313+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:39.313+0000 7f414f993640 20 osd.0 65 get_con_osd_cluster to osd.4 from_epoch 65 2026-02-11T23:30:39.313+0000 7f414f993640 20 osd.0 65 get_nextmap_reserved map_reservations: {65=1} 2026-02-11T23:30:39.313+0000 7f414f993640 20 osd.0 65 release_map epoch: 65 2026-02-11T23:30:39.313+0000 7f414f993640 10 osd.0 65 maybe_share_map: con v2:127.0.0.1:6836/3331151400 map epoch 64 -> 65 (shared) 2026-02-11T23:30:39.313+0000 7f414f993640 10 osd.0 65 send_incremental_map epoch range: (65, 65] to v2:127.0.0.1:6836/3331151400 2026-02-11T23:30:39.313+0000 7f414f993640 10 osd.0 65 add_map_inc_bl 65 252 bytes 2026-02-11T23:30:39.313+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- osd_map(65..65 src has 1..65) -- 0x4a31500 con 0x507ac00 2026-02-11T23:30:39.313+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_info2(6.0 6.0 6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=64/0 les/c/f=65/0/0 sis=64) e65/65) -- 0x373aa00 con 0x507ac00 2026-02-11T23:30:39.313+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:39.327+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:30:39.327+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:30:39.327+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:09.327782+0000) 2026-02-11T23:30:39.383+0000 7f41633eb640 10 osd.0 65 tick 2026-02-11T23:30:39.383+0000 7f41633eb640 20 osd.0 65 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:30:39.738+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 38 ==== osd_map(65..66 src has 1..66) ==== 878+0+0 (crc 0 0 0) 0x5955340 con 0x4acbc00 2026-02-11T23:30:39.738+0000 7f415cf02640 20 osd.0 65 OSD::ms_dispatch: osd_map(65..66 src has 1..66) 2026-02-11T23:30:39.738+0000 7f415cf02640 20 osd.0 65 _dispatch 0x5955340 osd_map(65..66 src has 1..66) 2026-02-11T23:30:39.738+0000 7f415cf02640 3 osd.0 65 handle_osd_map epochs [65,66], i have 65, src has [1,66] 2026-02-11T23:30:39.738+0000 7f415cf02640 10 osd.0 65 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:39.738+0000 7f415cf02640 10 osd.0 65 handle_osd_map got inc map for epoch 66 2026-02-11T23:30:39.738+0000 7f415cf02640 20 osd.0 65 got_full_map 66, nothing requested 2026-02-11T23:30:39.738+0000 7f415cf02640 20 osd.0 65 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:39.738+0000 7f415cf02640 20 osd.0 65 handle_osd_map pg_num_history pg_num_history(e66 pg_nums {1={27=1},2={34=1},3={42=1},4={49=1},5={56=1},6={64=1}} deleted_pools ) 2026-02-11T23:30:39.738+0000 7f415cf02640 10 snap_mapper record_purged_snaps purged_snaps {66={}} 2026-02-11T23:30:39.738+0000 7f415cf02640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:39.738+0000 7f415cf02640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e66 maps [1~66] lci=[5,66] tlb=1) 2026-02-11T23:30:39.739+0000 7f41591a6640 10 osd.0 65 _committed_osd_maps 66..66 2026-02-11T23:30:39.739+0000 7f41591a6640 10 osd.0 65 advance to epoch 66 (<= last 66 <= newest_map 66) 2026-02-11T23:30:39.739+0000 7f41591a6640 20 osd.0 66 consume_map version 66 2026-02-11T23:30:39.739+0000 7f41591a6640 20 osd.0 65 await_reserved_maps epoch:66 2026-02-11T23:30:39.739+0000 7f41591a6640 20 osd.0 65 await_reserved_maps done 0 2026-02-11T23:30:39.739+0000 7f41591a6640 20 osd.0 66 consume_map 66 -- publish done 2026-02-11T23:30:39.739+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 3 slots 2026-02-11T23:30:39.739+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:6.0slot:0x3758b00 2026-02-11T23:30:39.739+0000 7f41591a6640 20 osd.0 66 identify_splits_and_merges 6.0 e65 to e66 2026-02-11T23:30:39.739+0000 7f41591a6640 20 osd.0 66 identify_splits_and_merges 6.0 e65 to e66 pg_nums {64=1} 2026-02-11T23:30:39.739+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x37582c0 2026-02-11T23:30:39.739+0000 7f41591a6640 20 osd.0 66 identify_splits_and_merges 4.0 e65 to e66 2026-02-11T23:30:39.739+0000 7f41591a6640 20 osd.0 66 identify_splits_and_merges 4.0 e65 to e66 pg_nums {49=1} 2026-02-11T23:30:39.739+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3700dc0 2026-02-11T23:30:39.739+0000 7f41591a6640 20 osd.0 66 identify_splits_and_merges 1.0 e65 to e66 2026-02-11T23:30:39.739+0000 7f41591a6640 20 osd.0 66 identify_splits_and_merges 1.0 e65 to e66 pg_nums {27=1} 2026-02-11T23:30:39.739+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:39.739+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:39.739+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:39.739+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:39.739+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:39.739+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:39.739+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:39.739+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:39.739+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:39.739+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:39.739+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:39.739+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:39.739+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:39.739+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:39.739+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:39.739+0000 7f41591a6640 20 osd.0 66 prune_pg_created 2026-02-11T23:30:39.739+0000 7f41591a6640 10 osd.0:0.consume_map 66 (was 65) 2026-02-11T23:30:39.739+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 6.0 2026-02-11T23:30:39.739+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:39.739+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:39.739+0000 7f41591a6640 10 osd.0:1.consume_map 66 (was 65) 2026-02-11T23:30:39.739+0000 7f41591a6640 10 osd.0:2.consume_map 66 (was 65) 2026-02-11T23:30:39.739+0000 7f41591a6640 10 osd.0:3.consume_map 66 (was 65) 2026-02-11T23:30:39.739+0000 7f41591a6640 10 osd.0:4.consume_map 66 (was 65) 2026-02-11T23:30:39.739+0000 7f41591a6640 10 osd.0:5.consume_map 66 (was 65) 2026-02-11T23:30:39.739+0000 7f41591a6640 10 osd.0:6.consume_map 66 (was 65) 2026-02-11T23:30:39.739+0000 7f41591a6640 10 osd.0:7.consume_map 66 (was 65) 2026-02-11T23:30:39.739+0000 7f41591a6640 10 osd.0 66 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:39.739+0000 7f41591a6640 15 osd.0 66 enqueue_peering_evt 6.0 epoch_sent: 66 epoch_requested: 66 NullEvt 2026-02-11T23:30:39.739+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 66 epoch_requested: 66 NullEvt) class_id 2 prio 255 cost 10 e66) 2026-02-11T23:30:39.739+0000 7f41591a6640 15 osd.0 66 enqueue_peering_evt 4.0 epoch_sent: 66 epoch_requested: 66 NullEvt 2026-02-11T23:30:39.739+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 66 epoch_requested: 66 NullEvt) class_id 2 prio 255 cost 10 e66) 2026-02-11T23:30:39.739+0000 7f41591a6640 15 osd.0 66 enqueue_peering_evt 1.0 epoch_sent: 66 epoch_requested: 66 NullEvt 2026-02-11T23:30:39.739+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 66 epoch_requested: 66 NullEvt) class_id 2 prio 255 cost 10 e66) 2026-02-11T23:30:39.739+0000 7f41591a6640 7 osd.0 66 activate_map version 66 2026-02-11T23:30:39.739+0000 7f414f993640 20 osd.0 op_wq(0) _process 6.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:39.739+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 66 epoch_requested: 66 NullEvt) class_id 2immediate prio 255 cost 10 e66) queued 2026-02-11T23:30:39.739+0000 7f414f993640 20 osd.0 op_wq(0) _process 6.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:39.739+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 66 epoch_requested: 66 NullEvt) class_id 2immediate prio 255 cost 10 e66) pg 0x4aa0000 2026-02-11T23:30:39.739+0000 7f4150194640 20 osd.0 66 agent_entry empty queue 2026-02-11T23:30:39.739+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:39.739+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 66 epoch_requested: 66 NullEvt) class_id 2immediate prio 255 cost 10 e66) queued 2026-02-11T23:30:39.739+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:39.739+0000 7f414f993640 10 osd.0 pg_epoch: 65 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=64 crt=0'0 active mbc={}] handle_advance_map: 66 2026-02-11T23:30:39.739+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 66 epoch_requested: 66 NullEvt) class_id 2immediate prio 255 cost 10 e66) pg 0x5978000 2026-02-11T23:30:39.739+0000 7f414f993640 20 osd.0:0.update_pg_epoch 6.0 65 -> 66 2026-02-11T23:30:39.739+0000 7f414f993640 10 osd.0 pg_epoch: 65 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=64 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,2]/[4,2] -- 4/4 2026-02-11T23:30:39.739+0000 7f414f993640 10 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=64 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:39.739+0000 7f414b98b640 10 osd.0 pg_epoch: 65 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 66 2026-02-11T23:30:39.739+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 4.0 65 -> 66 2026-02-11T23:30:39.739+0000 7f414f993640 20 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=64 crt=0'0 active mbc={}] PeeringState::should_restart_peering new interval newup [4,2] newacting [4,2] 2026-02-11T23:30:39.739+0000 7f414f993640 10 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=64 crt=0'0 active mbc={}] state: should_restart_peering, transitioning to Reset 2026-02-11T23:30:39.739+0000 7f414b98b640 10 osd.0 pg_epoch: 65 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:39.739+0000 7f414f993640 5 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=64 crt=0'0 active mbc={}] exit Started/ReplicaActive/RepNotRecovering 0.425710 2 0.000024 2026-02-11T23:30:39.739+0000 7f414b98b640 10 osd.0 pg_epoch: 66 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:39.739+0000 7f414f993640 5 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=64 crt=0'0 active mbc={}] exit Started/ReplicaActive 0.425714 0 0.000000 2026-02-11T23:30:39.739+0000 7f414b98b640 10 osd.0 pg_epoch: 66 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:39.739+0000 7f414f993640 5 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=64 crt=0'0 active mbc={}] exit Started 0.425773 0 0.000000 2026-02-11T23:30:39.739+0000 7f414b98b640 20 osd.0 66 advance_pg new pool opts old pool opts 2026-02-11T23:30:39.739+0000 7f414f993640 5 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=64 crt=0'0 active mbc={}] enter Reset 2026-02-11T23:30:39.739+0000 7f414b98b640 10 osd.0 pg_epoch: 66 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_activate_map: epoch range: 66..66 2026-02-11T23:30:39.739+0000 7f414f993640 20 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=64 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset set_last_peering_reset 66 2026-02-11T23:30:39.739+0000 7f414b98b640 10 osd.0 pg_epoch: 66 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:39.739+0000 7f414f993640 10 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=66 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset Clearing blocked outgoing recovery messages 2026-02-11T23:30:39.739+0000 7f414b98b640 20 osd.0 pg_epoch: 66 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 53 while current is 66 2026-02-11T23:30:39.739+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:39.739+0000 7f414b98b640 10 osd.0 pg_epoch: 66 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 66 epoch_requested: 66 NullEvt 2026-02-11T23:30:39.739+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:39.739+0000 7f414f993640 10 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=66 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset Not blocking outgoing recovery messages 2026-02-11T23:30:39.739+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:39.739+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 66 epoch_requested: 66 NullEvt) class_id 2immediate prio 255 cost 10 e66) queued 2026-02-11T23:30:39.739+0000 7f414f993640 10 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=66 crt=0'0 active mbc={}] state: Reset advmap 2026-02-11T23:30:39.739+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:39.739+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 66 epoch_requested: 66 NullEvt) class_id 2immediate prio 255 cost 10 e66) pg 0x5946000 2026-02-11T23:30:39.739+0000 7f414f993640 20 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=66 crt=0'0 active mbc={}] PeeringState::should_restart_peering new interval newup [4,2] newacting [4,2] 2026-02-11T23:30:39.739+0000 7f414f993640 10 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=66 crt=0'0 active mbc={}] state: should restart peering, calling start_peering_interval again 2026-02-11T23:30:39.739+0000 7f414b98b640 10 osd.0 pg_epoch: 65 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 66 2026-02-11T23:30:39.739+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 1.0 65 -> 66 2026-02-11T23:30:39.739+0000 7f414f993640 20 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2,0] r=2 lpr=66 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset set_last_peering_reset 66 2026-02-11T23:30:39.739+0000 7f414b98b640 10 osd.0 pg_epoch: 65 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:39.739+0000 7f414f993640 15 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2] r=2 lpr=66 crt=0'0 active mbc={}] clear_stats 2026-02-11T23:30:39.739+0000 7f414b98b640 10 osd.0 pg_epoch: 66 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:39.739+0000 7f414b98b640 10 osd.0 pg_epoch: 66 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:39.739+0000 7f414b98b640 20 osd.0 66 advance_pg new pool opts old pool opts 2026-02-11T23:30:39.739+0000 7f414b98b640 10 osd.0 pg_epoch: 66 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_activate_map: epoch range: 66..66 2026-02-11T23:30:39.739+0000 7f414b98b640 10 osd.0 pg_epoch: 66 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:39.739+0000 7f414b98b640 20 osd.0 pg_epoch: 66 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 30 while current is 66 2026-02-11T23:30:39.739+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:39.739+0000 7f414f993640 10 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 active mbc={}] PeeringState::start_peering_interval : check_new_interval output: check_new_interval interval(64-65 up [4, 2, 0](4) acting [4, 2, 0](4)) up_thru 64 up_from 23 last_epoch_clean 0 interval(64-65 up [4, 2, 0](4) acting [4, 2, 0](4) maybe_went_rw) : primary up 23-64 includes interval 2026-02-11T23:30:39.739+0000 7f414b98b640 10 osd.0 pg_epoch: 66 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 66 epoch_requested: 66 NullEvt 2026-02-11T23:30:39.739+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:39.739+0000 7f414f993640 10 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=64) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 active mbc={}] PeeringState::start_peering_interval noting past ([64,65] all_participants={0, 2, 4} intervals=[([64,65] acting={0, 2, 4})]) 2026-02-11T23:30:39.739+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:39.739+0000 7f414f993640 20 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 active mbc={}] PeeringState::on_new_interval 2026-02-11T23:30:39.739+0000 7f414f993640 20 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 active mbc={}] PeeringState::on_new_interval upacting_features 0x3f0ffffffffdffff from [4,2]+[4,2] 2026-02-11T23:30:39.739+0000 7f414f993640 20 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 active mbc={}] PeeringState::on_new_interval checking missing set deletes flag. missing = missing(0 may_include_deletes = 1) 2026-02-11T23:30:39.739+0000 7f414f993640 10 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 active mbc={}] PeeringState::init_hb_stamps now [] 2026-02-11T23:30:39.739+0000 7f414f993640 10 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66 pruub=15.574158669s) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 active pruub 37.990097046s@ mbc={}] PeeringState::on_new_interval prior_readable_until_ub 37.990097046s (mnow 22.415937424s + 15.574158669s) 2026-02-11T23:30:39.739+0000 7f414f993640 10 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66 pruub=15.574158669s) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 active pruub 37.990097046s@ mbc={}] cancel_recovery 2026-02-11T23:30:39.739+0000 7f414f993640 10 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66 pruub=15.574158669s) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 active pruub 37.990097046s@ mbc={}] clear_recovery_state 2026-02-11T23:30:39.739+0000 7f414f993640 15 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66 pruub=15.574158669s) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 active pruub 37.990097046s@ mbc={}] _clear_recovery_state 2026-02-11T23:30:39.739+0000 7f414f993640 10 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66 pruub=15.574158669s) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 active pruub 37.990097046s@ mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:39.739+0000 7f414f993640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:39.739+0000 7f414f993640 20 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66 pruub=15.574158669s) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 active pruub 37.990097046s@ mbc={}] scrubber: FSM: ReplicaActive::exit() 2026-02-11T23:30:39.739+0000 7f414f993640 10 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66 pruub=15.574158669s) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 active pruub 37.990097046s@ mbc={}] scrubber: FSM: ReplicaActive::clear_remote_reservation(): pending_reservation_nonce 0, reservation_granted false 2026-02-11T23:30:39.739+0000 7f414f993640 10 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66 pruub=15.574158669s) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 active pruub 37.990097046s@ mbc={}] scrubber: FSM: -- state -->> NotActive 2026-02-11T23:30:39.739+0000 7f414f993640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:39.739+0000 7f414f993640 1 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66 pruub=15.574158669s) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 active pruub 37.990097046s@ mbc={}] PeeringState::start_peering_interval up [4,2,0] -> [4,2], acting [4,2,0] -> [4,2], acting_primary 4 -> 4, up_primary 4 -> 4, role 2 -> -1, features acting 4544132024016699391 upacting 4544132024016699391 2026-02-11T23:30:39.739+0000 7f414f993640 10 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66 pruub=15.574158669s) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown pruub 37.990097046s@ mbc={}] PeeringState::clear_primary_state clear_primary_state 2026-02-11T23:30:39.739+0000 7f414f993640 20 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66 pruub=15.574158669s) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown pruub 37.990097046s@ mbc={}] clear_primary_state 2026-02-11T23:30:39.739+0000 7f414f993640 10 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66 pruub=15.574158669s) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown pruub 37.990097046s@ mbc={}] release_backoffs [6:00000000::::0,MAX) 2026-02-11T23:30:39.739+0000 7f414f993640 10 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66 pruub=15.574158669s) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown pruub 37.990097046s@ mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:39.739+0000 7f414f993640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:39.739+0000 7f414f993640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:39.739+0000 7f414f993640 20 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66 pruub=15.574158669s) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown pruub 37.990097046s@ mbc={}] agent_stop 2026-02-11T23:30:39.739+0000 7f414f993640 10 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66 pruub=15.574158669s) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown pruub 37.990097046s@ mbc={}] on_change 2026-02-11T23:30:39.739+0000 7f414f993640 10 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66 pruub=15.574158669s) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown pruub 37.990097046s@ mbc={}] cancel_copy_ops 2026-02-11T23:30:39.739+0000 7f414f993640 10 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66 pruub=15.574158669s) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown pruub 37.990097046s@ mbc={}] cancel_flush_ops 2026-02-11T23:30:39.739+0000 7f414f993640 10 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66 pruub=15.574158669s) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown pruub 37.990097046s@ mbc={}] cancel_proxy_ops 2026-02-11T23:30:39.739+0000 7f414f993640 10 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66 pruub=15.574158669s) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown pruub 37.990097046s@ mbc={}] cancel_manifest_ops 2026-02-11T23:30:39.739+0000 7f414f993640 10 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66 pruub=15.574158669s) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown pruub 37.990097046s@ mbc={}] cancel_cls_gather_ops 2026-02-11T23:30:39.739+0000 7f414f993640 10 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66 pruub=15.574158669s) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown pruub 37.990097046s@ mbc={}] on_change_cleanup 2026-02-11T23:30:39.739+0000 7f414f993640 10 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66 pruub=15.574158669s) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown pruub 37.990097046s@ mbc={}] on_change 2026-02-11T23:30:39.739+0000 7f414f993640 20 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66 pruub=15.574158669s) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown pruub 37.990097046s@ mbc={}] exit NotTrimming 2026-02-11T23:30:39.739+0000 7f414f993640 20 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66 pruub=15.574158669s) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown pruub 37.990097046s@ mbc={}] enter NotTrimming 2026-02-11T23:30:39.739+0000 7f414f993640 10 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66 pruub=15.574158669s) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown NOTIFY pruub 37.990097046s@ mbc={}] plpg_on_role_change 2026-02-11T23:30:39.739+0000 7f414f993640 10 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66 pruub=15.574158669s) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown NOTIFY pruub 37.990097046s@ mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:39.739+0000 7f414f993640 10 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66 pruub=15.574158669s) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown NOTIFY pruub 37.990097046s@ mbc={}] plpg_on_pool_change 2026-02-11T23:30:39.739+0000 7f414f993640 20 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66 pruub=15.574158669s) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown NOTIFY pruub 37.990097046s@ mbc={}] hit_set_clear 2026-02-11T23:30:39.739+0000 7f414f993640 20 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66 pruub=15.574158669s) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown NOTIFY pruub 37.990097046s@ mbc={}] agent_stop 2026-02-11T23:30:39.739+0000 7f414f993640 20 osd.0 66 advance_pg new pool opts old pool opts 2026-02-11T23:30:39.739+0000 7f414f993640 10 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66 pruub=15.574158669s) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown NOTIFY pruub 37.990097046s@ mbc={}] handle_activate_map: epoch range: 66..66 2026-02-11T23:30:39.739+0000 7f414f993640 10 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66 pruub=15.574158669s) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown NOTIFY pruub 37.990097046s@ mbc={}] PeeringState::activate_map 2026-02-11T23:30:39.739+0000 7f414f993640 5 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66 pruub=15.574115753s) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown NOTIFY pruub 37.990097046s@ mbc={}] exit Reset 0.000070 1 0.000075 2026-02-11T23:30:39.739+0000 7f414f993640 5 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66 pruub=15.574115753s) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown NOTIFY pruub 37.990097046s@ mbc={}] enter Started 2026-02-11T23:30:39.739+0000 7f414f993640 5 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66 pruub=15.574115753s) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown NOTIFY pruub 37.990097046s@ mbc={}] enter Start 2026-02-11T23:30:39.739+0000 7f414f993640 1 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66 pruub=15.574115753s) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown NOTIFY pruub 37.990097046s@ mbc={}] state: transitioning to Stray 2026-02-11T23:30:39.739+0000 7f414f993640 5 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66 pruub=15.574115753s) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown NOTIFY pruub 37.990097046s@ mbc={}] exit Start 0.000002 0 0.000000 2026-02-11T23:30:39.739+0000 7f414f993640 5 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66 pruub=15.574115753s) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown NOTIFY pruub 37.990097046s@ mbc={}] enter Started/Stray 2026-02-11T23:30:39.739+0000 7f414f993640 20 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66 pruub=15.574115753s) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown NOTIFY pruub 37.990097046s@ mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 65 while current is 66 2026-02-11T23:30:39.739+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:39.739+0000 7f414f993640 10 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66 pruub=15.574115753s) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown NOTIFY pruub 37.990097046s@ mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:30:39.739+0000 7f414f993640 10 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66 pruub=15.574115753s) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown NOTIFY pruub 37.990097046s@ mbc={}] do_peering_event: epoch_sent: 66 epoch_requested: 66 NullEvt 2026-02-11T23:30:39.739+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:39.739+0000 7f414f993640 20 osd.0 66 get_con_osd_cluster to osd.4 from_epoch 66 2026-02-11T23:30:39.739+0000 7f414f993640 20 osd.0 66 get_nextmap_reserved map_reservations: {66=1} 2026-02-11T23:30:39.739+0000 7f414f993640 20 osd.0 66 release_map epoch: 66 2026-02-11T23:30:39.739+0000 7f414f993640 10 osd.0 66 maybe_share_map: con v2:127.0.0.1:6836/3331151400 map epoch 65 -> 66 (shared) 2026-02-11T23:30:39.739+0000 7f414f993640 10 osd.0 66 send_incremental_map epoch range: (66, 66] to v2:127.0.0.1:6836/3331151400 2026-02-11T23:30:39.739+0000 7f414f993640 10 osd.0 66 add_map_inc_bl 66 574 bytes 2026-02-11T23:30:39.739+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- osd_map(66..66 src has 1..66) -- 0x4a64fc0 con 0x507ac00 2026-02-11T23:30:39.739+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_notify2(6.0 6.0 (query:66 sent:66 6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66 pruub=15.574115753s) ([64,65] all_participants={0, 2, 4} intervals=[([64,65] acting={0, 2, 4})])) e66/66) -- 0x3739500 con 0x507ac00 2026-02-11T23:30:39.739+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:39.739+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:39.739+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:39.739+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:39.740+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 17 ==== pg_query2(6.0 6.0 query(info 0'0 epoch_sent 66) e66/66) ==== 158+0+0 (unknown 0 0 0) 0x4fca960 con 0x507ac00 2026-02-11T23:30:39.740+0000 7f416abfa640 15 osd.0 66 enqueue_peering_evt 6.0 epoch_sent: 66 epoch_requested: 66 MQuery 6.0 from 4 query_epoch 66 query: query(info 0'0 epoch_sent 66) 2026-02-11T23:30:39.740+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 66 epoch_requested: 66 MQuery 6.0 from 4 query_epoch 66 query: query(info 0'0 epoch_sent 66)) class_id 2 prio 255 cost 10 e66) 2026-02-11T23:30:39.740+0000 7f414b98b640 20 osd.0 op_wq(0) _process 6.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:39.740+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 66 epoch_requested: 66 MQuery 6.0 from 4 query_epoch 66 query: query(info 0'0 epoch_sent 66)) class_id 2immediate prio 255 cost 10 e66) queued 2026-02-11T23:30:39.740+0000 7f414b98b640 20 osd.0 op_wq(0) _process 6.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:39.740+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 66 epoch_requested: 66 MQuery 6.0 from 4 query_epoch 66 query: query(info 0'0 epoch_sent 66)) class_id 2immediate prio 255 cost 10 e66) pg 0x4aa0000 2026-02-11T23:30:39.740+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:39.740+0000 7f414b98b640 10 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66 pruub=15.574115753s) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown NOTIFY pruub 37.990097046s@ mbc={}] do_peering_event: epoch_sent: 66 epoch_requested: 66 MQuery 6.0 from 4 query_epoch 66 query: query(info 0'0 epoch_sent 66) 2026-02-11T23:30:39.740+0000 7f414b98b640 20 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown NOTIFY pruub 37.990097046s@ mbc={}] PeeringState::update_history advanced history from ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66 2026-02-11T23:30:39.740+0000 7f414b98b640 10 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown NOTIFY mbc={}] PeeringState::fulfill_info sending info 2026-02-11T23:30:39.740+0000 7f415aefe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 18 ==== osd_map(65..66 src has 1..66) ==== 878+0+0 (unknown 0 0 0) 0x4a31a40 con 0x507ac00 2026-02-11T23:30:39.740+0000 7f415aefe640 20 osd.0 66 OSD::ms_dispatch: osd_map(65..66 src has 1..66) 2026-02-11T23:30:39.740+0000 7f415aefe640 20 osd.0 66 _dispatch 0x4a31a40 osd_map(65..66 src has 1..66) 2026-02-11T23:30:39.740+0000 7f415aefe640 3 osd.0 66 handle_osd_map epochs [65,66], i have 66, src has [1,66] 2026-02-11T23:30:39.740+0000 7f415aefe640 10 osd.0 66 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:39.740+0000 7f415aefe640 10 osd.0 66 no new maps here, dropping 2026-02-11T23:30:39.740+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:39.740+0000 7f414b98b640 20 osd.0 66 get_con_osd_cluster to osd.4 from_epoch 66 2026-02-11T23:30:39.740+0000 7f414b98b640 20 osd.0 66 get_nextmap_reserved map_reservations: {66=1} 2026-02-11T23:30:39.740+0000 7f414b98b640 20 osd.0 66 release_map epoch: 66 2026-02-11T23:30:39.740+0000 7f414b98b640 10 osd.0 66 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 66 is not newer than session's projected_epoch of 66 2026-02-11T23:30:39.740+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_notify2(6.0 6.0 (query:66 sent:66 6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) ([64,65] all_participants={0, 2, 4} intervals=[([64,65] acting={0, 2, 4})])) e66/66) -- 0x373b100 con 0x507ac00 2026-02-11T23:30:39.740+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:39.740+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:39.740+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:39.966+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 19 ==== osd pg remove(epoch 67; pg6.0; ) ==== 32+0+0 (unknown 0 0 0) 0x3700000 con 0x507ac00 2026-02-11T23:30:39.966+0000 7f416abfa640 7 osd.0 66 handle_fast_pg_remove osd pg remove(epoch 67; pg6.0; ) from osd.4 2026-02-11T23:30:39.966+0000 7f416abfa640 15 osd.0 66 enqueue_peering_evt 6.0 epoch_sent: 67 epoch_requested: 67 DeleteStart 2026-02-11T23:30:39.966+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 67 epoch_requested: 67 DeleteStart) class_id 2 prio 255 cost 10 e67) 2026-02-11T23:30:39.966+0000 7f414b98b640 20 osd.0 op_wq(0) _process 6.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:39.966+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 67 epoch_requested: 67 DeleteStart) class_id 2immediate prio 255 cost 10 e67) queued 2026-02-11T23:30:39.966+0000 7f414b98b640 20 osd.0 op_wq(0) _process 6.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:39.966+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 67 epoch_requested: 67 DeleteStart) class_id 2immediate prio 255 cost 10 e67) pg 0x4aa0000 2026-02-11T23:30:39.966+0000 7f414b98b640 20 osd.0 op_wq _add_slot_waiter 6.0 peering, item epoch is 67, will wait on OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 67 epoch_requested: 67 DeleteStart) class_id 2immediate prio 255 cost 10 e67) 2026-02-11T23:30:39.966+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:39.966+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:40.128+0000 7f4147983640 5 osd.0 66 heartbeat osd_stat(store_statfs(0x384000/0x0/0x384000, data 0x3ad23/0x92000, compress 0x0/0x0/0x0, omap 0x66fb, meta 0x1a49905), peers [1,2,3,4] op hist []) 2026-02-11T23:30:40.128+0000 7f4147983640 20 osd.0 66 check_full_status cur ratio 0, physical ratio 0, new state none 2026-02-11T23:30:40.128+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6816/3043114013,v1:127.0.0.1:6817/3043114013] -- osd_ping(ping e66 up_from 5 ping_stamp 2026-02-11T23:30:40.128665+0000/22.804069519s send_stamp 22.804069519s delta_ub 2.011763096s) -- 0x4c9b500 con 0x48c4400 2026-02-11T23:30:40.128+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6814/3043114013,v1:127.0.0.1:6815/3043114013] -- osd_ping(ping e66 up_from 5 ping_stamp 2026-02-11T23:30:40.128665+0000/22.804069519s send_stamp 22.804069519s delta_ub 2.011763096s) -- 0x4c9b080 con 0x4640c00 2026-02-11T23:30:40.128+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6824/2365263801,v1:127.0.0.1:6825/2365263801] -- osd_ping(ping e66 up_from 5 ping_stamp 2026-02-11T23:30:40.128665+0000/22.804069519s send_stamp 22.804069519s delta_ub 4.566909790s) -- 0x4c9bb00 con 0x48c4000 2026-02-11T23:30:40.128+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6822/2365263801,v1:127.0.0.1:6823/2365263801] -- osd_ping(ping e66 up_from 5 ping_stamp 2026-02-11T23:30:40.128665+0000/22.804069519s send_stamp 22.804069519s delta_ub 4.566909790s) -- 0x4c9bc80 con 0x48cd000 2026-02-11T23:30:40.128+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6832/2699377081,v1:127.0.0.1:6833/2699377081] -- osd_ping(ping e66 up_from 5 ping_stamp 2026-02-11T23:30:40.128665+0000/22.804069519s send_stamp 22.804069519s delta_ub 7.219748974s) -- 0x4c9a300 con 0x48cd400 2026-02-11T23:30:40.128+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6830/2699377081,v1:127.0.0.1:6831/2699377081] -- osd_ping(ping e66 up_from 5 ping_stamp 2026-02-11T23:30:40.128665+0000/22.804069519s send_stamp 22.804069519s delta_ub 7.219748974s) -- 0x4c9b380 con 0x48cd800 2026-02-11T23:30:40.128+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6840/3331151400,v1:127.0.0.1:6841/3331151400] -- osd_ping(ping e66 up_from 5 ping_stamp 2026-02-11T23:30:40.128665+0000/22.804069519s send_stamp 22.804069519s delta_ub 9.836817741s) -- 0x4c9a180 con 0x507bc00 2026-02-11T23:30:40.128+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6838/3331151400,v1:127.0.0.1:6839/3331151400] -- osd_ping(ping e66 up_from 5 ping_stamp 2026-02-11T23:30:40.128665+0000/22.804069519s send_stamp 22.804069519s delta_ub 9.836817741s) -- 0x4c9a480 con 0x5282000 2026-02-11T23:30:40.128+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6816/3043114013 6 ==== osd_ping(ping_reply e64 up_from 10 ping_stamp 2026-02-11T23:30:40.128665+0000/22.804069519s send_stamp 20.792495728s delta_ub -2.011575699s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a600 con 0x48c4400 2026-02-11T23:30:40.128+0000 7f416abfa640 20 osd.0 66 get_con_osd_cluster to osd.1 from_epoch 66 2026-02-11T23:30:40.128+0000 7f416abfa640 20 osd.0 66 get_nextmap_reserved map_reservations: {66=1} 2026-02-11T23:30:40.128+0000 7f416abfa640 20 osd.0 66 release_map epoch: 66 2026-02-11T23:30:40.128+0000 7f416abfa640 10 osd.0 66 maybe_share_map: con v2:127.0.0.1:6812/3043114013 updating session's projected_epoch from 56 to ping map epoch of 64 2026-02-11T23:30:40.128+0000 7f416abfa640 10 osd.0 66 maybe_share_map: con v2:127.0.0.1:6812/3043114013 map epoch 64 -> 66 (shared) 2026-02-11T23:30:40.128+0000 7f416abfa640 10 osd.0 66 send_incremental_map epoch range: (65, 66] to v2:127.0.0.1:6812/3043114013 2026-02-11T23:30:40.128+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- osd_map(65..66 src has 1..66) -- 0x4a31880 con 0x4acb000 2026-02-11T23:30:40.128+0000 7f416abfa640 20 osd.0 66 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011728525s,-2.011575699s]) 2026-02-11T23:30:40.128+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6814/3043114013 6 ==== osd_ping(ping_reply e64 up_from 10 ping_stamp 2026-02-11T23:30:40.128665+0000/22.804069519s send_stamp 20.792522430s delta_ub -2.011548281s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a480 con 0x4640c00 2026-02-11T23:30:40.128+0000 7f416c3fd640 20 osd.0 66 get_con_osd_cluster to osd.1 from_epoch 66 2026-02-11T23:30:40.128+0000 7f416c3fd640 20 osd.0 66 get_nextmap_reserved map_reservations: {66=1} 2026-02-11T23:30:40.128+0000 7f416c3fd640 20 osd.0 66 release_map epoch: 66 2026-02-11T23:30:40.128+0000 7f416c3fd640 10 osd.0 66 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 66 is not newer than session's projected_epoch of 66 2026-02-11T23:30:40.128+0000 7f416c3fd640 20 osd.0 66 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011735678s,-2.011548281s]) 2026-02-11T23:30:40.128+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6832/2699377081 5 ==== osd_ping(ping_reply e62 up_from 19 ping_stamp 2026-02-11T23:30:40.128665+0000/22.804069519s send_stamp 15.584761620s delta_ub -7.219310284s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a600 con 0x48cd400 2026-02-11T23:30:40.128+0000 7f416abfa640 20 osd.0 66 get_con_osd_cluster to osd.3 from_epoch 66 2026-02-11T23:30:40.128+0000 7f416abfa640 20 osd.0 66 get_nextmap_reserved map_reservations: {66=1} 2026-02-11T23:30:40.128+0000 7f416abfa640 20 osd.0 66 release_map epoch: 66 2026-02-11T23:30:40.128+0000 7f416abfa640 10 osd.0 66 maybe_share_map: con v2:127.0.0.1:6828/2699377081 map epoch 62 -> 66 (shared) 2026-02-11T23:30:40.128+0000 7f416abfa640 10 osd.0 66 send_incremental_map epoch range: (63, 66] to v2:127.0.0.1:6828/2699377081 2026-02-11T23:30:40.128+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6828/2699377081,v1:127.0.0.1:6829/2699377081] -- osd_map(63..66 src has 1..66) -- 0x4a31c00 con 0x48ccc00 2026-02-11T23:30:40.128+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6830/2699377081 5 ==== osd_ping(ping_reply e62 up_from 19 ping_stamp 2026-02-11T23:30:40.128665+0000/22.804069519s send_stamp 15.584738731s delta_ub -7.219332695s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a480 con 0x48cd800 2026-02-11T23:30:40.128+0000 7f416abfa640 20 osd.0 66 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219512939s,-7.219310284s]) 2026-02-11T23:30:40.128+0000 7f416c3fd640 20 osd.0 66 get_con_osd_cluster to osd.3 from_epoch 66 2026-02-11T23:30:40.128+0000 7f416c3fd640 20 osd.0 66 get_nextmap_reserved map_reservations: {66=1} 2026-02-11T23:30:40.128+0000 7f416c3fd640 20 osd.0 66 release_map epoch: 66 2026-02-11T23:30:40.128+0000 7f416c3fd640 10 osd.0 66 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 66 is not newer than session's projected_epoch of 66 2026-02-11T23:30:40.128+0000 7f416c3fd640 20 osd.0 66 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219554424s,-7.219332695s]) 2026-02-11T23:30:40.128+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6822/2365263801 5 ==== osd_ping(ping_reply e68 up_from 14 ping_stamp 2026-02-11T23:30:40.128665+0000/22.804069519s send_stamp 18.237367630s delta_ub -4.566703320s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a600 con 0x48cd000 2026-02-11T23:30:40.128+0000 7f416abfa640 20 osd.0 66 get_con_osd_cluster to osd.2 from_epoch 66 2026-02-11T23:30:40.128+0000 7f416abfa640 20 osd.0 66 get_nextmap_reserved map_reservations: {66=1} 2026-02-11T23:30:40.128+0000 7f416abfa640 20 osd.0 66 release_map epoch: 66 2026-02-11T23:30:40.128+0000 7f416abfa640 10 osd.0 66 maybe_share_map: con v2:127.0.0.1:6820/2365263801 updating session's projected_epoch from 62 to ping map epoch of 68 2026-02-11T23:30:40.128+0000 7f416abfa640 10 osd.0 66 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 66 is not newer than session's projected_epoch of 68 2026-02-11T23:30:40.128+0000 7f416abfa640 20 osd.0 66 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566935539s,-4.566703320s]) 2026-02-11T23:30:40.128+0000 7f415aefe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.2 v2:127.0.0.1:6820/2365263801 3 ==== osd_map(67..68 src has 1..68) ==== 1208+0+0 (crc 0 0 0) 0x5955500 con 0x507b800 2026-02-11T23:30:40.128+0000 7f415aefe640 20 osd.0 66 OSD::ms_dispatch: osd_map(67..68 src has 1..68) 2026-02-11T23:30:40.128+0000 7f416cbfe640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6824/2365263801 5 ==== osd_ping(ping_reply e68 up_from 14 ping_stamp 2026-02-11T23:30:40.128665+0000/22.804069519s send_stamp 18.237470627s delta_ub -4.566601276s) ==== 2033+0+0 (crc 0 0 0) 0x4c9bb00 con 0x48c4000 2026-02-11T23:30:40.128+0000 7f415aefe640 20 osd.0 66 _dispatch 0x5955500 osd_map(67..68 src has 1..68) 2026-02-11T23:30:40.128+0000 7f416cbfe640 20 osd.0 66 get_con_osd_cluster to osd.2 from_epoch 66 2026-02-11T23:30:40.128+0000 7f416cbfe640 20 osd.0 66 get_nextmap_reserved map_reservations: {66=1} 2026-02-11T23:30:40.128+0000 7f416cbfe640 20 osd.0 66 release_map epoch: 66 2026-02-11T23:30:40.128+0000 7f416cbfe640 10 osd.0 66 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 66 is not newer than session's projected_epoch of 68 2026-02-11T23:30:40.128+0000 7f416cbfe640 20 osd.0 66 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566875458s,-4.566601276s]) 2026-02-11T23:30:40.128+0000 7f415aefe640 3 osd.0 66 handle_osd_map epochs [67,68], i have 66, src has [1,68] 2026-02-11T23:30:40.128+0000 7f415aefe640 10 osd.0 66 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:40.128+0000 7f415aefe640 10 osd.0 66 handle_osd_map got inc map for epoch 67 2026-02-11T23:30:40.128+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6840/3331151400 4 ==== osd_ping(ping_reply e67 up_from 23 ping_stamp 2026-02-11T23:30:40.128665+0000/22.804069519s send_stamp 12.967562675s delta_ub -9.836508751s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a600 con 0x507bc00 2026-02-11T23:30:40.128+0000 7f416abfa640 20 osd.0 66 get_con_osd_cluster to osd.4 from_epoch 66 2026-02-11T23:30:40.128+0000 7f416abfa640 20 osd.0 66 get_nextmap_reserved map_reservations: {66=1} 2026-02-11T23:30:40.128+0000 7f416abfa640 20 osd.0 66 release_map epoch: 66 2026-02-11T23:30:40.128+0000 7f416abfa640 10 osd.0 66 maybe_share_map: con v2:127.0.0.1:6836/3331151400 updating session's projected_epoch from 66 to ping map epoch of 67 2026-02-11T23:30:40.128+0000 7f416abfa640 10 osd.0 66 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 66 is not newer than session's projected_epoch of 67 2026-02-11T23:30:40.128+0000 7f415aefe640 20 osd.0 66 got_full_map 67, nothing requested 2026-02-11T23:30:40.128+0000 7f416abfa640 20 osd.0 66 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836813927s,-9.836508751s]) 2026-02-11T23:30:40.128+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6838/3331151400 4 ==== osd_ping(ping_reply e67 up_from 23 ping_stamp 2026-02-11T23:30:40.128665+0000/22.804069519s send_stamp 12.967711449s delta_ub -9.836359978s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a480 con 0x5282000 2026-02-11T23:30:40.128+0000 7f416c3fd640 20 osd.0 66 get_con_osd_cluster to osd.4 from_epoch 66 2026-02-11T23:30:40.128+0000 7f416c3fd640 20 osd.0 66 get_nextmap_reserved map_reservations: {66=1} 2026-02-11T23:30:40.128+0000 7f416c3fd640 20 osd.0 66 release_map epoch: 66 2026-02-11T23:30:40.128+0000 7f415aefe640 10 osd.0 66 handle_osd_map got inc map for epoch 68 2026-02-11T23:30:40.128+0000 7f416c3fd640 10 osd.0 66 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 66 is not newer than session's projected_epoch of 67 2026-02-11T23:30:40.128+0000 7f416c3fd640 20 osd.0 66 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836703300s,-9.836359978s]) 2026-02-11T23:30:40.128+0000 7f415aefe640 20 osd.0 66 got_full_map 68, nothing requested 2026-02-11T23:30:40.128+0000 7f415aefe640 20 osd.0 66 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:40.128+0000 7f415aefe640 20 osd.0 66 handle_osd_map pg_num_history pg_num_history(e68 pg_nums {1={27=1},2={34=1},3={42=1},4={49=1},5={56=1},6={64=1}} deleted_pools ) 2026-02-11T23:30:40.128+0000 7f415aefe640 10 snap_mapper record_purged_snaps purged_snaps {67={},68={}} 2026-02-11T23:30:40.128+0000 7f415aefe640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:40.128+0000 7f415aefe640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e68 maps [1~68] lci=[5,68] tlb=1) 2026-02-11T23:30:40.128+0000 7f415aefe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 20 ==== osd_map(67..67 src has 1..67) ==== 626+0+0 (unknown 0 0 0) 0x4a31880 con 0x507ac00 2026-02-11T23:30:40.128+0000 7f415aefe640 20 osd.0 66 OSD::ms_dispatch: osd_map(67..67 src has 1..67) 2026-02-11T23:30:40.128+0000 7f415aefe640 20 osd.0 66 _dispatch 0x4a31880 osd_map(67..67 src has 1..67) 2026-02-11T23:30:40.128+0000 7f415aefe640 3 osd.0 66 handle_osd_map epochs [67,67], i have 68, src has [1,67] 2026-02-11T23:30:40.128+0000 7f415aefe640 10 osd.0 66 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:30:40.128+0000 7f415aefe640 10 osd.0 66 no new maps here, dropping 2026-02-11T23:30:40.128+0000 7f41591a6640 10 osd.0 66 _committed_osd_maps 67..68 2026-02-11T23:30:40.128+0000 7f41591a6640 10 osd.0 66 advance to epoch 67 (<= last 68 <= newest_map 68) 2026-02-11T23:30:40.128+0000 7f41591a6640 10 osd.0 67 advance to epoch 68 (<= last 68 <= newest_map 68) 2026-02-11T23:30:40.128+0000 7f41591a6640 20 osd.0 68 consume_map version 68 2026-02-11T23:30:40.128+0000 7f41591a6640 20 osd.0 66 await_reserved_maps epoch:68 2026-02-11T23:30:40.128+0000 7f41591a6640 20 osd.0 66 await_reserved_maps done 0 2026-02-11T23:30:40.128+0000 7f41591a6640 20 osd.0 68 consume_map 68 -- publish done 2026-02-11T23:30:40.128+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 3 slots 2026-02-11T23:30:40.128+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:6.0slot:0x3758b00 2026-02-11T23:30:40.128+0000 7f41591a6640 20 osd.0 68 identify_splits_and_merges 6.0 e66 to e68 2026-02-11T23:30:40.128+0000 7f41591a6640 20 osd.0 68 identify_splits_and_merges 6.0 e66 to e68 pg_nums {64=1} 2026-02-11T23:30:40.128+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x37582c0 2026-02-11T23:30:40.128+0000 7f41591a6640 20 osd.0 68 identify_splits_and_merges 4.0 e66 to e68 2026-02-11T23:30:40.128+0000 7f41591a6640 20 osd.0 68 identify_splits_and_merges 4.0 e66 to e68 pg_nums {49=1} 2026-02-11T23:30:40.128+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3700dc0 2026-02-11T23:30:40.128+0000 7f41591a6640 20 osd.0 68 identify_splits_and_merges 1.0 e66 to e68 2026-02-11T23:30:40.128+0000 7f41591a6640 20 osd.0 68 identify_splits_and_merges 1.0 e66 to e68 pg_nums {27=1} 2026-02-11T23:30:40.128+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:40.128+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:40.128+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:40.128+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:40.128+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:40.128+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:40.128+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:40.128+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:40.128+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:40.128+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:40.128+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:40.128+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:40.128+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:40.128+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:40.128+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:40.128+0000 7f41591a6640 20 osd.0 68 prune_pg_created 2026-02-11T23:30:40.128+0000 7f41591a6640 10 osd.0:0.consume_map 68 (was 66) 2026-02-11T23:30:40.128+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 6.0 2026-02-11T23:30:40.128+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 6.0 pending_peering first epoch 67 <= 68, requeueing 2026-02-11T23:30:40.128+0000 7f41591a6640 20 osd.0:0._wake_pg_slot _wake_pg_slot 6.0 to_process <> waiting <> waiting_peering {67=} 2026-02-11T23:30:40.128+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:40.128+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:40.128+0000 7f41591a6640 10 osd.0:1.consume_map 68 (was 66) 2026-02-11T23:30:40.128+0000 7f41591a6640 10 osd.0:2.consume_map 68 (was 66) 2026-02-11T23:30:40.128+0000 7f41591a6640 10 osd.0:3.consume_map 68 (was 66) 2026-02-11T23:30:40.128+0000 7f41591a6640 10 osd.0:4.consume_map 68 (was 66) 2026-02-11T23:30:40.128+0000 7f41591a6640 10 osd.0:5.consume_map 68 (was 66) 2026-02-11T23:30:40.128+0000 7f41591a6640 10 osd.0:6.consume_map 68 (was 66) 2026-02-11T23:30:40.128+0000 7f41591a6640 10 osd.0:7.consume_map 68 (was 66) 2026-02-11T23:30:40.128+0000 7f41591a6640 10 osd.0 68 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:40.128+0000 7f414b98b640 20 osd.0 op_wq(0) _process 6.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:40.128+0000 7f41591a6640 15 osd.0 68 enqueue_peering_evt 6.0 epoch_sent: 68 epoch_requested: 68 NullEvt 2026-02-11T23:30:40.128+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 67 epoch_requested: 67 DeleteStart) class_id 2immediate prio 255 cost 10 e67) queued 2026-02-11T23:30:40.128+0000 7f414b98b640 20 osd.0 op_wq(0) _process 6.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:40.128+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 68 epoch_requested: 68 NullEvt) class_id 2 prio 255 cost 10 e68) 2026-02-11T23:30:40.128+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 67 epoch_requested: 67 DeleteStart) class_id 2immediate prio 255 cost 10 e67) pg 0x4aa0000 2026-02-11T23:30:40.128+0000 7f414b98b640 10 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown NOTIFY mbc={}] handle_advance_map: 67 2026-02-11T23:30:40.128+0000 7f41591a6640 15 osd.0 68 enqueue_peering_evt 4.0 epoch_sent: 68 epoch_requested: 68 NullEvt 2026-02-11T23:30:40.128+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 6.0 66 -> 67 2026-02-11T23:30:40.128+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 68 epoch_requested: 68 NullEvt) class_id 2 prio 255 cost 10 e68) 2026-02-11T23:30:40.128+0000 7f414b98b640 10 osd.0 pg_epoch: 66 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown NOTIFY mbc={}] PeeringState::advance_map handle_advance_map [4,2]/[4,2] -- 4/4 2026-02-11T23:30:40.128+0000 7f414f993640 20 osd.0 op_wq(0) _process 6.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:40.128+0000 7f414b98b640 10 osd.0 pg_epoch: 67 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown NOTIFY mbc={}] state: Started advmap 2026-02-11T23:30:40.128+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 68 epoch_requested: 68 NullEvt) class_id 2immediate prio 255 cost 10 e68) queued 2026-02-11T23:30:40.128+0000 7f414b98b640 10 osd.0 pg_epoch: 67 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown NOTIFY mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:40.128+0000 7f414b98b640 10 osd.0 pg_epoch: 67 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown NOTIFY mbc={}] plpg_on_pool_change 2026-02-11T23:30:40.128+0000 7f414b98b640 20 osd.0 pg_epoch: 67 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown NOTIFY mbc={}] hit_set_clear 2026-02-11T23:30:40.128+0000 7f41591a6640 15 osd.0 68 enqueue_peering_evt 1.0 epoch_sent: 68 epoch_requested: 68 NullEvt 2026-02-11T23:30:40.128+0000 7f414b98b640 20 osd.0 pg_epoch: 67 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown NOTIFY mbc={}] agent_stop 2026-02-11T23:30:40.128+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 68 epoch_requested: 68 NullEvt) class_id 2 prio 255 cost 10 e68) 2026-02-11T23:30:40.128+0000 7f414b98b640 20 osd.0 68 advance_pg new pool opts old pool opts 2026-02-11T23:30:40.128+0000 7f41591a6640 7 osd.0 68 activate_map version 68 2026-02-11T23:30:40.128+0000 7f414b98b640 10 osd.0 pg_epoch: 67 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown NOTIFY mbc={}] handle_advance_map: 68 2026-02-11T23:30:40.128+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 6.0 67 -> 68 2026-02-11T23:30:40.128+0000 7f414b98b640 10 osd.0 pg_epoch: 67 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown NOTIFY mbc={}] PeeringState::advance_map handle_advance_map [4,2]/[4,2] -- 4/4 2026-02-11T23:30:40.128+0000 7f414b98b640 10 osd.0 pg_epoch: 68 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown NOTIFY mbc={}] state: Started advmap 2026-02-11T23:30:40.128+0000 7f414b98b640 10 osd.0 pg_epoch: 68 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown NOTIFY mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:40.128+0000 7f414b98b640 10 osd.0 pg_epoch: 68 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown NOTIFY mbc={}] plpg_on_pool_change 2026-02-11T23:30:40.128+0000 7f4150194640 20 osd.0 68 agent_entry empty queue 2026-02-11T23:30:40.128+0000 7f414b98b640 20 osd.0 pg_epoch: 68 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown NOTIFY mbc={}] hit_set_clear 2026-02-11T23:30:40.128+0000 7f414b98b640 20 osd.0 pg_epoch: 68 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown NOTIFY mbc={}] agent_stop 2026-02-11T23:30:40.128+0000 7f414b98b640 20 osd.0 68 advance_pg new pool opts old pool opts 2026-02-11T23:30:40.128+0000 7f414b98b640 10 osd.0 pg_epoch: 68 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown NOTIFY mbc={}] handle_activate_map: epoch range: 67..68 2026-02-11T23:30:40.128+0000 7f414b98b640 10 osd.0 pg_epoch: 68 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown NOTIFY mbc={}] PeeringState::activate_map 2026-02-11T23:30:40.128+0000 7f414b98b640 20 osd.0 pg_epoch: 68 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown NOTIFY mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 66 while current is 68 2026-02-11T23:30:40.128+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:40.128+0000 7f414b98b640 10 osd.0 pg_epoch: 68 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown NOTIFY mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:30:40.128+0000 7f414b98b640 10 osd.0 pg_epoch: 68 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown NOTIFY mbc={}] do_peering_event: epoch_sent: 67 epoch_requested: 67 DeleteStart 2026-02-11T23:30:40.128+0000 7f414b98b640 5 osd.0 pg_epoch: 68 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown NOTIFY mbc={}] exit Started/Stray 0.388888 6 0.000022 2026-02-11T23:30:40.128+0000 7f414b98b640 5 osd.0 pg_epoch: 68 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown NOTIFY mbc={}] enter Started/ToDelete 2026-02-11T23:30:40.128+0000 7f414b98b640 5 osd.0 pg_epoch: 68 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown NOTIFY mbc={}] enter Started/ToDelete/WaitDeleteReserved 2026-02-11T23:30:40.128+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:40.128+0000 7f414b98b640 20 osd.0 68 get_con_osd_cluster to osd.4 from_epoch 68 2026-02-11T23:30:40.128+0000 7f414b98b640 20 osd.0 68 get_nextmap_reserved map_reservations: {68=1} 2026-02-11T23:30:40.128+0000 7f414b98b640 20 osd.0 68 release_map epoch: 68 2026-02-11T23:30:40.128+0000 7f414b98b640 10 osd.0 68 maybe_share_map: con v2:127.0.0.1:6836/3331151400 map epoch 67 -> 68 (shared) 2026-02-11T23:30:40.128+0000 7f414b98b640 10 osd.0 68 send_incremental_map epoch range: (68, 68] to v2:127.0.0.1:6836/3331151400 2026-02-11T23:30:40.128+0000 7f414b98b640 10 osd.0 68 add_map_inc_bl 68 574 bytes 2026-02-11T23:30:40.128+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- osd_map(68..68 src has 1..68) -- 0x5954380 con 0x507ac00 2026-02-11T23:30:40.128+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_notify2(6.0 6.0 (query:68 sent:68 6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) ([64,65] all_participants={0, 2, 4} intervals=[([64,65] acting={0, 2, 4})])) e68/68) -- 0x3739c00 con 0x507ac00 2026-02-11T23:30:40.128+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:40.128+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 68 epoch_requested: 68 NullEvt) class_id 2immediate prio 255 cost 10 e68) queued 2026-02-11T23:30:40.128+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:40.128+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 68 epoch_requested: 68 NullEvt) class_id 2immediate prio 255 cost 10 e68) pg 0x5978000 2026-02-11T23:30:40.128+0000 7f414f993640 20 osd.0 op_wq(0) _process 6.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:40.128+0000 7f414b98b640 10 osd.0 pg_epoch: 66 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 67 2026-02-11T23:30:40.128+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 68 epoch_requested: 68 NullEvt) class_id 2immediate prio 255 cost 10 e68) pg 0x4aa0000 2026-02-11T23:30:40.128+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 4.0 66 -> 67 2026-02-11T23:30:40.128+0000 7f414b98b640 10 osd.0 pg_epoch: 66 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:40.128+0000 7f414f993640 10 osd.0 pg_epoch: 68 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown NOTIFY mbc={}] do_peering_event: epoch_sent: 68 epoch_requested: 68 NullEvt 2026-02-11T23:30:40.128+0000 7f414b98b640 10 osd.0 pg_epoch: 67 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:40.128+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:40.128+0000 7f414b98b640 10 osd.0 pg_epoch: 67 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:40.128+0000 7f414b98b640 20 osd.0 68 advance_pg new pool opts old pool opts 2026-02-11T23:30:40.128+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:40.128+0000 7f414b98b640 10 osd.0 pg_epoch: 67 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 68 2026-02-11T23:30:40.128+0000 7f4152999640 15 osd.0 68 enqueue_peering_evt 6.0 epoch_sent: 68 epoch_requested: 68 DeleteReserved 2026-02-11T23:30:40.128+0000 7f4152999640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 68 epoch_requested: 68 DeleteReserved) class_id 2 prio 255 cost 10 e68) 2026-02-11T23:30:40.128+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 68 epoch_requested: 68 NullEvt) class_id 2immediate prio 255 cost 10 e68) queued 2026-02-11T23:30:40.128+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:40.128+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 68 epoch_requested: 68 NullEvt) class_id 2immediate prio 255 cost 10 e68) pg 0x5946000 2026-02-11T23:30:40.128+0000 7f414f993640 10 osd.0 pg_epoch: 66 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 67 2026-02-11T23:30:40.128+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 66 -> 67 2026-02-11T23:30:40.128+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 4.0 67 -> 68 2026-02-11T23:30:40.128+0000 7f414f993640 10 osd.0 pg_epoch: 66 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:40.128+0000 7f414b98b640 10 osd.0 pg_epoch: 67 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:40.128+0000 7f414f993640 10 osd.0 pg_epoch: 67 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:40.128+0000 7f414b98b640 10 osd.0 pg_epoch: 68 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:40.128+0000 7f414f993640 10 osd.0 pg_epoch: 67 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:40.128+0000 7f414b98b640 10 osd.0 pg_epoch: 68 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:40.128+0000 7f414b98b640 20 osd.0 68 advance_pg new pool opts old pool opts 2026-02-11T23:30:40.128+0000 7f414f993640 20 osd.0 68 advance_pg new pool opts old pool opts 2026-02-11T23:30:40.128+0000 7f414b98b640 10 osd.0 pg_epoch: 68 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_activate_map: epoch range: 67..68 2026-02-11T23:30:40.128+0000 7f414b98b640 10 osd.0 pg_epoch: 68 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:40.128+0000 7f414b98b640 20 osd.0 pg_epoch: 68 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 53 while current is 68 2026-02-11T23:30:40.128+0000 7f414f993640 10 osd.0 pg_epoch: 67 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 68 2026-02-11T23:30:40.128+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:40.128+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 67 -> 68 2026-02-11T23:30:40.128+0000 7f414b98b640 10 osd.0 pg_epoch: 68 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 68 epoch_requested: 68 NullEvt 2026-02-11T23:30:40.128+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:40.128+0000 7f414b98b640 20 osd.0 op_wq(0) _process 6.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:40.128+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 68 epoch_requested: 68 DeleteReserved) class_id 2immediate prio 255 cost 10 e68) queued 2026-02-11T23:30:40.128+0000 7f414f993640 10 osd.0 pg_epoch: 67 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:40.128+0000 7f414b98b640 20 osd.0 op_wq(0) _process 6.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:40.128+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 68 epoch_requested: 68 DeleteReserved) class_id 2immediate prio 255 cost 10 e68) pg 0x4aa0000 2026-02-11T23:30:40.128+0000 7f414f993640 10 osd.0 pg_epoch: 68 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:40.128+0000 7f414b98b640 10 osd.0 pg_epoch: 68 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown NOTIFY mbc={}] do_peering_event: epoch_sent: 68 epoch_requested: 68 DeleteReserved 2026-02-11T23:30:40.128+0000 7f414f993640 10 osd.0 pg_epoch: 68 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:40.128+0000 7f414f993640 20 osd.0 68 advance_pg new pool opts old pool opts 2026-02-11T23:30:40.128+0000 7f414b98b640 5 osd.0 pg_epoch: 68 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown NOTIFY mbc={}] exit Started/ToDelete/WaitDeleteReserved 0.000116 2 0.000009 2026-02-11T23:30:40.128+0000 7f414f993640 10 osd.0 pg_epoch: 68 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_activate_map: epoch range: 67..68 2026-02-11T23:30:40.128+0000 7f414f993640 10 osd.0 pg_epoch: 68 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:40.128+0000 7f414f993640 20 osd.0 pg_epoch: 68 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 30 while current is 68 2026-02-11T23:30:40.128+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:40.128+0000 7f414b98b640 5 osd.0 pg_epoch: 68 pg[6.0( empty local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown NOTIFY mbc={}] enter Started/ToDelete/Deleting 2026-02-11T23:30:40.128+0000 7f414f993640 10 osd.0 pg_epoch: 68 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 68 epoch_requested: 68 NullEvt 2026-02-11T23:30:40.128+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:40.128+0000 7f414b98b640 10 osd.0 pg_epoch: 68 pg[6.0( empty lb MIN local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 DELETING pi=[64,66)/1 crt=0'0 unknown NOTIFY mbc={}] on_removal 2026-02-11T23:30:40.128+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:40.128+0000 7f414b98b640 10 osd.0 pg_epoch: 68 pg[6.0( empty lb MIN local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 DELETING pi=[64,66)/1 crt=0'0 unknown NOTIFY mbc={}] on_shutdown 2026-02-11T23:30:40.128+0000 7f414b98b640 10 osd.0 pg_epoch: 68 pg[6.0( empty lb MIN local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 DELETING pi=[64,66)/1 crt=0'0 unknown NOTIFY mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:40.128+0000 7f414b98b640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:40.128+0000 7f414b98b640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:40.128+0000 7f414b98b640 10 osd.0 pg_epoch: 68 pg[6.0( empty lb MIN local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 DELETING pi=[64,66)/1 crt=0'0 unknown NOTIFY mbc={}] cancel_copy_ops 2026-02-11T23:30:40.128+0000 7f414b98b640 10 osd.0 pg_epoch: 68 pg[6.0( empty lb MIN local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 DELETING pi=[64,66)/1 crt=0'0 unknown NOTIFY mbc={}] cancel_flush_ops 2026-02-11T23:30:40.128+0000 7f414b98b640 10 osd.0 pg_epoch: 68 pg[6.0( empty lb MIN local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 DELETING pi=[64,66)/1 crt=0'0 unknown NOTIFY mbc={}] cancel_proxy_ops 2026-02-11T23:30:40.128+0000 7f414b98b640 10 osd.0 pg_epoch: 68 pg[6.0( empty lb MIN local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 DELETING pi=[64,66)/1 crt=0'0 unknown NOTIFY mbc={}] cancel_manifest_ops 2026-02-11T23:30:40.128+0000 7f414b98b640 10 osd.0 pg_epoch: 68 pg[6.0( empty lb MIN local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 DELETING pi=[64,66)/1 crt=0'0 unknown NOTIFY mbc={}] cancel_cls_gather_ops 2026-02-11T23:30:40.128+0000 7f414b98b640 10 osd.0 pg_epoch: 68 pg[6.0( empty lb MIN local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 DELETING pi=[64,66)/1 crt=0'0 unknown NOTIFY mbc={}] clear_backoffs 2026-02-11T23:30:40.128+0000 7f414b98b640 20 osd.0 pg_epoch: 68 pg[6.0( empty lb MIN local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 DELETING pi=[64,66)/1 crt=0'0 unknown NOTIFY mbc={}] exit NotTrimming 2026-02-11T23:30:40.128+0000 7f414b98b640 20 osd.0 pg_epoch: 68 pg[6.0( empty lb MIN local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 DELETING pi=[64,66)/1 crt=0'0 unknown NOTIFY mbc={}] enter NotTrimming 2026-02-11T23:30:40.128+0000 7f414b98b640 10 osd.0 pg_epoch: 68 pg[6.0( empty lb MIN local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 DELETING pi=[64,66)/1 crt=0'0 unknown NOTIFY mbc={}] on_change 2026-02-11T23:30:40.128+0000 7f414b98b640 10 osd.0 pg_epoch: 68 pg[6.0( empty lb MIN local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 DELETING pi=[64,66)/1 crt=0'0 unknown NOTIFY mbc={}] clear_async_reads 2026-02-11T23:30:40.128+0000 7f414b98b640 20 osd.0 pg_epoch: 68 pg[6.0( empty lb MIN local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 DELETING pi=[64,66)/1 crt=0'0 unknown NOTIFY mbc={}] clear_primary_state 2026-02-11T23:30:40.128+0000 7f414b98b640 10 osd.0 pg_epoch: 68 pg[6.0( empty lb MIN local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 DELETING pi=[64,66)/1 crt=0'0 unknown NOTIFY mbc={}] release_backoffs [6:00000000::::0,MAX) 2026-02-11T23:30:40.128+0000 7f414b98b640 10 osd.0 pg_epoch: 68 pg[6.0( empty lb MIN local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 DELETING pi=[64,66)/1 crt=0'0 unknown NOTIFY mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:40.128+0000 7f414b98b640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:40.128+0000 7f414b98b640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:40.128+0000 7f414b98b640 20 osd.0 pg_epoch: 68 pg[6.0( empty lb MIN local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 DELETING pi=[64,66)/1 crt=0'0 unknown NOTIFY mbc={}] agent_stop 2026-02-11T23:30:40.128+0000 7f414b98b640 10 osd.0 pg_epoch: 68 pg[6.0( empty lb MIN local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 DELETING pi=[64,66)/1 crt=0'0 unknown NOTIFY mbc={}] cancel_recovery 2026-02-11T23:30:40.128+0000 7f414b98b640 10 osd.0 pg_epoch: 68 pg[6.0( empty lb MIN local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 DELETING pi=[64,66)/1 crt=0'0 unknown NOTIFY mbc={}] clear_recovery_state 2026-02-11T23:30:40.128+0000 7f414b98b640 15 osd.0 pg_epoch: 68 pg[6.0( empty lb MIN local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 DELETING pi=[64,66)/1 crt=0'0 unknown NOTIFY mbc={}] _clear_recovery_state 2026-02-11T23:30:40.128+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:40.128+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:40.128+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:40.128+0000 7f414f993640 10 osd.0 68 queue_for_pg_delete on 6.0 e 68 2026-02-11T23:30:40.128+0000 7f414f993640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(6.0 PGDelete(pgid=6.0 epoch_queued=68) class_id 1 prio 5 cost 1048576 e68) 2026-02-11T23:30:40.128+0000 7f414f993640 20 osd.0 op_wq(0) _process 6.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:40.128+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(6.0 PGDelete(6.0 e68) class_id 1background_best_effort prio 5 qos_cost 1048576 cost 1048576 e68) queued 2026-02-11T23:30:40.128+0000 7f414f993640 20 osd.0 op_wq(0) _process 6.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:40.128+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(6.0 PGDelete(6.0 e68) class_id 1background_best_effort prio 5 qos_cost 1048576 cost 1048576 e68) pg 0x4aa0000 2026-02-11T23:30:40.128+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:40.128+0000 7f414f993640 10 osd.0 pg_epoch: 68 pg[6.0( empty lb MIN local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 DELETING pi=[64,66)/1 crt=0'0 unknown NOTIFY mbc={}] do_peering_event: epoch_sent: 68 epoch_requested: 68 DeleteSome 2026-02-11T23:30:40.128+0000 7f414f993640 10 osd.0 pg_epoch: 68 pg[6.0( empty lb MIN local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 DELETING pi=[64,66)/1 crt=0'0 unknown NOTIFY mbc={}] do_delete_work 2026-02-11T23:30:40.128+0000 7f414f993640 20 osd.0 pg_epoch: 68 pg[6.0( empty lb MIN local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 DELETING pi=[64,66)/1 crt=0'0 unknown NOTIFY mbc={}] do_delete_work [#6:00000000::::head#] 2026-02-11T23:30:40.128+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:40.128+0000 7f414f993640 20 osd.0 68 try_finish_pg_delete 6.0 0x4aa0000 2026-02-11T23:30:40.128+0000 7f414f993640 10 osd.0:0._detach_pg 6.0 0x4aa0000 2026-02-11T23:30:40.128+0000 7f414f993640 5 osd.0 pg_epoch: 68 pg[6.0( empty lb MIN local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 DELETING pi=[64,66)/1 crt=0'0 unknown NOTIFY mbc={}] exit Started/ToDelete/Deleting 0.000232 1 0.000033 2026-02-11T23:30:40.128+0000 7f414f993640 5 osd.0 pg_epoch: 68 pg[6.0( empty lb MIN local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown NOTIFY mbc={}] exit Started/ToDelete 0.000355 0 0.000000 2026-02-11T23:30:40.128+0000 7f414f993640 5 osd.0 pg_epoch: 68 pg[6.0( empty lb MIN local-lis/les=64/65 n=0 ec=64/64 lis/c=0/0 les/c/f=0/0/0 sis=66) [4,2] r=-1 lpr=66 pi=[64,66)/1 crt=0'0 unknown NOTIFY mbc={}] exit Started 0.389253 0 0.000000 2026-02-11T23:30:40.128+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:40.128+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:40.128+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:40.128+0000 7f414f993640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:40.128+0000 7f414f993640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:40.128+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:40.282+0000 7f4162bea640 10 osd.0 68 tick_without_osd_lock 2026-02-11T23:30:40.282+0000 7f4162bea640 0 osd.0 68 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:40.282+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:40.282+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.468 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:40.282+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:40.284, recovery is active?:false restrictions:<....> 2026-02-11T23:30:40.282+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:40.282+0000 7f4162bea640 10 osd.0 68 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:40.282+0000 7f4162bea640 20 osd.0 68 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:40.282+0000 7f4162bea640 10 osd.0 68 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:40.282+0000 7f4162bea640 20 osd.0 68 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:40.326+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:30:40.326+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:30:40.326+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:10.327845+0000) 2026-02-11T23:30:40.326+0000 7f415bf00640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-11T23:30:40.347+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.3 127.0.0.1:0/2699377081 4 ==== osd_ping(ping e66 up_from 19 ping_stamp 2026-02-11T23:30:40.347526+0000/15.803466797s send_stamp 15.803466797s delta_ub -7.219310284s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a480 con 0x48cc400 2026-02-11T23:30:40.347+0000 7f416c3fd640 20 osd.0 68 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219555855s,-7.219310284s]) 2026-02-11T23:30:40.347+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.3 127.0.0.1:0/2699377081 4 ==== osd_ping(ping e66 up_from 19 ping_stamp 2026-02-11T23:30:40.347526+0000/15.803466797s send_stamp 15.803466797s delta_ub -7.219310284s) ==== 2033+0+0 (crc 0 0 0) 0x4c9bb00 con 0x48cc800 2026-02-11T23:30:40.347+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e68 up_from 5 ping_stamp 2026-02-11T23:30:40.347526+0000/15.803466797s send_stamp 23.023021698s delta_ub 7.219555855s) -- 0x4c9b380 con 0x48cc400 2026-02-11T23:30:40.347+0000 7f416c3fd640 20 osd.0 68 get_con_osd_cluster to osd.3 from_epoch 68 2026-02-11T23:30:40.347+0000 7f416c3fd640 20 osd.0 68 get_nextmap_reserved map_reservations: {68=1} 2026-02-11T23:30:40.347+0000 7f416c3fd640 20 osd.0 68 release_map epoch: 68 2026-02-11T23:30:40.347+0000 7f416c3fd640 10 osd.0 68 maybe_share_map: con v2:127.0.0.1:6828/2699377081 map epoch 66 -> 68 (shared) 2026-02-11T23:30:40.347+0000 7f416c3fd640 10 osd.0 68 send_incremental_map epoch range: (67, 68] to v2:127.0.0.1:6828/2699377081 2026-02-11T23:30:40.347+0000 7f416c3fd640 10 osd.0 68 add_map_inc_bl 67 582 bytes 2026-02-11T23:30:40.347+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6828/2699377081,v1:127.0.0.1:6829/2699377081] -- osd_map(67..68 src has 1..68) -- 0x5954c40 con 0x48ccc00 2026-02-11T23:30:40.347+0000 7f416cbfe640 20 osd.0 68 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219720840s,-7.219310284s]) 2026-02-11T23:30:40.347+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e68 up_from 5 ping_stamp 2026-02-11T23:30:40.347526+0000/15.803466797s send_stamp 23.023187637s delta_ub 7.219720840s) -- 0x4c9ad80 con 0x48cc800 2026-02-11T23:30:40.347+0000 7f416cbfe640 20 osd.0 68 get_con_osd_cluster to osd.3 from_epoch 68 2026-02-11T23:30:40.347+0000 7f416cbfe640 20 osd.0 68 get_nextmap_reserved map_reservations: {68=1} 2026-02-11T23:30:40.347+0000 7f416cbfe640 20 osd.0 68 release_map epoch: 68 2026-02-11T23:30:40.347+0000 7f416cbfe640 10 osd.0 68 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 68 is not newer than session's projected_epoch of 68 2026-02-11T23:30:40.348+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 39 ==== osd_map(69..70 src has 1..70) ==== 1217+0+0 (crc 0 0 0) 0x5954c40 con 0x4acbc00 2026-02-11T23:30:40.348+0000 7f415cf02640 20 osd.0 68 OSD::ms_dispatch: osd_map(69..70 src has 1..70) 2026-02-11T23:30:40.348+0000 7f415cf02640 20 osd.0 68 _dispatch 0x5954c40 osd_map(69..70 src has 1..70) 2026-02-11T23:30:40.348+0000 7f415cf02640 3 osd.0 68 handle_osd_map epochs [69,70], i have 68, src has [1,70] 2026-02-11T23:30:40.348+0000 7f415cf02640 10 osd.0 68 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:40.348+0000 7f415cf02640 10 osd.0 68 handle_osd_map got inc map for epoch 69 2026-02-11T23:30:40.348+0000 7f415cf02640 20 osd.0 68 got_full_map 69, nothing requested 2026-02-11T23:30:40.348+0000 7f415cf02640 10 osd.0 68 handle_osd_map got inc map for epoch 70 2026-02-11T23:30:40.348+0000 7f415cf02640 20 osd.0 68 got_full_map 70, nothing requested 2026-02-11T23:30:40.348+0000 7f415cf02640 10 osd.0 68 _track_pools_and_pg_num_changes recording new pool 7 pg_num 1 2026-02-11T23:30:40.348+0000 7f415cf02640 20 osd.0 68 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:40.348+0000 7f415cf02640 20 osd.0 68 handle_osd_map pg_num_history pg_num_history(e70 pg_nums {1={27=1},2={34=1},3={42=1},4={49=1},5={56=1},6={64=1},7={70=1}} deleted_pools ) 2026-02-11T23:30:40.348+0000 7f415cf02640 10 snap_mapper record_purged_snaps purged_snaps {69={},70={}} 2026-02-11T23:30:40.348+0000 7f415cf02640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:40.348+0000 7f415cf02640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e70 maps [1~70] lci=[5,70] tlb=1) 2026-02-11T23:30:40.349+0000 7f41591a6640 10 osd.0 68 _committed_osd_maps 69..70 2026-02-11T23:30:40.349+0000 7f41591a6640 10 osd.0 68 advance to epoch 69 (<= last 70 <= newest_map 70) 2026-02-11T23:30:40.349+0000 7f41591a6640 10 osd.0 69 advance to epoch 70 (<= last 70 <= newest_map 70) 2026-02-11T23:30:40.349+0000 7f41591a6640 20 osd.0 70 consume_map version 70 2026-02-11T23:30:40.349+0000 7f41591a6640 20 osd.0 68 await_reserved_maps epoch:70 2026-02-11T23:30:40.349+0000 7f41591a6640 20 osd.0 68 await_reserved_maps done 0 2026-02-11T23:30:40.349+0000 7f41591a6640 20 osd.0 70 consume_map 70 -- publish done 2026-02-11T23:30:40.349+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 3 slots 2026-02-11T23:30:40.349+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:6.0slot:0x3758b00 2026-02-11T23:30:40.349+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot 6.0 has no pg and waiting_for_split 2026-02-11T23:30:40.349+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x37582c0 2026-02-11T23:30:40.349+0000 7f41591a6640 20 osd.0 70 identify_splits_and_merges 4.0 e68 to e70 2026-02-11T23:30:40.349+0000 7f41591a6640 20 osd.0 70 identify_splits_and_merges 4.0 e68 to e70 pg_nums {49=1} 2026-02-11T23:30:40.349+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3700dc0 2026-02-11T23:30:40.349+0000 7f41591a6640 20 osd.0 70 identify_splits_and_merges 1.0 e68 to e70 2026-02-11T23:30:40.349+0000 7f41591a6640 20 osd.0 70 identify_splits_and_merges 1.0 e68 to e70 pg_nums {27=1} 2026-02-11T23:30:40.349+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:40.349+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:40.349+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:40.349+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:40.349+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:40.349+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:40.349+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:40.349+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:40.349+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:40.349+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:40.349+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:40.349+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:40.349+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:40.349+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:40.349+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:40.349+0000 7f41591a6640 20 osd.0 70 prune_pg_created 2026-02-11T23:30:40.349+0000 7f41591a6640 10 osd.0:0.consume_map 70 (was 68) 2026-02-11T23:30:40.349+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 6.0 2026-02-11T23:30:40.349+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 6.0 empty, pruning 2026-02-11T23:30:40.349+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:40.349+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:40.349+0000 7f41591a6640 10 osd.0:1.consume_map 70 (was 68) 2026-02-11T23:30:40.349+0000 7f41591a6640 10 osd.0:2.consume_map 70 (was 68) 2026-02-11T23:30:40.349+0000 7f41591a6640 10 osd.0:3.consume_map 70 (was 68) 2026-02-11T23:30:40.349+0000 7f41591a6640 10 osd.0:4.consume_map 70 (was 68) 2026-02-11T23:30:40.349+0000 7f41591a6640 10 osd.0:5.consume_map 70 (was 68) 2026-02-11T23:30:40.349+0000 7f41591a6640 10 osd.0:6.consume_map 70 (was 68) 2026-02-11T23:30:40.349+0000 7f41591a6640 10 osd.0:7.consume_map 70 (was 68) 2026-02-11T23:30:40.349+0000 7f41591a6640 10 osd.0 70 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:40.349+0000 7f41591a6640 15 osd.0 70 enqueue_peering_evt 4.0 epoch_sent: 70 epoch_requested: 70 NullEvt 2026-02-11T23:30:40.349+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 70 epoch_requested: 70 NullEvt) class_id 2 prio 255 cost 10 e70) 2026-02-11T23:30:40.349+0000 7f41591a6640 15 osd.0 70 enqueue_peering_evt 1.0 epoch_sent: 70 epoch_requested: 70 NullEvt 2026-02-11T23:30:40.349+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 70 epoch_requested: 70 NullEvt) class_id 2 prio 255 cost 10 e70) 2026-02-11T23:30:40.349+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:40.349+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 70 epoch_requested: 70 NullEvt) class_id 2immediate prio 255 cost 10 e70) queued 2026-02-11T23:30:40.349+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:40.349+0000 7f41591a6640 7 osd.0 70 activate_map version 70 2026-02-11T23:30:40.349+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 70 epoch_requested: 70 NullEvt) class_id 2immediate prio 255 cost 10 e70) pg 0x5978000 2026-02-11T23:30:40.349+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:40.349+0000 7f414b98b640 10 osd.0 pg_epoch: 68 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 69 2026-02-11T23:30:40.349+0000 7f4150194640 20 osd.0 70 agent_entry empty queue 2026-02-11T23:30:40.349+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 70 epoch_requested: 70 NullEvt) class_id 2immediate prio 255 cost 10 e70) queued 2026-02-11T23:30:40.349+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:40.349+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 70 epoch_requested: 70 NullEvt) class_id 2immediate prio 255 cost 10 e70) pg 0x5946000 2026-02-11T23:30:40.349+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 4.0 68 -> 69 2026-02-11T23:30:40.349+0000 7f414f993640 10 osd.0 pg_epoch: 68 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 69 2026-02-11T23:30:40.349+0000 7f414b98b640 10 osd.0 pg_epoch: 68 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:40.349+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 68 -> 69 2026-02-11T23:30:40.349+0000 7f414b98b640 10 osd.0 pg_epoch: 69 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:40.349+0000 7f414f993640 10 osd.0 pg_epoch: 68 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:40.349+0000 7f414b98b640 10 osd.0 pg_epoch: 69 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:40.349+0000 7f414f993640 10 osd.0 pg_epoch: 69 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:40.349+0000 7f414b98b640 20 osd.0 70 advance_pg new pool opts old pool opts 2026-02-11T23:30:40.349+0000 7f414f993640 10 osd.0 pg_epoch: 69 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:40.349+0000 7f414b98b640 10 osd.0 pg_epoch: 69 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 70 2026-02-11T23:30:40.349+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 4.0 69 -> 70 2026-02-11T23:30:40.349+0000 7f414f993640 20 osd.0 70 advance_pg new pool opts old pool opts 2026-02-11T23:30:40.349+0000 7f414b98b640 10 osd.0 pg_epoch: 69 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:40.349+0000 7f414b98b640 10 osd.0 pg_epoch: 70 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:40.349+0000 7f414f993640 10 osd.0 pg_epoch: 69 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 70 2026-02-11T23:30:40.349+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 69 -> 70 2026-02-11T23:30:40.349+0000 7f414b98b640 10 osd.0 pg_epoch: 70 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:40.349+0000 7f414f993640 10 osd.0 pg_epoch: 69 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:40.349+0000 7f414f993640 10 osd.0 pg_epoch: 70 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:40.349+0000 7f414b98b640 20 osd.0 70 advance_pg new pool opts old pool opts 2026-02-11T23:30:40.349+0000 7f414f993640 10 osd.0 pg_epoch: 70 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:40.349+0000 7f414f993640 20 osd.0 70 advance_pg new pool opts old pool opts 2026-02-11T23:30:40.349+0000 7f414b98b640 10 osd.0 pg_epoch: 70 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_activate_map: epoch range: 69..70 2026-02-11T23:30:40.349+0000 7f414f993640 10 osd.0 pg_epoch: 70 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_activate_map: epoch range: 69..70 2026-02-11T23:30:40.349+0000 7f414b98b640 10 osd.0 pg_epoch: 70 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:40.349+0000 7f414f993640 10 osd.0 pg_epoch: 70 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:40.349+0000 7f414f993640 20 osd.0 pg_epoch: 70 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 30 while current is 70 2026-02-11T23:30:40.349+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:40.349+0000 7f414f993640 10 osd.0 pg_epoch: 70 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 70 epoch_requested: 70 NullEvt 2026-02-11T23:30:40.349+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:40.349+0000 7f414b98b640 20 osd.0 pg_epoch: 70 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 53 while current is 70 2026-02-11T23:30:40.349+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:40.349+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:40.349+0000 7f414b98b640 10 osd.0 pg_epoch: 70 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 70 epoch_requested: 70 NullEvt 2026-02-11T23:30:40.349+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:40.349+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:40.356+0000 7f41633eb640 10 osd.0 70 tick 2026-02-11T23:30:40.356+0000 7f41633eb640 20 osd.0 70 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:30:40.540+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.1 127.0.0.1:0/3043114013 10 ==== osd_ping(ping e69 up_from 10 ping_stamp 2026-02-11T23:30:40.540777+0000/21.204523087s send_stamp 21.204523087s delta_ub -2.011548281s) ==== 2033+0+0 (crc 0 0 0) 0x4c9ad80 con 0x4aca000 2026-02-11T23:30:40.540+0000 7f416cbfe640 20 osd.0 70 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011763096s,-2.011548281s]) 2026-02-11T23:30:40.540+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.1 127.0.0.1:0/3043114013 10 ==== osd_ping(ping e69 up_from 10 ping_stamp 2026-02-11T23:30:40.540777+0000/21.204523087s send_stamp 21.204523087s delta_ub -2.011548281s) ==== 2033+0+0 (crc 0 0 0) 0x4c9b380 con 0x4aca400 2026-02-11T23:30:40.540+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e70 up_from 5 ping_stamp 2026-02-11T23:30:40.540777+0000/21.204523087s send_stamp 23.216285706s delta_ub 2.011763096s) -- 0x4c9bb00 con 0x4aca000 2026-02-11T23:30:40.540+0000 7f416cbfe640 20 osd.0 70 get_con_osd_cluster to osd.1 from_epoch 70 2026-02-11T23:30:40.540+0000 7f416cbfe640 20 osd.0 70 get_nextmap_reserved map_reservations: {70=1} 2026-02-11T23:30:40.540+0000 7f416cbfe640 20 osd.0 70 release_map epoch: 70 2026-02-11T23:30:40.540+0000 7f416cbfe640 10 osd.0 70 maybe_share_map: con v2:127.0.0.1:6812/3043114013 updating session's projected_epoch from 66 to ping map epoch of 69 2026-02-11T23:30:40.540+0000 7f416cbfe640 10 osd.0 70 maybe_share_map: con v2:127.0.0.1:6812/3043114013 map epoch 69 -> 70 (shared) 2026-02-11T23:30:40.540+0000 7f416cbfe640 10 osd.0 70 send_incremental_map epoch range: (70, 70] to v2:127.0.0.1:6812/3043114013 2026-02-11T23:30:40.540+0000 7f416cbfe640 10 osd.0 70 add_map_inc_bl 70 591 bytes 2026-02-11T23:30:40.540+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- osd_map(70..70 src has 1..70) -- 0x4a31c00 con 0x4acb000 2026-02-11T23:30:40.540+0000 7f416c3fd640 20 osd.0 70 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011982679s,-2.011548281s]) 2026-02-11T23:30:40.540+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e70 up_from 5 ping_stamp 2026-02-11T23:30:40.540777+0000/21.204523087s send_stamp 23.216505051s delta_ub 2.011982679s) -- 0x4c9a480 con 0x4aca400 2026-02-11T23:30:40.540+0000 7f416c3fd640 20 osd.0 70 get_con_osd_cluster to osd.1 from_epoch 70 2026-02-11T23:30:40.540+0000 7f416c3fd640 20 osd.0 70 get_nextmap_reserved map_reservations: {70=1} 2026-02-11T23:30:40.540+0000 7f416c3fd640 20 osd.0 70 release_map epoch: 70 2026-02-11T23:30:40.540+0000 7f416c3fd640 10 osd.0 70 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 70 is not newer than session's projected_epoch of 70 2026-02-11T23:30:41.290+0000 7f4162bea640 10 osd.0 70 tick_without_osd_lock 2026-02-11T23:30:41.290+0000 7f4162bea640 0 osd.0 70 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:41.290+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:41.290+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.468 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:41.290+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:41.290, recovery is active?:false restrictions:<....> 2026-02-11T23:30:41.290+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:41.290+0000 7f4162bea640 10 osd.0 70 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:41.290+0000 7f4162bea640 20 osd.0 70 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:41.290+0000 7f4162bea640 10 osd.0 70 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:41.290+0000 7f4162bea640 20 osd.0 70 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:41.327+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:30:41.327+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:30:41.327+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:11.327920+0000) 2026-02-11T23:30:41.405+0000 7f41633eb640 10 osd.0 70 tick 2026-02-11T23:30:41.405+0000 7f41633eb640 20 osd.0 70 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:30:41.827+0000 7f4147983640 5 osd.0 70 heartbeat osd_stat(store_statfs(0x384000/0x0/0x384000, data 0x43193/0xa4000, compress 0x0/0x0/0x0, omap 0x7127, meta 0x1a48ed9), peers [1,2,3,4] op hist []) 2026-02-11T23:30:41.827+0000 7f4147983640 20 osd.0 70 check_full_status cur ratio 0, physical ratio 0, new state none 2026-02-11T23:30:41.827+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6816/3043114013,v1:127.0.0.1:6817/3043114013] -- osd_ping(ping e70 up_from 5 ping_stamp 2026-02-11T23:30:41.828881+0000/24.504287720s send_stamp 24.504287720s delta_ub 2.011982679s) -- 0x4c9b200 con 0x48c4400 2026-02-11T23:30:41.827+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6814/3043114013,v1:127.0.0.1:6815/3043114013] -- osd_ping(ping e70 up_from 5 ping_stamp 2026-02-11T23:30:41.828881+0000/24.504287720s send_stamp 24.504287720s delta_ub 2.011982679s) -- 0x4c9a900 con 0x4640c00 2026-02-11T23:30:41.827+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6824/2365263801,v1:127.0.0.1:6825/2365263801] -- osd_ping(ping e70 up_from 5 ping_stamp 2026-02-11T23:30:41.828881+0000/24.504287720s send_stamp 24.504287720s delta_ub 4.566875458s) -- 0x4c9be00 con 0x48c4000 2026-02-11T23:30:41.827+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6822/2365263801,v1:127.0.0.1:6823/2365263801] -- osd_ping(ping e70 up_from 5 ping_stamp 2026-02-11T23:30:41.828881+0000/24.504287720s send_stamp 24.504287720s delta_ub 4.566875458s) -- 0x5af8000 con 0x48cd000 2026-02-11T23:30:41.827+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6832/2699377081,v1:127.0.0.1:6833/2699377081] -- osd_ping(ping e70 up_from 5 ping_stamp 2026-02-11T23:30:41.828881+0000/24.504287720s send_stamp 24.504287720s delta_ub 7.219720840s) -- 0x5af8180 con 0x48cd400 2026-02-11T23:30:41.827+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6830/2699377081,v1:127.0.0.1:6831/2699377081] -- osd_ping(ping e70 up_from 5 ping_stamp 2026-02-11T23:30:41.828881+0000/24.504287720s send_stamp 24.504287720s delta_ub 7.219720840s) -- 0x5af8300 con 0x48cd800 2026-02-11T23:30:41.827+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6840/3331151400,v1:127.0.0.1:6841/3331151400] -- osd_ping(ping e70 up_from 5 ping_stamp 2026-02-11T23:30:41.828881+0000/24.504287720s send_stamp 24.504287720s delta_ub 9.836703300s) -- 0x5af8480 con 0x507bc00 2026-02-11T23:30:41.827+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6838/3331151400,v1:127.0.0.1:6839/3331151400] -- osd_ping(ping e70 up_from 5 ping_stamp 2026-02-11T23:30:41.828881+0000/24.504287720s send_stamp 24.504287720s delta_ub 9.836703300s) -- 0x5af8600 con 0x5282000 2026-02-11T23:30:41.827+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6832/2699377081 6 ==== osd_ping(ping_reply e73 up_from 19 ping_stamp 2026-02-11T23:30:41.828881+0000/24.504287720s send_stamp 17.284961700s delta_ub -7.219324112s) ==== 2033+0+0 (crc 0 0 0) 0x5af8480 con 0x48cd400 2026-02-11T23:30:41.827+0000 7f416abfa640 20 osd.0 70 get_con_osd_cluster to osd.3 from_epoch 70 2026-02-11T23:30:41.827+0000 7f416abfa640 20 osd.0 70 get_nextmap_reserved map_reservations: {70=1} 2026-02-11T23:30:41.827+0000 7f416abfa640 20 osd.0 70 release_map epoch: 70 2026-02-11T23:30:41.827+0000 7f416abfa640 10 osd.0 70 maybe_share_map: con v2:127.0.0.1:6828/2699377081 updating session's projected_epoch from 68 to ping map epoch of 73 2026-02-11T23:30:41.827+0000 7f416abfa640 10 osd.0 70 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 70 is not newer than session's projected_epoch of 73 2026-02-11T23:30:41.827+0000 7f416abfa640 20 osd.0 70 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219553947s,-7.219324112s]) 2026-02-11T23:30:41.827+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6830/2699377081 6 ==== osd_ping(ping_reply e73 up_from 19 ping_stamp 2026-02-11T23:30:41.828881+0000/24.504287720s send_stamp 17.285009384s delta_ub -7.219277382s) ==== 2033+0+0 (crc 0 0 0) 0x5af8600 con 0x48cd800 2026-02-11T23:30:41.827+0000 7f416c3fd640 20 osd.0 70 get_con_osd_cluster to osd.3 from_epoch 70 2026-02-11T23:30:41.827+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6840/3331151400 5 ==== osd_ping(ping_reply e73 up_from 23 ping_stamp 2026-02-11T23:30:41.828881+0000/24.504287720s send_stamp 14.667803764s delta_ub -9.836483955s) ==== 2033+0+0 (crc 0 0 0) 0x5af8480 con 0x507bc00 2026-02-11T23:30:41.827+0000 7f416c3fd640 20 osd.0 70 get_nextmap_reserved map_reservations: {70=1} 2026-02-11T23:30:41.827+0000 7f416c3fd640 20 osd.0 70 release_map epoch: 70 2026-02-11T23:30:41.827+0000 7f416c3fd640 10 osd.0 70 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 70 is not newer than session's projected_epoch of 73 2026-02-11T23:30:41.827+0000 7f415aefe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.3 v2:127.0.0.1:6828/2699377081 21 ==== osd_map(71..73 src has 1..73) ==== 1460+0+0 (crc 0 0 0) 0x59556c0 con 0x48ccc00 2026-02-11T23:30:41.827+0000 7f416c3fd640 20 osd.0 70 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219518185s,-7.219277382s]) 2026-02-11T23:30:41.827+0000 7f415aefe640 20 osd.0 70 OSD::ms_dispatch: osd_map(71..73 src has 1..73) 2026-02-11T23:30:41.827+0000 7f415aefe640 20 osd.0 70 _dispatch 0x59556c0 osd_map(71..73 src has 1..73) 2026-02-11T23:30:41.827+0000 7f416abfa640 20 osd.0 70 get_con_osd_cluster to osd.4 from_epoch 70 2026-02-11T23:30:41.827+0000 7f416abfa640 20 osd.0 70 get_nextmap_reserved map_reservations: {70=1} 2026-02-11T23:30:41.827+0000 7f416abfa640 20 osd.0 70 release_map epoch: 70 2026-02-11T23:30:41.827+0000 7f416abfa640 10 osd.0 70 maybe_share_map: con v2:127.0.0.1:6836/3331151400 updating session's projected_epoch from 68 to ping map epoch of 73 2026-02-11T23:30:41.827+0000 7f416abfa640 10 osd.0 70 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 70 is not newer than session's projected_epoch of 73 2026-02-11T23:30:41.827+0000 7f416abfa640 20 osd.0 70 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836742401s,-9.836483955s]) 2026-02-11T23:30:41.827+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6838/3331151400 5 ==== osd_ping(ping_reply e73 up_from 23 ping_stamp 2026-02-11T23:30:41.828881+0000/24.504287720s send_stamp 14.667847633s delta_ub -9.836438179s) ==== 2033+0+0 (crc 0 0 0) 0x5af8600 con 0x5282000 2026-02-11T23:30:41.827+0000 7f416c3fd640 20 osd.0 70 get_con_osd_cluster to osd.4 from_epoch 70 2026-02-11T23:30:41.827+0000 7f416c3fd640 20 osd.0 70 get_nextmap_reserved map_reservations: {70=1} 2026-02-11T23:30:41.827+0000 7f416c3fd640 20 osd.0 70 release_map epoch: 70 2026-02-11T23:30:41.827+0000 7f416c3fd640 10 osd.0 70 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 70 is not newer than session's projected_epoch of 73 2026-02-11T23:30:41.827+0000 7f416c3fd640 20 osd.0 70 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836707115s,-9.836438179s]) 2026-02-11T23:30:41.827+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6814/3043114013 7 ==== osd_ping(ping_reply e74 up_from 10 ping_stamp 2026-02-11T23:30:41.828881+0000/24.504287720s send_stamp 22.492725372s delta_ub -2.011562109s) ==== 2033+0+0 (crc 0 0 0) 0x5af8600 con 0x4640c00 2026-02-11T23:30:41.827+0000 7f416c3fd640 20 osd.0 70 get_con_osd_cluster to osd.1 from_epoch 70 2026-02-11T23:30:41.827+0000 7f416c3fd640 20 osd.0 70 get_nextmap_reserved map_reservations: {70=1} 2026-02-11T23:30:41.827+0000 7f416c3fd640 20 osd.0 70 release_map epoch: 70 2026-02-11T23:30:41.827+0000 7f416c3fd640 10 osd.0 70 maybe_share_map: con v2:127.0.0.1:6812/3043114013 updating session's projected_epoch from 70 to ping map epoch of 74 2026-02-11T23:30:41.827+0000 7f416c3fd640 10 osd.0 70 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 70 is not newer than session's projected_epoch of 74 2026-02-11T23:30:41.827+0000 7f416c3fd640 20 osd.0 70 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011844873s,-2.011562109s]) 2026-02-11T23:30:41.827+0000 7f415aefe640 3 osd.0 70 handle_osd_map epochs [71,73], i have 70, src has [1,73] 2026-02-11T23:30:41.827+0000 7f415aefe640 10 osd.0 70 handle_osd_map: received 0 full maps and 3 incremental maps 2026-02-11T23:30:41.827+0000 7f415aefe640 10 osd.0 70 handle_osd_map got inc map for epoch 71 2026-02-11T23:30:41.827+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6816/3043114013 7 ==== osd_ping(ping_reply e74 up_from 10 ping_stamp 2026-02-11T23:30:41.828881+0000/24.504287720s send_stamp 22.492887497s delta_ub -2.011399984s) ==== 2033+0+0 (crc 0 0 0) 0x5af8480 con 0x48c4400 2026-02-11T23:30:41.827+0000 7f416abfa640 20 osd.0 70 get_con_osd_cluster to osd.1 from_epoch 70 2026-02-11T23:30:41.827+0000 7f416abfa640 20 osd.0 70 get_nextmap_reserved map_reservations: {70=1} 2026-02-11T23:30:41.827+0000 7f416abfa640 20 osd.0 70 release_map epoch: 70 2026-02-11T23:30:41.827+0000 7f416abfa640 10 osd.0 70 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 70 is not newer than session's projected_epoch of 74 2026-02-11T23:30:41.827+0000 7f416abfa640 20 osd.0 70 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011717319s,-2.011399984s]) 2026-02-11T23:30:41.827+0000 7f415aefe640 20 osd.0 70 got_full_map 71, nothing requested 2026-02-11T23:30:41.827+0000 7f415aefe640 10 osd.0 70 handle_osd_map got inc map for epoch 72 2026-02-11T23:30:41.827+0000 7f415aefe640 20 osd.0 70 got_full_map 72, nothing requested 2026-02-11T23:30:41.827+0000 7f415aefe640 10 osd.0 70 handle_osd_map got inc map for epoch 73 2026-02-11T23:30:41.827+0000 7f415aefe640 20 osd.0 70 got_full_map 73, nothing requested 2026-02-11T23:30:41.828+0000 7f415aefe640 20 osd.0 70 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:41.828+0000 7f415aefe640 20 osd.0 70 handle_osd_map pg_num_history pg_num_history(e73 pg_nums {1={27=1},2={34=1},3={42=1},4={49=1},5={56=1},6={64=1},7={70=1}} deleted_pools ) 2026-02-11T23:30:41.828+0000 7f415aefe640 10 snap_mapper record_purged_snaps purged_snaps {71={},72={},73={}} 2026-02-11T23:30:41.828+0000 7f415aefe640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:41.828+0000 7f415aefe640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e73 maps [1~73] lci=[5,73] tlb=1) 2026-02-11T23:30:41.828+0000 7f415aefe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 21 ==== osd_map(71..73 src has 1..73) ==== 1460+0+0 (unknown 0 0 0) 0x4a31c00 con 0x507ac00 2026-02-11T23:30:41.828+0000 7f415aefe640 20 osd.0 70 OSD::ms_dispatch: osd_map(71..73 src has 1..73) 2026-02-11T23:30:41.828+0000 7f415aefe640 20 osd.0 70 _dispatch 0x4a31c00 osd_map(71..73 src has 1..73) 2026-02-11T23:30:41.828+0000 7f415aefe640 3 osd.0 70 handle_osd_map epochs [71,73], i have 73, src has [1,73] 2026-02-11T23:30:41.828+0000 7f415aefe640 10 osd.0 70 handle_osd_map: received 0 full maps and 3 incremental maps 2026-02-11T23:30:41.828+0000 7f415aefe640 10 osd.0 70 no new maps here, dropping 2026-02-11T23:30:41.828+0000 7f415aefe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 18 ==== osd_map(71..74 src has 1..74) ==== 2050+0+0 (unknown 0 0 0) 0x5954380 con 0x4acb000 2026-02-11T23:30:41.828+0000 7f415aefe640 20 osd.0 70 OSD::ms_dispatch: osd_map(71..74 src has 1..74) 2026-02-11T23:30:41.828+0000 7f415aefe640 20 osd.0 70 _dispatch 0x5954380 osd_map(71..74 src has 1..74) 2026-02-11T23:30:41.828+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6822/2365263801 6 ==== osd_ping(ping_reply e74 up_from 14 ping_stamp 2026-02-11T23:30:41.828881+0000/24.504287720s send_stamp 19.937585831s delta_ub -4.566700935s) ==== 2033+0+0 (crc 0 0 0) 0x5af8480 con 0x48cd000 2026-02-11T23:30:41.828+0000 7f416abfa640 20 osd.0 70 get_con_osd_cluster to osd.2 from_epoch 70 2026-02-11T23:30:41.828+0000 7f416abfa640 20 osd.0 70 get_nextmap_reserved map_reservations: {70=1} 2026-02-11T23:30:41.828+0000 7f416abfa640 20 osd.0 70 release_map epoch: 70 2026-02-11T23:30:41.828+0000 7f416abfa640 10 osd.0 70 maybe_share_map: con v2:127.0.0.1:6820/2365263801 updating session's projected_epoch from 68 to ping map epoch of 74 2026-02-11T23:30:41.828+0000 7f416abfa640 10 osd.0 70 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 70 is not newer than session's projected_epoch of 74 2026-02-11T23:30:41.828+0000 7f416abfa640 20 osd.0 70 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.567368031s,-4.566700935s]) 2026-02-11T23:30:41.828+0000 7f415aefe640 3 osd.0 70 handle_osd_map epochs [71,74], i have 73, src has [1,74] 2026-02-11T23:30:41.828+0000 7f415aefe640 10 osd.0 70 handle_osd_map: received 0 full maps and 4 incremental maps 2026-02-11T23:30:41.828+0000 7f415aefe640 10 osd.0 70 handle_osd_map got inc map for epoch 74 2026-02-11T23:30:41.828+0000 7f415aefe640 20 osd.0 70 got_full_map 74, nothing requested 2026-02-11T23:30:41.828+0000 7f416cbfe640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6824/2365263801 6 ==== osd_ping(ping_reply e74 up_from 14 ping_stamp 2026-02-11T23:30:41.828881+0000/24.504287720s send_stamp 19.938127518s delta_ub -4.566157818s) ==== 2033+0+0 (crc 0 0 0) 0x4c9be00 con 0x48c4000 2026-02-11T23:30:41.828+0000 7f416cbfe640 20 osd.0 70 get_con_osd_cluster to osd.2 from_epoch 70 2026-02-11T23:30:41.828+0000 7f416cbfe640 20 osd.0 70 get_nextmap_reserved map_reservations: {70=1} 2026-02-11T23:30:41.828+0000 7f416cbfe640 20 osd.0 70 release_map epoch: 70 2026-02-11T23:30:41.828+0000 7f416cbfe640 10 osd.0 70 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 70 is not newer than session's projected_epoch of 74 2026-02-11T23:30:41.828+0000 7f415aefe640 20 osd.0 70 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:41.828+0000 7f416cbfe640 20 osd.0 70 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566858292s,-4.566157818s]) 2026-02-11T23:30:41.828+0000 7f415aefe640 20 osd.0 70 handle_osd_map pg_num_history pg_num_history(e74 pg_nums {1={27=1},2={34=1},3={42=1},4={49=1},5={56=1},6={64=1},7={70=1}} deleted_pools ) 2026-02-11T23:30:41.828+0000 7f415aefe640 10 snap_mapper record_purged_snaps purged_snaps {74={}} 2026-02-11T23:30:41.828+0000 7f415aefe640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:41.828+0000 7f415aefe640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e74 maps [1~74] lci=[5,74] tlb=1) 2026-02-11T23:30:41.828+0000 7f415aefe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.2 v2:127.0.0.1:6820/2365263801 4 ==== osd_map(71..74 src has 1..74) ==== 2050+0+0 (crc 0 0 0) 0x5955880 con 0x507b800 2026-02-11T23:30:41.828+0000 7f415aefe640 20 osd.0 70 OSD::ms_dispatch: osd_map(71..74 src has 1..74) 2026-02-11T23:30:41.828+0000 7f415aefe640 20 osd.0 70 _dispatch 0x5955880 osd_map(71..74 src has 1..74) 2026-02-11T23:30:41.828+0000 7f415aefe640 3 osd.0 70 handle_osd_map epochs [71,74], i have 74, src has [1,74] 2026-02-11T23:30:41.828+0000 7f415aefe640 10 osd.0 70 handle_osd_map: received 0 full maps and 4 incremental maps 2026-02-11T23:30:41.828+0000 7f415aefe640 10 osd.0 70 no new maps here, dropping 2026-02-11T23:30:41.828+0000 7f41591a6640 10 osd.0 70 _committed_osd_maps 71..73 2026-02-11T23:30:41.828+0000 7f41591a6640 10 osd.0 70 advance to epoch 71 (<= last 73 <= newest_map 74) 2026-02-11T23:30:41.828+0000 7f41591a6640 10 osd.0 71 advance to epoch 72 (<= last 73 <= newest_map 74) 2026-02-11T23:30:41.828+0000 7f41591a6640 10 osd.0 72 advance to epoch 73 (<= last 73 <= newest_map 74) 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0 73 consume_map version 73 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0 70 await_reserved_maps epoch:73 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0 70 await_reserved_maps done 0 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0 73 consume_map 73 -- publish done 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x37582c0 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0 73 identify_splits_and_merges 4.0 e70 to e73 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0 73 identify_splits_and_merges 4.0 e70 to e73 pg_nums {49=1} 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3700dc0 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0 73 identify_splits_and_merges 1.0 e70 to e73 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0 73 identify_splits_and_merges 1.0 e70 to e73 pg_nums {27=1} 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0 73 prune_pg_created 2026-02-11T23:30:41.828+0000 7f41591a6640 10 osd.0:0.consume_map 73 (was 70) 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:41.828+0000 7f41591a6640 10 osd.0:1.consume_map 73 (was 70) 2026-02-11T23:30:41.828+0000 7f41591a6640 10 osd.0:2.consume_map 73 (was 70) 2026-02-11T23:30:41.828+0000 7f41591a6640 10 osd.0:3.consume_map 73 (was 70) 2026-02-11T23:30:41.828+0000 7f41591a6640 10 osd.0:4.consume_map 73 (was 70) 2026-02-11T23:30:41.828+0000 7f41591a6640 10 osd.0:5.consume_map 73 (was 70) 2026-02-11T23:30:41.828+0000 7f41591a6640 10 osd.0:6.consume_map 73 (was 70) 2026-02-11T23:30:41.828+0000 7f41591a6640 10 osd.0:7.consume_map 73 (was 70) 2026-02-11T23:30:41.828+0000 7f41591a6640 10 osd.0 73 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:41.828+0000 7f41591a6640 15 osd.0 73 enqueue_peering_evt 4.0 epoch_sent: 73 epoch_requested: 73 NullEvt 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 73 epoch_requested: 73 NullEvt) class_id 2 prio 255 cost 10 e73) 2026-02-11T23:30:41.828+0000 7f41591a6640 15 osd.0 73 enqueue_peering_evt 1.0 epoch_sent: 73 epoch_requested: 73 NullEvt 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 73 epoch_requested: 73 NullEvt) class_id 2 prio 255 cost 10 e73) 2026-02-11T23:30:41.828+0000 7f41591a6640 7 osd.0 73 activate_map version 73 2026-02-11T23:30:41.828+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:41.828+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 73 epoch_requested: 73 NullEvt) class_id 2immediate prio 255 cost 10 e73) queued 2026-02-11T23:30:41.828+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:41.828+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 73 epoch_requested: 73 NullEvt) class_id 2immediate prio 255 cost 10 e73) pg 0x5978000 2026-02-11T23:30:41.828+0000 7f4150194640 20 osd.0 73 agent_entry empty queue 2026-02-11T23:30:41.828+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:41.828+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 73 epoch_requested: 73 NullEvt) class_id 2immediate prio 255 cost 10 e73) queued 2026-02-11T23:30:41.828+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:41.828+0000 7f414b98b640 10 osd.0 pg_epoch: 70 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 71 2026-02-11T23:30:41.828+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 73 epoch_requested: 73 NullEvt) class_id 2immediate prio 255 cost 10 e73) pg 0x5946000 2026-02-11T23:30:41.828+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 4.0 70 -> 71 2026-02-11T23:30:41.828+0000 7f414b98b640 10 osd.0 pg_epoch: 70 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:41.828+0000 7f414f993640 10 osd.0 pg_epoch: 70 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 71 2026-02-11T23:30:41.828+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 70 -> 71 2026-02-11T23:30:41.828+0000 7f414b98b640 10 osd.0 pg_epoch: 71 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:41.828+0000 7f414f993640 10 osd.0 pg_epoch: 70 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:41.828+0000 7f414b98b640 10 osd.0 pg_epoch: 71 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:41.828+0000 7f414f993640 10 osd.0 pg_epoch: 71 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:41.828+0000 7f414b98b640 20 osd.0 73 advance_pg new pool opts old pool opts 2026-02-11T23:30:41.828+0000 7f414f993640 10 osd.0 pg_epoch: 71 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:41.828+0000 7f414b98b640 10 osd.0 pg_epoch: 71 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 72 2026-02-11T23:30:41.828+0000 7f414f993640 20 osd.0 73 advance_pg new pool opts old pool opts 2026-02-11T23:30:41.828+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 4.0 71 -> 72 2026-02-11T23:30:41.828+0000 7f414b98b640 10 osd.0 pg_epoch: 71 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:41.828+0000 7f414f993640 10 osd.0 pg_epoch: 71 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 72 2026-02-11T23:30:41.828+0000 7f414b98b640 10 osd.0 pg_epoch: 72 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:41.828+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 71 -> 72 2026-02-11T23:30:41.828+0000 7f414b98b640 10 osd.0 pg_epoch: 72 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:41.828+0000 7f414f993640 10 osd.0 pg_epoch: 71 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:41.828+0000 7f414f993640 10 osd.0 pg_epoch: 72 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:41.828+0000 7f414b98b640 20 osd.0 73 advance_pg new pool opts old pool opts 2026-02-11T23:30:41.828+0000 7f414f993640 10 osd.0 pg_epoch: 72 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:41.828+0000 7f414f993640 20 osd.0 73 advance_pg new pool opts old pool opts 2026-02-11T23:30:41.828+0000 7f414b98b640 10 osd.0 pg_epoch: 72 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 73 2026-02-11T23:30:41.828+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 4.0 72 -> 73 2026-02-11T23:30:41.828+0000 7f414f993640 10 osd.0 pg_epoch: 72 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 73 2026-02-11T23:30:41.828+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 72 -> 73 2026-02-11T23:30:41.828+0000 7f414b98b640 10 osd.0 pg_epoch: 72 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:41.828+0000 7f414f993640 10 osd.0 pg_epoch: 72 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:41.828+0000 7f414b98b640 10 osd.0 pg_epoch: 73 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:41.828+0000 7f414f993640 10 osd.0 pg_epoch: 73 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:41.828+0000 7f414b98b640 10 osd.0 pg_epoch: 73 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:41.828+0000 7f414f993640 10 osd.0 pg_epoch: 73 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:41.828+0000 7f414f993640 20 osd.0 73 advance_pg new pool opts old pool opts 2026-02-11T23:30:41.828+0000 7f414b98b640 20 osd.0 73 advance_pg new pool opts old pool opts 2026-02-11T23:30:41.828+0000 7f414f993640 10 osd.0 pg_epoch: 73 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_activate_map: epoch range: 71..73 2026-02-11T23:30:41.828+0000 7f41591a6640 10 osd.0 73 _committed_osd_maps 74..74 2026-02-11T23:30:41.828+0000 7f414b98b640 10 osd.0 pg_epoch: 73 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_activate_map: epoch range: 71..73 2026-02-11T23:30:41.828+0000 7f414f993640 10 osd.0 pg_epoch: 73 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:41.828+0000 7f414b98b640 10 osd.0 pg_epoch: 73 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:41.828+0000 7f414b98b640 20 osd.0 pg_epoch: 73 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 53 while current is 73 2026-02-11T23:30:41.828+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:41.828+0000 7f414f993640 20 osd.0 pg_epoch: 73 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map : Dirtying info: last_persisted is 30 while current is 73 2026-02-11T23:30:41.828+0000 7f41591a6640 10 osd.0 73 advance to epoch 74 (<= last 74 <= newest_map 74) 2026-02-11T23:30:41.828+0000 7f414b98b640 10 osd.0 pg_epoch: 73 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 73 epoch_requested: 73 NullEvt 2026-02-11T23:30:41.828+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0 74 consume_map version 74 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0 73 await_reserved_maps epoch:74 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0 73 await_reserved_maps done 0 2026-02-11T23:30:41.828+0000 7f414f993640 10 osd.0 pg_epoch: 73 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 73 epoch_requested: 73 NullEvt 2026-02-11T23:30:41.828+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0 74 consume_map 74 -- publish done 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:30:41.828+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x37582c0 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0 74 identify_splits_and_merges 4.0 e73 to e74 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0 74 identify_splits_and_merges 4.0 e73 to e74 pg_nums {49=1} 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3700dc0 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0 74 identify_splits_and_merges 1.0 e73 to e74 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0 74 identify_splits_and_merges 1.0 e73 to e74 pg_nums {27=1} 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:41.828+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:41.828+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0 74 prune_pg_created 2026-02-11T23:30:41.828+0000 7f41591a6640 10 osd.0:0.consume_map 74 (was 73) 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:41.828+0000 7f41591a6640 10 osd.0:1.consume_map 74 (was 73) 2026-02-11T23:30:41.828+0000 7f41591a6640 10 osd.0:2.consume_map 74 (was 73) 2026-02-11T23:30:41.828+0000 7f41591a6640 10 osd.0:3.consume_map 74 (was 73) 2026-02-11T23:30:41.828+0000 7f41591a6640 10 osd.0:4.consume_map 74 (was 73) 2026-02-11T23:30:41.828+0000 7f41591a6640 10 osd.0:5.consume_map 74 (was 73) 2026-02-11T23:30:41.828+0000 7f41591a6640 10 osd.0:6.consume_map 74 (was 73) 2026-02-11T23:30:41.828+0000 7f41591a6640 10 osd.0:7.consume_map 74 (was 73) 2026-02-11T23:30:41.828+0000 7f41591a6640 10 osd.0 74 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:41.828+0000 7f41591a6640 15 osd.0 74 enqueue_peering_evt 4.0 epoch_sent: 74 epoch_requested: 74 NullEvt 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 74 epoch_requested: 74 NullEvt) class_id 2 prio 255 cost 10 e74) 2026-02-11T23:30:41.828+0000 7f41591a6640 15 osd.0 74 enqueue_peering_evt 1.0 epoch_sent: 74 epoch_requested: 74 NullEvt 2026-02-11T23:30:41.828+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 74 epoch_requested: 74 NullEvt) class_id 2 prio 255 cost 10 e74) 2026-02-11T23:30:41.828+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:41.828+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 74 epoch_requested: 74 NullEvt) class_id 2immediate prio 255 cost 10 e74) queued 2026-02-11T23:30:41.828+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:41.828+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 74 epoch_requested: 74 NullEvt) class_id 2immediate prio 255 cost 10 e74) pg 0x5978000 2026-02-11T23:30:41.828+0000 7f414b98b640 10 osd.0 pg_epoch: 73 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 74 2026-02-11T23:30:41.828+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 4.0 73 -> 74 2026-02-11T23:30:41.828+0000 7f41591a6640 7 osd.0 74 activate_map version 74 2026-02-11T23:30:41.828+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:41.828+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 74 epoch_requested: 74 NullEvt) class_id 2immediate prio 255 cost 10 e74) queued 2026-02-11T23:30:41.828+0000 7f414b98b640 10 osd.0 pg_epoch: 73 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:41.828+0000 7f4150194640 20 osd.0 74 agent_entry empty queue 2026-02-11T23:30:41.828+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:41.828+0000 7f414b98b640 10 osd.0 pg_epoch: 74 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:41.828+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 74 epoch_requested: 74 NullEvt) class_id 2immediate prio 255 cost 10 e74) pg 0x5946000 2026-02-11T23:30:41.828+0000 7f414b98b640 10 osd.0 pg_epoch: 74 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:41.828+0000 7f414b98b640 20 osd.0 74 advance_pg new pool opts old pool opts 2026-02-11T23:30:41.828+0000 7f414f993640 10 osd.0 pg_epoch: 73 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 74 2026-02-11T23:30:41.828+0000 7f414b98b640 10 osd.0 pg_epoch: 74 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_activate_map: epoch range: 74..74 2026-02-11T23:30:41.828+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 73 -> 74 2026-02-11T23:30:41.828+0000 7f414b98b640 10 osd.0 pg_epoch: 74 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:41.828+0000 7f414f993640 10 osd.0 pg_epoch: 73 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:41.828+0000 7f414b98b640 20 osd.0 pg_epoch: 74 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 53 while current is 74 2026-02-11T23:30:41.828+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:41.828+0000 7f414f993640 10 osd.0 pg_epoch: 74 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:41.828+0000 7f414b98b640 10 osd.0 pg_epoch: 74 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 74 epoch_requested: 74 NullEvt 2026-02-11T23:30:41.828+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:41.828+0000 7f414f993640 10 osd.0 pg_epoch: 74 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:41.828+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:41.828+0000 7f414f993640 20 osd.0 74 advance_pg new pool opts old pool opts 2026-02-11T23:30:41.828+0000 7f414f993640 10 osd.0 pg_epoch: 74 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_activate_map: epoch range: 74..74 2026-02-11T23:30:41.828+0000 7f414f993640 10 osd.0 pg_epoch: 74 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:41.828+0000 7f414f993640 20 osd.0 pg_epoch: 74 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 73 while current is 74 2026-02-11T23:30:41.828+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:41.828+0000 7f414f993640 10 osd.0 pg_epoch: 74 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 74 epoch_requested: 74 NullEvt 2026-02-11T23:30:41.828+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:41.828+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:41.828+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:41.828+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:41.894+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.2 127.0.0.1:0/2365263801 4 ==== osd_ping(ping e74 up_from 14 ping_stamp 2026-02-11T23:30:41.894632+0000/20.003229141s send_stamp 20.003229141s delta_ub -4.566157818s) ==== 2033+0+0 (crc 0 0 0) 0x4c9be00 con 0x4641000 2026-02-11T23:30:41.894+0000 7f416cbfe640 20 osd.0 74 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566859245s,-4.566157818s]) 2026-02-11T23:30:41.894+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e74 up_from 5 ping_stamp 2026-02-11T23:30:41.894632+0000/20.003229141s send_stamp 24.570087433s delta_ub 4.566859245s) -- 0x4c9bb00 con 0x4641000 2026-02-11T23:30:41.894+0000 7f416cbfe640 20 osd.0 74 get_con_osd_cluster to osd.2 from_epoch 74 2026-02-11T23:30:41.894+0000 7f416cbfe640 20 osd.0 74 get_nextmap_reserved map_reservations: {74=1} 2026-02-11T23:30:41.894+0000 7f416cbfe640 20 osd.0 74 release_map epoch: 74 2026-02-11T23:30:41.894+0000 7f416cbfe640 10 osd.0 74 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 74 is not newer than session's projected_epoch of 74 2026-02-11T23:30:41.894+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.2 127.0.0.1:0/2365263801 4 ==== osd_ping(ping e74 up_from 14 ping_stamp 2026-02-11T23:30:41.894632+0000/20.003229141s send_stamp 20.003229141s delta_ub -4.566157818s) ==== 2033+0+0 (crc 0 0 0) 0x5af8600 con 0x48cc000 2026-02-11T23:30:41.894+0000 7f416c3fd640 20 osd.0 74 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566880703s,-4.566157818s]) 2026-02-11T23:30:41.894+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e74 up_from 5 ping_stamp 2026-02-11T23:30:41.894632+0000/20.003229141s send_stamp 24.570110321s delta_ub 4.566880703s) -- 0x4c9aa80 con 0x48cc000 2026-02-11T23:30:41.894+0000 7f416c3fd640 20 osd.0 74 get_con_osd_cluster to osd.2 from_epoch 74 2026-02-11T23:30:41.894+0000 7f416c3fd640 20 osd.0 74 get_nextmap_reserved map_reservations: {74=1} 2026-02-11T23:30:41.894+0000 7f416c3fd640 20 osd.0 74 release_map epoch: 74 2026-02-11T23:30:41.894+0000 7f416c3fd640 10 osd.0 74 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 74 is not newer than session's projected_epoch of 74 2026-02-11T23:30:42.015+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 40 ==== osd_map(75..76 src has 1..76) ==== 1200+0+0 (crc 0 0 0) 0x5955a40 con 0x4acbc00 2026-02-11T23:30:42.015+0000 7f415cf02640 20 osd.0 74 OSD::ms_dispatch: osd_map(75..76 src has 1..76) 2026-02-11T23:30:42.015+0000 7f415cf02640 20 osd.0 74 _dispatch 0x5955a40 osd_map(75..76 src has 1..76) 2026-02-11T23:30:42.015+0000 7f415cf02640 3 osd.0 74 handle_osd_map epochs [75,76], i have 74, src has [1,76] 2026-02-11T23:30:42.015+0000 7f415cf02640 10 osd.0 74 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:42.015+0000 7f415cf02640 10 osd.0 74 handle_osd_map got inc map for epoch 75 2026-02-11T23:30:42.015+0000 7f415cf02640 20 osd.0 74 got_full_map 75, nothing requested 2026-02-11T23:30:42.015+0000 7f415cf02640 10 osd.0 74 handle_osd_map got inc map for epoch 76 2026-02-11T23:30:42.015+0000 7f415cf02640 20 osd.0 74 got_full_map 76, nothing requested 2026-02-11T23:30:42.015+0000 7f415cf02640 20 osd.0 74 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:42.015+0000 7f415cf02640 20 osd.0 74 handle_osd_map pg_num_history pg_num_history(e76 pg_nums {1={27=1},2={34=1},3={42=1},4={49=1},5={56=1},6={64=1},7={70=1}} deleted_pools ) 2026-02-11T23:30:42.015+0000 7f415cf02640 10 snap_mapper record_purged_snaps purged_snaps {75={},76={}} 2026-02-11T23:30:42.015+0000 7f415cf02640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:42.015+0000 7f415cf02640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e76 maps [1~76] lci=[5,76] tlb=1) 2026-02-11T23:30:42.015+0000 7f41591a6640 10 osd.0 74 _committed_osd_maps 75..76 2026-02-11T23:30:42.015+0000 7f41591a6640 10 osd.0 74 advance to epoch 75 (<= last 76 <= newest_map 76) 2026-02-11T23:30:42.015+0000 7f41591a6640 10 osd.0 75 advance to epoch 76 (<= last 76 <= newest_map 76) 2026-02-11T23:30:42.015+0000 7f41591a6640 20 osd.0 76 consume_map version 76 2026-02-11T23:30:42.015+0000 7f41591a6640 20 osd.0 74 await_reserved_maps epoch:76 2026-02-11T23:30:42.015+0000 7f41591a6640 20 osd.0 74 await_reserved_maps done 0 2026-02-11T23:30:42.015+0000 7f41591a6640 20 osd.0 76 consume_map 76 -- publish done 2026-02-11T23:30:42.015+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:30:42.015+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x37582c0 2026-02-11T23:30:42.015+0000 7f41591a6640 20 osd.0 76 identify_splits_and_merges 4.0 e74 to e76 2026-02-11T23:30:42.015+0000 7f41591a6640 20 osd.0 76 identify_splits_and_merges 4.0 e74 to e76 pg_nums {49=1} 2026-02-11T23:30:42.015+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3700dc0 2026-02-11T23:30:42.015+0000 7f41591a6640 20 osd.0 76 identify_splits_and_merges 1.0 e74 to e76 2026-02-11T23:30:42.015+0000 7f41591a6640 20 osd.0 76 identify_splits_and_merges 1.0 e74 to e76 pg_nums {27=1} 2026-02-11T23:30:42.015+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:42.015+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:42.015+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:42.015+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:42.015+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:42.015+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:42.015+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:42.015+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:42.015+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:42.015+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:42.015+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:42.015+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:42.015+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:42.015+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:42.015+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:42.015+0000 7f41591a6640 20 osd.0 76 prune_pg_created 2026-02-11T23:30:42.015+0000 7f41591a6640 10 osd.0:0.consume_map 76 (was 74) 2026-02-11T23:30:42.015+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:42.015+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:42.015+0000 7f41591a6640 10 osd.0:1.consume_map 76 (was 74) 2026-02-11T23:30:42.015+0000 7f41591a6640 10 osd.0:2.consume_map 76 (was 74) 2026-02-11T23:30:42.015+0000 7f41591a6640 10 osd.0:3.consume_map 76 (was 74) 2026-02-11T23:30:42.015+0000 7f41591a6640 10 osd.0:4.consume_map 76 (was 74) 2026-02-11T23:30:42.015+0000 7f41591a6640 10 osd.0:5.consume_map 76 (was 74) 2026-02-11T23:30:42.015+0000 7f41591a6640 10 osd.0:6.consume_map 76 (was 74) 2026-02-11T23:30:42.015+0000 7f41591a6640 10 osd.0:7.consume_map 76 (was 74) 2026-02-11T23:30:42.015+0000 7f41591a6640 10 osd.0 76 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:42.015+0000 7f41591a6640 15 osd.0 76 enqueue_peering_evt 4.0 epoch_sent: 76 epoch_requested: 76 NullEvt 2026-02-11T23:30:42.015+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 76 epoch_requested: 76 NullEvt) class_id 2 prio 255 cost 10 e76) 2026-02-11T23:30:42.015+0000 7f41591a6640 15 osd.0 76 enqueue_peering_evt 1.0 epoch_sent: 76 epoch_requested: 76 NullEvt 2026-02-11T23:30:42.015+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 76 epoch_requested: 76 NullEvt) class_id 2 prio 255 cost 10 e76) 2026-02-11T23:30:42.015+0000 7f41591a6640 7 osd.0 76 activate_map version 76 2026-02-11T23:30:42.015+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:42.015+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 76 epoch_requested: 76 NullEvt) class_id 2immediate prio 255 cost 10 e76) queued 2026-02-11T23:30:42.015+0000 7f4150194640 20 osd.0 76 agent_entry empty queue 2026-02-11T23:30:42.015+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:42.015+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 76 epoch_requested: 76 NullEvt) class_id 2immediate prio 255 cost 10 e76) pg 0x5978000 2026-02-11T23:30:42.015+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:42.015+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 76 epoch_requested: 76 NullEvt) class_id 2immediate prio 255 cost 10 e76) queued 2026-02-11T23:30:42.015+0000 7f414b98b640 10 osd.0 pg_epoch: 74 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 75 2026-02-11T23:30:42.015+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:42.015+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 76 epoch_requested: 76 NullEvt) class_id 2immediate prio 255 cost 10 e76) pg 0x5946000 2026-02-11T23:30:42.015+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 4.0 74 -> 75 2026-02-11T23:30:42.015+0000 7f414f993640 10 osd.0 pg_epoch: 74 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 75 2026-02-11T23:30:42.015+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 74 -> 75 2026-02-11T23:30:42.015+0000 7f414f993640 10 osd.0 pg_epoch: 74 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:42.015+0000 7f414b98b640 10 osd.0 pg_epoch: 74 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:42.015+0000 7f414f993640 10 osd.0 pg_epoch: 75 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:42.015+0000 7f414b98b640 10 osd.0 pg_epoch: 75 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:42.015+0000 7f414f993640 10 osd.0 pg_epoch: 75 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:42.015+0000 7f414b98b640 10 osd.0 pg_epoch: 75 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:42.015+0000 7f414f993640 20 osd.0 76 advance_pg new pool opts old pool opts 2026-02-11T23:30:42.015+0000 7f414b98b640 20 osd.0 76 advance_pg new pool opts old pool opts 2026-02-11T23:30:42.015+0000 7f414f993640 10 osd.0 pg_epoch: 75 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 76 2026-02-11T23:30:42.015+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 75 -> 76 2026-02-11T23:30:42.015+0000 7f414b98b640 10 osd.0 pg_epoch: 75 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 76 2026-02-11T23:30:42.015+0000 7f414f993640 10 osd.0 pg_epoch: 75 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:42.015+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 4.0 75 -> 76 2026-02-11T23:30:42.015+0000 7f414f993640 10 osd.0 pg_epoch: 76 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:42.015+0000 7f414b98b640 10 osd.0 pg_epoch: 75 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:42.015+0000 7f414f993640 10 osd.0 pg_epoch: 76 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:42.015+0000 7f414f993640 20 osd.0 76 advance_pg new pool opts old pool opts 2026-02-11T23:30:42.015+0000 7f414b98b640 10 osd.0 pg_epoch: 76 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:42.015+0000 7f414f993640 10 osd.0 pg_epoch: 76 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_activate_map: epoch range: 75..76 2026-02-11T23:30:42.015+0000 7f414b98b640 10 osd.0 pg_epoch: 76 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:42.015+0000 7f414f993640 10 osd.0 pg_epoch: 76 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:42.015+0000 7f414b98b640 20 osd.0 76 advance_pg new pool opts old pool opts 2026-02-11T23:30:42.015+0000 7f414f993640 20 osd.0 pg_epoch: 76 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 73 while current is 76 2026-02-11T23:30:42.015+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:42.015+0000 7f414b98b640 10 osd.0 pg_epoch: 76 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_activate_map: epoch range: 75..76 2026-02-11T23:30:42.015+0000 7f414f993640 10 osd.0 pg_epoch: 76 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 76 epoch_requested: 76 NullEvt 2026-02-11T23:30:42.015+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:42.015+0000 7f414b98b640 10 osd.0 pg_epoch: 76 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:42.015+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:42.015+0000 7f414b98b640 20 osd.0 pg_epoch: 76 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 53 while current is 76 2026-02-11T23:30:42.015+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:42.015+0000 7f414b98b640 10 osd.0 pg_epoch: 76 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 76 epoch_requested: 76 NullEvt 2026-02-11T23:30:42.015+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:42.015+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:42.285+0000 7f4162bea640 10 osd.0 76 tick_without_osd_lock 2026-02-11T23:30:42.285+0000 7f4162bea640 0 osd.0 76 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:42.285+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:42.285+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:42.285+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:42.285+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.468 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:42.285+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:42.287, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:42.285+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:42.285+0000 7f4162bea640 10 osd.0 76 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:42.285+0000 7f4162bea640 20 osd.0 76 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:42.285+0000 7f4162bea640 10 osd.0 76 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:42.285+0000 7f4162bea640 20 osd.0 76 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:42.322+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 41 ==== osd_map(77..78 src has 1..78) ==== 895+0+0 (crc 0 0 0) 0x5955c00 con 0x4acbc00 2026-02-11T23:30:42.322+0000 7f415cf02640 20 osd.0 76 OSD::ms_dispatch: osd_map(77..78 src has 1..78) 2026-02-11T23:30:42.322+0000 7f415cf02640 20 osd.0 76 _dispatch 0x5955c00 osd_map(77..78 src has 1..78) 2026-02-11T23:30:42.322+0000 7f415cf02640 3 osd.0 76 handle_osd_map epochs [77,78], i have 76, src has [1,78] 2026-02-11T23:30:42.322+0000 7f415cf02640 10 osd.0 76 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:42.322+0000 7f415cf02640 10 osd.0 76 handle_osd_map got inc map for epoch 77 2026-02-11T23:30:42.322+0000 7f415cf02640 20 osd.0 76 got_full_map 77, nothing requested 2026-02-11T23:30:42.322+0000 7f415cf02640 10 osd.0 76 handle_osd_map got inc map for epoch 78 2026-02-11T23:30:42.322+0000 7f415cf02640 20 osd.0 76 got_full_map 78, nothing requested 2026-02-11T23:30:42.322+0000 7f415cf02640 10 osd.0 76 _track_pools_and_pg_num_changes recording new pool 8 pg_num 1 2026-02-11T23:30:42.322+0000 7f415cf02640 20 osd.0 76 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:42.322+0000 7f415cf02640 20 osd.0 76 handle_osd_map pg_num_history pg_num_history(e78 pg_nums {1={27=1},2={34=1},3={42=1},4={49=1},5={56=1},6={64=1},7={70=1},8={77=1}} deleted_pools ) 2026-02-11T23:30:42.322+0000 7f415cf02640 10 snap_mapper record_purged_snaps purged_snaps {77={},78={}} 2026-02-11T23:30:42.322+0000 7f415cf02640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:42.322+0000 7f415cf02640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e78 maps [1~78] lci=[5,78] tlb=1) 2026-02-11T23:30:42.323+0000 7f41591a6640 10 osd.0 76 _committed_osd_maps 77..78 2026-02-11T23:30:42.323+0000 7f41591a6640 10 osd.0 76 advance to epoch 77 (<= last 78 <= newest_map 78) 2026-02-11T23:30:42.323+0000 7f41591a6640 10 osd.0 77 advance to epoch 78 (<= last 78 <= newest_map 78) 2026-02-11T23:30:42.323+0000 7f41591a6640 20 osd.0 78 consume_map version 78 2026-02-11T23:30:42.323+0000 7f41591a6640 20 osd.0 76 await_reserved_maps epoch:78 2026-02-11T23:30:42.323+0000 7f41591a6640 20 osd.0 76 await_reserved_maps done 0 2026-02-11T23:30:42.323+0000 7f41591a6640 20 osd.0 78 consume_map 78 -- publish done 2026-02-11T23:30:42.323+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:30:42.323+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x37582c0 2026-02-11T23:30:42.323+0000 7f41591a6640 20 osd.0 78 identify_splits_and_merges 4.0 e76 to e78 2026-02-11T23:30:42.323+0000 7f41591a6640 20 osd.0 78 identify_splits_and_merges 4.0 e76 to e78 pg_nums {49=1} 2026-02-11T23:30:42.323+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3700dc0 2026-02-11T23:30:42.323+0000 7f41591a6640 20 osd.0 78 identify_splits_and_merges 1.0 e76 to e78 2026-02-11T23:30:42.323+0000 7f41591a6640 20 osd.0 78 identify_splits_and_merges 1.0 e76 to e78 pg_nums {27=1} 2026-02-11T23:30:42.323+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:42.323+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:42.323+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:42.323+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:42.323+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:42.323+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:42.323+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:42.323+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:42.323+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:42.323+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:42.323+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:42.323+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:42.323+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:42.323+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:42.323+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:42.323+0000 7f41591a6640 20 osd.0 78 prune_pg_created 2026-02-11T23:30:42.323+0000 7f41591a6640 10 osd.0:0.consume_map 78 (was 76) 2026-02-11T23:30:42.323+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:42.323+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:42.323+0000 7f41591a6640 10 osd.0:1.consume_map 78 (was 76) 2026-02-11T23:30:42.323+0000 7f41591a6640 10 osd.0:2.consume_map 78 (was 76) 2026-02-11T23:30:42.323+0000 7f41591a6640 10 osd.0:3.consume_map 78 (was 76) 2026-02-11T23:30:42.323+0000 7f41591a6640 10 osd.0:4.consume_map 78 (was 76) 2026-02-11T23:30:42.323+0000 7f41591a6640 10 osd.0:5.consume_map 78 (was 76) 2026-02-11T23:30:42.323+0000 7f41591a6640 10 osd.0:6.consume_map 78 (was 76) 2026-02-11T23:30:42.323+0000 7f41591a6640 10 osd.0:7.consume_map 78 (was 76) 2026-02-11T23:30:42.323+0000 7f41591a6640 10 osd.0 78 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:42.323+0000 7f41591a6640 15 osd.0 78 enqueue_peering_evt 4.0 epoch_sent: 78 epoch_requested: 78 NullEvt 2026-02-11T23:30:42.323+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 78 epoch_requested: 78 NullEvt) class_id 2 prio 255 cost 10 e78) 2026-02-11T23:30:42.323+0000 7f41591a6640 15 osd.0 78 enqueue_peering_evt 1.0 epoch_sent: 78 epoch_requested: 78 NullEvt 2026-02-11T23:30:42.323+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 78 epoch_requested: 78 NullEvt) class_id 2 prio 255 cost 10 e78) 2026-02-11T23:30:42.323+0000 7f41591a6640 7 osd.0 78 activate_map version 78 2026-02-11T23:30:42.323+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:42.323+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 78 epoch_requested: 78 NullEvt) class_id 2immediate prio 255 cost 10 e78) queued 2026-02-11T23:30:42.323+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:42.323+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 78 epoch_requested: 78 NullEvt) class_id 2immediate prio 255 cost 10 e78) pg 0x5978000 2026-02-11T23:30:42.323+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:42.323+0000 7f414b98b640 10 osd.0 pg_epoch: 76 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 77 2026-02-11T23:30:42.323+0000 7f4150194640 20 osd.0 78 agent_entry empty queue 2026-02-11T23:30:42.323+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 78 epoch_requested: 78 NullEvt) class_id 2immediate prio 255 cost 10 e78) queued 2026-02-11T23:30:42.323+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:42.323+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 78 epoch_requested: 78 NullEvt) class_id 2immediate prio 255 cost 10 e78) pg 0x5946000 2026-02-11T23:30:42.323+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 4.0 76 -> 77 2026-02-11T23:30:42.323+0000 7f414b98b640 10 osd.0 pg_epoch: 76 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:42.323+0000 7f414f993640 10 osd.0 pg_epoch: 76 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 77 2026-02-11T23:30:42.323+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 76 -> 77 2026-02-11T23:30:42.323+0000 7f414b98b640 10 osd.0 pg_epoch: 77 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:42.323+0000 7f414b98b640 10 osd.0 pg_epoch: 77 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:42.323+0000 7f414f993640 10 osd.0 pg_epoch: 76 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:42.323+0000 7f414b98b640 20 osd.0 78 advance_pg new pool opts old pool opts 2026-02-11T23:30:42.323+0000 7f414f993640 10 osd.0 pg_epoch: 77 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:42.323+0000 7f414b98b640 10 osd.0 pg_epoch: 77 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 78 2026-02-11T23:30:42.323+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 4.0 77 -> 78 2026-02-11T23:30:42.323+0000 7f414b98b640 10 osd.0 pg_epoch: 77 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:42.323+0000 7f414f993640 10 osd.0 pg_epoch: 77 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:42.323+0000 7f414b98b640 10 osd.0 pg_epoch: 78 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:42.323+0000 7f414b98b640 10 osd.0 pg_epoch: 78 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:42.323+0000 7f414b98b640 20 osd.0 78 advance_pg new pool opts old pool opts 2026-02-11T23:30:42.323+0000 7f414f993640 20 osd.0 78 advance_pg new pool opts old pool opts 2026-02-11T23:30:42.323+0000 7f414b98b640 10 osd.0 pg_epoch: 78 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_activate_map: epoch range: 77..78 2026-02-11T23:30:42.323+0000 7f414b98b640 10 osd.0 pg_epoch: 78 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:42.323+0000 7f414f993640 10 osd.0 pg_epoch: 77 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 78 2026-02-11T23:30:42.323+0000 7f414b98b640 20 osd.0 pg_epoch: 78 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 53 while current is 78 2026-02-11T23:30:42.323+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 77 -> 78 2026-02-11T23:30:42.323+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:42.323+0000 7f414b98b640 10 osd.0 pg_epoch: 78 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 78 epoch_requested: 78 NullEvt 2026-02-11T23:30:42.323+0000 7f414f993640 10 osd.0 pg_epoch: 77 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:42.323+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:42.323+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:42.323+0000 7f414f993640 10 osd.0 pg_epoch: 78 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:42.323+0000 7f414f993640 10 osd.0 pg_epoch: 78 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:42.323+0000 7f414f993640 20 osd.0 78 advance_pg new pool opts old pool opts 2026-02-11T23:30:42.323+0000 7f414f993640 10 osd.0 pg_epoch: 78 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_activate_map: epoch range: 77..78 2026-02-11T23:30:42.323+0000 7f414f993640 10 osd.0 pg_epoch: 78 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:42.323+0000 7f414f993640 20 osd.0 pg_epoch: 78 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 73 while current is 78 2026-02-11T23:30:42.323+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:42.323+0000 7f414f993640 10 osd.0 pg_epoch: 78 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 78 epoch_requested: 78 NullEvt 2026-02-11T23:30:42.323+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:42.323+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:42.327+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:30:42.327+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:30:42.327+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:12.327987+0000) 2026-02-11T23:30:42.372+0000 7f41633eb640 10 osd.0 78 tick 2026-02-11T23:30:42.372+0000 7f41633eb640 20 osd.0 78 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:30:42.484+0000 7f41683f5640 10 monclient: _renew_subs 2026-02-11T23:30:42.484+0000 7f41683f5640 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:30:42.484+0000 7f41683f5640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> v2:127.0.0.1:7180/0 -- mon_subscribe({osdmap=79}) -- 0x3814820 con 0x4acbc00 2026-02-11T23:30:42.619+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 42 ==== osd_map(79..79 src has 1..79) ==== 618+0+0 (crc 0 0 0) 0x5955dc0 con 0x4acbc00 2026-02-11T23:30:42.619+0000 7f415cf02640 20 osd.0 78 OSD::ms_dispatch: osd_map(79..79 src has 1..79) 2026-02-11T23:30:42.619+0000 7f415cf02640 20 osd.0 78 _dispatch 0x5955dc0 osd_map(79..79 src has 1..79) 2026-02-11T23:30:42.619+0000 7f415cf02640 3 osd.0 78 handle_osd_map epochs [79,79], i have 78, src has [1,79] 2026-02-11T23:30:42.619+0000 7f415cf02640 10 osd.0 78 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:30:42.619+0000 7f415cf02640 10 osd.0 78 handle_osd_map got inc map for epoch 79 2026-02-11T23:30:42.619+0000 7f415cf02640 20 osd.0 78 got_full_map 79, nothing requested 2026-02-11T23:30:42.619+0000 7f415cf02640 20 osd.0 78 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:42.619+0000 7f415cf02640 20 osd.0 78 handle_osd_map pg_num_history pg_num_history(e79 pg_nums {1={27=1},2={34=1},3={42=1},4={49=1},5={56=1},6={64=1},7={70=1},8={77=1}} deleted_pools ) 2026-02-11T23:30:42.619+0000 7f415cf02640 10 snap_mapper record_purged_snaps purged_snaps {79={}} 2026-02-11T23:30:42.619+0000 7f415cf02640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:42.619+0000 7f415cf02640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e79 maps [1~79] lci=[5,79] tlb=1) 2026-02-11T23:30:42.619+0000 7f41591a6640 10 osd.0 78 _committed_osd_maps 79..79 2026-02-11T23:30:42.619+0000 7f41591a6640 10 osd.0 78 advance to epoch 79 (<= last 79 <= newest_map 79) 2026-02-11T23:30:42.619+0000 7f41591a6640 20 osd.0 79 consume_map version 79 2026-02-11T23:30:42.619+0000 7f41591a6640 20 osd.0 78 await_reserved_maps epoch:79 2026-02-11T23:30:42.619+0000 7f41591a6640 20 osd.0 78 await_reserved_maps done 0 2026-02-11T23:30:42.619+0000 7f41591a6640 20 osd.0 79 consume_map 79 -- publish done 2026-02-11T23:30:42.619+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:30:42.619+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x37582c0 2026-02-11T23:30:42.619+0000 7f41591a6640 20 osd.0 79 identify_splits_and_merges 4.0 e78 to e79 2026-02-11T23:30:42.619+0000 7f41591a6640 20 osd.0 79 identify_splits_and_merges 4.0 e78 to e79 pg_nums {49=1} 2026-02-11T23:30:42.619+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3700dc0 2026-02-11T23:30:42.619+0000 7f41591a6640 20 osd.0 79 identify_splits_and_merges 1.0 e78 to e79 2026-02-11T23:30:42.619+0000 7f41591a6640 20 osd.0 79 identify_splits_and_merges 1.0 e78 to e79 pg_nums {27=1} 2026-02-11T23:30:42.619+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:42.619+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:42.619+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:42.619+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:42.619+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:42.619+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:42.619+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:42.619+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:42.619+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:42.619+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:42.619+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:42.619+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:42.619+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:42.619+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:42.619+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:42.619+0000 7f41591a6640 20 osd.0 79 prune_pg_created 2026-02-11T23:30:42.619+0000 7f41591a6640 10 osd.0:0.consume_map 79 (was 78) 2026-02-11T23:30:42.619+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:42.619+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:42.619+0000 7f41591a6640 10 osd.0:1.consume_map 79 (was 78) 2026-02-11T23:30:42.619+0000 7f41591a6640 10 osd.0:2.consume_map 79 (was 78) 2026-02-11T23:30:42.619+0000 7f41591a6640 10 osd.0:3.consume_map 79 (was 78) 2026-02-11T23:30:42.619+0000 7f41591a6640 10 osd.0:4.consume_map 79 (was 78) 2026-02-11T23:30:42.619+0000 7f41591a6640 10 osd.0:5.consume_map 79 (was 78) 2026-02-11T23:30:42.619+0000 7f41591a6640 10 osd.0:6.consume_map 79 (was 78) 2026-02-11T23:30:42.619+0000 7f41591a6640 10 osd.0:7.consume_map 79 (was 78) 2026-02-11T23:30:42.619+0000 7f41591a6640 10 osd.0 79 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:42.619+0000 7f41591a6640 15 osd.0 79 enqueue_peering_evt 4.0 epoch_sent: 79 epoch_requested: 79 NullEvt 2026-02-11T23:30:42.619+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 79 epoch_requested: 79 NullEvt) class_id 2 prio 255 cost 10 e79) 2026-02-11T23:30:42.619+0000 7f41591a6640 15 osd.0 79 enqueue_peering_evt 1.0 epoch_sent: 79 epoch_requested: 79 NullEvt 2026-02-11T23:30:42.619+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 79 epoch_requested: 79 NullEvt) class_id 2 prio 255 cost 10 e79) 2026-02-11T23:30:42.619+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:42.619+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 79 epoch_requested: 79 NullEvt) class_id 2immediate prio 255 cost 10 e79) queued 2026-02-11T23:30:42.619+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:42.619+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 79 epoch_requested: 79 NullEvt) class_id 2immediate prio 255 cost 10 e79) pg 0x5978000 2026-02-11T23:30:42.619+0000 7f41591a6640 7 osd.0 79 activate_map version 79 2026-02-11T23:30:42.619+0000 7f414b98b640 10 osd.0 pg_epoch: 78 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 79 2026-02-11T23:30:42.619+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:42.619+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 79 epoch_requested: 79 NullEvt) class_id 2immediate prio 255 cost 10 e79) queued 2026-02-11T23:30:42.619+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:42.619+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 79 epoch_requested: 79 NullEvt) class_id 2immediate prio 255 cost 10 e79) pg 0x5946000 2026-02-11T23:30:42.619+0000 7f4150194640 20 osd.0 79 agent_entry empty queue 2026-02-11T23:30:42.619+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 4.0 78 -> 79 2026-02-11T23:30:42.619+0000 7f414b98b640 10 osd.0 pg_epoch: 78 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:42.619+0000 7f414f993640 10 osd.0 pg_epoch: 78 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 79 2026-02-11T23:30:42.619+0000 7f414b98b640 10 osd.0 pg_epoch: 79 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:42.619+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 78 -> 79 2026-02-11T23:30:42.619+0000 7f414b98b640 10 osd.0 pg_epoch: 79 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:42.619+0000 7f414f993640 10 osd.0 pg_epoch: 78 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:42.619+0000 7f414b98b640 20 osd.0 79 advance_pg new pool opts old pool opts 2026-02-11T23:30:42.619+0000 7f414f993640 10 osd.0 pg_epoch: 79 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:42.619+0000 7f414b98b640 10 osd.0 pg_epoch: 79 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_activate_map: epoch range: 79..79 2026-02-11T23:30:42.619+0000 7f414f993640 10 osd.0 pg_epoch: 79 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:42.619+0000 7f414b98b640 10 osd.0 pg_epoch: 79 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:42.619+0000 7f414f993640 20 osd.0 79 advance_pg new pool opts old pool opts 2026-02-11T23:30:42.619+0000 7f414b98b640 20 osd.0 pg_epoch: 79 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 53 while current is 79 2026-02-11T23:30:42.619+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:42.619+0000 7f414f993640 10 osd.0 pg_epoch: 79 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_activate_map: epoch range: 79..79 2026-02-11T23:30:42.619+0000 7f414f993640 10 osd.0 pg_epoch: 79 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:42.619+0000 7f414b98b640 10 osd.0 pg_epoch: 79 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 79 epoch_requested: 79 NullEvt 2026-02-11T23:30:42.619+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:42.619+0000 7f414f993640 20 osd.0 pg_epoch: 79 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 73 while current is 79 2026-02-11T23:30:42.619+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:42.619+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:42.619+0000 7f414f993640 10 osd.0 pg_epoch: 79 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 79 epoch_requested: 79 NullEvt 2026-02-11T23:30:42.619+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:42.619+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:42.964+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.4 127.0.0.1:0/3331151400 4 ==== osd_ping(ping e79 up_from 23 ping_stamp 2026-02-11T23:30:42.964594+0000/15.803361893s send_stamp 15.803361893s delta_ub -9.836438179s) ==== 2033+0+0 (crc 0 0 0) 0x4c9aa80 con 0x507a800 2026-02-11T23:30:42.964+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.4 127.0.0.1:0/3331151400 4 ==== osd_ping(ping e79 up_from 23 ping_stamp 2026-02-11T23:30:42.964594+0000/15.803361893s send_stamp 15.803361893s delta_ub -9.836438179s) ==== 2033+0+0 (crc 0 0 0) 0x5af8480 con 0x507a400 2026-02-11T23:30:42.964+0000 7f416c3fd640 20 osd.0 79 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836744308s,-9.836438179s]) 2026-02-11T23:30:42.964+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e79 up_from 5 ping_stamp 2026-02-11T23:30:42.964594+0000/15.803361893s send_stamp 25.640106201s delta_ub 9.836744308s) -- 0x5af8600 con 0x507a800 2026-02-11T23:30:42.964+0000 7f416c3fd640 20 osd.0 79 get_con_osd_cluster to osd.4 from_epoch 79 2026-02-11T23:30:42.964+0000 7f416c3fd640 20 osd.0 79 get_nextmap_reserved map_reservations: {79=1} 2026-02-11T23:30:42.964+0000 7f416c3fd640 20 osd.0 79 release_map epoch: 79 2026-02-11T23:30:42.964+0000 7f416c3fd640 10 osd.0 79 maybe_share_map: con v2:127.0.0.1:6836/3331151400 updating session's projected_epoch from 73 to ping map epoch of 79 2026-02-11T23:30:42.964+0000 7f416c3fd640 10 osd.0 79 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 79 is not newer than session's projected_epoch of 79 2026-02-11T23:30:42.964+0000 7f416abfa640 20 osd.0 79 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836782455s,-9.836438179s]) 2026-02-11T23:30:42.964+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e79 up_from 5 ping_stamp 2026-02-11T23:30:42.964594+0000/15.803361893s send_stamp 25.640144348s delta_ub 9.836782455s) -- 0x5af8180 con 0x507a400 2026-02-11T23:30:42.964+0000 7f416abfa640 20 osd.0 79 get_con_osd_cluster to osd.4 from_epoch 79 2026-02-11T23:30:42.964+0000 7f416abfa640 20 osd.0 79 get_nextmap_reserved map_reservations: {79=1} 2026-02-11T23:30:42.964+0000 7f416abfa640 20 osd.0 79 release_map epoch: 79 2026-02-11T23:30:42.964+0000 7f416abfa640 10 osd.0 79 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 79 is not newer than session's projected_epoch of 79 2026-02-11T23:30:43.204+0000 7f415b6ff640 20 osd.0 79 reports for 0 queries 2026-02-11T23:30:43.204+0000 7f415b6ff640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] -- mgrreport(unknown.0 +0-0 packed 3294 daemon_metrics=2) -- 0x4f9ea80 con 0x4aca800 2026-02-11T23:30:43.204+0000 7f415b6ff640 15 osd.0 79 collect_pg_stats 2026-02-11T23:30:43.204+0000 7f415b6ff640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] -- pg_stats(0 pgs seq 21474836486 v 0) -- 0x5ab2300 con 0x4aca800 2026-02-11T23:30:43.279+0000 7f4162bea640 10 osd.0 79 tick_without_osd_lock 2026-02-11T23:30:43.279+0000 7f4162bea640 0 osd.0 79 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:43.279+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:43.279+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:43.279+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:43.279+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.430 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:43.279+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:43.279, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:43.279+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:43.279+0000 7f4162bea640 10 osd.0 79 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:43.279+0000 7f4162bea640 20 osd.0 79 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:43.279+0000 7f4162bea640 10 osd.0 79 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:43.279+0000 7f4162bea640 20 osd.0 79 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:43.327+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:30:43.327+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:30:43.327+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:13.328004+0000) 2026-02-11T23:30:43.373+0000 7f41633eb640 10 osd.0 79 tick 2026-02-11T23:30:43.373+0000 7f41633eb640 20 osd.0 79 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:30:43.847+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.3 127.0.0.1:0/2699377081 5 ==== osd_ping(ping e83 up_from 19 ping_stamp 2026-02-11T23:30:43.847724+0000/19.303663254s send_stamp 19.303663254s delta_ub -7.219277382s) ==== 2033+0+0 (crc 0 0 0) 0x5af8600 con 0x48cc400 2026-02-11T23:30:43.847+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.3 127.0.0.1:0/2699377081 5 ==== osd_ping(ping e83 up_from 19 ping_stamp 2026-02-11T23:30:43.847724+0000/19.303663254s send_stamp 19.303663254s delta_ub -7.219277382s) ==== 2033+0+0 (crc 0 0 0) 0x4c9bb00 con 0x48cc800 2026-02-11T23:30:43.847+0000 7f416c3fd640 20 osd.0 79 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219539165s,-7.219277382s]) 2026-02-11T23:30:43.847+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e79 up_from 5 ping_stamp 2026-02-11T23:30:43.847724+0000/19.303663254s send_stamp 26.523200989s delta_ub 7.219539165s) -- 0x4c9aa80 con 0x48cc400 2026-02-11T23:30:43.847+0000 7f416c3fd640 20 osd.0 79 get_con_osd_cluster to osd.3 from_epoch 79 2026-02-11T23:30:43.847+0000 7f416c3fd640 20 osd.0 79 get_nextmap_reserved map_reservations: {79=1} 2026-02-11T23:30:43.847+0000 7f416c3fd640 20 osd.0 79 release_map epoch: 79 2026-02-11T23:30:43.847+0000 7f416c3fd640 10 osd.0 79 maybe_share_map: con v2:127.0.0.1:6828/2699377081 updating session's projected_epoch from 73 to ping map epoch of 83 2026-02-11T23:30:43.847+0000 7f416c3fd640 10 osd.0 79 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 79 is not newer than session's projected_epoch of 83 2026-02-11T23:30:43.847+0000 7f416cbfe640 20 osd.0 79 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219561577s,-7.219277382s]) 2026-02-11T23:30:43.847+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e79 up_from 5 ping_stamp 2026-02-11T23:30:43.847724+0000/19.303663254s send_stamp 26.523223877s delta_ub 7.219561577s) -- 0x4c9be00 con 0x48cc800 2026-02-11T23:30:43.847+0000 7f416cbfe640 20 osd.0 79 get_con_osd_cluster to osd.3 from_epoch 79 2026-02-11T23:30:43.847+0000 7f416cbfe640 20 osd.0 79 get_nextmap_reserved map_reservations: {79=1} 2026-02-11T23:30:43.847+0000 7f416cbfe640 20 osd.0 79 release_map epoch: 79 2026-02-11T23:30:43.847+0000 7f416cbfe640 10 osd.0 79 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 79 is not newer than session's projected_epoch of 83 2026-02-11T23:30:43.847+0000 7f415aefe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.3 v2:127.0.0.1:6828/2699377081 22 ==== osd_map(80..83 src has 1..83) ==== 2042+0+0 (crc 0 0 0) 0x5954a80 con 0x48ccc00 2026-02-11T23:30:43.847+0000 7f415aefe640 20 osd.0 79 OSD::ms_dispatch: osd_map(80..83 src has 1..83) 2026-02-11T23:30:43.847+0000 7f415aefe640 20 osd.0 79 _dispatch 0x5954a80 osd_map(80..83 src has 1..83) 2026-02-11T23:30:43.847+0000 7f415aefe640 3 osd.0 79 handle_osd_map epochs [80,83], i have 79, src has [1,83] 2026-02-11T23:30:43.847+0000 7f415aefe640 10 osd.0 79 handle_osd_map: received 0 full maps and 4 incremental maps 2026-02-11T23:30:43.847+0000 7f415aefe640 10 osd.0 79 handle_osd_map got inc map for epoch 80 2026-02-11T23:30:43.847+0000 7f415aefe640 20 osd.0 79 got_full_map 80, nothing requested 2026-02-11T23:30:43.847+0000 7f415aefe640 10 osd.0 79 handle_osd_map got inc map for epoch 81 2026-02-11T23:30:43.848+0000 7f415aefe640 20 osd.0 79 got_full_map 81, nothing requested 2026-02-11T23:30:43.848+0000 7f415aefe640 10 osd.0 79 handle_osd_map got inc map for epoch 82 2026-02-11T23:30:43.848+0000 7f415aefe640 20 osd.0 79 got_full_map 82, nothing requested 2026-02-11T23:30:43.848+0000 7f415aefe640 10 osd.0 79 handle_osd_map got inc map for epoch 83 2026-02-11T23:30:43.848+0000 7f415aefe640 20 osd.0 79 got_full_map 83, nothing requested 2026-02-11T23:30:43.848+0000 7f415aefe640 20 osd.0 79 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:43.848+0000 7f415aefe640 20 osd.0 79 handle_osd_map pg_num_history pg_num_history(e83 pg_nums {1={27=1},2={34=1},3={42=1},4={49=1},5={56=1},6={64=1},7={70=1},8={77=1}} deleted_pools ) 2026-02-11T23:30:43.848+0000 7f415aefe640 10 snap_mapper record_purged_snaps purged_snaps {80={},81={},82={},83={}} 2026-02-11T23:30:43.848+0000 7f415aefe640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:43.848+0000 7f415aefe640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e83 maps [1~83] lci=[5,83] tlb=1) 2026-02-11T23:30:43.848+0000 7f41591a6640 10 osd.0 79 _committed_osd_maps 80..83 2026-02-11T23:30:43.848+0000 7f41591a6640 10 osd.0 79 advance to epoch 80 (<= last 83 <= newest_map 83) 2026-02-11T23:30:43.848+0000 7f41591a6640 10 osd.0 80 advance to epoch 81 (<= last 83 <= newest_map 83) 2026-02-11T23:30:43.848+0000 7f41591a6640 10 osd.0 81 advance to epoch 82 (<= last 83 <= newest_map 83) 2026-02-11T23:30:43.848+0000 7f41591a6640 10 osd.0 82 advance to epoch 83 (<= last 83 <= newest_map 83) 2026-02-11T23:30:43.848+0000 7f41591a6640 20 osd.0 83 consume_map version 83 2026-02-11T23:30:43.848+0000 7f41591a6640 20 osd.0 79 await_reserved_maps epoch:83 2026-02-11T23:30:43.848+0000 7f41591a6640 20 osd.0 79 await_reserved_maps done 0 2026-02-11T23:30:43.848+0000 7f41591a6640 20 osd.0 83 consume_map 83 -- publish done 2026-02-11T23:30:43.848+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:30:43.848+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x37582c0 2026-02-11T23:30:43.848+0000 7f41591a6640 20 osd.0 83 identify_splits_and_merges 4.0 e79 to e83 2026-02-11T23:30:43.848+0000 7f41591a6640 20 osd.0 83 identify_splits_and_merges 4.0 e79 to e83 pg_nums {49=1} 2026-02-11T23:30:43.848+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3700dc0 2026-02-11T23:30:43.848+0000 7f41591a6640 20 osd.0 83 identify_splits_and_merges 1.0 e79 to e83 2026-02-11T23:30:43.848+0000 7f41591a6640 20 osd.0 83 identify_splits_and_merges 1.0 e79 to e83 pg_nums {27=1} 2026-02-11T23:30:43.848+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:43.848+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:43.848+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:43.848+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:43.848+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:43.848+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:43.848+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:43.848+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:43.848+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:43.848+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:43.848+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:43.848+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:43.848+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:43.848+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:43.848+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:43.848+0000 7f41591a6640 20 osd.0 83 prune_pg_created 2026-02-11T23:30:43.848+0000 7f41591a6640 10 osd.0:0.consume_map 83 (was 79) 2026-02-11T23:30:43.848+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:43.848+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:43.848+0000 7f41591a6640 10 osd.0:1.consume_map 83 (was 79) 2026-02-11T23:30:43.848+0000 7f41591a6640 10 osd.0:2.consume_map 83 (was 79) 2026-02-11T23:30:43.848+0000 7f41591a6640 10 osd.0:3.consume_map 83 (was 79) 2026-02-11T23:30:43.848+0000 7f41591a6640 10 osd.0:4.consume_map 83 (was 79) 2026-02-11T23:30:43.848+0000 7f41591a6640 10 osd.0:5.consume_map 83 (was 79) 2026-02-11T23:30:43.848+0000 7f41591a6640 10 osd.0:6.consume_map 83 (was 79) 2026-02-11T23:30:43.848+0000 7f41591a6640 10 osd.0:7.consume_map 83 (was 79) 2026-02-11T23:30:43.848+0000 7f41591a6640 10 osd.0 83 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:43.848+0000 7f41591a6640 15 osd.0 83 enqueue_peering_evt 4.0 epoch_sent: 83 epoch_requested: 83 NullEvt 2026-02-11T23:30:43.848+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 83 epoch_requested: 83 NullEvt) class_id 2 prio 255 cost 10 e83) 2026-02-11T23:30:43.848+0000 7f41591a6640 15 osd.0 83 enqueue_peering_evt 1.0 epoch_sent: 83 epoch_requested: 83 NullEvt 2026-02-11T23:30:43.848+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 83 epoch_requested: 83 NullEvt) class_id 2 prio 255 cost 10 e83) 2026-02-11T23:30:43.848+0000 7f41591a6640 7 osd.0 83 activate_map version 83 2026-02-11T23:30:43.848+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:43.848+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 83 epoch_requested: 83 NullEvt) class_id 2immediate prio 255 cost 10 e83) queued 2026-02-11T23:30:43.848+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:43.848+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 83 epoch_requested: 83 NullEvt) class_id 2immediate prio 255 cost 10 e83) pg 0x5978000 2026-02-11T23:30:43.848+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:43.848+0000 7f4150194640 20 osd.0 83 agent_entry empty queue 2026-02-11T23:30:43.848+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 83 epoch_requested: 83 NullEvt) class_id 2immediate prio 255 cost 10 e83) queued 2026-02-11T23:30:43.848+0000 7f414f993640 10 osd.0 pg_epoch: 79 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 80 2026-02-11T23:30:43.848+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:43.848+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 83 epoch_requested: 83 NullEvt) class_id 2immediate prio 255 cost 10 e83) pg 0x5946000 2026-02-11T23:30:43.848+0000 7f414f993640 20 osd.0:0.update_pg_epoch 4.0 79 -> 80 2026-02-11T23:30:43.848+0000 7f414f993640 10 osd.0 pg_epoch: 79 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:43.848+0000 7f414b98b640 10 osd.0 pg_epoch: 79 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 80 2026-02-11T23:30:43.848+0000 7f414f993640 10 osd.0 pg_epoch: 80 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:43.848+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 1.0 79 -> 80 2026-02-11T23:30:43.848+0000 7f414f993640 10 osd.0 pg_epoch: 80 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:43.848+0000 7f414b98b640 10 osd.0 pg_epoch: 79 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:43.848+0000 7f414f993640 20 osd.0 83 advance_pg new pool opts old pool opts 2026-02-11T23:30:43.848+0000 7f414f993640 10 osd.0 pg_epoch: 80 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 81 2026-02-11T23:30:43.848+0000 7f414f993640 20 osd.0:0.update_pg_epoch 4.0 80 -> 81 2026-02-11T23:30:43.848+0000 7f414b98b640 10 osd.0 pg_epoch: 80 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:43.848+0000 7f414f993640 10 osd.0 pg_epoch: 80 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:43.848+0000 7f414f993640 10 osd.0 pg_epoch: 81 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:43.848+0000 7f414b98b640 10 osd.0 pg_epoch: 80 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:43.848+0000 7f414f993640 10 osd.0 pg_epoch: 81 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:43.848+0000 7f414f993640 20 osd.0 83 advance_pg new pool opts old pool opts 2026-02-11T23:30:43.848+0000 7f414b98b640 20 osd.0 83 advance_pg new pool opts old pool opts 2026-02-11T23:30:43.848+0000 7f414f993640 10 osd.0 pg_epoch: 81 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 82 2026-02-11T23:30:43.848+0000 7f414f993640 20 osd.0:0.update_pg_epoch 4.0 81 -> 82 2026-02-11T23:30:43.848+0000 7f414b98b640 10 osd.0 pg_epoch: 80 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 81 2026-02-11T23:30:43.848+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 1.0 80 -> 81 2026-02-11T23:30:43.848+0000 7f414f993640 10 osd.0 pg_epoch: 81 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:43.848+0000 7f414b98b640 10 osd.0 pg_epoch: 80 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:43.848+0000 7f414f993640 10 osd.0 pg_epoch: 82 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:43.848+0000 7f414b98b640 10 osd.0 pg_epoch: 81 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:43.848+0000 7f414f993640 10 osd.0 pg_epoch: 82 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:43.848+0000 7f414f993640 20 osd.0 83 advance_pg new pool opts old pool opts 2026-02-11T23:30:43.848+0000 7f414b98b640 10 osd.0 pg_epoch: 81 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:43.848+0000 7f414b98b640 20 osd.0 83 advance_pg new pool opts old pool opts 2026-02-11T23:30:43.848+0000 7f414f993640 10 osd.0 pg_epoch: 82 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 83 2026-02-11T23:30:43.848+0000 7f414f993640 20 osd.0:0.update_pg_epoch 4.0 82 -> 83 2026-02-11T23:30:43.848+0000 7f414b98b640 10 osd.0 pg_epoch: 81 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 82 2026-02-11T23:30:43.848+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 1.0 81 -> 82 2026-02-11T23:30:43.848+0000 7f414f993640 10 osd.0 pg_epoch: 82 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:43.848+0000 7f414b98b640 10 osd.0 pg_epoch: 81 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:43.848+0000 7f414f993640 10 osd.0 pg_epoch: 83 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:43.848+0000 7f414b98b640 10 osd.0 pg_epoch: 82 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:43.848+0000 7f414f993640 10 osd.0 pg_epoch: 83 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:43.848+0000 7f414f993640 20 osd.0 83 advance_pg new pool opts old pool opts 2026-02-11T23:30:43.848+0000 7f414b98b640 10 osd.0 pg_epoch: 82 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:43.848+0000 7f414b98b640 20 osd.0 83 advance_pg new pool opts old pool opts 2026-02-11T23:30:43.848+0000 7f414f993640 10 osd.0 pg_epoch: 83 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_activate_map: epoch range: 80..83 2026-02-11T23:30:43.848+0000 7f414f993640 10 osd.0 pg_epoch: 83 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:43.848+0000 7f414b98b640 10 osd.0 pg_epoch: 82 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 83 2026-02-11T23:30:43.848+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 1.0 82 -> 83 2026-02-11T23:30:43.848+0000 7f414f993640 20 osd.0 pg_epoch: 83 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 53 while current is 83 2026-02-11T23:30:43.848+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:43.848+0000 7f414b98b640 10 osd.0 pg_epoch: 82 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:43.848+0000 7f414f993640 10 osd.0 pg_epoch: 83 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 83 epoch_requested: 83 NullEvt 2026-02-11T23:30:43.848+0000 7f414b98b640 10 osd.0 pg_epoch: 83 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:43.848+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:43.848+0000 7f414b98b640 10 osd.0 pg_epoch: 83 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:43.848+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:43.848+0000 7f414b98b640 20 osd.0 83 advance_pg new pool opts old pool opts 2026-02-11T23:30:43.848+0000 7f414b98b640 10 osd.0 pg_epoch: 83 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_activate_map: epoch range: 80..83 2026-02-11T23:30:43.848+0000 7f414b98b640 10 osd.0 pg_epoch: 83 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:43.848+0000 7f414b98b640 20 osd.0 pg_epoch: 83 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 73 while current is 83 2026-02-11T23:30:43.848+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:43.848+0000 7f414b98b640 10 osd.0 pg_epoch: 83 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 83 epoch_requested: 83 NullEvt 2026-02-11T23:30:43.848+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:43.848+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:43.913+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 43 ==== osd_map(83..84 src has 1..84) ==== 1200+0+0 (crc 0 0 0) 0x5955340 con 0x4acbc00 2026-02-11T23:30:43.913+0000 7f415cf02640 20 osd.0 83 OSD::ms_dispatch: osd_map(83..84 src has 1..84) 2026-02-11T23:30:43.913+0000 7f415cf02640 20 osd.0 83 _dispatch 0x5955340 osd_map(83..84 src has 1..84) 2026-02-11T23:30:43.913+0000 7f415cf02640 3 osd.0 83 handle_osd_map epochs [83,84], i have 83, src has [1,84] 2026-02-11T23:30:43.913+0000 7f415cf02640 10 osd.0 83 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:43.913+0000 7f415cf02640 10 osd.0 83 handle_osd_map got inc map for epoch 84 2026-02-11T23:30:43.913+0000 7f415cf02640 20 osd.0 83 got_full_map 84, nothing requested 2026-02-11T23:30:43.913+0000 7f415cf02640 20 osd.0 83 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:43.913+0000 7f415cf02640 20 osd.0 83 handle_osd_map pg_num_history pg_num_history(e84 pg_nums {1={27=1},2={34=1},3={42=1},4={49=1},5={56=1},6={64=1},7={70=1},8={77=1}} deleted_pools ) 2026-02-11T23:30:43.913+0000 7f415cf02640 10 snap_mapper record_purged_snaps purged_snaps {84={}} 2026-02-11T23:30:43.913+0000 7f415cf02640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:43.913+0000 7f415cf02640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e84 maps [1~84] lci=[5,84] tlb=1) 2026-02-11T23:30:43.913+0000 7f41591a6640 10 osd.0 83 _committed_osd_maps 84..84 2026-02-11T23:30:43.913+0000 7f41591a6640 10 osd.0 83 advance to epoch 84 (<= last 84 <= newest_map 84) 2026-02-11T23:30:43.913+0000 7f41591a6640 20 osd.0 84 consume_map version 84 2026-02-11T23:30:43.913+0000 7f41591a6640 20 osd.0 83 await_reserved_maps epoch:84 2026-02-11T23:30:43.913+0000 7f41591a6640 20 osd.0 83 await_reserved_maps done 0 2026-02-11T23:30:43.913+0000 7f41591a6640 20 osd.0 84 consume_map 84 -- publish done 2026-02-11T23:30:43.913+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:30:43.913+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x37582c0 2026-02-11T23:30:43.913+0000 7f41591a6640 20 osd.0 84 identify_splits_and_merges 4.0 e83 to e84 2026-02-11T23:30:43.913+0000 7f41591a6640 20 osd.0 84 identify_splits_and_merges 4.0 e83 to e84 pg_nums {49=1} 2026-02-11T23:30:43.913+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3700dc0 2026-02-11T23:30:43.913+0000 7f41591a6640 20 osd.0 84 identify_splits_and_merges 1.0 e83 to e84 2026-02-11T23:30:43.913+0000 7f41591a6640 20 osd.0 84 identify_splits_and_merges 1.0 e83 to e84 pg_nums {27=1} 2026-02-11T23:30:43.913+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:43.913+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:43.913+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:43.913+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:43.913+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:43.913+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:43.913+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:43.913+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:43.913+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:43.913+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:43.913+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:43.913+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:43.913+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:43.913+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:43.913+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:43.913+0000 7f41591a6640 20 osd.0 84 prune_pg_created 2026-02-11T23:30:43.913+0000 7f41591a6640 10 osd.0:0.consume_map 84 (was 83) 2026-02-11T23:30:43.913+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:43.913+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:43.913+0000 7f41591a6640 10 osd.0:1.consume_map 84 (was 83) 2026-02-11T23:30:43.913+0000 7f41591a6640 10 osd.0:2.consume_map 84 (was 83) 2026-02-11T23:30:43.913+0000 7f41591a6640 10 osd.0:3.consume_map 84 (was 83) 2026-02-11T23:30:43.913+0000 7f41591a6640 10 osd.0:4.consume_map 84 (was 83) 2026-02-11T23:30:43.913+0000 7f41591a6640 10 osd.0:5.consume_map 84 (was 83) 2026-02-11T23:30:43.913+0000 7f41591a6640 10 osd.0:6.consume_map 84 (was 83) 2026-02-11T23:30:43.913+0000 7f41591a6640 10 osd.0:7.consume_map 84 (was 83) 2026-02-11T23:30:43.913+0000 7f41591a6640 10 osd.0 84 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:43.913+0000 7f41591a6640 15 osd.0 84 enqueue_peering_evt 4.0 epoch_sent: 84 epoch_requested: 84 NullEvt 2026-02-11T23:30:43.913+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 84 epoch_requested: 84 NullEvt) class_id 2 prio 255 cost 10 e84) 2026-02-11T23:30:43.913+0000 7f41591a6640 15 osd.0 84 enqueue_peering_evt 1.0 epoch_sent: 84 epoch_requested: 84 NullEvt 2026-02-11T23:30:43.913+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 84 epoch_requested: 84 NullEvt) class_id 2 prio 255 cost 10 e84) 2026-02-11T23:30:43.913+0000 7f41591a6640 7 osd.0 84 activate_map version 84 2026-02-11T23:30:43.913+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:43.913+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 84 epoch_requested: 84 NullEvt) class_id 2immediate prio 255 cost 10 e84) queued 2026-02-11T23:30:43.913+0000 7f4150194640 20 osd.0 84 agent_entry empty queue 2026-02-11T23:30:43.913+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:43.914+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 84 epoch_requested: 84 NullEvt) class_id 2immediate prio 255 cost 10 e84) pg 0x5978000 2026-02-11T23:30:43.914+0000 7f414f993640 10 osd.0 pg_epoch: 83 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 84 2026-02-11T23:30:43.914+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:43.914+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 84 epoch_requested: 84 NullEvt) class_id 2immediate prio 255 cost 10 e84) queued 2026-02-11T23:30:43.914+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:43.914+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 84 epoch_requested: 84 NullEvt) class_id 2immediate prio 255 cost 10 e84) pg 0x5946000 2026-02-11T23:30:43.914+0000 7f414f993640 20 osd.0:0.update_pg_epoch 4.0 83 -> 84 2026-02-11T23:30:43.914+0000 7f414f993640 10 osd.0 pg_epoch: 83 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:43.914+0000 7f414f993640 10 osd.0 pg_epoch: 84 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:43.914+0000 7f414b98b640 10 osd.0 pg_epoch: 83 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 84 2026-02-11T23:30:43.914+0000 7f414f993640 10 osd.0 pg_epoch: 84 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:43.914+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 1.0 83 -> 84 2026-02-11T23:30:43.914+0000 7f414f993640 20 osd.0 84 advance_pg new pool opts old pool opts 2026-02-11T23:30:43.914+0000 7f414b98b640 10 osd.0 pg_epoch: 83 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:43.914+0000 7f414f993640 10 osd.0 pg_epoch: 84 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_activate_map: epoch range: 84..84 2026-02-11T23:30:43.914+0000 7f414f993640 10 osd.0 pg_epoch: 84 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:43.914+0000 7f414b98b640 10 osd.0 pg_epoch: 84 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:43.914+0000 7f414f993640 20 osd.0 pg_epoch: 84 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 53 while current is 84 2026-02-11T23:30:43.914+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:43.914+0000 7f414b98b640 10 osd.0 pg_epoch: 84 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:43.914+0000 7f414f993640 10 osd.0 pg_epoch: 84 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 84 epoch_requested: 84 NullEvt 2026-02-11T23:30:43.914+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:43.914+0000 7f414b98b640 20 osd.0 84 advance_pg new pool opts old pool opts 2026-02-11T23:30:43.914+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:43.914+0000 7f414b98b640 10 osd.0 pg_epoch: 84 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_activate_map: epoch range: 84..84 2026-02-11T23:30:43.914+0000 7f414b98b640 10 osd.0 pg_epoch: 84 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:43.914+0000 7f414b98b640 20 osd.0 pg_epoch: 84 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 73 while current is 84 2026-02-11T23:30:43.914+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:43.914+0000 7f414b98b640 10 osd.0 pg_epoch: 84 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 84 epoch_requested: 84 NullEvt 2026-02-11T23:30:43.914+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:43.914+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:43.989+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 22 ==== pg_lease(4.0 pg_lease(ru 24.828193665s ub 32.828323364s int 16.000000000s) e83/83) ==== 58+0+0 (unknown 0 0 0) 0x5af8180 con 0x507ac00 2026-02-11T23:30:43.989+0000 7f416abfa640 15 osd.0 84 enqueue_peering_evt 4.0 epoch_sent: 83 epoch_requested: 83 MLease epoch 83 from osd.4 pg_lease(ru 24.828193665s ub 32.828323364s int 16.000000000s) 2026-02-11T23:30:43.989+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 83 epoch_requested: 83 MLease epoch 83 from osd.4 pg_lease(ru 24.828193665s ub 32.828323364s int 16.000000000s)) class_id 2 prio 255 cost 10 e83) 2026-02-11T23:30:43.989+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:43.989+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 83 epoch_requested: 83 MLease epoch 83 from osd.4 pg_lease(ru 24.828193665s ub 32.828323364s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e83) queued 2026-02-11T23:30:43.989+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:43.989+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 83 epoch_requested: 83 MLease epoch 83 from osd.4 pg_lease(ru 24.828193665s ub 32.828323364s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e83) pg 0x5978000 2026-02-11T23:30:43.989+0000 7f414f993640 10 osd.0 pg_epoch: 84 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 83 epoch_requested: 83 MLease epoch 83 from osd.4 pg_lease(ru 24.828193665s ub 32.828323364s int 16.000000000s) 2026-02-11T23:30:43.989+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:43.989+0000 7f414f993640 10 osd.0 pg_epoch: 84 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::proc_lease pg_lease(ru 24.828193665s ub 32.828323364s int 16.000000000s) 2026-02-11T23:30:43.989+0000 7f414f993640 20 osd.0 pg_epoch: 84 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::proc_lease peer_clock_delta_ub -9.836438179s -> ru 34.664630890s 2026-02-11T23:30:43.989+0000 7f414f993640 20 osd.0 pg_epoch: 84 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::proc_lease readable_until now 34.664630890s 2026-02-11T23:30:43.989+0000 7f414f993640 20 osd.0 pg_epoch: 84 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::proc_lease peer_clock_delta_lb -9.836782455s -> ruub 42.665103912s 2026-02-11T23:30:43.989+0000 7f414f993640 20 osd.0 pg_epoch: 84 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::proc_lease readable_until_ub now 42.665103912s 2026-02-11T23:30:43.989+0000 7f414f993640 20 osd.0 84 get_con_osd_cluster to osd.4 from_epoch 84 2026-02-11T23:30:43.989+0000 7f414f993640 20 osd.0 84 get_nextmap_reserved map_reservations: {84=1} 2026-02-11T23:30:43.989+0000 7f414f993640 20 osd.0 84 release_map epoch: 84 2026-02-11T23:30:43.989+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_lease_ack(4.0 pg_lease_ack(ruub 32.828323364s) e84/84) -- 0x3758420 con 0x507ac00 2026-02-11T23:30:43.989+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:43.989+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:44.040+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.1 127.0.0.1:0/3043114013 11 ==== osd_ping(ping e83 up_from 10 ping_stamp 2026-02-11T23:30:44.041000+0000/24.704746246s send_stamp 24.704746246s delta_ub -2.011399984s) ==== 2033+0+0 (crc 0 0 0) 0x4c9aa80 con 0x4aca400 2026-02-11T23:30:44.040+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.1 127.0.0.1:0/3043114013 11 ==== osd_ping(ping e83 up_from 10 ping_stamp 2026-02-11T23:30:44.041000+0000/24.704746246s send_stamp 24.704746246s delta_ub -2.011399984s) ==== 2033+0+0 (crc 0 0 0) 0x4c9be00 con 0x4aca000 2026-02-11T23:30:44.040+0000 7f416c3fd640 20 osd.0 84 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011749029s,-2.011399984s]) 2026-02-11T23:30:44.040+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e84 up_from 5 ping_stamp 2026-02-11T23:30:44.041000+0000/24.704746246s send_stamp 26.716493607s delta_ub 2.011749029s) -- 0x5af8600 con 0x4aca400 2026-02-11T23:30:44.040+0000 7f416c3fd640 20 osd.0 84 get_con_osd_cluster to osd.1 from_epoch 84 2026-02-11T23:30:44.040+0000 7f416c3fd640 20 osd.0 84 get_nextmap_reserved map_reservations: {84=1} 2026-02-11T23:30:44.040+0000 7f416c3fd640 20 osd.0 84 release_map epoch: 84 2026-02-11T23:30:44.040+0000 7f416c3fd640 10 osd.0 84 maybe_share_map: con v2:127.0.0.1:6812/3043114013 updating session's projected_epoch from 74 to ping map epoch of 83 2026-02-11T23:30:44.040+0000 7f416c3fd640 10 osd.0 84 maybe_share_map: con v2:127.0.0.1:6812/3043114013 map epoch 83 -> 84 (shared) 2026-02-11T23:30:44.040+0000 7f416c3fd640 10 osd.0 84 send_incremental_map epoch range: (84, 84] to v2:127.0.0.1:6812/3043114013 2026-02-11T23:30:44.040+0000 7f416c3fd640 10 osd.0 84 add_map_inc_bl 84 574 bytes 2026-02-11T23:30:44.040+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- osd_map(84..84 src has 1..84) -- 0x4a30700 con 0x4acb000 2026-02-11T23:30:44.040+0000 7f416cbfe640 20 osd.0 84 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011955261s,-2.011399984s]) 2026-02-11T23:30:44.040+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e84 up_from 5 ping_stamp 2026-02-11T23:30:44.041000+0000/24.704746246s send_stamp 26.716701508s delta_ub 2.011955261s) -- 0x4c9bb00 con 0x4aca000 2026-02-11T23:30:44.041+0000 7f416cbfe640 20 osd.0 84 get_con_osd_cluster to osd.1 from_epoch 84 2026-02-11T23:30:44.041+0000 7f416cbfe640 20 osd.0 84 get_nextmap_reserved map_reservations: {84=1} 2026-02-11T23:30:44.041+0000 7f416cbfe640 20 osd.0 84 release_map epoch: 84 2026-02-11T23:30:44.041+0000 7f416cbfe640 10 osd.0 84 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 84 is not newer than session's projected_epoch of 84 2026-02-11T23:30:44.299+0000 7f4162bea640 10 osd.0 84 tick_without_osd_lock 2026-02-11T23:30:44.299+0000 7f4162bea640 0 osd.0 84 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:44.299+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:44.299+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:44.299+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:44.299+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.430 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:44.299+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:44.300, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:44.299+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:44.299+0000 7f4162bea640 10 osd.0 84 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:44.299+0000 7f4162bea640 20 osd.0 84 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:44.299+0000 7f4162bea640 10 osd.0 84 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:44.300+0000 7f4162bea640 20 osd.0 84 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:44.327+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:30:44.327+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:30:44.327+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:14.328072+0000) 2026-02-11T23:30:44.410+0000 7f41633eb640 10 osd.0 84 tick 2026-02-11T23:30:44.410+0000 7f41633eb640 20 osd.0 84 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:30:44.664+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.4 127.0.0.1:0/3331151400 5 ==== osd_ping(ping e85 up_from 23 ping_stamp 2026-02-11T23:30:44.664812+0000/17.503580093s send_stamp 17.503580093s delta_ub -9.836365700s) ==== 2033+0+0 (crc 0 0 0) 0x5af8600 con 0x507a800 2026-02-11T23:30:44.664+0000 7f416c3fd640 20 osd.0 84 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836732864s,-9.836365700s]) 2026-02-11T23:30:44.664+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e84 up_from 5 ping_stamp 2026-02-11T23:30:44.664812+0000/17.503580093s send_stamp 27.340312958s delta_ub 9.836732864s) -- 0x4c9aa80 con 0x507a800 2026-02-11T23:30:44.664+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.4 127.0.0.1:0/3331151400 5 ==== osd_ping(ping e85 up_from 23 ping_stamp 2026-02-11T23:30:44.664812+0000/17.503580093s send_stamp 17.503580093s delta_ub -9.836365700s) ==== 2033+0+0 (crc 0 0 0) 0x5af8180 con 0x507a400 2026-02-11T23:30:44.664+0000 7f416c3fd640 20 osd.0 84 get_con_osd_cluster to osd.4 from_epoch 84 2026-02-11T23:30:44.664+0000 7f416c3fd640 20 osd.0 84 get_nextmap_reserved map_reservations: {84=1} 2026-02-11T23:30:44.664+0000 7f416c3fd640 20 osd.0 84 release_map epoch: 84 2026-02-11T23:30:44.664+0000 7f416c3fd640 10 osd.0 84 maybe_share_map: con v2:127.0.0.1:6836/3331151400 updating session's projected_epoch from 79 to ping map epoch of 85 2026-02-11T23:30:44.664+0000 7f416c3fd640 10 osd.0 84 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 84 is not newer than session's projected_epoch of 85 2026-02-11T23:30:44.664+0000 7f416abfa640 20 osd.0 84 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836769104s,-9.836365700s]) 2026-02-11T23:30:44.664+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e84 up_from 5 ping_stamp 2026-02-11T23:30:44.664812+0000/17.503580093s send_stamp 27.340349197s delta_ub 9.836769104s) -- 0x5af8480 con 0x507a400 2026-02-11T23:30:44.664+0000 7f416abfa640 20 osd.0 84 get_con_osd_cluster to osd.4 from_epoch 84 2026-02-11T23:30:44.664+0000 7f416abfa640 20 osd.0 84 get_nextmap_reserved map_reservations: {84=1} 2026-02-11T23:30:44.664+0000 7f416abfa640 20 osd.0 84 release_map epoch: 84 2026-02-11T23:30:44.664+0000 7f416abfa640 10 osd.0 84 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 84 is not newer than session's projected_epoch of 85 2026-02-11T23:30:44.664+0000 7f415aefe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 23 ==== osd_map(85..85 src has 1..85) ==== 635+0+0 (unknown 0 0 0) 0x4a30700 con 0x507ac00 2026-02-11T23:30:44.664+0000 7f415aefe640 20 osd.0 84 OSD::ms_dispatch: osd_map(85..85 src has 1..85) 2026-02-11T23:30:44.664+0000 7f415aefe640 20 osd.0 84 _dispatch 0x4a30700 osd_map(85..85 src has 1..85) 2026-02-11T23:30:44.664+0000 7f415aefe640 3 osd.0 84 handle_osd_map epochs [85,85], i have 84, src has [1,85] 2026-02-11T23:30:44.664+0000 7f415aefe640 10 osd.0 84 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:30:44.664+0000 7f415aefe640 10 osd.0 84 handle_osd_map got inc map for epoch 85 2026-02-11T23:30:44.664+0000 7f415aefe640 20 osd.0 84 got_full_map 85, nothing requested 2026-02-11T23:30:44.664+0000 7f415aefe640 10 osd.0 84 _track_pools_and_pg_num_changes recording new pool 9 pg_num 1 2026-02-11T23:30:44.664+0000 7f415aefe640 20 osd.0 84 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:44.664+0000 7f415aefe640 20 osd.0 84 handle_osd_map pg_num_history pg_num_history(e85 pg_nums {1={27=1},2={34=1},3={42=1},4={49=1},5={56=1},6={64=1},7={70=1},8={77=1},9={85=1}} deleted_pools ) 2026-02-11T23:30:44.664+0000 7f415aefe640 10 snap_mapper record_purged_snaps purged_snaps {85={}} 2026-02-11T23:30:44.664+0000 7f415aefe640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:44.664+0000 7f415aefe640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e85 maps [1~85] lci=[5,85] tlb=1) 2026-02-11T23:30:44.665+0000 7f41591a6640 10 osd.0 84 _committed_osd_maps 85..85 2026-02-11T23:30:44.665+0000 7f41591a6640 10 osd.0 84 advance to epoch 85 (<= last 85 <= newest_map 85) 2026-02-11T23:30:44.665+0000 7f41591a6640 20 osd.0 85 consume_map version 85 2026-02-11T23:30:44.665+0000 7f41591a6640 20 osd.0 84 await_reserved_maps epoch:85 2026-02-11T23:30:44.665+0000 7f41591a6640 20 osd.0 84 await_reserved_maps done 0 2026-02-11T23:30:44.665+0000 7f41591a6640 20 osd.0 85 consume_map 85 -- publish done 2026-02-11T23:30:44.665+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:30:44.665+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x37582c0 2026-02-11T23:30:44.665+0000 7f41591a6640 20 osd.0 85 identify_splits_and_merges 4.0 e84 to e85 2026-02-11T23:30:44.665+0000 7f41591a6640 20 osd.0 85 identify_splits_and_merges 4.0 e84 to e85 pg_nums {49=1} 2026-02-11T23:30:44.665+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3700dc0 2026-02-11T23:30:44.665+0000 7f41591a6640 20 osd.0 85 identify_splits_and_merges 1.0 e84 to e85 2026-02-11T23:30:44.665+0000 7f41591a6640 20 osd.0 85 identify_splits_and_merges 1.0 e84 to e85 pg_nums {27=1} 2026-02-11T23:30:44.665+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:44.665+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:44.665+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:44.665+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:44.665+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:44.665+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:44.665+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:44.665+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:44.665+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:44.665+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:44.665+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:44.665+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:44.665+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:44.665+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:44.665+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:44.665+0000 7f41591a6640 20 osd.0 85 prune_pg_created 2026-02-11T23:30:44.665+0000 7f41591a6640 10 osd.0:0.consume_map 85 (was 84) 2026-02-11T23:30:44.665+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:44.665+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:44.665+0000 7f41591a6640 10 osd.0:1.consume_map 85 (was 84) 2026-02-11T23:30:44.665+0000 7f41591a6640 10 osd.0:2.consume_map 85 (was 84) 2026-02-11T23:30:44.665+0000 7f41591a6640 10 osd.0:3.consume_map 85 (was 84) 2026-02-11T23:30:44.665+0000 7f41591a6640 10 osd.0:4.consume_map 85 (was 84) 2026-02-11T23:30:44.665+0000 7f41591a6640 10 osd.0:5.consume_map 85 (was 84) 2026-02-11T23:30:44.665+0000 7f41591a6640 10 osd.0:6.consume_map 85 (was 84) 2026-02-11T23:30:44.665+0000 7f41591a6640 10 osd.0:7.consume_map 85 (was 84) 2026-02-11T23:30:44.665+0000 7f41591a6640 10 osd.0 85 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:44.665+0000 7f41591a6640 15 osd.0 85 enqueue_peering_evt 4.0 epoch_sent: 85 epoch_requested: 85 NullEvt 2026-02-11T23:30:44.665+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 85 epoch_requested: 85 NullEvt) class_id 2 prio 255 cost 10 e85) 2026-02-11T23:30:44.665+0000 7f41591a6640 15 osd.0 85 enqueue_peering_evt 1.0 epoch_sent: 85 epoch_requested: 85 NullEvt 2026-02-11T23:30:44.665+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 85 epoch_requested: 85 NullEvt) class_id 2 prio 255 cost 10 e85) 2026-02-11T23:30:44.665+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:44.665+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 85 epoch_requested: 85 NullEvt) class_id 2immediate prio 255 cost 10 e85) queued 2026-02-11T23:30:44.665+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:44.665+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 85 epoch_requested: 85 NullEvt) class_id 2immediate prio 255 cost 10 e85) pg 0x5978000 2026-02-11T23:30:44.665+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:44.665+0000 7f414b98b640 10 osd.0 pg_epoch: 84 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 85 2026-02-11T23:30:44.665+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 4.0 84 -> 85 2026-02-11T23:30:44.665+0000 7f414b98b640 10 osd.0 pg_epoch: 84 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:44.665+0000 7f414b98b640 10 osd.0 pg_epoch: 85 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:44.665+0000 7f41591a6640 7 osd.0 85 activate_map version 85 2026-02-11T23:30:44.665+0000 7f414b98b640 10 osd.0 pg_epoch: 85 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:44.665+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:44.665+0000 7f414b98b640 20 osd.0 85 advance_pg new pool opts old pool opts 2026-02-11T23:30:44.665+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 85 epoch_requested: 85 NullEvt) class_id 2immediate prio 255 cost 10 e85) queued 2026-02-11T23:30:44.665+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:44.665+0000 7f414b98b640 10 osd.0 pg_epoch: 85 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_activate_map: epoch range: 85..85 2026-02-11T23:30:44.665+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 85 epoch_requested: 85 NullEvt) class_id 2immediate prio 255 cost 10 e85) pg 0x5946000 2026-02-11T23:30:44.665+0000 7f414b98b640 10 osd.0 pg_epoch: 85 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:44.665+0000 7f4150194640 20 osd.0 85 agent_entry empty queue 2026-02-11T23:30:44.665+0000 7f414b98b640 20 osd.0 pg_epoch: 85 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 53 while current is 85 2026-02-11T23:30:44.665+0000 7f414f993640 10 osd.0 pg_epoch: 84 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 85 2026-02-11T23:30:44.665+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 84 -> 85 2026-02-11T23:30:44.665+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:44.665+0000 7f414f993640 10 osd.0 pg_epoch: 84 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:44.665+0000 7f414b98b640 10 osd.0 pg_epoch: 85 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 85 epoch_requested: 85 NullEvt 2026-02-11T23:30:44.665+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:44.665+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:44.665+0000 7f414f993640 10 osd.0 pg_epoch: 85 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:44.665+0000 7f414f993640 10 osd.0 pg_epoch: 85 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:44.665+0000 7f414f993640 20 osd.0 85 advance_pg new pool opts old pool opts 2026-02-11T23:30:44.665+0000 7f414f993640 10 osd.0 pg_epoch: 85 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_activate_map: epoch range: 85..85 2026-02-11T23:30:44.665+0000 7f414f993640 10 osd.0 pg_epoch: 85 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:44.665+0000 7f414f993640 20 osd.0 pg_epoch: 85 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 73 while current is 85 2026-02-11T23:30:44.665+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:44.665+0000 7f414f993640 10 osd.0 pg_epoch: 85 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 85 epoch_requested: 85 NullEvt 2026-02-11T23:30:44.665+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:44.665+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:44.794+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.2 127.0.0.1:0/2365263801 5 ==== osd_ping(ping e85 up_from 14 ping_stamp 2026-02-11T23:30:44.794810+0000/22.903406143s send_stamp 22.903406143s delta_ub -4.566758633s) ==== 2033+0+0 (crc 0 0 0) 0x4c9bb00 con 0x4641000 2026-02-11T23:30:44.794+0000 7f416cbfe640 20 osd.0 85 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566894054s,-4.566758633s]) 2026-02-11T23:30:44.794+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e85 up_from 5 ping_stamp 2026-02-11T23:30:44.794810+0000/22.903406143s send_stamp 27.470300674s delta_ub 4.566894054s) -- 0x4c9be00 con 0x4641000 2026-02-11T23:30:44.794+0000 7f416cbfe640 20 osd.0 85 get_con_osd_cluster to osd.2 from_epoch 85 2026-02-11T23:30:44.794+0000 7f416cbfe640 20 osd.0 85 get_nextmap_reserved map_reservations: {85=1} 2026-02-11T23:30:44.794+0000 7f416cbfe640 20 osd.0 85 release_map epoch: 85 2026-02-11T23:30:44.794+0000 7f416cbfe640 10 osd.0 85 maybe_share_map: con v2:127.0.0.1:6820/2365263801 updating session's projected_epoch from 74 to ping map epoch of 85 2026-02-11T23:30:44.794+0000 7f416cbfe640 10 osd.0 85 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 85 is not newer than session's projected_epoch of 85 2026-02-11T23:30:44.794+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.2 127.0.0.1:0/2365263801 5 ==== osd_ping(ping e85 up_from 14 ping_stamp 2026-02-11T23:30:44.794810+0000/22.903406143s send_stamp 22.903406143s delta_ub -4.566758633s) ==== 2033+0+0 (crc 0 0 0) 0x4c9aa80 con 0x48cc000 2026-02-11T23:30:44.794+0000 7f416c3fd640 20 osd.0 85 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566914558s,-4.566758633s]) 2026-02-11T23:30:44.794+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e85 up_from 5 ping_stamp 2026-02-11T23:30:44.794810+0000/22.903406143s send_stamp 27.470321655s delta_ub 4.566914558s) -- 0x5af8600 con 0x48cc000 2026-02-11T23:30:44.794+0000 7f416c3fd640 20 osd.0 85 get_con_osd_cluster to osd.2 from_epoch 85 2026-02-11T23:30:44.794+0000 7f416c3fd640 20 osd.0 85 get_nextmap_reserved map_reservations: {85=1} 2026-02-11T23:30:44.794+0000 7f416c3fd640 20 osd.0 85 release_map epoch: 85 2026-02-11T23:30:44.794+0000 7f416c3fd640 10 osd.0 85 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 85 is not newer than session's projected_epoch of 85 2026-02-11T23:30:45.294+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.2 127.0.0.1:0/2365263801 6 ==== osd_ping(ping e86 up_from 14 ping_stamp 2026-02-11T23:30:45.295047+0000/23.403644562s send_stamp 23.403644562s delta_ub -4.566722393s) ==== 2033+0+0 (crc 0 0 0) 0x4c9be00 con 0x4641000 2026-02-11T23:30:45.294+0000 7f416cbfe640 20 osd.0 85 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566869736s,-4.566722393s]) 2026-02-11T23:30:45.294+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e85 up_from 5 ping_stamp 2026-02-11T23:30:45.295047+0000/23.403644562s send_stamp 27.970512390s delta_ub 4.566869736s) -- 0x4c9bb00 con 0x4641000 2026-02-11T23:30:45.294+0000 7f416cbfe640 20 osd.0 85 get_con_osd_cluster to osd.2 from_epoch 85 2026-02-11T23:30:45.294+0000 7f416cbfe640 20 osd.0 85 get_nextmap_reserved map_reservations: {85=1} 2026-02-11T23:30:45.294+0000 7f416cbfe640 20 osd.0 85 release_map epoch: 85 2026-02-11T23:30:45.294+0000 7f416cbfe640 10 osd.0 85 maybe_share_map: con v2:127.0.0.1:6820/2365263801 updating session's projected_epoch from 85 to ping map epoch of 86 2026-02-11T23:30:45.294+0000 7f416cbfe640 10 osd.0 85 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 85 is not newer than session's projected_epoch of 86 2026-02-11T23:30:45.294+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.2 127.0.0.1:0/2365263801 6 ==== osd_ping(ping e86 up_from 14 ping_stamp 2026-02-11T23:30:45.295047+0000/23.403644562s send_stamp 23.403644562s delta_ub -4.566722393s) ==== 2033+0+0 (crc 0 0 0) 0x5af8600 con 0x48cc000 2026-02-11T23:30:45.294+0000 7f416c3fd640 20 osd.0 85 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566914082s,-4.566722393s]) 2026-02-11T23:30:45.294+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e85 up_from 5 ping_stamp 2026-02-11T23:30:45.295047+0000/23.403644562s send_stamp 27.970558167s delta_ub 4.566914082s) -- 0x4c9aa80 con 0x48cc000 2026-02-11T23:30:45.294+0000 7f416c3fd640 20 osd.0 85 get_con_osd_cluster to osd.2 from_epoch 85 2026-02-11T23:30:45.294+0000 7f416c3fd640 20 osd.0 85 get_nextmap_reserved map_reservations: {85=1} 2026-02-11T23:30:45.294+0000 7f416c3fd640 20 osd.0 85 release_map epoch: 85 2026-02-11T23:30:45.294+0000 7f416c3fd640 10 osd.0 85 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 85 is not newer than session's projected_epoch of 86 2026-02-11T23:30:45.295+0000 7f415aefe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.2 v2:127.0.0.1:6820/2365263801 5 ==== osd_map(86..86 src has 1..86) ==== 296+0+0 (crc 0 0 0) 0x5955500 con 0x507b800 2026-02-11T23:30:45.295+0000 7f415aefe640 20 osd.0 85 OSD::ms_dispatch: osd_map(86..86 src has 1..86) 2026-02-11T23:30:45.295+0000 7f415aefe640 20 osd.0 85 _dispatch 0x5955500 osd_map(86..86 src has 1..86) 2026-02-11T23:30:45.295+0000 7f415aefe640 3 osd.0 85 handle_osd_map epochs [86,86], i have 85, src has [1,86] 2026-02-11T23:30:45.295+0000 7f415aefe640 10 osd.0 85 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:30:45.295+0000 7f415aefe640 10 osd.0 85 handle_osd_map got inc map for epoch 86 2026-02-11T23:30:45.295+0000 7f415aefe640 20 osd.0 85 got_full_map 86, nothing requested 2026-02-11T23:30:45.295+0000 7f415aefe640 20 osd.0 85 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:45.295+0000 7f415aefe640 20 osd.0 85 handle_osd_map pg_num_history pg_num_history(e86 pg_nums {1={27=1},2={34=1},3={42=1},4={49=1},5={56=1},6={64=1},7={70=1},8={77=1},9={85=1}} deleted_pools ) 2026-02-11T23:30:45.295+0000 7f415aefe640 10 snap_mapper record_purged_snaps purged_snaps {86={}} 2026-02-11T23:30:45.295+0000 7f415aefe640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:45.295+0000 7f415aefe640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e86 maps [1~86] lci=[5,86] tlb=1) 2026-02-11T23:30:45.295+0000 7f41591a6640 10 osd.0 85 _committed_osd_maps 86..86 2026-02-11T23:30:45.295+0000 7f41591a6640 10 osd.0 85 advance to epoch 86 (<= last 86 <= newest_map 86) 2026-02-11T23:30:45.295+0000 7f41591a6640 20 osd.0 86 consume_map version 86 2026-02-11T23:30:45.295+0000 7f41591a6640 20 osd.0 85 await_reserved_maps epoch:86 2026-02-11T23:30:45.295+0000 7f41591a6640 20 osd.0 85 await_reserved_maps done 0 2026-02-11T23:30:45.295+0000 7f41591a6640 20 osd.0 86 consume_map 86 -- publish done 2026-02-11T23:30:45.295+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:30:45.295+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x37582c0 2026-02-11T23:30:45.295+0000 7f41591a6640 20 osd.0 86 identify_splits_and_merges 4.0 e85 to e86 2026-02-11T23:30:45.295+0000 7f41591a6640 20 osd.0 86 identify_splits_and_merges 4.0 e85 to e86 pg_nums {49=1} 2026-02-11T23:30:45.295+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3700dc0 2026-02-11T23:30:45.295+0000 7f41591a6640 20 osd.0 86 identify_splits_and_merges 1.0 e85 to e86 2026-02-11T23:30:45.295+0000 7f41591a6640 20 osd.0 86 identify_splits_and_merges 1.0 e85 to e86 pg_nums {27=1} 2026-02-11T23:30:45.295+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:45.295+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:45.295+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:45.295+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:45.295+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:45.295+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:45.295+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:45.295+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:45.295+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:45.295+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:45.295+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:45.295+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:45.295+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:45.295+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:45.295+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:45.295+0000 7f41591a6640 20 osd.0 86 prune_pg_created 2026-02-11T23:30:45.295+0000 7f41591a6640 10 osd.0:0.consume_map 86 (was 85) 2026-02-11T23:30:45.295+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:45.295+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:45.295+0000 7f41591a6640 10 osd.0:1.consume_map 86 (was 85) 2026-02-11T23:30:45.295+0000 7f41591a6640 10 osd.0:2.consume_map 86 (was 85) 2026-02-11T23:30:45.295+0000 7f41591a6640 10 osd.0:3.consume_map 86 (was 85) 2026-02-11T23:30:45.295+0000 7f41591a6640 10 osd.0:4.consume_map 86 (was 85) 2026-02-11T23:30:45.295+0000 7f41591a6640 10 osd.0:5.consume_map 86 (was 85) 2026-02-11T23:30:45.295+0000 7f41591a6640 10 osd.0:6.consume_map 86 (was 85) 2026-02-11T23:30:45.295+0000 7f41591a6640 10 osd.0:7.consume_map 86 (was 85) 2026-02-11T23:30:45.295+0000 7f41591a6640 10 osd.0 86 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:45.295+0000 7f41591a6640 15 osd.0 86 enqueue_peering_evt 4.0 epoch_sent: 86 epoch_requested: 86 NullEvt 2026-02-11T23:30:45.295+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 86 epoch_requested: 86 NullEvt) class_id 2 prio 255 cost 10 e86) 2026-02-11T23:30:45.295+0000 7f41591a6640 15 osd.0 86 enqueue_peering_evt 1.0 epoch_sent: 86 epoch_requested: 86 NullEvt 2026-02-11T23:30:45.295+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 86 epoch_requested: 86 NullEvt) class_id 2 prio 255 cost 10 e86) 2026-02-11T23:30:45.295+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:45.295+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 86 epoch_requested: 86 NullEvt) class_id 2immediate prio 255 cost 10 e86) queued 2026-02-11T23:30:45.295+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:45.295+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 86 epoch_requested: 86 NullEvt) class_id 2immediate prio 255 cost 10 e86) pg 0x5978000 2026-02-11T23:30:45.295+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:45.295+0000 7f414b98b640 10 osd.0 pg_epoch: 85 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 86 2026-02-11T23:30:45.295+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 4.0 85 -> 86 2026-02-11T23:30:45.295+0000 7f414b98b640 10 osd.0 pg_epoch: 85 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:45.295+0000 7f414b98b640 10 osd.0 pg_epoch: 86 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:45.295+0000 7f41591a6640 7 osd.0 86 activate_map version 86 2026-02-11T23:30:45.295+0000 7f414b98b640 10 osd.0 pg_epoch: 86 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:45.295+0000 7f414b98b640 20 osd.0 86 advance_pg new pool opts old pool opts 2026-02-11T23:30:45.295+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:45.295+0000 7f414b98b640 10 osd.0 pg_epoch: 86 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_activate_map: epoch range: 86..86 2026-02-11T23:30:45.295+0000 7f414b98b640 10 osd.0 pg_epoch: 86 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:45.295+0000 7f4150194640 20 osd.0 86 agent_entry empty queue 2026-02-11T23:30:45.295+0000 7f414b98b640 20 osd.0 pg_epoch: 86 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 53 while current is 86 2026-02-11T23:30:45.295+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 86 epoch_requested: 86 NullEvt) class_id 2immediate prio 255 cost 10 e86) queued 2026-02-11T23:30:45.295+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:45.295+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:45.295+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 86 epoch_requested: 86 NullEvt) class_id 2immediate prio 255 cost 10 e86) pg 0x5946000 2026-02-11T23:30:45.295+0000 7f414b98b640 10 osd.0 pg_epoch: 86 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 86 epoch_requested: 86 NullEvt 2026-02-11T23:30:45.295+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:45.295+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:45.295+0000 7f414f993640 10 osd.0 pg_epoch: 85 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 86 2026-02-11T23:30:45.295+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 85 -> 86 2026-02-11T23:30:45.295+0000 7f414f993640 10 osd.0 pg_epoch: 85 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:45.295+0000 7f414f993640 10 osd.0 pg_epoch: 86 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:45.295+0000 7f414f993640 10 osd.0 pg_epoch: 86 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:45.295+0000 7f414f993640 20 osd.0 86 advance_pg new pool opts old pool opts 2026-02-11T23:30:45.295+0000 7f414f993640 10 osd.0 pg_epoch: 86 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_activate_map: epoch range: 86..86 2026-02-11T23:30:45.295+0000 7f414f993640 10 osd.0 pg_epoch: 86 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:45.295+0000 7f414f993640 20 osd.0 pg_epoch: 86 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 73 while current is 86 2026-02-11T23:30:45.295+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:45.295+0000 7f414f993640 10 osd.0 pg_epoch: 86 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 86 epoch_requested: 86 NullEvt 2026-02-11T23:30:45.295+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:45.295+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:45.327+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:30:45.327+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:30:45.327+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:15.328106+0000) 2026-02-11T23:30:45.349+0000 7f4162bea640 10 osd.0 86 tick_without_osd_lock 2026-02-11T23:30:45.349+0000 7f4162bea640 0 osd.0 86 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:45.349+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:45.349+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:45.349+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:45.349+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.430 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:45.349+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:45.349, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:45.349+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:45.349+0000 7f4162bea640 10 osd.0 86 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:45.349+0000 7f4162bea640 20 osd.0 86 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:45.349+0000 7f4162bea640 10 osd.0 86 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:45.349+0000 7f4162bea640 20 osd.0 86 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:45.436+0000 7f41633eb640 10 osd.0 86 tick 2026-02-11T23:30:45.436+0000 7f41633eb640 20 osd.0 86 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:30:46.273+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 19 ==== pg_lease(1.0 pg_lease(ru 34.937133789s ub 42.937232971s int 16.000000000s) e92/92) ==== 58+0+0 (unknown 0 0 0) 0x5af8480 con 0x4acb000 2026-02-11T23:30:46.273+0000 7f416abfa640 15 osd.0 86 enqueue_peering_evt 1.0 epoch_sent: 92 epoch_requested: 92 MLease epoch 92 from osd.1 pg_lease(ru 34.937133789s ub 42.937232971s int 16.000000000s) 2026-02-11T23:30:46.273+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 92 epoch_requested: 92 MLease epoch 92 from osd.1 pg_lease(ru 34.937133789s ub 42.937232971s int 16.000000000s)) class_id 2 prio 255 cost 10 e92) 2026-02-11T23:30:46.273+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:46.273+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 92 epoch_requested: 92 MLease epoch 92 from osd.1 pg_lease(ru 34.937133789s ub 42.937232971s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e92) queued 2026-02-11T23:30:46.273+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:46.273+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 92 epoch_requested: 92 MLease epoch 92 from osd.1 pg_lease(ru 34.937133789s ub 42.937232971s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e92) pg 0x5946000 2026-02-11T23:30:46.273+0000 7f414b98b640 20 osd.0 op_wq _add_slot_waiter 1.0 peering, item epoch is 92, will wait on OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 92 epoch_requested: 92 MLease epoch 92 from osd.1 pg_lease(ru 34.937133789s ub 42.937232971s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e92) 2026-02-11T23:30:46.273+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:46.273+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:46.328+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:30:46.328+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:30:46.328+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:16.328174+0000) 2026-02-11T23:30:46.394+0000 7f4162bea640 10 osd.0 86 tick_without_osd_lock 2026-02-11T23:30:46.394+0000 7f4162bea640 0 osd.0 86 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:46.394+0000 7f4162bea640 20 osd.0 86 tick_without_osd_lock max_waiting_epoch 92, requesting new map 2026-02-11T23:30:46.394+0000 7f4162bea640 10 monclient: _renew_subs 2026-02-11T23:30:46.394+0000 7f4162bea640 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:30:46.394+0000 7f4162bea640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> v2:127.0.0.1:7180/0 -- mon_subscribe({osdmap=87}) -- 0x3814680 con 0x4acbc00 2026-02-11T23:30:46.394+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:46.394+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:46.394+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:46.394+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.430 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:46.394+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:46.395, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:46.394+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:46.394+0000 7f4162bea640 10 osd.0 86 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:46.394+0000 7f4162bea640 20 osd.0 86 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:46.394+0000 7f4162bea640 10 osd.0 86 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:46.394+0000 7f4162bea640 20 osd.0 86 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:46.395+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 44 ==== osd_map(87..92 src has 1..92) ==== 3554+0+0 (crc 0 0 0) 0x5954c40 con 0x4acbc00 2026-02-11T23:30:46.395+0000 7f415cf02640 20 osd.0 86 OSD::ms_dispatch: osd_map(87..92 src has 1..92) 2026-02-11T23:30:46.395+0000 7f415cf02640 20 osd.0 86 _dispatch 0x5954c40 osd_map(87..92 src has 1..92) 2026-02-11T23:30:46.395+0000 7f415cf02640 3 osd.0 86 handle_osd_map epochs [87,92], i have 86, src has [1,92] 2026-02-11T23:30:46.395+0000 7f415cf02640 10 osd.0 86 handle_osd_map: received 0 full maps and 6 incremental maps 2026-02-11T23:30:46.395+0000 7f415cf02640 10 osd.0 86 handle_osd_map got inc map for epoch 87 2026-02-11T23:30:46.395+0000 7f415cf02640 20 osd.0 86 got_full_map 87, nothing requested 2026-02-11T23:30:46.395+0000 7f415cf02640 10 osd.0 86 handle_osd_map got inc map for epoch 88 2026-02-11T23:30:46.395+0000 7f415cf02640 20 osd.0 86 got_full_map 88, nothing requested 2026-02-11T23:30:46.395+0000 7f415cf02640 10 osd.0 86 handle_osd_map got inc map for epoch 89 2026-02-11T23:30:46.395+0000 7f415cf02640 20 osd.0 86 got_full_map 89, nothing requested 2026-02-11T23:30:46.395+0000 7f415cf02640 10 osd.0 86 handle_osd_map got inc map for epoch 90 2026-02-11T23:30:46.395+0000 7f415cf02640 20 osd.0 86 got_full_map 90, nothing requested 2026-02-11T23:30:46.395+0000 7f415cf02640 10 osd.0 86 handle_osd_map got inc map for epoch 91 2026-02-11T23:30:46.395+0000 7f415cf02640 20 osd.0 86 got_full_map 91, nothing requested 2026-02-11T23:30:46.395+0000 7f415cf02640 10 osd.0 86 handle_osd_map got inc map for epoch 92 2026-02-11T23:30:46.395+0000 7f415cf02640 20 osd.0 86 got_full_map 92, nothing requested 2026-02-11T23:30:46.395+0000 7f415cf02640 10 osd.0 86 _track_pools_and_pg_num_changes recording new pool 10 pg_num 1 2026-02-11T23:30:46.395+0000 7f415cf02640 20 osd.0 86 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:46.395+0000 7f415cf02640 20 osd.0 86 handle_osd_map pg_num_history pg_num_history(e92 pg_nums {1={27=1},2={34=1},3={42=1},4={49=1},5={56=1},6={64=1},7={70=1},8={77=1},9={85=1},10={92=1}} deleted_pools ) 2026-02-11T23:30:46.395+0000 7f415cf02640 10 snap_mapper record_purged_snaps purged_snaps {87={},88={},89={},90={},91={},92={}} 2026-02-11T23:30:46.395+0000 7f415cf02640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:46.395+0000 7f415cf02640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e92 maps [1~92] lci=[5,92] tlb=1) 2026-02-11T23:30:46.395+0000 7f41591a6640 10 osd.0 86 _committed_osd_maps 87..92 2026-02-11T23:30:46.395+0000 7f41591a6640 10 osd.0 86 advance to epoch 87 (<= last 92 <= newest_map 92) 2026-02-11T23:30:46.395+0000 7f41591a6640 10 osd.0 87 advance to epoch 88 (<= last 92 <= newest_map 92) 2026-02-11T23:30:46.395+0000 7f41591a6640 10 osd.0 88 advance to epoch 89 (<= last 92 <= newest_map 92) 2026-02-11T23:30:46.395+0000 7f41591a6640 10 osd.0 89 advance to epoch 90 (<= last 92 <= newest_map 92) 2026-02-11T23:30:46.395+0000 7f41591a6640 10 osd.0 90 advance to epoch 91 (<= last 92 <= newest_map 92) 2026-02-11T23:30:46.395+0000 7f41591a6640 10 osd.0 91 advance to epoch 92 (<= last 92 <= newest_map 92) 2026-02-11T23:30:46.395+0000 7f41591a6640 20 osd.0 92 consume_map version 92 2026-02-11T23:30:46.395+0000 7f41591a6640 20 osd.0 86 await_reserved_maps epoch:92 2026-02-11T23:30:46.395+0000 7f41591a6640 20 osd.0 86 await_reserved_maps done 0 2026-02-11T23:30:46.395+0000 7f41591a6640 20 osd.0 92 consume_map 92 -- publish done 2026-02-11T23:30:46.395+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:30:46.395+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x37582c0 2026-02-11T23:30:46.395+0000 7f41591a6640 20 osd.0 92 identify_splits_and_merges 4.0 e86 to e92 2026-02-11T23:30:46.395+0000 7f41591a6640 20 osd.0 92 identify_splits_and_merges 4.0 e86 to e92 pg_nums {49=1} 2026-02-11T23:30:46.395+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3700dc0 2026-02-11T23:30:46.395+0000 7f41591a6640 20 osd.0 92 identify_splits_and_merges 1.0 e86 to e92 2026-02-11T23:30:46.395+0000 7f41591a6640 20 osd.0 92 identify_splits_and_merges 1.0 e86 to e92 pg_nums {27=1} 2026-02-11T23:30:46.395+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:46.395+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:46.395+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:46.395+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:46.395+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:46.395+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:46.395+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:46.395+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:46.395+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:46.395+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:46.395+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:46.395+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:46.395+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:46.395+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:46.395+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:46.395+0000 7f41591a6640 20 osd.0 92 prune_pg_created 2026-02-11T23:30:46.395+0000 7f41591a6640 10 osd.0:0.consume_map 92 (was 86) 2026-02-11T23:30:46.395+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:46.395+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:46.395+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 pending_peering first epoch 92 <= 92, requeueing 2026-02-11T23:30:46.395+0000 7f41591a6640 20 osd.0:0._wake_pg_slot _wake_pg_slot 1.0 to_process <> waiting <> waiting_peering {92=} 2026-02-11T23:30:46.395+0000 7f41591a6640 10 osd.0:1.consume_map 92 (was 86) 2026-02-11T23:30:46.395+0000 7f41591a6640 10 osd.0:2.consume_map 92 (was 86) 2026-02-11T23:30:46.396+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:46.396+0000 7f41591a6640 10 osd.0:3.consume_map 92 (was 86) 2026-02-11T23:30:46.396+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 92 epoch_requested: 92 MLease epoch 92 from osd.1 pg_lease(ru 34.937133789s ub 42.937232971s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e92) queued 2026-02-11T23:30:46.396+0000 7f41591a6640 10 osd.0:4.consume_map 92 (was 86) 2026-02-11T23:30:46.396+0000 7f41591a6640 10 osd.0:5.consume_map 92 (was 86) 2026-02-11T23:30:46.396+0000 7f41591a6640 10 osd.0:6.consume_map 92 (was 86) 2026-02-11T23:30:46.396+0000 7f41591a6640 10 osd.0:7.consume_map 92 (was 86) 2026-02-11T23:30:46.396+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:46.396+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 92 epoch_requested: 92 MLease epoch 92 from osd.1 pg_lease(ru 34.937133789s ub 42.937232971s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e92) pg 0x5946000 2026-02-11T23:30:46.396+0000 7f41591a6640 10 osd.0 92 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:46.396+0000 7f41591a6640 15 osd.0 92 enqueue_peering_evt 4.0 epoch_sent: 92 epoch_requested: 92 NullEvt 2026-02-11T23:30:46.396+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 92 epoch_requested: 92 NullEvt) class_id 2 prio 255 cost 10 e92) 2026-02-11T23:30:46.396+0000 7f414b98b640 10 osd.0 pg_epoch: 86 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 87 2026-02-11T23:30:46.396+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 1.0 86 -> 87 2026-02-11T23:30:46.396+0000 7f41591a6640 15 osd.0 92 enqueue_peering_evt 1.0 epoch_sent: 92 epoch_requested: 92 NullEvt 2026-02-11T23:30:46.396+0000 7f414b98b640 10 osd.0 pg_epoch: 86 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:46.396+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 92 epoch_requested: 92 NullEvt) class_id 2 prio 255 cost 10 e92) 2026-02-11T23:30:46.396+0000 7f41591a6640 7 osd.0 92 activate_map version 92 2026-02-11T23:30:46.396+0000 7f414b98b640 10 osd.0 pg_epoch: 87 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:46.396+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:46.396+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 92 epoch_requested: 92 NullEvt) class_id 2immediate prio 255 cost 10 e92) queued 2026-02-11T23:30:46.396+0000 7f414b98b640 10 osd.0 pg_epoch: 87 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:46.396+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:46.396+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 92 epoch_requested: 92 NullEvt) class_id 2immediate prio 255 cost 10 e92) pg 0x5978000 2026-02-11T23:30:46.396+0000 7f414b98b640 20 osd.0 92 advance_pg new pool opts old pool opts 2026-02-11T23:30:46.396+0000 7f414b98b640 10 osd.0 pg_epoch: 87 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 88 2026-02-11T23:30:46.396+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 1.0 87 -> 88 2026-02-11T23:30:46.396+0000 7f4150194640 20 osd.0 92 agent_entry empty queue 2026-02-11T23:30:46.396+0000 7f414f993640 10 osd.0 pg_epoch: 86 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 87 2026-02-11T23:30:46.396+0000 7f414f993640 20 osd.0:0.update_pg_epoch 4.0 86 -> 87 2026-02-11T23:30:46.396+0000 7f414b98b640 10 osd.0 pg_epoch: 87 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:46.396+0000 7f414f993640 10 osd.0 pg_epoch: 86 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:46.396+0000 7f414b98b640 10 osd.0 pg_epoch: 88 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:46.396+0000 7f414f993640 10 osd.0 pg_epoch: 87 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:46.396+0000 7f414b98b640 10 osd.0 pg_epoch: 88 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:46.396+0000 7f414b98b640 20 osd.0 92 advance_pg new pool opts old pool opts 2026-02-11T23:30:46.396+0000 7f414f993640 10 osd.0 pg_epoch: 87 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:46.396+0000 7f414b98b640 10 osd.0 pg_epoch: 88 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 89 2026-02-11T23:30:46.396+0000 7f414f993640 20 osd.0 92 advance_pg new pool opts old pool opts 2026-02-11T23:30:46.396+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 1.0 88 -> 89 2026-02-11T23:30:46.396+0000 7f414f993640 10 osd.0 pg_epoch: 87 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 88 2026-02-11T23:30:46.396+0000 7f414f993640 20 osd.0:0.update_pg_epoch 4.0 87 -> 88 2026-02-11T23:30:46.396+0000 7f414b98b640 10 osd.0 pg_epoch: 88 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:46.396+0000 7f414f993640 10 osd.0 pg_epoch: 87 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:46.396+0000 7f414f993640 10 osd.0 pg_epoch: 88 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:46.396+0000 7f414f993640 10 osd.0 pg_epoch: 88 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:46.396+0000 7f414b98b640 10 osd.0 pg_epoch: 89 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:46.396+0000 7f414b98b640 10 osd.0 pg_epoch: 89 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:46.396+0000 7f414b98b640 20 osd.0 92 advance_pg new pool opts old pool opts 2026-02-11T23:30:46.396+0000 7f414b98b640 10 osd.0 pg_epoch: 89 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 90 2026-02-11T23:30:46.396+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 1.0 89 -> 90 2026-02-11T23:30:46.396+0000 7f414f993640 20 osd.0 92 advance_pg new pool opts old pool opts 2026-02-11T23:30:46.396+0000 7f414f993640 10 osd.0 pg_epoch: 88 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 89 2026-02-11T23:30:46.396+0000 7f414f993640 20 osd.0:0.update_pg_epoch 4.0 88 -> 89 2026-02-11T23:30:46.396+0000 7f414b98b640 10 osd.0 pg_epoch: 89 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:46.396+0000 7f414f993640 10 osd.0 pg_epoch: 88 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:46.396+0000 7f414b98b640 10 osd.0 pg_epoch: 90 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:46.396+0000 7f414f993640 10 osd.0 pg_epoch: 89 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:46.396+0000 7f414b98b640 10 osd.0 pg_epoch: 90 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:46.396+0000 7f414b98b640 20 osd.0 92 advance_pg new pool opts old pool opts 2026-02-11T23:30:46.396+0000 7f414b98b640 10 osd.0 pg_epoch: 90 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 91 2026-02-11T23:30:46.396+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 1.0 90 -> 91 2026-02-11T23:30:46.396+0000 7f414f993640 10 osd.0 pg_epoch: 89 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:46.396+0000 7f414b98b640 10 osd.0 pg_epoch: 90 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:46.396+0000 7f414f993640 20 osd.0 92 advance_pg new pool opts old pool opts 2026-02-11T23:30:46.396+0000 7f414b98b640 10 osd.0 pg_epoch: 91 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:46.396+0000 7f414f993640 10 osd.0 pg_epoch: 89 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 90 2026-02-11T23:30:46.396+0000 7f414f993640 20 osd.0:0.update_pg_epoch 4.0 89 -> 90 2026-02-11T23:30:46.396+0000 7f414b98b640 10 osd.0 pg_epoch: 91 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:46.396+0000 7f414b98b640 20 osd.0 92 advance_pg new pool opts old pool opts 2026-02-11T23:30:46.396+0000 7f414b98b640 10 osd.0 pg_epoch: 91 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 92 2026-02-11T23:30:46.396+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 1.0 91 -> 92 2026-02-11T23:30:46.396+0000 7f414f993640 10 osd.0 pg_epoch: 89 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:46.396+0000 7f414b98b640 10 osd.0 pg_epoch: 91 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:46.396+0000 7f414b98b640 10 osd.0 pg_epoch: 92 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:46.396+0000 7f414b98b640 10 osd.0 pg_epoch: 92 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:46.396+0000 7f414b98b640 20 osd.0 92 advance_pg new pool opts old pool opts 2026-02-11T23:30:46.396+0000 7f414f993640 10 osd.0 pg_epoch: 90 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:46.396+0000 7f414b98b640 10 osd.0 pg_epoch: 92 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_activate_map: epoch range: 87..92 2026-02-11T23:30:46.396+0000 7f414b98b640 10 osd.0 pg_epoch: 92 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:46.396+0000 7f414b98b640 20 osd.0 pg_epoch: 92 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 73 while current is 92 2026-02-11T23:30:46.396+0000 7f414f993640 10 osd.0 pg_epoch: 90 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:46.396+0000 7f414f993640 20 osd.0 92 advance_pg new pool opts old pool opts 2026-02-11T23:30:46.396+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:46.396+0000 7f414f993640 10 osd.0 pg_epoch: 90 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 91 2026-02-11T23:30:46.396+0000 7f414f993640 20 osd.0:0.update_pg_epoch 4.0 90 -> 91 2026-02-11T23:30:46.396+0000 7f414b98b640 10 osd.0 pg_epoch: 92 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 92 epoch_requested: 92 MLease epoch 92 from osd.1 pg_lease(ru 34.937133789s ub 42.937232971s int 16.000000000s) 2026-02-11T23:30:46.396+0000 7f414f993640 10 osd.0 pg_epoch: 90 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:46.396+0000 7f414b98b640 10 osd.0 pg_epoch: 92 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::proc_lease pg_lease(ru 34.937133789s ub 42.937232971s int 16.000000000s) 2026-02-11T23:30:46.396+0000 7f414f993640 10 osd.0 pg_epoch: 91 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:46.396+0000 7f414b98b640 20 osd.0 pg_epoch: 92 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::proc_lease peer_clock_delta_ub -2.011399984s -> ru 36.948535919s 2026-02-11T23:30:46.396+0000 7f414f993640 10 osd.0 pg_epoch: 91 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:46.396+0000 7f414f993640 20 osd.0 92 advance_pg new pool opts old pool opts 2026-02-11T23:30:46.396+0000 7f414b98b640 20 osd.0 pg_epoch: 92 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::proc_lease readable_until now 36.948535919s 2026-02-11T23:30:46.396+0000 7f414f993640 10 osd.0 pg_epoch: 91 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 92 2026-02-11T23:30:46.396+0000 7f414f993640 20 osd.0:0.update_pg_epoch 4.0 91 -> 92 2026-02-11T23:30:46.396+0000 7f414f993640 10 osd.0 pg_epoch: 91 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:46.396+0000 7f414b98b640 20 osd.0 pg_epoch: 92 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::proc_lease peer_clock_delta_lb -2.011955261s -> ruub 44.949188232s 2026-02-11T23:30:46.396+0000 7f414f993640 10 osd.0 pg_epoch: 92 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:46.396+0000 7f414b98b640 20 osd.0 pg_epoch: 92 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::proc_lease readable_until_ub now 44.949188232s 2026-02-11T23:30:46.396+0000 7f414f993640 10 osd.0 pg_epoch: 92 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:46.396+0000 7f414b98b640 20 osd.0 92 get_con_osd_cluster to osd.1 from_epoch 92 2026-02-11T23:30:46.396+0000 7f414f993640 20 osd.0 92 advance_pg new pool opts old pool opts 2026-02-11T23:30:46.396+0000 7f414b98b640 20 osd.0 92 get_nextmap_reserved map_reservations: {92=1} 2026-02-11T23:30:46.396+0000 7f414f993640 10 osd.0 pg_epoch: 92 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_activate_map: epoch range: 87..92 2026-02-11T23:30:46.396+0000 7f414b98b640 20 osd.0 92 release_map epoch: 92 2026-02-11T23:30:46.396+0000 7f414f993640 10 osd.0 pg_epoch: 92 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:46.396+0000 7f414f993640 20 osd.0 pg_epoch: 92 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 53 while current is 92 2026-02-11T23:30:46.396+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:46.396+0000 7f414f993640 10 osd.0 pg_epoch: 92 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 92 epoch_requested: 92 NullEvt 2026-02-11T23:30:46.396+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:46.396+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_lease_ack(1.0 pg_lease_ack(ruub 42.937232971s) e92/92) -- 0x3759340 con 0x4acb000 2026-02-11T23:30:46.396+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:46.396+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 92 epoch_requested: 92 NullEvt) class_id 2immediate prio 255 cost 10 e92) queued 2026-02-11T23:30:46.396+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:46.396+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:46.396+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:46.396+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 92 epoch_requested: 92 NullEvt) class_id 2immediate prio 255 cost 10 e92) pg 0x5946000 2026-02-11T23:30:46.396+0000 7f414f993640 10 osd.0 pg_epoch: 92 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 92 epoch_requested: 92 NullEvt 2026-02-11T23:30:46.396+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:46.396+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:46.444+0000 7f41633eb640 10 osd.0 92 tick 2026-02-11T23:30:46.444+0000 7f41633eb640 20 osd.0 92 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:30:46.528+0000 7f4147983640 5 osd.0 92 heartbeat osd_stat(store_statfs(0x384000/0x0/0x384000, data 0x5b341/0xd4000, compress 0x0/0x0/0x0, omap 0x880a, meta 0x1a477f6), peers [1,2,3,4] op hist []) 2026-02-11T23:30:46.528+0000 7f4147983640 20 osd.0 92 check_full_status cur ratio 0, physical ratio 0, new state none 2026-02-11T23:30:46.528+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6816/3043114013,v1:127.0.0.1:6817/3043114013] -- osd_ping(ping e92 up_from 5 ping_stamp 2026-02-11T23:30:46.529123+0000/29.204528809s send_stamp 29.204528809s delta_ub 2.011955261s) -- 0x5af8780 con 0x48c4400 2026-02-11T23:30:46.528+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6814/3043114013,v1:127.0.0.1:6815/3043114013] -- osd_ping(ping e92 up_from 5 ping_stamp 2026-02-11T23:30:46.529123+0000/29.204528809s send_stamp 29.204528809s delta_ub 2.011955261s) -- 0x5af8900 con 0x4640c00 2026-02-11T23:30:46.528+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6824/2365263801,v1:127.0.0.1:6825/2365263801] -- osd_ping(ping e92 up_from 5 ping_stamp 2026-02-11T23:30:46.529123+0000/29.204528809s send_stamp 29.204528809s delta_ub 4.566914082s) -- 0x5af8a80 con 0x48c4000 2026-02-11T23:30:46.528+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6822/2365263801,v1:127.0.0.1:6823/2365263801] -- osd_ping(ping e92 up_from 5 ping_stamp 2026-02-11T23:30:46.529123+0000/29.204528809s send_stamp 29.204528809s delta_ub 4.566914082s) -- 0x5af8c00 con 0x48cd000 2026-02-11T23:30:46.528+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6832/2699377081,v1:127.0.0.1:6833/2699377081] -- osd_ping(ping e92 up_from 5 ping_stamp 2026-02-11T23:30:46.529123+0000/29.204528809s send_stamp 29.204528809s delta_ub 7.219561577s) -- 0x5af8d80 con 0x48cd400 2026-02-11T23:30:46.528+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6830/2699377081,v1:127.0.0.1:6831/2699377081] -- osd_ping(ping e92 up_from 5 ping_stamp 2026-02-11T23:30:46.529123+0000/29.204528809s send_stamp 29.204528809s delta_ub 7.219561577s) -- 0x5af8300 con 0x48cd800 2026-02-11T23:30:46.528+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6840/3331151400,v1:127.0.0.1:6841/3331151400] -- osd_ping(ping e92 up_from 5 ping_stamp 2026-02-11T23:30:46.529123+0000/29.204528809s send_stamp 29.204528809s delta_ub 9.836769104s) -- 0x4c9ac00 con 0x507bc00 2026-02-11T23:30:46.528+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6838/3331151400,v1:127.0.0.1:6839/3331151400] -- osd_ping(ping e92 up_from 5 ping_stamp 2026-02-11T23:30:46.529123+0000/29.204528809s send_stamp 29.204528809s delta_ub 9.836769104s) -- 0x4c9af00 con 0x5282000 2026-02-11T23:30:46.528+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6816/3043114013 8 ==== osd_ping(ping_reply e92 up_from 10 ping_stamp 2026-02-11T23:30:46.529123+0000/29.204528809s send_stamp 27.192958832s delta_ub -2.011570215s) ==== 2033+0+0 (crc 0 0 0) 0x4c9ac00 con 0x48c4400 2026-02-11T23:30:46.529+0000 7f416abfa640 20 osd.0 92 get_con_osd_cluster to osd.1 from_epoch 92 2026-02-11T23:30:46.529+0000 7f416abfa640 20 osd.0 92 get_nextmap_reserved map_reservations: {92=1} 2026-02-11T23:30:46.529+0000 7f416abfa640 20 osd.0 92 release_map epoch: 92 2026-02-11T23:30:46.529+0000 7f416abfa640 10 osd.0 92 maybe_share_map: con v2:127.0.0.1:6812/3043114013 updating session's projected_epoch from 84 to ping map epoch of 92 2026-02-11T23:30:46.529+0000 7f416abfa640 10 osd.0 92 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 92 is not newer than session's projected_epoch of 92 2026-02-11T23:30:46.529+0000 7f416abfa640 20 osd.0 92 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011765957s,-2.011570215s]) 2026-02-11T23:30:46.529+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6814/3043114013 8 ==== osd_ping(ping_reply e92 up_from 10 ping_stamp 2026-02-11T23:30:46.529123+0000/29.204528809s send_stamp 27.193006516s delta_ub -2.011523008s) ==== 2033+0+0 (crc 0 0 0) 0x4c9af00 con 0x4640c00 2026-02-11T23:30:46.529+0000 7f416cbfe640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6824/2365263801 7 ==== osd_ping(ping_reply e91 up_from 14 ping_stamp 2026-02-11T23:30:46.529123+0000/29.204528809s send_stamp 24.637842178s delta_ub -4.566688061s) ==== 2033+0+0 (crc 0 0 0) 0x5af8a80 con 0x48c4000 2026-02-11T23:30:46.529+0000 7f416c3fd640 20 osd.0 92 get_con_osd_cluster to osd.1 from_epoch 92 2026-02-11T23:30:46.529+0000 7f416c3fd640 20 osd.0 92 get_nextmap_reserved map_reservations: {92=1} 2026-02-11T23:30:46.529+0000 7f416c3fd640 20 osd.0 92 release_map epoch: 92 2026-02-11T23:30:46.529+0000 7f416c3fd640 10 osd.0 92 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 92 is not newer than session's projected_epoch of 92 2026-02-11T23:30:46.529+0000 7f416c3fd640 20 osd.0 92 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011739016s,-2.011523008s]) 2026-02-11T23:30:46.529+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6840/3331151400 6 ==== osd_ping(ping_reply e92 up_from 23 ping_stamp 2026-02-11T23:30:46.529123+0000/29.204528809s send_stamp 19.368040085s delta_ub -9.836488724s) ==== 2033+0+0 (crc 0 0 0) 0x4c9ac00 con 0x507bc00 2026-02-11T23:30:46.529+0000 7f416abfa640 20 osd.0 92 get_con_osd_cluster to osd.4 from_epoch 92 2026-02-11T23:30:46.529+0000 7f416abfa640 20 osd.0 92 get_nextmap_reserved map_reservations: {92=1} 2026-02-11T23:30:46.529+0000 7f416abfa640 20 osd.0 92 release_map epoch: 92 2026-02-11T23:30:46.529+0000 7f416abfa640 10 osd.0 92 maybe_share_map: con v2:127.0.0.1:6836/3331151400 updating session's projected_epoch from 85 to ping map epoch of 92 2026-02-11T23:30:46.529+0000 7f416abfa640 10 osd.0 92 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 92 is not newer than session's projected_epoch of 92 2026-02-11T23:30:46.529+0000 7f416abfa640 20 osd.0 92 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836732864s,-9.836488724s]) 2026-02-11T23:30:46.529+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6822/2365263801 7 ==== osd_ping(ping_reply e91 up_from 14 ping_stamp 2026-02-11T23:30:46.529123+0000/29.204528809s send_stamp 24.637878418s delta_ub -4.566651821s) ==== 2033+0+0 (crc 0 0 0) 0x4c9ac00 con 0x48cd000 2026-02-11T23:30:46.529+0000 7f416abfa640 20 osd.0 92 get_con_osd_cluster to osd.2 from_epoch 92 2026-02-11T23:30:46.529+0000 7f416abfa640 20 osd.0 92 get_nextmap_reserved map_reservations: {92=1} 2026-02-11T23:30:46.529+0000 7f416abfa640 20 osd.0 92 release_map epoch: 92 2026-02-11T23:30:46.529+0000 7f416abfa640 10 osd.0 92 maybe_share_map: con v2:127.0.0.1:6820/2365263801 updating session's projected_epoch from 86 to ping map epoch of 91 2026-02-11T23:30:46.529+0000 7f416abfa640 10 osd.0 92 maybe_share_map: con v2:127.0.0.1:6820/2365263801 map epoch 91 -> 92 (shared) 2026-02-11T23:30:46.529+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6838/3331151400 6 ==== osd_ping(ping_reply e92 up_from 23 ping_stamp 2026-02-11T23:30:46.529123+0000/29.204528809s send_stamp 19.368061066s delta_ub -9.836468697s) ==== 2033+0+0 (crc 0 0 0) 0x4c9af00 con 0x5282000 2026-02-11T23:30:46.529+0000 7f416abfa640 10 osd.0 92 send_incremental_map epoch range: (92, 92] to v2:127.0.0.1:6820/2365263801 2026-02-11T23:30:46.529+0000 7f416abfa640 10 osd.0 92 add_map_inc_bl 92 592 bytes 2026-02-11T23:30:46.529+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6820/2365263801,v1:127.0.0.1:6821/2365263801] -- osd_map(92..92 src has 1..92) -- 0x4a31500 con 0x507b800 2026-02-11T23:30:46.529+0000 7f416abfa640 20 osd.0 92 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566919327s,-4.566651821s]) 2026-02-11T23:30:46.529+0000 7f416c3fd640 20 osd.0 92 get_con_osd_cluster to osd.4 from_epoch 92 2026-02-11T23:30:46.529+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6832/2699377081 7 ==== osd_ping(ping_reply e91 up_from 19 ping_stamp 2026-02-11T23:30:46.529123+0000/29.204528809s send_stamp 21.985198975s delta_ub -7.219332218s) ==== 2033+0+0 (crc 0 0 0) 0x4c9ac00 con 0x48cd400 2026-02-11T23:30:46.529+0000 7f416c3fd640 20 osd.0 92 get_nextmap_reserved map_reservations: {92=1} 2026-02-11T23:30:46.529+0000 7f416c3fd640 20 osd.0 92 release_map epoch: 92 2026-02-11T23:30:46.529+0000 7f416c3fd640 10 osd.0 92 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 92 is not newer than session's projected_epoch of 92 2026-02-11T23:30:46.529+0000 7f416c3fd640 20 osd.0 92 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836857796s,-9.836468697s]) 2026-02-11T23:30:46.529+0000 7f416cbfe640 20 osd.0 92 get_con_osd_cluster to osd.2 from_epoch 92 2026-02-11T23:30:46.529+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6830/2699377081 7 ==== osd_ping(ping_reply e91 up_from 19 ping_stamp 2026-02-11T23:30:46.529123+0000/29.204528809s send_stamp 21.985246658s delta_ub -7.219282150s) ==== 2033+0+0 (crc 0 0 0) 0x4c9af00 con 0x48cd800 2026-02-11T23:30:46.529+0000 7f416cbfe640 20 osd.0 92 get_nextmap_reserved map_reservations: {92=1} 2026-02-11T23:30:46.529+0000 7f416cbfe640 20 osd.0 92 release_map epoch: 92 2026-02-11T23:30:46.529+0000 7f416cbfe640 10 osd.0 92 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 92 is not newer than session's projected_epoch of 92 2026-02-11T23:30:46.529+0000 7f416cbfe640 20 osd.0 92 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.567098141s,-4.566688061s]) 2026-02-11T23:30:46.529+0000 7f416abfa640 20 osd.0 92 get_con_osd_cluster to osd.3 from_epoch 92 2026-02-11T23:30:46.529+0000 7f416abfa640 20 osd.0 92 get_nextmap_reserved map_reservations: {92=1} 2026-02-11T23:30:46.529+0000 7f416abfa640 20 osd.0 92 release_map epoch: 92 2026-02-11T23:30:46.529+0000 7f416abfa640 10 osd.0 92 maybe_share_map: con v2:127.0.0.1:6828/2699377081 updating session's projected_epoch from 83 to ping map epoch of 91 2026-02-11T23:30:46.529+0000 7f416abfa640 10 osd.0 92 maybe_share_map: con v2:127.0.0.1:6828/2699377081 map epoch 91 -> 92 (shared) 2026-02-11T23:30:46.529+0000 7f416abfa640 10 osd.0 92 send_incremental_map epoch range: (92, 92] to v2:127.0.0.1:6828/2699377081 2026-02-11T23:30:46.529+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6828/2699377081,v1:127.0.0.1:6829/2699377081] -- osd_map(92..92 src has 1..92) -- 0x4a64fc0 con 0x48ccc00 2026-02-11T23:30:46.529+0000 7f416abfa640 20 osd.0 92 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219763279s,-7.219332218s]) 2026-02-11T23:30:46.529+0000 7f416c3fd640 20 osd.0 92 get_con_osd_cluster to osd.3 from_epoch 92 2026-02-11T23:30:46.529+0000 7f416c3fd640 20 osd.0 92 get_nextmap_reserved map_reservations: {92=1} 2026-02-11T23:30:46.529+0000 7f416c3fd640 20 osd.0 92 release_map epoch: 92 2026-02-11T23:30:46.529+0000 7f416c3fd640 10 osd.0 92 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 92 is not newer than session's projected_epoch of 92 2026-02-11T23:30:46.529+0000 7f416c3fd640 20 osd.0 92 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219734669s,-7.219282150s]) 2026-02-11T23:30:46.530+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.2 v2:127.0.0.1:6820/2365263801 6 ==== pg_query2(10.0 10.0 query(info 0'0 epoch_sent 92) e92/92) ==== 158+0+0 (crc 0 0 0) 0x59681e0 con 0x507b800 2026-02-11T23:30:46.530+0000 7f416cbfe640 15 osd.0 92 enqueue_peering_evt 10.0 epoch_sent: 92 epoch_requested: 92 MQuery 10.0 from 2 query_epoch 92 query: query(info 0'0 epoch_sent 92) 2026-02-11T23:30:46.530+0000 7f416cbfe640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 92 epoch_requested: 92 MQuery 10.0 from 2 query_epoch 92 query: query(info 0'0 epoch_sent 92)) class_id 2 prio 255 cost 10 e92) 2026-02-11T23:30:46.530+0000 7f414b98b640 20 osd.0 op_wq(0) _process 10.0 (new) to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:46.530+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 92 epoch_requested: 92 MQuery 10.0 from 2 query_epoch 92 query: query(info 0'0 epoch_sent 92)) class_id 2immediate prio 255 cost 10 e92) queued 2026-02-11T23:30:46.530+0000 7f414b98b640 20 osd.0 op_wq(0) _process 10.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:46.530+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 92 epoch_requested: 92 MQuery 10.0 from 2 query_epoch 92 query: query(info 0'0 epoch_sent 92)) class_id 2immediate prio 255 cost 10 e92) pg 0 2026-02-11T23:30:46.530+0000 7f414b98b640 10 osd.0 92 handle_pg_query_nopg 10.0 2026-02-11T23:30:46.530+0000 7f414b98b640 10 osd.0 92 pg 10.0 dne 2026-02-11T23:30:46.530+0000 7f414b98b640 20 osd.0 92 get_con_osd_cluster to osd.2 from_epoch 92 2026-02-11T23:30:46.530+0000 7f414b98b640 20 osd.0 92 get_nextmap_reserved map_reservations: {92=1} 2026-02-11T23:30:46.530+0000 7f414b98b640 20 osd.0 92 release_map epoch: 92 2026-02-11T23:30:46.530+0000 7f414b98b640 10 osd.0 92 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 92 is not newer than session's projected_epoch of 92 2026-02-11T23:30:46.530+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6820/2365263801,v1:127.0.0.1:6821/2365263801] -- pg_notify2(10.0 10.0 (query:92 sent:92 10.0( DNE empty local-lis/les=0/0 n=0 ec=0/0 lis/c=0/0 les/c/f=0/0/0 sis=0) ([0,0] all_participants={} intervals=[])) e92/92) -- 0x3738e00 con 0x507b800 2026-02-11T23:30:46.530+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:46.530+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:46.582+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.2 v2:127.0.0.1:6820/2365263801 7 ==== PGlog(10.0 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 40.691310883s int 16.000000000s) e93/92) ==== 1172+0+0 (crc 0 0 0) 0x4a1a800 con 0x507b800 2026-02-11T23:30:46.582+0000 7f416cbfe640 15 osd.0 92 enqueue_peering_evt 10.0 epoch_sent: 93 epoch_requested: 92 MLogRec from 2 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 40.691310883s int 16.000000000s) +create_info 2026-02-11T23:30:46.582+0000 7f416cbfe640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 93 epoch_requested: 92 MLogRec from 2 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 40.691310883s int 16.000000000s) +create_info) class_id 2 prio 255 cost 10 e93) 2026-02-11T23:30:46.582+0000 7f414f993640 20 osd.0 op_wq(0) _process 10.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:46.582+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 93 epoch_requested: 92 MLogRec from 2 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 40.691310883s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e93) queued 2026-02-11T23:30:46.582+0000 7f414f993640 20 osd.0 op_wq(0) _process 10.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:46.582+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 93 epoch_requested: 92 MLogRec from 2 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 40.691310883s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e93) pg 0 2026-02-11T23:30:46.582+0000 7f414f993640 20 osd.0 op_wq(0) _process 10.0 map 93 > 92 2026-02-11T23:30:46.582+0000 7f414f993640 20 osd.0 op_wq _add_slot_waiter 10.0 peering, item epoch is 93, will wait on OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 93 epoch_requested: 92 MLogRec from 2 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 40.691310883s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e93) 2026-02-11T23:30:46.582+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:46.582+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:46.940+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.1 127.0.0.1:0/3043114013 12 ==== osd_ping(ping e93 up_from 10 ping_stamp 2026-02-11T23:30:46.941181+0000/27.604927063s send_stamp 27.604927063s delta_ub -2.011523008s) ==== 2033+0+0 (crc 0 0 0) 0x5af8a80 con 0x4aca000 2026-02-11T23:30:46.940+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.1 127.0.0.1:0/3043114013 12 ==== osd_ping(ping e93 up_from 10 ping_stamp 2026-02-11T23:30:46.941181+0000/27.604927063s send_stamp 27.604927063s delta_ub -2.011523008s) ==== 2033+0+0 (crc 0 0 0) 0x4c9af00 con 0x4aca400 2026-02-11T23:30:46.940+0000 7f416cbfe640 20 osd.0 92 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011741161s,-2.011523008s]) 2026-02-11T23:30:46.940+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e92 up_from 5 ping_stamp 2026-02-11T23:30:46.941181+0000/27.604927063s send_stamp 29.616666794s delta_ub 2.011741161s) -- 0x4c9be00 con 0x4aca000 2026-02-11T23:30:46.940+0000 7f416cbfe640 20 osd.0 92 get_con_osd_cluster to osd.1 from_epoch 92 2026-02-11T23:30:46.940+0000 7f416cbfe640 20 osd.0 92 get_nextmap_reserved map_reservations: {92=1} 2026-02-11T23:30:46.940+0000 7f416cbfe640 20 osd.0 92 release_map epoch: 92 2026-02-11T23:30:46.940+0000 7f416cbfe640 10 osd.0 92 maybe_share_map: con v2:127.0.0.1:6812/3043114013 updating session's projected_epoch from 92 to ping map epoch of 93 2026-02-11T23:30:46.940+0000 7f416cbfe640 10 osd.0 92 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 92 is not newer than session's projected_epoch of 93 2026-02-11T23:30:46.940+0000 7f416c3fd640 20 osd.0 92 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011759043s,-2.011523008s]) 2026-02-11T23:30:46.940+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e92 up_from 5 ping_stamp 2026-02-11T23:30:46.941181+0000/27.604927063s send_stamp 29.616685867s delta_ub 2.011759043s) -- 0x5af8300 con 0x4aca400 2026-02-11T23:30:46.940+0000 7f416c3fd640 20 osd.0 92 get_con_osd_cluster to osd.1 from_epoch 92 2026-02-11T23:30:46.940+0000 7f416c3fd640 20 osd.0 92 get_nextmap_reserved map_reservations: {92=1} 2026-02-11T23:30:46.940+0000 7f416c3fd640 20 osd.0 92 release_map epoch: 92 2026-02-11T23:30:46.940+0000 7f416c3fd640 10 osd.0 92 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 92 is not newer than session's projected_epoch of 93 2026-02-11T23:30:46.941+0000 7f415aefe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 20 ==== osd_map(93..93 src has 1..93) ==== 296+0+0 (unknown 0 0 0) 0x5954000 con 0x4acb000 2026-02-11T23:30:46.941+0000 7f415aefe640 20 osd.0 92 OSD::ms_dispatch: osd_map(93..93 src has 1..93) 2026-02-11T23:30:46.941+0000 7f415aefe640 20 osd.0 92 _dispatch 0x5954000 osd_map(93..93 src has 1..93) 2026-02-11T23:30:46.941+0000 7f415aefe640 3 osd.0 92 handle_osd_map epochs [93,93], i have 92, src has [1,93] 2026-02-11T23:30:46.941+0000 7f415aefe640 10 osd.0 92 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:30:46.941+0000 7f415aefe640 10 osd.0 92 handle_osd_map got inc map for epoch 93 2026-02-11T23:30:46.941+0000 7f415aefe640 20 osd.0 92 got_full_map 93, nothing requested 2026-02-11T23:30:46.941+0000 7f415aefe640 20 osd.0 92 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:46.941+0000 7f415aefe640 20 osd.0 92 handle_osd_map pg_num_history pg_num_history(e93 pg_nums {1={27=1},2={34=1},3={42=1},4={49=1},5={56=1},6={64=1},7={70=1},8={77=1},9={85=1},10={92=1}} deleted_pools ) 2026-02-11T23:30:46.941+0000 7f415aefe640 10 snap_mapper record_purged_snaps purged_snaps {93={}} 2026-02-11T23:30:46.941+0000 7f415aefe640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:46.941+0000 7f415aefe640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e93 maps [1~93] lci=[5,93] tlb=1) 2026-02-11T23:30:46.941+0000 7f41591a6640 10 osd.0 92 _committed_osd_maps 93..93 2026-02-11T23:30:46.941+0000 7f41591a6640 10 osd.0 92 advance to epoch 93 (<= last 93 <= newest_map 93) 2026-02-11T23:30:46.941+0000 7f41591a6640 20 osd.0 93 consume_map version 93 2026-02-11T23:30:46.941+0000 7f41591a6640 20 osd.0 92 await_reserved_maps epoch:93 2026-02-11T23:30:46.941+0000 7f41591a6640 20 osd.0 92 await_reserved_maps done 0 2026-02-11T23:30:46.941+0000 7f41591a6640 20 osd.0 93 consume_map 93 -- publish done 2026-02-11T23:30:46.941+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 3 slots 2026-02-11T23:30:46.941+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:10.0slot:0x37594a0 2026-02-11T23:30:46.941+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot 10.0 has no pg and waiting_for_split 2026-02-11T23:30:46.941+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x37582c0 2026-02-11T23:30:46.941+0000 7f41591a6640 20 osd.0 93 identify_splits_and_merges 4.0 e92 to e93 2026-02-11T23:30:46.941+0000 7f41591a6640 20 osd.0 93 identify_splits_and_merges 4.0 e92 to e93 pg_nums {49=1} 2026-02-11T23:30:46.941+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3700dc0 2026-02-11T23:30:46.941+0000 7f41591a6640 20 osd.0 93 identify_splits_and_merges 1.0 e92 to e93 2026-02-11T23:30:46.941+0000 7f41591a6640 20 osd.0 93 identify_splits_and_merges 1.0 e92 to e93 pg_nums {27=1} 2026-02-11T23:30:46.941+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:46.941+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:46.941+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:46.941+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:46.941+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:46.941+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:46.941+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:46.941+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:46.941+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:46.941+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:46.941+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:46.941+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:46.941+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:46.941+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:46.941+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:46.941+0000 7f41591a6640 20 osd.0 93 prune_pg_created 2026-02-11T23:30:46.941+0000 7f41591a6640 10 osd.0:0.consume_map 93 (was 92) 2026-02-11T23:30:46.941+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 10.0 2026-02-11T23:30:46.941+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 10.0 pending_peering first epoch 93 <= 93, requeueing 2026-02-11T23:30:46.941+0000 7f41591a6640 20 osd.0:0._wake_pg_slot _wake_pg_slot 10.0 to_process <> waiting <> waiting_peering {93=} 2026-02-11T23:30:46.941+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:46.941+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:46.941+0000 7f41591a6640 10 osd.0:1.consume_map 93 (was 92) 2026-02-11T23:30:46.941+0000 7f41591a6640 10 osd.0:2.consume_map 93 (was 92) 2026-02-11T23:30:46.941+0000 7f41591a6640 10 osd.0:3.consume_map 93 (was 92) 2026-02-11T23:30:46.941+0000 7f41591a6640 10 osd.0:4.consume_map 93 (was 92) 2026-02-11T23:30:46.941+0000 7f41591a6640 10 osd.0:5.consume_map 93 (was 92) 2026-02-11T23:30:46.941+0000 7f41591a6640 10 osd.0:6.consume_map 93 (was 92) 2026-02-11T23:30:46.941+0000 7f41591a6640 10 osd.0:7.consume_map 93 (was 92) 2026-02-11T23:30:46.941+0000 7f414f993640 20 osd.0 op_wq(0) _process 10.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:46.941+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 93 epoch_requested: 92 MLogRec from 2 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 40.691310883s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e93) queued 2026-02-11T23:30:46.941+0000 7f414f993640 20 osd.0 op_wq(0) _process 10.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:46.941+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 93 epoch_requested: 92 MLogRec from 2 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 40.691310883s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e93) pg 0 2026-02-11T23:30:46.941+0000 7f414f993640 20 osd.0 op_wq(0) _process 10.0 no pg, should create on OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 93 epoch_requested: 92 MLogRec from 2 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 40.691310883s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e93) 2026-02-11T23:30:46.941+0000 7f414f993640 10 osd.0 93 _make_pg 10.0 2026-02-11T23:30:46.941+0000 7f414f993640 10 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:10, shard:-1, shard_prefix: , prefixes: {}] SnapMapper 2026-02-11T23:30:46.941+0000 7f414f993640 20 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:10, shard:-1, shard_prefix: , prefixes: {}] update_bits new_bits: 0 2026-02-11T23:30:46.941+0000 7f414f993640 20 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:10, shard:-1, shard_prefix: , prefixes: {"000000000000000A."}] update_bits prefix updated 2026-02-11T23:30:46.941+0000 7f414f993640 10 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:10, shard:-1, shard_prefix: , prefixes: {"000000000000000A."}] reset_prefix_itr::from <0> to ::update_bits 2026-02-11T23:30:46.941+0000 7f414f993640 5 osd.0 pg_epoch: 92 pg[10.0(unlocked)] enter Initial 2026-02-11T23:30:46.941+0000 7f414f993640 20 osd.0 pg_epoch: 92 pg[10.0(unlocked)] enter NotTrimming 2026-02-11T23:30:46.941+0000 7f414f993640 10 osd.0 pg_epoch: 92 pg[10.0(unlocked)] scrubber: FSM: -- state -->> NotActive 2026-02-11T23:30:46.941+0000 7f414f993640 10 osd.0 pg_epoch: 92 pg[10.0( DNE empty local-lis/les=0/0 n=0 ec=0/0 lis/c=0/0 les/c/f=0/0/0 sis=0) [] r=-1 lpr=0 crt=0'0 unknown mbc={}] PeeringState::init init role 2 up [2,1,0] acting [2,1,0] history ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92 past_intervals ([0,0] all_participants={} intervals=[]) 2026-02-11T23:30:46.941+0000 7f414f993640 20 osd.0 pg_epoch: 92 pg[10.0( empty local-lis/les=0/0 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=0 crt=0'0 unknown mbc={}] PeeringState::on_new_interval 2026-02-11T23:30:46.941+0000 7f414f993640 20 osd.0 pg_epoch: 92 pg[10.0( empty local-lis/les=0/0 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=0 crt=0'0 unknown mbc={}] PeeringState::on_new_interval upacting_features 0x3f0ffffffffdffff from [2,1,0]+[2,1,0] 2026-02-11T23:30:46.941+0000 7f414f993640 20 osd.0 pg_epoch: 92 pg[10.0( empty local-lis/les=0/0 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=0 crt=0'0 unknown mbc={}] PeeringState::on_new_interval checking missing set deletes flag. missing = missing(0 may_include_deletes = 1) 2026-02-11T23:30:46.941+0000 7f414f993640 10 osd.0 pg_epoch: 92 pg[10.0( empty local-lis/les=0/0 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=0 crt=0'0 unknown mbc={}] PeeringState::init_hb_stamps now [0x5685960] 2026-02-11T23:30:46.941+0000 7f414f993640 10 osd.0 pg_epoch: 92 pg[10.0( empty local-lis/les=0/0 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=0 crt=0'0 unknown mbc={}] PeeringState::on_new_interval prior_readable_until_ub 0.000000000s (mnow 29.617609024s + 0.000000000s) 2026-02-11T23:30:46.941+0000 7f414f993640 10 osd.0 pg_epoch: 92 pg[10.0( empty local-lis/les=0/0 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=0 crt=0'0 unknown mbc={}] cancel_recovery 2026-02-11T23:30:46.941+0000 7f414f993640 10 osd.0 pg_epoch: 92 pg[10.0( empty local-lis/les=0/0 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=0 crt=0'0 unknown mbc={}] clear_recovery_state 2026-02-11T23:30:46.941+0000 7f414f993640 15 osd.0 pg_epoch: 92 pg[10.0( empty local-lis/les=0/0 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=0 crt=0'0 unknown mbc={}] _clear_recovery_state 2026-02-11T23:30:46.941+0000 7f414f993640 10 osd.0 pg_epoch: 92 pg[10.0( empty local-lis/les=0/0 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=0 crt=0'0 unknown mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:46.941+0000 7f414f993640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:46.941+0000 7f414f993640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:46.941+0000 7f414f993640 6 write_log_and_missing with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, writeout_from: 4294967295'18446744073709551615, trimmed: , trimmed_dups: , clear_divergent_priors: 0 2026-02-11T23:30:46.941+0000 7f414f993640 10 _write_log_and_missing clearing up to 0'0 dirty_to_dups=0'0 dirty_from_dups=4294967295'18446744073709551615 write_from_dups=4294967295'18446744073709551615 trimmed_dups.size()=0 2026-02-11T23:30:46.941+0000 7f414f993640 10 _write_log_and_missing going to encode log.dups.size()=0 2026-02-11T23:30:46.941+0000 7f414f993640 10 _write_log_and_missing 1st round encoded log.dups.size()=0 2026-02-11T23:30:46.941+0000 7f414f993640 10 _write_log_and_missing 2st round encoded log.dups.size()=0 2026-02-11T23:30:46.941+0000 7f414f993640 10 end of _write_log_and_missing 2026-02-11T23:30:46.941+0000 7f414f993640 10 osd.0 pg_epoch: 92 pg[10.0( empty local-lis/les=0/0 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=0 crt=0'0 unknown mbc={}] handle_initialize 2026-02-11T23:30:46.941+0000 7f414f993640 5 osd.0 pg_epoch: 92 pg[10.0( empty local-lis/les=0/0 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=0 crt=0'0 unknown mbc={}] exit Initial 0.000098 0 0.000000 2026-02-11T23:30:46.941+0000 7f414f993640 5 osd.0 pg_epoch: 92 pg[10.0( empty local-lis/les=0/0 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=0 crt=0'0 unknown mbc={}] enter Reset 2026-02-11T23:30:46.941+0000 7f414f993640 20 osd.0 pg_epoch: 92 pg[10.0( empty local-lis/les=0/0 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=0 crt=0'0 unknown mbc={}] PeeringState::set_last_peering_reset set_last_peering_reset 92 2026-02-11T23:30:46.941+0000 7f414f993640 10 osd.0 pg_epoch: 92 pg[10.0( empty local-lis/les=0/0 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 unknown mbc={}] PeeringState::set_last_peering_reset Clearing blocked outgoing recovery messages 2026-02-11T23:30:46.941+0000 7f414f993640 10 osd.0 pg_epoch: 92 pg[10.0( empty local-lis/les=0/0 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 unknown mbc={}] PeeringState::set_last_peering_reset Not blocking outgoing recovery messages 2026-02-11T23:30:46.941+0000 7f414f993640 10 osd.0 pg_epoch: 92 pg[10.0( empty local-lis/les=0/0 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 unknown mbc={}] handle_activate_map: epoch range: 92..92 2026-02-11T23:30:46.941+0000 7f414f993640 10 osd.0 pg_epoch: 92 pg[10.0( empty local-lis/les=0/0 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 unknown mbc={}] PeeringState::activate_map 2026-02-11T23:30:46.941+0000 7f414f993640 5 osd.0 pg_epoch: 92 pg[10.0( empty local-lis/les=0/0 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 unknown mbc={}] exit Reset 0.000016 1 0.000016 2026-02-11T23:30:46.941+0000 7f414f993640 5 osd.0 pg_epoch: 92 pg[10.0( empty local-lis/les=0/0 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 unknown mbc={}] enter Started 2026-02-11T23:30:46.941+0000 7f414f993640 5 osd.0 pg_epoch: 92 pg[10.0( empty local-lis/les=0/0 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 unknown mbc={}] enter Start 2026-02-11T23:30:46.941+0000 7f414f993640 1 osd.0 pg_epoch: 92 pg[10.0( empty local-lis/les=0/0 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 unknown mbc={}] state: transitioning to Stray 2026-02-11T23:30:46.941+0000 7f414f993640 5 osd.0 pg_epoch: 92 pg[10.0( empty local-lis/les=0/0 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 unknown mbc={}] exit Start 0.000003 0 0.000000 2026-02-11T23:30:46.941+0000 7f414f993640 5 osd.0 pg_epoch: 92 pg[10.0( empty local-lis/les=0/0 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 unknown mbc={}] enter Started/Stray 2026-02-11T23:30:46.941+0000 7f414f993640 20 osd.0 pg_epoch: 92 pg[10.0( empty local-lis/les=0/0 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 unknown mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 92 while current is 92 2026-02-11T23:30:46.941+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:46.941+0000 7f414f993640 10 osd.0 pg_epoch: 92 pg[10.0( empty local-lis/les=0/0 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 unknown mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:30:46.942+0000 7f414f993640 10 osd.0 93 handle_pg_create_info new pg pg[10.0( empty local-lis/les=0/0 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 unknown mbc={}] 2026-02-11T23:30:46.942+0000 7f414f993640 10 osd.0:0._attach_pg 10.0 0x4aa0000 2026-02-11T23:30:46.942+0000 7f414f993640 20 osd.0:0._wake_pg_slot _wake_pg_slot 10.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:46.942+0000 7f414f993640 20 osd.0 93 identify_splits_and_merges 10.0 e92 to e93 2026-02-11T23:30:46.942+0000 7f414f993640 20 osd.0 93 identify_splits_and_merges 10.0 e92 to e93 pg_nums {92=1} 2026-02-11T23:30:46.942+0000 7f414f993640 10 osd.0:0._prime_splits 2026-02-11T23:30:46.942+0000 7f414f993640 10 osd.0 pg_epoch: 92 pg[10.0( empty local-lis/les=0/0 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 unknown mbc={}] handle_advance_map: 93 2026-02-11T23:30:46.942+0000 7f41591a6640 10 osd.0 93 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:46.942+0000 7f414f993640 20 osd.0:0.update_pg_epoch 10.0 92 -> 93 2026-02-11T23:30:46.942+0000 7f414f993640 10 osd.0 pg_epoch: 92 pg[10.0( empty local-lis/les=0/0 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 unknown mbc={}] PeeringState::advance_map handle_advance_map [2,1,0]/[2,1,0] -- 2/2 2026-02-11T23:30:46.942+0000 7f41591a6640 15 osd.0 93 enqueue_peering_evt 10.0 epoch_sent: 93 epoch_requested: 93 NullEvt 2026-02-11T23:30:46.942+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:46.942+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 93 epoch_requested: 93 NullEvt) class_id 2 prio 255 cost 10 e93) 2026-02-11T23:30:46.942+0000 7f414f993640 10 osd.0 pg_epoch: 93 pg[10.0( empty local-lis/les=0/0 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 unknown mbc={}] state: Started advmap 2026-02-11T23:30:46.942+0000 7f414f993640 10 osd.0 pg_epoch: 93 pg[10.0( empty local-lis/les=0/0 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 unknown mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:46.942+0000 7f414f993640 20 osd.0 93 advance_pg new pool opts old pool opts 2026-02-11T23:30:46.942+0000 7f414f993640 10 osd.0 pg_epoch: 93 pg[10.0( empty local-lis/les=0/0 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 unknown mbc={}] handle_activate_map: epoch range: 93..93 2026-02-11T23:30:46.942+0000 7f414b98b640 20 osd.0 op_wq(0) _process 10.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:46.942+0000 7f414f993640 10 osd.0 pg_epoch: 93 pg[10.0( empty local-lis/les=0/0 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 unknown mbc={}] PeeringState::activate_map 2026-02-11T23:30:46.942+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 93 epoch_requested: 93 NullEvt) class_id 2immediate prio 255 cost 10 e93) queued 2026-02-11T23:30:46.942+0000 7f414f993640 20 osd.0 pg_epoch: 93 pg[10.0( empty local-lis/les=0/0 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 unknown mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 92 while current is 93 2026-02-11T23:30:46.942+0000 7f41591a6640 15 osd.0 93 enqueue_peering_evt 4.0 epoch_sent: 93 epoch_requested: 93 NullEvt 2026-02-11T23:30:46.942+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 93 epoch_requested: 93 NullEvt) class_id 2 prio 255 cost 10 e93) 2026-02-11T23:30:46.942+0000 7f41591a6640 15 osd.0 93 enqueue_peering_evt 1.0 epoch_sent: 93 epoch_requested: 93 NullEvt 2026-02-11T23:30:46.942+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:46.942+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 93 epoch_requested: 93 NullEvt) class_id 2 prio 255 cost 10 e93) 2026-02-11T23:30:46.942+0000 7f414f993640 10 osd.0 pg_epoch: 93 pg[10.0( empty local-lis/les=0/0 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 unknown mbc={}] do_peering_event: epoch_sent: 93 epoch_requested: 92 MLogRec from 2 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 40.691310883s int 16.000000000s) +create_info 2026-02-11T23:30:46.942+0000 7f41591a6640 7 osd.0 93 activate_map version 93 2026-02-11T23:30:46.942+0000 7f414f993640 10 osd.0 pg_epoch: 93 pg[10.0( empty local-lis/les=0/0 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 unknown mbc={}] state: got info+log from osd.2 10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) log((0'0,0'0], crt=0'0) 2026-02-11T23:30:46.942+0000 7f414f993640 10 merge_log log((0'0,0'0], crt=0'0) from osd.2 into log((0'0,0'0], crt=0'0) 2026-02-11T23:30:46.942+0000 7f414f993640 5 merge_log_dups log.dups.size()=0olog.dups.size()=0 2026-02-11T23:30:46.942+0000 7f414f993640 5 end of merge_log_dups changed=0 log.dups.size()=0 olog.dups.size()=0 2026-02-11T23:30:46.942+0000 7f414f993640 10 merge_log result log((0'0,0'0], crt=0'0) missing(0 may_include_deletes = 1) changed=0 2026-02-11T23:30:46.942+0000 7f4150194640 20 osd.0 93 agent_entry empty queue 2026-02-11T23:30:46.942+0000 7f414f993640 10 osd.0 pg_epoch: 93 pg[10.0( empty local-lis/les=0/0 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 unknown mbc={}] PeeringState::proc_lease pg_lease(ru 0.000000000s ub 40.691310883s int 16.000000000s) 2026-02-11T23:30:46.942+0000 7f414f993640 20 osd.0 pg_epoch: 93 pg[10.0( empty local-lis/les=0/0 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 unknown mbc={}] PeeringState::proc_lease peer_clock_delta_lb -4.567098141s -> ruub 45.258411407s 2026-02-11T23:30:46.942+0000 7f414f993640 20 osd.0 pg_epoch: 93 pg[10.0( empty local-lis/les=0/0 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 unknown mbc={}] PeeringState::proc_lease readable_until_ub now 45.258411407s 2026-02-11T23:30:46.942+0000 7f414f993640 5 osd.0 pg_epoch: 93 pg[10.0( empty local-lis/les=0/0 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 unknown mbc={}] exit Started/Stray 0.000101 3 0.000020 2026-02-11T23:30:46.942+0000 7f414f993640 5 osd.0 pg_epoch: 93 pg[10.0( empty local-lis/les=0/0 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 unknown mbc={}] enter Started/ReplicaActive 2026-02-11T23:30:46.942+0000 7f414f993640 5 osd.0 pg_epoch: 93 pg[10.0( empty local-lis/les=0/0 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 unknown mbc={}] enter Started/ReplicaActive/RepNotRecovering 2026-02-11T23:30:46.942+0000 7f414f993640 10 osd.0 pg_epoch: 93 pg[10.0( empty local-lis/les=0/0 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 unknown mbc={}] state: In ReplicaActive, about to call activate 2026-02-11T23:30:46.942+0000 7f414f993640 10 osd.0 pg_epoch: 93 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 unknown mbc={}] PeeringState::activate activate - no missing, moving last_complete 0'0 -> 0'0 2026-02-11T23:30:46.942+0000 7f414f993640 10 osd.0 pg_epoch: 93 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 unknown mbc={}] scrubber: on_replica_activate 2026-02-11T23:30:46.942+0000 7f414f993640 20 scrubberFSM event: --vvvv---- ReplicaActivate 2026-02-11T23:30:46.942+0000 7f414f993640 10 osd.0 pg_epoch: 93 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 unknown mbc={}] scrubber: FSM: -- state -->> ReplicaActive 2026-02-11T23:30:46.942+0000 7f414f993640 10 osd.0 pg_epoch: 93 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 unknown mbc={}] scrubber: FSM: -- state -->> ReplicaActive/ReplicaIdle 2026-02-11T23:30:46.942+0000 7f414f993640 20 scrubberFSM event: --^^^^---- ReplicaActivate 2026-02-11T23:30:46.942+0000 7f414f993640 10 osd.0 pg_epoch: 93 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 unknown mbc={}] state: Activate Finished 2026-02-11T23:30:46.942+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:46.942+0000 7f414b98b640 20 osd.0 op_wq(0) _process 10.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:46.942+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 93 epoch_requested: 93 NullEvt) class_id 2immediate prio 255 cost 10 e93) pg 0x4aa0000 2026-02-11T23:30:46.942+0000 7f414b98b640 10 osd.0 pg_epoch: 93 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 unknown mbc={}] do_peering_event: epoch_sent: 93 epoch_requested: 93 NullEvt 2026-02-11T23:30:46.942+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:46.942+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:46.942+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 93 epoch_requested: 93 NullEvt) class_id 2immediate prio 255 cost 10 e93) queued 2026-02-11T23:30:46.942+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:46.942+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 93 epoch_requested: 93 NullEvt) class_id 2immediate prio 255 cost 10 e93) pg 0x5978000 2026-02-11T23:30:46.942+0000 7f414f993640 10 osd.0 pg_epoch: 92 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 93 2026-02-11T23:30:46.942+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:46.942+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 93 epoch_requested: 93 NullEvt) class_id 2immediate prio 255 cost 10 e93) queued 2026-02-11T23:30:46.942+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:46.942+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 93 epoch_requested: 93 NullEvt) class_id 2immediate prio 255 cost 10 e93) pg 0x5946000 2026-02-11T23:30:46.942+0000 7f414f993640 20 osd.0:0.update_pg_epoch 4.0 92 -> 93 2026-02-11T23:30:46.942+0000 7f414b98b640 10 osd.0 pg_epoch: 92 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 93 2026-02-11T23:30:46.942+0000 7f414f993640 10 osd.0 pg_epoch: 92 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:46.942+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 1.0 92 -> 93 2026-02-11T23:30:46.942+0000 7f414b98b640 10 osd.0 pg_epoch: 92 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:46.942+0000 7f414f993640 10 osd.0 pg_epoch: 93 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:46.942+0000 7f414b98b640 10 osd.0 pg_epoch: 93 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:46.942+0000 7f414f993640 10 osd.0 pg_epoch: 93 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:46.942+0000 7f414f993640 20 osd.0 93 advance_pg new pool opts old pool opts 2026-02-11T23:30:46.942+0000 7f414f993640 10 osd.0 pg_epoch: 93 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_activate_map: epoch range: 93..93 2026-02-11T23:30:46.942+0000 7f414f993640 10 osd.0 pg_epoch: 93 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:46.942+0000 7f414b98b640 10 osd.0 pg_epoch: 93 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:46.942+0000 7f414f993640 20 osd.0 pg_epoch: 93 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 53 while current is 93 2026-02-11T23:30:46.942+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:46.942+0000 7f414f993640 10 osd.0 pg_epoch: 93 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 93 epoch_requested: 93 NullEvt 2026-02-11T23:30:46.942+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:46.942+0000 7f414b98b640 20 osd.0 93 advance_pg new pool opts old pool opts 2026-02-11T23:30:46.942+0000 7f414b98b640 10 osd.0 pg_epoch: 93 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_activate_map: epoch range: 93..93 2026-02-11T23:30:46.942+0000 7f414f993640 15 osd.0 93 enqueue_peering_evt 10.0 epoch_sent: 93 epoch_requested: 93 ActivateCommitted from 93 processed at 93 2026-02-11T23:30:46.942+0000 7f414b98b640 10 osd.0 pg_epoch: 93 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:46.942+0000 7f414f993640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 93 epoch_requested: 93 ActivateCommitted from 93 processed at 93) class_id 2 prio 255 cost 10 e93) 2026-02-11T23:30:46.942+0000 7f414b98b640 20 osd.0 pg_epoch: 93 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 73 while current is 93 2026-02-11T23:30:46.942+0000 7f414f993640 20 osd.0 op_wq(0) _process 10.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:46.942+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 93 epoch_requested: 93 ActivateCommitted from 93 processed at 93) class_id 2immediate prio 255 cost 10 e93) queued 2026-02-11T23:30:46.942+0000 7f414f993640 20 osd.0 op_wq(0) _process 10.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:46.942+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 93 epoch_requested: 93 ActivateCommitted from 93 processed at 93) class_id 2immediate prio 255 cost 10 e93) pg 0x4aa0000 2026-02-11T23:30:46.942+0000 7f414f993640 10 osd.0 pg_epoch: 93 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 unknown mbc={}] do_peering_event: epoch_sent: 93 epoch_requested: 93 ActivateCommitted from 93 processed at 93 2026-02-11T23:30:46.942+0000 7f414f993640 10 osd.0 pg_epoch: 93 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 unknown mbc={}] state: react 93 telling primary 2026-02-11T23:30:46.942+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:46.942+0000 7f414f993640 20 osd.0 93 get_con_osd_cluster to osd.2 from_epoch 93 2026-02-11T23:30:46.942+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:46.942+0000 7f414f993640 20 osd.0 93 get_nextmap_reserved map_reservations: {93=1} 2026-02-11T23:30:46.942+0000 7f414b98b640 10 osd.0 pg_epoch: 93 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 93 epoch_requested: 93 NullEvt 2026-02-11T23:30:46.942+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:46.942+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:46.942+0000 7f414f993640 20 osd.0 93 release_map epoch: 93 2026-02-11T23:30:46.942+0000 7f414f993640 10 osd.0 93 maybe_share_map: con v2:127.0.0.1:6820/2365263801 map epoch 92 -> 93 (shared) 2026-02-11T23:30:46.942+0000 7f414f993640 10 osd.0 93 send_incremental_map epoch range: (93, 93] to v2:127.0.0.1:6820/2365263801 2026-02-11T23:30:46.942+0000 7f414f993640 10 osd.0 93 add_map_inc_bl 93 252 bytes 2026-02-11T23:30:46.942+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6820/2365263801,v1:127.0.0.1:6821/2365263801] -- osd_map(93..93 src has 1..93) -- 0x4a308c0 con 0x507b800 2026-02-11T23:30:46.942+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6820/2365263801,v1:127.0.0.1:6821/2365263801] -- pg_info2(10.0 10.0 10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/0 les/c/f=93/0/0 sis=92) e93/93) -- 0x373a300 con 0x507b800 2026-02-11T23:30:46.942+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:46.942+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.2 v2:127.0.0.1:6820/2365263801 8 ==== pg_lease(10.0 pg_lease(ru 40.691310883s ub 41.051261902s int 16.000000000s) e93/93) ==== 58+0+0 (crc 0 0 0) 0x4c9be00 con 0x507b800 2026-02-11T23:30:46.942+0000 7f416cbfe640 15 osd.0 93 enqueue_peering_evt 10.0 epoch_sent: 93 epoch_requested: 93 MLease epoch 93 from osd.2 pg_lease(ru 40.691310883s ub 41.051261902s int 16.000000000s) 2026-02-11T23:30:46.942+0000 7f416cbfe640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 93 epoch_requested: 93 MLease epoch 93 from osd.2 pg_lease(ru 40.691310883s ub 41.051261902s int 16.000000000s)) class_id 2 prio 255 cost 10 e93) 2026-02-11T23:30:46.942+0000 7f414b98b640 20 osd.0 op_wq(0) _process 10.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:46.942+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 93 epoch_requested: 93 MLease epoch 93 from osd.2 pg_lease(ru 40.691310883s ub 41.051261902s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e93) queued 2026-02-11T23:30:46.942+0000 7f414b98b640 20 osd.0 op_wq(0) _process 10.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:46.942+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 93 epoch_requested: 93 MLease epoch 93 from osd.2 pg_lease(ru 40.691310883s ub 41.051261902s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e93) pg 0x4aa0000 2026-02-11T23:30:46.942+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:46.942+0000 7f414b98b640 10 osd.0 pg_epoch: 93 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 93 epoch_requested: 93 MLease epoch 93 from osd.2 pg_lease(ru 40.691310883s ub 41.051261902s int 16.000000000s) 2026-02-11T23:30:46.942+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.2 v2:127.0.0.1:6820/2365263801 9 ==== pg_info2(10.0 10.0 10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/0 les/c/f=93/0/0 sis=92) e93/93) ==== 1126+0+0 (crc 0 0 0) 0x373a300 con 0x507b800 2026-02-11T23:30:46.942+0000 7f414b98b640 10 osd.0 pg_epoch: 93 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 active mbc={}] PeeringState::proc_lease pg_lease(ru 40.691310883s ub 41.051261902s int 16.000000000s) 2026-02-11T23:30:46.942+0000 7f416cbfe640 15 osd.0 93 enqueue_peering_evt 10.0 epoch_sent: 93 epoch_requested: 93 MInfoRec from 2 info: 10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/0 les/c/f=93/0/0 sis=92) pg_lease(ru 40.691310883s ub 41.051261902s int 16.000000000s) 2026-02-11T23:30:46.942+0000 7f416cbfe640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 93 epoch_requested: 93 MInfoRec from 2 info: 10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/0 les/c/f=93/0/0 sis=92) pg_lease(ru 40.691310883s ub 41.051261902s int 16.000000000s)) class_id 2 prio 255 cost 10 e93) 2026-02-11T23:30:46.942+0000 7f414b98b640 20 osd.0 pg_epoch: 93 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 active mbc={}] PeeringState::proc_lease peer_clock_delta_ub -4.566688061s -> ru 45.258003235s 2026-02-11T23:30:46.942+0000 7f414b98b640 20 osd.0 pg_epoch: 93 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 active mbc={}] PeeringState::proc_lease readable_until now 45.258003235s 2026-02-11T23:30:46.942+0000 7f414b98b640 20 osd.0 pg_epoch: 93 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 active mbc={}] PeeringState::proc_lease peer_clock_delta_lb -4.567098141s -> ruub 45.618358612s 2026-02-11T23:30:46.942+0000 7f414b98b640 20 osd.0 pg_epoch: 93 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 active mbc={}] PeeringState::proc_lease readable_until_ub now 45.618358612s 2026-02-11T23:30:46.942+0000 7f414f993640 20 osd.0 op_wq(0) _process 10.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:46.942+0000 7f414b98b640 20 osd.0 93 get_con_osd_cluster to osd.2 from_epoch 93 2026-02-11T23:30:46.942+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 93 epoch_requested: 93 MInfoRec from 2 info: 10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/0 les/c/f=93/0/0 sis=92) pg_lease(ru 40.691310883s ub 41.051261902s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e93) queued 2026-02-11T23:30:46.942+0000 7f414b98b640 20 osd.0 93 get_nextmap_reserved map_reservations: {93=1} 2026-02-11T23:30:46.942+0000 7f414b98b640 20 osd.0 93 release_map epoch: 93 2026-02-11T23:30:46.942+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6820/2365263801,v1:127.0.0.1:6821/2365263801] -- pg_lease_ack(10.0 pg_lease_ack(ruub 41.051261902s) e93/93) -- 0x3759600 con 0x507b800 2026-02-11T23:30:46.942+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:46.942+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:46.942+0000 7f414f993640 20 osd.0 op_wq(0) _process 10.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:46.942+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 93 epoch_requested: 93 MInfoRec from 2 info: 10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/0 les/c/f=93/0/0 sis=92) pg_lease(ru 40.691310883s ub 41.051261902s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e93) pg 0x4aa0000 2026-02-11T23:30:46.942+0000 7f414f993640 10 osd.0 pg_epoch: 93 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=0/0 les/c/f=0/0/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 93 epoch_requested: 93 MInfoRec from 2 info: 10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/0 les/c/f=93/0/0 sis=92) pg_lease(ru 40.691310883s ub 41.051261902s int 16.000000000s) 2026-02-11T23:30:46.942+0000 7f414f993640 20 osd.0 pg_epoch: 93 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/0 les/c/f=93/0/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 active mbc={}] PeeringState::update_history advanced history from ec=92/92 lis/c=92/0 les/c/f=93/0/0 sis=92 2026-02-11T23:30:46.942+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:46.942+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:46.942+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.2 v2:127.0.0.1:6820/2365263801 10 ==== pg_info2(10.0 10.0 10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=92) e93/93) ==== 1126+0+0 (crc 0 0 0) 0x373a300 con 0x507b800 2026-02-11T23:30:46.942+0000 7f416cbfe640 15 osd.0 93 enqueue_peering_evt 10.0 epoch_sent: 93 epoch_requested: 93 MInfoRec from 2 info: 10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=92) pg_lease(ru 40.691310883s ub 41.051261902s int 16.000000000s) 2026-02-11T23:30:46.942+0000 7f416cbfe640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 93 epoch_requested: 93 MInfoRec from 2 info: 10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=92) pg_lease(ru 40.691310883s ub 41.051261902s int 16.000000000s)) class_id 2 prio 255 cost 10 e93) 2026-02-11T23:30:46.942+0000 7f414f993640 20 osd.0 op_wq(0) _process 10.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:46.942+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 93 epoch_requested: 93 MInfoRec from 2 info: 10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=92) pg_lease(ru 40.691310883s ub 41.051261902s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e93) queued 2026-02-11T23:30:46.942+0000 7f414f993640 20 osd.0 op_wq(0) _process 10.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:46.942+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 93 epoch_requested: 93 MInfoRec from 2 info: 10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=92) pg_lease(ru 40.691310883s ub 41.051261902s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e93) pg 0x4aa0000 2026-02-11T23:30:46.942+0000 7f414f993640 10 osd.0 pg_epoch: 93 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/0 les/c/f=93/0/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 93 epoch_requested: 93 MInfoRec from 2 info: 10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=92) pg_lease(ru 40.691310883s ub 41.051261902s int 16.000000000s) 2026-02-11T23:30:46.942+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:46.942+0000 7f414f993640 20 osd.0 pg_epoch: 93 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 active mbc={}] PeeringState::update_history advanced history from ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=92 2026-02-11T23:30:46.942+0000 7f414f993640 20 osd.0 pg_epoch: 93 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 active mbc={}] PeeringState::update_history clearing past_intervals 2026-02-11T23:30:46.942+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:46.942+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:46.942+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:46.942+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:46.942+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:46.942+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:47.294+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.2 v2:127.0.0.1:6820/2365263801 12 ==== pg_query2(10.0 10.0 query(info 0'0 epoch_sent 95) e95/95) ==== 158+0+0 (crc 0 0 0) 0x59681e0 con 0x507b800 2026-02-11T23:30:47.294+0000 7f415aefe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.2 v2:127.0.0.1:6820/2365263801 11 ==== osd_map(93..95 src has 1..95) ==== 1460+0+0 (crc 0 0 0) 0x4a308c0 con 0x507b800 2026-02-11T23:30:47.294+0000 7f415aefe640 20 osd.0 93 OSD::ms_dispatch: osd_map(93..95 src has 1..95) 2026-02-11T23:30:47.294+0000 7f415aefe640 20 osd.0 93 _dispatch 0x4a308c0 osd_map(93..95 src has 1..95) 2026-02-11T23:30:47.294+0000 7f416cbfe640 15 osd.0 93 enqueue_peering_evt 10.0 epoch_sent: 95 epoch_requested: 95 MQuery 10.0 from 2 query_epoch 95 query: query(info 0'0 epoch_sent 95) 2026-02-11T23:30:47.294+0000 7f416cbfe640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 95 epoch_requested: 95 MQuery 10.0 from 2 query_epoch 95 query: query(info 0'0 epoch_sent 95)) class_id 2 prio 255 cost 10 e95) 2026-02-11T23:30:47.294+0000 7f414b98b640 20 osd.0 op_wq(0) _process 10.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:47.294+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 95 epoch_requested: 95 MQuery 10.0 from 2 query_epoch 95 query: query(info 0'0 epoch_sent 95)) class_id 2immediate prio 255 cost 10 e95) queued 2026-02-11T23:30:47.294+0000 7f414b98b640 20 osd.0 op_wq(0) _process 10.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:47.294+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 95 epoch_requested: 95 MQuery 10.0 from 2 query_epoch 95 query: query(info 0'0 epoch_sent 95)) class_id 2immediate prio 255 cost 10 e95) pg 0x4aa0000 2026-02-11T23:30:47.294+0000 7f414b98b640 20 osd.0 op_wq _add_slot_waiter 10.0 peering, item epoch is 95, will wait on OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 95 epoch_requested: 95 MQuery 10.0 from 2 query_epoch 95 query: query(info 0'0 epoch_sent 95)) class_id 2immediate prio 255 cost 10 e95) 2026-02-11T23:30:47.294+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:47.294+0000 7f415aefe640 3 osd.0 93 handle_osd_map epochs [93,95], i have 93, src has [1,95] 2026-02-11T23:30:47.294+0000 7f415aefe640 10 osd.0 93 handle_osd_map: received 0 full maps and 3 incremental maps 2026-02-11T23:30:47.294+0000 7f415aefe640 10 osd.0 93 handle_osd_map got inc map for epoch 94 2026-02-11T23:30:47.294+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:47.294+0000 7f415aefe640 20 osd.0 93 got_full_map 94, nothing requested 2026-02-11T23:30:47.294+0000 7f415aefe640 10 osd.0 93 handle_osd_map got inc map for epoch 95 2026-02-11T23:30:47.294+0000 7f415aefe640 20 osd.0 93 got_full_map 95, nothing requested 2026-02-11T23:30:47.294+0000 7f415aefe640 20 osd.0 93 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:47.294+0000 7f415aefe640 20 osd.0 93 handle_osd_map pg_num_history pg_num_history(e95 pg_nums {1={27=1},2={34=1},3={42=1},4={49=1},5={56=1},6={64=1},7={70=1},8={77=1},9={85=1},10={92=1}} deleted_pools ) 2026-02-11T23:30:47.294+0000 7f415aefe640 10 snap_mapper record_purged_snaps purged_snaps {94={},95={}} 2026-02-11T23:30:47.294+0000 7f415aefe640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:47.294+0000 7f415aefe640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e95 maps [1~95] lci=[5,95] tlb=1) 2026-02-11T23:30:47.295+0000 7f41591a6640 10 osd.0 93 _committed_osd_maps 94..95 2026-02-11T23:30:47.295+0000 7f41591a6640 10 osd.0 93 advance to epoch 94 (<= last 95 <= newest_map 95) 2026-02-11T23:30:47.295+0000 7f41591a6640 10 osd.0 94 advance to epoch 95 (<= last 95 <= newest_map 95) 2026-02-11T23:30:47.295+0000 7f41591a6640 20 osd.0 95 consume_map version 95 2026-02-11T23:30:47.295+0000 7f41591a6640 20 osd.0 93 await_reserved_maps epoch:95 2026-02-11T23:30:47.295+0000 7f41591a6640 20 osd.0 93 await_reserved_maps done 0 2026-02-11T23:30:47.295+0000 7f41591a6640 20 osd.0 95 consume_map 95 -- publish done 2026-02-11T23:30:47.295+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 3 slots 2026-02-11T23:30:47.295+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:10.0slot:0x37594a0 2026-02-11T23:30:47.295+0000 7f41591a6640 20 osd.0 95 identify_splits_and_merges 10.0 e93 to e95 2026-02-11T23:30:47.295+0000 7f41591a6640 20 osd.0 95 identify_splits_and_merges 10.0 e93 to e95 pg_nums {92=1} 2026-02-11T23:30:47.295+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x37582c0 2026-02-11T23:30:47.295+0000 7f41591a6640 20 osd.0 95 identify_splits_and_merges 4.0 e93 to e95 2026-02-11T23:30:47.295+0000 7f41591a6640 20 osd.0 95 identify_splits_and_merges 4.0 e93 to e95 pg_nums {49=1} 2026-02-11T23:30:47.295+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3700dc0 2026-02-11T23:30:47.295+0000 7f41591a6640 20 osd.0 95 identify_splits_and_merges 1.0 e93 to e95 2026-02-11T23:30:47.295+0000 7f41591a6640 20 osd.0 95 identify_splits_and_merges 1.0 e93 to e95 pg_nums {27=1} 2026-02-11T23:30:47.295+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:47.295+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:47.295+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:47.295+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:47.295+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:47.295+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:47.295+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:47.295+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:47.295+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:47.295+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:47.295+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:47.295+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:47.295+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:47.295+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:47.295+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:47.295+0000 7f41591a6640 20 osd.0 95 prune_pg_created 2026-02-11T23:30:47.295+0000 7f41591a6640 10 osd.0:0.consume_map 95 (was 93) 2026-02-11T23:30:47.295+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 10.0 2026-02-11T23:30:47.295+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 10.0 pending_peering first epoch 95 <= 95, requeueing 2026-02-11T23:30:47.295+0000 7f41591a6640 20 osd.0:0._wake_pg_slot _wake_pg_slot 10.0 to_process <> waiting <> waiting_peering {95=} 2026-02-11T23:30:47.295+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:47.295+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:47.295+0000 7f41591a6640 10 osd.0:1.consume_map 95 (was 93) 2026-02-11T23:30:47.295+0000 7f41591a6640 10 osd.0:2.consume_map 95 (was 93) 2026-02-11T23:30:47.295+0000 7f41591a6640 10 osd.0:3.consume_map 95 (was 93) 2026-02-11T23:30:47.295+0000 7f41591a6640 10 osd.0:4.consume_map 95 (was 93) 2026-02-11T23:30:47.295+0000 7f41591a6640 10 osd.0:5.consume_map 95 (was 93) 2026-02-11T23:30:47.295+0000 7f41591a6640 10 osd.0:6.consume_map 95 (was 93) 2026-02-11T23:30:47.295+0000 7f41591a6640 10 osd.0:7.consume_map 95 (was 93) 2026-02-11T23:30:47.295+0000 7f414b98b640 20 osd.0 op_wq(0) _process 10.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:47.295+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 95 epoch_requested: 95 MQuery 10.0 from 2 query_epoch 95 query: query(info 0'0 epoch_sent 95)) class_id 2immediate prio 255 cost 10 e95) queued 2026-02-11T23:30:47.295+0000 7f414b98b640 20 osd.0 op_wq(0) _process 10.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:47.295+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 95 epoch_requested: 95 MQuery 10.0 from 2 query_epoch 95 query: query(info 0'0 epoch_sent 95)) class_id 2immediate prio 255 cost 10 e95) pg 0x4aa0000 2026-02-11T23:30:47.295+0000 7f41591a6640 10 osd.0 95 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 pg_epoch: 93 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 active mbc={}] handle_advance_map: 94 2026-02-11T23:30:47.295+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 10.0 93 -> 94 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 pg_epoch: 93 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [2,1,0]/[2,1,0] -- 2/2 2026-02-11T23:30:47.295+0000 7f41591a6640 15 osd.0 95 enqueue_peering_evt 10.0 epoch_sent: 95 epoch_requested: 95 NullEvt 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 pg_epoch: 94 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:47.295+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 95 epoch_requested: 95 NullEvt) class_id 2 prio 255 cost 10 e95) 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 pg_epoch: 94 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 pg_epoch: 94 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 active mbc={}] plpg_on_pool_change 2026-02-11T23:30:47.295+0000 7f414b98b640 20 osd.0 pg_epoch: 94 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 active mbc={}] hit_set_clear 2026-02-11T23:30:47.295+0000 7f41591a6640 15 osd.0 95 enqueue_peering_evt 4.0 epoch_sent: 95 epoch_requested: 95 NullEvt 2026-02-11T23:30:47.295+0000 7f414b98b640 20 osd.0 pg_epoch: 94 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 active mbc={}] agent_stop 2026-02-11T23:30:47.295+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 95 epoch_requested: 95 NullEvt) class_id 2 prio 255 cost 10 e95) 2026-02-11T23:30:47.295+0000 7f414b98b640 20 osd.0 95 advance_pg new pool opts old pool opts 2026-02-11T23:30:47.295+0000 7f414f993640 20 osd.0 op_wq(0) _process 10.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:47.295+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 95 epoch_requested: 95 NullEvt) class_id 2immediate prio 255 cost 10 e95) queued 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 pg_epoch: 94 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 active mbc={}] handle_advance_map: 95 2026-02-11T23:30:47.295+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 10.0 94 -> 95 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 pg_epoch: 94 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [2,1]/[2,1] -- 2/2 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:47.295+0000 7f414b98b640 20 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 active mbc={}] PeeringState::should_restart_peering new interval newup [2,1] newacting [2,1] 2026-02-11T23:30:47.295+0000 7f41591a6640 15 osd.0 95 enqueue_peering_evt 1.0 epoch_sent: 95 epoch_requested: 95 NullEvt 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 active mbc={}] state: should_restart_peering, transitioning to Reset 2026-02-11T23:30:47.295+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 95 epoch_requested: 95 NullEvt) class_id 2 prio 255 cost 10 e95) 2026-02-11T23:30:47.295+0000 7f414b98b640 5 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 active mbc={}] exit Started/ReplicaActive/RepNotRecovering 0.353400 7 0.000093 2026-02-11T23:30:47.295+0000 7f414b98b640 5 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 active mbc={}] exit Started/ReplicaActive 0.353407 0 0.000000 2026-02-11T23:30:47.295+0000 7f41591a6640 7 osd.0 95 activate_map version 95 2026-02-11T23:30:47.295+0000 7f414b98b640 5 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 active mbc={}] exit Started 0.353521 0 0.000000 2026-02-11T23:30:47.295+0000 7f414b98b640 5 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 active mbc={}] enter Reset 2026-02-11T23:30:47.295+0000 7f414b98b640 20 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=92) [2,1,0] r=2 lpr=92 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset set_last_peering_reset 95 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=92) [2,1,0] r=2 lpr=95 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset Clearing blocked outgoing recovery messages 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=92) [2,1,0] r=2 lpr=95 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset Not blocking outgoing recovery messages 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=92) [2,1,0] r=2 lpr=95 crt=0'0 active mbc={}] state: Reset advmap 2026-02-11T23:30:47.295+0000 7f414b98b640 20 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=92) [2,1,0] r=2 lpr=95 crt=0'0 active mbc={}] PeeringState::should_restart_peering new interval newup [2,1] newacting [2,1] 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=92) [2,1,0] r=2 lpr=95 crt=0'0 active mbc={}] state: should restart peering, calling start_peering_interval again 2026-02-11T23:30:47.295+0000 7f414b98b640 20 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=92) [2,1,0] r=2 lpr=95 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset set_last_peering_reset 95 2026-02-11T23:30:47.295+0000 7f4150194640 20 osd.0 95 agent_entry empty queue 2026-02-11T23:30:47.295+0000 7f414b98b640 15 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=92) [2,1] r=2 lpr=95 crt=0'0 active mbc={}] clear_stats 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=92) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 active mbc={}] PeeringState::start_peering_interval : check_new_interval output: check_new_interval interval(92-94 up [2, 1, 0](2) acting [2, 1, 0](2)) up_thru 92 up_from 14 last_epoch_clean 93 interval(92-94 up [2, 1, 0](2) acting [2, 1, 0](2) maybe_went_rw) : primary up 14-92 includes interval 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=92) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 active mbc={}] PeeringState::start_peering_interval noting past ([92,94] all_participants={0, 1, 2} intervals=[([92,94] acting={0, 1, 2})]) 2026-02-11T23:30:47.295+0000 7f414b98b640 20 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 active mbc={}] PeeringState::on_new_interval 2026-02-11T23:30:47.295+0000 7f414b98b640 20 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 active mbc={}] PeeringState::on_new_interval upacting_features 0x3f0ffffffffdffff from [2,1]+[2,1] 2026-02-11T23:30:47.295+0000 7f414b98b640 20 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 active mbc={}] PeeringState::on_new_interval checking missing set deletes flag. missing = missing(0 may_include_deletes = 1) 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 active mbc={}] PeeringState::init_hb_stamps now [] 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.647118568s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 active pruub 45.618358612s@ mbc={}] PeeringState::on_new_interval prior_readable_until_ub 45.618358612s (mnow 29.971240997s + 15.647118568s) 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.647118568s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 active pruub 45.618358612s@ mbc={}] cancel_recovery 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.647118568s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 active pruub 45.618358612s@ mbc={}] clear_recovery_state 2026-02-11T23:30:47.295+0000 7f414b98b640 15 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.647118568s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 active pruub 45.618358612s@ mbc={}] _clear_recovery_state 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.647118568s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 active pruub 45.618358612s@ mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:47.295+0000 7f414b98b640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:47.295+0000 7f414b98b640 20 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.647118568s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 active pruub 45.618358612s@ mbc={}] scrubber: FSM: ReplicaActive::exit() 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.647118568s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 active pruub 45.618358612s@ mbc={}] scrubber: FSM: ReplicaActive::clear_remote_reservation(): pending_reservation_nonce 0, reservation_granted false 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.647118568s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 active pruub 45.618358612s@ mbc={}] scrubber: FSM: -- state -->> NotActive 2026-02-11T23:30:47.295+0000 7f414b98b640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:47.295+0000 7f414b98b640 1 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.647118568s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 active pruub 45.618358612s@ mbc={}] PeeringState::start_peering_interval up [2,1,0] -> [2,1], acting [2,1,0] -> [2,1], acting_primary 2 -> 2, up_primary 2 -> 2, role 2 -> -1, features acting 4544132024016699391 upacting 4544132024016699391 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.647118568s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown pruub 45.618358612s@ mbc={}] PeeringState::clear_primary_state clear_primary_state 2026-02-11T23:30:47.295+0000 7f414b98b640 20 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.647118568s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown pruub 45.618358612s@ mbc={}] clear_primary_state 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.647118568s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown pruub 45.618358612s@ mbc={}] release_backoffs [10:00000000::::0,MAX) 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.647118568s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown pruub 45.618358612s@ mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:47.295+0000 7f414b98b640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:47.295+0000 7f414b98b640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:47.295+0000 7f414b98b640 20 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.647118568s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown pruub 45.618358612s@ mbc={}] agent_stop 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.647118568s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown pruub 45.618358612s@ mbc={}] on_change 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.647118568s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown pruub 45.618358612s@ mbc={}] cancel_copy_ops 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.647118568s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown pruub 45.618358612s@ mbc={}] cancel_flush_ops 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.647118568s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown pruub 45.618358612s@ mbc={}] cancel_proxy_ops 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.647118568s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown pruub 45.618358612s@ mbc={}] cancel_manifest_ops 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.647118568s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown pruub 45.618358612s@ mbc={}] cancel_cls_gather_ops 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.647118568s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown pruub 45.618358612s@ mbc={}] on_change_cleanup 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.647118568s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown pruub 45.618358612s@ mbc={}] on_change 2026-02-11T23:30:47.295+0000 7f414b98b640 20 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.647118568s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown pruub 45.618358612s@ mbc={}] exit NotTrimming 2026-02-11T23:30:47.295+0000 7f414b98b640 20 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.647118568s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown pruub 45.618358612s@ mbc={}] enter NotTrimming 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.647118568s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] plpg_on_role_change 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.647118568s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.647118568s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] plpg_on_pool_change 2026-02-11T23:30:47.295+0000 7f414b98b640 20 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.647118568s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] hit_set_clear 2026-02-11T23:30:47.295+0000 7f414b98b640 20 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.647118568s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] agent_stop 2026-02-11T23:30:47.295+0000 7f414b98b640 20 osd.0 95 advance_pg new pool opts old pool opts 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.647118568s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] handle_activate_map: epoch range: 94..95 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.647118568s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] PeeringState::activate_map 2026-02-11T23:30:47.295+0000 7f414b98b640 5 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.647050858s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] exit Reset 0.000120 1 0.000121 2026-02-11T23:30:47.295+0000 7f414b98b640 5 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.647050858s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] enter Started 2026-02-11T23:30:47.295+0000 7f414b98b640 5 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.647050858s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] enter Start 2026-02-11T23:30:47.295+0000 7f414b98b640 1 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.647050858s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] state: transitioning to Stray 2026-02-11T23:30:47.295+0000 7f414b98b640 5 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.647050858s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] exit Start 0.000002 0 0.000000 2026-02-11T23:30:47.295+0000 7f414b98b640 5 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.647050858s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] enter Started/Stray 2026-02-11T23:30:47.295+0000 7f414b98b640 20 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.647050858s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 93 while current is 95 2026-02-11T23:30:47.295+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.647050858s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.647050858s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] do_peering_event: epoch_sent: 95 epoch_requested: 95 MQuery 10.0 from 2 query_epoch 95 query: query(info 0'0 epoch_sent 95) 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.647021294s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] PeeringState::fulfill_info sending info 2026-02-11T23:30:47.295+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:47.295+0000 7f414b98b640 20 osd.0 95 get_con_osd_cluster to osd.2 from_epoch 95 2026-02-11T23:30:47.295+0000 7f414b98b640 20 osd.0 95 get_nextmap_reserved map_reservations: {95=1} 2026-02-11T23:30:47.295+0000 7f414b98b640 20 osd.0 95 release_map epoch: 95 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 95 maybe_share_map: con v2:127.0.0.1:6820/2365263801 map epoch 93 -> 95 (shared) 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 95 send_incremental_map epoch range: (94, 95] to v2:127.0.0.1:6820/2365263801 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 95 add_map_inc_bl 94 574 bytes 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 95 add_map_inc_bl 95 574 bytes 2026-02-11T23:30:47.295+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6820/2365263801,v1:127.0.0.1:6821/2365263801] -- osd_map(94..95 src has 1..95) -- 0x59548c0 con 0x507b800 2026-02-11T23:30:47.295+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6820/2365263801,v1:127.0.0.1:6821/2365263801] -- pg_notify2(10.0 10.0 (query:95 sent:95 10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.647050858s) ([92,94] all_participants={0, 1, 2} intervals=[([92,94] acting={0, 1, 2})])) e95/95) -- 0x5a8e300 con 0x507b800 2026-02-11T23:30:47.295+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6820/2365263801,v1:127.0.0.1:6821/2365263801] -- pg_notify2(10.0 10.0 (query:95 sent:95 10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.647021294s) ([92,94] all_participants={0, 1, 2} intervals=[([92,94] acting={0, 1, 2})])) e95/95) -- 0x5a8c700 con 0x507b800 2026-02-11T23:30:47.295+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:47.295+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 95 epoch_requested: 95 NullEvt) class_id 2immediate prio 255 cost 10 e95) queued 2026-02-11T23:30:47.295+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:47.295+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 95 epoch_requested: 95 NullEvt) class_id 2immediate prio 255 cost 10 e95) pg 0x5978000 2026-02-11T23:30:47.295+0000 7f414f993640 20 osd.0 op_wq(0) _process 10.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:47.295+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 95 epoch_requested: 95 NullEvt) class_id 2immediate prio 255 cost 10 e95) pg 0x4aa0000 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 pg_epoch: 93 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 94 2026-02-11T23:30:47.295+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 4.0 93 -> 94 2026-02-11T23:30:47.295+0000 7f414f993640 10 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.647021294s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] do_peering_event: epoch_sent: 95 epoch_requested: 95 NullEvt 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 pg_epoch: 93 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:47.295+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 pg_epoch: 94 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:47.295+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:47.295+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 95 epoch_requested: 95 NullEvt) class_id 2immediate prio 255 cost 10 e95) queued 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 pg_epoch: 94 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:47.295+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:47.295+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 95 epoch_requested: 95 NullEvt) class_id 2immediate prio 255 cost 10 e95) pg 0x5946000 2026-02-11T23:30:47.295+0000 7f414b98b640 20 osd.0 95 advance_pg new pool opts old pool opts 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 pg_epoch: 94 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 95 2026-02-11T23:30:47.295+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 4.0 94 -> 95 2026-02-11T23:30:47.295+0000 7f414f993640 10 osd.0 pg_epoch: 93 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 94 2026-02-11T23:30:47.295+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 93 -> 94 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 pg_epoch: 94 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:47.295+0000 7f414f993640 10 osd.0 pg_epoch: 93 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 pg_epoch: 95 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:47.295+0000 7f414f993640 10 osd.0 pg_epoch: 94 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 pg_epoch: 95 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:47.295+0000 7f414b98b640 20 osd.0 95 advance_pg new pool opts old pool opts 2026-02-11T23:30:47.295+0000 7f414f993640 10 osd.0 pg_epoch: 94 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 pg_epoch: 95 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_activate_map: epoch range: 94..95 2026-02-11T23:30:47.295+0000 7f414f993640 20 osd.0 95 advance_pg new pool opts old pool opts 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 pg_epoch: 95 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:47.295+0000 7f414b98b640 20 osd.0 pg_epoch: 95 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map : Dirtying info: last_persisted is 53 while current is 95 2026-02-11T23:30:47.295+0000 7f414f993640 10 osd.0 pg_epoch: 94 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 95 2026-02-11T23:30:47.295+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 94 -> 95 2026-02-11T23:30:47.295+0000 7f414f993640 10 osd.0 pg_epoch: 94 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:47.295+0000 7f414f993640 10 osd.0 pg_epoch: 95 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:47.295+0000 7f414f993640 10 osd.0 pg_epoch: 95 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:47.295+0000 7f414f993640 20 osd.0 95 advance_pg new pool opts old pool opts 2026-02-11T23:30:47.295+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:47.295+0000 7f414f993640 10 osd.0 pg_epoch: 95 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_activate_map: epoch range: 94..95 2026-02-11T23:30:47.295+0000 7f414f993640 10 osd.0 pg_epoch: 95 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:47.295+0000 7f414f993640 20 osd.0 pg_epoch: 95 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 73 while current is 95 2026-02-11T23:30:47.295+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:47.295+0000 7f414f993640 10 osd.0 pg_epoch: 95 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 95 epoch_requested: 95 NullEvt 2026-02-11T23:30:47.295+0000 7f414b98b640 10 osd.0 pg_epoch: 95 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 95 epoch_requested: 95 NullEvt 2026-02-11T23:30:47.295+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:47.295+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:47.295+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:47.295+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:47.296+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:47.296+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:47.327+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:30:47.327+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:30:47.327+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:17.328227+0000) 2026-02-11T23:30:47.347+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.3 127.0.0.1:0/2699377081 6 ==== osd_ping(ping e93 up_from 19 ping_stamp 2026-02-11T23:30:47.347923+0000/22.803861618s send_stamp 22.803861618s delta_ub -7.219282150s) ==== 2033+0+0 (crc 0 0 0) 0x5af8300 con 0x48cc400 2026-02-11T23:30:47.347+0000 7f416c3fd640 20 osd.0 95 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219541550s,-7.219282150s]) 2026-02-11T23:30:47.347+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.3 127.0.0.1:0/2699377081 6 ==== osd_ping(ping e93 up_from 19 ping_stamp 2026-02-11T23:30:47.347923+0000/22.803861618s send_stamp 22.803861618s delta_ub -7.219282150s) ==== 2033+0+0 (crc 0 0 0) 0x4c9be00 con 0x48cc800 2026-02-11T23:30:47.347+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e95 up_from 5 ping_stamp 2026-02-11T23:30:47.347923+0000/22.803861618s send_stamp 30.023403168s delta_ub 7.219541550s) -- 0x4c9af00 con 0x48cc400 2026-02-11T23:30:47.347+0000 7f416c3fd640 20 osd.0 95 get_con_osd_cluster to osd.3 from_epoch 95 2026-02-11T23:30:47.347+0000 7f416c3fd640 20 osd.0 95 get_nextmap_reserved map_reservations: {95=1} 2026-02-11T23:30:47.347+0000 7f416c3fd640 20 osd.0 95 release_map epoch: 95 2026-02-11T23:30:47.347+0000 7f416c3fd640 10 osd.0 95 maybe_share_map: con v2:127.0.0.1:6828/2699377081 updating session's projected_epoch from 92 to ping map epoch of 93 2026-02-11T23:30:47.347+0000 7f416c3fd640 10 osd.0 95 maybe_share_map: con v2:127.0.0.1:6828/2699377081 map epoch 93 -> 95 (shared) 2026-02-11T23:30:47.347+0000 7f416c3fd640 10 osd.0 95 send_incremental_map epoch range: (94, 95] to v2:127.0.0.1:6828/2699377081 2026-02-11T23:30:47.347+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6828/2699377081,v1:127.0.0.1:6829/2699377081] -- osd_map(94..95 src has 1..95) -- 0x4a30fc0 con 0x48ccc00 2026-02-11T23:30:47.347+0000 7f416cbfe640 20 osd.0 95 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219576359s,-7.219282150s]) 2026-02-11T23:30:47.347+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e95 up_from 5 ping_stamp 2026-02-11T23:30:47.347923+0000/22.803861618s send_stamp 30.023437500s delta_ub 7.219576359s) -- 0x5af8a80 con 0x48cc800 2026-02-11T23:30:47.347+0000 7f416cbfe640 20 osd.0 95 get_con_osd_cluster to osd.3 from_epoch 95 2026-02-11T23:30:47.347+0000 7f416cbfe640 20 osd.0 95 get_nextmap_reserved map_reservations: {95=1} 2026-02-11T23:30:47.347+0000 7f416cbfe640 20 osd.0 95 release_map epoch: 95 2026-02-11T23:30:47.347+0000 7f416cbfe640 10 osd.0 95 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 95 is not newer than session's projected_epoch of 95 2026-02-11T23:30:47.365+0000 7f4162bea640 10 osd.0 95 tick_without_osd_lock 2026-02-11T23:30:47.365+0000 7f4162bea640 0 osd.0 95 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:47.365+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:47.365+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:47.365+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:47.365+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.430 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:47.365+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:47.365, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:47.365+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:47.365+0000 7f4162bea640 10 osd.0 95 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:47.365+0000 7f4162bea640 20 osd.0 95 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:47.365+0000 7f4162bea640 10 osd.0 95 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:47.365+0000 7f4162bea640 20 osd.0 95 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:47.393+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 45 ==== osd_map(95..96 src has 1..96) ==== 1208+0+0 (crc 0 0 0) 0x4a30fc0 con 0x4acbc00 2026-02-11T23:30:47.393+0000 7f415cf02640 20 osd.0 95 OSD::ms_dispatch: osd_map(95..96 src has 1..96) 2026-02-11T23:30:47.393+0000 7f415cf02640 20 osd.0 95 _dispatch 0x4a30fc0 osd_map(95..96 src has 1..96) 2026-02-11T23:30:47.393+0000 7f415cf02640 3 osd.0 95 handle_osd_map epochs [95,96], i have 95, src has [1,96] 2026-02-11T23:30:47.393+0000 7f415cf02640 10 osd.0 95 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:47.393+0000 7f415cf02640 10 osd.0 95 handle_osd_map got inc map for epoch 96 2026-02-11T23:30:47.393+0000 7f415cf02640 20 osd.0 95 got_full_map 96, nothing requested 2026-02-11T23:30:47.393+0000 7f415cf02640 20 osd.0 95 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:47.393+0000 7f415cf02640 20 osd.0 95 handle_osd_map pg_num_history pg_num_history(e96 pg_nums {1={27=1},2={34=1},3={42=1},4={49=1},5={56=1},6={64=1},7={70=1},8={77=1},9={85=1},10={92=1}} deleted_pools ) 2026-02-11T23:30:47.393+0000 7f415cf02640 10 snap_mapper record_purged_snaps purged_snaps {96={}} 2026-02-11T23:30:47.393+0000 7f415cf02640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:47.393+0000 7f415cf02640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e96 maps [1~96] lci=[5,96] tlb=1) 2026-02-11T23:30:47.393+0000 7f41591a6640 10 osd.0 95 _committed_osd_maps 96..96 2026-02-11T23:30:47.393+0000 7f41591a6640 10 osd.0 95 advance to epoch 96 (<= last 96 <= newest_map 96) 2026-02-11T23:30:47.393+0000 7f41591a6640 20 osd.0 96 consume_map version 96 2026-02-11T23:30:47.393+0000 7f41591a6640 20 osd.0 95 await_reserved_maps epoch:96 2026-02-11T23:30:47.393+0000 7f41591a6640 20 osd.0 95 await_reserved_maps done 0 2026-02-11T23:30:47.393+0000 7f41591a6640 20 osd.0 96 consume_map 96 -- publish done 2026-02-11T23:30:47.393+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 3 slots 2026-02-11T23:30:47.393+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:10.0slot:0x37594a0 2026-02-11T23:30:47.393+0000 7f41591a6640 20 osd.0 96 identify_splits_and_merges 10.0 e95 to e96 2026-02-11T23:30:47.394+0000 7f41591a6640 20 osd.0 96 identify_splits_and_merges 10.0 e95 to e96 pg_nums {92=1} 2026-02-11T23:30:47.394+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x37582c0 2026-02-11T23:30:47.394+0000 7f41591a6640 20 osd.0 96 identify_splits_and_merges 4.0 e95 to e96 2026-02-11T23:30:47.394+0000 7f41591a6640 20 osd.0 96 identify_splits_and_merges 4.0 e95 to e96 pg_nums {49=1} 2026-02-11T23:30:47.394+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3700dc0 2026-02-11T23:30:47.394+0000 7f41591a6640 20 osd.0 96 identify_splits_and_merges 1.0 e95 to e96 2026-02-11T23:30:47.394+0000 7f41591a6640 20 osd.0 96 identify_splits_and_merges 1.0 e95 to e96 pg_nums {27=1} 2026-02-11T23:30:47.394+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:47.394+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:47.394+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:47.394+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:47.394+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:47.394+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:47.394+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:47.394+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:47.394+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:47.394+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:47.394+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:47.394+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:47.394+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:47.394+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:47.394+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:47.394+0000 7f41591a6640 20 osd.0 96 prune_pg_created 2026-02-11T23:30:47.394+0000 7f41591a6640 10 osd.0:0.consume_map 96 (was 95) 2026-02-11T23:30:47.394+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 10.0 2026-02-11T23:30:47.394+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:47.394+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:47.394+0000 7f41591a6640 10 osd.0:1.consume_map 96 (was 95) 2026-02-11T23:30:47.394+0000 7f41591a6640 10 osd.0:2.consume_map 96 (was 95) 2026-02-11T23:30:47.394+0000 7f41591a6640 10 osd.0:3.consume_map 96 (was 95) 2026-02-11T23:30:47.394+0000 7f41591a6640 10 osd.0:4.consume_map 96 (was 95) 2026-02-11T23:30:47.394+0000 7f41591a6640 10 osd.0:5.consume_map 96 (was 95) 2026-02-11T23:30:47.394+0000 7f41591a6640 10 osd.0:6.consume_map 96 (was 95) 2026-02-11T23:30:47.394+0000 7f41591a6640 10 osd.0:7.consume_map 96 (was 95) 2026-02-11T23:30:47.394+0000 7f41591a6640 10 osd.0 96 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:47.394+0000 7f41591a6640 15 osd.0 96 enqueue_peering_evt 10.0 epoch_sent: 96 epoch_requested: 96 NullEvt 2026-02-11T23:30:47.394+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 96 epoch_requested: 96 NullEvt) class_id 2 prio 255 cost 10 e96) 2026-02-11T23:30:47.394+0000 7f41591a6640 15 osd.0 96 enqueue_peering_evt 4.0 epoch_sent: 96 epoch_requested: 96 NullEvt 2026-02-11T23:30:47.394+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 96 epoch_requested: 96 NullEvt) class_id 2 prio 255 cost 10 e96) 2026-02-11T23:30:47.394+0000 7f41591a6640 15 osd.0 96 enqueue_peering_evt 1.0 epoch_sent: 96 epoch_requested: 96 NullEvt 2026-02-11T23:30:47.394+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 96 epoch_requested: 96 NullEvt) class_id 2 prio 255 cost 10 e96) 2026-02-11T23:30:47.394+0000 7f414f993640 20 osd.0 op_wq(0) _process 10.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:47.394+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 96 epoch_requested: 96 NullEvt) class_id 2immediate prio 255 cost 10 e96) queued 2026-02-11T23:30:47.394+0000 7f414f993640 20 osd.0 op_wq(0) _process 10.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:47.394+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 96 epoch_requested: 96 NullEvt) class_id 2immediate prio 255 cost 10 e96) pg 0x4aa0000 2026-02-11T23:30:47.394+0000 7f41591a6640 7 osd.0 96 activate_map version 96 2026-02-11T23:30:47.394+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:47.394+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 96 epoch_requested: 96 NullEvt) class_id 2immediate prio 255 cost 10 e96) queued 2026-02-11T23:30:47.394+0000 7f414f993640 10 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.647021294s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] handle_advance_map: 96 2026-02-11T23:30:47.394+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:47.394+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 96 epoch_requested: 96 NullEvt) class_id 2immediate prio 255 cost 10 e96) pg 0x5978000 2026-02-11T23:30:47.394+0000 7f414f993640 20 osd.0:0.update_pg_epoch 10.0 95 -> 96 2026-02-11T23:30:47.394+0000 7f4150194640 20 osd.0 96 agent_entry empty queue 2026-02-11T23:30:47.394+0000 7f414f993640 10 osd.0 pg_epoch: 95 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.647021294s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] PeeringState::advance_map handle_advance_map [2,1]/[2,1] -- 2/2 2026-02-11T23:30:47.394+0000 7f414f993640 10 osd.0 pg_epoch: 96 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.647021294s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] state: Started advmap 2026-02-11T23:30:47.394+0000 7f414f993640 10 osd.0 pg_epoch: 96 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.647021294s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:47.394+0000 7f414f993640 10 osd.0 pg_epoch: 96 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.647021294s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] plpg_on_pool_change 2026-02-11T23:30:47.394+0000 7f414f993640 20 osd.0 pg_epoch: 96 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.647021294s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] hit_set_clear 2026-02-11T23:30:47.394+0000 7f414f993640 20 osd.0 pg_epoch: 96 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.647021294s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] agent_stop 2026-02-11T23:30:47.394+0000 7f414f993640 20 osd.0 96 advance_pg new pool opts old pool opts 2026-02-11T23:30:47.394+0000 7f414f993640 10 osd.0 pg_epoch: 96 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.647021294s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] handle_activate_map: epoch range: 96..96 2026-02-11T23:30:47.394+0000 7f414f993640 10 osd.0 pg_epoch: 96 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.647021294s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] PeeringState::activate_map 2026-02-11T23:30:47.394+0000 7f414b98b640 10 osd.0 pg_epoch: 95 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 96 2026-02-11T23:30:47.394+0000 7f414f993640 20 osd.0 pg_epoch: 96 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.548597336s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 95 while current is 96 2026-02-11T23:30:47.394+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:47.394+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 4.0 95 -> 96 2026-02-11T23:30:47.394+0000 7f414f993640 10 osd.0 pg_epoch: 96 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.548597336s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:30:47.394+0000 7f414f993640 10 osd.0 pg_epoch: 96 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.548597336s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] do_peering_event: epoch_sent: 96 epoch_requested: 96 NullEvt 2026-02-11T23:30:47.394+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:47.394+0000 7f414f993640 20 osd.0 96 get_con_osd_cluster to osd.2 from_epoch 96 2026-02-11T23:30:47.394+0000 7f414f993640 20 osd.0 96 get_nextmap_reserved map_reservations: {96=1} 2026-02-11T23:30:47.394+0000 7f414f993640 20 osd.0 96 release_map epoch: 96 2026-02-11T23:30:47.394+0000 7f414b98b640 10 osd.0 pg_epoch: 95 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:47.394+0000 7f414f993640 10 osd.0 96 maybe_share_map: con v2:127.0.0.1:6820/2365263801 map epoch 95 -> 96 (shared) 2026-02-11T23:30:47.394+0000 7f414f993640 10 osd.0 96 send_incremental_map epoch range: (96, 96] to v2:127.0.0.1:6820/2365263801 2026-02-11T23:30:47.394+0000 7f414b98b640 10 osd.0 pg_epoch: 96 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:47.394+0000 7f414b98b640 10 osd.0 pg_epoch: 96 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:47.394+0000 7f414b98b640 20 osd.0 96 advance_pg new pool opts old pool opts 2026-02-11T23:30:47.394+0000 7f414b98b640 10 osd.0 pg_epoch: 96 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_activate_map: epoch range: 96..96 2026-02-11T23:30:47.394+0000 7f414b98b640 10 osd.0 pg_epoch: 96 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:47.394+0000 7f414b98b640 20 osd.0 pg_epoch: 96 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 95 while current is 96 2026-02-11T23:30:47.394+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:47.394+0000 7f414b98b640 10 osd.0 pg_epoch: 96 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 96 epoch_requested: 96 NullEvt 2026-02-11T23:30:47.394+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:47.394+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:47.394+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 96 epoch_requested: 96 NullEvt) class_id 2immediate prio 255 cost 10 e96) queued 2026-02-11T23:30:47.394+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:47.394+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 96 epoch_requested: 96 NullEvt) class_id 2immediate prio 255 cost 10 e96) pg 0x5946000 2026-02-11T23:30:47.394+0000 7f414f993640 10 osd.0 96 add_map_inc_bl 96 582 bytes 2026-02-11T23:30:47.394+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6820/2365263801,v1:127.0.0.1:6821/2365263801] -- osd_map(96..96 src has 1..96) -- 0x4a31180 con 0x507b800 2026-02-11T23:30:47.394+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6820/2365263801,v1:127.0.0.1:6821/2365263801] -- pg_notify2(10.0 10.0 (query:96 sent:96 10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.548597336s) ([92,94] all_participants={0, 1, 2} intervals=[([92,94] acting={0, 1, 2})])) e96/96) -- 0x373b800 con 0x507b800 2026-02-11T23:30:47.394+0000 7f414b98b640 10 osd.0 pg_epoch: 95 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 96 2026-02-11T23:30:47.394+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 1.0 95 -> 96 2026-02-11T23:30:47.394+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:47.394+0000 7f414b98b640 10 osd.0 pg_epoch: 95 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:47.394+0000 7f414b98b640 10 osd.0 pg_epoch: 96 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:47.394+0000 7f414b98b640 10 osd.0 pg_epoch: 96 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:47.394+0000 7f414b98b640 20 osd.0 96 advance_pg new pool opts old pool opts 2026-02-11T23:30:47.394+0000 7f414b98b640 10 osd.0 pg_epoch: 96 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_activate_map: epoch range: 96..96 2026-02-11T23:30:47.394+0000 7f414b98b640 10 osd.0 pg_epoch: 96 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:47.394+0000 7f414b98b640 20 osd.0 pg_epoch: 96 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 73 while current is 96 2026-02-11T23:30:47.394+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:47.394+0000 7f414b98b640 10 osd.0 pg_epoch: 96 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 96 epoch_requested: 96 NullEvt 2026-02-11T23:30:47.394+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:47.394+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:47.404+0000 7f41633eb640 10 osd.0 96 tick 2026-02-11T23:30:47.404+0000 7f41633eb640 20 osd.0 96 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:30:47.615+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.2 v2:127.0.0.1:6820/2365263801 13 ==== osd pg remove(epoch 97; pg10.0; ) ==== 32+0+0 (crc 0 0 0) 0x3759600 con 0x507b800 2026-02-11T23:30:47.615+0000 7f416cbfe640 7 osd.0 96 handle_fast_pg_remove osd pg remove(epoch 97; pg10.0; ) from osd.2 2026-02-11T23:30:47.616+0000 7f416cbfe640 15 osd.0 96 enqueue_peering_evt 10.0 epoch_sent: 97 epoch_requested: 97 DeleteStart 2026-02-11T23:30:47.616+0000 7f416cbfe640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 97 epoch_requested: 97 DeleteStart) class_id 2 prio 255 cost 10 e97) 2026-02-11T23:30:47.616+0000 7f414f993640 20 osd.0 op_wq(0) _process 10.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:47.616+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 97 epoch_requested: 97 DeleteStart) class_id 2immediate prio 255 cost 10 e97) queued 2026-02-11T23:30:47.616+0000 7f414f993640 20 osd.0 op_wq(0) _process 10.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:47.616+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 97 epoch_requested: 97 DeleteStart) class_id 2immediate prio 255 cost 10 e97) pg 0x4aa0000 2026-02-11T23:30:47.616+0000 7f414f993640 20 osd.0 op_wq _add_slot_waiter 10.0 peering, item epoch is 97, will wait on OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 97 epoch_requested: 97 DeleteStart) class_id 2immediate prio 255 cost 10 e97) 2026-02-11T23:30:47.616+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:47.616+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:48.205+0000 7f415b6ff640 20 osd.0 96 reports for 0 queries 2026-02-11T23:30:48.205+0000 7f415b6ff640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] -- mgrreport(unknown.0 +0-0 packed 3294 daemon_metrics=2) -- 0x4f9f880 con 0x4aca800 2026-02-11T23:30:48.205+0000 7f415b6ff640 15 osd.0 96 collect_pg_stats 2026-02-11T23:30:48.205+0000 7f415b6ff640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] -- pg_stats(0 pgs seq 21474836487 v 0) -- 0x5ab2600 con 0x4aca800 2026-02-11T23:30:48.328+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:30:48.328+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:30:48.328+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:18.328248+0000) 2026-02-11T23:30:48.384+0000 7f4162bea640 10 osd.0 96 tick_without_osd_lock 2026-02-11T23:30:48.384+0000 7f4162bea640 0 osd.0 96 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:48.384+0000 7f4162bea640 20 osd.0 96 tick_without_osd_lock max_waiting_epoch 97, requesting new map 2026-02-11T23:30:48.384+0000 7f4162bea640 10 monclient: _renew_subs 2026-02-11T23:30:48.384+0000 7f4162bea640 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:30:48.384+0000 7f4162bea640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> v2:127.0.0.1:7180/0 -- mon_subscribe({osdmap=97}) -- 0x38144e0 con 0x4acbc00 2026-02-11T23:30:48.384+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:48.384+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.399 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:48.384+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:48.385, recovery is active?:false restrictions:<....> 2026-02-11T23:30:48.384+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:48.384+0000 7f4162bea640 10 osd.0 96 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:48.384+0000 7f4162bea640 20 osd.0 96 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:48.384+0000 7f4162bea640 10 osd.0 96 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:48.384+0000 7f4162bea640 20 osd.0 96 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:48.385+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 46 ==== osd_map(97..98 src has 1..98) ==== 1200+0+0 (crc 0 0 0) 0x4a31180 con 0x4acbc00 2026-02-11T23:30:48.385+0000 7f415cf02640 20 osd.0 96 OSD::ms_dispatch: osd_map(97..98 src has 1..98) 2026-02-11T23:30:48.385+0000 7f415cf02640 20 osd.0 96 _dispatch 0x4a31180 osd_map(97..98 src has 1..98) 2026-02-11T23:30:48.385+0000 7f415cf02640 3 osd.0 96 handle_osd_map epochs [97,98], i have 96, src has [1,98] 2026-02-11T23:30:48.385+0000 7f415cf02640 10 osd.0 96 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:48.385+0000 7f415cf02640 10 osd.0 96 handle_osd_map got inc map for epoch 97 2026-02-11T23:30:48.385+0000 7f415cf02640 20 osd.0 96 got_full_map 97, nothing requested 2026-02-11T23:30:48.385+0000 7f415cf02640 10 osd.0 96 handle_osd_map got inc map for epoch 98 2026-02-11T23:30:48.385+0000 7f415cf02640 20 osd.0 96 got_full_map 98, nothing requested 2026-02-11T23:30:48.385+0000 7f415cf02640 20 osd.0 96 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:48.385+0000 7f415cf02640 20 osd.0 96 handle_osd_map pg_num_history pg_num_history(e98 pg_nums {1={27=1},2={34=1},3={42=1},4={49=1},5={56=1},6={64=1},7={70=1},8={77=1},9={85=1},10={92=1}} deleted_pools ) 2026-02-11T23:30:48.385+0000 7f415cf02640 10 snap_mapper record_purged_snaps purged_snaps {97={},98={}} 2026-02-11T23:30:48.385+0000 7f415cf02640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:48.385+0000 7f415cf02640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e98 maps [1~98] lci=[5,98] tlb=1) 2026-02-11T23:30:48.385+0000 7f41591a6640 10 osd.0 96 _committed_osd_maps 97..98 2026-02-11T23:30:48.385+0000 7f41591a6640 10 osd.0 96 advance to epoch 97 (<= last 98 <= newest_map 98) 2026-02-11T23:30:48.385+0000 7f41591a6640 10 osd.0 97 advance to epoch 98 (<= last 98 <= newest_map 98) 2026-02-11T23:30:48.385+0000 7f41591a6640 20 osd.0 98 consume_map version 98 2026-02-11T23:30:48.385+0000 7f41591a6640 20 osd.0 96 await_reserved_maps epoch:98 2026-02-11T23:30:48.385+0000 7f41591a6640 20 osd.0 96 await_reserved_maps done 0 2026-02-11T23:30:48.385+0000 7f41591a6640 20 osd.0 98 consume_map 98 -- publish done 2026-02-11T23:30:48.385+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 3 slots 2026-02-11T23:30:48.385+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:10.0slot:0x37594a0 2026-02-11T23:30:48.385+0000 7f41591a6640 20 osd.0 98 identify_splits_and_merges 10.0 e96 to e98 2026-02-11T23:30:48.385+0000 7f41591a6640 20 osd.0 98 identify_splits_and_merges 10.0 e96 to e98 pg_nums {92=1} 2026-02-11T23:30:48.385+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x37582c0 2026-02-11T23:30:48.385+0000 7f41591a6640 20 osd.0 98 identify_splits_and_merges 4.0 e96 to e98 2026-02-11T23:30:48.385+0000 7f41591a6640 20 osd.0 98 identify_splits_and_merges 4.0 e96 to e98 pg_nums {49=1} 2026-02-11T23:30:48.385+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3700dc0 2026-02-11T23:30:48.385+0000 7f41591a6640 20 osd.0 98 identify_splits_and_merges 1.0 e96 to e98 2026-02-11T23:30:48.385+0000 7f41591a6640 20 osd.0 98 identify_splits_and_merges 1.0 e96 to e98 pg_nums {27=1} 2026-02-11T23:30:48.385+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:48.385+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:48.385+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:48.385+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:48.385+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:48.385+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:48.385+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:48.385+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:48.385+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:48.385+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:48.385+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:48.385+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:48.385+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:48.385+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:48.385+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:48.385+0000 7f41591a6640 20 osd.0 98 prune_pg_created 2026-02-11T23:30:48.385+0000 7f41591a6640 10 osd.0:0.consume_map 98 (was 96) 2026-02-11T23:30:48.385+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 10.0 2026-02-11T23:30:48.385+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 10.0 pending_peering first epoch 97 <= 98, requeueing 2026-02-11T23:30:48.385+0000 7f41591a6640 20 osd.0:0._wake_pg_slot _wake_pg_slot 10.0 to_process <> waiting <> waiting_peering {97=} 2026-02-11T23:30:48.385+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:48.385+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:48.385+0000 7f41591a6640 10 osd.0:1.consume_map 98 (was 96) 2026-02-11T23:30:48.385+0000 7f41591a6640 10 osd.0:2.consume_map 98 (was 96) 2026-02-11T23:30:48.385+0000 7f41591a6640 10 osd.0:3.consume_map 98 (was 96) 2026-02-11T23:30:48.385+0000 7f41591a6640 10 osd.0:4.consume_map 98 (was 96) 2026-02-11T23:30:48.385+0000 7f41591a6640 10 osd.0:5.consume_map 98 (was 96) 2026-02-11T23:30:48.385+0000 7f41591a6640 10 osd.0:6.consume_map 98 (was 96) 2026-02-11T23:30:48.385+0000 7f41591a6640 10 osd.0:7.consume_map 98 (was 96) 2026-02-11T23:30:48.385+0000 7f41591a6640 10 osd.0 98 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:48.385+0000 7f41591a6640 15 osd.0 98 enqueue_peering_evt 10.0 epoch_sent: 98 epoch_requested: 98 NullEvt 2026-02-11T23:30:48.385+0000 7f414f993640 20 osd.0 op_wq(0) _process 10.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:48.385+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 97 epoch_requested: 97 DeleteStart) class_id 2immediate prio 255 cost 10 e97) queued 2026-02-11T23:30:48.385+0000 7f414f993640 20 osd.0 op_wq(0) _process 10.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:48.385+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 98 epoch_requested: 98 NullEvt) class_id 2 prio 255 cost 10 e98) 2026-02-11T23:30:48.385+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 97 epoch_requested: 97 DeleteStart) class_id 2immediate prio 255 cost 10 e97) pg 0x4aa0000 2026-02-11T23:30:48.385+0000 7f41591a6640 15 osd.0 98 enqueue_peering_evt 4.0 epoch_sent: 98 epoch_requested: 98 NullEvt 2026-02-11T23:30:48.385+0000 7f414b98b640 20 osd.0 op_wq(0) _process 10.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:48.385+0000 7f414f993640 10 osd.0 pg_epoch: 96 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.548597336s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] handle_advance_map: 97 2026-02-11T23:30:48.385+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 98 epoch_requested: 98 NullEvt) class_id 2immediate prio 255 cost 10 e98) queued 2026-02-11T23:30:48.385+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 98 epoch_requested: 98 NullEvt) class_id 2 prio 255 cost 10 e98) 2026-02-11T23:30:48.385+0000 7f414f993640 20 osd.0:0.update_pg_epoch 10.0 96 -> 97 2026-02-11T23:30:48.385+0000 7f41591a6640 15 osd.0 98 enqueue_peering_evt 1.0 epoch_sent: 98 epoch_requested: 98 NullEvt 2026-02-11T23:30:48.385+0000 7f414f993640 10 osd.0 pg_epoch: 96 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.548597336s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] PeeringState::advance_map handle_advance_map [2,1]/[2,1] -- 2/2 2026-02-11T23:30:48.385+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 98 epoch_requested: 98 NullEvt) class_id 2 prio 255 cost 10 e98) 2026-02-11T23:30:48.385+0000 7f41591a6640 7 osd.0 98 activate_map version 98 2026-02-11T23:30:48.385+0000 7f414f993640 10 osd.0 pg_epoch: 97 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.548597336s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] state: Started advmap 2026-02-11T23:30:48.385+0000 7f414f993640 10 osd.0 pg_epoch: 97 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.548597336s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:48.385+0000 7f414f993640 10 osd.0 pg_epoch: 97 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.548597336s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] plpg_on_pool_change 2026-02-11T23:30:48.385+0000 7f4150194640 20 osd.0 98 agent_entry empty queue 2026-02-11T23:30:48.385+0000 7f414f993640 20 osd.0 pg_epoch: 97 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.548597336s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] hit_set_clear 2026-02-11T23:30:48.385+0000 7f414f993640 20 osd.0 pg_epoch: 97 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.548597336s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] agent_stop 2026-02-11T23:30:48.385+0000 7f414f993640 20 osd.0 98 advance_pg new pool opts old pool opts 2026-02-11T23:30:48.385+0000 7f414f993640 10 osd.0 pg_epoch: 97 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.548597336s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] handle_advance_map: 98 2026-02-11T23:30:48.385+0000 7f414f993640 20 osd.0:0.update_pg_epoch 10.0 97 -> 98 2026-02-11T23:30:48.385+0000 7f414f993640 10 osd.0 pg_epoch: 97 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.548597336s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] PeeringState::advance_map handle_advance_map [2,1]/[2,1] -- 2/2 2026-02-11T23:30:48.385+0000 7f414f993640 10 osd.0 pg_epoch: 98 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.548597336s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] state: Started advmap 2026-02-11T23:30:48.385+0000 7f414f993640 10 osd.0 pg_epoch: 98 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.548597336s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:48.385+0000 7f414f993640 10 osd.0 pg_epoch: 98 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.548597336s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] plpg_on_pool_change 2026-02-11T23:30:48.385+0000 7f414f993640 20 osd.0 pg_epoch: 98 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.548597336s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] hit_set_clear 2026-02-11T23:30:48.385+0000 7f414f993640 20 osd.0 pg_epoch: 98 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.548597336s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] agent_stop 2026-02-11T23:30:48.385+0000 7f414f993640 20 osd.0 98 advance_pg new pool opts old pool opts 2026-02-11T23:30:48.385+0000 7f414f993640 10 osd.0 pg_epoch: 98 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.548597336s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] handle_activate_map: epoch range: 97..98 2026-02-11T23:30:48.385+0000 7f414f993640 10 osd.0 pg_epoch: 98 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=15.548597336s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] PeeringState::activate_map 2026-02-11T23:30:48.385+0000 7f414f993640 20 osd.0 pg_epoch: 98 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=14.556807518s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 95 while current is 98 2026-02-11T23:30:48.385+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:48.385+0000 7f414f993640 10 osd.0 pg_epoch: 98 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=14.556807518s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:30:48.385+0000 7f414f993640 10 osd.0 pg_epoch: 98 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=14.556807518s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] do_peering_event: epoch_sent: 97 epoch_requested: 97 DeleteStart 2026-02-11T23:30:48.385+0000 7f414f993640 5 osd.0 pg_epoch: 98 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=14.556807518s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] exit Started/Stray 1.090256 9 0.000054 2026-02-11T23:30:48.385+0000 7f414f993640 5 osd.0 pg_epoch: 98 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=14.556807518s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] enter Started/ToDelete 2026-02-11T23:30:48.385+0000 7f414f993640 5 osd.0 pg_epoch: 98 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=14.556807518s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] enter Started/ToDelete/WaitDeleteReserved 2026-02-11T23:30:48.385+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:48.385+0000 7f414f993640 20 osd.0 98 get_con_osd_cluster to osd.2 from_epoch 98 2026-02-11T23:30:48.385+0000 7f414f993640 20 osd.0 98 get_nextmap_reserved map_reservations: {98=1} 2026-02-11T23:30:48.385+0000 7f414f993640 20 osd.0 98 release_map epoch: 98 2026-02-11T23:30:48.385+0000 7f414f993640 10 osd.0 98 maybe_share_map: con v2:127.0.0.1:6820/2365263801 map epoch 96 -> 98 (shared) 2026-02-11T23:30:48.385+0000 7f414f993640 10 osd.0 98 send_incremental_map epoch range: (97, 98] to v2:127.0.0.1:6820/2365263801 2026-02-11T23:30:48.385+0000 7f414f993640 10 osd.0 98 add_map_inc_bl 97 574 bytes 2026-02-11T23:30:48.386+0000 7f414f993640 10 osd.0 98 add_map_inc_bl 98 574 bytes 2026-02-11T23:30:48.386+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6820/2365263801,v1:127.0.0.1:6821/2365263801] -- osd_map(97..98 src has 1..98) -- 0x4a316c0 con 0x507b800 2026-02-11T23:30:48.386+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6820/2365263801,v1:127.0.0.1:6821/2365263801] -- pg_notify2(10.0 10.0 (query:98 sent:98 10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=14.556807518s) ([92,94] all_participants={0, 1, 2} intervals=[([92,94] acting={0, 1, 2})])) e98/98) -- 0x5b22000 con 0x507b800 2026-02-11T23:30:48.386+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:48.386+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 98 epoch_requested: 98 NullEvt) class_id 2immediate prio 255 cost 10 e98) queued 2026-02-11T23:30:48.386+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:48.386+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 98 epoch_requested: 98 NullEvt) class_id 2immediate prio 255 cost 10 e98) pg 0x5978000 2026-02-11T23:30:48.386+0000 7f414b98b640 20 osd.0 op_wq(0) _process 10.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:48.386+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 98 epoch_requested: 98 NullEvt) class_id 2immediate prio 255 cost 10 e98) pg 0x4aa0000 2026-02-11T23:30:48.386+0000 7f414f993640 10 osd.0 pg_epoch: 96 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 97 2026-02-11T23:30:48.386+0000 7f414f993640 20 osd.0:0.update_pg_epoch 4.0 96 -> 97 2026-02-11T23:30:48.386+0000 7f414f993640 10 osd.0 pg_epoch: 96 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:48.386+0000 7f414b98b640 10 osd.0 pg_epoch: 98 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=14.556807518s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] do_peering_event: epoch_sent: 98 epoch_requested: 98 NullEvt 2026-02-11T23:30:48.386+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:48.386+0000 7f414f993640 10 osd.0 pg_epoch: 97 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:48.386+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:48.386+0000 7f414f993640 10 osd.0 pg_epoch: 97 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:48.386+0000 7f4152999640 15 osd.0 98 enqueue_peering_evt 10.0 epoch_sent: 98 epoch_requested: 98 DeleteReserved 2026-02-11T23:30:48.386+0000 7f414f993640 20 osd.0 98 advance_pg new pool opts old pool opts 2026-02-11T23:30:48.386+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 98 epoch_requested: 98 NullEvt) class_id 2immediate prio 255 cost 10 e98) queued 2026-02-11T23:30:48.386+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:48.386+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 98 epoch_requested: 98 NullEvt) class_id 2immediate prio 255 cost 10 e98) pg 0x5946000 2026-02-11T23:30:48.386+0000 7f414f993640 10 osd.0 pg_epoch: 97 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 98 2026-02-11T23:30:48.386+0000 7f414f993640 20 osd.0:0.update_pg_epoch 4.0 97 -> 98 2026-02-11T23:30:48.386+0000 7f4152999640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 98 epoch_requested: 98 DeleteReserved) class_id 2 prio 255 cost 10 e98) 2026-02-11T23:30:48.386+0000 7f414b98b640 10 osd.0 pg_epoch: 96 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 97 2026-02-11T23:30:48.386+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 1.0 96 -> 97 2026-02-11T23:30:48.386+0000 7f414f993640 10 osd.0 pg_epoch: 97 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:48.386+0000 7f414b98b640 10 osd.0 pg_epoch: 96 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:48.386+0000 7f414f993640 10 osd.0 pg_epoch: 98 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:48.386+0000 7f414b98b640 10 osd.0 pg_epoch: 97 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:48.386+0000 7f414b98b640 10 osd.0 pg_epoch: 97 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:48.386+0000 7f414f993640 10 osd.0 pg_epoch: 98 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:48.386+0000 7f414b98b640 20 osd.0 98 advance_pg new pool opts old pool opts 2026-02-11T23:30:48.386+0000 7f414f993640 20 osd.0 98 advance_pg new pool opts old pool opts 2026-02-11T23:30:48.386+0000 7f414b98b640 10 osd.0 pg_epoch: 97 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 98 2026-02-11T23:30:48.386+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 1.0 97 -> 98 2026-02-11T23:30:48.386+0000 7f414b98b640 10 osd.0 pg_epoch: 97 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:48.386+0000 7f414f993640 10 osd.0 pg_epoch: 98 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_activate_map: epoch range: 97..98 2026-02-11T23:30:48.386+0000 7f414b98b640 10 osd.0 pg_epoch: 98 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:48.386+0000 7f414f993640 10 osd.0 pg_epoch: 98 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:48.386+0000 7f414b98b640 10 osd.0 pg_epoch: 98 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:48.386+0000 7f414b98b640 20 osd.0 98 advance_pg new pool opts old pool opts 2026-02-11T23:30:48.386+0000 7f414f993640 20 osd.0 pg_epoch: 98 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 95 while current is 98 2026-02-11T23:30:48.386+0000 7f414b98b640 10 osd.0 pg_epoch: 98 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_activate_map: epoch range: 97..98 2026-02-11T23:30:48.386+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:48.386+0000 7f414b98b640 10 osd.0 pg_epoch: 98 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:48.386+0000 7f414f993640 10 osd.0 pg_epoch: 98 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 98 epoch_requested: 98 NullEvt 2026-02-11T23:30:48.386+0000 7f414b98b640 20 osd.0 pg_epoch: 98 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 73 while current is 98 2026-02-11T23:30:48.386+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:48.386+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:48.386+0000 7f414f993640 20 osd.0 op_wq(0) _process 10.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:48.386+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 98 epoch_requested: 98 DeleteReserved) class_id 2immediate prio 255 cost 10 e98) queued 2026-02-11T23:30:48.386+0000 7f414f993640 20 osd.0 op_wq(0) _process 10.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:48.386+0000 7f414b98b640 10 osd.0 pg_epoch: 98 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 98 epoch_requested: 98 NullEvt 2026-02-11T23:30:48.386+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:48.386+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 98 epoch_requested: 98 DeleteReserved) class_id 2immediate prio 255 cost 10 e98) pg 0x4aa0000 2026-02-11T23:30:48.386+0000 7f414f993640 10 osd.0 pg_epoch: 98 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=14.556807518s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] do_peering_event: epoch_sent: 98 epoch_requested: 98 DeleteReserved 2026-02-11T23:30:48.386+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:48.386+0000 7f414f993640 5 osd.0 pg_epoch: 98 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=14.556807518s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] exit Started/ToDelete/WaitDeleteReserved 0.000277 2 0.000022 2026-02-11T23:30:48.386+0000 7f414f993640 5 osd.0 pg_epoch: 98 pg[10.0( empty local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=14.556807518s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] enter Started/ToDelete/Deleting 2026-02-11T23:30:48.386+0000 7f414f993640 10 osd.0 pg_epoch: 98 pg[10.0( empty lb MIN local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=14.556807518s) [2,1] r=-1 lpr=95 DELETING pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] on_removal 2026-02-11T23:30:48.386+0000 7f414f993640 10 osd.0 pg_epoch: 98 pg[10.0( empty lb MIN local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=14.556807518s) [2,1] r=-1 lpr=95 DELETING pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] on_shutdown 2026-02-11T23:30:48.386+0000 7f414f993640 10 osd.0 pg_epoch: 98 pg[10.0( empty lb MIN local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=14.556807518s) [2,1] r=-1 lpr=95 DELETING pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:48.386+0000 7f414f993640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:48.386+0000 7f414f993640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:48.386+0000 7f414f993640 10 osd.0 pg_epoch: 98 pg[10.0( empty lb MIN local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=14.556807518s) [2,1] r=-1 lpr=95 DELETING pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] cancel_copy_ops 2026-02-11T23:30:48.386+0000 7f414f993640 10 osd.0 pg_epoch: 98 pg[10.0( empty lb MIN local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=14.556807518s) [2,1] r=-1 lpr=95 DELETING pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] cancel_flush_ops 2026-02-11T23:30:48.386+0000 7f414f993640 10 osd.0 pg_epoch: 98 pg[10.0( empty lb MIN local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=14.556807518s) [2,1] r=-1 lpr=95 DELETING pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] cancel_proxy_ops 2026-02-11T23:30:48.386+0000 7f414f993640 10 osd.0 pg_epoch: 98 pg[10.0( empty lb MIN local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=14.556807518s) [2,1] r=-1 lpr=95 DELETING pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] cancel_manifest_ops 2026-02-11T23:30:48.386+0000 7f414f993640 10 osd.0 pg_epoch: 98 pg[10.0( empty lb MIN local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=14.556807518s) [2,1] r=-1 lpr=95 DELETING pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] cancel_cls_gather_ops 2026-02-11T23:30:48.386+0000 7f414f993640 10 osd.0 pg_epoch: 98 pg[10.0( empty lb MIN local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=14.556807518s) [2,1] r=-1 lpr=95 DELETING pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] clear_backoffs 2026-02-11T23:30:48.386+0000 7f414f993640 20 osd.0 pg_epoch: 98 pg[10.0( empty lb MIN local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=14.556807518s) [2,1] r=-1 lpr=95 DELETING pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] exit NotTrimming 2026-02-11T23:30:48.386+0000 7f414f993640 20 osd.0 pg_epoch: 98 pg[10.0( empty lb MIN local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=14.556807518s) [2,1] r=-1 lpr=95 DELETING pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] enter NotTrimming 2026-02-11T23:30:48.386+0000 7f414f993640 10 osd.0 pg_epoch: 98 pg[10.0( empty lb MIN local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=14.556807518s) [2,1] r=-1 lpr=95 DELETING pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] on_change 2026-02-11T23:30:48.386+0000 7f414f993640 10 osd.0 pg_epoch: 98 pg[10.0( empty lb MIN local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=14.556807518s) [2,1] r=-1 lpr=95 DELETING pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] clear_async_reads 2026-02-11T23:30:48.386+0000 7f414f993640 20 osd.0 pg_epoch: 98 pg[10.0( empty lb MIN local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=14.556807518s) [2,1] r=-1 lpr=95 DELETING pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] clear_primary_state 2026-02-11T23:30:48.386+0000 7f414f993640 10 osd.0 pg_epoch: 98 pg[10.0( empty lb MIN local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=14.556807518s) [2,1] r=-1 lpr=95 DELETING pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] release_backoffs [10:00000000::::0,MAX) 2026-02-11T23:30:48.386+0000 7f414f993640 10 osd.0 pg_epoch: 98 pg[10.0( empty lb MIN local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=14.556807518s) [2,1] r=-1 lpr=95 DELETING pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:48.386+0000 7f414f993640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:48.386+0000 7f414f993640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:48.386+0000 7f414f993640 20 osd.0 pg_epoch: 98 pg[10.0( empty lb MIN local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=14.556807518s) [2,1] r=-1 lpr=95 DELETING pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] agent_stop 2026-02-11T23:30:48.386+0000 7f414f993640 10 osd.0 pg_epoch: 98 pg[10.0( empty lb MIN local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=14.556807518s) [2,1] r=-1 lpr=95 DELETING pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] cancel_recovery 2026-02-11T23:30:48.386+0000 7f414f993640 10 osd.0 pg_epoch: 98 pg[10.0( empty lb MIN local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=14.556807518s) [2,1] r=-1 lpr=95 DELETING pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] clear_recovery_state 2026-02-11T23:30:48.386+0000 7f414f993640 15 osd.0 pg_epoch: 98 pg[10.0( empty lb MIN local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=14.556807518s) [2,1] r=-1 lpr=95 DELETING pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] _clear_recovery_state 2026-02-11T23:30:48.386+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:48.386+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:48.386+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:48.386+0000 7f414f993640 10 osd.0 98 queue_for_pg_delete on 10.0 e 98 2026-02-11T23:30:48.386+0000 7f414f993640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(10.0 PGDelete(pgid=10.0 epoch_queued=98) class_id 1 prio 5 cost 1048576 e98) 2026-02-11T23:30:48.386+0000 7f414f993640 20 osd.0 op_wq(0) _process 10.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:48.386+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(10.0 PGDelete(10.0 e98) class_id 1background_best_effort prio 5 qos_cost 1048576 cost 1048576 e98) queued 2026-02-11T23:30:48.386+0000 7f414f993640 20 osd.0 op_wq(0) _process 10.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:48.386+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(10.0 PGDelete(10.0 e98) class_id 1background_best_effort prio 5 qos_cost 1048576 cost 1048576 e98) pg 0x4aa0000 2026-02-11T23:30:48.386+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:48.386+0000 7f414f993640 10 osd.0 pg_epoch: 98 pg[10.0( empty lb MIN local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=14.556807518s) [2,1] r=-1 lpr=95 DELETING pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] do_peering_event: epoch_sent: 98 epoch_requested: 98 DeleteSome 2026-02-11T23:30:48.386+0000 7f414f993640 10 osd.0 pg_epoch: 98 pg[10.0( empty lb MIN local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=14.556807518s) [2,1] r=-1 lpr=95 DELETING pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] do_delete_work 2026-02-11T23:30:48.386+0000 7f414f993640 20 osd.0 pg_epoch: 98 pg[10.0( empty lb MIN local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=14.556807518s) [2,1] r=-1 lpr=95 DELETING pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] do_delete_work [#10:00000000::::head#] 2026-02-11T23:30:48.386+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:48.386+0000 7f414f993640 20 osd.0 98 try_finish_pg_delete 10.0 0x4aa0000 2026-02-11T23:30:48.386+0000 7f414f993640 10 osd.0:0._detach_pg 10.0 0x4aa0000 2026-02-11T23:30:48.386+0000 7f414f993640 5 osd.0 pg_epoch: 98 pg[10.0( empty lb MIN local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=14.556807518s) [2,1] r=-1 lpr=95 DELETING pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] exit Started/ToDelete/Deleting 0.000440 1 0.000073 2026-02-11T23:30:48.386+0000 7f414f993640 5 osd.0 pg_epoch: 98 pg[10.0( empty lb MIN local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=14.556807518s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] exit Started/ToDelete 0.000732 0 0.000000 2026-02-11T23:30:48.386+0000 7f414f993640 5 osd.0 pg_epoch: 98 pg[10.0( empty lb MIN local-lis/les=92/93 n=0 ec=92/92 lis/c=92/92 les/c/f=93/93/0 sis=95 pruub=14.556807518s) [2,1] r=-1 lpr=95 pi=[92,95)/1 crt=0'0 unknown NOTIFY pruub 45.618358612s@ mbc={}] exit Started 1.091002 0 0.000000 2026-02-11T23:30:48.386+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:48.386+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:48.386+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:48.386+0000 7f414f993640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:48.386+0000 7f414f993640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:48.386+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:48.403+0000 7f41633eb640 10 osd.0 98 tick 2026-02-11T23:30:48.403+0000 7f41633eb640 20 osd.0 98 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:30:48.829+0000 7f4147983640 5 osd.0 98 heartbeat osd_stat(store_statfs(0x384000/0x0/0x384000, data 0x6bdf1/0xf2000, compress 0x0/0x0/0x0, omap 0x9236, meta 0x1a46dca), peers [1,2,3,4] op hist []) 2026-02-11T23:30:48.829+0000 7f4147983640 20 osd.0 98 check_full_status cur ratio 0, physical ratio 0, new state none 2026-02-11T23:30:48.829+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6816/3043114013,v1:127.0.0.1:6817/3043114013] -- osd_ping(ping e98 up_from 5 ping_stamp 2026-02-11T23:30:48.829379+0000/31.504785538s send_stamp 31.504785538s delta_ub 2.011759043s) -- 0x4c9bb00 con 0x48c4400 2026-02-11T23:30:48.829+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6814/3043114013,v1:127.0.0.1:6815/3043114013] -- osd_ping(ping e98 up_from 5 ping_stamp 2026-02-11T23:30:48.829379+0000/31.504785538s send_stamp 31.504785538s delta_ub 2.011759043s) -- 0x4c9a180 con 0x4640c00 2026-02-11T23:30:48.829+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6824/2365263801,v1:127.0.0.1:6825/2365263801] -- osd_ping(ping e98 up_from 5 ping_stamp 2026-02-11T23:30:48.829379+0000/31.504785538s send_stamp 31.504785538s delta_ub 4.567098141s) -- 0x4c9a300 con 0x48c4000 2026-02-11T23:30:48.829+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6822/2365263801,v1:127.0.0.1:6823/2365263801] -- osd_ping(ping e98 up_from 5 ping_stamp 2026-02-11T23:30:48.829379+0000/31.504785538s send_stamp 31.504785538s delta_ub 4.567098141s) -- 0x4c9bc80 con 0x48cd000 2026-02-11T23:30:48.829+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6832/2699377081,v1:127.0.0.1:6833/2699377081] -- osd_ping(ping e98 up_from 5 ping_stamp 2026-02-11T23:30:48.829379+0000/31.504785538s send_stamp 31.504785538s delta_ub 7.219576359s) -- 0x4c9b500 con 0x48cd400 2026-02-11T23:30:48.829+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6830/2699377081,v1:127.0.0.1:6831/2699377081] -- osd_ping(ping e98 up_from 5 ping_stamp 2026-02-11T23:30:48.829379+0000/31.504785538s send_stamp 31.504785538s delta_ub 7.219576359s) -- 0x4c9b980 con 0x48cd800 2026-02-11T23:30:48.829+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6840/3331151400,v1:127.0.0.1:6841/3331151400] -- osd_ping(ping e98 up_from 5 ping_stamp 2026-02-11T23:30:48.829379+0000/31.504785538s send_stamp 31.504785538s delta_ub 9.836857796s) -- 0x4c9a600 con 0x507bc00 2026-02-11T23:30:48.829+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6838/3331151400,v1:127.0.0.1:6839/3331151400] -- osd_ping(ping e98 up_from 5 ping_stamp 2026-02-11T23:30:48.829379+0000/31.504785538s send_stamp 31.504785538s delta_ub 9.836857796s) -- 0x4c9b200 con 0x5282000 2026-02-11T23:30:48.829+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6816/3043114013 9 ==== osd_ping(ping_reply e97 up_from 10 ping_stamp 2026-02-11T23:30:48.829379+0000/31.504785538s send_stamp 29.493196487s delta_ub -2.011588335s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a600 con 0x48c4400 2026-02-11T23:30:48.829+0000 7f416abfa640 20 osd.0 98 get_con_osd_cluster to osd.1 from_epoch 98 2026-02-11T23:30:48.829+0000 7f416abfa640 20 osd.0 98 get_nextmap_reserved map_reservations: {98=1} 2026-02-11T23:30:48.829+0000 7f416abfa640 20 osd.0 98 release_map epoch: 98 2026-02-11T23:30:48.829+0000 7f416abfa640 10 osd.0 98 maybe_share_map: con v2:127.0.0.1:6812/3043114013 updating session's projected_epoch from 93 to ping map epoch of 97 2026-02-11T23:30:48.829+0000 7f416abfa640 10 osd.0 98 maybe_share_map: con v2:127.0.0.1:6812/3043114013 map epoch 97 -> 98 (shared) 2026-02-11T23:30:48.829+0000 7f416abfa640 10 osd.0 98 send_incremental_map epoch range: (98, 98] to v2:127.0.0.1:6812/3043114013 2026-02-11T23:30:48.829+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- osd_map(98..98 src has 1..98) -- 0x5954fc0 con 0x4acb000 2026-02-11T23:30:48.829+0000 7f416abfa640 20 osd.0 98 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011723042s,-2.011588335s]) 2026-02-11T23:30:48.829+0000 7f416cbfe640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6824/2365263801 8 ==== osd_ping(ping_reply e98 up_from 14 ping_stamp 2026-02-11T23:30:48.829379+0000/31.504785538s send_stamp 26.938104630s delta_ub -4.566680908s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a300 con 0x48c4000 2026-02-11T23:30:48.829+0000 7f416cbfe640 20 osd.0 98 get_con_osd_cluster to osd.2 from_epoch 98 2026-02-11T23:30:48.829+0000 7f416cbfe640 20 osd.0 98 get_nextmap_reserved map_reservations: {98=1} 2026-02-11T23:30:48.829+0000 7f416cbfe640 20 osd.0 98 release_map epoch: 98 2026-02-11T23:30:48.829+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6830/2699377081 8 ==== osd_ping(ping_reply e98 up_from 19 ping_stamp 2026-02-11T23:30:48.829379+0000/31.504785538s send_stamp 24.285446167s delta_ub -7.219338417s) ==== 2033+0+0 (crc 0 0 0) 0x4c9b200 con 0x48cd800 2026-02-11T23:30:48.829+0000 7f416cbfe640 10 osd.0 98 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 98 is not newer than session's projected_epoch of 98 2026-02-11T23:30:48.829+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6822/2365263801 8 ==== osd_ping(ping_reply e98 up_from 14 ping_stamp 2026-02-11T23:30:48.829379+0000/31.504785538s send_stamp 26.938085556s delta_ub -4.566699982s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a600 con 0x48cd000 2026-02-11T23:30:48.829+0000 7f416cbfe640 20 osd.0 98 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566875935s,-4.566680908s]) 2026-02-11T23:30:48.829+0000 7f416abfa640 20 osd.0 98 get_con_osd_cluster to osd.2 from_epoch 98 2026-02-11T23:30:48.829+0000 7f416abfa640 20 osd.0 98 get_nextmap_reserved map_reservations: {98=1} 2026-02-11T23:30:48.829+0000 7f416abfa640 20 osd.0 98 release_map epoch: 98 2026-02-11T23:30:48.829+0000 7f416abfa640 10 osd.0 98 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 98 is not newer than session's projected_epoch of 98 2026-02-11T23:30:48.829+0000 7f416abfa640 20 osd.0 98 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566906929s,-4.566699982s]) 2026-02-11T23:30:48.829+0000 7f416c3fd640 20 osd.0 98 get_con_osd_cluster to osd.3 from_epoch 98 2026-02-11T23:30:48.829+0000 7f416c3fd640 20 osd.0 98 get_nextmap_reserved map_reservations: {98=1} 2026-02-11T23:30:48.829+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6840/3331151400 7 ==== osd_ping(ping_reply e98 up_from 23 ping_stamp 2026-02-11T23:30:48.829379+0000/31.504785538s send_stamp 21.668292999s delta_ub -9.836490631s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a600 con 0x507bc00 2026-02-11T23:30:48.829+0000 7f416c3fd640 20 osd.0 98 release_map epoch: 98 2026-02-11T23:30:48.829+0000 7f416c3fd640 10 osd.0 98 maybe_share_map: con v2:127.0.0.1:6828/2699377081 updating session's projected_epoch from 95 to ping map epoch of 98 2026-02-11T23:30:48.829+0000 7f416c3fd640 10 osd.0 98 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 98 is not newer than session's projected_epoch of 98 2026-02-11T23:30:48.829+0000 7f416c3fd640 20 osd.0 98 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219567299s,-7.219338417s]) 2026-02-11T23:30:48.829+0000 7f416abfa640 20 osd.0 98 get_con_osd_cluster to osd.4 from_epoch 98 2026-02-11T23:30:48.829+0000 7f416abfa640 20 osd.0 98 get_nextmap_reserved map_reservations: {98=1} 2026-02-11T23:30:48.829+0000 7f416abfa640 20 osd.0 98 release_map epoch: 98 2026-02-11T23:30:48.829+0000 7f416abfa640 10 osd.0 98 maybe_share_map: con v2:127.0.0.1:6836/3331151400 updating session's projected_epoch from 92 to ping map epoch of 98 2026-02-11T23:30:48.829+0000 7f416abfa640 10 osd.0 98 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 98 is not newer than session's projected_epoch of 98 2026-02-11T23:30:48.829+0000 7f416abfa640 20 osd.0 98 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836751938s,-9.836490631s]) 2026-02-11T23:30:48.829+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6814/3043114013 9 ==== osd_ping(ping_reply e97 up_from 10 ping_stamp 2026-02-11T23:30:48.829379+0000/31.504785538s send_stamp 29.493247986s delta_ub -2.011535645s) ==== 2033+0+0 (crc 0 0 0) 0x4c9b200 con 0x4640c00 2026-02-11T23:30:48.829+0000 7f416c3fd640 20 osd.0 98 get_con_osd_cluster to osd.1 from_epoch 98 2026-02-11T23:30:48.829+0000 7f416c3fd640 20 osd.0 98 get_nextmap_reserved map_reservations: {98=1} 2026-02-11T23:30:48.829+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6832/2699377081 8 ==== osd_ping(ping_reply e98 up_from 19 ping_stamp 2026-02-11T23:30:48.829379+0000/31.504785538s send_stamp 24.285484314s delta_ub -7.219299793s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a600 con 0x48cd400 2026-02-11T23:30:48.829+0000 7f416c3fd640 20 osd.0 98 release_map epoch: 98 2026-02-11T23:30:48.829+0000 7f416c3fd640 10 osd.0 98 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 98 is not newer than session's projected_epoch of 98 2026-02-11T23:30:48.829+0000 7f416c3fd640 20 osd.0 98 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011813641s,-2.011535645s]) 2026-02-11T23:30:48.829+0000 7f416abfa640 20 osd.0 98 get_con_osd_cluster to osd.3 from_epoch 98 2026-02-11T23:30:48.829+0000 7f416abfa640 20 osd.0 98 get_nextmap_reserved map_reservations: {98=1} 2026-02-11T23:30:48.829+0000 7f416abfa640 20 osd.0 98 release_map epoch: 98 2026-02-11T23:30:48.829+0000 7f416abfa640 10 osd.0 98 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 98 is not newer than session's projected_epoch of 98 2026-02-11T23:30:48.829+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6838/3331151400 7 ==== osd_ping(ping_reply e98 up_from 23 ping_stamp 2026-02-11T23:30:48.829379+0000/31.504785538s send_stamp 21.668319702s delta_ub -9.836465836s) ==== 2033+0+0 (crc 0 0 0) 0x4c9b200 con 0x5282000 2026-02-11T23:30:48.829+0000 7f416abfa640 20 osd.0 98 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219602585s,-7.219299793s]) 2026-02-11T23:30:48.829+0000 7f416c3fd640 20 osd.0 98 get_con_osd_cluster to osd.4 from_epoch 98 2026-02-11T23:30:48.829+0000 7f416c3fd640 20 osd.0 98 get_nextmap_reserved map_reservations: {98=1} 2026-02-11T23:30:48.829+0000 7f416c3fd640 20 osd.0 98 release_map epoch: 98 2026-02-11T23:30:48.829+0000 7f416c3fd640 10 osd.0 98 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 98 is not newer than session's projected_epoch of 98 2026-02-11T23:30:48.829+0000 7f416c3fd640 20 osd.0 98 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836788177s,-9.836465836s]) 2026-02-11T23:30:49.328+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:30:49.328+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:30:49.328+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:19.328282+0000) 2026-02-11T23:30:49.363+0000 7f41633eb640 10 osd.0 98 tick 2026-02-11T23:30:49.363+0000 7f41633eb640 20 osd.0 98 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:30:49.396+0000 7f4162bea640 10 osd.0 98 tick_without_osd_lock 2026-02-11T23:30:49.396+0000 7f4162bea640 0 osd.0 98 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:49.396+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:49.396+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:49.396+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:49.396+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.399 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:49.396+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:49.396, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:49.396+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:49.396+0000 7f4162bea640 10 osd.0 98 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:49.396+0000 7f4162bea640 20 osd.0 98 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:49.396+0000 7f4162bea640 10 osd.0 98 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:49.396+0000 7f4162bea640 20 osd.0 98 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:49.841+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.1 127.0.0.1:0/3043114013 13 ==== osd_ping(ping e98 up_from 10 ping_stamp 2026-02-11T23:30:49.841370+0000/30.505115509s send_stamp 30.505115509s delta_ub -2.011535645s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a300 con 0x4aca000 2026-02-11T23:30:49.841+0000 7f416cbfe640 20 osd.0 98 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011742830s,-2.011535645s]) 2026-02-11T23:30:49.841+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e98 up_from 5 ping_stamp 2026-02-11T23:30:49.841370+0000/30.505115509s send_stamp 32.516857147s delta_ub 2.011742830s) -- 0x5af8a80 con 0x4aca000 2026-02-11T23:30:49.841+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.1 127.0.0.1:0/3043114013 13 ==== osd_ping(ping e98 up_from 10 ping_stamp 2026-02-11T23:30:49.841370+0000/30.505115509s send_stamp 30.505115509s delta_ub -2.011535645s) ==== 2033+0+0 (crc 0 0 0) 0x4c9b200 con 0x4aca400 2026-02-11T23:30:49.841+0000 7f416cbfe640 20 osd.0 98 get_con_osd_cluster to osd.1 from_epoch 98 2026-02-11T23:30:49.841+0000 7f416cbfe640 20 osd.0 98 get_nextmap_reserved map_reservations: {98=1} 2026-02-11T23:30:49.841+0000 7f416cbfe640 20 osd.0 98 release_map epoch: 98 2026-02-11T23:30:49.841+0000 7f416cbfe640 10 osd.0 98 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 98 is not newer than session's projected_epoch of 98 2026-02-11T23:30:49.841+0000 7f416c3fd640 20 osd.0 98 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011761904s,-2.011535645s]) 2026-02-11T23:30:49.841+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e98 up_from 5 ping_stamp 2026-02-11T23:30:49.841370+0000/30.505115509s send_stamp 32.516880035s delta_ub 2.011761904s) -- 0x4c9b980 con 0x4aca400 2026-02-11T23:30:49.841+0000 7f416c3fd640 20 osd.0 98 get_con_osd_cluster to osd.1 from_epoch 98 2026-02-11T23:30:49.841+0000 7f416c3fd640 20 osd.0 98 get_nextmap_reserved map_reservations: {98=1} 2026-02-11T23:30:49.841+0000 7f416c3fd640 20 osd.0 98 release_map epoch: 98 2026-02-11T23:30:49.841+0000 7f416c3fd640 10 osd.0 98 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 98 is not newer than session's projected_epoch of 98 2026-02-11T23:30:49.964+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.4 127.0.0.1:0/3331151400 6 ==== osd_ping(ping e98 up_from 23 ping_stamp 2026-02-11T23:30:49.965044+0000/22.803812027s send_stamp 22.803812027s delta_ub -9.836465836s) ==== 2033+0+0 (crc 0 0 0) 0x4c9b980 con 0x507a800 2026-02-11T23:30:49.964+0000 7f416c3fd640 20 osd.0 98 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836692810s,-9.836465836s]) 2026-02-11T23:30:49.964+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e98 up_from 5 ping_stamp 2026-02-11T23:30:49.965044+0000/22.803812027s send_stamp 32.640502930s delta_ub 9.836692810s) -- 0x4c9b200 con 0x507a800 2026-02-11T23:30:49.964+0000 7f416c3fd640 20 osd.0 98 get_con_osd_cluster to osd.4 from_epoch 98 2026-02-11T23:30:49.964+0000 7f416c3fd640 20 osd.0 98 get_nextmap_reserved map_reservations: {98=1} 2026-02-11T23:30:49.964+0000 7f416c3fd640 20 osd.0 98 release_map epoch: 98 2026-02-11T23:30:49.964+0000 7f416c3fd640 10 osd.0 98 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 98 is not newer than session's projected_epoch of 98 2026-02-11T23:30:49.964+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.4 127.0.0.1:0/3331151400 6 ==== osd_ping(ping e98 up_from 23 ping_stamp 2026-02-11T23:30:49.965044+0000/22.803812027s send_stamp 22.803812027s delta_ub -9.836465836s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a600 con 0x507a400 2026-02-11T23:30:49.964+0000 7f416abfa640 20 osd.0 98 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836714745s,-9.836465836s]) 2026-02-11T23:30:49.964+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e98 up_from 5 ping_stamp 2026-02-11T23:30:49.965044+0000/22.803812027s send_stamp 32.640525818s delta_ub 9.836714745s) -- 0x4c9b500 con 0x507a400 2026-02-11T23:30:49.964+0000 7f416abfa640 20 osd.0 98 get_con_osd_cluster to osd.4 from_epoch 98 2026-02-11T23:30:49.964+0000 7f416abfa640 20 osd.0 98 get_nextmap_reserved map_reservations: {98=1} 2026-02-11T23:30:49.964+0000 7f416abfa640 20 osd.0 98 release_map epoch: 98 2026-02-11T23:30:49.964+0000 7f416abfa640 10 osd.0 98 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 98 is not newer than session's projected_epoch of 98 2026-02-11T23:30:49.995+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.2 127.0.0.1:0/2365263801 7 ==== osd_ping(ping e98 up_from 14 ping_stamp 2026-02-11T23:30:49.995244+0000/28.103841782s send_stamp 28.103841782s delta_ub -4.566680908s) ==== 2033+0+0 (crc 0 0 0) 0x5af8a80 con 0x4641000 2026-02-11T23:30:49.995+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.2 127.0.0.1:0/2365263801 7 ==== osd_ping(ping e98 up_from 14 ping_stamp 2026-02-11T23:30:49.995244+0000/28.103841782s send_stamp 28.103841782s delta_ub -4.566680908s) ==== 2033+0+0 (crc 0 0 0) 0x4c9b200 con 0x48cc000 2026-02-11T23:30:49.995+0000 7f416cbfe640 20 osd.0 98 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566922665s,-4.566680908s]) 2026-02-11T23:30:49.995+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e98 up_from 5 ping_stamp 2026-02-11T23:30:49.995244+0000/28.103841782s send_stamp 32.670761108s delta_ub 4.566922665s) -- 0x4c9a300 con 0x4641000 2026-02-11T23:30:49.995+0000 7f416cbfe640 20 osd.0 98 get_con_osd_cluster to osd.2 from_epoch 98 2026-02-11T23:30:49.995+0000 7f416cbfe640 20 osd.0 98 get_nextmap_reserved map_reservations: {98=1} 2026-02-11T23:30:49.995+0000 7f416cbfe640 20 osd.0 98 release_map epoch: 98 2026-02-11T23:30:49.995+0000 7f416cbfe640 10 osd.0 98 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 98 is not newer than session's projected_epoch of 98 2026-02-11T23:30:49.995+0000 7f416c3fd640 20 osd.0 98 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.567008018s,-4.566680908s]) 2026-02-11T23:30:49.995+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e98 up_from 5 ping_stamp 2026-02-11T23:30:49.995244+0000/28.103841782s send_stamp 32.670848846s delta_ub 4.567008018s) -- 0x4c9b980 con 0x48cc000 2026-02-11T23:30:49.995+0000 7f416c3fd640 20 osd.0 98 get_con_osd_cluster to osd.2 from_epoch 98 2026-02-11T23:30:49.995+0000 7f416c3fd640 20 osd.0 98 get_nextmap_reserved map_reservations: {98=1} 2026-02-11T23:30:49.995+0000 7f416c3fd640 20 osd.0 98 release_map epoch: 98 2026-02-11T23:30:49.995+0000 7f416c3fd640 10 osd.0 98 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 98 is not newer than session's projected_epoch of 98 2026-02-11T23:30:50.328+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:30:50.328+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:30:50.328+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:20.328322+0000) 2026-02-11T23:30:50.328+0000 7f415bf00640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-11T23:30:50.375+0000 7f41633eb640 10 osd.0 98 tick 2026-02-11T23:30:50.375+0000 7f41633eb640 20 osd.0 98 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:30:50.425+0000 7f4162bea640 10 osd.0 98 tick_without_osd_lock 2026-02-11T23:30:50.425+0000 7f4162bea640 0 osd.0 98 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:50.425+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:50.425+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:50.425+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:50.425+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.399 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:50.425+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:50.426, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:50.425+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:50.425+0000 7f4162bea640 10 osd.0 98 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:50.425+0000 7f4162bea640 20 osd.0 98 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:50.425+0000 7f4162bea640 10 osd.0 98 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:50.425+0000 7f4162bea640 20 osd.0 98 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:50.846+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.3 127.0.0.1:0/2699377081 7 ==== osd_ping(ping e98 up_from 19 ping_stamp 2026-02-11T23:30:50.848120+0000/26.304059982s send_stamp 26.304059982s delta_ub -7.219299793s) ==== 2033+0+0 (crc 0 0 0) 0x4c9b980 con 0x48cc400 2026-02-11T23:30:50.846+0000 7f416c3fd640 20 osd.0 98 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219522476s,-7.219299793s]) 2026-02-11T23:30:50.846+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e98 up_from 5 ping_stamp 2026-02-11T23:30:50.848120+0000/26.304059982s send_stamp 33.523582458s delta_ub 7.219522476s) -- 0x4c9b200 con 0x48cc400 2026-02-11T23:30:50.846+0000 7f416c3fd640 20 osd.0 98 get_con_osd_cluster to osd.3 from_epoch 98 2026-02-11T23:30:50.846+0000 7f416c3fd640 20 osd.0 98 get_nextmap_reserved map_reservations: {98=1} 2026-02-11T23:30:50.846+0000 7f416c3fd640 20 osd.0 98 release_map epoch: 98 2026-02-11T23:30:50.846+0000 7f416c3fd640 10 osd.0 98 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 98 is not newer than session's projected_epoch of 98 2026-02-11T23:30:50.846+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.3 127.0.0.1:0/2699377081 7 ==== osd_ping(ping e98 up_from 19 ping_stamp 2026-02-11T23:30:50.848120+0000/26.304059982s send_stamp 26.304059982s delta_ub -7.219299793s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a300 con 0x48cc800 2026-02-11T23:30:50.846+0000 7f416cbfe640 20 osd.0 98 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219550133s,-7.219299793s]) 2026-02-11T23:30:50.846+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e98 up_from 5 ping_stamp 2026-02-11T23:30:50.848120+0000/26.304059982s send_stamp 33.523609161s delta_ub 7.219550133s) -- 0x5af8a80 con 0x48cc800 2026-02-11T23:30:50.846+0000 7f416cbfe640 20 osd.0 98 get_con_osd_cluster to osd.3 from_epoch 98 2026-02-11T23:30:50.846+0000 7f416cbfe640 20 osd.0 98 get_nextmap_reserved map_reservations: {98=1} 2026-02-11T23:30:50.846+0000 7f416cbfe640 20 osd.0 98 release_map epoch: 98 2026-02-11T23:30:50.846+0000 7f416cbfe640 10 osd.0 98 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 98 is not newer than session's projected_epoch of 98 2026-02-11T23:30:51.327+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:30:51.327+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:30:51.327+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:21.328388+0000) 2026-02-11T23:30:51.362+0000 7f41633eb640 10 osd.0 98 tick 2026-02-11T23:30:51.362+0000 7f41633eb640 20 osd.0 98 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:30:51.400+0000 7f4162bea640 10 osd.0 98 tick_without_osd_lock 2026-02-11T23:30:51.400+0000 7f4162bea640 0 osd.0 98 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:51.400+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:51.400+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:51.400+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:51.400+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.399 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:51.400+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:51.401, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:51.400+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:51.400+0000 7f4162bea640 10 osd.0 98 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:51.400+0000 7f4162bea640 20 osd.0 98 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:51.400+0000 7f4162bea640 10 osd.0 98 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:51.400+0000 7f4162bea640 20 osd.0 98 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:51.988+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 24 ==== pg_lease(4.0 pg_lease(ru 32.828323364s ub 40.828445435s int 16.000000000s) e98/98) ==== 58+0+0 (unknown 0 0 0) 0x4c9b500 con 0x507ac00 2026-02-11T23:30:51.988+0000 7f416abfa640 15 osd.0 98 enqueue_peering_evt 4.0 epoch_sent: 98 epoch_requested: 98 MLease epoch 98 from osd.4 pg_lease(ru 32.828323364s ub 40.828445435s int 16.000000000s) 2026-02-11T23:30:51.988+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 98 epoch_requested: 98 MLease epoch 98 from osd.4 pg_lease(ru 32.828323364s ub 40.828445435s int 16.000000000s)) class_id 2 prio 255 cost 10 e98) 2026-02-11T23:30:51.988+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:51.988+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 98 epoch_requested: 98 MLease epoch 98 from osd.4 pg_lease(ru 32.828323364s ub 40.828445435s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e98) queued 2026-02-11T23:30:51.988+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:51.988+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 98 epoch_requested: 98 MLease epoch 98 from osd.4 pg_lease(ru 32.828323364s ub 40.828445435s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e98) pg 0x5978000 2026-02-11T23:30:51.988+0000 7f414b98b640 10 osd.0 pg_epoch: 98 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 98 epoch_requested: 98 MLease epoch 98 from osd.4 pg_lease(ru 32.828323364s ub 40.828445435s int 16.000000000s) 2026-02-11T23:30:51.988+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:51.988+0000 7f414b98b640 10 osd.0 pg_epoch: 98 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::proc_lease pg_lease(ru 32.828323364s ub 40.828445435s int 16.000000000s) 2026-02-11T23:30:51.988+0000 7f414b98b640 20 osd.0 pg_epoch: 98 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::proc_lease peer_clock_delta_ub -9.836465836s -> ru 42.664787292s 2026-02-11T23:30:51.988+0000 7f414b98b640 20 osd.0 pg_epoch: 98 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::proc_lease readable_until now 42.664787292s 2026-02-11T23:30:51.988+0000 7f414b98b640 20 osd.0 pg_epoch: 98 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::proc_lease peer_clock_delta_lb -9.836714745s -> ruub 50.665157318s 2026-02-11T23:30:51.988+0000 7f414b98b640 20 osd.0 pg_epoch: 98 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::proc_lease readable_until_ub now 50.665157318s 2026-02-11T23:30:51.988+0000 7f414b98b640 20 osd.0 98 get_con_osd_cluster to osd.4 from_epoch 98 2026-02-11T23:30:51.988+0000 7f414b98b640 20 osd.0 98 get_nextmap_reserved map_reservations: {98=1} 2026-02-11T23:30:51.988+0000 7f414b98b640 20 osd.0 98 release_map epoch: 98 2026-02-11T23:30:51.988+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_lease_ack(4.0 pg_lease_ack(ruub 40.828445435s) e98/98) -- 0x3759760 con 0x507ac00 2026-02-11T23:30:51.988+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:51.988+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:52.327+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:30:52.327+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:30:52.327+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:22.328448+0000) 2026-02-11T23:30:52.328+0000 7f4147983640 5 osd.0 98 heartbeat osd_stat(store_statfs(0x384000/0x0/0x384000, data 0x6f4d9/0xf8000, compress 0x0/0x0/0x0, omap 0x94c1, meta 0x1a46b3f), peers [1,2,3,4] op hist []) 2026-02-11T23:30:52.328+0000 7f4147983640 20 osd.0 98 check_full_status cur ratio 0, physical ratio 0, new state none 2026-02-11T23:30:52.328+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6816/3043114013,v1:127.0.0.1:6817/3043114013] -- osd_ping(ping e98 up_from 5 ping_stamp 2026-02-11T23:30:52.329552+0000/35.004955292s send_stamp 35.004955292s delta_ub 2.011761904s) -- 0x5af8600 con 0x48c4400 2026-02-11T23:30:52.328+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6814/3043114013,v1:127.0.0.1:6815/3043114013] -- osd_ping(ping e98 up_from 5 ping_stamp 2026-02-11T23:30:52.329552+0000/35.004955292s send_stamp 35.004955292s delta_ub 2.011761904s) -- 0x5af8900 con 0x4640c00 2026-02-11T23:30:52.328+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6824/2365263801,v1:127.0.0.1:6825/2365263801] -- osd_ping(ping e98 up_from 5 ping_stamp 2026-02-11T23:30:52.329552+0000/35.004955292s send_stamp 35.004955292s delta_ub 4.567008018s) -- 0x5af8000 con 0x48c4000 2026-02-11T23:30:52.328+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6822/2365263801,v1:127.0.0.1:6823/2365263801] -- osd_ping(ping e98 up_from 5 ping_stamp 2026-02-11T23:30:52.329552+0000/35.004955292s send_stamp 35.004955292s delta_ub 4.567008018s) -- 0x5af8f00 con 0x48cd000 2026-02-11T23:30:52.328+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6832/2699377081,v1:127.0.0.1:6833/2699377081] -- osd_ping(ping e98 up_from 5 ping_stamp 2026-02-11T23:30:52.329552+0000/35.004955292s send_stamp 35.004955292s delta_ub 7.219550133s) -- 0x4c9a780 con 0x48cd400 2026-02-11T23:30:52.328+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6830/2699377081,v1:127.0.0.1:6831/2699377081] -- osd_ping(ping e98 up_from 5 ping_stamp 2026-02-11T23:30:52.329552+0000/35.004955292s send_stamp 35.004955292s delta_ub 7.219550133s) -- 0x4c9a900 con 0x48cd800 2026-02-11T23:30:52.328+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6840/3331151400,v1:127.0.0.1:6841/3331151400] -- osd_ping(ping e98 up_from 5 ping_stamp 2026-02-11T23:30:52.329552+0000/35.004955292s send_stamp 35.004955292s delta_ub 9.836714745s) -- 0x4c9a480 con 0x507bc00 2026-02-11T23:30:52.328+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6838/3331151400,v1:127.0.0.1:6839/3331151400] -- osd_ping(ping e98 up_from 5 ping_stamp 2026-02-11T23:30:52.329552+0000/35.004955292s send_stamp 35.004955292s delta_ub 9.836714745s) -- 0x4c9b380 con 0x5282000 2026-02-11T23:30:52.328+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6816/3043114013 10 ==== osd_ping(ping_reply e98 up_from 10 ping_stamp 2026-02-11T23:30:52.329552+0000/35.004955292s send_stamp 32.993358612s delta_ub -2.011597633s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a780 con 0x48c4400 2026-02-11T23:30:52.328+0000 7f416abfa640 20 osd.0 98 get_con_osd_cluster to osd.1 from_epoch 98 2026-02-11T23:30:52.328+0000 7f416abfa640 20 osd.0 98 get_nextmap_reserved map_reservations: {98=1} 2026-02-11T23:30:52.328+0000 7f416abfa640 20 osd.0 98 release_map epoch: 98 2026-02-11T23:30:52.328+0000 7f416abfa640 10 osd.0 98 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 98 is not newer than session's projected_epoch of 98 2026-02-11T23:30:52.328+0000 7f416abfa640 20 osd.0 98 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011719465s,-2.011597633s]) 2026-02-11T23:30:52.328+0000 7f416cbfe640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6824/2365263801 9 ==== osd_ping(ping_reply e98 up_from 14 ping_stamp 2026-02-11T23:30:52.329552+0000/35.004955292s send_stamp 30.438236237s delta_ub -4.566720486s) ==== 2033+0+0 (crc 0 0 0) 0x4c9b680 con 0x48c4000 2026-02-11T23:30:52.328+0000 7f416cbfe640 20 osd.0 98 get_con_osd_cluster to osd.2 from_epoch 98 2026-02-11T23:30:52.328+0000 7f416cbfe640 20 osd.0 98 get_nextmap_reserved map_reservations: {98=1} 2026-02-11T23:30:52.328+0000 7f416cbfe640 20 osd.0 98 release_map epoch: 98 2026-02-11T23:30:52.328+0000 7f416cbfe640 10 osd.0 98 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 98 is not newer than session's projected_epoch of 98 2026-02-11T23:30:52.328+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6814/3043114013 10 ==== osd_ping(ping_reply e98 up_from 10 ping_stamp 2026-02-11T23:30:52.329552+0000/35.004955292s send_stamp 32.993381500s delta_ub -2.011577845s) ==== 2033+0+0 (crc 0 0 0) 0x4c9b380 con 0x4640c00 2026-02-11T23:30:52.328+0000 7f416cbfe640 20 osd.0 98 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566860676s,-4.566720486s]) 2026-02-11T23:30:52.328+0000 7f416c3fd640 20 osd.0 98 get_con_osd_cluster to osd.1 from_epoch 98 2026-02-11T23:30:52.328+0000 7f416c3fd640 20 osd.0 98 get_nextmap_reserved map_reservations: {98=1} 2026-02-11T23:30:52.328+0000 7f416c3fd640 20 osd.0 98 release_map epoch: 98 2026-02-11T23:30:52.328+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6832/2699377081 9 ==== osd_ping(ping_reply e98 up_from 19 ping_stamp 2026-02-11T23:30:52.329552+0000/35.004955292s send_stamp 27.785596848s delta_ub -7.219361305s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a480 con 0x48cd400 2026-02-11T23:30:52.328+0000 7f416c3fd640 10 osd.0 98 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 98 is not newer than session's projected_epoch of 98 2026-02-11T23:30:52.328+0000 7f416c3fd640 20 osd.0 98 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011728287s,-2.011577845s]) 2026-02-11T23:30:52.328+0000 7f416abfa640 20 osd.0 98 get_con_osd_cluster to osd.3 from_epoch 98 2026-02-11T23:30:52.328+0000 7f416abfa640 20 osd.0 98 get_nextmap_reserved map_reservations: {98=1} 2026-02-11T23:30:52.328+0000 7f416abfa640 20 osd.0 98 release_map epoch: 98 2026-02-11T23:30:52.328+0000 7f416abfa640 10 osd.0 98 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 98 is not newer than session's projected_epoch of 98 2026-02-11T23:30:52.328+0000 7f416abfa640 20 osd.0 98 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219523907s,-7.219361305s]) 2026-02-11T23:30:52.328+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6830/2699377081 9 ==== osd_ping(ping_reply e98 up_from 19 ping_stamp 2026-02-11T23:30:52.329552+0000/35.004955292s send_stamp 27.785625458s delta_ub -7.219332218s) ==== 2033+0+0 (crc 0 0 0) 0x4c9b380 con 0x48cd800 2026-02-11T23:30:52.328+0000 7f416c3fd640 20 osd.0 98 get_con_osd_cluster to osd.3 from_epoch 98 2026-02-11T23:30:52.328+0000 7f416c3fd640 20 osd.0 98 get_nextmap_reserved map_reservations: {98=1} 2026-02-11T23:30:52.328+0000 7f416c3fd640 20 osd.0 98 release_map epoch: 98 2026-02-11T23:30:52.328+0000 7f416c3fd640 10 osd.0 98 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 98 is not newer than session's projected_epoch of 98 2026-02-11T23:30:52.328+0000 7f416c3fd640 20 osd.0 98 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219517231s,-7.219332218s]) 2026-02-11T23:30:52.328+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6822/2365263801 9 ==== osd_ping(ping_reply e98 up_from 14 ping_stamp 2026-02-11T23:30:52.329552+0000/35.004955292s send_stamp 30.438253403s delta_ub -4.566704750s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a480 con 0x48cd000 2026-02-11T23:30:52.328+0000 7f416abfa640 20 osd.0 98 get_con_osd_cluster to osd.2 from_epoch 98 2026-02-11T23:30:52.328+0000 7f416abfa640 20 osd.0 98 get_nextmap_reserved map_reservations: {98=1} 2026-02-11T23:30:52.328+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6838/3331151400 8 ==== osd_ping(ping_reply e98 up_from 23 ping_stamp 2026-02-11T23:30:52.329552+0000/35.004955292s send_stamp 25.168472290s delta_ub -9.836485863s) ==== 2033+0+0 (crc 0 0 0) 0x4c9b380 con 0x5282000 2026-02-11T23:30:52.328+0000 7f416abfa640 20 osd.0 98 release_map epoch: 98 2026-02-11T23:30:52.328+0000 7f416abfa640 10 osd.0 98 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 98 is not newer than session's projected_epoch of 98 2026-02-11T23:30:52.328+0000 7f416abfa640 20 osd.0 98 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566898823s,-4.566704750s]) 2026-02-11T23:30:52.328+0000 7f416c3fd640 20 osd.0 98 get_con_osd_cluster to osd.4 from_epoch 98 2026-02-11T23:30:52.328+0000 7f416c3fd640 20 osd.0 98 get_nextmap_reserved map_reservations: {98=1} 2026-02-11T23:30:52.328+0000 7f416c3fd640 20 osd.0 98 release_map epoch: 98 2026-02-11T23:30:52.328+0000 7f416c3fd640 10 osd.0 98 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 98 is not newer than session's projected_epoch of 98 2026-02-11T23:30:52.328+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6840/3331151400 8 ==== osd_ping(ping_reply e98 up_from 23 ping_stamp 2026-02-11T23:30:52.329552+0000/35.004955292s send_stamp 25.168485641s delta_ub -9.836471558s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a480 con 0x507bc00 2026-02-11T23:30:52.328+0000 7f416c3fd640 20 osd.0 98 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836701393s,-9.836485863s]) 2026-02-11T23:30:52.328+0000 7f416abfa640 20 osd.0 98 get_con_osd_cluster to osd.4 from_epoch 98 2026-02-11T23:30:52.328+0000 7f416abfa640 20 osd.0 98 get_nextmap_reserved map_reservations: {98=1} 2026-02-11T23:30:52.328+0000 7f416abfa640 20 osd.0 98 release_map epoch: 98 2026-02-11T23:30:52.328+0000 7f416abfa640 10 osd.0 98 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 98 is not newer than session's projected_epoch of 98 2026-02-11T23:30:52.328+0000 7f416abfa640 20 osd.0 98 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836700439s,-9.836471558s]) 2026-02-11T23:30:52.363+0000 7f41633eb640 10 osd.0 98 tick 2026-02-11T23:30:52.363+0000 7f41633eb640 20 osd.0 98 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:30:52.377+0000 7f4162bea640 10 osd.0 98 tick_without_osd_lock 2026-02-11T23:30:52.377+0000 7f4162bea640 0 osd.0 98 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:52.377+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:52.377+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.399 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:52.377+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:52.378, recovery is active?:false restrictions:<....> 2026-02-11T23:30:52.377+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:52.377+0000 7f4162bea640 10 osd.0 98 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:52.377+0000 7f4162bea640 20 osd.0 98 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:52.377+0000 7f4162bea640 10 osd.0 98 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:52.377+0000 7f4162bea640 20 osd.0 98 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:52.485+0000 7f41683f5640 10 monclient: _renew_subs 2026-02-11T23:30:52.485+0000 7f41683f5640 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:30:52.485+0000 7f41683f5640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> v2:127.0.0.1:7180/0 -- mon_subscribe({osdmap=99}) -- 0x38149c0 con 0x4acbc00 2026-02-11T23:30:52.547+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.3 127.0.0.1:0/2699377081 8 ==== osd_ping(ping e98 up_from 19 ping_stamp 2026-02-11T23:30:52.548320+0000/28.004259109s send_stamp 28.004259109s delta_ub -7.219332218s) ==== 2033+0+0 (crc 0 0 0) 0x4c9b380 con 0x48cc400 2026-02-11T23:30:52.547+0000 7f416c3fd640 20 osd.0 98 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219511986s,-7.219332218s]) 2026-02-11T23:30:52.547+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e98 up_from 5 ping_stamp 2026-02-11T23:30:52.548320+0000/28.004259109s send_stamp 35.223773956s delta_ub 7.219511986s) -- 0x4c9a900 con 0x48cc400 2026-02-11T23:30:52.547+0000 7f416c3fd640 20 osd.0 98 get_con_osd_cluster to osd.3 from_epoch 98 2026-02-11T23:30:52.547+0000 7f416c3fd640 20 osd.0 98 get_nextmap_reserved map_reservations: {98=1} 2026-02-11T23:30:52.547+0000 7f416c3fd640 20 osd.0 98 release_map epoch: 98 2026-02-11T23:30:52.547+0000 7f416c3fd640 10 osd.0 98 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 98 is not newer than session's projected_epoch of 98 2026-02-11T23:30:52.547+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.3 127.0.0.1:0/2699377081 8 ==== osd_ping(ping e98 up_from 19 ping_stamp 2026-02-11T23:30:52.548320+0000/28.004259109s send_stamp 28.004259109s delta_ub -7.219332218s) ==== 2033+0+0 (crc 0 0 0) 0x4c9b680 con 0x48cc800 2026-02-11T23:30:52.547+0000 7f416cbfe640 20 osd.0 98 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219548225s,-7.219332218s]) 2026-02-11T23:30:52.547+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e98 up_from 5 ping_stamp 2026-02-11T23:30:52.548320+0000/28.004259109s send_stamp 35.223804474s delta_ub 7.219548225s) -- 0x4c9ad80 con 0x48cc800 2026-02-11T23:30:52.547+0000 7f416cbfe640 20 osd.0 98 get_con_osd_cluster to osd.3 from_epoch 98 2026-02-11T23:30:52.547+0000 7f416cbfe640 20 osd.0 98 get_nextmap_reserved map_reservations: {98=1} 2026-02-11T23:30:52.547+0000 7f416cbfe640 20 osd.0 98 release_map epoch: 98 2026-02-11T23:30:52.547+0000 7f416cbfe640 10 osd.0 98 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 98 is not newer than session's projected_epoch of 98 2026-02-11T23:30:52.864+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.4 127.0.0.1:0/3331151400 7 ==== osd_ping(ping e98 up_from 23 ping_stamp 2026-02-11T23:30:52.865247+0000/25.704013824s send_stamp 25.704013824s delta_ub -9.836471558s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a900 con 0x507a800 2026-02-11T23:30:52.864+0000 7f416c3fd640 20 osd.0 98 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836722374s,-9.836471558s]) 2026-02-11T23:30:52.864+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e98 up_from 5 ping_stamp 2026-02-11T23:30:52.865247+0000/25.704013824s send_stamp 35.540737152s delta_ub 9.836722374s) -- 0x4c9b380 con 0x507a800 2026-02-11T23:30:52.864+0000 7f416c3fd640 20 osd.0 98 get_con_osd_cluster to osd.4 from_epoch 98 2026-02-11T23:30:52.864+0000 7f416c3fd640 20 osd.0 98 get_nextmap_reserved map_reservations: {98=1} 2026-02-11T23:30:52.864+0000 7f416c3fd640 20 osd.0 98 release_map epoch: 98 2026-02-11T23:30:52.864+0000 7f416c3fd640 10 osd.0 98 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 98 is not newer than session's projected_epoch of 98 2026-02-11T23:30:52.864+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.4 127.0.0.1:0/3331151400 7 ==== osd_ping(ping e98 up_from 23 ping_stamp 2026-02-11T23:30:52.865247+0000/25.704013824s send_stamp 25.704013824s delta_ub -9.836471558s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a480 con 0x507a400 2026-02-11T23:30:52.864+0000 7f416abfa640 20 osd.0 98 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836741447s,-9.836471558s]) 2026-02-11T23:30:52.864+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e98 up_from 5 ping_stamp 2026-02-11T23:30:52.865247+0000/25.704013824s send_stamp 35.540760040s delta_ub 9.836741447s) -- 0x4c9a780 con 0x507a400 2026-02-11T23:30:52.864+0000 7f416abfa640 20 osd.0 98 get_con_osd_cluster to osd.4 from_epoch 98 2026-02-11T23:30:52.864+0000 7f416abfa640 20 osd.0 98 get_nextmap_reserved map_reservations: {98=1} 2026-02-11T23:30:52.864+0000 7f416abfa640 20 osd.0 98 release_map epoch: 98 2026-02-11T23:30:52.864+0000 7f416abfa640 10 osd.0 98 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 98 is not newer than session's projected_epoch of 98 2026-02-11T23:30:53.047+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.3 127.0.0.1:0/2699377081 9 ==== osd_ping(ping e98 up_from 19 ping_stamp 2026-02-11T23:30:53.048505+0000/28.504444122s send_stamp 28.504444122s delta_ub -7.219398499s) ==== 2033+0+0 (crc 0 0 0) 0x4c9b380 con 0x48cc400 2026-02-11T23:30:53.047+0000 7f416c3fd640 20 osd.0 98 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219533920s,-7.219398499s]) 2026-02-11T23:30:53.047+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e98 up_from 5 ping_stamp 2026-02-11T23:30:53.048505+0000/28.504444122s send_stamp 35.723979950s delta_ub 7.219533920s) -- 0x4c9a900 con 0x48cc400 2026-02-11T23:30:53.047+0000 7f416c3fd640 20 osd.0 98 get_con_osd_cluster to osd.3 from_epoch 98 2026-02-11T23:30:53.047+0000 7f416c3fd640 20 osd.0 98 get_nextmap_reserved map_reservations: {98=1} 2026-02-11T23:30:53.047+0000 7f416c3fd640 20 osd.0 98 release_map epoch: 98 2026-02-11T23:30:53.047+0000 7f416c3fd640 10 osd.0 98 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 98 is not newer than session's projected_epoch of 98 2026-02-11T23:30:53.047+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.3 127.0.0.1:0/2699377081 9 ==== osd_ping(ping e98 up_from 19 ping_stamp 2026-02-11T23:30:53.048505+0000/28.504444122s send_stamp 28.504444122s delta_ub -7.219398499s) ==== 2033+0+0 (crc 0 0 0) 0x4c9ad80 con 0x48cc800 2026-02-11T23:30:53.047+0000 7f416cbfe640 20 osd.0 98 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219551563s,-7.219398499s]) 2026-02-11T23:30:53.047+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e98 up_from 5 ping_stamp 2026-02-11T23:30:53.048505+0000/28.504444122s send_stamp 35.723999023s delta_ub 7.219551563s) -- 0x4c9b680 con 0x48cc800 2026-02-11T23:30:53.047+0000 7f416cbfe640 20 osd.0 98 get_con_osd_cluster to osd.3 from_epoch 98 2026-02-11T23:30:53.047+0000 7f416cbfe640 20 osd.0 98 get_nextmap_reserved map_reservations: {98=1} 2026-02-11T23:30:53.047+0000 7f416cbfe640 20 osd.0 98 release_map epoch: 98 2026-02-11T23:30:53.047+0000 7f416cbfe640 10 osd.0 98 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 98 is not newer than session's projected_epoch of 98 2026-02-11T23:30:53.087+0000 7f416c3fd640 1 --2- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] >> conn(0x507b400 0x527f080 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).accept 2026-02-11T23:30:53.087+0000 7f416cbfe640 1 --2- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] >> conn(0x507b400 0x527f080 unknown :-1 s=BANNER_ACCEPTING pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:53.087+0000 7f416cbfe640 10 osd.0 98 ms_handle_fast_authentication new session 0x4a316c0 con 0x507b400 entity client.admin addr 2026-02-11T23:30:53.087+0000 7f416cbfe640 1 --2- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] >> 127.0.0.1:0/863698824 conn(0x507b400 0x527f080 crc :-1 s=READY pgs=3 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=client.4237 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-11T23:30:53.087+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== client.4237 127.0.0.1:0/863698824 1 ==== command(tid 1: {"prefix": "get_command_descriptions"}) ==== 62+0+0 (crc 0 0 0) 0x3759600 con 0x507b400 2026-02-11T23:30:53.087+0000 7f415cf02640 20 osd.0 98 OSD::ms_dispatch: command(tid 1: {"prefix": "get_command_descriptions"}) 2026-02-11T23:30:53.087+0000 7f415cf02640 20 osd.0 98 _dispatch 0x3759600 command(tid 1: {"prefix": "get_command_descriptions"}) 2026-02-11T23:30:53.087+0000 7f416bbfc640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> 127.0.0.1:0/863698824 -- command_reply(tid 1: 0 ) -- 0x37002c0 con 0x507b400 2026-02-11T23:30:53.095+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== client.4237 127.0.0.1:0/863698824 2 ==== command(tid 2: {"prefix": "flush_pg_stats"}) ==== 52+0+0 (crc 0 0 0) 0x37002c0 con 0x507b400 2026-02-11T23:30:53.095+0000 7f415cf02640 20 osd.0 98 OSD::ms_dispatch: command(tid 2: {"prefix": "flush_pg_stats"}) 2026-02-11T23:30:53.095+0000 7f415cf02640 20 osd.0 98 _dispatch 0x37002c0 command(tid 2: {"prefix": "flush_pg_stats"}) 2026-02-11T23:30:53.095+0000 7f416bbfc640 15 osd.0 98 collect_pg_stats 2026-02-11T23:30:53.095+0000 7f416bbfc640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] -- pg_stats(0 pgs seq 21474836488 v 0) -- 0x5b80000 con 0x4aca800 2026-02-11T23:30:53.095+0000 7f416bbfc640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> 127.0.0.1:0/863698824 -- command_reply(tid 2: 0 ) -- 0x3759600 con 0x507b400 2026-02-11T23:30:53.101+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== client.4237 127.0.0.1:0/863698824 3 ==== command(tid 3: {"prefix": "flush_pg_stats"}) ==== 52+0+0 (crc 0 0 0) 0x3759600 con 0x507b400 2026-02-11T23:30:53.101+0000 7f415cf02640 20 osd.0 98 OSD::ms_dispatch: command(tid 3: {"prefix": "flush_pg_stats"}) 2026-02-11T23:30:53.101+0000 7f415cf02640 20 osd.0 98 _dispatch 0x3759600 command(tid 3: {"prefix": "flush_pg_stats"}) 2026-02-11T23:30:53.101+0000 7f416bbfc640 15 osd.0 98 collect_pg_stats 2026-02-11T23:30:53.101+0000 7f416bbfc640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] -- pg_stats(0 pgs seq 21474836489 v 0) -- 0x5b80300 con 0x4aca800 2026-02-11T23:30:53.101+0000 7f416bbfc640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> 127.0.0.1:0/863698824 -- command_reply(tid 3: 0 ) -- 0x37002c0 con 0x507b400 2026-02-11T23:30:53.101+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] >> 127.0.0.1:0/863698824 conn(0x507b400 msgr2=0x527f080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 80 2026-02-11T23:30:53.101+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] >> 127.0.0.1:0/863698824 conn(0x507b400 msgr2=0x527f080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-11T23:30:53.101+0000 7f416cbfe640 1 --2- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] >> 127.0.0.1:0/863698824 conn(0x507b400 0x527f080 crc :-1 s=READY pgs=3 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-11T23:30:53.101+0000 7f416cbfe640 1 --2- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] >> 127.0.0.1:0/863698824 conn(0x507b400 0x527f080 crc :-1 s=READY pgs=3 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-11T23:30:53.101+0000 7f415cf02640 2 osd.0 98 ms_handle_reset con 0x507b400 session 0x4a316c0 2026-02-11T23:30:53.204+0000 7f415b6ff640 20 osd.0 98 reports for 0 queries 2026-02-11T23:30:53.204+0000 7f415b6ff640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] -- mgrreport(unknown.0 +0-0 packed 3294 daemon_metrics=2) -- 0x4643c00 con 0x4aca800 2026-02-11T23:30:53.204+0000 7f415b6ff640 15 osd.0 98 collect_pg_stats 2026-02-11T23:30:53.204+0000 7f415b6ff640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] -- pg_stats(0 pgs seq 21474836490 v 0) -- 0x4a9e600 con 0x4aca800 2026-02-11T23:30:53.327+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:30:53.327+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:30:53.327+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:23.328523+0000) 2026-02-11T23:30:53.334+0000 7f4162bea640 10 osd.0 98 tick_without_osd_lock 2026-02-11T23:30:53.334+0000 7f4162bea640 0 osd.0 98 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:53.334+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:53.334+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:53.334+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:53.334+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.367 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:53.334+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:53.335, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:53.334+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:53.334+0000 7f4162bea640 10 osd.0 98 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:53.334+0000 7f4162bea640 20 osd.0 98 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:53.334+0000 7f4162bea640 10 osd.0 98 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:53.334+0000 7f4162bea640 20 osd.0 98 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:53.347+0000 7f41633eb640 10 osd.0 98 tick 2026-02-11T23:30:53.347+0000 7f41633eb640 20 osd.0 98 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:30:53.940+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.1 127.0.0.1:0/3043114013 14 ==== osd_ping(ping e98 up_from 10 ping_stamp 2026-02-11T23:30:53.941578+0000/34.605323792s send_stamp 34.605323792s delta_ub -2.011577845s) ==== 2033+0+0 (crc 0 0 0) 0x4c9b680 con 0x4aca000 2026-02-11T23:30:53.940+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.1 127.0.0.1:0/3043114013 14 ==== osd_ping(ping e98 up_from 10 ping_stamp 2026-02-11T23:30:53.941578+0000/34.605323792s send_stamp 34.605323792s delta_ub -2.011577845s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a900 con 0x4aca400 2026-02-11T23:30:53.940+0000 7f416cbfe640 20 osd.0 98 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011803627s,-2.011577845s]) 2026-02-11T23:30:53.940+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e98 up_from 5 ping_stamp 2026-02-11T23:30:53.941578+0000/34.605323792s send_stamp 36.617126465s delta_ub 2.011803627s) -- 0x4c9ad80 con 0x4aca000 2026-02-11T23:30:53.940+0000 7f416cbfe640 20 osd.0 98 get_con_osd_cluster to osd.1 from_epoch 98 2026-02-11T23:30:53.940+0000 7f416cbfe640 20 osd.0 98 get_nextmap_reserved map_reservations: {98=1} 2026-02-11T23:30:53.940+0000 7f416cbfe640 20 osd.0 98 release_map epoch: 98 2026-02-11T23:30:53.940+0000 7f416cbfe640 10 osd.0 98 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 98 is not newer than session's projected_epoch of 98 2026-02-11T23:30:53.940+0000 7f416c3fd640 20 osd.0 98 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011831522s,-2.011577845s]) 2026-02-11T23:30:53.940+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e98 up_from 5 ping_stamp 2026-02-11T23:30:53.941578+0000/34.605323792s send_stamp 36.617153168s delta_ub 2.011831522s) -- 0x4c9b380 con 0x4aca400 2026-02-11T23:30:53.940+0000 7f416c3fd640 20 osd.0 98 get_con_osd_cluster to osd.1 from_epoch 98 2026-02-11T23:30:53.940+0000 7f416c3fd640 20 osd.0 98 get_nextmap_reserved map_reservations: {98=1} 2026-02-11T23:30:53.940+0000 7f416c3fd640 20 osd.0 98 release_map epoch: 98 2026-02-11T23:30:53.940+0000 7f416c3fd640 10 osd.0 98 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 98 is not newer than session's projected_epoch of 98 2026-02-11T23:30:54.272+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 21 ==== pg_lease(1.0 pg_lease(ru 42.937232971s ub 50.937343597s int 16.000000000s) e98/98) ==== 58+0+0 (unknown 0 0 0) 0x4c9a780 con 0x4acb000 2026-02-11T23:30:54.272+0000 7f416abfa640 15 osd.0 98 enqueue_peering_evt 1.0 epoch_sent: 98 epoch_requested: 98 MLease epoch 98 from osd.1 pg_lease(ru 42.937232971s ub 50.937343597s int 16.000000000s) 2026-02-11T23:30:54.272+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 98 epoch_requested: 98 MLease epoch 98 from osd.1 pg_lease(ru 42.937232971s ub 50.937343597s int 16.000000000s)) class_id 2 prio 255 cost 10 e98) 2026-02-11T23:30:54.272+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:54.272+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 98 epoch_requested: 98 MLease epoch 98 from osd.1 pg_lease(ru 42.937232971s ub 50.937343597s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e98) queued 2026-02-11T23:30:54.272+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:54.272+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 98 epoch_requested: 98 MLease epoch 98 from osd.1 pg_lease(ru 42.937232971s ub 50.937343597s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e98) pg 0x5946000 2026-02-11T23:30:54.272+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:54.272+0000 7f414b98b640 10 osd.0 pg_epoch: 98 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 98 epoch_requested: 98 MLease epoch 98 from osd.1 pg_lease(ru 42.937232971s ub 50.937343597s int 16.000000000s) 2026-02-11T23:30:54.272+0000 7f414b98b640 10 osd.0 pg_epoch: 98 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::proc_lease pg_lease(ru 42.937232971s ub 50.937343597s int 16.000000000s) 2026-02-11T23:30:54.272+0000 7f414b98b640 20 osd.0 pg_epoch: 98 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::proc_lease peer_clock_delta_ub -2.011577845s -> ru 44.948806763s 2026-02-11T23:30:54.272+0000 7f414b98b640 20 osd.0 pg_epoch: 98 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::proc_lease readable_until now 44.948806763s 2026-02-11T23:30:54.272+0000 7f414b98b640 20 osd.0 pg_epoch: 98 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::proc_lease peer_clock_delta_lb -2.011831522s -> ruub 52.949176788s 2026-02-11T23:30:54.272+0000 7f414b98b640 20 osd.0 pg_epoch: 98 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::proc_lease readable_until_ub now 52.949176788s 2026-02-11T23:30:54.272+0000 7f414b98b640 20 osd.0 98 get_con_osd_cluster to osd.1 from_epoch 98 2026-02-11T23:30:54.272+0000 7f414b98b640 20 osd.0 98 get_nextmap_reserved map_reservations: {98=1} 2026-02-11T23:30:54.272+0000 7f414b98b640 20 osd.0 98 release_map epoch: 98 2026-02-11T23:30:54.272+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_lease_ack(1.0 pg_lease_ack(ruub 50.937343597s) e98/98) -- 0x37009a0 con 0x4acb000 2026-02-11T23:30:54.272+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:54.272+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:54.297+0000 7f4162bea640 10 osd.0 98 tick_without_osd_lock 2026-02-11T23:30:54.297+0000 7f4162bea640 0 osd.0 98 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:54.297+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:54.297+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.367 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:54.297+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:54.298, recovery is active?:false restrictions:<....> 2026-02-11T23:30:54.297+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:54.297+0000 7f4162bea640 10 osd.0 98 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:54.297+0000 7f4162bea640 20 osd.0 98 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:54.297+0000 7f4162bea640 10 osd.0 98 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:54.297+0000 7f4162bea640 20 osd.0 98 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:54.326+0000 7f41633eb640 10 osd.0 98 tick 2026-02-11T23:30:54.326+0000 7f41633eb640 20 osd.0 98 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:30:54.327+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:30:54.327+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:30:54.327+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:24.328602+0000) 2026-02-11T23:30:54.694+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.2 127.0.0.1:0/2365263801 8 ==== osd_ping(ping e98 up_from 14 ping_stamp 2026-02-11T23:30:54.695550+0000/32.804145813s send_stamp 32.804145813s delta_ub -4.566704750s) ==== 2033+0+0 (crc 0 0 0) 0x4c9ad80 con 0x4641000 2026-02-11T23:30:54.694+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.2 127.0.0.1:0/2365263801 8 ==== osd_ping(ping e98 up_from 14 ping_stamp 2026-02-11T23:30:54.695550+0000/32.804145813s send_stamp 32.804145813s delta_ub -4.566704750s) ==== 2033+0+0 (crc 0 0 0) 0x4c9b380 con 0x48cc000 2026-02-11T23:30:54.694+0000 7f416cbfe640 20 osd.0 98 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566910744s,-4.566704750s]) 2026-02-11T23:30:54.694+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e98 up_from 5 ping_stamp 2026-02-11T23:30:54.695550+0000/32.804145813s send_stamp 37.371055603s delta_ub 4.566910744s) -- 0x4c9b680 con 0x4641000 2026-02-11T23:30:54.694+0000 7f416cbfe640 20 osd.0 98 get_con_osd_cluster to osd.2 from_epoch 98 2026-02-11T23:30:54.694+0000 7f416cbfe640 20 osd.0 98 get_nextmap_reserved map_reservations: {98=1} 2026-02-11T23:30:54.694+0000 7f416cbfe640 20 osd.0 98 release_map epoch: 98 2026-02-11T23:30:54.694+0000 7f416cbfe640 10 osd.0 98 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 98 is not newer than session's projected_epoch of 98 2026-02-11T23:30:54.694+0000 7f416c3fd640 20 osd.0 98 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566933155s,-4.566704750s]) 2026-02-11T23:30:54.694+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e98 up_from 5 ping_stamp 2026-02-11T23:30:54.695550+0000/32.804145813s send_stamp 37.371082306s delta_ub 4.566933155s) -- 0x4c9a900 con 0x48cc000 2026-02-11T23:30:54.694+0000 7f416c3fd640 20 osd.0 98 get_con_osd_cluster to osd.2 from_epoch 98 2026-02-11T23:30:54.694+0000 7f416c3fd640 20 osd.0 98 get_nextmap_reserved map_reservations: {98=1} 2026-02-11T23:30:54.694+0000 7f416c3fd640 20 osd.0 98 release_map epoch: 98 2026-02-11T23:30:54.694+0000 7f416c3fd640 10 osd.0 98 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 98 is not newer than session's projected_epoch of 98 2026-02-11T23:30:55.228+0000 7f4147983640 5 osd.0 98 heartbeat osd_stat(store_statfs(0x384000/0x0/0x384000, data 0x6f4d9/0xf8000, compress 0x0/0x0/0x0, omap 0x94c1, meta 0x1a46b3f), peers [1,2,3,4] op hist []) 2026-02-11T23:30:55.228+0000 7f4147983640 20 osd.0 98 check_full_status cur ratio 0, physical ratio 0, new state none 2026-02-11T23:30:55.228+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6816/3043114013,v1:127.0.0.1:6817/3043114013] -- osd_ping(ping e98 up_from 5 ping_stamp 2026-02-11T23:30:55.229786+0000/37.905189514s send_stamp 37.905189514s delta_ub 2.011831522s) -- 0x4c9b080 con 0x48c4400 2026-02-11T23:30:55.228+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6814/3043114013,v1:127.0.0.1:6815/3043114013] -- osd_ping(ping e98 up_from 5 ping_stamp 2026-02-11T23:30:55.229786+0000/37.905189514s send_stamp 37.905189514s delta_ub 2.011831522s) -- 0x4c9b800 con 0x4640c00 2026-02-11T23:30:55.228+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6824/2365263801,v1:127.0.0.1:6825/2365263801] -- osd_ping(ping e98 up_from 5 ping_stamp 2026-02-11T23:30:55.229786+0000/37.905189514s send_stamp 37.905189514s delta_ub 4.566933155s) -- 0x4c9aa80 con 0x48c4000 2026-02-11T23:30:55.228+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6822/2365263801,v1:127.0.0.1:6823/2365263801] -- osd_ping(ping e98 up_from 5 ping_stamp 2026-02-11T23:30:55.229786+0000/37.905189514s send_stamp 37.905189514s delta_ub 4.566933155s) -- 0x4a89980 con 0x48cd000 2026-02-11T23:30:55.228+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6832/2699377081,v1:127.0.0.1:6833/2699377081] -- osd_ping(ping e98 up_from 5 ping_stamp 2026-02-11T23:30:55.229786+0000/37.905189514s send_stamp 37.905189514s delta_ub 7.219551563s) -- 0x5af8a80 con 0x48cd400 2026-02-11T23:30:55.228+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6830/2699377081,v1:127.0.0.1:6831/2699377081] -- osd_ping(ping e98 up_from 5 ping_stamp 2026-02-11T23:30:55.229786+0000/37.905189514s send_stamp 37.905189514s delta_ub 7.219551563s) -- 0x5af8000 con 0x48cd800 2026-02-11T23:30:55.228+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6840/3331151400,v1:127.0.0.1:6841/3331151400] -- osd_ping(ping e98 up_from 5 ping_stamp 2026-02-11T23:30:55.229786+0000/37.905189514s send_stamp 37.905189514s delta_ub 9.836741447s) -- 0x5af8180 con 0x507bc00 2026-02-11T23:30:55.228+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6838/3331151400,v1:127.0.0.1:6839/3331151400] -- osd_ping(ping e98 up_from 5 ping_stamp 2026-02-11T23:30:55.229786+0000/37.905189514s send_stamp 37.905189514s delta_ub 9.836741447s) -- 0x5af8480 con 0x5282000 2026-02-11T23:30:55.228+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6814/3043114013 11 ==== osd_ping(ping_reply e98 up_from 10 ping_stamp 2026-02-11T23:30:55.229786+0000/37.905189514s send_stamp 35.893634796s delta_ub -2.011560440s) ==== 2033+0+0 (crc 0 0 0) 0x5af8480 con 0x4640c00 2026-02-11T23:30:55.228+0000 7f416c3fd640 20 osd.0 98 get_con_osd_cluster to osd.1 from_epoch 98 2026-02-11T23:30:55.228+0000 7f416c3fd640 20 osd.0 98 get_nextmap_reserved map_reservations: {98=1} 2026-02-11T23:30:55.228+0000 7f416c3fd640 20 osd.0 98 release_map epoch: 98 2026-02-11T23:30:55.228+0000 7f416c3fd640 10 osd.0 98 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 98 is not newer than session's projected_epoch of 98 2026-02-11T23:30:55.228+0000 7f416c3fd640 20 osd.0 98 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011724234s,-2.011560440s]) 2026-02-11T23:30:55.228+0000 7f416cbfe640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6824/2365263801 10 ==== osd_ping(ping_reply e98 up_from 14 ping_stamp 2026-02-11T23:30:55.229786+0000/37.905189514s send_stamp 33.338493347s delta_ub -4.566697121s) ==== 2033+0+0 (crc 0 0 0) 0x4c9aa80 con 0x48c4000 2026-02-11T23:30:55.228+0000 7f416cbfe640 20 osd.0 98 get_con_osd_cluster to osd.2 from_epoch 98 2026-02-11T23:30:55.228+0000 7f416cbfe640 20 osd.0 98 get_nextmap_reserved map_reservations: {98=1} 2026-02-11T23:30:55.228+0000 7f416cbfe640 20 osd.0 98 release_map epoch: 98 2026-02-11T23:30:55.228+0000 7f416cbfe640 10 osd.0 98 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 98 is not newer than session's projected_epoch of 98 2026-02-11T23:30:55.228+0000 7f416cbfe640 20 osd.0 98 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566874504s,-4.566697121s]) 2026-02-11T23:30:55.228+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6830/2699377081 10 ==== osd_ping(ping_reply e98 up_from 19 ping_stamp 2026-02-11T23:30:55.229786+0000/37.905189514s send_stamp 30.685878754s delta_ub -7.219313622s) ==== 2033+0+0 (crc 0 0 0) 0x5af8480 con 0x48cd800 2026-02-11T23:30:55.228+0000 7f416c3fd640 20 osd.0 98 get_con_osd_cluster to osd.3 from_epoch 98 2026-02-11T23:30:55.228+0000 7f416c3fd640 20 osd.0 98 get_nextmap_reserved map_reservations: {98=1} 2026-02-11T23:30:55.228+0000 7f416c3fd640 20 osd.0 98 release_map epoch: 98 2026-02-11T23:30:55.228+0000 7f416c3fd640 10 osd.0 98 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 98 is not newer than session's projected_epoch of 98 2026-02-11T23:30:55.228+0000 7f416c3fd640 20 osd.0 98 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219511509s,-7.219313622s]) 2026-02-11T23:30:55.228+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6838/3331151400 9 ==== osd_ping(ping_reply e98 up_from 23 ping_stamp 2026-02-11T23:30:55.229786+0000/37.905189514s send_stamp 28.068710327s delta_ub -9.836481094s) ==== 2033+0+0 (crc 0 0 0) 0x5af8480 con 0x5282000 2026-02-11T23:30:55.228+0000 7f416c3fd640 20 osd.0 98 get_con_osd_cluster to osd.4 from_epoch 98 2026-02-11T23:30:55.228+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6816/3043114013 11 ==== osd_ping(ping_reply e98 up_from 10 ping_stamp 2026-02-11T23:30:55.229786+0000/37.905189514s send_stamp 35.893653870s delta_ub -2.011538982s) ==== 2033+0+0 (crc 0 0 0) 0x5af8180 con 0x48c4400 2026-02-11T23:30:55.228+0000 7f416c3fd640 20 osd.0 98 get_nextmap_reserved map_reservations: {98=1} 2026-02-11T23:30:55.228+0000 7f416c3fd640 20 osd.0 98 release_map epoch: 98 2026-02-11T23:30:55.228+0000 7f416c3fd640 10 osd.0 98 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 98 is not newer than session's projected_epoch of 98 2026-02-11T23:30:55.228+0000 7f416c3fd640 20 osd.0 98 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836691856s,-9.836481094s]) 2026-02-11T23:30:55.228+0000 7f416abfa640 20 osd.0 98 get_con_osd_cluster to osd.1 from_epoch 98 2026-02-11T23:30:55.228+0000 7f416abfa640 20 osd.0 98 get_nextmap_reserved map_reservations: {98=1} 2026-02-11T23:30:55.228+0000 7f416abfa640 20 osd.0 98 release_map epoch: 98 2026-02-11T23:30:55.228+0000 7f416abfa640 10 osd.0 98 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 98 is not newer than session's projected_epoch of 98 2026-02-11T23:30:55.228+0000 7f416abfa640 20 osd.0 98 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011759043s,-2.011538982s]) 2026-02-11T23:30:55.228+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6832/2699377081 10 ==== osd_ping(ping_reply e98 up_from 19 ping_stamp 2026-02-11T23:30:55.229786+0000/37.905189514s send_stamp 30.685855865s delta_ub -7.219336510s) ==== 2033+0+0 (crc 0 0 0) 0x5af8180 con 0x48cd400 2026-02-11T23:30:55.228+0000 7f416abfa640 20 osd.0 98 get_con_osd_cluster to osd.3 from_epoch 98 2026-02-11T23:30:55.228+0000 7f416abfa640 20 osd.0 98 get_nextmap_reserved map_reservations: {98=1} 2026-02-11T23:30:55.228+0000 7f416abfa640 20 osd.0 98 release_map epoch: 98 2026-02-11T23:30:55.228+0000 7f416abfa640 10 osd.0 98 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 98 is not newer than session's projected_epoch of 98 2026-02-11T23:30:55.228+0000 7f416abfa640 20 osd.0 98 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219576836s,-7.219336510s]) 2026-02-11T23:30:55.228+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6840/3331151400 9 ==== osd_ping(ping_reply e98 up_from 23 ping_stamp 2026-02-11T23:30:55.229786+0000/37.905189514s send_stamp 28.068685532s delta_ub -9.836505890s) ==== 2033+0+0 (crc 0 0 0) 0x5af8180 con 0x507bc00 2026-02-11T23:30:55.228+0000 7f416abfa640 20 osd.0 98 get_con_osd_cluster to osd.4 from_epoch 98 2026-02-11T23:30:55.228+0000 7f416abfa640 20 osd.0 98 get_nextmap_reserved map_reservations: {98=1} 2026-02-11T23:30:55.228+0000 7f416abfa640 20 osd.0 98 release_map epoch: 98 2026-02-11T23:30:55.228+0000 7f416abfa640 10 osd.0 98 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 98 is not newer than session's projected_epoch of 98 2026-02-11T23:30:55.228+0000 7f416abfa640 20 osd.0 98 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836756706s,-9.836505890s]) 2026-02-11T23:30:55.228+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6822/2365263801 10 ==== osd_ping(ping_reply e98 up_from 14 ping_stamp 2026-02-11T23:30:55.229786+0000/37.905189514s send_stamp 33.338520050s delta_ub -4.566670895s) ==== 2033+0+0 (crc 0 0 0) 0x5af8180 con 0x48cd000 2026-02-11T23:30:55.228+0000 7f416abfa640 20 osd.0 98 get_con_osd_cluster to osd.2 from_epoch 98 2026-02-11T23:30:55.228+0000 7f416abfa640 20 osd.0 98 get_nextmap_reserved map_reservations: {98=1} 2026-02-11T23:30:55.228+0000 7f416abfa640 20 osd.0 98 release_map epoch: 98 2026-02-11T23:30:55.228+0000 7f416abfa640 10 osd.0 98 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 98 is not newer than session's projected_epoch of 98 2026-02-11T23:30:55.228+0000 7f416abfa640 20 osd.0 98 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566936493s,-4.566670895s]) 2026-02-11T23:30:55.268+0000 7f4162bea640 10 osd.0 98 tick_without_osd_lock 2026-02-11T23:30:55.268+0000 7f4162bea640 0 osd.0 98 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:55.268+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:55.268+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:55.268+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:55.268+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.367 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:55.268+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:55.270, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:55.268+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:55.268+0000 7f4162bea640 10 osd.0 98 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:55.268+0000 7f4162bea640 20 osd.0 98 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:55.268+0000 7f4162bea640 10 osd.0 98 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:55.268+0000 7f4162bea640 20 osd.0 98 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:55.321+0000 7f41633eb640 10 osd.0 98 tick 2026-02-11T23:30:55.321+0000 7f41633eb640 20 osd.0 98 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:30:55.327+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:30:55.327+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:30:55.327+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:25.328675+0000) 2026-02-11T23:30:56.232+0000 7f4162bea640 10 osd.0 98 tick_without_osd_lock 2026-02-11T23:30:56.232+0000 7f4162bea640 0 osd.0 98 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:56.232+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:56.232+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:56.232+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:56.232+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.367 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:56.232+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:56.234, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:56.232+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:56.232+0000 7f4162bea640 10 osd.0 98 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:56.232+0000 7f4162bea640 20 osd.0 98 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:56.232+0000 7f4162bea640 10 osd.0 98 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:56.232+0000 7f4162bea640 20 osd.0 98 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:56.284+0000 7f41633eb640 10 osd.0 98 tick 2026-02-11T23:30:56.284+0000 7f41633eb640 20 osd.0 98 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:30:56.327+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:30:56.327+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:30:56.327+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:26.328747+0000) 2026-02-11T23:30:56.840+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.1 127.0.0.1:0/3043114013 15 ==== osd_ping(ping e98 up_from 10 ping_stamp 2026-02-11T23:30:56.841780+0000/37.505523682s send_stamp 37.505523682s delta_ub -2.011538982s) ==== 2033+0+0 (crc 0 0 0) 0x4c9aa80 con 0x4aca000 2026-02-11T23:30:56.840+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.1 127.0.0.1:0/3043114013 15 ==== osd_ping(ping e98 up_from 10 ping_stamp 2026-02-11T23:30:56.841780+0000/37.505523682s send_stamp 37.505523682s delta_ub -2.011538982s) ==== 2033+0+0 (crc 0 0 0) 0x5af8480 con 0x4aca400 2026-02-11T23:30:56.840+0000 7f416cbfe640 20 osd.0 98 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011749268s,-2.011538982s]) 2026-02-11T23:30:56.840+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e98 up_from 5 ping_stamp 2026-02-11T23:30:56.841780+0000/37.505523682s send_stamp 39.517272949s delta_ub 2.011749268s) -- 0x4c9b680 con 0x4aca000 2026-02-11T23:30:56.840+0000 7f416cbfe640 20 osd.0 98 get_con_osd_cluster to osd.1 from_epoch 98 2026-02-11T23:30:56.840+0000 7f416cbfe640 20 osd.0 98 get_nextmap_reserved map_reservations: {98=1} 2026-02-11T23:30:56.840+0000 7f416cbfe640 20 osd.0 98 release_map epoch: 98 2026-02-11T23:30:56.840+0000 7f416cbfe640 10 osd.0 98 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 98 is not newer than session's projected_epoch of 98 2026-02-11T23:30:56.840+0000 7f416c3fd640 20 osd.0 98 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011770010s,-2.011538982s]) 2026-02-11T23:30:56.840+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e98 up_from 5 ping_stamp 2026-02-11T23:30:56.841780+0000/37.505523682s send_stamp 39.517295837s delta_ub 2.011770010s) -- 0x5af8300 con 0x4aca400 2026-02-11T23:30:56.840+0000 7f416c3fd640 20 osd.0 98 get_con_osd_cluster to osd.1 from_epoch 98 2026-02-11T23:30:56.840+0000 7f416c3fd640 20 osd.0 98 get_nextmap_reserved map_reservations: {98=1} 2026-02-11T23:30:56.840+0000 7f416c3fd640 20 osd.0 98 release_map epoch: 98 2026-02-11T23:30:56.840+0000 7f416c3fd640 10 osd.0 98 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 98 is not newer than session's projected_epoch of 98 2026-02-11T23:30:56.964+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.4 127.0.0.1:0/3331151400 8 ==== osd_ping(ping e98 up_from 23 ping_stamp 2026-02-11T23:30:56.965431+0000/29.804199219s send_stamp 29.804199219s delta_ub -9.836481094s) ==== 2033+0+0 (crc 0 0 0) 0x5af8300 con 0x507a800 2026-02-11T23:30:56.964+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.4 127.0.0.1:0/3331151400 8 ==== osd_ping(ping e98 up_from 23 ping_stamp 2026-02-11T23:30:56.965431+0000/29.804199219s send_stamp 29.804199219s delta_ub -9.836481094s) ==== 2033+0+0 (crc 0 0 0) 0x5af8180 con 0x507a400 2026-02-11T23:30:56.964+0000 7f416c3fd640 20 osd.0 98 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836735725s,-9.836481094s]) 2026-02-11T23:30:56.964+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e98 up_from 5 ping_stamp 2026-02-11T23:30:56.965431+0000/29.804199219s send_stamp 39.640937805s delta_ub 9.836735725s) -- 0x5af8480 con 0x507a800 2026-02-11T23:30:56.964+0000 7f416c3fd640 20 osd.0 98 get_con_osd_cluster to osd.4 from_epoch 98 2026-02-11T23:30:56.964+0000 7f416c3fd640 20 osd.0 98 get_nextmap_reserved map_reservations: {98=1} 2026-02-11T23:30:56.964+0000 7f416c3fd640 20 osd.0 98 release_map epoch: 98 2026-02-11T23:30:56.964+0000 7f416c3fd640 10 osd.0 98 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 98 is not newer than session's projected_epoch of 98 2026-02-11T23:30:56.964+0000 7f416abfa640 20 osd.0 98 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836760521s,-9.836481094s]) 2026-02-11T23:30:56.964+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e98 up_from 5 ping_stamp 2026-02-11T23:30:56.965431+0000/29.804199219s send_stamp 39.640956879s delta_ub 9.836760521s) -- 0x5af8a80 con 0x507a400 2026-02-11T23:30:56.964+0000 7f416abfa640 20 osd.0 98 get_con_osd_cluster to osd.4 from_epoch 98 2026-02-11T23:30:56.964+0000 7f416abfa640 20 osd.0 98 get_nextmap_reserved map_reservations: {98=1} 2026-02-11T23:30:56.964+0000 7f416abfa640 20 osd.0 98 release_map epoch: 98 2026-02-11T23:30:56.964+0000 7f416abfa640 10 osd.0 98 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 98 is not newer than session's projected_epoch of 98 2026-02-11T23:30:57.222+0000 7f4162bea640 10 osd.0 98 tick_without_osd_lock 2026-02-11T23:30:57.222+0000 7f4162bea640 0 osd.0 98 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:57.222+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:57.222+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.367 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:57.222+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:57.224, recovery is active?:false restrictions:<....> 2026-02-11T23:30:57.222+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:57.222+0000 7f4162bea640 10 osd.0 98 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:57.222+0000 7f4162bea640 20 osd.0 98 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:57.222+0000 7f4162bea640 10 osd.0 98 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:57.222+0000 7f4162bea640 20 osd.0 98 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:57.266+0000 7f41633eb640 10 osd.0 98 tick 2026-02-11T23:30:57.266+0000 7f41633eb640 20 osd.0 98 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:30:57.327+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:30:57.327+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:30:57.327+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:27.328817+0000) 2026-02-11T23:30:57.747+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.3 127.0.0.1:0/2699377081 10 ==== osd_ping(ping e98 up_from 19 ping_stamp 2026-02-11T23:30:57.748725+0000/33.204666138s send_stamp 33.204666138s delta_ub -7.219313622s) ==== 2033+0+0 (crc 0 0 0) 0x4c9b680 con 0x48cc800 2026-02-11T23:30:57.747+0000 7f416cbfe640 20 osd.0 98 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219570160s,-7.219313622s]) 2026-02-11T23:30:57.747+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.3 127.0.0.1:0/2699377081 10 ==== osd_ping(ping e98 up_from 19 ping_stamp 2026-02-11T23:30:57.748725+0000/33.204666138s send_stamp 33.204666138s delta_ub -7.219313622s) ==== 2033+0+0 (crc 0 0 0) 0x5af8480 con 0x48cc400 2026-02-11T23:30:57.747+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e98 up_from 5 ping_stamp 2026-02-11T23:30:57.748725+0000/33.204666138s send_stamp 40.424236298s delta_ub 7.219570160s) -- 0x4c9aa80 con 0x48cc800 2026-02-11T23:30:57.747+0000 7f416cbfe640 20 osd.0 98 get_con_osd_cluster to osd.3 from_epoch 98 2026-02-11T23:30:57.747+0000 7f416cbfe640 20 osd.0 98 get_nextmap_reserved map_reservations: {98=1} 2026-02-11T23:30:57.747+0000 7f416cbfe640 20 osd.0 98 release_map epoch: 98 2026-02-11T23:30:57.747+0000 7f416cbfe640 10 osd.0 98 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 98 is not newer than session's projected_epoch of 98 2026-02-11T23:30:57.747+0000 7f416c3fd640 20 osd.0 98 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219594955s,-7.219313622s]) 2026-02-11T23:30:57.747+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e98 up_from 5 ping_stamp 2026-02-11T23:30:57.748725+0000/33.204666138s send_stamp 40.424259186s delta_ub 7.219594955s) -- 0x5af8300 con 0x48cc400 2026-02-11T23:30:57.747+0000 7f416c3fd640 20 osd.0 98 get_con_osd_cluster to osd.3 from_epoch 98 2026-02-11T23:30:57.747+0000 7f416c3fd640 20 osd.0 98 get_nextmap_reserved map_reservations: {98=1} 2026-02-11T23:30:57.747+0000 7f416c3fd640 20 osd.0 98 release_map epoch: 98 2026-02-11T23:30:57.747+0000 7f416c3fd640 10 osd.0 98 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 98 is not newer than session's projected_epoch of 98 2026-02-11T23:30:58.020+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 47 ==== osd_map(99..99 src has 1..99) ==== 296+0+0 (crc 0 0 0) 0x59556c0 con 0x4acbc00 2026-02-11T23:30:58.020+0000 7f415cf02640 20 osd.0 98 OSD::ms_dispatch: osd_map(99..99 src has 1..99) 2026-02-11T23:30:58.020+0000 7f415cf02640 20 osd.0 98 _dispatch 0x59556c0 osd_map(99..99 src has 1..99) 2026-02-11T23:30:58.020+0000 7f415cf02640 3 osd.0 98 handle_osd_map epochs [99,99], i have 98, src has [1,99] 2026-02-11T23:30:58.020+0000 7f415cf02640 10 osd.0 98 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:30:58.020+0000 7f415cf02640 10 osd.0 98 handle_osd_map got inc map for epoch 99 2026-02-11T23:30:58.020+0000 7f415cf02640 20 osd.0 98 got_full_map 99, nothing requested 2026-02-11T23:30:58.020+0000 7f415cf02640 10 osd.0 98 _track_pools_and_pg_num_changes recording final pg_pool_t for pool 2 2026-02-11T23:30:58.020+0000 7f415cf02640 20 osd.0 98 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:58.020+0000 7f415cf02640 20 osd.0 98 handle_osd_map pg_num_history pg_num_history(e99 pg_nums {1={27=1},2={34=1},3={42=1},4={49=1},5={56=1},6={64=1},7={70=1},8={77=1},9={85=1},10={92=1}} deleted_pools 99,2) 2026-02-11T23:30:58.020+0000 7f415cf02640 10 snap_mapper record_purged_snaps purged_snaps {99={}} 2026-02-11T23:30:58.020+0000 7f415cf02640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:58.020+0000 7f415cf02640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e99 maps [1~99] lci=[5,99] tlb=1) 2026-02-11T23:30:58.021+0000 7f41591a6640 10 osd.0 98 _committed_osd_maps 99..99 2026-02-11T23:30:58.021+0000 7f41591a6640 10 osd.0 98 advance to epoch 99 (<= last 99 <= newest_map 99) 2026-02-11T23:30:58.021+0000 7f41591a6640 20 osd.0 99 consume_map version 99 2026-02-11T23:30:58.021+0000 7f41591a6640 20 osd.0 98 await_reserved_maps epoch:99 2026-02-11T23:30:58.021+0000 7f41591a6640 20 osd.0 98 await_reserved_maps done 0 2026-02-11T23:30:58.021+0000 7f41591a6640 20 osd.0 99 consume_map 99 -- publish done 2026-02-11T23:30:58.021+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 3 slots 2026-02-11T23:30:58.021+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:10.0slot:0x37594a0 2026-02-11T23:30:58.021+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot 10.0 has no pg and waiting_for_split 2026-02-11T23:30:58.021+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x37582c0 2026-02-11T23:30:58.021+0000 7f41591a6640 20 osd.0 99 identify_splits_and_merges 4.0 e98 to e99 2026-02-11T23:30:58.021+0000 7f41591a6640 20 osd.0 99 identify_splits_and_merges 4.0 e98 to e99 pg_nums {49=1} 2026-02-11T23:30:58.021+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3700dc0 2026-02-11T23:30:58.021+0000 7f41591a6640 20 osd.0 99 identify_splits_and_merges 1.0 e98 to e99 2026-02-11T23:30:58.021+0000 7f41591a6640 20 osd.0 99 identify_splits_and_merges 1.0 e98 to e99 pg_nums {27=1} 2026-02-11T23:30:58.021+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:58.021+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:58.021+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:58.021+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:58.021+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:58.021+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:58.021+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:58.021+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:58.021+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:58.021+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:58.021+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:58.021+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:58.021+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:58.021+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:58.021+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:58.021+0000 7f41591a6640 20 osd.0 99 prune_pg_created 2026-02-11T23:30:58.021+0000 7f41591a6640 10 osd.0:0.consume_map 99 (was 98) 2026-02-11T23:30:58.021+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 10.0 2026-02-11T23:30:58.021+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 10.0 empty, pruning 2026-02-11T23:30:58.021+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:58.021+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:58.021+0000 7f41591a6640 10 osd.0:1.consume_map 99 (was 98) 2026-02-11T23:30:58.021+0000 7f41591a6640 10 osd.0:2.consume_map 99 (was 98) 2026-02-11T23:30:58.021+0000 7f41591a6640 10 osd.0:3.consume_map 99 (was 98) 2026-02-11T23:30:58.021+0000 7f41591a6640 10 osd.0:4.consume_map 99 (was 98) 2026-02-11T23:30:58.021+0000 7f41591a6640 10 osd.0:5.consume_map 99 (was 98) 2026-02-11T23:30:58.021+0000 7f41591a6640 10 osd.0:6.consume_map 99 (was 98) 2026-02-11T23:30:58.021+0000 7f41591a6640 10 osd.0:7.consume_map 99 (was 98) 2026-02-11T23:30:58.021+0000 7f41591a6640 10 osd.0 99 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:58.021+0000 7f41591a6640 15 osd.0 99 enqueue_peering_evt 4.0 epoch_sent: 99 epoch_requested: 99 NullEvt 2026-02-11T23:30:58.021+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 99 epoch_requested: 99 NullEvt) class_id 2 prio 255 cost 10 e99) 2026-02-11T23:30:58.021+0000 7f41591a6640 15 osd.0 99 enqueue_peering_evt 1.0 epoch_sent: 99 epoch_requested: 99 NullEvt 2026-02-11T23:30:58.021+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 99 epoch_requested: 99 NullEvt) class_id 2 prio 255 cost 10 e99) 2026-02-11T23:30:58.021+0000 7f41591a6640 7 osd.0 99 activate_map version 99 2026-02-11T23:30:58.021+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:58.021+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 99 epoch_requested: 99 NullEvt) class_id 2immediate prio 255 cost 10 e99) queued 2026-02-11T23:30:58.021+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:58.021+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 99 epoch_requested: 99 NullEvt) class_id 2immediate prio 255 cost 10 e99) pg 0x5978000 2026-02-11T23:30:58.021+0000 7f4150194640 20 osd.0 99 agent_entry empty queue 2026-02-11T23:30:58.021+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:58.021+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 99 epoch_requested: 99 NullEvt) class_id 2immediate prio 255 cost 10 e99) queued 2026-02-11T23:30:58.021+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:58.021+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 99 epoch_requested: 99 NullEvt) class_id 2immediate prio 255 cost 10 e99) pg 0x5946000 2026-02-11T23:30:58.021+0000 7f414f993640 10 osd.0 pg_epoch: 98 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 99 2026-02-11T23:30:58.021+0000 7f414f993640 20 osd.0:0.update_pg_epoch 4.0 98 -> 99 2026-02-11T23:30:58.021+0000 7f414f993640 10 osd.0 pg_epoch: 98 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:58.021+0000 7f414b98b640 10 osd.0 pg_epoch: 98 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 99 2026-02-11T23:30:58.021+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 1.0 98 -> 99 2026-02-11T23:30:58.021+0000 7f414f993640 10 osd.0 pg_epoch: 99 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:58.021+0000 7f414b98b640 10 osd.0 pg_epoch: 98 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:58.021+0000 7f414f993640 10 osd.0 pg_epoch: 99 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:58.021+0000 7f414b98b640 10 osd.0 pg_epoch: 99 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:58.021+0000 7f414b98b640 10 osd.0 pg_epoch: 99 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:58.021+0000 7f414b98b640 20 osd.0 99 advance_pg new pool opts old pool opts 2026-02-11T23:30:58.021+0000 7f414f993640 20 osd.0 99 advance_pg new pool opts old pool opts 2026-02-11T23:30:58.021+0000 7f414b98b640 10 osd.0 pg_epoch: 99 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_activate_map: epoch range: 99..99 2026-02-11T23:30:58.021+0000 7f414b98b640 10 osd.0 pg_epoch: 99 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:58.021+0000 7f414b98b640 20 osd.0 pg_epoch: 99 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 73 while current is 99 2026-02-11T23:30:58.021+0000 7f414f993640 10 osd.0 pg_epoch: 99 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_activate_map: epoch range: 99..99 2026-02-11T23:30:58.021+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:58.021+0000 7f414f993640 10 osd.0 pg_epoch: 99 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:58.021+0000 7f414b98b640 10 osd.0 pg_epoch: 99 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 99 epoch_requested: 99 NullEvt 2026-02-11T23:30:58.021+0000 7f414f993640 20 osd.0 pg_epoch: 99 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 95 while current is 99 2026-02-11T23:30:58.021+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:58.021+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:58.021+0000 7f414f993640 10 osd.0 pg_epoch: 99 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 99 epoch_requested: 99 NullEvt 2026-02-11T23:30:58.021+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:58.021+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:58.021+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:58.204+0000 7f415b6ff640 20 osd.0 99 reports for 0 queries 2026-02-11T23:30:58.204+0000 7f415b6ff640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] -- mgrreport(unknown.0 +0-0 packed 3294 daemon_metrics=2) -- 0x4643880 con 0x4aca800 2026-02-11T23:30:58.204+0000 7f415b6ff640 15 osd.0 99 collect_pg_stats 2026-02-11T23:30:58.204+0000 7f415b6ff640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] -- pg_stats(0 pgs seq 21474836491 v 0) -- 0x4a9e300 con 0x4aca800 2026-02-11T23:30:58.224+0000 7f4162bea640 10 osd.0 99 tick_without_osd_lock 2026-02-11T23:30:58.225+0000 7f4162bea640 0 osd.0 99 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:58.225+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:58.225+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.341 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:58.225+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:58.226, recovery is active?:false restrictions:<....> 2026-02-11T23:30:58.225+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:58.225+0000 7f4162bea640 10 osd.0 99 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:58.225+0000 7f4162bea640 20 osd.0 99 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:58.225+0000 7f4162bea640 10 osd.0 99 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:58.225+0000 7f4162bea640 20 osd.0 99 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:58.298+0000 7f41633eb640 10 osd.0 99 tick 2026-02-11T23:30:58.298+0000 7f41633eb640 10 osd.0 99 maybe_update_heartbeat_peers forcing update after 20.147743 seconds 2026-02-11T23:30:58.298+0000 7f41633eb640 10 osd.0 99 reset_heartbeat_peers 2026-02-11T23:30:58.298+0000 7f41633eb640 10 osd.0 99 maybe_update_heartbeat_peers updating 2026-02-11T23:30:58.298+0000 7f41633eb640 10 osd.0 99 adding neighbor peer osd.1 2026-02-11T23:30:58.298+0000 7f41633eb640 10 osd.0 99 adding neighbor peer osd.4 2026-02-11T23:30:58.298+0000 7f41633eb640 10 osd.0 99 maybe_update_heartbeat_peers 4 peers, extras 1,2,3,4 2026-02-11T23:30:58.298+0000 7f41633eb640 20 osd.0 99 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:30:58.327+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:30:58.327+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:30:58.327+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:28.328839+0000) 2026-02-11T23:30:59.255+0000 7f41633eb640 10 osd.0 99 tick 2026-02-11T23:30:59.255+0000 7f41633eb640 20 osd.0 99 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:30:59.264+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.4 127.0.0.1:0/3331151400 9 ==== osd_ping(ping e103 up_from 23 ping_stamp 2026-02-11T23:30:59.265633+0000/32.104400635s send_stamp 32.104400635s delta_ub -9.836546898s) ==== 2033+0+0 (crc 0 0 0) 0x5af8300 con 0x507a800 2026-02-11T23:30:59.264+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.4 127.0.0.1:0/3331151400 9 ==== osd_ping(ping e103 up_from 23 ping_stamp 2026-02-11T23:30:59.265633+0000/32.104400635s send_stamp 32.104400635s delta_ub -9.836546898s) ==== 2033+0+0 (crc 0 0 0) 0x5af8a80 con 0x507a400 2026-02-11T23:30:59.264+0000 7f416abfa640 20 osd.0 99 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836732864s,-9.836546898s]) 2026-02-11T23:30:59.264+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e99 up_from 5 ping_stamp 2026-02-11T23:30:59.265633+0000/32.104400635s send_stamp 41.941135406s delta_ub 9.836732864s) -- 0x5af8180 con 0x507a400 2026-02-11T23:30:59.264+0000 7f416abfa640 20 osd.0 99 get_con_osd_cluster to osd.4 from_epoch 99 2026-02-11T23:30:59.264+0000 7f416abfa640 20 osd.0 99 get_nextmap_reserved map_reservations: {99=1} 2026-02-11T23:30:59.264+0000 7f416abfa640 20 osd.0 99 release_map epoch: 99 2026-02-11T23:30:59.264+0000 7f416abfa640 10 osd.0 99 maybe_share_map: con v2:127.0.0.1:6836/3331151400 updating session's projected_epoch from 98 to ping map epoch of 103 2026-02-11T23:30:59.264+0000 7f416abfa640 10 osd.0 99 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 99 is not newer than session's projected_epoch of 103 2026-02-11T23:30:59.264+0000 7f416c3fd640 20 osd.0 99 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836755753s,-9.836546898s]) 2026-02-11T23:30:59.264+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e99 up_from 5 ping_stamp 2026-02-11T23:30:59.265633+0000/32.104400635s send_stamp 41.941154480s delta_ub 9.836755753s) -- 0x5af8480 con 0x507a800 2026-02-11T23:30:59.264+0000 7f416c3fd640 20 osd.0 99 get_con_osd_cluster to osd.4 from_epoch 99 2026-02-11T23:30:59.264+0000 7f416c3fd640 20 osd.0 99 get_nextmap_reserved map_reservations: {99=1} 2026-02-11T23:30:59.264+0000 7f416c3fd640 20 osd.0 99 release_map epoch: 99 2026-02-11T23:30:59.264+0000 7f416c3fd640 10 osd.0 99 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 99 is not newer than session's projected_epoch of 103 2026-02-11T23:30:59.265+0000 7f415aefe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 25 ==== osd_map(100..103 src has 1..103) ==== 1076+0+0 (unknown 0 0 0) 0x5954fc0 con 0x507ac00 2026-02-11T23:30:59.265+0000 7f415aefe640 20 osd.0 99 OSD::ms_dispatch: osd_map(100..103 src has 1..103) 2026-02-11T23:30:59.265+0000 7f415aefe640 20 osd.0 99 _dispatch 0x5954fc0 osd_map(100..103 src has 1..103) 2026-02-11T23:30:59.265+0000 7f415aefe640 3 osd.0 99 handle_osd_map epochs [100,103], i have 99, src has [1,103] 2026-02-11T23:30:59.265+0000 7f415aefe640 10 osd.0 99 handle_osd_map: received 0 full maps and 4 incremental maps 2026-02-11T23:30:59.265+0000 7f415aefe640 10 osd.0 99 handle_osd_map got inc map for epoch 100 2026-02-11T23:30:59.265+0000 7f415aefe640 20 osd.0 99 got_full_map 100, nothing requested 2026-02-11T23:30:59.265+0000 7f415aefe640 10 osd.0 99 handle_osd_map got inc map for epoch 101 2026-02-11T23:30:59.265+0000 7f415aefe640 20 osd.0 99 got_full_map 101, nothing requested 2026-02-11T23:30:59.265+0000 7f415aefe640 10 osd.0 99 handle_osd_map got inc map for epoch 102 2026-02-11T23:30:59.265+0000 7f415aefe640 20 osd.0 99 got_full_map 102, nothing requested 2026-02-11T23:30:59.265+0000 7f415aefe640 10 osd.0 99 handle_osd_map got inc map for epoch 103 2026-02-11T23:30:59.265+0000 7f415aefe640 20 osd.0 99 got_full_map 103, nothing requested 2026-02-11T23:30:59.265+0000 7f415aefe640 10 osd.0 99 _track_pools_and_pg_num_changes recording final pg_pool_t for pool 3 2026-02-11T23:30:59.265+0000 7f415aefe640 10 osd.0 99 _track_pools_and_pg_num_changes recording final pg_pool_t for pool 5 2026-02-11T23:30:59.265+0000 7f415aefe640 10 osd.0 99 _track_pools_and_pg_num_changes recording final pg_pool_t for pool 6 2026-02-11T23:30:59.265+0000 7f415aefe640 10 osd.0 99 _track_pools_and_pg_num_changes recording final pg_pool_t for pool 7 2026-02-11T23:30:59.265+0000 7f415aefe640 20 osd.0 99 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:59.265+0000 7f415aefe640 20 osd.0 99 handle_osd_map pg_num_history pg_num_history(e103 pg_nums {1={27=1},2={34=1},3={42=1},4={49=1},5={56=1},6={64=1},7={70=1},8={77=1},9={85=1},10={92=1}} deleted_pools 99,2,100,3,101,5,102,6,103,7) 2026-02-11T23:30:59.265+0000 7f415aefe640 10 snap_mapper record_purged_snaps purged_snaps {100={},101={},102={},103={}} 2026-02-11T23:30:59.265+0000 7f415aefe640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:59.265+0000 7f415aefe640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e103 maps [1~103] lci=[5,103] tlb=1) 2026-02-11T23:30:59.265+0000 7f41591a6640 10 osd.0 99 _committed_osd_maps 100..103 2026-02-11T23:30:59.265+0000 7f41591a6640 10 osd.0 99 advance to epoch 100 (<= last 103 <= newest_map 103) 2026-02-11T23:30:59.265+0000 7f41591a6640 10 osd.0 100 advance to epoch 101 (<= last 103 <= newest_map 103) 2026-02-11T23:30:59.265+0000 7f41591a6640 10 osd.0 101 advance to epoch 102 (<= last 103 <= newest_map 103) 2026-02-11T23:30:59.265+0000 7f41591a6640 10 osd.0 102 advance to epoch 103 (<= last 103 <= newest_map 103) 2026-02-11T23:30:59.265+0000 7f41591a6640 20 osd.0 103 consume_map version 103 2026-02-11T23:30:59.265+0000 7f41591a6640 20 osd.0 99 await_reserved_maps epoch:103 2026-02-11T23:30:59.265+0000 7f41591a6640 20 osd.0 99 await_reserved_maps done 0 2026-02-11T23:30:59.265+0000 7f41591a6640 20 osd.0 103 consume_map 103 -- publish done 2026-02-11T23:30:59.265+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:30:59.265+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x37582c0 2026-02-11T23:30:59.265+0000 7f41591a6640 20 osd.0 103 identify_splits_and_merges 4.0 e99 to e103 2026-02-11T23:30:59.265+0000 7f41591a6640 20 osd.0 103 identify_splits_and_merges 4.0 e99 to e103 pg_nums {49=1} 2026-02-11T23:30:59.265+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3700dc0 2026-02-11T23:30:59.265+0000 7f41591a6640 20 osd.0 103 identify_splits_and_merges 1.0 e99 to e103 2026-02-11T23:30:59.265+0000 7f41591a6640 20 osd.0 103 identify_splits_and_merges 1.0 e99 to e103 pg_nums {27=1} 2026-02-11T23:30:59.265+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:59.265+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:59.265+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:59.265+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:59.265+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:59.265+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:59.266+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:59.266+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:59.266+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:59.266+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:59.266+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:59.266+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:59.266+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:59.266+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:59.266+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:59.266+0000 7f41591a6640 20 osd.0 103 prune_pg_created 2026-02-11T23:30:59.266+0000 7f41591a6640 10 osd.0:0.consume_map 103 (was 99) 2026-02-11T23:30:59.266+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:59.266+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:59.266+0000 7f41591a6640 10 osd.0:1.consume_map 103 (was 99) 2026-02-11T23:30:59.266+0000 7f41591a6640 10 osd.0:2.consume_map 103 (was 99) 2026-02-11T23:30:59.266+0000 7f41591a6640 10 osd.0:3.consume_map 103 (was 99) 2026-02-11T23:30:59.266+0000 7f41591a6640 10 osd.0:4.consume_map 103 (was 99) 2026-02-11T23:30:59.266+0000 7f41591a6640 10 osd.0:5.consume_map 103 (was 99) 2026-02-11T23:30:59.266+0000 7f41591a6640 10 osd.0:6.consume_map 103 (was 99) 2026-02-11T23:30:59.266+0000 7f41591a6640 10 osd.0:7.consume_map 103 (was 99) 2026-02-11T23:30:59.266+0000 7f41591a6640 10 osd.0 103 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:59.266+0000 7f41591a6640 15 osd.0 103 enqueue_peering_evt 4.0 epoch_sent: 103 epoch_requested: 103 NullEvt 2026-02-11T23:30:59.266+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 103 epoch_requested: 103 NullEvt) class_id 2 prio 255 cost 10 e103) 2026-02-11T23:30:59.266+0000 7f41591a6640 15 osd.0 103 enqueue_peering_evt 1.0 epoch_sent: 103 epoch_requested: 103 NullEvt 2026-02-11T23:30:59.266+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 103 epoch_requested: 103 NullEvt) class_id 2 prio 255 cost 10 e103) 2026-02-11T23:30:59.266+0000 7f41591a6640 7 osd.0 103 activate_map version 103 2026-02-11T23:30:59.266+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:59.266+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 103 epoch_requested: 103 NullEvt) class_id 2immediate prio 255 cost 10 e103) queued 2026-02-11T23:30:59.266+0000 7f4150194640 20 osd.0 103 agent_entry empty queue 2026-02-11T23:30:59.266+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:59.266+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 103 epoch_requested: 103 NullEvt) class_id 2immediate prio 255 cost 10 e103) pg 0x5978000 2026-02-11T23:30:59.266+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:59.266+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 103 epoch_requested: 103 NullEvt) class_id 2immediate prio 255 cost 10 e103) queued 2026-02-11T23:30:59.266+0000 7f414b98b640 10 osd.0 pg_epoch: 99 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 100 2026-02-11T23:30:59.266+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:59.266+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 103 epoch_requested: 103 NullEvt) class_id 2immediate prio 255 cost 10 e103) pg 0x5946000 2026-02-11T23:30:59.266+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 4.0 99 -> 100 2026-02-11T23:30:59.266+0000 7f414b98b640 10 osd.0 pg_epoch: 99 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:59.266+0000 7f414f993640 10 osd.0 pg_epoch: 99 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 100 2026-02-11T23:30:59.266+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 99 -> 100 2026-02-11T23:30:59.266+0000 7f414f993640 10 osd.0 pg_epoch: 99 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:59.266+0000 7f414b98b640 10 osd.0 pg_epoch: 100 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:59.266+0000 7f414f993640 10 osd.0 pg_epoch: 100 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:59.266+0000 7f414b98b640 10 osd.0 pg_epoch: 100 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:59.266+0000 7f414f993640 10 osd.0 pg_epoch: 100 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:59.266+0000 7f414f993640 20 osd.0 103 advance_pg new pool opts old pool opts 2026-02-11T23:30:59.266+0000 7f414b98b640 20 osd.0 103 advance_pg new pool opts old pool opts 2026-02-11T23:30:59.266+0000 7f414f993640 10 osd.0 pg_epoch: 100 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 101 2026-02-11T23:30:59.266+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 100 -> 101 2026-02-11T23:30:59.266+0000 7f414b98b640 10 osd.0 pg_epoch: 100 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 101 2026-02-11T23:30:59.266+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 4.0 100 -> 101 2026-02-11T23:30:59.266+0000 7f414f993640 10 osd.0 pg_epoch: 100 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:59.266+0000 7f414f993640 10 osd.0 pg_epoch: 101 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:59.266+0000 7f414b98b640 10 osd.0 pg_epoch: 100 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:59.266+0000 7f414f993640 10 osd.0 pg_epoch: 101 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:59.266+0000 7f414b98b640 10 osd.0 pg_epoch: 101 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:59.266+0000 7f414f993640 20 osd.0 103 advance_pg new pool opts old pool opts 2026-02-11T23:30:59.266+0000 7f414b98b640 10 osd.0 pg_epoch: 101 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:59.266+0000 7f414b98b640 20 osd.0 103 advance_pg new pool opts old pool opts 2026-02-11T23:30:59.266+0000 7f414f993640 10 osd.0 pg_epoch: 101 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 102 2026-02-11T23:30:59.266+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 101 -> 102 2026-02-11T23:30:59.266+0000 7f414b98b640 10 osd.0 pg_epoch: 101 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 102 2026-02-11T23:30:59.266+0000 7f414f993640 10 osd.0 pg_epoch: 101 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:59.266+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 4.0 101 -> 102 2026-02-11T23:30:59.266+0000 7f414f993640 10 osd.0 pg_epoch: 102 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:59.266+0000 7f414b98b640 10 osd.0 pg_epoch: 101 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:59.266+0000 7f414f993640 10 osd.0 pg_epoch: 102 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:59.266+0000 7f414b98b640 10 osd.0 pg_epoch: 102 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:59.266+0000 7f414f993640 20 osd.0 103 advance_pg new pool opts old pool opts 2026-02-11T23:30:59.266+0000 7f414b98b640 10 osd.0 pg_epoch: 102 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:59.266+0000 7f414b98b640 20 osd.0 103 advance_pg new pool opts old pool opts 2026-02-11T23:30:59.266+0000 7f414f993640 10 osd.0 pg_epoch: 102 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 103 2026-02-11T23:30:59.266+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 102 -> 103 2026-02-11T23:30:59.266+0000 7f414b98b640 10 osd.0 pg_epoch: 102 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 103 2026-02-11T23:30:59.266+0000 7f414f993640 10 osd.0 pg_epoch: 102 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:59.266+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 4.0 102 -> 103 2026-02-11T23:30:59.266+0000 7f414b98b640 10 osd.0 pg_epoch: 102 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:59.266+0000 7f414f993640 10 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:59.266+0000 7f414b98b640 10 osd.0 pg_epoch: 103 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:59.266+0000 7f414b98b640 10 osd.0 pg_epoch: 103 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:59.266+0000 7f414b98b640 20 osd.0 103 advance_pg new pool opts old pool opts 2026-02-11T23:30:59.266+0000 7f414f993640 10 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:59.266+0000 7f414f993640 20 osd.0 103 advance_pg new pool opts old pool opts 2026-02-11T23:30:59.266+0000 7f414f993640 10 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_activate_map: epoch range: 100..103 2026-02-11T23:30:59.266+0000 7f414b98b640 10 osd.0 pg_epoch: 103 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_activate_map: epoch range: 100..103 2026-02-11T23:30:59.266+0000 7f414f993640 10 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:59.266+0000 7f414b98b640 10 osd.0 pg_epoch: 103 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:59.266+0000 7f414f993640 20 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 73 while current is 103 2026-02-11T23:30:59.266+0000 7f414b98b640 20 osd.0 pg_epoch: 103 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 95 while current is 103 2026-02-11T23:30:59.266+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:59.266+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:59.266+0000 7f414f993640 10 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 103 epoch_requested: 103 NullEvt 2026-02-11T23:30:59.266+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:59.266+0000 7f414b98b640 10 osd.0 pg_epoch: 103 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 103 epoch_requested: 103 NullEvt 2026-02-11T23:30:59.266+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:59.266+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:59.266+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:59.267+0000 7f4162bea640 10 osd.0 103 tick_without_osd_lock 2026-02-11T23:30:59.267+0000 7f4162bea640 0 osd.0 103 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:59.267+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:59.267+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.341 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:59.267+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:59.268, recovery is active?:false restrictions:<....> 2026-02-11T23:30:59.267+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:59.267+0000 7f4162bea640 10 osd.0 103 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:59.267+0000 7f4162bea640 20 osd.0 103 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:59.267+0000 7f4162bea640 10 osd.0 103 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:59.267+0000 7f4162bea640 20 osd.0 103 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:59.327+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:30:59.327+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:30:59.327+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:29.328886+0000) 2026-02-11T23:30:59.726+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 48 ==== osd_map(104..105 src has 1..105) ==== 556+0+0 (crc 0 0 0) 0x5954380 con 0x4acbc00 2026-02-11T23:30:59.726+0000 7f415cf02640 20 osd.0 103 OSD::ms_dispatch: osd_map(104..105 src has 1..105) 2026-02-11T23:30:59.726+0000 7f415cf02640 20 osd.0 103 _dispatch 0x5954380 osd_map(104..105 src has 1..105) 2026-02-11T23:30:59.726+0000 7f415cf02640 3 osd.0 103 handle_osd_map epochs [104,105], i have 103, src has [1,105] 2026-02-11T23:30:59.726+0000 7f415cf02640 10 osd.0 103 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:59.726+0000 7f415cf02640 10 osd.0 103 handle_osd_map got inc map for epoch 104 2026-02-11T23:30:59.726+0000 7f415cf02640 20 osd.0 103 got_full_map 104, nothing requested 2026-02-11T23:30:59.726+0000 7f415cf02640 10 osd.0 103 handle_osd_map got inc map for epoch 105 2026-02-11T23:30:59.726+0000 7f415cf02640 20 osd.0 103 got_full_map 105, nothing requested 2026-02-11T23:30:59.727+0000 7f415cf02640 10 osd.0 103 _track_pools_and_pg_num_changes recording final pg_pool_t for pool 8 2026-02-11T23:30:59.727+0000 7f415cf02640 10 osd.0 103 _track_pools_and_pg_num_changes recording final pg_pool_t for pool 9 2026-02-11T23:30:59.727+0000 7f415cf02640 20 osd.0 103 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:59.727+0000 7f415cf02640 20 osd.0 103 handle_osd_map pg_num_history pg_num_history(e105 pg_nums {1={27=1},2={34=1},3={42=1},4={49=1},5={56=1},6={64=1},7={70=1},8={77=1},9={85=1},10={92=1}} deleted_pools 99,2,100,3,101,5,102,6,103,7,104,8,105,9) 2026-02-11T23:30:59.727+0000 7f415cf02640 10 snap_mapper record_purged_snaps purged_snaps {104={},105={}} 2026-02-11T23:30:59.727+0000 7f415cf02640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:59.727+0000 7f415cf02640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e105 maps [1~105] lci=[5,105] tlb=1) 2026-02-11T23:30:59.727+0000 7f41591a6640 10 osd.0 103 _committed_osd_maps 104..105 2026-02-11T23:30:59.727+0000 7f41591a6640 10 osd.0 103 advance to epoch 104 (<= last 105 <= newest_map 105) 2026-02-11T23:30:59.727+0000 7f41591a6640 10 osd.0 104 advance to epoch 105 (<= last 105 <= newest_map 105) 2026-02-11T23:30:59.727+0000 7f41591a6640 20 osd.0 105 consume_map version 105 2026-02-11T23:30:59.727+0000 7f41591a6640 20 osd.0 103 await_reserved_maps epoch:105 2026-02-11T23:30:59.727+0000 7f41591a6640 20 osd.0 103 await_reserved_maps done 0 2026-02-11T23:30:59.727+0000 7f41591a6640 20 osd.0 105 consume_map 105 -- publish done 2026-02-11T23:30:59.727+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:30:59.727+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x37582c0 2026-02-11T23:30:59.727+0000 7f41591a6640 20 osd.0 105 identify_splits_and_merges 4.0 e103 to e105 2026-02-11T23:30:59.727+0000 7f41591a6640 20 osd.0 105 identify_splits_and_merges 4.0 e103 to e105 pg_nums {49=1} 2026-02-11T23:30:59.727+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3700dc0 2026-02-11T23:30:59.727+0000 7f41591a6640 20 osd.0 105 identify_splits_and_merges 1.0 e103 to e105 2026-02-11T23:30:59.727+0000 7f41591a6640 20 osd.0 105 identify_splits_and_merges 1.0 e103 to e105 pg_nums {27=1} 2026-02-11T23:30:59.727+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:59.727+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:59.727+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:59.727+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:59.727+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:59.727+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:59.727+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:59.727+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:59.727+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:59.727+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:59.727+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:59.727+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:59.727+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:59.727+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:59.727+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:59.727+0000 7f41591a6640 20 osd.0 105 prune_pg_created 2026-02-11T23:30:59.727+0000 7f41591a6640 10 osd.0:0.consume_map 105 (was 103) 2026-02-11T23:30:59.727+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:59.727+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:59.727+0000 7f41591a6640 10 osd.0:1.consume_map 105 (was 103) 2026-02-11T23:30:59.727+0000 7f41591a6640 10 osd.0:2.consume_map 105 (was 103) 2026-02-11T23:30:59.727+0000 7f41591a6640 10 osd.0:3.consume_map 105 (was 103) 2026-02-11T23:30:59.727+0000 7f41591a6640 10 osd.0:4.consume_map 105 (was 103) 2026-02-11T23:30:59.727+0000 7f41591a6640 10 osd.0:5.consume_map 105 (was 103) 2026-02-11T23:30:59.727+0000 7f41591a6640 10 osd.0:6.consume_map 105 (was 103) 2026-02-11T23:30:59.727+0000 7f41591a6640 10 osd.0:7.consume_map 105 (was 103) 2026-02-11T23:30:59.727+0000 7f41591a6640 10 osd.0 105 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:59.727+0000 7f41591a6640 15 osd.0 105 enqueue_peering_evt 4.0 epoch_sent: 105 epoch_requested: 105 NullEvt 2026-02-11T23:30:59.727+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 105 epoch_requested: 105 NullEvt) class_id 2 prio 255 cost 10 e105) 2026-02-11T23:30:59.727+0000 7f41591a6640 15 osd.0 105 enqueue_peering_evt 1.0 epoch_sent: 105 epoch_requested: 105 NullEvt 2026-02-11T23:30:59.727+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:59.727+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 105 epoch_requested: 105 NullEvt) class_id 2 prio 255 cost 10 e105) 2026-02-11T23:30:59.727+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 105 epoch_requested: 105 NullEvt) class_id 2immediate prio 255 cost 10 e105) queued 2026-02-11T23:30:59.727+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:59.727+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 105 epoch_requested: 105 NullEvt) class_id 2immediate prio 255 cost 10 e105) pg 0x5978000 2026-02-11T23:30:59.727+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:59.727+0000 7f414b98b640 10 osd.0 pg_epoch: 103 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 104 2026-02-11T23:30:59.727+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 4.0 103 -> 104 2026-02-11T23:30:59.727+0000 7f414b98b640 10 osd.0 pg_epoch: 103 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:59.727+0000 7f414b98b640 10 osd.0 pg_epoch: 104 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:59.727+0000 7f41591a6640 7 osd.0 105 activate_map version 105 2026-02-11T23:30:59.727+0000 7f414b98b640 10 osd.0 pg_epoch: 104 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:59.727+0000 7f414b98b640 20 osd.0 105 advance_pg new pool opts old pool opts 2026-02-11T23:30:59.727+0000 7f414b98b640 10 osd.0 pg_epoch: 104 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 105 2026-02-11T23:30:59.727+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:59.727+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 105 epoch_requested: 105 NullEvt) class_id 2immediate prio 255 cost 10 e105) queued 2026-02-11T23:30:59.727+0000 7f4150194640 20 osd.0 105 agent_entry empty queue 2026-02-11T23:30:59.727+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:59.727+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 105 epoch_requested: 105 NullEvt) class_id 2immediate prio 255 cost 10 e105) pg 0x5946000 2026-02-11T23:30:59.727+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 4.0 104 -> 105 2026-02-11T23:30:59.727+0000 7f414b98b640 10 osd.0 pg_epoch: 104 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:59.727+0000 7f414f993640 10 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 104 2026-02-11T23:30:59.727+0000 7f414b98b640 10 osd.0 pg_epoch: 105 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:59.727+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 103 -> 104 2026-02-11T23:30:59.727+0000 7f414b98b640 10 osd.0 pg_epoch: 105 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:59.727+0000 7f414b98b640 20 osd.0 105 advance_pg new pool opts old pool opts 2026-02-11T23:30:59.727+0000 7f414f993640 10 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:59.727+0000 7f414b98b640 10 osd.0 pg_epoch: 105 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_activate_map: epoch range: 104..105 2026-02-11T23:30:59.727+0000 7f414b98b640 10 osd.0 pg_epoch: 105 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:59.727+0000 7f414f993640 10 osd.0 pg_epoch: 104 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:59.727+0000 7f414b98b640 20 osd.0 pg_epoch: 105 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 95 while current is 105 2026-02-11T23:30:59.727+0000 7f414f993640 10 osd.0 pg_epoch: 104 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:59.727+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:59.727+0000 7f414b98b640 10 osd.0 pg_epoch: 105 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 105 epoch_requested: 105 NullEvt 2026-02-11T23:30:59.727+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:59.727+0000 7f414f993640 20 osd.0 105 advance_pg new pool opts old pool opts 2026-02-11T23:30:59.727+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:59.727+0000 7f414f993640 10 osd.0 pg_epoch: 104 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 105 2026-02-11T23:30:59.727+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 104 -> 105 2026-02-11T23:30:59.727+0000 7f414f993640 10 osd.0 pg_epoch: 104 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:59.727+0000 7f414f993640 10 osd.0 pg_epoch: 105 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:59.727+0000 7f414f993640 10 osd.0 pg_epoch: 105 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:59.727+0000 7f414f993640 20 osd.0 105 advance_pg new pool opts old pool opts 2026-02-11T23:30:59.727+0000 7f414f993640 10 osd.0 pg_epoch: 105 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_activate_map: epoch range: 104..105 2026-02-11T23:30:59.727+0000 7f414f993640 10 osd.0 pg_epoch: 105 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:59.727+0000 7f414f993640 20 osd.0 pg_epoch: 105 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 73 while current is 105 2026-02-11T23:30:59.727+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:59.727+0000 7f414f993640 10 osd.0 pg_epoch: 105 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 105 epoch_requested: 105 NullEvt 2026-02-11T23:30:59.727+0000 7f414f993640 10 log is not dirty 2026-02-11T23:30:59.727+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:59.989+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 26 ==== pg_lease(4.0 pg_lease(ru 40.828445435s ub 48.828575134s int 16.000000000s) e104/104) ==== 58+0+0 (unknown 0 0 0) 0x5af8180 con 0x507ac00 2026-02-11T23:30:59.989+0000 7f416abfa640 15 osd.0 105 enqueue_peering_evt 4.0 epoch_sent: 104 epoch_requested: 104 MLease epoch 104 from osd.4 pg_lease(ru 40.828445435s ub 48.828575134s int 16.000000000s) 2026-02-11T23:30:59.989+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 104 epoch_requested: 104 MLease epoch 104 from osd.4 pg_lease(ru 40.828445435s ub 48.828575134s int 16.000000000s)) class_id 2 prio 255 cost 10 e104) 2026-02-11T23:30:59.989+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:59.989+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 104 epoch_requested: 104 MLease epoch 104 from osd.4 pg_lease(ru 40.828445435s ub 48.828575134s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e104) queued 2026-02-11T23:30:59.989+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:59.989+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 104 epoch_requested: 104 MLease epoch 104 from osd.4 pg_lease(ru 40.828445435s ub 48.828575134s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e104) pg 0x5978000 2026-02-11T23:30:59.989+0000 7f414b98b640 10 osd.0 pg_epoch: 105 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 104 epoch_requested: 104 MLease epoch 104 from osd.4 pg_lease(ru 40.828445435s ub 48.828575134s int 16.000000000s) 2026-02-11T23:30:59.989+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:59.989+0000 7f414b98b640 10 osd.0 pg_epoch: 105 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::proc_lease pg_lease(ru 40.828445435s ub 48.828575134s int 16.000000000s) 2026-02-11T23:30:59.989+0000 7f414b98b640 20 osd.0 pg_epoch: 105 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::proc_lease peer_clock_delta_ub -9.836546898s -> ru 50.664989471s 2026-02-11T23:30:59.989+0000 7f414b98b640 20 osd.0 pg_epoch: 105 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::proc_lease readable_until now 50.664989471s 2026-02-11T23:30:59.989+0000 7f414b98b640 20 osd.0 pg_epoch: 105 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::proc_lease peer_clock_delta_lb -9.836755753s -> ruub 58.665332794s 2026-02-11T23:30:59.989+0000 7f414b98b640 20 osd.0 pg_epoch: 105 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::proc_lease readable_until_ub now 58.665332794s 2026-02-11T23:30:59.989+0000 7f414b98b640 20 osd.0 105 get_con_osd_cluster to osd.4 from_epoch 105 2026-02-11T23:30:59.989+0000 7f414b98b640 20 osd.0 105 get_nextmap_reserved map_reservations: {105=1} 2026-02-11T23:30:59.989+0000 7f414b98b640 20 osd.0 105 release_map epoch: 105 2026-02-11T23:30:59.989+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_lease_ack(4.0 pg_lease_ack(ruub 48.828575134s) e105/105) -- 0x3701e40 con 0x507ac00 2026-02-11T23:30:59.989+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:30:59.989+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:59.995+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.2 127.0.0.1:0/2365263801 9 ==== osd_ping(ping e103 up_from 14 ping_stamp 2026-02-11T23:30:59.995882+0000/38.104476929s send_stamp 38.104476929s delta_ub -4.566670895s) ==== 2033+0+0 (crc 0 0 0) 0x4c9b680 con 0x4641000 2026-02-11T23:30:59.995+0000 7f416cbfe640 20 osd.0 105 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566875458s,-4.566670895s]) 2026-02-11T23:30:59.995+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e105 up_from 5 ping_stamp 2026-02-11T23:30:59.995882+0000/38.104476929s send_stamp 42.671352386s delta_ub 4.566875458s) -- 0x4c9ad80 con 0x4641000 2026-02-11T23:30:59.995+0000 7f416cbfe640 20 osd.0 105 get_con_osd_cluster to osd.2 from_epoch 105 2026-02-11T23:30:59.995+0000 7f416cbfe640 20 osd.0 105 get_nextmap_reserved map_reservations: {105=1} 2026-02-11T23:30:59.995+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.2 127.0.0.1:0/2365263801 9 ==== osd_ping(ping e103 up_from 14 ping_stamp 2026-02-11T23:30:59.995882+0000/38.104476929s send_stamp 38.104476929s delta_ub -4.566670895s) ==== 2033+0+0 (crc 0 0 0) 0x5af8480 con 0x48cc000 2026-02-11T23:30:59.995+0000 7f416cbfe640 20 osd.0 105 release_map epoch: 105 2026-02-11T23:30:59.995+0000 7f416cbfe640 10 osd.0 105 maybe_share_map: con v2:127.0.0.1:6820/2365263801 updating session's projected_epoch from 98 to ping map epoch of 103 2026-02-11T23:30:59.995+0000 7f416cbfe640 10 osd.0 105 maybe_share_map: con v2:127.0.0.1:6820/2365263801 map epoch 103 -> 105 (shared) 2026-02-11T23:30:59.995+0000 7f416cbfe640 10 osd.0 105 send_incremental_map epoch range: (104, 105] to v2:127.0.0.1:6820/2365263801 2026-02-11T23:30:59.995+0000 7f416cbfe640 10 osd.0 105 add_map_inc_bl 104 252 bytes 2026-02-11T23:30:59.995+0000 7f416cbfe640 10 osd.0 105 add_map_inc_bl 105 252 bytes 2026-02-11T23:30:59.995+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6820/2365263801,v1:127.0.0.1:6821/2365263801] -- osd_map(104..105 src has 1..105) -- 0x5955a40 con 0x507b800 2026-02-11T23:30:59.995+0000 7f416c3fd640 20 osd.0 105 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.567171097s,-4.566670895s]) 2026-02-11T23:30:59.995+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e105 up_from 5 ping_stamp 2026-02-11T23:30:59.995882+0000/38.104476929s send_stamp 42.671649933s delta_ub 4.567171097s) -- 0x5af8300 con 0x48cc000 2026-02-11T23:30:59.995+0000 7f416c3fd640 20 osd.0 105 get_con_osd_cluster to osd.2 from_epoch 105 2026-02-11T23:30:59.995+0000 7f416c3fd640 20 osd.0 105 get_nextmap_reserved map_reservations: {105=1} 2026-02-11T23:30:59.995+0000 7f416c3fd640 20 osd.0 105 release_map epoch: 105 2026-02-11T23:30:59.995+0000 7f416c3fd640 10 osd.0 105 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 105 is not newer than session's projected_epoch of 105 2026-02-11T23:31:00.011+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 49 ==== osd_map(105..106 src has 1..106) ==== 556+0+0 (crc 0 0 0) 0x5955a40 con 0x4acbc00 2026-02-11T23:31:00.011+0000 7f415cf02640 20 osd.0 105 OSD::ms_dispatch: osd_map(105..106 src has 1..106) 2026-02-11T23:31:00.011+0000 7f415cf02640 20 osd.0 105 _dispatch 0x5955a40 osd_map(105..106 src has 1..106) 2026-02-11T23:31:00.011+0000 7f415cf02640 3 osd.0 105 handle_osd_map epochs [105,106], i have 105, src has [1,106] 2026-02-11T23:31:00.011+0000 7f415cf02640 10 osd.0 105 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:31:00.011+0000 7f415cf02640 10 osd.0 105 handle_osd_map got inc map for epoch 106 2026-02-11T23:31:00.011+0000 7f415cf02640 20 osd.0 105 got_full_map 106, nothing requested 2026-02-11T23:31:00.011+0000 7f415cf02640 10 osd.0 105 _track_pools_and_pg_num_changes recording final pg_pool_t for pool 10 2026-02-11T23:31:00.011+0000 7f415cf02640 20 osd.0 105 trim_maps: min=1 oldest_map=1 2026-02-11T23:31:00.011+0000 7f415cf02640 20 osd.0 105 handle_osd_map pg_num_history pg_num_history(e106 pg_nums {1={27=1},2={34=1},3={42=1},4={49=1},5={56=1},6={64=1},7={70=1},8={77=1},9={85=1},10={92=1}} deleted_pools 99,2,100,3,101,5,102,6,103,7,104,8,105,9,106,10) 2026-02-11T23:31:00.011+0000 7f415cf02640 10 snap_mapper record_purged_snaps purged_snaps {106={}} 2026-02-11T23:31:00.011+0000 7f415cf02640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:31:00.011+0000 7f415cf02640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e106 maps [1~106] lci=[5,106] tlb=1) 2026-02-11T23:31:00.011+0000 7f41591a6640 10 osd.0 105 _committed_osd_maps 106..106 2026-02-11T23:31:00.011+0000 7f41591a6640 10 osd.0 105 advance to epoch 106 (<= last 106 <= newest_map 106) 2026-02-11T23:31:00.011+0000 7f41591a6640 20 osd.0 106 consume_map version 106 2026-02-11T23:31:00.011+0000 7f41591a6640 20 osd.0 105 await_reserved_maps epoch:106 2026-02-11T23:31:00.011+0000 7f41591a6640 20 osd.0 105 await_reserved_maps done 0 2026-02-11T23:31:00.011+0000 7f41591a6640 20 osd.0 106 consume_map 106 -- publish done 2026-02-11T23:31:00.011+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:31:00.011+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x37582c0 2026-02-11T23:31:00.011+0000 7f41591a6640 20 osd.0 106 identify_splits_and_merges 4.0 e105 to e106 2026-02-11T23:31:00.011+0000 7f41591a6640 20 osd.0 106 identify_splits_and_merges 4.0 e105 to e106 pg_nums {49=1} 2026-02-11T23:31:00.011+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3700dc0 2026-02-11T23:31:00.011+0000 7f41591a6640 20 osd.0 106 identify_splits_and_merges 1.0 e105 to e106 2026-02-11T23:31:00.011+0000 7f41591a6640 20 osd.0 106 identify_splits_and_merges 1.0 e105 to e106 pg_nums {27=1} 2026-02-11T23:31:00.011+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:00.011+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:00.011+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:00.011+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:00.011+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:00.011+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:00.011+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:00.011+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:00.011+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:00.011+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:00.011+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:00.011+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:00.011+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:00.011+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:00.011+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:00.011+0000 7f41591a6640 20 osd.0 106 prune_pg_created 2026-02-11T23:31:00.011+0000 7f41591a6640 10 osd.0:0.consume_map 106 (was 105) 2026-02-11T23:31:00.011+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:31:00.011+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:31:00.011+0000 7f41591a6640 10 osd.0:1.consume_map 106 (was 105) 2026-02-11T23:31:00.011+0000 7f41591a6640 10 osd.0:2.consume_map 106 (was 105) 2026-02-11T23:31:00.011+0000 7f41591a6640 10 osd.0:3.consume_map 106 (was 105) 2026-02-11T23:31:00.011+0000 7f41591a6640 10 osd.0:4.consume_map 106 (was 105) 2026-02-11T23:31:00.011+0000 7f41591a6640 10 osd.0:5.consume_map 106 (was 105) 2026-02-11T23:31:00.011+0000 7f41591a6640 10 osd.0:6.consume_map 106 (was 105) 2026-02-11T23:31:00.011+0000 7f41591a6640 10 osd.0:7.consume_map 106 (was 105) 2026-02-11T23:31:00.011+0000 7f41591a6640 10 osd.0 106 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:31:00.011+0000 7f41591a6640 15 osd.0 106 enqueue_peering_evt 4.0 epoch_sent: 106 epoch_requested: 106 NullEvt 2026-02-11T23:31:00.011+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 106 epoch_requested: 106 NullEvt) class_id 2 prio 255 cost 10 e106) 2026-02-11T23:31:00.011+0000 7f41591a6640 15 osd.0 106 enqueue_peering_evt 1.0 epoch_sent: 106 epoch_requested: 106 NullEvt 2026-02-11T23:31:00.011+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 106 epoch_requested: 106 NullEvt) class_id 2 prio 255 cost 10 e106) 2026-02-11T23:31:00.011+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:00.011+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 106 epoch_requested: 106 NullEvt) class_id 2immediate prio 255 cost 10 e106) queued 2026-02-11T23:31:00.011+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:00.011+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 106 epoch_requested: 106 NullEvt) class_id 2immediate prio 255 cost 10 e106) pg 0x5978000 2026-02-11T23:31:00.011+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:00.011+0000 7f414b98b640 10 osd.0 pg_epoch: 105 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 106 2026-02-11T23:31:00.011+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 4.0 105 -> 106 2026-02-11T23:31:00.011+0000 7f414b98b640 10 osd.0 pg_epoch: 105 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:31:00.011+0000 7f414b98b640 10 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:31:00.011+0000 7f41591a6640 7 osd.0 106 activate_map version 106 2026-02-11T23:31:00.011+0000 7f414b98b640 10 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:31:00.011+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:00.011+0000 7f414b98b640 20 osd.0 106 advance_pg new pool opts old pool opts 2026-02-11T23:31:00.011+0000 7f4150194640 20 osd.0 106 agent_entry empty queue 2026-02-11T23:31:00.011+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 106 epoch_requested: 106 NullEvt) class_id 2immediate prio 255 cost 10 e106) queued 2026-02-11T23:31:00.011+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:00.011+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 106 epoch_requested: 106 NullEvt) class_id 2immediate prio 255 cost 10 e106) pg 0x5946000 2026-02-11T23:31:00.011+0000 7f414b98b640 10 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_activate_map: epoch range: 106..106 2026-02-11T23:31:00.011+0000 7f414b98b640 10 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:31:00.011+0000 7f414b98b640 20 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 95 while current is 106 2026-02-11T23:31:00.011+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:00.011+0000 7f414f993640 10 osd.0 pg_epoch: 105 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 106 2026-02-11T23:31:00.011+0000 7f414b98b640 10 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 106 epoch_requested: 106 NullEvt 2026-02-11T23:31:00.011+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 105 -> 106 2026-02-11T23:31:00.011+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:00.011+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:00.011+0000 7f414f993640 10 osd.0 pg_epoch: 105 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:31:00.011+0000 7f414f993640 10 osd.0 pg_epoch: 106 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:31:00.011+0000 7f414f993640 10 osd.0 pg_epoch: 106 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:31:00.011+0000 7f414f993640 20 osd.0 106 advance_pg new pool opts old pool opts 2026-02-11T23:31:00.011+0000 7f414f993640 10 osd.0 pg_epoch: 106 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_activate_map: epoch range: 106..106 2026-02-11T23:31:00.011+0000 7f414f993640 10 osd.0 pg_epoch: 106 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:31:00.011+0000 7f414f993640 20 osd.0 pg_epoch: 106 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 73 while current is 106 2026-02-11T23:31:00.011+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:00.011+0000 7f414f993640 10 osd.0 pg_epoch: 106 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 106 epoch_requested: 106 NullEvt 2026-02-11T23:31:00.011+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:00.011+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:00.230+0000 7f4162bea640 10 osd.0 106 tick_without_osd_lock 2026-02-11T23:31:00.230+0000 7f4162bea640 0 osd.0 106 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:31:00.230+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:00.230+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.341 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:00.230+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:00.232, recovery is active?:false restrictions:<....> 2026-02-11T23:31:00.230+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:00.230+0000 7f4162bea640 10 osd.0 106 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:00.230+0000 7f4162bea640 20 osd.0 106 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:00.230+0000 7f4162bea640 10 osd.0 106 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:00.230+0000 7f4162bea640 20 osd.0 106 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:00.302+0000 7f41633eb640 10 osd.0 106 tick 2026-02-11T23:31:00.302+0000 7f41633eb640 20 osd.0 106 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:00.327+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:00.327+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:00.327+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:30.328926+0000) 2026-02-11T23:31:00.327+0000 7f415bf00640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-11T23:31:00.528+0000 7f4147983640 5 osd.0 106 heartbeat osd_stat(store_statfs(0x384000/0x0/0x384000, data 0x7a075/0x116000, compress 0x0/0x0/0x0, omap 0x9eed, meta 0x1a46113), peers [1,2,3,4] op hist []) 2026-02-11T23:31:00.528+0000 7f4147983640 20 osd.0 106 check_full_status cur ratio 0, physical ratio 0, new state none 2026-02-11T23:31:00.528+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6816/3043114013,v1:127.0.0.1:6817/3043114013] -- osd_ping(ping e106 up_from 5 ping_stamp 2026-02-11T23:31:00.529987+0000/43.205394745s send_stamp 43.205394745s delta_ub 2.011770010s) -- 0x5af8780 con 0x48c4400 2026-02-11T23:31:00.528+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6814/3043114013,v1:127.0.0.1:6815/3043114013] -- osd_ping(ping e106 up_from 5 ping_stamp 2026-02-11T23:31:00.529987+0000/43.205394745s send_stamp 43.205394745s delta_ub 2.011770010s) -- 0x5af8c00 con 0x4640c00 2026-02-11T23:31:00.528+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6824/2365263801,v1:127.0.0.1:6825/2365263801] -- osd_ping(ping e106 up_from 5 ping_stamp 2026-02-11T23:31:00.529987+0000/43.205394745s send_stamp 43.205394745s delta_ub 4.567171097s) -- 0x5af8d80 con 0x48c4000 2026-02-11T23:31:00.528+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6822/2365263801,v1:127.0.0.1:6823/2365263801] -- osd_ping(ping e106 up_from 5 ping_stamp 2026-02-11T23:31:00.529987+0000/43.205394745s send_stamp 43.205394745s delta_ub 4.567171097s) -- 0x5af8600 con 0x48cd000 2026-02-11T23:31:00.528+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6832/2699377081,v1:127.0.0.1:6833/2699377081] -- osd_ping(ping e106 up_from 5 ping_stamp 2026-02-11T23:31:00.529987+0000/43.205394745s send_stamp 43.205394745s delta_ub 7.219594955s) -- 0x5af9080 con 0x48cd400 2026-02-11T23:31:00.528+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6830/2699377081,v1:127.0.0.1:6831/2699377081] -- osd_ping(ping e106 up_from 5 ping_stamp 2026-02-11T23:31:00.529987+0000/43.205394745s send_stamp 43.205394745s delta_ub 7.219594955s) -- 0x5af9200 con 0x48cd800 2026-02-11T23:31:00.528+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6840/3331151400,v1:127.0.0.1:6841/3331151400] -- osd_ping(ping e106 up_from 5 ping_stamp 2026-02-11T23:31:00.529987+0000/43.205394745s send_stamp 43.205394745s delta_ub 9.836755753s) -- 0x5af9380 con 0x507bc00 2026-02-11T23:31:00.528+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6838/3331151400,v1:127.0.0.1:6839/3331151400] -- osd_ping(ping e106 up_from 5 ping_stamp 2026-02-11T23:31:00.529987+0000/43.205394745s send_stamp 43.205394745s delta_ub 9.836755753s) -- 0x4c9a600 con 0x5282000 2026-02-11T23:31:00.528+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6816/3043114013 12 ==== osd_ping(ping_reply e103 up_from 10 ping_stamp 2026-02-11T23:31:00.529987+0000/43.205394745s send_stamp 41.193832397s delta_ub -2.011561155s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a480 con 0x48c4400 2026-02-11T23:31:00.528+0000 7f416abfa640 20 osd.0 106 get_con_osd_cluster to osd.1 from_epoch 106 2026-02-11T23:31:00.528+0000 7f416abfa640 20 osd.0 106 get_nextmap_reserved map_reservations: {106=1} 2026-02-11T23:31:00.528+0000 7f416abfa640 20 osd.0 106 release_map epoch: 106 2026-02-11T23:31:00.528+0000 7f416abfa640 10 osd.0 106 maybe_share_map: con v2:127.0.0.1:6812/3043114013 updating session's projected_epoch from 98 to ping map epoch of 103 2026-02-11T23:31:00.528+0000 7f416abfa640 10 osd.0 106 maybe_share_map: con v2:127.0.0.1:6812/3043114013 map epoch 103 -> 106 (shared) 2026-02-11T23:31:00.528+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6814/3043114013 12 ==== osd_ping(ping_reply e103 up_from 10 ping_stamp 2026-02-11T23:31:00.529987+0000/43.205394745s send_stamp 41.193855286s delta_ub -2.011537552s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a600 con 0x4640c00 2026-02-11T23:31:00.528+0000 7f416abfa640 10 osd.0 106 send_incremental_map epoch range: (104, 106] to v2:127.0.0.1:6812/3043114013 2026-02-11T23:31:00.528+0000 7f416cbfe640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6824/2365263801 11 ==== osd_ping(ping_reply e105 up_from 14 ping_stamp 2026-02-11T23:31:00.529987+0000/43.205394745s send_stamp 38.638732910s delta_ub -4.566663265s) ==== 2033+0+0 (crc 0 0 0) 0x5af8d80 con 0x48c4000 2026-02-11T23:31:00.528+0000 7f416abfa640 10 osd.0 106 add_map_inc_bl 106 252 bytes 2026-02-11T23:31:00.528+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- osd_map(104..106 src has 1..106) -- 0x4a64fc0 con 0x4acb000 2026-02-11T23:31:00.529+0000 7f416abfa640 20 osd.0 106 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011722326s,-2.011561155s]) 2026-02-11T23:31:00.529+0000 7f416c3fd640 20 osd.0 106 get_con_osd_cluster to osd.1 from_epoch 106 2026-02-11T23:31:00.529+0000 7f416c3fd640 20 osd.0 106 get_nextmap_reserved map_reservations: {106=1} 2026-02-11T23:31:00.529+0000 7f416c3fd640 20 osd.0 106 release_map epoch: 106 2026-02-11T23:31:00.529+0000 7f416c3fd640 10 osd.0 106 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 106 is not newer than session's projected_epoch of 106 2026-02-11T23:31:00.529+0000 7f416c3fd640 20 osd.0 106 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011870861s,-2.011537552s]) 2026-02-11T23:31:00.529+0000 7f416cbfe640 20 osd.0 106 get_con_osd_cluster to osd.2 from_epoch 106 2026-02-11T23:31:00.529+0000 7f416cbfe640 20 osd.0 106 get_nextmap_reserved map_reservations: {106=1} 2026-02-11T23:31:00.529+0000 7f416cbfe640 20 osd.0 106 release_map epoch: 106 2026-02-11T23:31:00.529+0000 7f416cbfe640 10 osd.0 106 maybe_share_map: con v2:127.0.0.1:6820/2365263801 map epoch 105 -> 106 (shared) 2026-02-11T23:31:00.529+0000 7f416cbfe640 10 osd.0 106 send_incremental_map epoch range: (106, 106] to v2:127.0.0.1:6820/2365263801 2026-02-11T23:31:00.529+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6820/2365263801,v1:127.0.0.1:6821/2365263801] -- osd_map(106..106 src has 1..106) -- 0x5955c00 con 0x507b800 2026-02-11T23:31:00.529+0000 7f416cbfe640 20 osd.0 106 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.567011356s,-4.566663265s]) 2026-02-11T23:31:00.529+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6822/2365263801 11 ==== osd_ping(ping_reply e105 up_from 14 ping_stamp 2026-02-11T23:31:00.529987+0000/43.205394745s send_stamp 38.638698578s delta_ub -4.566694736s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a480 con 0x48cd000 2026-02-11T23:31:00.529+0000 7f416abfa640 20 osd.0 106 get_con_osd_cluster to osd.2 from_epoch 106 2026-02-11T23:31:00.529+0000 7f416abfa640 20 osd.0 106 get_nextmap_reserved map_reservations: {106=1} 2026-02-11T23:31:00.529+0000 7f416abfa640 20 osd.0 106 release_map epoch: 106 2026-02-11T23:31:00.529+0000 7f416abfa640 10 osd.0 106 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 106 is not newer than session's projected_epoch of 106 2026-02-11T23:31:00.529+0000 7f416abfa640 20 osd.0 106 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.567062855s,-4.566694736s]) 2026-02-11T23:31:00.529+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6838/3331151400 10 ==== osd_ping(ping_reply e107 up_from 23 ping_stamp 2026-02-11T23:31:00.529987+0000/43.205394745s send_stamp 33.369098663s delta_ub -9.836295128s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a600 con 0x5282000 2026-02-11T23:31:00.529+0000 7f416c3fd640 20 osd.0 106 get_con_osd_cluster to osd.4 from_epoch 106 2026-02-11T23:31:00.529+0000 7f416c3fd640 20 osd.0 106 get_nextmap_reserved map_reservations: {106=1} 2026-02-11T23:31:00.529+0000 7f416c3fd640 20 osd.0 106 release_map epoch: 106 2026-02-11T23:31:00.529+0000 7f416c3fd640 10 osd.0 106 maybe_share_map: con v2:127.0.0.1:6836/3331151400 updating session's projected_epoch from 103 to ping map epoch of 107 2026-02-11T23:31:00.529+0000 7f416c3fd640 10 osd.0 106 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 106 is not newer than session's projected_epoch of 107 2026-02-11T23:31:00.529+0000 7f416c3fd640 20 osd.0 106 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836691856s,-9.836295128s]) 2026-02-11T23:31:00.529+0000 7f415aefe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 27 ==== osd_map(107..107 src has 1..107) ==== 618+0+0 (unknown 0 0 0) 0x4a30700 con 0x507ac00 2026-02-11T23:31:00.529+0000 7f415aefe640 20 osd.0 106 OSD::ms_dispatch: osd_map(107..107 src has 1..107) 2026-02-11T23:31:00.529+0000 7f415aefe640 20 osd.0 106 _dispatch 0x4a30700 osd_map(107..107 src has 1..107) 2026-02-11T23:31:00.529+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6840/3331151400 10 ==== osd_ping(ping_reply e107 up_from 23 ping_stamp 2026-02-11T23:31:00.529987+0000/43.205394745s send_stamp 33.368896484s delta_ub -9.836494446s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a480 con 0x507bc00 2026-02-11T23:31:00.529+0000 7f415aefe640 3 osd.0 106 handle_osd_map epochs [107,107], i have 106, src has [1,107] 2026-02-11T23:31:00.529+0000 7f415aefe640 10 osd.0 106 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:31:00.529+0000 7f415aefe640 10 osd.0 106 handle_osd_map got inc map for epoch 107 2026-02-11T23:31:00.529+0000 7f416abfa640 20 osd.0 106 get_con_osd_cluster to osd.4 from_epoch 106 2026-02-11T23:31:00.529+0000 7f416abfa640 20 osd.0 106 get_nextmap_reserved map_reservations: {106=1} 2026-02-11T23:31:00.529+0000 7f416abfa640 20 osd.0 106 release_map epoch: 106 2026-02-11T23:31:00.529+0000 7f416abfa640 10 osd.0 106 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 106 is not newer than session's projected_epoch of 107 2026-02-11T23:31:00.529+0000 7f416abfa640 20 osd.0 106 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836935997s,-9.836494446s]) 2026-02-11T23:31:00.529+0000 7f415aefe640 20 osd.0 106 got_full_map 107, nothing requested 2026-02-11T23:31:00.529+0000 7f415aefe640 20 osd.0 106 trim_maps: min=1 oldest_map=1 2026-02-11T23:31:00.529+0000 7f415aefe640 20 osd.0 106 handle_osd_map pg_num_history pg_num_history(e107 pg_nums {1={27=1},2={34=1},3={42=1},4={49=1},5={56=1},6={64=1},7={70=1},8={77=1},9={85=1},10={92=1}} deleted_pools 99,2,100,3,101,5,102,6,103,7,104,8,105,9,106,10) 2026-02-11T23:31:00.529+0000 7f415aefe640 10 snap_mapper record_purged_snaps purged_snaps {107={}} 2026-02-11T23:31:00.529+0000 7f415aefe640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:31:00.529+0000 7f415aefe640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e107 maps [1~107] lci=[5,107] tlb=1) 2026-02-11T23:31:00.529+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6830/2699377081 11 ==== osd_ping(ping_reply e108 up_from 19 ping_stamp 2026-02-11T23:31:00.529987+0000/43.205394745s send_stamp 35.986061096s delta_ub -7.219334126s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a600 con 0x48cd800 2026-02-11T23:31:00.529+0000 7f416c3fd640 20 osd.0 106 get_con_osd_cluster to osd.3 from_epoch 106 2026-02-11T23:31:00.529+0000 7f416c3fd640 20 osd.0 106 get_nextmap_reserved map_reservations: {106=1} 2026-02-11T23:31:00.529+0000 7f416c3fd640 20 osd.0 106 release_map epoch: 106 2026-02-11T23:31:00.529+0000 7f416c3fd640 10 osd.0 106 maybe_share_map: con v2:127.0.0.1:6828/2699377081 updating session's projected_epoch from 98 to ping map epoch of 108 2026-02-11T23:31:00.529+0000 7f416c3fd640 10 osd.0 106 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 106 is not newer than session's projected_epoch of 108 2026-02-11T23:31:00.529+0000 7f416c3fd640 20 osd.0 106 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219877720s,-7.219334126s]) 2026-02-11T23:31:00.529+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6832/2699377081 11 ==== osd_ping(ping_reply e108 up_from 19 ping_stamp 2026-02-11T23:31:00.529987+0000/43.205394745s send_stamp 35.986438751s delta_ub -7.218953133s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a480 con 0x48cd400 2026-02-11T23:31:00.529+0000 7f416abfa640 20 osd.0 106 get_con_osd_cluster to osd.3 from_epoch 106 2026-02-11T23:31:00.529+0000 7f416abfa640 20 osd.0 106 get_nextmap_reserved map_reservations: {106=1} 2026-02-11T23:31:00.529+0000 7f416abfa640 20 osd.0 106 release_map epoch: 106 2026-02-11T23:31:00.529+0000 7f416abfa640 10 osd.0 106 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 106 is not newer than session's projected_epoch of 108 2026-02-11T23:31:00.529+0000 7f416abfa640 20 osd.0 106 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219534874s,-7.218953133s]) 2026-02-11T23:31:00.529+0000 7f415aefe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.3 v2:127.0.0.1:6828/2699377081 23 ==== osd_map(107..108 src has 1..108) ==== 1200+0+0 (crc 0 0 0) 0x5955c00 con 0x48ccc00 2026-02-11T23:31:00.529+0000 7f415aefe640 20 osd.0 106 OSD::ms_dispatch: osd_map(107..108 src has 1..108) 2026-02-11T23:31:00.529+0000 7f415aefe640 20 osd.0 106 _dispatch 0x5955c00 osd_map(107..108 src has 1..108) 2026-02-11T23:31:00.529+0000 7f415aefe640 3 osd.0 106 handle_osd_map epochs [107,108], i have 107, src has [1,108] 2026-02-11T23:31:00.529+0000 7f415aefe640 10 osd.0 106 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:31:00.529+0000 7f415aefe640 10 osd.0 106 handle_osd_map got inc map for epoch 108 2026-02-11T23:31:00.529+0000 7f415aefe640 20 osd.0 106 got_full_map 108, nothing requested 2026-02-11T23:31:00.529+0000 7f415aefe640 20 osd.0 106 trim_maps: min=1 oldest_map=1 2026-02-11T23:31:00.529+0000 7f415aefe640 20 osd.0 106 handle_osd_map pg_num_history pg_num_history(e108 pg_nums {1={27=1},2={34=1},3={42=1},4={49=1},5={56=1},6={64=1},7={70=1},8={77=1},9={85=1},10={92=1}} deleted_pools 99,2,100,3,101,5,102,6,103,7,104,8,105,9,106,10) 2026-02-11T23:31:00.529+0000 7f415aefe640 10 snap_mapper record_purged_snaps purged_snaps {108={}} 2026-02-11T23:31:00.529+0000 7f415aefe640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:31:00.529+0000 7f415aefe640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e108 maps [1~108] lci=[5,108] tlb=1) 2026-02-11T23:31:00.529+0000 7f41591a6640 10 osd.0 106 _committed_osd_maps 107..107 2026-02-11T23:31:00.529+0000 7f41591a6640 10 osd.0 106 advance to epoch 107 (<= last 107 <= newest_map 108) 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0 107 consume_map version 107 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0 106 await_reserved_maps epoch:107 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0 106 await_reserved_maps done 0 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0 107 consume_map 107 -- publish done 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x37582c0 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0 107 identify_splits_and_merges 4.0 e106 to e107 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0 107 identify_splits_and_merges 4.0 e106 to e107 pg_nums {49=1} 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3700dc0 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0 107 identify_splits_and_merges 1.0 e106 to e107 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0 107 identify_splits_and_merges 1.0 e106 to e107 pg_nums {27=1} 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0 107 prune_pg_created 2026-02-11T23:31:00.529+0000 7f41591a6640 10 osd.0:0.consume_map 107 (was 106) 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:31:00.529+0000 7f41591a6640 10 osd.0:1.consume_map 107 (was 106) 2026-02-11T23:31:00.529+0000 7f41591a6640 10 osd.0:2.consume_map 107 (was 106) 2026-02-11T23:31:00.529+0000 7f41591a6640 10 osd.0:3.consume_map 107 (was 106) 2026-02-11T23:31:00.529+0000 7f41591a6640 10 osd.0:4.consume_map 107 (was 106) 2026-02-11T23:31:00.529+0000 7f41591a6640 10 osd.0:5.consume_map 107 (was 106) 2026-02-11T23:31:00.529+0000 7f41591a6640 10 osd.0:6.consume_map 107 (was 106) 2026-02-11T23:31:00.529+0000 7f41591a6640 10 osd.0:7.consume_map 107 (was 106) 2026-02-11T23:31:00.529+0000 7f41591a6640 10 osd.0 107 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:31:00.529+0000 7f41591a6640 15 osd.0 107 enqueue_peering_evt 4.0 epoch_sent: 107 epoch_requested: 107 NullEvt 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 107 epoch_requested: 107 NullEvt) class_id 2 prio 255 cost 10 e107) 2026-02-11T23:31:00.529+0000 7f41591a6640 15 osd.0 107 enqueue_peering_evt 1.0 epoch_sent: 107 epoch_requested: 107 NullEvt 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 107 epoch_requested: 107 NullEvt) class_id 2 prio 255 cost 10 e107) 2026-02-11T23:31:00.529+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:00.529+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 107 epoch_requested: 107 NullEvt) class_id 2immediate prio 255 cost 10 e107) queued 2026-02-11T23:31:00.529+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:00.529+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 107 epoch_requested: 107 NullEvt) class_id 2immediate prio 255 cost 10 e107) pg 0x5978000 2026-02-11T23:31:00.529+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:00.529+0000 7f414b98b640 10 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 107 2026-02-11T23:31:00.529+0000 7f41591a6640 7 osd.0 107 activate_map version 107 2026-02-11T23:31:00.529+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 4.0 106 -> 107 2026-02-11T23:31:00.529+0000 7f414b98b640 10 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:31:00.529+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:00.529+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 107 epoch_requested: 107 NullEvt) class_id 2immediate prio 255 cost 10 e107) queued 2026-02-11T23:31:00.529+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:00.529+0000 7f414b98b640 10 osd.0 pg_epoch: 107 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:31:00.529+0000 7f4150194640 20 osd.0 107 agent_entry empty queue 2026-02-11T23:31:00.529+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 107 epoch_requested: 107 NullEvt) class_id 2immediate prio 255 cost 10 e107) pg 0x5946000 2026-02-11T23:31:00.529+0000 7f414b98b640 10 osd.0 pg_epoch: 107 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:31:00.529+0000 7f414b98b640 20 osd.0 107 advance_pg new pool opts old pool opts 2026-02-11T23:31:00.529+0000 7f414f993640 10 osd.0 pg_epoch: 106 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] handle_advance_map: 107 2026-02-11T23:31:00.529+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 106 -> 107 2026-02-11T23:31:00.529+0000 7f414b98b640 10 osd.0 pg_epoch: 107 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_activate_map: epoch range: 107..107 2026-02-11T23:31:00.529+0000 7f414f993640 10 osd.0 pg_epoch: 106 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1]/[1] -- 1/1 2026-02-11T23:31:00.529+0000 7f414b98b640 10 osd.0 pg_epoch: 107 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:31:00.529+0000 7f414f993640 10 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:31:00.529+0000 7f414b98b640 20 osd.0 pg_epoch: 107 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 95 while current is 107 2026-02-11T23:31:00.529+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:00.529+0000 7f414f993640 20 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::should_restart_peering new interval newup [1] newacting [1] 2026-02-11T23:31:00.529+0000 7f414b98b640 10 osd.0 pg_epoch: 107 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 107 epoch_requested: 107 NullEvt 2026-02-11T23:31:00.529+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:00.529+0000 7f414f993640 10 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] state: should_restart_peering, transitioning to Reset 2026-02-11T23:31:00.529+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:00.529+0000 7f414f993640 5 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] exit Started/ReplicaActive/RepNotRecovering 30.258088 164 0.000483 2026-02-11T23:31:00.529+0000 7f414f993640 5 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] exit Started/ReplicaActive 30.258092 0 0.000000 2026-02-11T23:31:00.529+0000 7f414f993640 5 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] exit Started 30.358499 0 0.000000 2026-02-11T23:31:00.529+0000 7f414f993640 5 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] enter Reset 2026-02-11T23:31:00.529+0000 7f414f993640 20 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=29 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset set_last_peering_reset 107 2026-02-11T23:31:00.529+0000 7f414f993640 10 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=107 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset Clearing blocked outgoing recovery messages 2026-02-11T23:31:00.529+0000 7f414f993640 10 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=107 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset Not blocking outgoing recovery messages 2026-02-11T23:31:00.529+0000 7f414f993640 10 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=107 crt=0'0 active mbc={}] state: Reset advmap 2026-02-11T23:31:00.529+0000 7f414f993640 20 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=107 crt=0'0 active mbc={}] PeeringState::should_restart_peering new interval newup [1] newacting [1] 2026-02-11T23:31:00.529+0000 7f414f993640 10 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=107 crt=0'0 active mbc={}] state: should restart peering, calling start_peering_interval again 2026-02-11T23:31:00.529+0000 7f414f993640 20 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1,0] r=1 lpr=107 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset set_last_peering_reset 107 2026-02-11T23:31:00.529+0000 7f414f993640 15 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1] r=1 lpr=107 crt=0'0 active mbc={}] clear_stats 2026-02-11T23:31:00.529+0000 7f414f993640 10 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 active mbc={}] PeeringState::start_peering_interval : check_new_interval output: check_new_interval interval(29-106 up [1, 0](1) acting [1, 0](1)) up_thru 90 up_from 10 last_epoch_clean 30 interval(29-106 up [1, 0](1) acting [1, 0](1) maybe_went_rw) : primary up 10-90 includes interval 2026-02-11T23:31:00.529+0000 7f414f993640 10 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=29) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 active mbc={}] PeeringState::start_peering_interval noting past ([29,106] all_participants={0, 1} intervals=[([29,106] acting={0, 1})]) 2026-02-11T23:31:00.529+0000 7f414f993640 20 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 active mbc={}] PeeringState::on_new_interval 2026-02-11T23:31:00.529+0000 7f414f993640 20 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 active mbc={}] PeeringState::on_new_interval upacting_features 0x3f0ffffffffdffff from [1]+[1] 2026-02-11T23:31:00.529+0000 7f414f993640 20 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 active mbc={}] PeeringState::on_new_interval checking missing set deletes flag. missing = missing(0 may_include_deletes = 1) 2026-02-11T23:31:00.529+0000 7f414f993640 10 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 active mbc={}] PeeringState::init_hb_stamps now [] 2026-02-11T23:31:00.529+0000 7f414f993640 10 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742717743s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 active pruub 52.949176788s@ mbc={}] PeeringState::on_new_interval prior_readable_until_ub 52.949176788s (mnow 43.206455231s + 9.742717743s) 2026-02-11T23:31:00.529+0000 7f414f993640 10 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742717743s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 active pruub 52.949176788s@ mbc={}] cancel_recovery 2026-02-11T23:31:00.529+0000 7f414f993640 10 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742717743s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 active pruub 52.949176788s@ mbc={}] clear_recovery_state 2026-02-11T23:31:00.529+0000 7f414f993640 15 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742717743s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 active pruub 52.949176788s@ mbc={}] _clear_recovery_state 2026-02-11T23:31:00.529+0000 7f414f993640 10 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742717743s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 active pruub 52.949176788s@ mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:31:00.529+0000 7f414f993640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:31:00.529+0000 7f414f993640 20 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742717743s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 active pruub 52.949176788s@ mbc={}] scrubber: FSM: ReplicaActive::exit() 2026-02-11T23:31:00.529+0000 7f414f993640 10 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742717743s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 active pruub 52.949176788s@ mbc={}] scrubber: FSM: ReplicaActive::clear_remote_reservation(): pending_reservation_nonce 0, reservation_granted false 2026-02-11T23:31:00.529+0000 7f414f993640 10 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742717743s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 active pruub 52.949176788s@ mbc={}] scrubber: FSM: -- state -->> NotActive 2026-02-11T23:31:00.529+0000 7f414f993640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:31:00.529+0000 7f414f993640 1 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742717743s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 active pruub 52.949176788s@ mbc={}] PeeringState::start_peering_interval up [1,0] -> [1], acting [1,0] -> [1], acting_primary 1 -> 1, up_primary 1 -> 1, role 1 -> -1, features acting 4544132024016699391 upacting 4544132024016699391 2026-02-11T23:31:00.529+0000 7f414f993640 10 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742717743s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown pruub 52.949176788s@ mbc={}] PeeringState::clear_primary_state clear_primary_state 2026-02-11T23:31:00.529+0000 7f414f993640 20 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742717743s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown pruub 52.949176788s@ mbc={}] clear_primary_state 2026-02-11T23:31:00.529+0000 7f414f993640 10 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742717743s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown pruub 52.949176788s@ mbc={}] release_backoffs [1:00000000::::0,MAX) 2026-02-11T23:31:00.529+0000 7f414f993640 10 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742717743s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown pruub 52.949176788s@ mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:31:00.529+0000 7f414f993640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:31:00.529+0000 7f414f993640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:31:00.529+0000 7f414f993640 20 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742717743s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown pruub 52.949176788s@ mbc={}] agent_stop 2026-02-11T23:31:00.529+0000 7f414f993640 10 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742717743s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown pruub 52.949176788s@ mbc={}] on_change 2026-02-11T23:31:00.529+0000 7f414f993640 10 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742717743s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown pruub 52.949176788s@ mbc={}] cancel_copy_ops 2026-02-11T23:31:00.529+0000 7f414f993640 10 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742717743s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown pruub 52.949176788s@ mbc={}] cancel_flush_ops 2026-02-11T23:31:00.529+0000 7f414f993640 10 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742717743s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown pruub 52.949176788s@ mbc={}] cancel_proxy_ops 2026-02-11T23:31:00.529+0000 7f414f993640 10 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742717743s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown pruub 52.949176788s@ mbc={}] cancel_manifest_ops 2026-02-11T23:31:00.529+0000 7f414f993640 10 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742717743s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown pruub 52.949176788s@ mbc={}] cancel_cls_gather_ops 2026-02-11T23:31:00.529+0000 7f414f993640 10 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742717743s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown pruub 52.949176788s@ mbc={}] on_change_cleanup 2026-02-11T23:31:00.529+0000 7f414f993640 10 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742717743s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown pruub 52.949176788s@ mbc={}] on_change 2026-02-11T23:31:00.529+0000 7f414f993640 20 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742717743s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown pruub 52.949176788s@ mbc={}] exit NotTrimming 2026-02-11T23:31:00.529+0000 7f414f993640 20 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742717743s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown pruub 52.949176788s@ mbc={}] enter NotTrimming 2026-02-11T23:31:00.529+0000 7f414f993640 10 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742717743s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] plpg_on_role_change 2026-02-11T23:31:00.529+0000 7f414f993640 10 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742717743s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:31:00.529+0000 7f414f993640 10 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742717743s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] plpg_on_pool_change 2026-02-11T23:31:00.529+0000 7f414f993640 20 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742717743s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] hit_set_clear 2026-02-11T23:31:00.529+0000 7f414f993640 20 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742717743s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] agent_stop 2026-02-11T23:31:00.529+0000 7f414f993640 20 osd.0 107 advance_pg new pool opts old pool opts 2026-02-11T23:31:00.529+0000 7f414f993640 10 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742717743s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] handle_activate_map: epoch range: 107..107 2026-02-11T23:31:00.529+0000 7f414f993640 10 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742717743s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] PeeringState::activate_map 2026-02-11T23:31:00.529+0000 7f414f993640 5 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742657661s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] exit Reset 0.000098 1 0.000099 2026-02-11T23:31:00.529+0000 7f41591a6640 10 osd.0 107 _committed_osd_maps 108..108 2026-02-11T23:31:00.529+0000 7f41591a6640 10 osd.0 107 advance to epoch 108 (<= last 108 <= newest_map 108) 2026-02-11T23:31:00.529+0000 7f414f993640 5 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742657661s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] enter Started 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0 108 consume_map version 108 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0 107 await_reserved_maps epoch:108 2026-02-11T23:31:00.529+0000 7f414f993640 5 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742657661s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] enter Start 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0 107 await_reserved_maps done 0 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0 108 consume_map 108 -- publish done 2026-02-11T23:31:00.529+0000 7f414f993640 1 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742657661s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] state: transitioning to Stray 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:31:00.529+0000 7f414f993640 5 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742657661s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] exit Start 0.000006 0 0.000000 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x37582c0 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0 108 identify_splits_and_merges 4.0 e107 to e108 2026-02-11T23:31:00.529+0000 7f414f993640 5 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742657661s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] enter Started/Stray 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0 108 identify_splits_and_merges 4.0 e107 to e108 pg_nums {49=1} 2026-02-11T23:31:00.529+0000 7f414f993640 20 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742657661s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 73 while current is 107 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3700dc0 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0 108 identify_splits_and_merges 1.0 e107 to e108 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0 108 identify_splits_and_merges 1.0 e107 to e108 pg_nums {27=1} 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:00.529+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:00.529+0000 7f414f993640 10 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742657661s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0 108 prune_pg_created 2026-02-11T23:31:00.529+0000 7f414f993640 10 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742657661s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] do_peering_event: epoch_sent: 107 epoch_requested: 107 NullEvt 2026-02-11T23:31:00.529+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:00.529+0000 7f414f993640 20 osd.0 108 get_con_osd_cluster to osd.1 from_epoch 107 2026-02-11T23:31:00.529+0000 7f41591a6640 10 osd.0:0.consume_map 108 (was 107) 2026-02-11T23:31:00.529+0000 7f414f993640 20 osd.0 108 get_nextmap_reserved map_reservations: {108=1} 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:31:00.529+0000 7f414f993640 20 osd.0 108 release_map epoch: 108 2026-02-11T23:31:00.529+0000 7f41591a6640 10 osd.0:1.consume_map 108 (was 107) 2026-02-11T23:31:00.529+0000 7f41591a6640 10 osd.0:2.consume_map 108 (was 107) 2026-02-11T23:31:00.529+0000 7f414f993640 10 osd.0 108 maybe_share_map: con v2:127.0.0.1:6812/3043114013 map epoch 106 -> 107 (shared) 2026-02-11T23:31:00.529+0000 7f41591a6640 10 osd.0:3.consume_map 108 (was 107) 2026-02-11T23:31:00.529+0000 7f414f993640 10 osd.0 108 send_incremental_map epoch range: (107, 107] to v2:127.0.0.1:6812/3043114013 2026-02-11T23:31:00.529+0000 7f41591a6640 10 osd.0:4.consume_map 108 (was 107) 2026-02-11T23:31:00.529+0000 7f41591a6640 10 osd.0:5.consume_map 108 (was 107) 2026-02-11T23:31:00.529+0000 7f41591a6640 10 osd.0:6.consume_map 108 (was 107) 2026-02-11T23:31:00.529+0000 7f41591a6640 10 osd.0:7.consume_map 108 (was 107) 2026-02-11T23:31:00.529+0000 7f41591a6640 10 osd.0 108 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:31:00.529+0000 7f41591a6640 15 osd.0 108 enqueue_peering_evt 4.0 epoch_sent: 108 epoch_requested: 108 NullEvt 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 108 epoch_requested: 108 NullEvt) class_id 2 prio 255 cost 10 e108) 2026-02-11T23:31:00.529+0000 7f41591a6640 15 osd.0 108 enqueue_peering_evt 1.0 epoch_sent: 108 epoch_requested: 108 NullEvt 2026-02-11T23:31:00.529+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 108 epoch_requested: 108 NullEvt) class_id 2 prio 255 cost 10 e108) 2026-02-11T23:31:00.529+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:00.529+0000 7f41591a6640 7 osd.0 108 activate_map version 108 2026-02-11T23:31:00.529+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 108 epoch_requested: 108 NullEvt) class_id 2immediate prio 255 cost 10 e108) queued 2026-02-11T23:31:00.529+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:00.529+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 108 epoch_requested: 108 NullEvt) class_id 2immediate prio 255 cost 10 e108) pg 0x5978000 2026-02-11T23:31:00.529+0000 7f4150194640 20 osd.0 108 agent_entry empty queue 2026-02-11T23:31:00.529+0000 7f414f993640 10 osd.0 108 add_map_inc_bl 107 574 bytes 2026-02-11T23:31:00.529+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- osd_map(107..107 src has 1..108) -- 0x59541c0 con 0x4acb000 2026-02-11T23:31:00.529+0000 7f414b98b640 10 osd.0 pg_epoch: 107 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 108 2026-02-11T23:31:00.529+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 4.0 107 -> 108 2026-02-11T23:31:00.529+0000 7f414b98b640 10 osd.0 pg_epoch: 107 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:31:00.529+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_notify2(1.0 1.0 (query:107 sent:107 1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742657661s) ([29,106] all_participants={0, 1} intervals=[([29,106] acting={0, 1})])) e107/107) -- 0x5b22700 con 0x4acb000 2026-02-11T23:31:00.529+0000 7f414b98b640 10 osd.0 pg_epoch: 108 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:31:00.529+0000 7f414b98b640 10 osd.0 pg_epoch: 108 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:31:00.529+0000 7f414b98b640 20 osd.0 108 advance_pg new pool opts old pool opts 2026-02-11T23:31:00.529+0000 7f414b98b640 10 osd.0 pg_epoch: 108 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_activate_map: epoch range: 108..108 2026-02-11T23:31:00.529+0000 7f414b98b640 10 osd.0 pg_epoch: 108 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:31:00.529+0000 7f414b98b640 20 osd.0 pg_epoch: 108 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 95 while current is 108 2026-02-11T23:31:00.529+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:00.529+0000 7f414b98b640 10 osd.0 pg_epoch: 108 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 108 epoch_requested: 108 NullEvt 2026-02-11T23:31:00.529+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:00.529+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:00.529+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 108 epoch_requested: 108 NullEvt) class_id 2immediate prio 255 cost 10 e108) queued 2026-02-11T23:31:00.529+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:00.529+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 108 epoch_requested: 108 NullEvt) class_id 2immediate prio 255 cost 10 e108) pg 0x5946000 2026-02-11T23:31:00.529+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:00.529+0000 7f414b98b640 10 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742657661s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] handle_advance_map: 108 2026-02-11T23:31:00.529+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 1.0 107 -> 108 2026-02-11T23:31:00.529+0000 7f414b98b640 10 osd.0 pg_epoch: 107 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742657661s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] PeeringState::advance_map handle_advance_map [1]/[1] -- 1/1 2026-02-11T23:31:00.529+0000 7f414b98b640 10 osd.0 pg_epoch: 108 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742657661s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] state: Started advmap 2026-02-11T23:31:00.529+0000 7f414b98b640 10 osd.0 pg_epoch: 108 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742657661s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:31:00.529+0000 7f414b98b640 10 osd.0 pg_epoch: 108 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742657661s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] plpg_on_pool_change 2026-02-11T23:31:00.529+0000 7f414b98b640 20 osd.0 pg_epoch: 108 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742657661s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] hit_set_clear 2026-02-11T23:31:00.529+0000 7f414b98b640 20 osd.0 pg_epoch: 108 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742657661s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] agent_stop 2026-02-11T23:31:00.529+0000 7f414b98b640 20 osd.0 108 advance_pg new pool opts old pool opts 2026-02-11T23:31:00.529+0000 7f414b98b640 10 osd.0 pg_epoch: 108 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742657661s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] handle_activate_map: epoch range: 108..108 2026-02-11T23:31:00.529+0000 7f414b98b640 10 osd.0 pg_epoch: 108 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742657661s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] PeeringState::activate_map 2026-02-11T23:31:00.529+0000 7f414b98b640 20 osd.0 pg_epoch: 108 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742480278s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 107 while current is 108 2026-02-11T23:31:00.529+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:00.529+0000 7f414b98b640 10 osd.0 pg_epoch: 108 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742480278s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:31:00.529+0000 7f414b98b640 10 osd.0 pg_epoch: 108 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742480278s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] do_peering_event: epoch_sent: 108 epoch_requested: 108 NullEvt 2026-02-11T23:31:00.529+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:00.529+0000 7f414b98b640 20 osd.0 108 get_con_osd_cluster to osd.1 from_epoch 108 2026-02-11T23:31:00.529+0000 7f414b98b640 20 osd.0 108 get_nextmap_reserved map_reservations: {108=1} 2026-02-11T23:31:00.530+0000 7f414b98b640 20 osd.0 108 release_map epoch: 108 2026-02-11T23:31:00.530+0000 7f414b98b640 10 osd.0 108 maybe_share_map: con v2:127.0.0.1:6812/3043114013 map epoch 107 -> 108 (shared) 2026-02-11T23:31:00.530+0000 7f414b98b640 10 osd.0 108 send_incremental_map epoch range: (108, 108] to v2:127.0.0.1:6812/3043114013 2026-02-11T23:31:00.530+0000 7f414b98b640 10 osd.0 108 add_map_inc_bl 108 574 bytes 2026-02-11T23:31:00.530+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- osd_map(108..108 src has 1..108) -- 0x5954e00 con 0x4acb000 2026-02-11T23:31:00.530+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_notify2(1.0 1.0 (query:108 sent:108 1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742480278s) ([29,106] all_participants={0, 1} intervals=[([29,106] acting={0, 1})])) e108/108) -- 0x5a8ce00 con 0x4acb000 2026-02-11T23:31:00.530+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:00.530+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:00.530+0000 7f415aefe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 22 ==== osd_map(107..107 src has 1..108) ==== 618+0+0 (unknown 0 0 0) 0x5954e00 con 0x4acb000 2026-02-11T23:31:00.530+0000 7f415aefe640 20 osd.0 108 OSD::ms_dispatch: osd_map(107..107 src has 1..108) 2026-02-11T23:31:00.530+0000 7f415aefe640 20 osd.0 108 _dispatch 0x5954e00 osd_map(107..107 src has 1..108) 2026-02-11T23:31:00.530+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 23 ==== pg_query2(1.0 1.0 query(info 0'0 epoch_sent 107) e107/107) ==== 158+0+0 (unknown 0 0 0) 0x4fca960 con 0x4acb000 2026-02-11T23:31:00.530+0000 7f415aefe640 3 osd.0 108 handle_osd_map epochs [107,107], i have 108, src has [1,108] 2026-02-11T23:31:00.530+0000 7f415aefe640 10 osd.0 108 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:31:00.530+0000 7f415aefe640 10 osd.0 108 no new maps here, dropping 2026-02-11T23:31:00.530+0000 7f416abfa640 15 osd.0 108 enqueue_peering_evt 1.0 epoch_sent: 107 epoch_requested: 107 MQuery 1.0 from 1 query_epoch 107 query: query(info 0'0 epoch_sent 107) 2026-02-11T23:31:00.530+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 107 epoch_requested: 107 MQuery 1.0 from 1 query_epoch 107 query: query(info 0'0 epoch_sent 107)) class_id 2 prio 255 cost 10 e107) 2026-02-11T23:31:00.530+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:00.530+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 107 epoch_requested: 107 MQuery 1.0 from 1 query_epoch 107 query: query(info 0'0 epoch_sent 107)) class_id 2immediate prio 255 cost 10 e107) queued 2026-02-11T23:31:00.530+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:00.530+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 107 epoch_requested: 107 MQuery 1.0 from 1 query_epoch 107 query: query(info 0'0 epoch_sent 107)) class_id 2immediate prio 255 cost 10 e107) pg 0x5946000 2026-02-11T23:31:00.530+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:00.530+0000 7f414b98b640 10 osd.0 pg_epoch: 108 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.742480278s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] do_peering_event: epoch_sent: 107 epoch_requested: 107 MQuery 1.0 from 1 query_epoch 107 query: query(info 0'0 epoch_sent 107) 2026-02-11T23:31:00.530+0000 7f414b98b640 10 osd.0 pg_epoch: 108 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.741745949s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] PeeringState::fulfill_info sending info 2026-02-11T23:31:00.530+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:00.530+0000 7f414b98b640 20 osd.0 108 get_con_osd_cluster to osd.1 from_epoch 108 2026-02-11T23:31:00.530+0000 7f414b98b640 20 osd.0 108 get_nextmap_reserved map_reservations: {108=1} 2026-02-11T23:31:00.530+0000 7f414b98b640 20 osd.0 108 release_map epoch: 108 2026-02-11T23:31:00.530+0000 7f414b98b640 10 osd.0 108 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 108 is not newer than session's projected_epoch of 108 2026-02-11T23:31:00.530+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_notify2(1.0 1.0 (query:107 sent:108 1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.741745949s) ([29,106] all_participants={0, 1} intervals=[([29,106] acting={0, 1})])) e108/107) -- 0x5a8ea00 con 0x4acb000 2026-02-11T23:31:00.530+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:00.603+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 24 ==== osd pg remove(epoch 109; pg1.0; ) ==== 32+0+0 (unknown 0 0 0) 0x3759340 con 0x4acb000 2026-02-11T23:31:00.603+0000 7f416abfa640 7 osd.0 108 handle_fast_pg_remove osd pg remove(epoch 109; pg1.0; ) from osd.1 2026-02-11T23:31:00.603+0000 7f416abfa640 15 osd.0 108 enqueue_peering_evt 1.0 epoch_sent: 109 epoch_requested: 109 DeleteStart 2026-02-11T23:31:00.603+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 109 epoch_requested: 109 DeleteStart) class_id 2 prio 255 cost 10 e109) 2026-02-11T23:31:00.603+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:00.603+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 109 epoch_requested: 109 DeleteStart) class_id 2immediate prio 255 cost 10 e109) queued 2026-02-11T23:31:00.603+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:00.603+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 109 epoch_requested: 109 DeleteStart) class_id 2immediate prio 255 cost 10 e109) pg 0x5946000 2026-02-11T23:31:00.603+0000 7f414f993640 20 osd.0 op_wq _add_slot_waiter 1.0 peering, item epoch is 109, will wait on OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 109 epoch_requested: 109 DeleteStart) class_id 2immediate prio 255 cost 10 e109) 2026-02-11T23:31:00.603+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:00.603+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:01.028+0000 7f4147983640 5 osd.0 108 heartbeat osd_stat(store_statfs(0x384000/0x0/0x384000, data 0x7a075/0x116000, compress 0x0/0x0/0x0, omap 0x9eed, meta 0x1a46113), peers [1,2,3,4] op hist []) 2026-02-11T23:31:01.028+0000 7f4147983640 20 osd.0 108 check_full_status cur ratio 0, physical ratio 0, new state none 2026-02-11T23:31:01.028+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6816/3043114013,v1:127.0.0.1:6817/3043114013] -- osd_ping(ping e108 up_from 5 ping_stamp 2026-02-11T23:31:01.030205+0000/43.705608368s send_stamp 43.705608368s delta_ub 2.011870861s) -- 0x4c9b500 con 0x48c4400 2026-02-11T23:31:01.028+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6814/3043114013,v1:127.0.0.1:6815/3043114013] -- osd_ping(ping e108 up_from 5 ping_stamp 2026-02-11T23:31:01.030205+0000/43.705608368s send_stamp 43.705608368s delta_ub 2.011870861s) -- 0x5af8f00 con 0x4640c00 2026-02-11T23:31:01.028+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6824/2365263801,v1:127.0.0.1:6825/2365263801] -- osd_ping(ping e108 up_from 5 ping_stamp 2026-02-11T23:31:01.030205+0000/43.705608368s send_stamp 43.705608368s delta_ub 4.567062855s) -- 0x5af8a80 con 0x48c4000 2026-02-11T23:31:01.028+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6822/2365263801,v1:127.0.0.1:6823/2365263801] -- osd_ping(ping e108 up_from 5 ping_stamp 2026-02-11T23:31:01.030205+0000/43.705608368s send_stamp 43.705608368s delta_ub 4.567062855s) -- 0x5af8180 con 0x48cd000 2026-02-11T23:31:01.028+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6832/2699377081,v1:127.0.0.1:6833/2699377081] -- osd_ping(ping e108 up_from 5 ping_stamp 2026-02-11T23:31:01.030205+0000/43.705608368s send_stamp 43.705608368s delta_ub 7.219534874s) -- 0x5af8780 con 0x48cd400 2026-02-11T23:31:01.028+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6830/2699377081,v1:127.0.0.1:6831/2699377081] -- osd_ping(ping e108 up_from 5 ping_stamp 2026-02-11T23:31:01.030205+0000/43.705608368s send_stamp 43.705608368s delta_ub 7.219534874s) -- 0x5af8600 con 0x48cd800 2026-02-11T23:31:01.028+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6840/3331151400,v1:127.0.0.1:6841/3331151400] -- osd_ping(ping e108 up_from 5 ping_stamp 2026-02-11T23:31:01.030205+0000/43.705608368s send_stamp 43.705608368s delta_ub 9.836935997s) -- 0x5af9080 con 0x507bc00 2026-02-11T23:31:01.028+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6838/3331151400,v1:127.0.0.1:6839/3331151400] -- osd_ping(ping e108 up_from 5 ping_stamp 2026-02-11T23:31:01.030205+0000/43.705608368s send_stamp 43.705608368s delta_ub 9.836935997s) -- 0x5af9380 con 0x5282000 2026-02-11T23:31:01.029+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6840/3331151400 11 ==== osd_ping(ping_reply e107 up_from 23 ping_stamp 2026-02-11T23:31:01.030205+0000/43.705608368s send_stamp 33.869129181s delta_ub -9.836481094s) ==== 2033+0+0 (crc 0 0 0) 0x5af9080 con 0x507bc00 2026-02-11T23:31:01.029+0000 7f416abfa640 20 osd.0 108 get_con_osd_cluster to osd.4 from_epoch 108 2026-02-11T23:31:01.029+0000 7f416abfa640 20 osd.0 108 get_nextmap_reserved map_reservations: {108=1} 2026-02-11T23:31:01.029+0000 7f416abfa640 20 osd.0 108 release_map epoch: 108 2026-02-11T23:31:01.029+0000 7f416abfa640 10 osd.0 108 maybe_share_map: con v2:127.0.0.1:6836/3331151400 map epoch 107 -> 108 (shared) 2026-02-11T23:31:01.029+0000 7f416abfa640 10 osd.0 108 send_incremental_map epoch range: (108, 108] to v2:127.0.0.1:6836/3331151400 2026-02-11T23:31:01.029+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6830/2699377081 12 ==== osd_ping(ping_reply e108 up_from 19 ping_stamp 2026-02-11T23:31:01.030205+0000/43.705608368s send_stamp 36.486324310s delta_ub -7.219288349s) ==== 2033+0+0 (crc 0 0 0) 0x5af9380 con 0x48cd800 2026-02-11T23:31:01.029+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- osd_map(108..108 src has 1..108) -- 0x59541c0 con 0x507ac00 2026-02-11T23:31:01.029+0000 7f416abfa640 20 osd.0 108 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836695671s,-9.836481094s]) 2026-02-11T23:31:01.029+0000 7f416c3fd640 20 osd.0 108 get_con_osd_cluster to osd.3 from_epoch 108 2026-02-11T23:31:01.029+0000 7f416c3fd640 20 osd.0 108 get_nextmap_reserved map_reservations: {108=1} 2026-02-11T23:31:01.029+0000 7f416c3fd640 20 osd.0 108 release_map epoch: 108 2026-02-11T23:31:01.029+0000 7f416c3fd640 10 osd.0 108 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 108 is not newer than session's projected_epoch of 108 2026-02-11T23:31:01.029+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6832/2699377081 12 ==== osd_ping(ping_reply e108 up_from 19 ping_stamp 2026-02-11T23:31:01.030205+0000/43.705608368s send_stamp 36.486289978s delta_ub -7.219318867s) ==== 2033+0+0 (crc 0 0 0) 0x5af9080 con 0x48cd400 2026-02-11T23:31:01.029+0000 7f416c3fd640 20 osd.0 108 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219526291s,-7.219288349s]) 2026-02-11T23:31:01.029+0000 7f416abfa640 20 osd.0 108 get_con_osd_cluster to osd.3 from_epoch 108 2026-02-11T23:31:01.029+0000 7f416abfa640 20 osd.0 108 get_nextmap_reserved map_reservations: {108=1} 2026-02-11T23:31:01.029+0000 7f416abfa640 20 osd.0 108 release_map epoch: 108 2026-02-11T23:31:01.029+0000 7f416abfa640 10 osd.0 108 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 108 is not newer than session's projected_epoch of 108 2026-02-11T23:31:01.029+0000 7f416abfa640 20 osd.0 108 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219563484s,-7.219318867s]) 2026-02-11T23:31:01.029+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6838/3331151400 11 ==== osd_ping(ping_reply e107 up_from 23 ping_stamp 2026-02-11T23:31:01.030205+0000/43.705608368s send_stamp 33.869152069s delta_ub -9.836461067s) ==== 2033+0+0 (crc 0 0 0) 0x5af9380 con 0x5282000 2026-02-11T23:31:01.029+0000 7f416c3fd640 20 osd.0 108 get_con_osd_cluster to osd.4 from_epoch 108 2026-02-11T23:31:01.029+0000 7f416c3fd640 20 osd.0 108 get_nextmap_reserved map_reservations: {108=1} 2026-02-11T23:31:01.029+0000 7f416c3fd640 20 osd.0 108 release_map epoch: 108 2026-02-11T23:31:01.029+0000 7f416c3fd640 10 osd.0 108 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 108 is not newer than session's projected_epoch of 108 2026-02-11T23:31:01.029+0000 7f416c3fd640 20 osd.0 108 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836714745s,-9.836461067s]) 2026-02-11T23:31:01.029+0000 7f416cbfe640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6824/2365263801 12 ==== osd_ping(ping_reply e109 up_from 14 ping_stamp 2026-02-11T23:31:01.030205+0000/43.705608368s send_stamp 39.138935089s delta_ub -4.566675663s) ==== 2033+0+0 (crc 0 0 0) 0x4c9bc80 con 0x48c4000 2026-02-11T23:31:01.029+0000 7f416cbfe640 20 osd.0 108 get_con_osd_cluster to osd.2 from_epoch 108 2026-02-11T23:31:01.029+0000 7f416cbfe640 20 osd.0 108 get_nextmap_reserved map_reservations: {108=1} 2026-02-11T23:31:01.029+0000 7f416cbfe640 20 osd.0 108 release_map epoch: 108 2026-02-11T23:31:01.029+0000 7f416cbfe640 10 osd.0 108 maybe_share_map: con v2:127.0.0.1:6820/2365263801 updating session's projected_epoch from 106 to ping map epoch of 109 2026-02-11T23:31:01.029+0000 7f416cbfe640 10 osd.0 108 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 108 is not newer than session's projected_epoch of 109 2026-02-11T23:31:01.029+0000 7f416cbfe640 20 osd.0 108 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.567010403s,-4.566675663s]) 2026-02-11T23:31:01.029+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6822/2365263801 12 ==== osd_ping(ping_reply e109 up_from 14 ping_stamp 2026-02-11T23:31:01.030205+0000/43.705608368s send_stamp 39.139110565s delta_ub -4.566502094s) ==== 2033+0+0 (crc 0 0 0) 0x5af9080 con 0x48cd000 2026-02-11T23:31:01.029+0000 7f415aefe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.2 v2:127.0.0.1:6820/2365263801 14 ==== osd_map(109..109 src has 1..109) ==== 626+0+0 (crc 0 0 0) 0x5955dc0 con 0x507b800 2026-02-11T23:31:01.029+0000 7f415aefe640 20 osd.0 108 OSD::ms_dispatch: osd_map(109..109 src has 1..109) 2026-02-11T23:31:01.029+0000 7f416abfa640 20 osd.0 108 get_con_osd_cluster to osd.2 from_epoch 108 2026-02-11T23:31:01.029+0000 7f415aefe640 20 osd.0 108 _dispatch 0x5955dc0 osd_map(109..109 src has 1..109) 2026-02-11T23:31:01.029+0000 7f416abfa640 20 osd.0 108 get_nextmap_reserved map_reservations: {108=1} 2026-02-11T23:31:01.029+0000 7f416abfa640 20 osd.0 108 release_map epoch: 108 2026-02-11T23:31:01.029+0000 7f416abfa640 10 osd.0 108 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 108 is not newer than session's projected_epoch of 109 2026-02-11T23:31:01.029+0000 7f416abfa640 20 osd.0 108 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566872597s,-4.566502094s]) 2026-02-11T23:31:01.029+0000 7f415aefe640 3 osd.0 108 handle_osd_map epochs [109,109], i have 108, src has [1,109] 2026-02-11T23:31:01.029+0000 7f415aefe640 10 osd.0 108 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:31:01.029+0000 7f415aefe640 10 osd.0 108 handle_osd_map got inc map for epoch 109 2026-02-11T23:31:01.029+0000 7f415aefe640 20 osd.0 108 got_full_map 109, nothing requested 2026-02-11T23:31:01.029+0000 7f415aefe640 20 osd.0 108 trim_maps: min=1 oldest_map=1 2026-02-11T23:31:01.029+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6816/3043114013 13 ==== osd_ping(ping_reply e110 up_from 10 ping_stamp 2026-02-11T23:31:01.030205+0000/43.705608368s send_stamp 41.694068909s delta_ub -2.011541843s) ==== 2033+0+0 (crc 0 0 0) 0x5af9080 con 0x48c4400 2026-02-11T23:31:01.029+0000 7f416abfa640 20 osd.0 108 get_con_osd_cluster to osd.1 from_epoch 108 2026-02-11T23:31:01.029+0000 7f416abfa640 20 osd.0 108 get_nextmap_reserved map_reservations: {108=1} 2026-02-11T23:31:01.029+0000 7f416abfa640 20 osd.0 108 release_map epoch: 108 2026-02-11T23:31:01.029+0000 7f416abfa640 10 osd.0 108 maybe_share_map: con v2:127.0.0.1:6812/3043114013 updating session's projected_epoch from 108 to ping map epoch of 110 2026-02-11T23:31:01.029+0000 7f416abfa640 10 osd.0 108 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 108 is not newer than session's projected_epoch of 110 2026-02-11T23:31:01.029+0000 7f416abfa640 20 osd.0 108 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011987925s,-2.011541843s]) 2026-02-11T23:31:01.029+0000 7f415aefe640 20 osd.0 108 handle_osd_map pg_num_history pg_num_history(e109 pg_nums {1={27=1},2={34=1},3={42=1},4={49=1},5={56=1},6={64=1},7={70=1},8={77=1},9={85=1},10={92=1}} deleted_pools 99,2,100,3,101,5,102,6,103,7,104,8,105,9,106,10) 2026-02-11T23:31:01.029+0000 7f415aefe640 10 snap_mapper record_purged_snaps purged_snaps {109={}} 2026-02-11T23:31:01.029+0000 7f415aefe640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:31:01.029+0000 7f415aefe640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e109 maps [1~109] lci=[5,109] tlb=1) 2026-02-11T23:31:01.029+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6814/3043114013 13 ==== osd_ping(ping_reply e110 up_from 10 ping_stamp 2026-02-11T23:31:01.030205+0000/43.705608368s send_stamp 41.694370270s delta_ub -2.011240244s) ==== 2033+0+0 (crc 0 0 0) 0x5af9380 con 0x4640c00 2026-02-11T23:31:01.029+0000 7f416c3fd640 20 osd.0 108 get_con_osd_cluster to osd.1 from_epoch 108 2026-02-11T23:31:01.029+0000 7f416c3fd640 20 osd.0 108 get_nextmap_reserved map_reservations: {108=1} 2026-02-11T23:31:01.029+0000 7f416c3fd640 20 osd.0 108 release_map epoch: 108 2026-02-11T23:31:01.029+0000 7f416c3fd640 10 osd.0 108 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 108 is not newer than session's projected_epoch of 110 2026-02-11T23:31:01.029+0000 7f416c3fd640 20 osd.0 108 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011705399s,-2.011240244s]) 2026-02-11T23:31:01.029+0000 7f415aefe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 25 ==== osd_map(109..110 src has 1..110) ==== 1208+0+0 (unknown 0 0 0) 0x59541c0 con 0x4acb000 2026-02-11T23:31:01.029+0000 7f415aefe640 20 osd.0 108 OSD::ms_dispatch: osd_map(109..110 src has 1..110) 2026-02-11T23:31:01.029+0000 7f415aefe640 20 osd.0 108 _dispatch 0x59541c0 osd_map(109..110 src has 1..110) 2026-02-11T23:31:01.029+0000 7f415aefe640 3 osd.0 108 handle_osd_map epochs [109,110], i have 109, src has [1,110] 2026-02-11T23:31:01.029+0000 7f415aefe640 10 osd.0 108 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:31:01.029+0000 7f415aefe640 10 osd.0 108 handle_osd_map got inc map for epoch 110 2026-02-11T23:31:01.029+0000 7f415aefe640 20 osd.0 108 got_full_map 110, nothing requested 2026-02-11T23:31:01.029+0000 7f415aefe640 20 osd.0 108 trim_maps: min=1 oldest_map=1 2026-02-11T23:31:01.029+0000 7f415aefe640 20 osd.0 108 handle_osd_map pg_num_history pg_num_history(e110 pg_nums {1={27=1},2={34=1},3={42=1},4={49=1},5={56=1},6={64=1},7={70=1},8={77=1},9={85=1},10={92=1}} deleted_pools 99,2,100,3,101,5,102,6,103,7,104,8,105,9,106,10) 2026-02-11T23:31:01.029+0000 7f415aefe640 10 snap_mapper record_purged_snaps purged_snaps {110={}} 2026-02-11T23:31:01.029+0000 7f415aefe640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:31:01.029+0000 7f415aefe640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e110 maps [1~110] lci=[5,110] tlb=1) 2026-02-11T23:31:01.029+0000 7f41591a6640 10 osd.0 108 _committed_osd_maps 109..109 2026-02-11T23:31:01.029+0000 7f41591a6640 10 osd.0 108 advance to epoch 109 (<= last 109 <= newest_map 110) 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0 109 consume_map version 109 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0 108 await_reserved_maps epoch:109 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0 108 await_reserved_maps done 0 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0 109 consume_map 109 -- publish done 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x37582c0 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0 109 identify_splits_and_merges 4.0 e108 to e109 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0 109 identify_splits_and_merges 4.0 e108 to e109 pg_nums {49=1} 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3700dc0 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0 109 identify_splits_and_merges 1.0 e108 to e109 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0 109 identify_splits_and_merges 1.0 e108 to e109 pg_nums {27=1} 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0 109 prune_pg_created 2026-02-11T23:31:01.029+0000 7f41591a6640 10 osd.0:0.consume_map 109 (was 108) 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 pending_peering first epoch 109 <= 109, requeueing 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0:0._wake_pg_slot _wake_pg_slot 1.0 to_process <> waiting <> waiting_peering {109=} 2026-02-11T23:31:01.029+0000 7f41591a6640 10 osd.0:1.consume_map 109 (was 108) 2026-02-11T23:31:01.029+0000 7f41591a6640 10 osd.0:2.consume_map 109 (was 108) 2026-02-11T23:31:01.029+0000 7f41591a6640 10 osd.0:3.consume_map 109 (was 108) 2026-02-11T23:31:01.029+0000 7f41591a6640 10 osd.0:4.consume_map 109 (was 108) 2026-02-11T23:31:01.029+0000 7f41591a6640 10 osd.0:5.consume_map 109 (was 108) 2026-02-11T23:31:01.029+0000 7f41591a6640 10 osd.0:6.consume_map 109 (was 108) 2026-02-11T23:31:01.029+0000 7f41591a6640 10 osd.0:7.consume_map 109 (was 108) 2026-02-11T23:31:01.029+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:01.029+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 109 epoch_requested: 109 DeleteStart) class_id 2immediate prio 255 cost 10 e109) queued 2026-02-11T23:31:01.029+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:01.029+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 109 epoch_requested: 109 DeleteStart) class_id 2immediate prio 255 cost 10 e109) pg 0x5946000 2026-02-11T23:31:01.029+0000 7f41591a6640 10 osd.0 109 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:31:01.029+0000 7f41591a6640 15 osd.0 109 enqueue_peering_evt 4.0 epoch_sent: 109 epoch_requested: 109 NullEvt 2026-02-11T23:31:01.029+0000 7f414f993640 10 osd.0 pg_epoch: 108 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.741745949s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] handle_advance_map: 109 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 109 epoch_requested: 109 NullEvt) class_id 2 prio 255 cost 10 e109) 2026-02-11T23:31:01.029+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 108 -> 109 2026-02-11T23:31:01.029+0000 7f414f993640 10 osd.0 pg_epoch: 108 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.741745949s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] PeeringState::advance_map handle_advance_map [1]/[1] -- 1/1 2026-02-11T23:31:01.029+0000 7f41591a6640 15 osd.0 109 enqueue_peering_evt 1.0 epoch_sent: 109 epoch_requested: 109 NullEvt 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 109 epoch_requested: 109 NullEvt) class_id 2 prio 255 cost 10 e109) 2026-02-11T23:31:01.029+0000 7f41591a6640 7 osd.0 109 activate_map version 109 2026-02-11T23:31:01.029+0000 7f414f993640 10 osd.0 pg_epoch: 109 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.741745949s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] state: Started advmap 2026-02-11T23:31:01.029+0000 7f414f993640 10 osd.0 pg_epoch: 109 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.741745949s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:31:01.029+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:01.029+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 109 epoch_requested: 109 NullEvt) class_id 2immediate prio 255 cost 10 e109) queued 2026-02-11T23:31:01.029+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:01.029+0000 7f414f993640 20 osd.0 109 advance_pg new pool opts old pool opts 2026-02-11T23:31:01.029+0000 7f4150194640 20 osd.0 109 agent_entry empty queue 2026-02-11T23:31:01.029+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 109 epoch_requested: 109 NullEvt) class_id 2immediate prio 255 cost 10 e109) pg 0x5978000 2026-02-11T23:31:01.029+0000 7f414f993640 10 osd.0 pg_epoch: 109 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.741745949s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] handle_activate_map: epoch range: 109..109 2026-02-11T23:31:01.029+0000 7f414f993640 10 osd.0 pg_epoch: 109 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.741745949s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] PeeringState::activate_map 2026-02-11T23:31:01.029+0000 7f414b98b640 10 osd.0 pg_epoch: 108 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 109 2026-02-11T23:31:01.029+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 4.0 108 -> 109 2026-02-11T23:31:01.029+0000 7f414f993640 20 osd.0 pg_epoch: 109 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.242665291s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 107 while current is 109 2026-02-11T23:31:01.029+0000 7f414b98b640 10 osd.0 pg_epoch: 108 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4]/[4] -- 4/4 2026-02-11T23:31:01.029+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:01.029+0000 7f414b98b640 10 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:31:01.029+0000 7f414f993640 10 osd.0 pg_epoch: 109 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.242665291s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] do_peering_event: epoch_sent: 109 epoch_requested: 109 DeleteStart 2026-02-11T23:31:01.029+0000 7f414b98b640 20 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::should_restart_peering new interval newup [4] newacting [4] 2026-02-11T23:31:01.029+0000 7f414b98b640 10 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] state: should_restart_peering, transitioning to Reset 2026-02-11T23:31:01.029+0000 7f414f993640 5 osd.0 pg_epoch: 109 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.242665291s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] exit Started/Stray 0.499984 8 0.000054 2026-02-11T23:31:01.029+0000 7f414b98b640 5 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] exit Started/ReplicaActive/RepNotRecovering 25.041932 127 0.000376 2026-02-11T23:31:01.029+0000 7f414f993640 5 osd.0 pg_epoch: 109 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.242665291s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] enter Started/ToDelete 2026-02-11T23:31:01.029+0000 7f414b98b640 5 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] exit Started/ReplicaActive 25.041937 0 0.000000 2026-02-11T23:31:01.029+0000 7f414f993640 5 osd.0 pg_epoch: 109 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.242665291s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] enter Started/ToDelete/WaitDeleteReserved 2026-02-11T23:31:01.029+0000 7f414b98b640 5 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] exit Started 25.141920 0 0.000000 2026-02-11T23:31:01.029+0000 7f414b98b640 5 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] enter Reset 2026-02-11T23:31:01.029+0000 7f414b98b640 20 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=52 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset set_last_peering_reset 109 2026-02-11T23:31:01.029+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:01.029+0000 7f414b98b640 10 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=109 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset Clearing blocked outgoing recovery messages 2026-02-11T23:31:01.029+0000 7f414f993640 20 osd.0 109 get_con_osd_cluster to osd.1 from_epoch 109 2026-02-11T23:31:01.029+0000 7f414f993640 20 osd.0 109 get_nextmap_reserved map_reservations: {109=1} 2026-02-11T23:31:01.029+0000 7f414f993640 20 osd.0 109 release_map epoch: 109 2026-02-11T23:31:01.029+0000 7f414b98b640 10 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=109 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset Not blocking outgoing recovery messages 2026-02-11T23:31:01.029+0000 7f414f993640 10 osd.0 109 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 109 is not newer than session's projected_epoch of 110 2026-02-11T23:31:01.029+0000 7f414b98b640 10 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=109 crt=0'0 active mbc={}] state: Reset advmap 2026-02-11T23:31:01.029+0000 7f414b98b640 20 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=109 crt=0'0 active mbc={}] PeeringState::should_restart_peering new interval newup [4] newacting [4] 2026-02-11T23:31:01.029+0000 7f414b98b640 10 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=109 crt=0'0 active mbc={}] state: should restart peering, calling start_peering_interval again 2026-02-11T23:31:01.029+0000 7f414b98b640 20 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4,0] r=1 lpr=109 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset set_last_peering_reset 109 2026-02-11T23:31:01.029+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_notify2(1.0 1.0 (query:109 sent:109 1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.242665291s) ([29,106] all_participants={0, 1} intervals=[([29,106] acting={0, 1})])) e109/109) -- 0x5b22e00 con 0x4acb000 2026-02-11T23:31:01.029+0000 7f414b98b640 15 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4] r=1 lpr=109 crt=0'0 active mbc={}] clear_stats 2026-02-11T23:31:01.029+0000 7f414b98b640 10 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 active mbc={}] PeeringState::start_peering_interval : check_new_interval output: check_new_interval interval(52-108 up [4, 0](4) acting [4, 0](4)) up_thru 66 up_from 23 last_epoch_clean 53 interval(52-108 up [4, 0](4) acting [4, 0](4) maybe_went_rw) : primary up 23-66 includes interval 2026-02-11T23:31:01.029+0000 7f4152999640 15 osd.0 109 enqueue_peering_evt 1.0 epoch_sent: 109 epoch_requested: 109 DeleteReserved 2026-02-11T23:31:01.029+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:01.029+0000 7f41591a6640 10 osd.0 109 _committed_osd_maps 110..110 2026-02-11T23:31:01.029+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 109 epoch_requested: 109 NullEvt) class_id 2immediate prio 255 cost 10 e109) queued 2026-02-11T23:31:01.029+0000 7f41591a6640 10 osd.0 109 advance to epoch 110 (<= last 110 <= newest_map 110) 2026-02-11T23:31:01.029+0000 7f4152999640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 109 epoch_requested: 109 DeleteReserved) class_id 2 prio 255 cost 10 e109) 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0 110 consume_map version 110 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0 109 await_reserved_maps epoch:110 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0 109 await_reserved_maps done 0 2026-02-11T23:31:01.029+0000 7f414b98b640 10 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=52) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 active mbc={}] PeeringState::start_peering_interval noting past ([52,108] all_participants={0, 4} intervals=[([52,108] acting={0, 4})]) 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0 110 consume_map 110 -- publish done 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x37582c0 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0 110 identify_splits_and_merges 4.0 e109 to e110 2026-02-11T23:31:01.029+0000 7f414b98b640 20 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 active mbc={}] PeeringState::on_new_interval 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0 110 identify_splits_and_merges 4.0 e109 to e110 pg_nums {49=1} 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3700dc0 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0 110 identify_splits_and_merges 1.0 e109 to e110 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0 110 identify_splits_and_merges 1.0 e109 to e110 pg_nums {27=1} 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:01.029+0000 7f414b98b640 20 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 active mbc={}] PeeringState::on_new_interval upacting_features 0x3f0ffffffffdffff from [4]+[4] 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:01.029+0000 7f414b98b640 20 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 active mbc={}] PeeringState::on_new_interval checking missing set deletes flag. missing = missing(0 may_include_deletes = 1) 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:01.029+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:01.029+0000 7f414b98b640 10 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 active mbc={}] PeeringState::init_hb_stamps now [] 2026-02-11T23:31:01.029+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 109 epoch_requested: 109 NullEvt) class_id 2immediate prio 255 cost 10 e109) pg 0x5946000 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:01.029+0000 7f414f993640 10 osd.0 pg_epoch: 109 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.242665291s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] do_peering_event: epoch_sent: 109 epoch_requested: 109 NullEvt 2026-02-11T23:31:01.029+0000 7f414b98b640 10 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958727837s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 active pruub 58.665332794s@ mbc={}] PeeringState::on_new_interval prior_readable_until_ub 58.665332794s (mnow 43.706604004s + 14.958727837s) 2026-02-11T23:31:01.029+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:01.029+0000 7f414b98b640 10 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958727837s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 active pruub 58.665332794s@ mbc={}] cancel_recovery 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:01.029+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:01.029+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 109 epoch_requested: 109 DeleteReserved) class_id 2immediate prio 255 cost 10 e109) queued 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:01.029+0000 7f414b98b640 10 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958727837s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 active pruub 58.665332794s@ mbc={}] clear_recovery_state 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0 110 prune_pg_created 2026-02-11T23:31:01.029+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:01.029+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 109 epoch_requested: 109 DeleteReserved) class_id 2immediate prio 255 cost 10 e109) pg 0x5946000 2026-02-11T23:31:01.029+0000 7f414b98b640 15 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958727837s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 active pruub 58.665332794s@ mbc={}] _clear_recovery_state 2026-02-11T23:31:01.029+0000 7f41591a6640 10 osd.0:0.consume_map 110 (was 109) 2026-02-11T23:31:01.029+0000 7f414f993640 10 osd.0 pg_epoch: 109 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.242665291s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] do_peering_event: epoch_sent: 109 epoch_requested: 109 DeleteReserved 2026-02-11T23:31:01.029+0000 7f414b98b640 10 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958727837s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 active pruub 58.665332794s@ mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:31:01.029+0000 7f414f993640 5 osd.0 pg_epoch: 109 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.242665291s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] exit Started/ToDelete/WaitDeleteReserved 0.000103 2 0.000022 2026-02-11T23:31:01.029+0000 7f414b98b640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:31:01.029+0000 7f414f993640 5 osd.0 pg_epoch: 109 pg[1.0( empty local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.242665291s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] enter Started/ToDelete/Deleting 2026-02-11T23:31:01.029+0000 7f41591a6640 10 osd.0:1.consume_map 110 (was 109) 2026-02-11T23:31:01.029+0000 7f414b98b640 20 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958727837s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 active pruub 58.665332794s@ mbc={}] scrubber: FSM: ReplicaActive::exit() 2026-02-11T23:31:01.029+0000 7f41591a6640 10 osd.0:2.consume_map 110 (was 109) 2026-02-11T23:31:01.029+0000 7f41591a6640 10 osd.0:3.consume_map 110 (was 109) 2026-02-11T23:31:01.029+0000 7f41591a6640 10 osd.0:4.consume_map 110 (was 109) 2026-02-11T23:31:01.029+0000 7f414f993640 10 osd.0 pg_epoch: 109 pg[1.0( empty lb MIN local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.242665291s) [1] r=-1 lpr=107 DELETING pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] on_removal 2026-02-11T23:31:01.029+0000 7f414b98b640 10 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958727837s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 active pruub 58.665332794s@ mbc={}] scrubber: FSM: ReplicaActive::clear_remote_reservation(): pending_reservation_nonce 0, reservation_granted false 2026-02-11T23:31:01.029+0000 7f41591a6640 10 osd.0:5.consume_map 110 (was 109) 2026-02-11T23:31:01.029+0000 7f414f993640 10 osd.0 pg_epoch: 109 pg[1.0( empty lb MIN local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.242665291s) [1] r=-1 lpr=107 DELETING pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] on_shutdown 2026-02-11T23:31:01.029+0000 7f41591a6640 10 osd.0:6.consume_map 110 (was 109) 2026-02-11T23:31:01.029+0000 7f414b98b640 10 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958727837s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 active pruub 58.665332794s@ mbc={}] scrubber: FSM: -- state -->> NotActive 2026-02-11T23:31:01.029+0000 7f41591a6640 10 osd.0:7.consume_map 110 (was 109) 2026-02-11T23:31:01.029+0000 7f414b98b640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:31:01.029+0000 7f41591a6640 10 osd.0 110 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:31:01.029+0000 7f414f993640 10 osd.0 pg_epoch: 109 pg[1.0( empty lb MIN local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.242665291s) [1] r=-1 lpr=107 DELETING pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:31:01.029+0000 7f414f993640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:31:01.029+0000 7f41591a6640 15 osd.0 110 enqueue_peering_evt 4.0 epoch_sent: 110 epoch_requested: 110 NullEvt 2026-02-11T23:31:01.029+0000 7f414f993640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:31:01.029+0000 7f414b98b640 1 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958727837s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 active pruub 58.665332794s@ mbc={}] PeeringState::start_peering_interval up [4,0] -> [4], acting [4,0] -> [4], acting_primary 4 -> 4, up_primary 4 -> 4, role 1 -> -1, features acting 4544132024016699391 upacting 4544132024016699391 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 110 epoch_requested: 110 NullEvt) class_id 2 prio 255 cost 10 e110) 2026-02-11T23:31:01.029+0000 7f41591a6640 15 osd.0 110 enqueue_peering_evt 1.0 epoch_sent: 110 epoch_requested: 110 NullEvt 2026-02-11T23:31:01.029+0000 7f414f993640 10 osd.0 pg_epoch: 109 pg[1.0( empty lb MIN local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.242665291s) [1] r=-1 lpr=107 DELETING pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] cancel_copy_ops 2026-02-11T23:31:01.029+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 110 epoch_requested: 110 NullEvt) class_id 2 prio 255 cost 10 e110) 2026-02-11T23:31:01.029+0000 7f414b98b640 10 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958727837s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown pruub 58.665332794s@ mbc={}] PeeringState::clear_primary_state clear_primary_state 2026-02-11T23:31:01.029+0000 7f41591a6640 7 osd.0 110 activate_map version 110 2026-02-11T23:31:01.029+0000 7f414f993640 10 osd.0 pg_epoch: 109 pg[1.0( empty lb MIN local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.242665291s) [1] r=-1 lpr=107 DELETING pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] cancel_flush_ops 2026-02-11T23:31:01.029+0000 7f414b98b640 20 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958727837s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown pruub 58.665332794s@ mbc={}] clear_primary_state 2026-02-11T23:31:01.029+0000 7f4150194640 20 osd.0 110 agent_entry empty queue 2026-02-11T23:31:01.029+0000 7f414f993640 10 osd.0 pg_epoch: 109 pg[1.0( empty lb MIN local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.242665291s) [1] r=-1 lpr=107 DELETING pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] cancel_proxy_ops 2026-02-11T23:31:01.029+0000 7f414f993640 10 osd.0 pg_epoch: 109 pg[1.0( empty lb MIN local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.242665291s) [1] r=-1 lpr=107 DELETING pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] cancel_manifest_ops 2026-02-11T23:31:01.029+0000 7f414b98b640 10 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958727837s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown pruub 58.665332794s@ mbc={}] release_backoffs [4:00000000::::0,MAX) 2026-02-11T23:31:01.029+0000 7f414f993640 10 osd.0 pg_epoch: 109 pg[1.0( empty lb MIN local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.242665291s) [1] r=-1 lpr=107 DELETING pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] cancel_cls_gather_ops 2026-02-11T23:31:01.029+0000 7f414b98b640 10 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958727837s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown pruub 58.665332794s@ mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:31:01.029+0000 7f414b98b640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:31:01.029+0000 7f414b98b640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:31:01.029+0000 7f414f993640 10 osd.0 pg_epoch: 109 pg[1.0( empty lb MIN local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.242665291s) [1] r=-1 lpr=107 DELETING pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] clear_backoffs 2026-02-11T23:31:01.029+0000 7f414b98b640 20 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958727837s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown pruub 58.665332794s@ mbc={}] agent_stop 2026-02-11T23:31:01.029+0000 7f414f993640 20 osd.0 pg_epoch: 109 pg[1.0( empty lb MIN local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.242665291s) [1] r=-1 lpr=107 DELETING pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] exit NotTrimming 2026-02-11T23:31:01.029+0000 7f414b98b640 10 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958727837s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown pruub 58.665332794s@ mbc={}] on_change 2026-02-11T23:31:01.029+0000 7f414f993640 20 osd.0 pg_epoch: 109 pg[1.0( empty lb MIN local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.242665291s) [1] r=-1 lpr=107 DELETING pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] enter NotTrimming 2026-02-11T23:31:01.029+0000 7f414b98b640 10 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958727837s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown pruub 58.665332794s@ mbc={}] cancel_copy_ops 2026-02-11T23:31:01.029+0000 7f414b98b640 10 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958727837s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown pruub 58.665332794s@ mbc={}] cancel_flush_ops 2026-02-11T23:31:01.029+0000 7f414f993640 10 osd.0 pg_epoch: 109 pg[1.0( empty lb MIN local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.242665291s) [1] r=-1 lpr=107 DELETING pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] on_change 2026-02-11T23:31:01.029+0000 7f414b98b640 10 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958727837s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown pruub 58.665332794s@ mbc={}] cancel_proxy_ops 2026-02-11T23:31:01.029+0000 7f414f993640 10 osd.0 pg_epoch: 109 pg[1.0( empty lb MIN local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.242665291s) [1] r=-1 lpr=107 DELETING pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] clear_async_reads 2026-02-11T23:31:01.029+0000 7f414b98b640 10 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958727837s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown pruub 58.665332794s@ mbc={}] cancel_manifest_ops 2026-02-11T23:31:01.029+0000 7f414b98b640 10 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958727837s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown pruub 58.665332794s@ mbc={}] cancel_cls_gather_ops 2026-02-11T23:31:01.029+0000 7f414f993640 20 osd.0 pg_epoch: 109 pg[1.0( empty lb MIN local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.242665291s) [1] r=-1 lpr=107 DELETING pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] clear_primary_state 2026-02-11T23:31:01.029+0000 7f414b98b640 10 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958727837s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown pruub 58.665332794s@ mbc={}] on_change_cleanup 2026-02-11T23:31:01.030+0000 7f414b98b640 10 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958727837s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown pruub 58.665332794s@ mbc={}] on_change 2026-02-11T23:31:01.030+0000 7f414f993640 10 osd.0 pg_epoch: 109 pg[1.0( empty lb MIN local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.242665291s) [1] r=-1 lpr=107 DELETING pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] release_backoffs [1:00000000::::0,MAX) 2026-02-11T23:31:01.030+0000 7f414f993640 10 osd.0 pg_epoch: 109 pg[1.0( empty lb MIN local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.242665291s) [1] r=-1 lpr=107 DELETING pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:31:01.030+0000 7f414b98b640 20 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958727837s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown pruub 58.665332794s@ mbc={}] exit NotTrimming 2026-02-11T23:31:01.030+0000 7f414f993640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:31:01.030+0000 7f414f993640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:31:01.030+0000 7f414b98b640 20 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958727837s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown pruub 58.665332794s@ mbc={}] enter NotTrimming 2026-02-11T23:31:01.030+0000 7f414b98b640 10 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958727837s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] plpg_on_role_change 2026-02-11T23:31:01.030+0000 7f414f993640 20 osd.0 pg_epoch: 109 pg[1.0( empty lb MIN local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.242665291s) [1] r=-1 lpr=107 DELETING pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] agent_stop 2026-02-11T23:31:01.030+0000 7f414b98b640 10 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958727837s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:31:01.030+0000 7f414f993640 10 osd.0 pg_epoch: 109 pg[1.0( empty lb MIN local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.242665291s) [1] r=-1 lpr=107 DELETING pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] cancel_recovery 2026-02-11T23:31:01.030+0000 7f414f993640 10 osd.0 pg_epoch: 109 pg[1.0( empty lb MIN local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.242665291s) [1] r=-1 lpr=107 DELETING pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] clear_recovery_state 2026-02-11T23:31:01.030+0000 7f414b98b640 10 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958727837s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] plpg_on_pool_change 2026-02-11T23:31:01.030+0000 7f414f993640 15 osd.0 pg_epoch: 109 pg[1.0( empty lb MIN local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.242665291s) [1] r=-1 lpr=107 DELETING pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] _clear_recovery_state 2026-02-11T23:31:01.030+0000 7f414b98b640 20 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958727837s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] hit_set_clear 2026-02-11T23:31:01.030+0000 7f414b98b640 20 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958727837s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] agent_stop 2026-02-11T23:31:01.030+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:01.030+0000 7f414b98b640 20 osd.0 110 advance_pg new pool opts old pool opts 2026-02-11T23:31:01.030+0000 7f414b98b640 10 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958727837s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] handle_activate_map: epoch range: 109..109 2026-02-11T23:31:01.030+0000 7f414b98b640 10 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958727837s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] PeeringState::activate_map 2026-02-11T23:31:01.030+0000 7f414b98b640 5 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958574295s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] exit Reset 0.000224 1 0.000220 2026-02-11T23:31:01.030+0000 7f414b98b640 5 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958574295s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] enter Started 2026-02-11T23:31:01.030+0000 7f414b98b640 5 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958574295s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] enter Start 2026-02-11T23:31:01.030+0000 7f414b98b640 1 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958574295s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] state: transitioning to Stray 2026-02-11T23:31:01.030+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:01.030+0000 7f414b98b640 5 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958574295s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] exit Start 0.000008 0 0.000000 2026-02-11T23:31:01.030+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 110 epoch_requested: 110 NullEvt) class_id 2immediate prio 255 cost 10 e110) queued 2026-02-11T23:31:01.030+0000 7f414b98b640 5 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958574295s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] enter Started/Stray 2026-02-11T23:31:01.030+0000 7f414b98b640 20 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958574295s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 95 while current is 109 2026-02-11T23:31:01.030+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:01.030+0000 7f414b98b640 10 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958574295s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:31:01.030+0000 7f414b98b640 10 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958574295s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] do_peering_event: epoch_sent: 109 epoch_requested: 109 NullEvt 2026-02-11T23:31:01.030+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:01.030+0000 7f414b98b640 20 osd.0 110 get_con_osd_cluster to osd.4 from_epoch 109 2026-02-11T23:31:01.030+0000 7f414b98b640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:01.030+0000 7f414b98b640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:01.030+0000 7f414b98b640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6836/3331151400 map epoch 108 -> 109 (shared) 2026-02-11T23:31:01.030+0000 7f414b98b640 10 osd.0 110 send_incremental_map epoch range: (109, 109] to v2:127.0.0.1:6836/3331151400 2026-02-11T23:31:01.030+0000 7f414b98b640 10 osd.0 110 add_map_inc_bl 109 582 bytes 2026-02-11T23:31:01.030+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- osd_map(109..109 src has 1..110) -- 0x5955180 con 0x507ac00 2026-02-11T23:31:01.030+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_notify2(4.0 4.0 (query:109 sent:109 4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958574295s) ([52,108] all_participants={0, 4} intervals=[([52,108] acting={0, 4})])) e109/109) -- 0x5a8f100 con 0x507ac00 2026-02-11T23:31:01.030+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:01.030+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 110 epoch_requested: 110 NullEvt) class_id 2immediate prio 255 cost 10 e110) queued 2026-02-11T23:31:01.030+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:01.030+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 110 epoch_requested: 110 NullEvt) class_id 2immediate prio 255 cost 10 e110) pg 0x5946000 2026-02-11T23:31:01.030+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:01.030+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 110 epoch_requested: 110 NullEvt) class_id 2immediate prio 255 cost 10 e110) pg 0x5978000 2026-02-11T23:31:01.030+0000 7f414b98b640 10 osd.0 pg_epoch: 109 pg[1.0( empty lb MIN local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.242665291s) [1] r=-1 lpr=107 DELETING pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] handle_advance_map: 110 2026-02-11T23:31:01.030+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 1.0 109 -> 110 2026-02-11T23:31:01.030+0000 7f414f993640 10 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958574295s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] handle_advance_map: 110 2026-02-11T23:31:01.030+0000 7f414b98b640 10 osd.0 pg_epoch: 109 pg[1.0( empty lb MIN local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.242665291s) [1] r=-1 lpr=107 DELETING pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] PeeringState::advance_map handle_advance_map [1]/[1] -- 1/1 2026-02-11T23:31:01.030+0000 7f414f993640 20 osd.0:0.update_pg_epoch 4.0 109 -> 110 2026-02-11T23:31:01.030+0000 7f414b98b640 10 osd.0 pg_epoch: 110 pg[1.0( empty lb MIN local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.242665291s) [1] r=-1 lpr=107 DELETING pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] state: Started advmap 2026-02-11T23:31:01.030+0000 7f414f993640 10 osd.0 pg_epoch: 109 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958574295s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] PeeringState::advance_map handle_advance_map [4]/[4] -- 4/4 2026-02-11T23:31:01.030+0000 7f414b98b640 10 osd.0 pg_epoch: 110 pg[1.0( empty lb MIN local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.242665291s) [1] r=-1 lpr=107 DELETING pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:31:01.030+0000 7f414f993640 10 osd.0 pg_epoch: 110 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958574295s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] state: Started advmap 2026-02-11T23:31:01.030+0000 7f414b98b640 20 osd.0 110 advance_pg new pool opts old pool opts 2026-02-11T23:31:01.030+0000 7f414f993640 10 osd.0 pg_epoch: 110 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958574295s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:31:01.030+0000 7f414b98b640 10 osd.0 pg_epoch: 110 pg[1.0( empty lb MIN local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.242665291s) [1] r=-1 lpr=107 DELETING pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] handle_activate_map: epoch range: 110..110 2026-02-11T23:31:01.030+0000 7f414f993640 10 osd.0 pg_epoch: 110 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958574295s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] plpg_on_pool_change 2026-02-11T23:31:01.030+0000 7f414b98b640 10 osd.0 pg_epoch: 110 pg[1.0( empty lb MIN local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.242665291s) [1] r=-1 lpr=107 DELETING pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] PeeringState::activate_map 2026-02-11T23:31:01.030+0000 7f414f993640 20 osd.0 pg_epoch: 110 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958574295s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] hit_set_clear 2026-02-11T23:31:01.030+0000 7f414b98b640 20 osd.0 pg_epoch: 110 pg[1.0( empty lb MIN local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.242665291s) [1] r=-1 lpr=107 DELETING pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 109 while current is 110 2026-02-11T23:31:01.030+0000 7f414f993640 20 osd.0 pg_epoch: 110 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958574295s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] agent_stop 2026-02-11T23:31:01.030+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:01.030+0000 7f414f993640 20 osd.0 110 advance_pg new pool opts old pool opts 2026-02-11T23:31:01.030+0000 7f414b98b640 10 osd.0 pg_epoch: 110 pg[1.0( empty lb MIN local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.242665291s) [1] r=-1 lpr=107 DELETING pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] do_peering_event: epoch_sent: 110 epoch_requested: 110 NullEvt 2026-02-11T23:31:01.030+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:01.030+0000 7f414f993640 10 osd.0 pg_epoch: 110 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958574295s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] handle_activate_map: epoch range: 110..110 2026-02-11T23:31:01.030+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:01.030+0000 7f414f993640 10 osd.0 pg_epoch: 110 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958574295s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] PeeringState::activate_map 2026-02-11T23:31:01.030+0000 7f414f993640 20 osd.0 pg_epoch: 110 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958383560s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 109 while current is 110 2026-02-11T23:31:01.030+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:01.030+0000 7f414f993640 10 osd.0 pg_epoch: 110 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958383560s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:31:01.030+0000 7f414f993640 10 osd.0 pg_epoch: 110 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958383560s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] do_peering_event: epoch_sent: 110 epoch_requested: 110 NullEvt 2026-02-11T23:31:01.030+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:01.030+0000 7f414f993640 20 osd.0 110 get_con_osd_cluster to osd.4 from_epoch 110 2026-02-11T23:31:01.030+0000 7f414f993640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:01.030+0000 7f414f993640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:01.030+0000 7f414f993640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6836/3331151400 map epoch 109 -> 110 (shared) 2026-02-11T23:31:01.030+0000 7f414f993640 10 osd.0 110 send_incremental_map epoch range: (110, 110] to v2:127.0.0.1:6836/3331151400 2026-02-11T23:31:01.030+0000 7f414f993640 10 osd.0 110 add_map_inc_bl 110 574 bytes 2026-02-11T23:31:01.030+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- osd_map(110..110 src has 1..110) -- 0x4a30e00 con 0x507ac00 2026-02-11T23:31:01.030+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_notify2(4.0 4.0 (query:110 sent:110 4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958383560s) ([52,108] all_participants={0, 4} intervals=[([52,108] acting={0, 4})])) e110/110) -- 0x5b23500 con 0x507ac00 2026-02-11T23:31:01.030+0000 7f414f993640 10 osd.0 110 queue_for_pg_delete on 1.0 e 109 2026-02-11T23:31:01.030+0000 7f414f993640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGDelete(pgid=1.0 epoch_queued=109) class_id 1 prio 5 cost 1048576 e109) 2026-02-11T23:31:01.030+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:01.030+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGDelete(1.0 e109) class_id 1background_best_effort prio 5 qos_cost 1048576 cost 1048576 e109) queued 2026-02-11T23:31:01.030+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:01.030+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGDelete(1.0 e109) class_id 1background_best_effort prio 5 qos_cost 1048576 cost 1048576 e109) pg 0x5946000 2026-02-11T23:31:01.030+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:01.030+0000 7f414f993640 10 osd.0 pg_epoch: 110 pg[1.0( empty lb MIN local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.242665291s) [1] r=-1 lpr=107 DELETING pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] do_peering_event: epoch_sent: 109 epoch_requested: 109 DeleteSome 2026-02-11T23:31:01.030+0000 7f414f993640 10 osd.0 pg_epoch: 110 pg[1.0( empty lb MIN local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.242665291s) [1] r=-1 lpr=107 DELETING pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] do_delete_work 2026-02-11T23:31:01.030+0000 7f414f993640 20 osd.0 pg_epoch: 110 pg[1.0( empty lb MIN local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.242665291s) [1] r=-1 lpr=107 DELETING pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] do_delete_work [#1:00000000::::head#] 2026-02-11T23:31:01.030+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:01.030+0000 7f414f993640 20 osd.0 110 try_finish_pg_delete 1.0 0x5946000 2026-02-11T23:31:01.030+0000 7f414f993640 10 osd.0:0._detach_pg 1.0 0x5946000 2026-02-11T23:31:01.030+0000 7f414f993640 5 osd.0 pg_epoch: 110 pg[1.0( empty lb MIN local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.242665291s) [1] r=-1 lpr=107 DELETING pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] exit Started/ToDelete/Deleting 0.000526 4 0.000117 2026-02-11T23:31:01.030+0000 7f414f993640 5 osd.0 pg_epoch: 110 pg[1.0( empty lb MIN local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.242665291s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] exit Started/ToDelete 0.000642 0 0.000000 2026-02-11T23:31:01.030+0000 7f414f993640 5 osd.0 pg_epoch: 110 pg[1.0( empty lb MIN local-lis/les=29/30 n=0 ec=27/27 lis/c=29/29 les/c/f=30/30/0 sis=107 pruub=9.242665291s) [1] r=-1 lpr=107 pi=[29,107)/1 crt=0'0 unknown NOTIFY pruub 52.949176788s@ mbc={}] exit Started 0.500649 0 0.000000 2026-02-11T23:31:01.030+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:01.030+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:01.030+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:01.030+0000 7f414f993640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:31:01.030+0000 7f414f993640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:31:01.030+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:01.031+0000 7f415aefe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 28 ==== osd_map(109..109 src has 1..110) ==== 626+0+0 (unknown 0 0 0) 0x4a30e00 con 0x507ac00 2026-02-11T23:31:01.031+0000 7f415aefe640 20 osd.0 110 OSD::ms_dispatch: osd_map(109..109 src has 1..110) 2026-02-11T23:31:01.031+0000 7f415aefe640 20 osd.0 110 _dispatch 0x4a30e00 osd_map(109..109 src has 1..110) 2026-02-11T23:31:01.031+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 29 ==== pg_query2(4.0 4.0 query(info 0'0 epoch_sent 109) e109/109) ==== 158+0+0 (unknown 0 0 0) 0x4fca960 con 0x507ac00 2026-02-11T23:31:01.031+0000 7f415aefe640 3 osd.0 110 handle_osd_map epochs [109,109], i have 110, src has [1,110] 2026-02-11T23:31:01.031+0000 7f415aefe640 10 osd.0 110 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:31:01.031+0000 7f415aefe640 10 osd.0 110 no new maps here, dropping 2026-02-11T23:31:01.031+0000 7f416abfa640 15 osd.0 110 enqueue_peering_evt 4.0 epoch_sent: 109 epoch_requested: 109 MQuery 4.0 from 4 query_epoch 109 query: query(info 0'0 epoch_sent 109) 2026-02-11T23:31:01.031+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 109 epoch_requested: 109 MQuery 4.0 from 4 query_epoch 109 query: query(info 0'0 epoch_sent 109)) class_id 2 prio 255 cost 10 e109) 2026-02-11T23:31:01.031+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:01.031+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 109 epoch_requested: 109 MQuery 4.0 from 4 query_epoch 109 query: query(info 0'0 epoch_sent 109)) class_id 2immediate prio 255 cost 10 e109) queued 2026-02-11T23:31:01.031+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:01.031+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 109 epoch_requested: 109 MQuery 4.0 from 4 query_epoch 109 query: query(info 0'0 epoch_sent 109)) class_id 2immediate prio 255 cost 10 e109) pg 0x5978000 2026-02-11T23:31:01.031+0000 7f414f993640 10 osd.0 pg_epoch: 110 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.958383560s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] do_peering_event: epoch_sent: 109 epoch_requested: 109 MQuery 4.0 from 4 query_epoch 109 query: query(info 0'0 epoch_sent 109) 2026-02-11T23:31:01.031+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:01.031+0000 7f414f993640 10 osd.0 pg_epoch: 110 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.957548141s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] PeeringState::fulfill_info sending info 2026-02-11T23:31:01.031+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:01.031+0000 7f414f993640 20 osd.0 110 get_con_osd_cluster to osd.4 from_epoch 110 2026-02-11T23:31:01.031+0000 7f414f993640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:01.031+0000 7f414f993640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:01.031+0000 7f414f993640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:01.031+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_notify2(4.0 4.0 (query:109 sent:110 4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.957548141s) ([52,108] all_participants={0, 4} intervals=[([52,108] acting={0, 4})])) e110/109) -- 0x5b23c00 con 0x507ac00 2026-02-11T23:31:01.031+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:01.080+0000 7f416c3fd640 1 --2- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] >> conn(0x5956c00 0x527f600 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).accept 2026-02-11T23:31:01.080+0000 7f416cbfe640 1 --2- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] >> conn(0x5956c00 0x527f600 unknown :-1 s=BANNER_ACCEPTING pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:31:01.080+0000 7f416cbfe640 10 osd.0 110 ms_handle_fast_authentication new session 0x5954a80 con 0x5956c00 entity client.admin addr 2026-02-11T23:31:01.080+0000 7f416cbfe640 1 --2- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] >> 127.0.0.1:0/4165534873 conn(0x5956c00 0x527f600 crc :-1 s=READY pgs=3 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=client.4297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-11T23:31:01.080+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== client.4297 127.0.0.1:0/4165534873 1 ==== command(tid 1: {"prefix": "get_command_descriptions"}) ==== 62+0+0 (crc 0 0 0) 0x37002c0 con 0x5956c00 2026-02-11T23:31:01.080+0000 7f415cf02640 20 osd.0 110 OSD::ms_dispatch: command(tid 1: {"prefix": "get_command_descriptions"}) 2026-02-11T23:31:01.080+0000 7f415cf02640 20 osd.0 110 _dispatch 0x37002c0 command(tid 1: {"prefix": "get_command_descriptions"}) 2026-02-11T23:31:01.081+0000 7f416bbfc640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> 127.0.0.1:0/4165534873 -- command_reply(tid 1: 0 ) -- 0x3759600 con 0x5956c00 2026-02-11T23:31:01.089+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== client.4297 127.0.0.1:0/4165534873 2 ==== command(tid 2: {"prefix": "flush_pg_stats"}) ==== 52+0+0 (crc 0 0 0) 0x3759600 con 0x5956c00 2026-02-11T23:31:01.089+0000 7f415cf02640 20 osd.0 110 OSD::ms_dispatch: command(tid 2: {"prefix": "flush_pg_stats"}) 2026-02-11T23:31:01.089+0000 7f415cf02640 20 osd.0 110 _dispatch 0x3759600 command(tid 2: {"prefix": "flush_pg_stats"}) 2026-02-11T23:31:01.089+0000 7f416bbfc640 15 osd.0 110 collect_pg_stats 2026-02-11T23:31:01.089+0000 7f416bbfc640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] -- pg_stats(0 pgs seq 21474836492 v 0) -- 0x5b80600 con 0x4aca800 2026-02-11T23:31:01.089+0000 7f416bbfc640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> 127.0.0.1:0/4165534873 -- command_reply(tid 2: 0 ) -- 0x37002c0 con 0x5956c00 2026-02-11T23:31:01.095+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== client.4297 127.0.0.1:0/4165534873 3 ==== command(tid 3: {"prefix": "flush_pg_stats"}) ==== 52+0+0 (crc 0 0 0) 0x37002c0 con 0x5956c00 2026-02-11T23:31:01.095+0000 7f415cf02640 20 osd.0 110 OSD::ms_dispatch: command(tid 3: {"prefix": "flush_pg_stats"}) 2026-02-11T23:31:01.095+0000 7f415cf02640 20 osd.0 110 _dispatch 0x37002c0 command(tid 3: {"prefix": "flush_pg_stats"}) 2026-02-11T23:31:01.095+0000 7f416bbfc640 15 osd.0 110 collect_pg_stats 2026-02-11T23:31:01.095+0000 7f416bbfc640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] -- pg_stats(0 pgs seq 21474836493 v 0) -- 0x4aa0000 con 0x4aca800 2026-02-11T23:31:01.095+0000 7f416bbfc640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> 127.0.0.1:0/4165534873 -- command_reply(tid 3: 0 ) -- 0x3759600 con 0x5956c00 2026-02-11T23:31:01.095+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] >> 127.0.0.1:0/4165534873 conn(0x5956c00 msgr2=0x527f600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 80 2026-02-11T23:31:01.095+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] >> 127.0.0.1:0/4165534873 conn(0x5956c00 msgr2=0x527f600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-11T23:31:01.095+0000 7f416cbfe640 1 --2- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] >> 127.0.0.1:0/4165534873 conn(0x5956c00 0x527f600 crc :-1 s=READY pgs=3 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-11T23:31:01.095+0000 7f416cbfe640 1 --2- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] >> 127.0.0.1:0/4165534873 conn(0x5956c00 0x527f600 crc :-1 s=READY pgs=3 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-11T23:31:01.095+0000 7f415cf02640 2 osd.0 110 ms_handle_reset con 0x5956c00 session 0x5954a80 2026-02-11T23:31:01.115+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 30 ==== osd pg remove(epoch 111; pg4.0; ) ==== 32+0+0 (unknown 0 0 0) 0x3759340 con 0x507ac00 2026-02-11T23:31:01.115+0000 7f416abfa640 7 osd.0 110 handle_fast_pg_remove osd pg remove(epoch 111; pg4.0; ) from osd.4 2026-02-11T23:31:01.115+0000 7f416abfa640 15 osd.0 110 enqueue_peering_evt 4.0 epoch_sent: 111 epoch_requested: 111 DeleteStart 2026-02-11T23:31:01.115+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 111 epoch_requested: 111 DeleteStart) class_id 2 prio 255 cost 10 e111) 2026-02-11T23:31:01.115+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:01.115+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 111 epoch_requested: 111 DeleteStart) class_id 2immediate prio 255 cost 10 e111) queued 2026-02-11T23:31:01.115+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:01.115+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 111 epoch_requested: 111 DeleteStart) class_id 2immediate prio 255 cost 10 e111) pg 0x5978000 2026-02-11T23:31:01.115+0000 7f414b98b640 20 osd.0 op_wq _add_slot_waiter 4.0 peering, item epoch is 111, will wait on OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 111 epoch_requested: 111 DeleteStart) class_id 2immediate prio 255 cost 10 e111) 2026-02-11T23:31:01.115+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:01.115+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:01.240+0000 7f4162bea640 10 osd.0 110 tick_without_osd_lock 2026-02-11T23:31:01.240+0000 7f4162bea640 0 osd.0 110 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:31:01.240+0000 7f4162bea640 20 osd.0 110 tick_without_osd_lock max_waiting_epoch 111, requesting new map 2026-02-11T23:31:01.240+0000 7f4162bea640 10 monclient: _renew_subs 2026-02-11T23:31:01.240+0000 7f4162bea640 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:31:01.240+0000 7f4162bea640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> v2:127.0.0.1:7180/0 -- mon_subscribe({osdmap=111}) -- 0x3814820 con 0x4acbc00 2026-02-11T23:31:01.240+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:01.240+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:01.240+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:01.240+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.341 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:01.240+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:01.241, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:01.240+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:01.240+0000 7f4162bea640 10 osd.0 110 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:01.240+0000 7f4162bea640 20 osd.0 110 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:01.240+0000 7f4162bea640 10 osd.0 110 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:01.240+0000 7f4162bea640 20 osd.0 110 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:01.240+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 50 ==== osd_map(111..111 src has 1..111) ==== 296+0+0 (crc 0 0 0) 0x5955340 con 0x4acbc00 2026-02-11T23:31:01.240+0000 7f415cf02640 20 osd.0 110 OSD::ms_dispatch: osd_map(111..111 src has 1..111) 2026-02-11T23:31:01.240+0000 7f415cf02640 20 osd.0 110 _dispatch 0x5955340 osd_map(111..111 src has 1..111) 2026-02-11T23:31:01.240+0000 7f415cf02640 3 osd.0 110 handle_osd_map epochs [111,111], i have 110, src has [1,111] 2026-02-11T23:31:01.240+0000 7f415cf02640 10 osd.0 110 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:31:01.240+0000 7f415cf02640 10 osd.0 110 handle_osd_map got inc map for epoch 111 2026-02-11T23:31:01.240+0000 7f415cf02640 20 osd.0 110 got_full_map 111, nothing requested 2026-02-11T23:31:01.240+0000 7f415cf02640 20 osd.0 110 trim_maps: min=1 oldest_map=1 2026-02-11T23:31:01.240+0000 7f415cf02640 20 osd.0 110 handle_osd_map pg_num_history pg_num_history(e111 pg_nums {1={27=1},2={34=1},3={42=1},4={49=1},5={56=1},6={64=1},7={70=1},8={77=1},9={85=1},10={92=1}} deleted_pools 99,2,100,3,101,5,102,6,103,7,104,8,105,9,106,10) 2026-02-11T23:31:01.240+0000 7f415cf02640 10 snap_mapper record_purged_snaps purged_snaps {111={}} 2026-02-11T23:31:01.240+0000 7f415cf02640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:31:01.240+0000 7f415cf02640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e111 maps [1~111] lci=[5,111] tlb=1) 2026-02-11T23:31:01.241+0000 7f41591a6640 10 osd.0 110 _committed_osd_maps 111..111 2026-02-11T23:31:01.241+0000 7f41591a6640 10 osd.0 110 advance to epoch 111 (<= last 111 <= newest_map 111) 2026-02-11T23:31:01.241+0000 7f41591a6640 20 osd.0 111 consume_map version 111 2026-02-11T23:31:01.241+0000 7f41591a6640 20 osd.0 110 await_reserved_maps epoch:111 2026-02-11T23:31:01.241+0000 7f41591a6640 20 osd.0 110 await_reserved_maps done 0 2026-02-11T23:31:01.241+0000 7f41591a6640 20 osd.0 111 consume_map 111 -- publish done 2026-02-11T23:31:01.241+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:31:01.241+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x37582c0 2026-02-11T23:31:01.241+0000 7f41591a6640 20 osd.0 111 identify_splits_and_merges 4.0 e110 to e111 2026-02-11T23:31:01.241+0000 7f41591a6640 20 osd.0 111 identify_splits_and_merges 4.0 e110 to e111 pg_nums {49=1} 2026-02-11T23:31:01.241+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3700dc0 2026-02-11T23:31:01.241+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot 1.0 has no pg and waiting_for_split 2026-02-11T23:31:01.241+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:01.241+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:01.241+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:01.241+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:01.241+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:01.241+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:01.241+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:01.241+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:01.241+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:01.241+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:01.241+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:01.241+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:01.241+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:01.241+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:01.241+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:01.241+0000 7f41591a6640 20 osd.0 111 prune_pg_created 2026-02-11T23:31:01.241+0000 7f41591a6640 10 osd.0:0.consume_map 111 (was 110) 2026-02-11T23:31:01.241+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:31:01.241+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 pending_peering first epoch 111 <= 111, requeueing 2026-02-11T23:31:01.241+0000 7f41591a6640 20 osd.0:0._wake_pg_slot _wake_pg_slot 4.0 to_process <> waiting <> waiting_peering {111=} 2026-02-11T23:31:01.241+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:31:01.241+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 empty, pruning 2026-02-11T23:31:01.241+0000 7f41591a6640 10 osd.0:1.consume_map 111 (was 110) 2026-02-11T23:31:01.241+0000 7f41591a6640 10 osd.0:2.consume_map 111 (was 110) 2026-02-11T23:31:01.241+0000 7f41591a6640 10 osd.0:3.consume_map 111 (was 110) 2026-02-11T23:31:01.241+0000 7f41591a6640 10 osd.0:4.consume_map 111 (was 110) 2026-02-11T23:31:01.241+0000 7f41591a6640 10 osd.0:5.consume_map 111 (was 110) 2026-02-11T23:31:01.241+0000 7f41591a6640 10 osd.0:6.consume_map 111 (was 110) 2026-02-11T23:31:01.241+0000 7f41591a6640 10 osd.0:7.consume_map 111 (was 110) 2026-02-11T23:31:01.241+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:01.241+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 111 epoch_requested: 111 DeleteStart) class_id 2immediate prio 255 cost 10 e111) queued 2026-02-11T23:31:01.241+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:01.241+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 111 epoch_requested: 111 DeleteStart) class_id 2immediate prio 255 cost 10 e111) pg 0x5978000 2026-02-11T23:31:01.241+0000 7f41591a6640 10 osd.0 111 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:31:01.241+0000 7f41591a6640 15 osd.0 111 enqueue_peering_evt 4.0 epoch_sent: 111 epoch_requested: 111 NullEvt 2026-02-11T23:31:01.241+0000 7f414b98b640 10 osd.0 pg_epoch: 110 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.957548141s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] handle_advance_map: 111 2026-02-11T23:31:01.241+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 111 epoch_requested: 111 NullEvt) class_id 2 prio 255 cost 10 e111) 2026-02-11T23:31:01.241+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 4.0 110 -> 111 2026-02-11T23:31:01.241+0000 7f414b98b640 10 osd.0 pg_epoch: 110 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.957548141s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] PeeringState::advance_map handle_advance_map [4]/[4] -- 4/4 2026-02-11T23:31:01.241+0000 7f41591a6640 7 osd.0 111 activate_map version 111 2026-02-11T23:31:01.241+0000 7f414b98b640 10 osd.0 pg_epoch: 111 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.957548141s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] state: Started advmap 2026-02-11T23:31:01.241+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:01.241+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 111 epoch_requested: 111 NullEvt) class_id 2immediate prio 255 cost 10 e111) queued 2026-02-11T23:31:01.241+0000 7f4150194640 20 osd.0 111 agent_entry empty queue 2026-02-11T23:31:01.241+0000 7f414b98b640 10 osd.0 pg_epoch: 111 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.957548141s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:31:01.241+0000 7f414b98b640 20 osd.0 111 advance_pg new pool opts old pool opts 2026-02-11T23:31:01.241+0000 7f414b98b640 10 osd.0 pg_epoch: 111 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.957548141s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] handle_activate_map: epoch range: 111..111 2026-02-11T23:31:01.241+0000 7f414b98b640 10 osd.0 pg_epoch: 111 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.957548141s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] PeeringState::activate_map 2026-02-11T23:31:01.241+0000 7f414b98b640 20 osd.0 pg_epoch: 111 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.747307777s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 109 while current is 111 2026-02-11T23:31:01.241+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:01.241+0000 7f414b98b640 10 osd.0 pg_epoch: 111 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.747307777s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] do_peering_event: epoch_sent: 111 epoch_requested: 111 DeleteStart 2026-02-11T23:31:01.241+0000 7f414b98b640 5 osd.0 pg_epoch: 111 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.747307777s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] exit Started/Stray 0.211249 8 0.000059 2026-02-11T23:31:01.241+0000 7f414b98b640 5 osd.0 pg_epoch: 111 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.747307777s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] enter Started/ToDelete 2026-02-11T23:31:01.241+0000 7f414b98b640 5 osd.0 pg_epoch: 111 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.747307777s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] enter Started/ToDelete/WaitDeleteReserved 2026-02-11T23:31:01.241+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:01.241+0000 7f414b98b640 20 osd.0 111 get_con_osd_cluster to osd.4 from_epoch 111 2026-02-11T23:31:01.241+0000 7f414b98b640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:01.241+0000 7f414b98b640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:01.241+0000 7f414b98b640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6836/3331151400 map epoch 110 -> 111 (shared) 2026-02-11T23:31:01.241+0000 7f414b98b640 10 osd.0 111 send_incremental_map epoch range: (111, 111] to v2:127.0.0.1:6836/3331151400 2026-02-11T23:31:01.241+0000 7f414b98b640 10 osd.0 111 add_map_inc_bl 111 252 bytes 2026-02-11T23:31:01.241+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- osd_map(111..111 src has 1..111) -- 0x4aa2000 con 0x507ac00 2026-02-11T23:31:01.241+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_notify2(4.0 4.0 (query:111 sent:111 4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.747307777s) ([52,108] all_participants={0, 4} intervals=[([52,108] acting={0, 4})])) e111/111) -- 0x5a8f800 con 0x507ac00 2026-02-11T23:31:01.241+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:01.241+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:01.241+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 111 epoch_requested: 111 NullEvt) class_id 2immediate prio 255 cost 10 e111) pg 0x5978000 2026-02-11T23:31:01.241+0000 7f414f993640 10 osd.0 pg_epoch: 111 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.747307777s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] do_peering_event: epoch_sent: 111 epoch_requested: 111 NullEvt 2026-02-11T23:31:01.241+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:01.241+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:01.241+0000 7f4152999640 15 osd.0 111 enqueue_peering_evt 4.0 epoch_sent: 111 epoch_requested: 111 DeleteReserved 2026-02-11T23:31:01.241+0000 7f4152999640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 111 epoch_requested: 111 DeleteReserved) class_id 2 prio 255 cost 10 e111) 2026-02-11T23:31:01.241+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:01.241+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 111 epoch_requested: 111 DeleteReserved) class_id 2immediate prio 255 cost 10 e111) queued 2026-02-11T23:31:01.241+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:01.241+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 111 epoch_requested: 111 DeleteReserved) class_id 2immediate prio 255 cost 10 e111) pg 0x5978000 2026-02-11T23:31:01.241+0000 7f414f993640 10 osd.0 pg_epoch: 111 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.747307777s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] do_peering_event: epoch_sent: 111 epoch_requested: 111 DeleteReserved 2026-02-11T23:31:01.241+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:01.241+0000 7f414f993640 5 osd.0 pg_epoch: 111 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.747307777s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] exit Started/ToDelete/WaitDeleteReserved 0.000149 2 0.000013 2026-02-11T23:31:01.241+0000 7f414f993640 5 osd.0 pg_epoch: 111 pg[4.0( empty local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.747307777s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] enter Started/ToDelete/Deleting 2026-02-11T23:31:01.241+0000 7f414f993640 10 osd.0 pg_epoch: 111 pg[4.0( empty lb MIN local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.747307777s) [4] r=-1 lpr=109 DELETING pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] on_removal 2026-02-11T23:31:01.241+0000 7f414f993640 10 osd.0 pg_epoch: 111 pg[4.0( empty lb MIN local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.747307777s) [4] r=-1 lpr=109 DELETING pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] on_shutdown 2026-02-11T23:31:01.241+0000 7f414f993640 10 osd.0 pg_epoch: 111 pg[4.0( empty lb MIN local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.747307777s) [4] r=-1 lpr=109 DELETING pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:31:01.241+0000 7f414f993640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:31:01.241+0000 7f414f993640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:31:01.241+0000 7f414f993640 10 osd.0 pg_epoch: 111 pg[4.0( empty lb MIN local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.747307777s) [4] r=-1 lpr=109 DELETING pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] cancel_copy_ops 2026-02-11T23:31:01.241+0000 7f414f993640 10 osd.0 pg_epoch: 111 pg[4.0( empty lb MIN local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.747307777s) [4] r=-1 lpr=109 DELETING pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] cancel_flush_ops 2026-02-11T23:31:01.241+0000 7f414f993640 10 osd.0 pg_epoch: 111 pg[4.0( empty lb MIN local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.747307777s) [4] r=-1 lpr=109 DELETING pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] cancel_proxy_ops 2026-02-11T23:31:01.241+0000 7f414f993640 10 osd.0 pg_epoch: 111 pg[4.0( empty lb MIN local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.747307777s) [4] r=-1 lpr=109 DELETING pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] cancel_manifest_ops 2026-02-11T23:31:01.241+0000 7f414f993640 10 osd.0 pg_epoch: 111 pg[4.0( empty lb MIN local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.747307777s) [4] r=-1 lpr=109 DELETING pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] cancel_cls_gather_ops 2026-02-11T23:31:01.241+0000 7f414f993640 10 osd.0 pg_epoch: 111 pg[4.0( empty lb MIN local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.747307777s) [4] r=-1 lpr=109 DELETING pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] clear_backoffs 2026-02-11T23:31:01.241+0000 7f414f993640 20 osd.0 pg_epoch: 111 pg[4.0( empty lb MIN local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.747307777s) [4] r=-1 lpr=109 DELETING pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] exit NotTrimming 2026-02-11T23:31:01.241+0000 7f414f993640 20 osd.0 pg_epoch: 111 pg[4.0( empty lb MIN local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.747307777s) [4] r=-1 lpr=109 DELETING pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] enter NotTrimming 2026-02-11T23:31:01.241+0000 7f414f993640 10 osd.0 pg_epoch: 111 pg[4.0( empty lb MIN local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.747307777s) [4] r=-1 lpr=109 DELETING pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] on_change 2026-02-11T23:31:01.241+0000 7f414f993640 10 osd.0 pg_epoch: 111 pg[4.0( empty lb MIN local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.747307777s) [4] r=-1 lpr=109 DELETING pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] clear_async_reads 2026-02-11T23:31:01.241+0000 7f414f993640 20 osd.0 pg_epoch: 111 pg[4.0( empty lb MIN local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.747307777s) [4] r=-1 lpr=109 DELETING pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] clear_primary_state 2026-02-11T23:31:01.241+0000 7f414f993640 10 osd.0 pg_epoch: 111 pg[4.0( empty lb MIN local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.747307777s) [4] r=-1 lpr=109 DELETING pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] release_backoffs [4:00000000::::0,MAX) 2026-02-11T23:31:01.241+0000 7f414f993640 10 osd.0 pg_epoch: 111 pg[4.0( empty lb MIN local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.747307777s) [4] r=-1 lpr=109 DELETING pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:31:01.241+0000 7f414f993640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:31:01.241+0000 7f414f993640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:31:01.241+0000 7f414f993640 20 osd.0 pg_epoch: 111 pg[4.0( empty lb MIN local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.747307777s) [4] r=-1 lpr=109 DELETING pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] agent_stop 2026-02-11T23:31:01.241+0000 7f414f993640 10 osd.0 pg_epoch: 111 pg[4.0( empty lb MIN local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.747307777s) [4] r=-1 lpr=109 DELETING pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] cancel_recovery 2026-02-11T23:31:01.241+0000 7f414f993640 10 osd.0 pg_epoch: 111 pg[4.0( empty lb MIN local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.747307777s) [4] r=-1 lpr=109 DELETING pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] clear_recovery_state 2026-02-11T23:31:01.241+0000 7f414f993640 15 osd.0 pg_epoch: 111 pg[4.0( empty lb MIN local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.747307777s) [4] r=-1 lpr=109 DELETING pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] _clear_recovery_state 2026-02-11T23:31:01.241+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:01.241+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:01.241+0000 7f414f993640 10 osd.0 111 queue_for_pg_delete on 4.0 e 111 2026-02-11T23:31:01.241+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:01.241+0000 7f414f993640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGDelete(pgid=4.0 epoch_queued=111) class_id 1 prio 5 cost 1048576 e111) 2026-02-11T23:31:01.241+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:01.241+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGDelete(4.0 e111) class_id 1background_best_effort prio 5 qos_cost 1048576 cost 1048576 e111) queued 2026-02-11T23:31:01.241+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:01.241+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGDelete(4.0 e111) class_id 1background_best_effort prio 5 qos_cost 1048576 cost 1048576 e111) pg 0x5978000 2026-02-11T23:31:01.241+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:01.241+0000 7f414f993640 10 osd.0 pg_epoch: 111 pg[4.0( empty lb MIN local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.747307777s) [4] r=-1 lpr=109 DELETING pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] do_peering_event: epoch_sent: 111 epoch_requested: 111 DeleteSome 2026-02-11T23:31:01.241+0000 7f414f993640 10 osd.0 pg_epoch: 111 pg[4.0( empty lb MIN local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.747307777s) [4] r=-1 lpr=109 DELETING pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] do_delete_work 2026-02-11T23:31:01.241+0000 7f414f993640 20 osd.0 pg_epoch: 111 pg[4.0( empty lb MIN local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.747307777s) [4] r=-1 lpr=109 DELETING pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] do_delete_work [#4:00000000::::head#] 2026-02-11T23:31:01.241+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:01.241+0000 7f414f993640 20 osd.0 111 try_finish_pg_delete 4.0 0x5978000 2026-02-11T23:31:01.241+0000 7f414f993640 10 osd.0:0._detach_pg 4.0 0x5978000 2026-02-11T23:31:01.241+0000 7f414f993640 5 osd.0 pg_epoch: 111 pg[4.0( empty lb MIN local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.747307777s) [4] r=-1 lpr=109 DELETING pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] exit Started/ToDelete/Deleting 0.000402 1 0.000057 2026-02-11T23:31:01.241+0000 7f414f993640 5 osd.0 pg_epoch: 111 pg[4.0( empty lb MIN local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.747307777s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] exit Started/ToDelete 0.000560 0 0.000000 2026-02-11T23:31:01.241+0000 7f414f993640 5 osd.0 pg_epoch: 111 pg[4.0( empty lb MIN local-lis/les=52/53 n=0 ec=49/49 lis/c=52/52 les/c/f=53/53/0 sis=109 pruub=14.747307777s) [4] r=-1 lpr=109 pi=[52,109)/1 crt=0'0 unknown NOTIFY pruub 58.665332794s@ mbc={}] exit Started 0.211831 0 0.000000 2026-02-11T23:31:01.241+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:01.241+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:01.241+0000 7f414f993640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:31:01.241+0000 7f414f993640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:31:01.241+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:01.241+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:01.326+0000 7f41633eb640 10 osd.0 111 tick 2026-02-11T23:31:01.326+0000 7f41633eb640 20 osd.0 111 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:01.327+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:01.327+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:01.327+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:31.329009+0000) 2026-02-11T23:31:02.198+0000 7f4162bea640 10 osd.0 111 tick_without_osd_lock 2026-02-11T23:31:02.198+0000 7f4162bea640 0 osd.0 111 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:31:02.198+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:02.198+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:02.198+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:02.198+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.341 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:02.198+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:02.198, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:02.198+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:02.198+0000 7f4162bea640 10 osd.0 111 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:02.198+0000 7f4162bea640 20 osd.0 111 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:02.198+0000 7f4162bea640 10 osd.0 111 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:02.198+0000 7f4162bea640 20 osd.0 111 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:02.327+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:02.327+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:02.327+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:32.329070+0000) 2026-02-11T23:31:02.372+0000 7f41633eb640 10 osd.0 111 tick 2026-02-11T23:31:02.372+0000 7f41633eb640 20 osd.0 111 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:02.487+0000 7f41683f5640 10 monclient: _renew_subs 2026-02-11T23:31:02.487+0000 7f41683f5640 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:31:02.487+0000 7f41683f5640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> v2:127.0.0.1:7180/0 -- mon_subscribe({osdmap=112}) -- 0x3814b60 con 0x4acbc00 2026-02-11T23:31:02.740+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.1 127.0.0.1:0/3043114013 16 ==== osd_ping(ping e110 up_from 10 ping_stamp 2026-02-11T23:31:02.741942+0000/43.405689240s send_stamp 43.405689240s delta_ub -2.011240244s) ==== 2033+0+0 (crc 0 0 0) 0x4c9bb00 con 0x4aca000 2026-02-11T23:31:02.740+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.1 127.0.0.1:0/3043114013 16 ==== osd_ping(ping e110 up_from 10 ping_stamp 2026-02-11T23:31:02.741942+0000/43.405689240s send_stamp 43.405689240s delta_ub -2.011240244s) ==== 2033+0+0 (crc 0 0 0) 0x5af9380 con 0x4aca400 2026-02-11T23:31:02.740+0000 7f416cbfe640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011762381s,-2.011240244s]) 2026-02-11T23:31:02.740+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:02.741942+0000/43.405689240s send_stamp 45.417449951s delta_ub 2.011762381s) -- 0x4c9a300 con 0x4aca000 2026-02-11T23:31:02.740+0000 7f416cbfe640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:02.740+0000 7f416cbfe640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:02.740+0000 7f416cbfe640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:02.740+0000 7f416cbfe640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/3043114013 map epoch 110 -> 111 (shared) 2026-02-11T23:31:02.740+0000 7f416cbfe640 10 osd.0 111 send_incremental_map epoch range: (111, 111] to v2:127.0.0.1:6812/3043114013 2026-02-11T23:31:02.740+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- osd_map(111..111 src has 1..111) -- 0x5955500 con 0x4acb000 2026-02-11T23:31:02.740+0000 7f416c3fd640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011806011s,-2.011240244s]) 2026-02-11T23:31:02.740+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:02.741942+0000/43.405689240s send_stamp 45.417491913s delta_ub 2.011806011s) -- 0x5af8600 con 0x4aca400 2026-02-11T23:31:02.740+0000 7f416c3fd640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:02.740+0000 7f416c3fd640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:02.740+0000 7f416c3fd640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:02.740+0000 7f416c3fd640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:03.205+0000 7f415b6ff640 20 osd.0 111 reports for 0 queries 2026-02-11T23:31:03.205+0000 7f415b6ff640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] -- mgrreport(unknown.0 +0-0 packed 3294 daemon_metrics=2) -- 0x4f9f180 con 0x4aca800 2026-02-11T23:31:03.205+0000 7f415b6ff640 15 osd.0 111 collect_pg_stats 2026-02-11T23:31:03.205+0000 7f415b6ff640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] -- pg_stats(0 pgs seq 21474836494 v 0) -- 0x5afc600 con 0x4aca800 2026-02-11T23:31:03.227+0000 7f4162bea640 10 osd.0 111 tick_without_osd_lock 2026-02-11T23:31:03.227+0000 7f4162bea640 0 osd.0 111 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:31:03.227+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:03.227+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:03.227+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:03.227+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.314 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:03.227+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:03.229, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:03.227+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:03.227+0000 7f4162bea640 10 osd.0 111 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:03.227+0000 7f4162bea640 20 osd.0 111 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:03.227+0000 7f4162bea640 10 osd.0 111 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:03.227+0000 7f4162bea640 20 osd.0 111 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:03.327+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:03.327+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:03.327+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:33.329137+0000) 2026-02-11T23:31:03.408+0000 7f41633eb640 10 osd.0 111 tick 2026-02-11T23:31:03.408+0000 7f41633eb640 20 osd.0 111 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:03.647+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.3 127.0.0.1:0/2699377081 11 ==== osd_ping(ping e110 up_from 19 ping_stamp 2026-02-11T23:31:03.648900+0000/39.104839325s send_stamp 39.104839325s delta_ub -7.219288349s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a300 con 0x48cc800 2026-02-11T23:31:03.647+0000 7f416cbfe640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219568729s,-7.219288349s]) 2026-02-11T23:31:03.647+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.3 127.0.0.1:0/2699377081 11 ==== osd_ping(ping e110 up_from 19 ping_stamp 2026-02-11T23:31:03.648900+0000/39.104839325s send_stamp 39.104839325s delta_ub -7.219288349s) ==== 2033+0+0 (crc 0 0 0) 0x5af8600 con 0x48cc400 2026-02-11T23:31:03.647+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:03.648900+0000/39.104839325s send_stamp 46.324409485s delta_ub 7.219568729s) -- 0x4c9bb00 con 0x48cc800 2026-02-11T23:31:03.647+0000 7f416cbfe640 20 osd.0 111 get_con_osd_cluster to osd.3 from_epoch 111 2026-02-11T23:31:03.647+0000 7f416cbfe640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:03.647+0000 7f416cbfe640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:03.647+0000 7f416cbfe640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6828/2699377081 updating session's projected_epoch from 108 to ping map epoch of 110 2026-02-11T23:31:03.647+0000 7f416cbfe640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6828/2699377081 map epoch 110 -> 111 (shared) 2026-02-11T23:31:03.647+0000 7f416cbfe640 10 osd.0 111 send_incremental_map epoch range: (111, 111] to v2:127.0.0.1:6828/2699377081 2026-02-11T23:31:03.647+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6828/2699377081,v1:127.0.0.1:6829/2699377081] -- osd_map(111..111 src has 1..111) -- 0x5954c40 con 0x48ccc00 2026-02-11T23:31:03.647+0000 7f416c3fd640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219613552s,-7.219288349s]) 2026-02-11T23:31:03.647+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:03.648900+0000/39.104839325s send_stamp 46.324455261s delta_ub 7.219613552s) -- 0x5af9380 con 0x48cc400 2026-02-11T23:31:03.647+0000 7f416c3fd640 20 osd.0 111 get_con_osd_cluster to osd.3 from_epoch 111 2026-02-11T23:31:03.647+0000 7f416c3fd640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:03.647+0000 7f416c3fd640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:03.647+0000 7f416c3fd640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:03.929+0000 7f4147983640 5 osd.0 111 heartbeat osd_stat(store_statfs(0x384000/0x0/0x384000, data 0x7f242/0x120000, compress 0x0/0x0/0x0, omap 0xa403, meta 0x1a45bfd), peers [1,2,3,4] op hist []) 2026-02-11T23:31:03.929+0000 7f4147983640 20 osd.0 111 check_full_status cur ratio 0, physical ratio 0, new state none 2026-02-11T23:31:03.929+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6816/3043114013,v1:127.0.0.1:6817/3043114013] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:03.930406+0000/46.605812073s send_stamp 46.605812073s delta_ub 2.011806011s) -- 0x5af8000 con 0x48c4400 2026-02-11T23:31:03.929+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6814/3043114013,v1:127.0.0.1:6815/3043114013] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:03.930406+0000/46.605812073s send_stamp 46.605812073s delta_ub 2.011806011s) -- 0x5af8900 con 0x4640c00 2026-02-11T23:31:03.929+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6824/2365263801,v1:127.0.0.1:6825/2365263801] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:03.930406+0000/46.605812073s send_stamp 46.605812073s delta_ub 4.566872597s) -- 0x5af8480 con 0x48c4000 2026-02-11T23:31:03.929+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6822/2365263801,v1:127.0.0.1:6823/2365263801] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:03.930406+0000/46.605812073s send_stamp 46.605812073s delta_ub 4.566872597s) -- 0x5af8300 con 0x48cd000 2026-02-11T23:31:03.929+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6832/2699377081,v1:127.0.0.1:6833/2699377081] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:03.930406+0000/46.605812073s send_stamp 46.605812073s delta_ub 7.219613552s) -- 0x5af8c00 con 0x48cd400 2026-02-11T23:31:03.929+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6830/2699377081,v1:127.0.0.1:6831/2699377081] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:03.930406+0000/46.605812073s send_stamp 46.605812073s delta_ub 7.219613552s) -- 0x5af9500 con 0x48cd800 2026-02-11T23:31:03.929+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6840/3331151400,v1:127.0.0.1:6841/3331151400] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:03.930406+0000/46.605812073s send_stamp 46.605812073s delta_ub 9.836714745s) -- 0x5af9680 con 0x507bc00 2026-02-11T23:31:03.929+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6838/3331151400,v1:127.0.0.1:6839/3331151400] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:03.930406+0000/46.605812073s send_stamp 46.605812073s delta_ub 9.836714745s) -- 0x5af8d80 con 0x5282000 2026-02-11T23:31:03.929+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6822/2365263801 13 ==== osd_ping(ping_reply e111 up_from 14 ping_stamp 2026-02-11T23:31:03.930406+0000/46.605812073s send_stamp 42.039108276s delta_ub -4.566703320s) ==== 2033+0+0 (crc 0 0 0) 0x5af9680 con 0x48cd000 2026-02-11T23:31:03.929+0000 7f416abfa640 20 osd.0 111 get_con_osd_cluster to osd.2 from_epoch 111 2026-02-11T23:31:03.929+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6814/3043114013 14 ==== osd_ping(ping_reply e111 up_from 10 ping_stamp 2026-02-11T23:31:03.930406+0000/46.605812073s send_stamp 44.594257355s delta_ub -2.011555672s) ==== 2033+0+0 (crc 0 0 0) 0x5af8d80 con 0x4640c00 2026-02-11T23:31:03.929+0000 7f416abfa640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:03.929+0000 7f416abfa640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:03.929+0000 7f416abfa640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6820/2365263801 updating session's projected_epoch from 109 to ping map epoch of 111 2026-02-11T23:31:03.929+0000 7f416abfa640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:03.929+0000 7f416abfa640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566866398s,-4.566703320s]) 2026-02-11T23:31:03.929+0000 7f416c3fd640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:03.929+0000 7f416cbfe640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6824/2365263801 13 ==== osd_ping(ping_reply e111 up_from 14 ping_stamp 2026-02-11T23:31:03.930406+0000/46.605812073s send_stamp 42.039131165s delta_ub -4.566679001s) ==== 2033+0+0 (crc 0 0 0) 0x5af8480 con 0x48c4000 2026-02-11T23:31:03.929+0000 7f416c3fd640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:03.929+0000 7f416c3fd640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:03.929+0000 7f416c3fd640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:03.929+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6816/3043114013 14 ==== osd_ping(ping_reply e111 up_from 10 ping_stamp 2026-02-11T23:31:03.930406+0000/46.605812073s send_stamp 44.594280243s delta_ub -2.011533022s) ==== 2033+0+0 (crc 0 0 0) 0x5af9680 con 0x48c4400 2026-02-11T23:31:03.929+0000 7f416c3fd640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011736393s,-2.011555672s]) 2026-02-11T23:31:03.929+0000 7f416cbfe640 20 osd.0 111 get_con_osd_cluster to osd.2 from_epoch 111 2026-02-11T23:31:03.929+0000 7f416cbfe640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:03.929+0000 7f416cbfe640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:03.929+0000 7f416cbfe640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:03.929+0000 7f416cbfe640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566877365s,-4.566679001s]) 2026-02-11T23:31:03.929+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6830/2699377081 13 ==== osd_ping(ping_reply e111 up_from 19 ping_stamp 2026-02-11T23:31:03.930406+0000/46.605812073s send_stamp 39.386466980s delta_ub -7.219345093s) ==== 2033+0+0 (crc 0 0 0) 0x5af8d80 con 0x48cd800 2026-02-11T23:31:03.929+0000 7f416c3fd640 20 osd.0 111 get_con_osd_cluster to osd.3 from_epoch 111 2026-02-11T23:31:03.929+0000 7f416c3fd640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:03.929+0000 7f416c3fd640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:03.929+0000 7f416c3fd640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:03.929+0000 7f416c3fd640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219554901s,-7.219345093s]) 2026-02-11T23:31:03.929+0000 7f416abfa640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:03.929+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6838/3331151400 12 ==== osd_ping(ping_reply e111 up_from 23 ping_stamp 2026-02-11T23:31:03.930406+0000/46.605812073s send_stamp 36.769309998s delta_ub -9.836502075s) ==== 2033+0+0 (crc 0 0 0) 0x5af8d80 con 0x5282000 2026-02-11T23:31:03.929+0000 7f416abfa640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:03.929+0000 7f416abfa640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:03.929+0000 7f416abfa640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:03.929+0000 7f416abfa640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011751413s,-2.011533022s]) 2026-02-11T23:31:03.929+0000 7f416c3fd640 20 osd.0 111 get_con_osd_cluster to osd.4 from_epoch 111 2026-02-11T23:31:03.929+0000 7f416c3fd640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:03.929+0000 7f416c3fd640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:03.929+0000 7f416c3fd640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:03.929+0000 7f416c3fd640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836733818s,-9.836502075s]) 2026-02-11T23:31:03.929+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6832/2699377081 13 ==== osd_ping(ping_reply e111 up_from 19 ping_stamp 2026-02-11T23:31:03.930406+0000/46.605812073s send_stamp 39.386493683s delta_ub -7.219320297s) ==== 2033+0+0 (crc 0 0 0) 0x5af9680 con 0x48cd400 2026-02-11T23:31:03.929+0000 7f416abfa640 20 osd.0 111 get_con_osd_cluster to osd.3 from_epoch 111 2026-02-11T23:31:03.929+0000 7f416abfa640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:03.929+0000 7f416abfa640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:03.929+0000 7f416abfa640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:03.929+0000 7f416abfa640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219558716s,-7.219320297s]) 2026-02-11T23:31:03.929+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6840/3331151400 12 ==== osd_ping(ping_reply e111 up_from 23 ping_stamp 2026-02-11T23:31:03.930406+0000/46.605812073s send_stamp 36.769336700s delta_ub -9.836475372s) ==== 2033+0+0 (crc 0 0 0) 0x5af9680 con 0x507bc00 2026-02-11T23:31:03.929+0000 7f416abfa640 20 osd.0 111 get_con_osd_cluster to osd.4 from_epoch 111 2026-02-11T23:31:03.929+0000 7f416abfa640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:03.929+0000 7f416abfa640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:03.929+0000 7f416abfa640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:03.929+0000 7f416abfa640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836729050s,-9.836475372s]) 2026-02-11T23:31:03.964+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.4 127.0.0.1:0/3331151400 10 ==== osd_ping(ping e111 up_from 23 ping_stamp 2026-02-11T23:31:03.965848+0000/36.804615021s send_stamp 36.804615021s delta_ub -9.836475372s) ==== 2033+0+0 (crc 0 0 0) 0x5af9680 con 0x507a400 2026-02-11T23:31:03.964+0000 7f416abfa640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836684227s,-9.836475372s]) 2026-02-11T23:31:03.964+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.4 127.0.0.1:0/3331151400 10 ==== osd_ping(ping e111 up_from 23 ping_stamp 2026-02-11T23:31:03.965848+0000/36.804615021s send_stamp 36.804615021s delta_ub -9.836475372s) ==== 2033+0+0 (crc 0 0 0) 0x5af8d80 con 0x507a800 2026-02-11T23:31:03.964+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:03.965848+0000/36.804615021s send_stamp 46.641300201s delta_ub 9.836684227s) -- 0x5af8c00 con 0x507a400 2026-02-11T23:31:03.964+0000 7f416abfa640 20 osd.0 111 get_con_osd_cluster to osd.4 from_epoch 111 2026-02-11T23:31:03.964+0000 7f416abfa640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:03.964+0000 7f416abfa640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:03.964+0000 7f416abfa640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:03.964+0000 7f416c3fd640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836701393s,-9.836475372s]) 2026-02-11T23:31:03.964+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:03.965848+0000/36.804615021s send_stamp 46.641315460s delta_ub 9.836701393s) -- 0x5af9500 con 0x507a800 2026-02-11T23:31:03.964+0000 7f416c3fd640 20 osd.0 111 get_con_osd_cluster to osd.4 from_epoch 111 2026-02-11T23:31:03.964+0000 7f416c3fd640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:03.964+0000 7f416c3fd640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:03.964+0000 7f416c3fd640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:04.254+0000 7f4162bea640 10 osd.0 111 tick_without_osd_lock 2026-02-11T23:31:04.254+0000 7f4162bea640 0 osd.0 111 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:31:04.254+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:04.254+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:04.254+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:04.254+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.314 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:04.254+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:04.256, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:04.254+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:04.254+0000 7f4162bea640 10 osd.0 111 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:04.254+0000 7f4162bea640 20 osd.0 111 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:04.254+0000 7f4162bea640 10 osd.0 111 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:04.254+0000 7f4162bea640 20 osd.0 111 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:04.327+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:04.327+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:04.327+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:34.329189+0000) 2026-02-11T23:31:04.362+0000 7f41633eb640 10 osd.0 111 tick 2026-02-11T23:31:04.362+0000 7f41633eb640 20 osd.0 111 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:04.694+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.2 127.0.0.1:0/2365263801 10 ==== osd_ping(ping e111 up_from 14 ping_stamp 2026-02-11T23:31:04.696068+0000/42.804664612s send_stamp 42.804664612s delta_ub -4.566679001s) ==== 2033+0+0 (crc 0 0 0) 0x5af8480 con 0x4641000 2026-02-11T23:31:04.694+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.2 127.0.0.1:0/2365263801 10 ==== osd_ping(ping e111 up_from 14 ping_stamp 2026-02-11T23:31:04.696068+0000/42.804664612s send_stamp 42.804664612s delta_ub -4.566679001s) ==== 2033+0+0 (crc 0 0 0) 0x5af9500 con 0x48cc000 2026-02-11T23:31:04.694+0000 7f416cbfe640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566896915s,-4.566679001s]) 2026-02-11T23:31:04.694+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:04.696068+0000/42.804664612s send_stamp 47.371562958s delta_ub 4.566896915s) -- 0x4c9bb00 con 0x4641000 2026-02-11T23:31:04.694+0000 7f416cbfe640 20 osd.0 111 get_con_osd_cluster to osd.2 from_epoch 111 2026-02-11T23:31:04.694+0000 7f416cbfe640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:04.694+0000 7f416cbfe640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:04.694+0000 7f416cbfe640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:04.694+0000 7f416c3fd640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566931248s,-4.566679001s]) 2026-02-11T23:31:04.694+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:04.696068+0000/42.804664612s send_stamp 47.371597290s delta_ub 4.566931248s) -- 0x5af8d80 con 0x48cc000 2026-02-11T23:31:04.694+0000 7f416c3fd640 20 osd.0 111 get_con_osd_cluster to osd.2 from_epoch 111 2026-02-11T23:31:04.694+0000 7f416c3fd640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:04.694+0000 7f416c3fd640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:04.694+0000 7f416c3fd640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:05.064+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.4 127.0.0.1:0/3331151400 11 ==== osd_ping(ping e111 up_from 23 ping_stamp 2026-02-11T23:31:05.066001+0000/37.904769897s send_stamp 37.904769897s delta_ub -9.836586952s) ==== 2033+0+0 (crc 0 0 0) 0x5af8d80 con 0x507a800 2026-02-11T23:31:05.064+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.4 127.0.0.1:0/3331151400 11 ==== osd_ping(ping e111 up_from 23 ping_stamp 2026-02-11T23:31:05.066001+0000/37.904769897s send_stamp 37.904769897s delta_ub -9.836586952s) ==== 2033+0+0 (crc 0 0 0) 0x5af8c00 con 0x507a400 2026-02-11T23:31:05.064+0000 7f416c3fd640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836717606s,-9.836586952s]) 2026-02-11T23:31:05.064+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:05.066001+0000/37.904769897s send_stamp 47.741485596s delta_ub 9.836717606s) -- 0x5af9500 con 0x507a800 2026-02-11T23:31:05.064+0000 7f416c3fd640 20 osd.0 111 get_con_osd_cluster to osd.4 from_epoch 111 2026-02-11T23:31:05.064+0000 7f416c3fd640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:05.064+0000 7f416c3fd640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:05.064+0000 7f416c3fd640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:05.064+0000 7f416abfa640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836737633s,-9.836586952s]) 2026-02-11T23:31:05.064+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:05.066001+0000/37.904769897s send_stamp 47.741504669s delta_ub 9.836737633s) -- 0x5af9680 con 0x507a400 2026-02-11T23:31:05.064+0000 7f416abfa640 20 osd.0 111 get_con_osd_cluster to osd.4 from_epoch 111 2026-02-11T23:31:05.064+0000 7f416abfa640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:05.064+0000 7f416abfa640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:05.064+0000 7f416abfa640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:05.242+0000 7f4162bea640 10 osd.0 111 tick_without_osd_lock 2026-02-11T23:31:05.242+0000 7f4162bea640 0 osd.0 111 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:31:05.242+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:05.242+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:05.242+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:05.242+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.314 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:05.242+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:05.243, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:05.242+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:05.242+0000 7f4162bea640 10 osd.0 111 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:05.242+0000 7f4162bea640 20 osd.0 111 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:05.242+0000 7f4162bea640 10 osd.0 111 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:05.242+0000 7f4162bea640 20 osd.0 111 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:05.327+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:05.327+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:05.327+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:35.329265+0000) 2026-02-11T23:31:05.328+0000 7f41633eb640 10 osd.0 111 tick 2026-02-11T23:31:05.328+0000 7f41633eb640 20 osd.0 111 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:05.564+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.4 127.0.0.1:0/3331151400 12 ==== osd_ping(ping e111 up_from 23 ping_stamp 2026-02-11T23:31:05.566172+0000/38.404941559s send_stamp 38.404941559s delta_ub -9.836578369s) ==== 2033+0+0 (crc 0 0 0) 0x5af9500 con 0x507a800 2026-02-11T23:31:05.564+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.4 127.0.0.1:0/3331151400 12 ==== osd_ping(ping e111 up_from 23 ping_stamp 2026-02-11T23:31:05.566172+0000/38.404941559s send_stamp 38.404941559s delta_ub -9.836578369s) ==== 2033+0+0 (crc 0 0 0) 0x5af9680 con 0x507a400 2026-02-11T23:31:05.564+0000 7f416c3fd640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836713791s,-9.836578369s]) 2026-02-11T23:31:05.564+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:05.566172+0000/38.404941559s send_stamp 48.241657257s delta_ub 9.836713791s) -- 0x5af8d80 con 0x507a800 2026-02-11T23:31:05.564+0000 7f416c3fd640 20 osd.0 111 get_con_osd_cluster to osd.4 from_epoch 111 2026-02-11T23:31:05.564+0000 7f416c3fd640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:05.564+0000 7f416c3fd640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:05.564+0000 7f416c3fd640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:05.564+0000 7f416abfa640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836733818s,-9.836578369s]) 2026-02-11T23:31:05.564+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:05.566172+0000/38.404941559s send_stamp 48.241672516s delta_ub 9.836733818s) -- 0x5af8c00 con 0x507a400 2026-02-11T23:31:05.564+0000 7f416abfa640 20 osd.0 111 get_con_osd_cluster to osd.4 from_epoch 111 2026-02-11T23:31:05.564+0000 7f416abfa640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:05.564+0000 7f416abfa640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:05.564+0000 7f416abfa640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:06.228+0000 7f4162bea640 10 osd.0 111 tick_without_osd_lock 2026-02-11T23:31:06.228+0000 7f4162bea640 0 osd.0 111 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:31:06.228+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:06.228+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.314 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:06.228+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:06.229, recovery is active?:false restrictions:<....> 2026-02-11T23:31:06.228+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:06.228+0000 7f4162bea640 10 osd.0 111 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:06.228+0000 7f4162bea640 20 osd.0 111 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:06.228+0000 7f4162bea640 10 osd.0 111 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:06.228+0000 7f4162bea640 20 osd.0 111 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:06.324+0000 7f41633eb640 10 osd.0 111 tick 2026-02-11T23:31:06.324+0000 7f41633eb640 20 osd.0 111 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:06.328+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:06.328+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:06.328+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:36.329334+0000) 2026-02-11T23:31:06.395+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.2 127.0.0.1:0/2365263801 11 ==== osd_ping(ping e111 up_from 14 ping_stamp 2026-02-11T23:31:06.396249+0000/44.504844666s send_stamp 44.504844666s delta_ub -4.566732883s) ==== 2033+0+0 (crc 0 0 0) 0x4c9bb00 con 0x4641000 2026-02-11T23:31:06.395+0000 7f416cbfe640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566889286s,-4.566732883s]) 2026-02-11T23:31:06.395+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.2 127.0.0.1:0/2365263801 11 ==== osd_ping(ping e111 up_from 14 ping_stamp 2026-02-11T23:31:06.396249+0000/44.504844666s send_stamp 44.504844666s delta_ub -4.566732883s) ==== 2033+0+0 (crc 0 0 0) 0x5af8d80 con 0x48cc000 2026-02-11T23:31:06.395+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:06.396249+0000/44.504844666s send_stamp 49.071735382s delta_ub 4.566889286s) -- 0x5af8480 con 0x4641000 2026-02-11T23:31:06.395+0000 7f416cbfe640 20 osd.0 111 get_con_osd_cluster to osd.2 from_epoch 111 2026-02-11T23:31:06.395+0000 7f416cbfe640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:06.395+0000 7f416cbfe640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:06.395+0000 7f416cbfe640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:06.395+0000 7f416c3fd640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566909313s,-4.566732883s]) 2026-02-11T23:31:06.395+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:06.396249+0000/44.504844666s send_stamp 49.071754456s delta_ub 4.566909313s) -- 0x5af9500 con 0x48cc000 2026-02-11T23:31:06.395+0000 7f416c3fd640 20 osd.0 111 get_con_osd_cluster to osd.2 from_epoch 111 2026-02-11T23:31:06.395+0000 7f416c3fd640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:06.395+0000 7f416c3fd640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:06.395+0000 7f416c3fd640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:06.829+0000 7f4147983640 5 osd.0 111 heartbeat osd_stat(store_statfs(0x384000/0x0/0x384000, data 0x7f242/0x120000, compress 0x0/0x0/0x0, omap 0xa403, meta 0x1a45bfd), peers [1,2,3,4] op hist []) 2026-02-11T23:31:06.829+0000 7f4147983640 20 osd.0 111 check_full_status cur ratio 0, physical ratio 0, new state none 2026-02-11T23:31:06.829+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6816/3043114013,v1:127.0.0.1:6817/3043114013] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:06.830602+0000/49.506008148s send_stamp 49.506008148s delta_ub 2.011751413s) -- 0x5af8a80 con 0x48c4400 2026-02-11T23:31:06.829+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6814/3043114013,v1:127.0.0.1:6815/3043114013] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:06.830602+0000/49.506008148s send_stamp 49.506008148s delta_ub 2.011751413s) -- 0x5af8180 con 0x4640c00 2026-02-11T23:31:06.829+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6824/2365263801,v1:127.0.0.1:6825/2365263801] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:06.830602+0000/49.506008148s send_stamp 49.506008148s delta_ub 4.566909313s) -- 0x5af9800 con 0x48c4000 2026-02-11T23:31:06.829+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6822/2365263801,v1:127.0.0.1:6823/2365263801] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:06.830602+0000/49.506008148s send_stamp 49.506008148s delta_ub 4.566909313s) -- 0x5af9980 con 0x48cd000 2026-02-11T23:31:06.829+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6832/2699377081,v1:127.0.0.1:6833/2699377081] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:06.830602+0000/49.506008148s send_stamp 49.506008148s delta_ub 7.219558716s) -- 0x4c9a180 con 0x48cd400 2026-02-11T23:31:06.829+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6830/2699377081,v1:127.0.0.1:6831/2699377081] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:06.830602+0000/49.506008148s send_stamp 49.506008148s delta_ub 7.219558716s) -- 0x4c9b980 con 0x48cd800 2026-02-11T23:31:06.829+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6840/3331151400,v1:127.0.0.1:6841/3331151400] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:06.830602+0000/49.506008148s send_stamp 49.506008148s delta_ub 9.836733818s) -- 0x4c9b200 con 0x507bc00 2026-02-11T23:31:06.829+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6838/3331151400,v1:127.0.0.1:6839/3331151400] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:06.830602+0000/49.506008148s send_stamp 49.506008148s delta_ub 9.836733818s) -- 0x4c9b380 con 0x5282000 2026-02-11T23:31:06.829+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6816/3043114013 15 ==== osd_ping(ping_reply e111 up_from 10 ping_stamp 2026-02-11T23:31:06.830602+0000/49.506008148s send_stamp 47.494403839s delta_ub -2.011605501s) ==== 2033+0+0 (crc 0 0 0) 0x4c9b200 con 0x48c4400 2026-02-11T23:31:06.829+0000 7f416abfa640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:06.829+0000 7f416abfa640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:06.829+0000 7f416abfa640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:06.829+0000 7f416abfa640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:06.829+0000 7f416abfa640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011696577s,-2.011605501s]) 2026-02-11T23:31:06.829+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6814/3043114013 15 ==== osd_ping(ping_reply e111 up_from 10 ping_stamp 2026-02-11T23:31:06.830602+0000/49.506008148s send_stamp 47.494415283s delta_ub -2.011593103s) ==== 2033+0+0 (crc 0 0 0) 0x4c9b380 con 0x4640c00 2026-02-11T23:31:06.829+0000 7f416c3fd640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:06.829+0000 7f416c3fd640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:06.829+0000 7f416c3fd640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:06.829+0000 7f416c3fd640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:06.829+0000 7f416c3fd640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011694908s,-2.011593103s]) 2026-02-11T23:31:06.829+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6838/3331151400 13 ==== osd_ping(ping_reply e111 up_from 23 ping_stamp 2026-02-11T23:31:06.830602+0000/49.506008148s send_stamp 39.669452667s delta_ub -9.836554527s) ==== 2033+0+0 (crc 0 0 0) 0x4c9b380 con 0x5282000 2026-02-11T23:31:06.829+0000 7f416c3fd640 20 osd.0 111 get_con_osd_cluster to osd.4 from_epoch 111 2026-02-11T23:31:06.829+0000 7f416c3fd640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:06.829+0000 7f416c3fd640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:06.829+0000 7f416c3fd640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:06.829+0000 7f416c3fd640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836675644s,-9.836554527s]) 2026-02-11T23:31:06.829+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6822/2365263801 14 ==== osd_ping(ping_reply e111 up_from 14 ping_stamp 2026-02-11T23:31:06.830602+0000/49.506008148s send_stamp 44.939289093s delta_ub -4.566720486s) ==== 2033+0+0 (crc 0 0 0) 0x4c9b200 con 0x48cd000 2026-02-11T23:31:06.829+0000 7f416abfa640 20 osd.0 111 get_con_osd_cluster to osd.2 from_epoch 111 2026-02-11T23:31:06.829+0000 7f416abfa640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:06.829+0000 7f416abfa640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:06.829+0000 7f416abfa640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:06.829+0000 7f416abfa640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566847324s,-4.566720486s]) 2026-02-11T23:31:06.829+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6830/2699377081 14 ==== osd_ping(ping_reply e111 up_from 19 ping_stamp 2026-02-11T23:31:06.830602+0000/49.506008148s send_stamp 42.286636353s delta_ub -7.219370842s) ==== 2033+0+0 (crc 0 0 0) 0x4c9b380 con 0x48cd800 2026-02-11T23:31:06.829+0000 7f416c3fd640 20 osd.0 111 get_con_osd_cluster to osd.3 from_epoch 111 2026-02-11T23:31:06.829+0000 7f416c3fd640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:06.829+0000 7f416c3fd640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:06.829+0000 7f416c3fd640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:06.829+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6840/3331151400 13 ==== osd_ping(ping_reply e111 up_from 23 ping_stamp 2026-02-11T23:31:06.830602+0000/49.506008148s send_stamp 39.669467926s delta_ub -9.836538315s) ==== 2033+0+0 (crc 0 0 0) 0x4c9b200 con 0x507bc00 2026-02-11T23:31:06.829+0000 7f416c3fd640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219507217s,-7.219370842s]) 2026-02-11T23:31:06.829+0000 7f416abfa640 20 osd.0 111 get_con_osd_cluster to osd.4 from_epoch 111 2026-02-11T23:31:06.829+0000 7f416abfa640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:06.829+0000 7f416abfa640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:06.829+0000 7f416abfa640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:06.829+0000 7f416abfa640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836675644s,-9.836538315s]) 2026-02-11T23:31:06.829+0000 7f416cbfe640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6824/2365263801 14 ==== osd_ping(ping_reply e111 up_from 14 ping_stamp 2026-02-11T23:31:06.830602+0000/49.506008148s send_stamp 44.939308167s delta_ub -4.566699505s) ==== 2033+0+0 (crc 0 0 0) 0x5af9800 con 0x48c4000 2026-02-11T23:31:06.829+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6832/2699377081 14 ==== osd_ping(ping_reply e111 up_from 19 ping_stamp 2026-02-11T23:31:06.830602+0000/49.506008148s send_stamp 42.286651611s delta_ub -7.219355583s) ==== 2033+0+0 (crc 0 0 0) 0x4c9b200 con 0x48cd400 2026-02-11T23:31:06.829+0000 7f416cbfe640 20 osd.0 111 get_con_osd_cluster to osd.2 from_epoch 111 2026-02-11T23:31:06.829+0000 7f416cbfe640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:06.829+0000 7f416cbfe640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:06.829+0000 7f416cbfe640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:06.829+0000 7f416cbfe640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566851139s,-4.566699505s]) 2026-02-11T23:31:06.829+0000 7f416abfa640 20 osd.0 111 get_con_osd_cluster to osd.3 from_epoch 111 2026-02-11T23:31:06.829+0000 7f416abfa640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:06.829+0000 7f416abfa640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:06.829+0000 7f416abfa640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:06.829+0000 7f416abfa640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219515324s,-7.219355583s]) 2026-02-11T23:31:07.262+0000 7f4162bea640 10 osd.0 111 tick_without_osd_lock 2026-02-11T23:31:07.262+0000 7f4162bea640 0 osd.0 111 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:31:07.262+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:07.262+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.314 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:07.262+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:07.263, recovery is active?:false restrictions:<....> 2026-02-11T23:31:07.262+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:07.262+0000 7f4162bea640 10 osd.0 111 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:07.262+0000 7f4162bea640 20 osd.0 111 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:07.262+0000 7f4162bea640 10 osd.0 111 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:07.262+0000 7f4162bea640 20 osd.0 111 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:07.265+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.4 127.0.0.1:0/3331151400 13 ==== osd_ping(ping e111 up_from 23 ping_stamp 2026-02-11T23:31:07.266340+0000/40.105106354s send_stamp 40.105106354s delta_ub -9.836538315s) ==== 2033+0+0 (crc 0 0 0) 0x4c9b380 con 0x507a800 2026-02-11T23:31:07.265+0000 7f416c3fd640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836706161s,-9.836538315s]) 2026-02-11T23:31:07.265+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.4 127.0.0.1:0/3331151400 13 ==== osd_ping(ping e111 up_from 23 ping_stamp 2026-02-11T23:31:07.266340+0000/40.105106354s send_stamp 40.105106354s delta_ub -9.836538315s) ==== 2033+0+0 (crc 0 0 0) 0x4c9b200 con 0x507a400 2026-02-11T23:31:07.265+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:07.266340+0000/40.105106354s send_stamp 49.941810608s delta_ub 9.836706161s) -- 0x4c9b980 con 0x507a800 2026-02-11T23:31:07.265+0000 7f416c3fd640 20 osd.0 111 get_con_osd_cluster to osd.4 from_epoch 111 2026-02-11T23:31:07.265+0000 7f416c3fd640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:07.265+0000 7f416c3fd640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:07.265+0000 7f416c3fd640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:07.265+0000 7f416abfa640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836723328s,-9.836538315s]) 2026-02-11T23:31:07.265+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:07.266340+0000/40.105106354s send_stamp 49.941833496s delta_ub 9.836723328s) -- 0x4c9a180 con 0x507a400 2026-02-11T23:31:07.265+0000 7f416abfa640 20 osd.0 111 get_con_osd_cluster to osd.4 from_epoch 111 2026-02-11T23:31:07.265+0000 7f416abfa640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:07.265+0000 7f416abfa640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:07.265+0000 7f416abfa640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:07.298+0000 7f41633eb640 10 osd.0 111 tick 2026-02-11T23:31:07.298+0000 7f41633eb640 20 osd.0 111 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:07.328+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:07.328+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:07.328+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:37.329408+0000) 2026-02-11T23:31:07.495+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.2 127.0.0.1:0/2365263801 12 ==== osd_ping(ping e111 up_from 14 ping_stamp 2026-02-11T23:31:07.496430+0000/45.605026245s send_stamp 45.605026245s delta_ub -4.566699505s) ==== 2033+0+0 (crc 0 0 0) 0x5af9800 con 0x4641000 2026-02-11T23:31:07.495+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.2 127.0.0.1:0/2365263801 12 ==== osd_ping(ping e111 up_from 14 ping_stamp 2026-02-11T23:31:07.496430+0000/45.605026245s send_stamp 45.605026245s delta_ub -4.566699505s) ==== 2033+0+0 (crc 0 0 0) 0x4c9b980 con 0x48cc000 2026-02-11T23:31:07.495+0000 7f416cbfe640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566894531s,-4.566699505s]) 2026-02-11T23:31:07.495+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:07.496430+0000/45.605026245s send_stamp 50.171920776s delta_ub 4.566894531s) -- 0x5af8480 con 0x4641000 2026-02-11T23:31:07.495+0000 7f416cbfe640 20 osd.0 111 get_con_osd_cluster to osd.2 from_epoch 111 2026-02-11T23:31:07.495+0000 7f416cbfe640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:07.495+0000 7f416cbfe640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:07.495+0000 7f416cbfe640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:07.495+0000 7f416c3fd640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566919804s,-4.566699505s]) 2026-02-11T23:31:07.495+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:07.496430+0000/45.605026245s send_stamp 50.171943665s delta_ub 4.566919804s) -- 0x4c9b380 con 0x48cc000 2026-02-11T23:31:07.495+0000 7f416c3fd640 20 osd.0 111 get_con_osd_cluster to osd.2 from_epoch 111 2026-02-11T23:31:07.495+0000 7f416c3fd640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:07.495+0000 7f416c3fd640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:07.495+0000 7f416c3fd640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:08.205+0000 7f415b6ff640 20 osd.0 111 reports for 0 queries 2026-02-11T23:31:08.205+0000 7f415b6ff640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] -- mgrreport(unknown.0 +0-0 packed 3294 daemon_metrics=2) -- 0x4f9e700 con 0x4aca800 2026-02-11T23:31:08.205+0000 7f415b6ff640 15 osd.0 111 collect_pg_stats 2026-02-11T23:31:08.205+0000 7f415b6ff640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] -- pg_stats(0 pgs seq 21474836495 v 0) -- 0x5afc300 con 0x4aca800 2026-02-11T23:31:08.263+0000 7f41633eb640 10 osd.0 111 tick 2026-02-11T23:31:08.263+0000 7f41633eb640 20 osd.0 111 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:08.280+0000 7f4162bea640 10 osd.0 111 tick_without_osd_lock 2026-02-11T23:31:08.280+0000 7f4162bea640 0 osd.0 111 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:31:08.280+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:08.280+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.292 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:08.280+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:08.281, recovery is active?:false restrictions:<....> 2026-02-11T23:31:08.280+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:08.280+0000 7f4162bea640 10 osd.0 111 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:08.280+0000 7f4162bea640 20 osd.0 111 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:08.280+0000 7f4162bea640 10 osd.0 111 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:08.280+0000 7f4162bea640 20 osd.0 111 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:08.328+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:08.328+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:08.328+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:38.329486+0000) 2026-02-11T23:31:08.640+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.1 127.0.0.1:0/3043114013 17 ==== osd_ping(ping e111 up_from 10 ping_stamp 2026-02-11T23:31:08.642106+0000/49.305850983s send_stamp 49.305850983s delta_ub -2.011593103s) ==== 2033+0+0 (crc 0 0 0) 0x4c9b380 con 0x4aca400 2026-02-11T23:31:08.640+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.1 127.0.0.1:0/3043114013 17 ==== osd_ping(ping e111 up_from 10 ping_stamp 2026-02-11T23:31:08.642106+0000/49.305850983s send_stamp 49.305850983s delta_ub -2.011593103s) ==== 2033+0+0 (crc 0 0 0) 0x5af8480 con 0x4aca000 2026-02-11T23:31:08.640+0000 7f416c3fd640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011731148s,-2.011593103s]) 2026-02-11T23:31:08.640+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:08.642106+0000/49.305850983s send_stamp 51.317584991s delta_ub 2.011731148s) -- 0x4c9b980 con 0x4aca400 2026-02-11T23:31:08.640+0000 7f416c3fd640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:08.640+0000 7f416c3fd640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:08.640+0000 7f416c3fd640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:08.640+0000 7f416c3fd640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:08.640+0000 7f416cbfe640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011752129s,-2.011593103s]) 2026-02-11T23:31:08.640+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:08.642106+0000/49.305850983s send_stamp 51.317604065s delta_ub 2.011752129s) -- 0x5af9800 con 0x4aca000 2026-02-11T23:31:08.640+0000 7f416cbfe640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:08.640+0000 7f416cbfe640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:08.640+0000 7f416cbfe640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:08.640+0000 7f416cbfe640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:08.966+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.4 127.0.0.1:0/3331151400 14 ==== osd_ping(ping e111 up_from 23 ping_stamp 2026-02-11T23:31:08.966499+0000/41.805267334s send_stamp 41.805267334s delta_ub -9.836585045s) ==== 2033+0+0 (crc 0 0 0) 0x4c9b980 con 0x507a800 2026-02-11T23:31:08.966+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.4 127.0.0.1:0/3331151400 14 ==== osd_ping(ping e111 up_from 23 ping_stamp 2026-02-11T23:31:08.966499+0000/41.805267334s send_stamp 41.805267334s delta_ub -9.836585045s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a180 con 0x507a400 2026-02-11T23:31:08.966+0000 7f416c3fd640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836724281s,-9.836585045s]) 2026-02-11T23:31:08.966+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:08.966499+0000/41.805267334s send_stamp 51.641990662s delta_ub 9.836724281s) -- 0x4c9b380 con 0x507a800 2026-02-11T23:31:08.966+0000 7f416c3fd640 20 osd.0 111 get_con_osd_cluster to osd.4 from_epoch 111 2026-02-11T23:31:08.966+0000 7f416c3fd640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:08.966+0000 7f416c3fd640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:08.966+0000 7f416c3fd640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:08.966+0000 7f416abfa640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836745262s,-9.836585045s]) 2026-02-11T23:31:08.966+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:08.966499+0000/41.805267334s send_stamp 51.642013550s delta_ub 9.836745262s) -- 0x4c9b200 con 0x507a400 2026-02-11T23:31:08.966+0000 7f416abfa640 20 osd.0 111 get_con_osd_cluster to osd.4 from_epoch 111 2026-02-11T23:31:08.966+0000 7f416abfa640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:08.966+0000 7f416abfa640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:08.966+0000 7f416abfa640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:09.196+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.2 127.0.0.1:0/2365263801 13 ==== osd_ping(ping e111 up_from 14 ping_stamp 2026-02-11T23:31:09.196641+0000/47.305236816s send_stamp 47.305236816s delta_ub -4.566722870s) ==== 2033+0+0 (crc 0 0 0) 0x5af9800 con 0x4641000 2026-02-11T23:31:09.196+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.2 127.0.0.1:0/2365263801 13 ==== osd_ping(ping e111 up_from 14 ping_stamp 2026-02-11T23:31:09.196641+0000/47.305236816s send_stamp 47.305236816s delta_ub -4.566722870s) ==== 2033+0+0 (crc 0 0 0) 0x4c9b380 con 0x48cc000 2026-02-11T23:31:09.196+0000 7f416cbfe640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566891670s,-4.566722870s]) 2026-02-11T23:31:09.196+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:09.196641+0000/47.305236816s send_stamp 51.872127533s delta_ub 4.566891670s) -- 0x5af8480 con 0x4641000 2026-02-11T23:31:09.196+0000 7f416cbfe640 20 osd.0 111 get_con_osd_cluster to osd.2 from_epoch 111 2026-02-11T23:31:09.196+0000 7f416cbfe640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:09.196+0000 7f416cbfe640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:09.196+0000 7f416cbfe640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:09.196+0000 7f416c3fd640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566915035s,-4.566722870s]) 2026-02-11T23:31:09.196+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:09.196641+0000/47.305236816s send_stamp 51.872154236s delta_ub 4.566915035s) -- 0x4c9b980 con 0x48cc000 2026-02-11T23:31:09.196+0000 7f416c3fd640 20 osd.0 111 get_con_osd_cluster to osd.2 from_epoch 111 2026-02-11T23:31:09.196+0000 7f416c3fd640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:09.196+0000 7f416c3fd640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:09.196+0000 7f416c3fd640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:09.264+0000 7f41633eb640 10 osd.0 111 tick 2026-02-11T23:31:09.264+0000 7f41633eb640 20 osd.0 111 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:09.320+0000 7f4162bea640 10 osd.0 111 tick_without_osd_lock 2026-02-11T23:31:09.320+0000 7f4162bea640 0 osd.0 111 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:31:09.320+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:09.320+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:09.320+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:09.320+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.292 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:09.320+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:09.320, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:09.320+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:09.320+0000 7f4162bea640 10 osd.0 111 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:09.320+0000 7f4162bea640 20 osd.0 111 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:09.320+0000 7f4162bea640 10 osd.0 111 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:09.320+0000 7f4162bea640 20 osd.0 111 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:09.329+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:09.329+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:09.329+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:39.329522+0000) 2026-02-11T23:31:09.548+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.3 127.0.0.1:0/2699377081 12 ==== osd_ping(ping e111 up_from 19 ping_stamp 2026-02-11T23:31:09.549078+0000/45.005016327s send_stamp 45.005016327s delta_ub -7.219355583s) ==== 2033+0+0 (crc 0 0 0) 0x5af8480 con 0x48cc800 2026-02-11T23:31:09.548+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.3 127.0.0.1:0/2699377081 12 ==== osd_ping(ping e111 up_from 19 ping_stamp 2026-02-11T23:31:09.549078+0000/45.005016327s send_stamp 45.005016327s delta_ub -7.219355583s) ==== 2033+0+0 (crc 0 0 0) 0x4c9b980 con 0x48cc400 2026-02-11T23:31:09.548+0000 7f416cbfe640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219552040s,-7.219355583s]) 2026-02-11T23:31:09.548+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:09.549078+0000/45.005016327s send_stamp 52.224571228s delta_ub 7.219552040s) -- 0x5af9800 con 0x48cc800 2026-02-11T23:31:09.548+0000 7f416cbfe640 20 osd.0 111 get_con_osd_cluster to osd.3 from_epoch 111 2026-02-11T23:31:09.548+0000 7f416cbfe640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:09.548+0000 7f416cbfe640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:09.548+0000 7f416cbfe640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:09.548+0000 7f416c3fd640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219573021s,-7.219355583s]) 2026-02-11T23:31:09.548+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:09.549078+0000/45.005016327s send_stamp 52.224590302s delta_ub 7.219573021s) -- 0x4c9b380 con 0x48cc400 2026-02-11T23:31:09.548+0000 7f416c3fd640 20 osd.0 111 get_con_osd_cluster to osd.3 from_epoch 111 2026-02-11T23:31:09.548+0000 7f416c3fd640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:09.548+0000 7f416c3fd640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:09.548+0000 7f416c3fd640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:09.730+0000 7f4147983640 5 osd.0 111 heartbeat osd_stat(store_statfs(0x384000/0x0/0x384000, data 0x7f242/0x120000, compress 0x0/0x0/0x0, omap 0xa403, meta 0x1a45bfd), peers [1,2,3,4] op hist []) 2026-02-11T23:31:09.730+0000 7f4147983640 20 osd.0 111 check_full_status cur ratio 0, physical ratio 0, new state none 2026-02-11T23:31:09.730+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6816/3043114013,v1:127.0.0.1:6817/3043114013] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:09.730757+0000/52.406162262s send_stamp 52.406162262s delta_ub 2.011752129s) -- 0x4c9a900 con 0x48c4400 2026-02-11T23:31:09.730+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6814/3043114013,v1:127.0.0.1:6815/3043114013] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:09.730757+0000/52.406162262s send_stamp 52.406162262s delta_ub 2.011752129s) -- 0x4c9b800 con 0x4640c00 2026-02-11T23:31:09.730+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6824/2365263801,v1:127.0.0.1:6825/2365263801] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:09.730757+0000/52.406162262s send_stamp 52.406162262s delta_ub 4.566915035s) -- 0x4c9b080 con 0x48c4000 2026-02-11T23:31:09.730+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6822/2365263801,v1:127.0.0.1:6823/2365263801] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:09.730757+0000/52.406162262s send_stamp 52.406162262s delta_ub 4.566915035s) -- 0x4c9a780 con 0x48cd000 2026-02-11T23:31:09.730+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6832/2699377081,v1:127.0.0.1:6833/2699377081] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:09.730757+0000/52.406162262s send_stamp 52.406162262s delta_ub 7.219573021s) -- 0x4c9a480 con 0x48cd400 2026-02-11T23:31:09.730+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6830/2699377081,v1:127.0.0.1:6831/2699377081] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:09.730757+0000/52.406162262s send_stamp 52.406162262s delta_ub 7.219573021s) -- 0x4c9b500 con 0x48cd800 2026-02-11T23:31:09.730+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6840/3331151400,v1:127.0.0.1:6841/3331151400] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:09.730757+0000/52.406162262s send_stamp 52.406162262s delta_ub 9.836745262s) -- 0x5af9b00 con 0x507bc00 2026-02-11T23:31:09.730+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6838/3331151400,v1:127.0.0.1:6839/3331151400] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:09.730757+0000/52.406162262s send_stamp 52.406162262s delta_ub 9.836745262s) -- 0x5af9c80 con 0x5282000 2026-02-11T23:31:09.730+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6816/3043114013 16 ==== osd_ping(ping_reply e111 up_from 10 ping_stamp 2026-02-11T23:31:09.730757+0000/52.406162262s send_stamp 50.394561768s delta_ub -2.011599541s) ==== 2033+0+0 (crc 0 0 0) 0x5af8780 con 0x48c4400 2026-02-11T23:31:09.730+0000 7f416abfa640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:09.730+0000 7f416abfa640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:09.730+0000 7f416abfa640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:09.730+0000 7f416abfa640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:09.730+0000 7f416abfa640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011705160s,-2.011599541s]) 2026-02-11T23:31:09.730+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6840/3331151400 14 ==== osd_ping(ping_reply e111 up_from 23 ping_stamp 2026-02-11T23:31:09.730757+0000/52.406162262s send_stamp 42.569618225s delta_ub -9.836544991s) ==== 2033+0+0 (crc 0 0 0) 0x5af8780 con 0x507bc00 2026-02-11T23:31:09.730+0000 7f416abfa640 20 osd.0 111 get_con_osd_cluster to osd.4 from_epoch 111 2026-02-11T23:31:09.730+0000 7f416abfa640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:09.730+0000 7f416abfa640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:09.730+0000 7f416abfa640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:09.730+0000 7f416abfa640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836677551s,-9.836544991s]) 2026-02-11T23:31:09.730+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6814/3043114013 16 ==== osd_ping(ping_reply e111 up_from 10 ping_stamp 2026-02-11T23:31:09.730757+0000/52.406162262s send_stamp 50.394580841s delta_ub -2.011583567s) ==== 2033+0+0 (crc 0 0 0) 0x5af9c80 con 0x4640c00 2026-02-11T23:31:09.730+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6832/2699377081 15 ==== osd_ping(ping_reply e111 up_from 19 ping_stamp 2026-02-11T23:31:09.730757+0000/52.406162262s send_stamp 45.186809540s delta_ub -7.219352245s) ==== 2033+0+0 (crc 0 0 0) 0x5af8780 con 0x48cd400 2026-02-11T23:31:09.730+0000 7f416cbfe640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6824/2365263801 15 ==== osd_ping(ping_reply e111 up_from 14 ping_stamp 2026-02-11T23:31:09.730757+0000/52.406162262s send_stamp 47.839454651s delta_ub -4.566710949s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a180 con 0x48c4000 2026-02-11T23:31:09.730+0000 7f416c3fd640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:09.730+0000 7f416c3fd640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:09.730+0000 7f416c3fd640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:09.730+0000 7f416c3fd640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:09.730+0000 7f416c3fd640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011728525s,-2.011583567s]) 2026-02-11T23:31:09.730+0000 7f416abfa640 20 osd.0 111 get_con_osd_cluster to osd.3 from_epoch 111 2026-02-11T23:31:09.730+0000 7f416abfa640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:09.730+0000 7f416abfa640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:09.730+0000 7f416abfa640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:09.730+0000 7f416abfa640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219513893s,-7.219352245s]) 2026-02-11T23:31:09.730+0000 7f416cbfe640 20 osd.0 111 get_con_osd_cluster to osd.2 from_epoch 111 2026-02-11T23:31:09.730+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6830/2699377081 15 ==== osd_ping(ping_reply e111 up_from 19 ping_stamp 2026-02-11T23:31:09.730757+0000/52.406162262s send_stamp 45.186794281s delta_ub -7.219368458s) ==== 2033+0+0 (crc 0 0 0) 0x5af9c80 con 0x48cd800 2026-02-11T23:31:09.730+0000 7f416cbfe640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:09.730+0000 7f416cbfe640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:09.730+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6822/2365263801 15 ==== osd_ping(ping_reply e111 up_from 14 ping_stamp 2026-02-11T23:31:09.730757+0000/52.406162262s send_stamp 47.839469910s delta_ub -4.566692352s) ==== 2033+0+0 (crc 0 0 0) 0x5af8780 con 0x48cd000 2026-02-11T23:31:09.730+0000 7f416cbfe640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:09.730+0000 7f416cbfe640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566881180s,-4.566710949s]) 2026-02-11T23:31:09.730+0000 7f416c3fd640 20 osd.0 111 get_con_osd_cluster to osd.3 from_epoch 111 2026-02-11T23:31:09.730+0000 7f416c3fd640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:09.730+0000 7f416c3fd640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:09.730+0000 7f416c3fd640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:09.730+0000 7f416c3fd640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219551086s,-7.219368458s]) 2026-02-11T23:31:09.730+0000 7f416abfa640 20 osd.0 111 get_con_osd_cluster to osd.2 from_epoch 111 2026-02-11T23:31:09.730+0000 7f416abfa640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:09.730+0000 7f416abfa640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:09.730+0000 7f416abfa640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:09.730+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6838/3331151400 14 ==== osd_ping(ping_reply e111 up_from 23 ping_stamp 2026-02-11T23:31:09.730757+0000/52.406162262s send_stamp 42.569633484s delta_ub -9.836530685s) ==== 2033+0+0 (crc 0 0 0) 0x5af9c80 con 0x5282000 2026-02-11T23:31:09.730+0000 7f416abfa640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566886902s,-4.566692352s]) 2026-02-11T23:31:09.730+0000 7f416c3fd640 20 osd.0 111 get_con_osd_cluster to osd.4 from_epoch 111 2026-02-11T23:31:09.730+0000 7f416c3fd640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:09.730+0000 7f416c3fd640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:09.730+0000 7f416c3fd640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:09.730+0000 7f416c3fd640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836732864s,-9.836530685s]) 2026-02-11T23:31:09.741+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.1 127.0.0.1:0/3043114013 18 ==== osd_ping(ping e111 up_from 10 ping_stamp 2026-02-11T23:31:09.742245+0000/50.405990601s send_stamp 50.405990601s delta_ub -2.011583567s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a180 con 0x4aca000 2026-02-11T23:31:09.741+0000 7f416cbfe640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011709929s,-2.011583567s]) 2026-02-11T23:31:09.741+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:09.742245+0000/50.405990601s send_stamp 52.417701721s delta_ub 2.011709929s) -- 0x4c9b200 con 0x4aca000 2026-02-11T23:31:09.742+0000 7f416cbfe640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:09.742+0000 7f416cbfe640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:09.742+0000 7f416cbfe640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:09.742+0000 7f416cbfe640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:09.741+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.1 127.0.0.1:0/3043114013 18 ==== osd_ping(ping e111 up_from 10 ping_stamp 2026-02-11T23:31:09.742245+0000/50.405990601s send_stamp 50.405990601s delta_ub -2.011583567s) ==== 2033+0+0 (crc 0 0 0) 0x5af9c80 con 0x4aca400 2026-02-11T23:31:09.742+0000 7f416c3fd640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011747360s,-2.011583567s]) 2026-02-11T23:31:09.742+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:09.742245+0000/50.405990601s send_stamp 52.417736053s delta_ub 2.011747360s) -- 0x4c9a480 con 0x4aca400 2026-02-11T23:31:09.742+0000 7f416c3fd640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:09.742+0000 7f416c3fd640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:09.742+0000 7f416c3fd640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:09.742+0000 7f416c3fd640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:10.229+0000 7f41633eb640 10 osd.0 111 tick 2026-02-11T23:31:10.229+0000 7f41633eb640 20 osd.0 111 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:10.297+0000 7f4162bea640 10 osd.0 111 tick_without_osd_lock 2026-02-11T23:31:10.297+0000 7f4162bea640 0 osd.0 111 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:31:10.297+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:10.297+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.292 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:10.297+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:10.297, recovery is active?:false restrictions:<....> 2026-02-11T23:31:10.297+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:10.297+0000 7f4162bea640 10 osd.0 111 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:10.297+0000 7f4162bea640 20 osd.0 111 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:10.297+0000 7f4162bea640 10 osd.0 111 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:10.297+0000 7f4162bea640 20 osd.0 111 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:10.329+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:10.329+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:10.329+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:40.329591+0000) 2026-02-11T23:31:10.329+0000 7f415bf00640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-11T23:31:11.235+0000 7f41633eb640 10 osd.0 111 tick 2026-02-11T23:31:11.235+0000 7f41633eb640 20 osd.0 111 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:11.249+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.3 127.0.0.1:0/2699377081 13 ==== osd_ping(ping e111 up_from 19 ping_stamp 2026-02-11T23:31:11.249258+0000/46.705196381s send_stamp 46.705196381s delta_ub -7.219352245s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a480 con 0x48cc400 2026-02-11T23:31:11.249+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.3 127.0.0.1:0/2699377081 13 ==== osd_ping(ping e111 up_from 19 ping_stamp 2026-02-11T23:31:11.249258+0000/46.705196381s send_stamp 46.705196381s delta_ub -7.219352245s) ==== 2033+0+0 (crc 0 0 0) 0x4c9b200 con 0x48cc800 2026-02-11T23:31:11.249+0000 7f416c3fd640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219569683s,-7.219352245s]) 2026-02-11T23:31:11.249+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:11.249258+0000/46.705196381s send_stamp 53.924766541s delta_ub 7.219569683s) -- 0x5af9c80 con 0x48cc400 2026-02-11T23:31:11.249+0000 7f416c3fd640 20 osd.0 111 get_con_osd_cluster to osd.3 from_epoch 111 2026-02-11T23:31:11.249+0000 7f416c3fd640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:11.249+0000 7f416c3fd640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:11.249+0000 7f416c3fd640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:11.249+0000 7f416cbfe640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219594002s,-7.219352245s]) 2026-02-11T23:31:11.249+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:11.249258+0000/46.705196381s send_stamp 53.924789429s delta_ub 7.219594002s) -- 0x4c9a180 con 0x48cc800 2026-02-11T23:31:11.249+0000 7f416cbfe640 20 osd.0 111 get_con_osd_cluster to osd.3 from_epoch 111 2026-02-11T23:31:11.249+0000 7f416cbfe640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:11.249+0000 7f416cbfe640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:11.249+0000 7f416cbfe640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:11.279+0000 7f4162bea640 10 osd.0 111 tick_without_osd_lock 2026-02-11T23:31:11.279+0000 7f4162bea640 0 osd.0 111 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:31:11.279+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:11.279+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:11.279+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:11.279+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.292 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:11.279+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:11.280, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:11.279+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:11.279+0000 7f4162bea640 10 osd.0 111 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:11.279+0000 7f4162bea640 20 osd.0 111 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:11.279+0000 7f4162bea640 10 osd.0 111 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:11.279+0000 7f4162bea640 20 osd.0 111 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:11.329+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:11.329+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:11.329+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:41.329683+0000) 2026-02-11T23:31:12.248+0000 7f41633eb640 10 osd.0 111 tick 2026-02-11T23:31:12.248+0000 7f41633eb640 20 osd.0 111 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:12.316+0000 7f4162bea640 10 osd.0 111 tick_without_osd_lock 2026-02-11T23:31:12.317+0000 7f4162bea640 0 osd.0 111 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:31:12.317+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:12.317+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.292 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:12.317+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:12.317, recovery is active?:false restrictions:<....> 2026-02-11T23:31:12.317+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:12.317+0000 7f4162bea640 10 osd.0 111 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:12.317+0000 7f4162bea640 20 osd.0 111 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:12.317+0000 7f4162bea640 10 osd.0 111 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:12.317+0000 7f4162bea640 20 osd.0 111 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:12.329+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:12.329+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:12.329+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:42.329759+0000) 2026-02-11T23:31:13.206+0000 7f415b6ff640 20 osd.0 111 reports for 0 queries 2026-02-11T23:31:13.206+0000 7f415b6ff640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] -- mgrreport(unknown.0 +0-0 packed 3294 daemon_metrics=2) -- 0x4643880 con 0x4aca800 2026-02-11T23:31:13.206+0000 7f415b6ff640 15 osd.0 111 collect_pg_stats 2026-02-11T23:31:13.206+0000 7f415b6ff640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] -- pg_stats(0 pgs seq 21474836496 v 0) -- 0x4aa0900 con 0x4aca800 2026-02-11T23:31:13.217+0000 7f41633eb640 10 osd.0 111 tick 2026-02-11T23:31:13.217+0000 7f41633eb640 20 osd.0 111 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:13.314+0000 7f4162bea640 10 osd.0 111 tick_without_osd_lock 2026-02-11T23:31:13.314+0000 7f4162bea640 0 osd.0 111 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:31:13.314+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:13.314+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.272 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:13.314+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:13.314, recovery is active?:false restrictions:<....> 2026-02-11T23:31:13.314+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:13.314+0000 7f4162bea640 10 osd.0 111 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:13.314+0000 7f4162bea640 20 osd.0 111 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:13.314+0000 7f4162bea640 10 osd.0 111 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:13.314+0000 7f4162bea640 20 osd.0 111 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:13.329+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:13.329+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:13.329+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:43.329833+0000) 2026-02-11T23:31:13.830+0000 7f4147983640 5 osd.0 111 heartbeat osd_stat(store_statfs(0x384000/0x0/0x384000, data 0x7f242/0x120000, compress 0x0/0x0/0x0, omap 0xa403, meta 0x1a45bfd), peers [1,2,3,4] op hist []) 2026-02-11T23:31:13.830+0000 7f4147983640 20 osd.0 111 check_full_status cur ratio 0, physical ratio 0, new state none 2026-02-11T23:31:13.830+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6816/3043114013,v1:127.0.0.1:6817/3043114013] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:13.830921+0000/56.506328583s send_stamp 56.506328583s delta_ub 2.011747360s) -- 0x5af9e00 con 0x48c4400 2026-02-11T23:31:13.830+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6814/3043114013,v1:127.0.0.1:6815/3043114013] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:13.830921+0000/56.506328583s send_stamp 56.506328583s delta_ub 2.011747360s) -- 0x4ec2000 con 0x4640c00 2026-02-11T23:31:13.830+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6824/2365263801,v1:127.0.0.1:6825/2365263801] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:13.830921+0000/56.506328583s send_stamp 56.506328583s delta_ub 4.566886902s) -- 0x4ec2180 con 0x48c4000 2026-02-11T23:31:13.830+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6822/2365263801,v1:127.0.0.1:6823/2365263801] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:13.830921+0000/56.506328583s send_stamp 56.506328583s delta_ub 4.566886902s) -- 0x4ec2300 con 0x48cd000 2026-02-11T23:31:13.830+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6832/2699377081,v1:127.0.0.1:6833/2699377081] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:13.830921+0000/56.506328583s send_stamp 56.506328583s delta_ub 7.219594002s) -- 0x4ec2480 con 0x48cd400 2026-02-11T23:31:13.830+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6830/2699377081,v1:127.0.0.1:6831/2699377081] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:13.830921+0000/56.506328583s send_stamp 56.506328583s delta_ub 7.219594002s) -- 0x4ec2600 con 0x48cd800 2026-02-11T23:31:13.830+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6840/3331151400,v1:127.0.0.1:6841/3331151400] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:13.830921+0000/56.506328583s send_stamp 56.506328583s delta_ub 9.836732864s) -- 0x4ec2780 con 0x507bc00 2026-02-11T23:31:13.830+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6838/3331151400,v1:127.0.0.1:6839/3331151400] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:13.830921+0000/56.506328583s send_stamp 56.506328583s delta_ub 9.836732864s) -- 0x4ec2900 con 0x5282000 2026-02-11T23:31:13.830+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6816/3043114013 17 ==== osd_ping(ping_reply e111 up_from 10 ping_stamp 2026-02-11T23:31:13.830921+0000/56.506328583s send_stamp 54.494720459s delta_ub -2.011606693s) ==== 2033+0+0 (crc 0 0 0) 0x4ec2780 con 0x48c4400 2026-02-11T23:31:13.830+0000 7f416abfa640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:13.830+0000 7f416abfa640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:13.830+0000 7f416abfa640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:13.830+0000 7f416abfa640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:13.830+0000 7f416abfa640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011706114s,-2.011606693s]) 2026-02-11T23:31:13.830+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6840/3331151400 15 ==== osd_ping(ping_reply e111 up_from 23 ping_stamp 2026-02-11T23:31:13.830921+0000/56.506328583s send_stamp 46.669792175s delta_ub -9.836536407s) ==== 2033+0+0 (crc 0 0 0) 0x4ec2780 con 0x507bc00 2026-02-11T23:31:13.830+0000 7f416abfa640 20 osd.0 111 get_con_osd_cluster to osd.4 from_epoch 111 2026-02-11T23:31:13.830+0000 7f416abfa640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:13.830+0000 7f416abfa640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:13.830+0000 7f416abfa640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:13.830+0000 7f416abfa640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836669922s,-9.836536407s]) 2026-02-11T23:31:13.830+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6822/2365263801 16 ==== osd_ping(ping_reply e111 up_from 14 ping_stamp 2026-02-11T23:31:13.830921+0000/56.506328583s send_stamp 51.939636230s delta_ub -4.566690922s) ==== 2033+0+0 (crc 0 0 0) 0x4ec2780 con 0x48cd000 2026-02-11T23:31:13.830+0000 7f416abfa640 20 osd.0 111 get_con_osd_cluster to osd.2 from_epoch 111 2026-02-11T23:31:13.830+0000 7f416abfa640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:13.830+0000 7f416cbfe640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6824/2365263801 16 ==== osd_ping(ping_reply e111 up_from 14 ping_stamp 2026-02-11T23:31:13.830921+0000/56.506328583s send_stamp 51.939613342s delta_ub -4.566710949s) ==== 2033+0+0 (crc 0 0 0) 0x4ec2180 con 0x48c4000 2026-02-11T23:31:13.830+0000 7f416abfa640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:13.830+0000 7f416abfa640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:13.830+0000 7f416abfa640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566841125s,-4.566690922s]) 2026-02-11T23:31:13.830+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6838/3331151400 15 ==== osd_ping(ping_reply e111 up_from 23 ping_stamp 2026-02-11T23:31:13.830921+0000/56.506328583s send_stamp 46.669811249s delta_ub -9.836517334s) ==== 2033+0+0 (crc 0 0 0) 0x4ec2900 con 0x5282000 2026-02-11T23:31:13.830+0000 7f416cbfe640 20 osd.0 111 get_con_osd_cluster to osd.2 from_epoch 111 2026-02-11T23:31:13.830+0000 7f416cbfe640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:13.830+0000 7f416cbfe640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:13.830+0000 7f416cbfe640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:13.830+0000 7f416cbfe640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566868782s,-4.566710949s]) 2026-02-11T23:31:13.830+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6832/2699377081 16 ==== osd_ping(ping_reply e111 up_from 19 ping_stamp 2026-02-11T23:31:13.830921+0000/56.506328583s send_stamp 49.286998749s delta_ub -7.219326019s) ==== 2033+0+0 (crc 0 0 0) 0x4ec2780 con 0x48cd400 2026-02-11T23:31:13.830+0000 7f416c3fd640 20 osd.0 111 get_con_osd_cluster to osd.4 from_epoch 111 2026-02-11T23:31:13.830+0000 7f416c3fd640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:13.830+0000 7f416c3fd640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:13.830+0000 7f416c3fd640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:13.830+0000 7f416c3fd640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836696625s,-9.836517334s]) 2026-02-11T23:31:13.830+0000 7f416abfa640 20 osd.0 111 get_con_osd_cluster to osd.3 from_epoch 111 2026-02-11T23:31:13.830+0000 7f416abfa640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:13.830+0000 7f416abfa640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:13.830+0000 7f416abfa640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:13.830+0000 7f416abfa640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219518185s,-7.219326019s]) 2026-02-11T23:31:13.830+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6814/3043114013 17 ==== osd_ping(ping_reply e111 up_from 10 ping_stamp 2026-02-11T23:31:13.830921+0000/56.506328583s send_stamp 54.494781494s delta_ub -2.011545420s) ==== 2033+0+0 (crc 0 0 0) 0x4ec2900 con 0x4640c00 2026-02-11T23:31:13.830+0000 7f416c3fd640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:13.830+0000 7f416c3fd640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:13.830+0000 7f416c3fd640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:13.830+0000 7f416c3fd640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:13.830+0000 7f416c3fd640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011745453s,-2.011545420s]) 2026-02-11T23:31:13.830+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6830/2699377081 16 ==== osd_ping(ping_reply e111 up_from 19 ping_stamp 2026-02-11T23:31:13.830921+0000/56.506328583s send_stamp 49.286979675s delta_ub -7.219347000s) ==== 2033+0+0 (crc 0 0 0) 0x4ec2900 con 0x48cd800 2026-02-11T23:31:13.830+0000 7f416c3fd640 20 osd.0 111 get_con_osd_cluster to osd.3 from_epoch 111 2026-02-11T23:31:13.830+0000 7f416c3fd640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:13.830+0000 7f416c3fd640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:13.830+0000 7f416c3fd640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:13.830+0000 7f416c3fd640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219559193s,-7.219347000s]) 2026-02-11T23:31:13.842+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.1 127.0.0.1:0/3043114013 19 ==== osd_ping(ping e111 up_from 10 ping_stamp 2026-02-11T23:31:13.842411+0000/54.506156921s send_stamp 54.506156921s delta_ub -2.011545420s) ==== 2033+0+0 (crc 0 0 0) 0x4ec2180 con 0x4aca000 2026-02-11T23:31:13.842+0000 7f416cbfe640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011701822s,-2.011545420s]) 2026-02-11T23:31:13.842+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:13.842411+0000/54.506156921s send_stamp 56.517856598s delta_ub 2.011701822s) -- 0x4c9a180 con 0x4aca000 2026-02-11T23:31:13.842+0000 7f416cbfe640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:13.842+0000 7f416cbfe640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:13.842+0000 7f416cbfe640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:13.842+0000 7f416cbfe640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:13.842+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.1 127.0.0.1:0/3043114013 19 ==== osd_ping(ping e111 up_from 10 ping_stamp 2026-02-11T23:31:13.842411+0000/54.506156921s send_stamp 54.506156921s delta_ub -2.011545420s) ==== 2033+0+0 (crc 0 0 0) 0x4ec2900 con 0x4aca400 2026-02-11T23:31:13.842+0000 7f416c3fd640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011721373s,-2.011545420s]) 2026-02-11T23:31:13.842+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:13.842411+0000/54.506156921s send_stamp 56.517879486s delta_ub 2.011721373s) -- 0x4ec2600 con 0x4aca400 2026-02-11T23:31:13.842+0000 7f416c3fd640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:13.842+0000 7f416c3fd640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:13.842+0000 7f416c3fd640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:13.842+0000 7f416c3fd640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:14.211+0000 7f41633eb640 10 osd.0 111 tick 2026-02-11T23:31:14.211+0000 7f41633eb640 20 osd.0 111 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:14.329+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:14.329+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:14.329+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:44.329882+0000) 2026-02-11T23:31:14.347+0000 7f4162bea640 10 osd.0 111 tick_without_osd_lock 2026-02-11T23:31:14.347+0000 7f4162bea640 0 osd.0 111 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:31:14.347+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:14.347+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:14.347+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:14.347+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.272 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:14.347+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:14.348, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:14.347+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:14.347+0000 7f4162bea640 10 osd.0 111 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:14.347+0000 7f4162bea640 20 osd.0 111 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:14.347+0000 7f4162bea640 10 osd.0 111 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:14.347+0000 7f4162bea640 20 osd.0 111 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:14.496+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.2 127.0.0.1:0/2365263801 14 ==== osd_ping(ping e111 up_from 14 ping_stamp 2026-02-11T23:31:14.496815+0000/52.605411530s send_stamp 52.605411530s delta_ub -4.566690922s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a180 con 0x4641000 2026-02-11T23:31:14.496+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.2 127.0.0.1:0/2365263801 14 ==== osd_ping(ping e111 up_from 14 ping_stamp 2026-02-11T23:31:14.496815+0000/52.605411530s send_stamp 52.605411530s delta_ub -4.566690922s) ==== 2033+0+0 (crc 0 0 0) 0x4ec2600 con 0x48cc000 2026-02-11T23:31:14.496+0000 7f416cbfe640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566901207s,-4.566690922s]) 2026-02-11T23:31:14.496+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:14.496815+0000/52.605411530s send_stamp 57.172313690s delta_ub 4.566901207s) -- 0x4ec2180 con 0x4641000 2026-02-11T23:31:14.496+0000 7f416cbfe640 20 osd.0 111 get_con_osd_cluster to osd.2 from_epoch 111 2026-02-11T23:31:14.496+0000 7f416cbfe640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:14.496+0000 7f416cbfe640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:14.496+0000 7f416cbfe640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:14.496+0000 7f416c3fd640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566925526s,-4.566690922s]) 2026-02-11T23:31:14.496+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:14.496815+0000/52.605411530s send_stamp 57.172336578s delta_ub 4.566925526s) -- 0x4ec2900 con 0x48cc000 2026-02-11T23:31:14.496+0000 7f416c3fd640 20 osd.0 111 get_con_osd_cluster to osd.2 from_epoch 111 2026-02-11T23:31:14.496+0000 7f416c3fd640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:14.496+0000 7f416c3fd640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:14.496+0000 7f416c3fd640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:14.749+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.3 127.0.0.1:0/2699377081 14 ==== osd_ping(ping e111 up_from 19 ping_stamp 2026-02-11T23:31:14.749451+0000/50.205394745s send_stamp 50.205394745s delta_ub -7.219326019s) ==== 2033+0+0 (crc 0 0 0) 0x4ec2900 con 0x48cc400 2026-02-11T23:31:14.749+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.3 127.0.0.1:0/2699377081 14 ==== osd_ping(ping e111 up_from 19 ping_stamp 2026-02-11T23:31:14.749451+0000/50.205394745s send_stamp 50.205394745s delta_ub -7.219326019s) ==== 2033+0+0 (crc 0 0 0) 0x4ec2180 con 0x48cc800 2026-02-11T23:31:14.749+0000 7f416c3fd640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219550610s,-7.219326019s]) 2026-02-11T23:31:14.749+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:14.749451+0000/50.205394745s send_stamp 57.424942017s delta_ub 7.219550610s) -- 0x4ec2600 con 0x48cc400 2026-02-11T23:31:14.749+0000 7f416c3fd640 20 osd.0 111 get_con_osd_cluster to osd.3 from_epoch 111 2026-02-11T23:31:14.749+0000 7f416c3fd640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:14.749+0000 7f416c3fd640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:14.749+0000 7f416c3fd640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:14.749+0000 7f416cbfe640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219571114s,-7.219326019s]) 2026-02-11T23:31:14.749+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:14.749451+0000/50.205394745s send_stamp 57.424961090s delta_ub 7.219571114s) -- 0x4c9a180 con 0x48cc800 2026-02-11T23:31:14.749+0000 7f416cbfe640 20 osd.0 111 get_con_osd_cluster to osd.3 from_epoch 111 2026-02-11T23:31:14.749+0000 7f416cbfe640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:14.749+0000 7f416cbfe640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:14.749+0000 7f416cbfe640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:14.866+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.4 127.0.0.1:0/3331151400 15 ==== osd_ping(ping e111 up_from 23 ping_stamp 2026-02-11T23:31:14.866675+0000/47.705444336s send_stamp 47.705444336s delta_ub -9.836517334s) ==== 2033+0+0 (crc 0 0 0) 0x4ec2600 con 0x507a800 2026-02-11T23:31:14.866+0000 7f416c3fd640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836708069s,-9.836517334s]) 2026-02-11T23:31:14.866+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:14.866675+0000/47.705444336s send_stamp 57.542152405s delta_ub 9.836708069s) -- 0x4ec2900 con 0x507a800 2026-02-11T23:31:14.866+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.4 127.0.0.1:0/3331151400 15 ==== osd_ping(ping e111 up_from 23 ping_stamp 2026-02-11T23:31:14.866675+0000/47.705444336s send_stamp 47.705444336s delta_ub -9.836517334s) ==== 2033+0+0 (crc 0 0 0) 0x4ec2780 con 0x507a400 2026-02-11T23:31:14.866+0000 7f416c3fd640 20 osd.0 111 get_con_osd_cluster to osd.4 from_epoch 111 2026-02-11T23:31:14.866+0000 7f416c3fd640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:14.866+0000 7f416c3fd640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:14.866+0000 7f416c3fd640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:14.866+0000 7f416abfa640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836735725s,-9.836517334s]) 2026-02-11T23:31:14.866+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:14.866675+0000/47.705444336s send_stamp 57.542175293s delta_ub 9.836735725s) -- 0x4ec2480 con 0x507a400 2026-02-11T23:31:14.866+0000 7f416abfa640 20 osd.0 111 get_con_osd_cluster to osd.4 from_epoch 111 2026-02-11T23:31:14.866+0000 7f416abfa640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:14.866+0000 7f416abfa640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:14.866+0000 7f416abfa640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:15.189+0000 7f41633eb640 10 osd.0 111 tick 2026-02-11T23:31:15.189+0000 7f41633eb640 20 osd.0 111 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:15.329+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:15.329+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:15.329+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:45.329946+0000) 2026-02-11T23:31:15.348+0000 7f4162bea640 10 osd.0 111 tick_without_osd_lock 2026-02-11T23:31:15.349+0000 7f4162bea640 0 osd.0 111 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:31:15.349+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:15.349+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:15.349+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:15.349+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.272 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:15.349+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:15.349, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:15.349+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:15.349+0000 7f4162bea640 10 osd.0 111 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:15.349+0000 7f4162bea640 20 osd.0 111 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:15.349+0000 7f4162bea640 10 osd.0 111 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:15.349+0000 7f4162bea640 20 osd.0 111 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:16.155+0000 7f41633eb640 10 osd.0 111 tick 2026-02-11T23:31:16.155+0000 7f41633eb640 20 osd.0 111 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:16.196+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.2 127.0.0.1:0/2365263801 15 ==== osd_ping(ping e111 up_from 14 ping_stamp 2026-02-11T23:31:16.196985+0000/54.305583954s send_stamp 54.305583954s delta_ub -4.566696167s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a180 con 0x4641000 2026-02-11T23:31:16.196+0000 7f416cbfe640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566894054s,-4.566696167s]) 2026-02-11T23:31:16.196+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.2 127.0.0.1:0/2365263801 15 ==== osd_ping(ping e111 up_from 14 ping_stamp 2026-02-11T23:31:16.196985+0000/54.305583954s send_stamp 54.305583954s delta_ub -4.566696167s) ==== 2033+0+0 (crc 0 0 0) 0x4ec2900 con 0x48cc000 2026-02-11T23:31:16.196+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:16.196985+0000/54.305583954s send_stamp 58.872474670s delta_ub 4.566894054s) -- 0x4ec2180 con 0x4641000 2026-02-11T23:31:16.196+0000 7f416cbfe640 20 osd.0 111 get_con_osd_cluster to osd.2 from_epoch 111 2026-02-11T23:31:16.196+0000 7f416cbfe640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:16.196+0000 7f416cbfe640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:16.196+0000 7f416cbfe640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:16.196+0000 7f416c3fd640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566912651s,-4.566696167s]) 2026-02-11T23:31:16.196+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:16.196985+0000/54.305583954s send_stamp 58.872497559s delta_ub 4.566912651s) -- 0x4ec2600 con 0x48cc000 2026-02-11T23:31:16.196+0000 7f416c3fd640 20 osd.0 111 get_con_osd_cluster to osd.2 from_epoch 111 2026-02-11T23:31:16.196+0000 7f416c3fd640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:16.196+0000 7f416c3fd640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:16.196+0000 7f416c3fd640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:16.329+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:16.329+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:16.329+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:46.330009+0000) 2026-02-11T23:31:16.362+0000 7f4162bea640 10 osd.0 111 tick_without_osd_lock 2026-02-11T23:31:16.362+0000 7f4162bea640 0 osd.0 111 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:31:16.362+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:16.362+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:16.362+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:16.362+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.272 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:16.362+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:16.363, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:16.362+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:16.362+0000 7f4162bea640 10 osd.0 111 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:16.362+0000 7f4162bea640 20 osd.0 111 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:16.362+0000 7f4162bea640 10 osd.0 111 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:16.362+0000 7f4162bea640 20 osd.0 111 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:16.742+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.1 127.0.0.1:0/3043114013 20 ==== osd_ping(ping e111 up_from 10 ping_stamp 2026-02-11T23:31:16.742562+0000/57.406307220s send_stamp 57.406307220s delta_ub -2.011619568s) ==== 2033+0+0 (crc 0 0 0) 0x4ec2600 con 0x4aca400 2026-02-11T23:31:16.742+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.1 127.0.0.1:0/3043114013 20 ==== osd_ping(ping e111 up_from 10 ping_stamp 2026-02-11T23:31:16.742562+0000/57.406307220s send_stamp 57.406307220s delta_ub -2.011619568s) ==== 2033+0+0 (crc 0 0 0) 0x4ec2180 con 0x4aca000 2026-02-11T23:31:16.742+0000 7f416c3fd640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011749268s,-2.011619568s]) 2026-02-11T23:31:16.742+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:16.742562+0000/57.406307220s send_stamp 59.418056488s delta_ub 2.011749268s) -- 0x4ec2900 con 0x4aca400 2026-02-11T23:31:16.742+0000 7f416c3fd640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:16.742+0000 7f416c3fd640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:16.742+0000 7f416c3fd640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:16.742+0000 7f416c3fd640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:16.742+0000 7f416cbfe640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011774540s,-2.011619568s]) 2026-02-11T23:31:16.742+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:16.742562+0000/57.406307220s send_stamp 59.418083191s delta_ub 2.011774540s) -- 0x4c9a180 con 0x4aca000 2026-02-11T23:31:16.742+0000 7f416cbfe640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:16.742+0000 7f416cbfe640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:16.742+0000 7f416cbfe640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:16.742+0000 7f416cbfe640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.117+0000 7f41633eb640 10 osd.0 111 tick 2026-02-11T23:31:17.117+0000 7f41633eb640 20 osd.0 111 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:17.295+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.2 127.0.0.1:0/2365263801 16 ==== osd_ping(ping e111 up_from 14 ping_stamp 2026-02-11T23:31:17.297148+0000/55.405742645s send_stamp 55.405742645s delta_ub -4.566702366s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a180 con 0x4641000 2026-02-11T23:31:17.295+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.2 127.0.0.1:0/2365263801 16 ==== osd_ping(ping e111 up_from 14 ping_stamp 2026-02-11T23:31:17.297148+0000/55.405742645s send_stamp 55.405742645s delta_ub -4.566702366s) ==== 2033+0+0 (crc 0 0 0) 0x4ec2900 con 0x48cc000 2026-02-11T23:31:17.295+0000 7f416cbfe640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566903591s,-4.566702366s]) 2026-02-11T23:31:17.295+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:17.297148+0000/55.405742645s send_stamp 59.972648621s delta_ub 4.566903591s) -- 0x4ec2180 con 0x4641000 2026-02-11T23:31:17.295+0000 7f416cbfe640 20 osd.0 111 get_con_osd_cluster to osd.2 from_epoch 111 2026-02-11T23:31:17.295+0000 7f416cbfe640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.295+0000 7f416cbfe640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.295+0000 7f416cbfe640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.295+0000 7f416c3fd640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566923618s,-4.566702366s]) 2026-02-11T23:31:17.295+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:17.297148+0000/55.405742645s send_stamp 59.972667694s delta_ub 4.566923618s) -- 0x4ec2600 con 0x48cc000 2026-02-11T23:31:17.295+0000 7f416c3fd640 20 osd.0 111 get_con_osd_cluster to osd.2 from_epoch 111 2026-02-11T23:31:17.295+0000 7f416c3fd640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.295+0000 7f416c3fd640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.295+0000 7f416c3fd640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.328+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:17.328+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:17.328+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:47.330088+0000) 2026-02-11T23:31:17.396+0000 7f4162bea640 10 osd.0 111 tick_without_osd_lock 2026-02-11T23:31:17.396+0000 7f4162bea640 0 osd.0 111 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:31:17.396+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:17.396+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:17.396+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:17.396+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.272 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:17.396+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:17.397, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:17.396+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:17.396+0000 7f4162bea640 10 osd.0 111 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:17.396+0000 7f4162bea640 20 osd.0 111 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:17.396+0000 7f4162bea640 10 osd.0 111 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:17.396+0000 7f4162bea640 20 osd.0 111 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:17.796+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.2 127.0.0.1:0/2365263801 17 ==== osd_ping(ping e111 up_from 14 ping_stamp 2026-02-11T23:31:17.797345+0000/55.905941010s send_stamp 55.905941010s delta_ub -4.566697121s) ==== 2033+0+0 (crc 0 0 0) 0x4ec2180 con 0x4641000 2026-02-11T23:31:17.796+0000 7f416cbfe640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566901684s,-4.566697121s]) 2026-02-11T23:31:17.796+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:17.797345+0000/55.905941010s send_stamp 60.472843170s delta_ub 4.566901684s) -- 0x4c9a180 con 0x4641000 2026-02-11T23:31:17.796+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.2 127.0.0.1:0/2365263801 17 ==== osd_ping(ping e111 up_from 14 ping_stamp 2026-02-11T23:31:17.797345+0000/55.905941010s send_stamp 55.905941010s delta_ub -4.566697121s) ==== 2033+0+0 (crc 0 0 0) 0x4ec2600 con 0x48cc000 2026-02-11T23:31:17.796+0000 7f416cbfe640 20 osd.0 111 get_con_osd_cluster to osd.2 from_epoch 111 2026-02-11T23:31:17.796+0000 7f416cbfe640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.796+0000 7f416cbfe640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.796+0000 7f416cbfe640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.796+0000 7f416c3fd640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566926003s,-4.566697121s]) 2026-02-11T23:31:17.796+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:17.797345+0000/55.905941010s send_stamp 60.472869873s delta_ub 4.566926003s) -- 0x4ec2900 con 0x48cc000 2026-02-11T23:31:17.796+0000 7f416c3fd640 20 osd.0 111 get_con_osd_cluster to osd.2 from_epoch 111 2026-02-11T23:31:17.796+0000 7f416c3fd640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.796+0000 7f416c3fd640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.796+0000 7f416c3fd640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:18.165+0000 7f41633eb640 10 osd.0 111 tick 2026-02-11T23:31:18.165+0000 7f41633eb640 20 osd.0 111 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:18.205+0000 7f415b6ff640 20 osd.0 111 reports for 0 queries 2026-02-11T23:31:18.205+0000 7f415b6ff640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] -- mgrreport(unknown.0 +0-0 packed 3294 daemon_metrics=2) -- 0x4f9f880 con 0x4aca800 2026-02-11T23:31:18.205+0000 7f415b6ff640 15 osd.0 111 collect_pg_stats 2026-02-11T23:31:18.206+0000 7f415b6ff640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] -- pg_stats(0 pgs seq 21474836497 v 0) -- 0x4aa0c00 con 0x4aca800 2026-02-11T23:31:18.328+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:18.328+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:18.328+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:48.330117+0000) 2026-02-11T23:31:18.380+0000 7f4162bea640 10 osd.0 111 tick_without_osd_lock 2026-02-11T23:31:18.380+0000 7f4162bea640 0 osd.0 111 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:31:18.380+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:18.380+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:18.380+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:18.380+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.253 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:18.380+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:18.382, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:18.380+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:18.380+0000 7f4162bea640 10 osd.0 111 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:18.380+0000 7f4162bea640 20 osd.0 111 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:18.380+0000 7f4162bea640 10 osd.0 111 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:18.380+0000 7f4162bea640 20 osd.0 111 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:19.129+0000 7f4147983640 5 osd.0 111 heartbeat osd_stat(store_statfs(0x384000/0x0/0x384000, data 0x7f242/0x120000, compress 0x0/0x0/0x0, omap 0xa403, meta 0x1a45bfd), peers [1,2,3,4] op hist []) 2026-02-11T23:31:19.129+0000 7f4147983640 20 osd.0 111 check_full_status cur ratio 0, physical ratio 0, new state none 2026-02-11T23:31:19.129+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6816/3043114013,v1:127.0.0.1:6817/3043114013] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:19.131096+0000/61.806503296s send_stamp 61.806503296s delta_ub 2.011774540s) -- 0x4ec2a80 con 0x48c4400 2026-02-11T23:31:19.129+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6814/3043114013,v1:127.0.0.1:6815/3043114013] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:19.131096+0000/61.806503296s send_stamp 61.806503296s delta_ub 2.011774540s) -- 0x4ec2c00 con 0x4640c00 2026-02-11T23:31:19.129+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6824/2365263801,v1:127.0.0.1:6825/2365263801] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:19.131096+0000/61.806503296s send_stamp 61.806503296s delta_ub 4.566926003s) -- 0x4ec2d80 con 0x48c4000 2026-02-11T23:31:19.129+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6822/2365263801,v1:127.0.0.1:6823/2365263801] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:19.131096+0000/61.806503296s send_stamp 61.806503296s delta_ub 4.566926003s) -- 0x4ec2f00 con 0x48cd000 2026-02-11T23:31:19.129+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6832/2699377081,v1:127.0.0.1:6833/2699377081] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:19.131096+0000/61.806503296s send_stamp 61.806503296s delta_ub 7.219571114s) -- 0x4ec3080 con 0x48cd400 2026-02-11T23:31:19.129+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6830/2699377081,v1:127.0.0.1:6831/2699377081] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:19.131096+0000/61.806503296s send_stamp 61.806503296s delta_ub 7.219571114s) -- 0x4ec3200 con 0x48cd800 2026-02-11T23:31:19.129+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6840/3331151400,v1:127.0.0.1:6841/3331151400] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:19.131096+0000/61.806503296s send_stamp 61.806503296s delta_ub 9.836735725s) -- 0x4ec3380 con 0x507bc00 2026-02-11T23:31:19.129+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6838/3331151400,v1:127.0.0.1:6839/3331151400] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:19.131096+0000/61.806503296s send_stamp 61.806503296s delta_ub 9.836735725s) -- 0x4ec3500 con 0x5282000 2026-02-11T23:31:19.129+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6816/3043114013 18 ==== osd_ping(ping_reply e111 up_from 10 ping_stamp 2026-02-11T23:31:19.131096+0000/61.806503296s send_stamp 59.794906616s delta_ub -2.011595964s) ==== 2033+0+0 (crc 0 0 0) 0x4ec3380 con 0x48c4400 2026-02-11T23:31:19.129+0000 7f416abfa640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:19.129+0000 7f416abfa640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:19.129+0000 7f416abfa640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:19.129+0000 7f416abfa640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:19.129+0000 7f416abfa640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011709452s,-2.011595964s]) 2026-02-11T23:31:19.129+0000 7f416cbfe640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6824/2365263801 17 ==== osd_ping(ping_reply e111 up_from 14 ping_stamp 2026-02-11T23:31:19.131096+0000/61.806503296s send_stamp 57.239776611s delta_ub -4.566725254s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a780 con 0x48c4000 2026-02-11T23:31:19.129+0000 7f416cbfe640 20 osd.0 111 get_con_osd_cluster to osd.2 from_epoch 111 2026-02-11T23:31:19.129+0000 7f416cbfe640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:19.129+0000 7f416cbfe640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:19.129+0000 7f416cbfe640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:19.129+0000 7f416cbfe640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566856861s,-4.566725254s]) 2026-02-11T23:31:19.129+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6814/3043114013 18 ==== osd_ping(ping_reply e111 up_from 10 ping_stamp 2026-02-11T23:31:19.131096+0000/61.806503296s send_stamp 59.794929504s delta_ub -2.011574507s) ==== 2033+0+0 (crc 0 0 0) 0x4ec3500 con 0x4640c00 2026-02-11T23:31:19.129+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6840/3331151400 16 ==== osd_ping(ping_reply e111 up_from 23 ping_stamp 2026-02-11T23:31:19.131096+0000/61.806503296s send_stamp 51.969974518s delta_ub -9.836529732s) ==== 2033+0+0 (crc 0 0 0) 0x4ec3380 con 0x507bc00 2026-02-11T23:31:19.129+0000 7f416c3fd640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:19.129+0000 7f416c3fd640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:19.129+0000 7f416c3fd640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:19.129+0000 7f416c3fd640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:19.129+0000 7f416c3fd640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011728048s,-2.011574507s]) 2026-02-11T23:31:19.129+0000 7f416abfa640 20 osd.0 111 get_con_osd_cluster to osd.4 from_epoch 111 2026-02-11T23:31:19.129+0000 7f416abfa640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:19.129+0000 7f416abfa640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:19.129+0000 7f416abfa640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:19.129+0000 7f416abfa640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836707115s,-9.836529732s]) 2026-02-11T23:31:19.129+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6838/3331151400 16 ==== osd_ping(ping_reply e111 up_from 23 ping_stamp 2026-02-11T23:31:19.131096+0000/61.806503296s send_stamp 51.969989777s delta_ub -9.836510658s) ==== 2033+0+0 (crc 0 0 0) 0x4ec3500 con 0x5282000 2026-02-11T23:31:19.129+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6822/2365263801 17 ==== osd_ping(ping_reply e111 up_from 14 ping_stamp 2026-02-11T23:31:19.131096+0000/61.806503296s send_stamp 57.239791870s delta_ub -4.566708088s) ==== 2033+0+0 (crc 0 0 0) 0x4ec3380 con 0x48cd000 2026-02-11T23:31:19.129+0000 7f416c3fd640 20 osd.0 111 get_con_osd_cluster to osd.4 from_epoch 111 2026-02-11T23:31:19.129+0000 7f416c3fd640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:19.129+0000 7f416c3fd640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:19.129+0000 7f416c3fd640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:19.129+0000 7f416c3fd640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836698532s,-9.836510658s]) 2026-02-11T23:31:19.129+0000 7f416abfa640 20 osd.0 111 get_con_osd_cluster to osd.2 from_epoch 111 2026-02-11T23:31:19.130+0000 7f416abfa640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:19.130+0000 7f416abfa640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:19.130+0000 7f416abfa640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:19.130+0000 7f416abfa640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566909790s,-4.566708088s]) 2026-02-11T23:31:19.130+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6830/2699377081 17 ==== osd_ping(ping_reply e111 up_from 19 ping_stamp 2026-02-11T23:31:19.131096+0000/61.806503296s send_stamp 54.587173462s delta_ub -7.219325542s) ==== 2033+0+0 (crc 0 0 0) 0x4ec3500 con 0x48cd800 2026-02-11T23:31:19.130+0000 7f416c3fd640 20 osd.0 111 get_con_osd_cluster to osd.3 from_epoch 111 2026-02-11T23:31:19.130+0000 7f416c3fd640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:19.130+0000 7f416c3fd640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:19.130+0000 7f416c3fd640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:19.130+0000 7f416c3fd640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219540596s,-7.219325542s]) 2026-02-11T23:31:19.130+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6832/2699377081 17 ==== osd_ping(ping_reply e111 up_from 19 ping_stamp 2026-02-11T23:31:19.131096+0000/61.806503296s send_stamp 54.587154388s delta_ub -7.219348431s) ==== 2033+0+0 (crc 0 0 0) 0x4ec3380 con 0x48cd400 2026-02-11T23:31:19.130+0000 7f416abfa640 20 osd.0 111 get_con_osd_cluster to osd.3 from_epoch 111 2026-02-11T23:31:19.130+0000 7f416abfa640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:19.130+0000 7f416abfa640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:19.130+0000 7f416abfa640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:19.130+0000 7f416abfa640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219571114s,-7.219348431s]) 2026-02-11T23:31:19.207+0000 7f41633eb640 10 osd.0 111 tick 2026-02-11T23:31:19.207+0000 7f41633eb640 10 osd.0 111 maybe_update_heartbeat_peers forcing update after 20.908446 seconds 2026-02-11T23:31:19.207+0000 7f41633eb640 10 osd.0 111 reset_heartbeat_peers 2026-02-11T23:31:19.207+0000 7f41633eb640 10 osd.0 111 maybe_update_heartbeat_peers updating 2026-02-11T23:31:19.207+0000 7f41633eb640 10 osd.0 111 adding neighbor peer osd.1 2026-02-11T23:31:19.207+0000 7f41633eb640 10 osd.0 111 adding neighbor peer osd.4 2026-02-11T23:31:19.207+0000 7f41633eb640 10 osd.0 111 maybe_update_heartbeat_peers 4 peers, extras 1,2,3,4 2026-02-11T23:31:19.207+0000 7f41633eb640 20 osd.0 111 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:19.328+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:19.328+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:19.328+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:49.330194+0000) 2026-02-11T23:31:19.346+0000 7f4162bea640 10 osd.0 111 tick_without_osd_lock 2026-02-11T23:31:19.346+0000 7f4162bea640 0 osd.0 111 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:31:19.346+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:19.346+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:19.346+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:19.346+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.253 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:19.346+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:19.348, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:19.346+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:19.346+0000 7f4162bea640 10 osd.0 111 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:19.346+0000 7f4162bea640 20 osd.0 111 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:19.346+0000 7f4162bea640 10 osd.0 111 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:19.346+0000 7f4162bea640 20 osd.0 111 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:19.565+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.4 127.0.0.1:0/3331151400 16 ==== osd_ping(ping e111 up_from 23 ping_stamp 2026-02-11T23:31:19.566856+0000/52.405624390s send_stamp 52.405624390s delta_ub -9.836510658s) ==== 2033+0+0 (crc 0 0 0) 0x4ec3380 con 0x507a400 2026-02-11T23:31:19.565+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.4 127.0.0.1:0/3331151400 16 ==== osd_ping(ping e111 up_from 23 ping_stamp 2026-02-11T23:31:19.566856+0000/52.405624390s send_stamp 52.405624390s delta_ub -9.836510658s) ==== 2033+0+0 (crc 0 0 0) 0x4ec3500 con 0x507a800 2026-02-11T23:31:19.565+0000 7f416abfa640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836721420s,-9.836510658s]) 2026-02-11T23:31:19.565+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:19.566856+0000/52.405624390s send_stamp 62.242343903s delta_ub 9.836721420s) -- 0x4ec3080 con 0x507a400 2026-02-11T23:31:19.565+0000 7f416abfa640 20 osd.0 111 get_con_osd_cluster to osd.4 from_epoch 111 2026-02-11T23:31:19.565+0000 7f416abfa640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:19.565+0000 7f416abfa640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:19.565+0000 7f416abfa640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:19.565+0000 7f416c3fd640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836748123s,-9.836510658s]) 2026-02-11T23:31:19.565+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:19.566856+0000/52.405624390s send_stamp 62.242370605s delta_ub 9.836748123s) -- 0x4ec3200 con 0x507a800 2026-02-11T23:31:19.565+0000 7f416c3fd640 20 osd.0 111 get_con_osd_cluster to osd.4 from_epoch 111 2026-02-11T23:31:19.565+0000 7f416c3fd640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:19.565+0000 7f416c3fd640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:19.565+0000 7f416c3fd640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:20.048+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.3 127.0.0.1:0/2699377081 15 ==== osd_ping(ping e111 up_from 19 ping_stamp 2026-02-11T23:31:20.049661+0000/55.505599976s send_stamp 55.505599976s delta_ub -7.219325542s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a780 con 0x48cc800 2026-02-11T23:31:20.048+0000 7f416cbfe640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219567299s,-7.219325542s]) 2026-02-11T23:31:20.048+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.3 127.0.0.1:0/2699377081 15 ==== osd_ping(ping e111 up_from 19 ping_stamp 2026-02-11T23:31:20.049661+0000/55.505599976s send_stamp 55.505599976s delta_ub -7.219325542s) ==== 2033+0+0 (crc 0 0 0) 0x4ec3200 con 0x48cc400 2026-02-11T23:31:20.048+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:20.049661+0000/55.505599976s send_stamp 62.725170135s delta_ub 7.219567299s) -- 0x4c9a900 con 0x48cc800 2026-02-11T23:31:20.048+0000 7f416cbfe640 20 osd.0 111 get_con_osd_cluster to osd.3 from_epoch 111 2026-02-11T23:31:20.048+0000 7f416cbfe640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:20.048+0000 7f416cbfe640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:20.048+0000 7f416cbfe640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:20.048+0000 7f416c3fd640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219593048s,-7.219325542s]) 2026-02-11T23:31:20.048+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:20.049661+0000/55.505599976s send_stamp 62.725193024s delta_ub 7.219593048s) -- 0x4ec3500 con 0x48cc400 2026-02-11T23:31:20.048+0000 7f416c3fd640 20 osd.0 111 get_con_osd_cluster to osd.3 from_epoch 111 2026-02-11T23:31:20.048+0000 7f416c3fd640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:20.048+0000 7f416c3fd640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:20.048+0000 7f416c3fd640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:20.096+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.2 127.0.0.1:0/2365263801 18 ==== osd_ping(ping e111 up_from 14 ping_stamp 2026-02-11T23:31:20.097515+0000/58.206108093s send_stamp 58.206108093s delta_ub -4.566708088s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a900 con 0x4641000 2026-02-11T23:31:20.096+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.2 127.0.0.1:0/2365263801 18 ==== osd_ping(ping e111 up_from 14 ping_stamp 2026-02-11T23:31:20.097515+0000/58.206108093s send_stamp 58.206108093s delta_ub -4.566708088s) ==== 2033+0+0 (crc 0 0 0) 0x4ec3500 con 0x48cc000 2026-02-11T23:31:20.096+0000 7f416cbfe640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566879272s,-4.566708088s]) 2026-02-11T23:31:20.096+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:20.097515+0000/58.206108093s send_stamp 62.772991180s delta_ub 4.566879272s) -- 0x4c9a780 con 0x4641000 2026-02-11T23:31:20.096+0000 7f416cbfe640 20 osd.0 111 get_con_osd_cluster to osd.2 from_epoch 111 2026-02-11T23:31:20.096+0000 7f416cbfe640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:20.096+0000 7f416cbfe640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:20.096+0000 7f416cbfe640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:20.096+0000 7f416c3fd640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566896915s,-4.566708088s]) 2026-02-11T23:31:20.096+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:20.097515+0000/58.206108093s send_stamp 62.773010254s delta_ub 4.566896915s) -- 0x4ec3200 con 0x48cc000 2026-02-11T23:31:20.096+0000 7f416c3fd640 20 osd.0 111 get_con_osd_cluster to osd.2 from_epoch 111 2026-02-11T23:31:20.096+0000 7f416c3fd640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:20.096+0000 7f416c3fd640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:20.096+0000 7f416c3fd640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:20.241+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.1 127.0.0.1:0/3043114013 21 ==== osd_ping(ping e111 up_from 10 ping_stamp 2026-02-11T23:31:20.242694+0000/60.906436920s send_stamp 60.906436920s delta_ub -2.011574507s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a780 con 0x4aca000 2026-02-11T23:31:20.241+0000 7f416cbfe640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011730671s,-2.011574507s]) 2026-02-11T23:31:20.241+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:20.242694+0000/60.906436920s send_stamp 62.918167114s delta_ub 2.011730671s) -- 0x4c9a900 con 0x4aca000 2026-02-11T23:31:20.241+0000 7f416cbfe640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:20.241+0000 7f416cbfe640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:20.241+0000 7f416cbfe640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:20.241+0000 7f416cbfe640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:20.241+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.1 127.0.0.1:0/3043114013 21 ==== osd_ping(ping e111 up_from 10 ping_stamp 2026-02-11T23:31:20.242694+0000/60.906436920s send_stamp 60.906436920s delta_ub -2.011574507s) ==== 2033+0+0 (crc 0 0 0) 0x4ec3200 con 0x4aca400 2026-02-11T23:31:20.241+0000 7f416c3fd640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011752367s,-2.011574507s]) 2026-02-11T23:31:20.241+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:20.242694+0000/60.906436920s send_stamp 62.918193817s delta_ub 2.011752367s) -- 0x4ec3500 con 0x4aca400 2026-02-11T23:31:20.241+0000 7f416c3fd640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:20.241+0000 7f416c3fd640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:20.241+0000 7f416c3fd640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:20.241+0000 7f416c3fd640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:20.248+0000 7f41633eb640 10 osd.0 111 tick 2026-02-11T23:31:20.248+0000 7f41633eb640 20 osd.0 111 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:20.300+0000 7f4162bea640 10 osd.0 111 tick_without_osd_lock 2026-02-11T23:31:20.300+0000 7f4162bea640 0 osd.0 111 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:31:20.300+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:20.300+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:20.300+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:20.300+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.253 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:20.300+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:20.301, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:20.300+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:20.300+0000 7f4162bea640 10 osd.0 111 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:20.300+0000 7f4162bea640 20 osd.0 111 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:20.300+0000 7f4162bea640 10 osd.0 111 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:20.300+0000 7f4162bea640 20 osd.0 111 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:20.328+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:20.328+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:20.328+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:50.330264+0000) 2026-02-11T23:31:20.328+0000 7f415bf00640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-11T23:31:21.251+0000 7f4162bea640 10 osd.0 111 tick_without_osd_lock 2026-02-11T23:31:21.251+0000 7f4162bea640 0 osd.0 111 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:31:21.251+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:21.251+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:21.251+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:21.251+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.253 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:21.251+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:21.252, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:21.251+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:21.251+0000 7f4162bea640 10 osd.0 111 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:21.251+0000 7f4162bea640 20 osd.0 111 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:21.251+0000 7f4162bea640 10 osd.0 111 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:21.251+0000 7f4162bea640 20 osd.0 111 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:21.255+0000 7f41633eb640 10 osd.0 111 tick 2026-02-11T23:31:21.255+0000 7f41633eb640 20 osd.0 111 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:21.329+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:21.329+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:21.329+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:51.330344+0000) 2026-02-11T23:31:21.865+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.4 127.0.0.1:0/3331151400 17 ==== osd_ping(ping e111 up_from 23 ping_stamp 2026-02-11T23:31:21.867037+0000/54.705806732s send_stamp 54.705806732s delta_ub -9.836581230s) ==== 2033+0+0 (crc 0 0 0) 0x4ec3500 con 0x507a800 2026-02-11T23:31:21.865+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.4 127.0.0.1:0/3331151400 17 ==== osd_ping(ping e111 up_from 23 ping_stamp 2026-02-11T23:31:21.867037+0000/54.705806732s send_stamp 54.705806732s delta_ub -9.836581230s) ==== 2033+0+0 (crc 0 0 0) 0x4ec3080 con 0x507a400 2026-02-11T23:31:21.865+0000 7f416c3fd640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836719513s,-9.836581230s]) 2026-02-11T23:31:21.865+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:21.867037+0000/54.705806732s send_stamp 64.542526245s delta_ub 9.836719513s) -- 0x4ec3200 con 0x507a800 2026-02-11T23:31:21.865+0000 7f416c3fd640 20 osd.0 111 get_con_osd_cluster to osd.4 from_epoch 111 2026-02-11T23:31:21.865+0000 7f416c3fd640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:21.865+0000 7f416c3fd640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:21.865+0000 7f416c3fd640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:21.865+0000 7f416abfa640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836741447s,-9.836581230s]) 2026-02-11T23:31:21.865+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:21.867037+0000/54.705806732s send_stamp 64.542549133s delta_ub 9.836741447s) -- 0x4ec3380 con 0x507a400 2026-02-11T23:31:21.865+0000 7f416abfa640 20 osd.0 111 get_con_osd_cluster to osd.4 from_epoch 111 2026-02-11T23:31:21.865+0000 7f416abfa640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:21.865+0000 7f416abfa640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:21.865+0000 7f416abfa640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:22.291+0000 7f41633eb640 10 osd.0 111 tick 2026-02-11T23:31:22.291+0000 7f41633eb640 20 osd.0 111 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:22.293+0000 7f4162bea640 10 osd.0 111 tick_without_osd_lock 2026-02-11T23:31:22.293+0000 7f4162bea640 0 osd.0 111 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:31:22.293+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:22.293+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.253 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:22.293+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:22.295, recovery is active?:false restrictions:<....> 2026-02-11T23:31:22.293+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:22.293+0000 7f4162bea640 10 osd.0 111 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:22.293+0000 7f4162bea640 20 osd.0 111 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:22.293+0000 7f4162bea640 10 osd.0 111 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:22.293+0000 7f4162bea640 20 osd.0 111 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:22.329+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:22.329+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:22.329+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:52.330389+0000) 2026-02-11T23:31:22.348+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.3 127.0.0.1:0/2699377081 16 ==== osd_ping(ping e111 up_from 19 ping_stamp 2026-02-11T23:31:22.349865+0000/57.805805206s send_stamp 57.805805206s delta_ub -7.219371796s) ==== 2033+0+0 (crc 0 0 0) 0x4ec3200 con 0x48cc400 2026-02-11T23:31:22.348+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.3 127.0.0.1:0/2699377081 16 ==== osd_ping(ping e111 up_from 19 ping_stamp 2026-02-11T23:31:22.349865+0000/57.805805206s send_stamp 57.805805206s delta_ub -7.219371796s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a900 con 0x48cc800 2026-02-11T23:31:22.348+0000 7f416c3fd640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219549179s,-7.219371796s]) 2026-02-11T23:31:22.348+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:22.349865+0000/57.805805206s send_stamp 65.025352478s delta_ub 7.219549179s) -- 0x4ec3500 con 0x48cc400 2026-02-11T23:31:22.348+0000 7f416c3fd640 20 osd.0 111 get_con_osd_cluster to osd.3 from_epoch 111 2026-02-11T23:31:22.348+0000 7f416c3fd640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:22.348+0000 7f416c3fd640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:22.348+0000 7f416c3fd640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:22.348+0000 7f416cbfe640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219571114s,-7.219371796s]) 2026-02-11T23:31:22.348+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:22.349865+0000/57.805805206s send_stamp 65.025375366s delta_ub 7.219571114s) -- 0x4c9a780 con 0x48cc800 2026-02-11T23:31:22.348+0000 7f416cbfe640 20 osd.0 111 get_con_osd_cluster to osd.3 from_epoch 111 2026-02-11T23:31:22.348+0000 7f416cbfe640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:22.348+0000 7f416cbfe640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:22.348+0000 7f416cbfe640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:22.629+0000 7f4147983640 5 osd.0 111 heartbeat osd_stat(store_statfs(0x384000/0x0/0x384000, data 0x7f242/0x120000, compress 0x0/0x0/0x0, omap 0xa403, meta 0x1a45bfd), peers [1,2,3,4] op hist []) 2026-02-11T23:31:22.629+0000 7f4147983640 20 osd.0 111 check_full_status cur ratio 0, physical ratio 0, new state none 2026-02-11T23:31:22.629+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6816/3043114013,v1:127.0.0.1:6817/3043114013] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:22.631273+0000/65.306678772s send_stamp 65.306678772s delta_ub 2.011752367s) -- 0x4ec3680 con 0x48c4400 2026-02-11T23:31:22.629+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6814/3043114013,v1:127.0.0.1:6815/3043114013] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:22.631273+0000/65.306678772s send_stamp 65.306678772s delta_ub 2.011752367s) -- 0x4ec3800 con 0x4640c00 2026-02-11T23:31:22.630+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6824/2365263801,v1:127.0.0.1:6825/2365263801] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:22.631273+0000/65.306678772s send_stamp 65.306678772s delta_ub 4.566896915s) -- 0x4ec2180 con 0x48c4000 2026-02-11T23:31:22.630+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6822/2365263801,v1:127.0.0.1:6823/2365263801] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:22.631273+0000/65.306678772s send_stamp 65.306678772s delta_ub 4.566896915s) -- 0x4ec2d80 con 0x48cd000 2026-02-11T23:31:22.630+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6832/2699377081,v1:127.0.0.1:6833/2699377081] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:22.631273+0000/65.306678772s send_stamp 65.306678772s delta_ub 7.219571114s) -- 0x4ec2000 con 0x48cd400 2026-02-11T23:31:22.630+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6830/2699377081,v1:127.0.0.1:6831/2699377081] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:22.631273+0000/65.306678772s send_stamp 65.306678772s delta_ub 7.219571114s) -- 0x4ec2300 con 0x48cd800 2026-02-11T23:31:22.630+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6840/3331151400,v1:127.0.0.1:6841/3331151400] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:22.631273+0000/65.306678772s send_stamp 65.306678772s delta_ub 9.836741447s) -- 0x4ec3980 con 0x507bc00 2026-02-11T23:31:22.630+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6838/3331151400,v1:127.0.0.1:6839/3331151400] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:22.631273+0000/65.306678772s send_stamp 65.306678772s delta_ub 9.836741447s) -- 0x4ec3b00 con 0x5282000 2026-02-11T23:31:22.630+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6816/3043114013 19 ==== osd_ping(ping_reply e111 up_from 10 ping_stamp 2026-02-11T23:31:22.631273+0000/65.306678772s send_stamp 63.295085907s delta_ub -2.011592865s) ==== 2033+0+0 (crc 0 0 0) 0x4ec3980 con 0x48c4400 2026-02-11T23:31:22.630+0000 7f416abfa640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:22.630+0000 7f416abfa640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:22.630+0000 7f416abfa640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:22.630+0000 7f416abfa640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:22.630+0000 7f416abfa640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011711597s,-2.011592865s]) 2026-02-11T23:31:22.630+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6840/3331151400 17 ==== osd_ping(ping_reply e111 up_from 23 ping_stamp 2026-02-11T23:31:22.631273+0000/65.306678772s send_stamp 55.470149994s delta_ub -9.836530685s) ==== 2033+0+0 (crc 0 0 0) 0x4ec3980 con 0x507bc00 2026-02-11T23:31:22.630+0000 7f416abfa640 20 osd.0 111 get_con_osd_cluster to osd.4 from_epoch 111 2026-02-11T23:31:22.630+0000 7f416abfa640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:22.630+0000 7f416abfa640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:22.630+0000 7f416abfa640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:22.630+0000 7f416abfa640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836674690s,-9.836530685s]) 2026-02-11T23:31:22.630+0000 7f416cbfe640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6824/2365263801 18 ==== osd_ping(ping_reply e111 up_from 14 ping_stamp 2026-02-11T23:31:22.631273+0000/65.306678772s send_stamp 60.739974976s delta_ub -4.566705227s) ==== 2033+0+0 (crc 0 0 0) 0x4ec2180 con 0x48c4000 2026-02-11T23:31:22.630+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6814/3043114013 19 ==== osd_ping(ping_reply e111 up_from 10 ping_stamp 2026-02-11T23:31:22.631273+0000/65.306678772s send_stamp 63.295112610s delta_ub -2.011569738s) ==== 2033+0+0 (crc 0 0 0) 0x4ec3b00 con 0x4640c00 2026-02-11T23:31:22.630+0000 7f416cbfe640 20 osd.0 111 get_con_osd_cluster to osd.2 from_epoch 111 2026-02-11T23:31:22.630+0000 7f416cbfe640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:22.630+0000 7f416cbfe640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:22.630+0000 7f416cbfe640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:22.630+0000 7f416cbfe640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566856384s,-4.566705227s]) 2026-02-11T23:31:22.630+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6832/2699377081 18 ==== osd_ping(ping_reply e111 up_from 19 ping_stamp 2026-02-11T23:31:22.631273+0000/65.306678772s send_stamp 58.087322235s delta_ub -7.219357014s) ==== 2033+0+0 (crc 0 0 0) 0x4ec3980 con 0x48cd400 2026-02-11T23:31:22.630+0000 7f416abfa640 20 osd.0 111 get_con_osd_cluster to osd.3 from_epoch 111 2026-02-11T23:31:22.630+0000 7f416abfa640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:22.630+0000 7f416abfa640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:22.630+0000 7f416abfa640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:22.630+0000 7f416abfa640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219518185s,-7.219357014s]) 2026-02-11T23:31:22.630+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6822/2365263801 18 ==== osd_ping(ping_reply e111 up_from 14 ping_stamp 2026-02-11T23:31:22.631273+0000/65.306678772s send_stamp 60.739994049s delta_ub -4.566686153s) ==== 2033+0+0 (crc 0 0 0) 0x4ec3980 con 0x48cd000 2026-02-11T23:31:22.630+0000 7f416c3fd640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:22.630+0000 7f416c3fd640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:22.630+0000 7f416c3fd640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:22.630+0000 7f416c3fd640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:22.630+0000 7f416c3fd640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011740685s,-2.011569738s]) 2026-02-11T23:31:22.630+0000 7f416abfa640 20 osd.0 111 get_con_osd_cluster to osd.2 from_epoch 111 2026-02-11T23:31:22.630+0000 7f416abfa640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:22.630+0000 7f416abfa640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:22.630+0000 7f416abfa640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:22.630+0000 7f416abfa640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566879272s,-4.566686153s]) 2026-02-11T23:31:22.630+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6830/2699377081 18 ==== osd_ping(ping_reply e111 up_from 19 ping_stamp 2026-02-11T23:31:22.631273+0000/65.306678772s send_stamp 58.087345123s delta_ub -7.219331741s) ==== 2033+0+0 (crc 0 0 0) 0x4ec3b00 con 0x48cd800 2026-02-11T23:31:22.630+0000 7f416c3fd640 20 osd.0 111 get_con_osd_cluster to osd.3 from_epoch 111 2026-02-11T23:31:22.630+0000 7f416c3fd640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:22.630+0000 7f416c3fd640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:22.630+0000 7f416c3fd640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:22.630+0000 7f416c3fd640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219534397s,-7.219331741s]) 2026-02-11T23:31:22.630+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6838/3331151400 17 ==== osd_ping(ping_reply e111 up_from 23 ping_stamp 2026-02-11T23:31:22.631273+0000/65.306678772s send_stamp 55.470176697s delta_ub -9.836499214s) ==== 2033+0+0 (crc 0 0 0) 0x4ec3b00 con 0x5282000 2026-02-11T23:31:22.630+0000 7f416c3fd640 20 osd.0 111 get_con_osd_cluster to osd.4 from_epoch 111 2026-02-11T23:31:22.630+0000 7f416c3fd640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:22.630+0000 7f416c3fd640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:22.630+0000 7f416c3fd640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:22.630+0000 7f416c3fd640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836715698s,-9.836499214s]) 2026-02-11T23:31:23.206+0000 7f415b6ff640 20 osd.0 111 reports for 0 queries 2026-02-11T23:31:23.206+0000 7f415b6ff640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] -- mgrreport(unknown.0 +0-0 packed 3294 daemon_metrics=2) -- 0x4f9ea80 con 0x4aca800 2026-02-11T23:31:23.206+0000 7f415b6ff640 15 osd.0 111 collect_pg_stats 2026-02-11T23:31:23.206+0000 7f415b6ff640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] -- pg_stats(0 pgs seq 21474836498 v 0) -- 0x4aa0f00 con 0x4aca800 2026-02-11T23:31:23.273+0000 7f41633eb640 10 osd.0 111 tick 2026-02-11T23:31:23.273+0000 7f41633eb640 20 osd.0 111 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:23.329+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:23.329+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:23.329+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:53.330438+0000) 2026-02-11T23:31:23.340+0000 7f4162bea640 10 osd.0 111 tick_without_osd_lock 2026-02-11T23:31:23.340+0000 7f4162bea640 0 osd.0 111 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:31:23.340+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:23.340+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.236 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:23.340+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:23.341, recovery is active?:false restrictions:<....> 2026-02-11T23:31:23.340+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:23.340+0000 7f4162bea640 10 osd.0 111 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:23.340+0000 7f4162bea640 20 osd.0 111 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:23.340+0000 7f4162bea640 10 osd.0 111 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:23.340+0000 7f4162bea640 20 osd.0 111 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:23.730+0000 7f4147983640 5 osd.0 111 heartbeat osd_stat(store_statfs(0x384000/0x0/0x384000, data 0x7f242/0x120000, compress 0x0/0x0/0x0, omap 0xa403, meta 0x1a45bfd), peers [1,2,3,4] op hist []) 2026-02-11T23:31:23.730+0000 7f4147983640 20 osd.0 111 check_full_status cur ratio 0, physical ratio 0, new state none 2026-02-11T23:31:23.730+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6816/3043114013,v1:127.0.0.1:6817/3043114013] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:23.731454+0000/66.406860352s send_stamp 66.406860352s delta_ub 2.011740685s) -- 0x4ec3c80 con 0x48c4400 2026-02-11T23:31:23.730+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6814/3043114013,v1:127.0.0.1:6815/3043114013] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:23.731454+0000/66.406860352s send_stamp 66.406860352s delta_ub 2.011740685s) -- 0x4ec3e00 con 0x4640c00 2026-02-11T23:31:23.730+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6824/2365263801,v1:127.0.0.1:6825/2365263801] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:23.731454+0000/66.406860352s send_stamp 66.406860352s delta_ub 4.566879272s) -- 0x4a89200 con 0x48c4000 2026-02-11T23:31:23.730+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6822/2365263801,v1:127.0.0.1:6823/2365263801] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:23.731454+0000/66.406860352s send_stamp 66.406860352s delta_ub 4.566879272s) -- 0x4a89980 con 0x48cd000 2026-02-11T23:31:23.730+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6832/2699377081,v1:127.0.0.1:6833/2699377081] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:23.731454+0000/66.406860352s send_stamp 66.406860352s delta_ub 7.219534397s) -- 0x4c9b680 con 0x48cd400 2026-02-11T23:31:23.730+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6830/2699377081,v1:127.0.0.1:6831/2699377081] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:23.731454+0000/66.406860352s send_stamp 66.406860352s delta_ub 7.219534397s) -- 0x4c9ad80 con 0x48cd800 2026-02-11T23:31:23.730+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6840/3331151400,v1:127.0.0.1:6841/3331151400] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:23.731454+0000/66.406860352s send_stamp 66.406860352s delta_ub 9.836715698s) -- 0x4c9af00 con 0x507bc00 2026-02-11T23:31:23.730+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6838/3331151400,v1:127.0.0.1:6839/3331151400] -- osd_ping(ping e111 up_from 5 ping_stamp 2026-02-11T23:31:23.731454+0000/66.406860352s send_stamp 66.406860352s delta_ub 9.836715698s) -- 0x4c9a480 con 0x5282000 2026-02-11T23:31:23.730+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6816/3043114013 20 ==== osd_ping(ping_reply e111 up_from 10 ping_stamp 2026-02-11T23:31:23.731454+0000/66.406860352s send_stamp 64.395256042s delta_ub -2.011601925s) ==== 2033+0+0 (crc 0 0 0) 0x4c9af00 con 0x48c4400 2026-02-11T23:31:23.730+0000 7f416abfa640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:23.730+0000 7f416abfa640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:23.730+0000 7f416abfa640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:23.730+0000 7f416abfa640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:23.730+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6814/3043114013 20 ==== osd_ping(ping_reply e111 up_from 10 ping_stamp 2026-02-11T23:31:23.731454+0000/66.406860352s send_stamp 64.395286560s delta_ub -2.011575222s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a480 con 0x4640c00 2026-02-11T23:31:23.730+0000 7f416abfa640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011742592s,-2.011601925s]) 2026-02-11T23:31:23.730+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6832/2699377081 19 ==== osd_ping(ping_reply e111 up_from 19 ping_stamp 2026-02-11T23:31:23.731454+0000/66.406860352s send_stamp 59.187500000s delta_ub -7.219358921s) ==== 2033+0+0 (crc 0 0 0) 0x4c9af00 con 0x48cd400 2026-02-11T23:31:23.730+0000 7f416c3fd640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:23.730+0000 7f416c3fd640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:23.730+0000 7f416c3fd640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:23.730+0000 7f416c3fd640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:23.730+0000 7f416c3fd640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011733055s,-2.011575222s]) 2026-02-11T23:31:23.730+0000 7f416abfa640 20 osd.0 111 get_con_osd_cluster to osd.3 from_epoch 111 2026-02-11T23:31:23.730+0000 7f416abfa640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:23.730+0000 7f416abfa640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:23.730+0000 7f416abfa640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:23.730+0000 7f416abfa640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219546318s,-7.219358921s]) 2026-02-11T23:31:23.730+0000 7f416cbfe640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6824/2365263801 19 ==== osd_ping(ping_reply e111 up_from 14 ping_stamp 2026-02-11T23:31:23.731454+0000/66.406860352s send_stamp 61.840187073s delta_ub -4.566671371s) ==== 2033+0+0 (crc 0 0 0) 0x4a89200 con 0x48c4000 2026-02-11T23:31:23.730+0000 7f416cbfe640 20 osd.0 111 get_con_osd_cluster to osd.2 from_epoch 111 2026-02-11T23:31:23.730+0000 7f416cbfe640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:23.730+0000 7f416cbfe640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:23.730+0000 7f416cbfe640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:23.730+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6840/3331151400 18 ==== osd_ping(ping_reply e111 up_from 23 ping_stamp 2026-02-11T23:31:23.731454+0000/66.406860352s send_stamp 56.570323944s delta_ub -9.836535454s) ==== 2033+0+0 (crc 0 0 0) 0x4c9af00 con 0x507bc00 2026-02-11T23:31:23.730+0000 7f416cbfe640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566866398s,-4.566671371s]) 2026-02-11T23:31:23.730+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6830/2699377081 19 ==== osd_ping(ping_reply e111 up_from 19 ping_stamp 2026-02-11T23:31:23.731454+0000/66.406860352s send_stamp 59.187519073s delta_ub -7.219339371s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a480 con 0x48cd800 2026-02-11T23:31:23.730+0000 7f416c3fd640 20 osd.0 111 get_con_osd_cluster to osd.3 from_epoch 111 2026-02-11T23:31:23.730+0000 7f416c3fd640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:23.730+0000 7f416c3fd640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:23.730+0000 7f416c3fd640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:23.730+0000 7f416c3fd640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219543457s,-7.219339371s]) 2026-02-11T23:31:23.730+0000 7f416abfa640 20 osd.0 111 get_con_osd_cluster to osd.4 from_epoch 111 2026-02-11T23:31:23.730+0000 7f416abfa640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:23.730+0000 7f416abfa640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:23.730+0000 7f416abfa640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:23.730+0000 7f416abfa640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836751938s,-9.836535454s]) 2026-02-11T23:31:23.730+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6838/3331151400 18 ==== osd_ping(ping_reply e111 up_from 23 ping_stamp 2026-02-11T23:31:23.731454+0000/66.406860352s send_stamp 56.570354462s delta_ub -9.836505890s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a480 con 0x5282000 2026-02-11T23:31:23.730+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6822/2365263801 19 ==== osd_ping(ping_reply e111 up_from 14 ping_stamp 2026-02-11T23:31:23.731454+0000/66.406860352s send_stamp 61.840152740s delta_ub -4.566704750s) ==== 2033+0+0 (crc 0 0 0) 0x4c9af00 con 0x48cd000 2026-02-11T23:31:23.730+0000 7f416c3fd640 20 osd.0 111 get_con_osd_cluster to osd.4 from_epoch 111 2026-02-11T23:31:23.730+0000 7f416c3fd640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:23.730+0000 7f416c3fd640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:23.730+0000 7f416c3fd640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:23.730+0000 7f416c3fd640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836735725s,-9.836505890s]) 2026-02-11T23:31:23.730+0000 7f416abfa640 20 osd.0 111 get_con_osd_cluster to osd.2 from_epoch 111 2026-02-11T23:31:23.730+0000 7f416abfa640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:23.730+0000 7f416abfa640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:23.730+0000 7f416abfa640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:23.730+0000 7f416abfa640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566948414s,-4.566704750s]) 2026-02-11T23:31:24.196+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.2 127.0.0.1:0/2365263801 19 ==== osd_ping(ping e111 up_from 14 ping_stamp 2026-02-11T23:31:24.197702+0000/62.306301117s send_stamp 62.306301117s delta_ub -4.566671371s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a480 con 0x48cc000 2026-02-11T23:31:24.196+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.2 127.0.0.1:0/2365263801 19 ==== osd_ping(ping e111 up_from 14 ping_stamp 2026-02-11T23:31:24.197702+0000/62.306301117s send_stamp 62.306301117s delta_ub -4.566671371s) ==== 2033+0+0 (crc 0 0 0) 0x4a89200 con 0x4641000 2026-02-11T23:31:24.196+0000 7f416c3fd640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566899776s,-4.566671371s]) 2026-02-11T23:31:24.196+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:24.197702+0000/62.306301117s send_stamp 66.873199463s delta_ub 4.566899776s) -- 0x4c9ad80 con 0x48cc000 2026-02-11T23:31:24.196+0000 7f416c3fd640 20 osd.0 111 get_con_osd_cluster to osd.2 from_epoch 111 2026-02-11T23:31:24.196+0000 7f416c3fd640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:24.196+0000 7f416c3fd640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:24.196+0000 7f416c3fd640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:24.196+0000 7f416cbfe640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566924572s,-4.566671371s]) 2026-02-11T23:31:24.196+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:24.197702+0000/62.306301117s send_stamp 66.873222351s delta_ub 4.566924572s) -- 0x4ec2180 con 0x4641000 2026-02-11T23:31:24.196+0000 7f416cbfe640 20 osd.0 111 get_con_osd_cluster to osd.2 from_epoch 111 2026-02-11T23:31:24.196+0000 7f416cbfe640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:24.196+0000 7f416cbfe640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:24.196+0000 7f416cbfe640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:24.275+0000 7f41633eb640 10 osd.0 111 tick 2026-02-11T23:31:24.275+0000 7f41633eb640 20 osd.0 111 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:24.329+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:24.329+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:24.329+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:54.330513+0000) 2026-02-11T23:31:24.386+0000 7f4162bea640 10 osd.0 111 tick_without_osd_lock 2026-02-11T23:31:24.386+0000 7f4162bea640 0 osd.0 111 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:31:24.386+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:24.386+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:24.386+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:24.386+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.236 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:24.386+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:24.387, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:24.386+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:24.386+0000 7f4162bea640 10 osd.0 111 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:24.386+0000 7f4162bea640 20 osd.0 111 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:24.386+0000 7f4162bea640 10 osd.0 111 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:24.386+0000 7f4162bea640 20 osd.0 111 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:24.648+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.3 127.0.0.1:0/2699377081 17 ==== osd_ping(ping e111 up_from 19 ping_stamp 2026-02-11T23:31:24.650057+0000/60.105998993s send_stamp 60.105998993s delta_ub -7.219339371s) ==== 2033+0+0 (crc 0 0 0) 0x4ec2180 con 0x48cc800 2026-02-11T23:31:24.648+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.3 127.0.0.1:0/2699377081 17 ==== osd_ping(ping e111 up_from 19 ping_stamp 2026-02-11T23:31:24.650057+0000/60.105998993s send_stamp 60.105998993s delta_ub -7.219339371s) ==== 2033+0+0 (crc 0 0 0) 0x4c9ad80 con 0x48cc400 2026-02-11T23:31:24.648+0000 7f416cbfe640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219567776s,-7.219339371s]) 2026-02-11T23:31:24.648+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:24.650057+0000/60.105998993s send_stamp 67.325561523s delta_ub 7.219567776s) -- 0x4a89200 con 0x48cc800 2026-02-11T23:31:24.648+0000 7f416cbfe640 20 osd.0 111 get_con_osd_cluster to osd.3 from_epoch 111 2026-02-11T23:31:24.648+0000 7f416cbfe640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:24.648+0000 7f416cbfe640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:24.648+0000 7f416cbfe640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:24.648+0000 7f416c3fd640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219584942s,-7.219339371s]) 2026-02-11T23:31:24.648+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:24.650057+0000/60.105998993s send_stamp 67.325584412s delta_ub 7.219584942s) -- 0x4c9a480 con 0x48cc400 2026-02-11T23:31:24.648+0000 7f416c3fd640 20 osd.0 111 get_con_osd_cluster to osd.3 from_epoch 111 2026-02-11T23:31:24.648+0000 7f416c3fd640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:24.648+0000 7f416c3fd640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:24.648+0000 7f416c3fd640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:25.226+0000 7f41633eb640 10 osd.0 111 tick 2026-02-11T23:31:25.226+0000 7f41633eb640 20 osd.0 111 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:25.329+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:25.329+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:25.329+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:55.330589+0000) 2026-02-11T23:31:25.412+0000 7f4162bea640 10 osd.0 111 tick_without_osd_lock 2026-02-11T23:31:25.412+0000 7f4162bea640 0 osd.0 111 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:31:25.412+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:25.412+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.236 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:25.412+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:25.413, recovery is active?:false restrictions:<....> 2026-02-11T23:31:25.412+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:25.412+0000 7f4162bea640 10 osd.0 111 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:25.412+0000 7f4162bea640 20 osd.0 111 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:25.412+0000 7f4162bea640 10 osd.0 111 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:25.412+0000 7f4162bea640 20 osd.0 111 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:25.541+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.1 127.0.0.1:0/3043114013 22 ==== osd_ping(ping e111 up_from 10 ping_stamp 2026-02-11T23:31:25.542880+0000/66.206626892s send_stamp 66.206626892s delta_ub -2.011575222s) ==== 2033+0+0 (crc 0 0 0) 0x4a89200 con 0x4aca000 2026-02-11T23:31:25.541+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.1 127.0.0.1:0/3043114013 22 ==== osd_ping(ping e111 up_from 10 ping_stamp 2026-02-11T23:31:25.542880+0000/66.206626892s send_stamp 66.206626892s delta_ub -2.011575222s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a480 con 0x4aca400 2026-02-11T23:31:25.541+0000 7f416cbfe640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011738777s,-2.011575222s]) 2026-02-11T23:31:25.541+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:25.542880+0000/66.206626892s send_stamp 68.218360901s delta_ub 2.011738777s) -- 0x4ec2180 con 0x4aca000 2026-02-11T23:31:25.541+0000 7f416cbfe640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:25.541+0000 7f416cbfe640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:25.541+0000 7f416cbfe640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:25.541+0000 7f416cbfe640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:25.541+0000 7f416c3fd640 20 osd.0 111 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011764050s,-2.011575222s]) 2026-02-11T23:31:25.541+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e111 up_from 5 ping_stamp 2026-02-11T23:31:25.542880+0000/66.206626892s send_stamp 68.218391418s delta_ub 2.011764050s) -- 0x4c9ad80 con 0x4aca400 2026-02-11T23:31:25.541+0000 7f416c3fd640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:25.541+0000 7f416c3fd640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:25.541+0000 7f416c3fd640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:25.541+0000 7f416c3fd640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:26.252+0000 7f41633eb640 10 osd.0 111 tick 2026-02-11T23:31:26.252+0000 7f41633eb640 20 osd.0 111 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:26.329+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:26.329+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:26.329+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:56.330669+0000) 2026-02-11T23:31:26.431+0000 7f4162bea640 10 osd.0 111 tick_without_osd_lock 2026-02-11T23:31:26.431+0000 7f4162bea640 0 osd.0 111 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:31:26.431+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:26.431+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.236 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:26.431+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:26.433, recovery is active?:false restrictions:<....> 2026-02-11T23:31:26.431+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:26.431+0000 7f4162bea640 10 osd.0 111 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:26.431+0000 7f4162bea640 20 osd.0 111 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:26.431+0000 7f4162bea640 10 osd.0 111 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:26.431+0000 7f4162bea640 20 osd.0 111 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:27.077+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 51 ==== osd_map(112..112 src has 1..112) ==== 618+0+0 (crc 0 0 0) 0x5954c40 con 0x4acbc00 2026-02-11T23:31:27.077+0000 7f415cf02640 20 osd.0 111 OSD::ms_dispatch: osd_map(112..112 src has 1..112) 2026-02-11T23:31:27.077+0000 7f415cf02640 20 osd.0 111 _dispatch 0x5954c40 osd_map(112..112 src has 1..112) 2026-02-11T23:31:27.077+0000 7f415cf02640 3 osd.0 111 handle_osd_map epochs [112,112], i have 111, src has [1,112] 2026-02-11T23:31:27.077+0000 7f415cf02640 10 osd.0 111 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:31:27.077+0000 7f415cf02640 10 osd.0 111 handle_osd_map got inc map for epoch 112 2026-02-11T23:31:27.077+0000 7f415cf02640 20 osd.0 111 got_full_map 112, nothing requested 2026-02-11T23:31:27.077+0000 7f415cf02640 20 osd.0 111 trim_maps: min=1 oldest_map=1 2026-02-11T23:31:27.077+0000 7f415cf02640 20 osd.0 111 handle_osd_map pg_num_history pg_num_history(e112 pg_nums {1={27=1},2={34=1},3={42=1},4={49=1},5={56=1},6={64=1},7={70=1},8={77=1},9={85=1},10={92=1}} deleted_pools 99,2,100,3,101,5,102,6,103,7,104,8,105,9,106,10) 2026-02-11T23:31:27.077+0000 7f415cf02640 10 snap_mapper record_purged_snaps purged_snaps {112={}} 2026-02-11T23:31:27.077+0000 7f415cf02640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:31:27.077+0000 7f415cf02640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e112 maps [1~112] lci=[5,112] tlb=1) 2026-02-11T23:31:27.078+0000 7f41591a6640 10 osd.0 111 _committed_osd_maps 112..112 2026-02-11T23:31:27.078+0000 7f41591a6640 10 osd.0 111 advance to epoch 112 (<= last 112 <= newest_map 112) 2026-02-11T23:31:27.078+0000 7f41591a6640 20 osd.0 112 consume_map version 112 2026-02-11T23:31:27.078+0000 7f41591a6640 20 osd.0 111 await_reserved_maps epoch:112 2026-02-11T23:31:27.078+0000 7f41591a6640 20 osd.0 111 await_reserved_maps done 0 2026-02-11T23:31:27.078+0000 7f41591a6640 20 osd.0 112 consume_map 112 -- publish done 2026-02-11T23:31:27.078+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 1 slots 2026-02-11T23:31:27.078+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x37582c0 2026-02-11T23:31:27.078+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot 4.0 has no pg and waiting_for_split 2026-02-11T23:31:27.078+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:27.078+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:27.078+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:27.078+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:27.078+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:27.078+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:27.078+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:27.078+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:27.078+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:27.078+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:27.078+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:27.078+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:27.078+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:27.078+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:27.078+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:27.078+0000 7f41591a6640 20 osd.0 112 prune_pg_created 2026-02-11T23:31:27.078+0000 7f41591a6640 10 osd.0:0.consume_map 112 (was 111) 2026-02-11T23:31:27.078+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:31:27.078+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 empty, pruning 2026-02-11T23:31:27.078+0000 7f41591a6640 10 osd.0:1.consume_map 112 (was 111) 2026-02-11T23:31:27.078+0000 7f41591a6640 10 osd.0:2.consume_map 112 (was 111) 2026-02-11T23:31:27.078+0000 7f41591a6640 10 osd.0:3.consume_map 112 (was 111) 2026-02-11T23:31:27.078+0000 7f41591a6640 10 osd.0:4.consume_map 112 (was 111) 2026-02-11T23:31:27.078+0000 7f41591a6640 10 osd.0:5.consume_map 112 (was 111) 2026-02-11T23:31:27.078+0000 7f41591a6640 10 osd.0:6.consume_map 112 (was 111) 2026-02-11T23:31:27.078+0000 7f41591a6640 10 osd.0:7.consume_map 112 (was 111) 2026-02-11T23:31:27.078+0000 7f41591a6640 10 osd.0 112 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:31:27.078+0000 7f41591a6640 7 osd.0 112 activate_map version 112 2026-02-11T23:31:27.078+0000 7f4150194640 20 osd.0 112 agent_entry empty queue 2026-02-11T23:31:27.079+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 27 ==== pg_query2(1.0 1.0 query(info 0'0 epoch_sent 112) e112/112) ==== 158+0+0 (unknown 0 0 0) 0x4fca960 con 0x4acb000 2026-02-11T23:31:27.079+0000 7f415aefe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 26 ==== osd_map(112..112 src has 1..112) ==== 618+0+0 (unknown 0 0 0) 0x4aa2000 con 0x4acb000 2026-02-11T23:31:27.079+0000 7f415aefe640 20 osd.0 112 OSD::ms_dispatch: osd_map(112..112 src has 1..112) 2026-02-11T23:31:27.079+0000 7f415aefe640 20 osd.0 112 _dispatch 0x4aa2000 osd_map(112..112 src has 1..112) 2026-02-11T23:31:27.079+0000 7f416abfa640 15 osd.0 112 enqueue_peering_evt 1.0 epoch_sent: 112 epoch_requested: 112 MQuery 1.0 from 1 query_epoch 112 query: query(info 0'0 epoch_sent 112) 2026-02-11T23:31:27.079+0000 7f415aefe640 3 osd.0 112 handle_osd_map epochs [112,112], i have 112, src has [1,112] 2026-02-11T23:31:27.079+0000 7f415aefe640 10 osd.0 112 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:31:27.079+0000 7f415aefe640 10 osd.0 112 no new maps here, dropping 2026-02-11T23:31:27.079+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 112 epoch_requested: 112 MQuery 1.0 from 1 query_epoch 112 query: query(info 0'0 epoch_sent 112)) class_id 2 prio 255 cost 10 e112) 2026-02-11T23:31:27.079+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 (new) to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.079+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 112 epoch_requested: 112 MQuery 1.0 from 1 query_epoch 112 query: query(info 0'0 epoch_sent 112)) class_id 2immediate prio 255 cost 10 e112) queued 2026-02-11T23:31:27.079+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.079+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 112 epoch_requested: 112 MQuery 1.0 from 1 query_epoch 112 query: query(info 0'0 epoch_sent 112)) class_id 2immediate prio 255 cost 10 e112) pg 0 2026-02-11T23:31:27.079+0000 7f414b98b640 10 osd.0 112 handle_pg_query_nopg 1.0 2026-02-11T23:31:27.079+0000 7f414b98b640 10 osd.0 112 pg 1.0 dne 2026-02-11T23:31:27.079+0000 7f414b98b640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:27.079+0000 7f414b98b640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:27.079+0000 7f414b98b640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:27.079+0000 7f414b98b640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/3043114013 map epoch 111 -> 112 (shared) 2026-02-11T23:31:27.079+0000 7f414b98b640 10 osd.0 112 send_incremental_map epoch range: (112, 112] to v2:127.0.0.1:6812/3043114013 2026-02-11T23:31:27.079+0000 7f414b98b640 10 osd.0 112 add_map_inc_bl 112 574 bytes 2026-02-11T23:31:27.079+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- osd_map(112..112 src has 1..112) -- 0x4aa21c0 con 0x4acb000 2026-02-11T23:31:27.079+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_notify2(1.0 1.0 (query:112 sent:112 1.0( DNE empty local-lis/les=0/0 n=0 ec=0/0 lis/c=0/0 les/c/f=0/0/0 sis=0) ([0,0] all_participants={} intervals=[])) e112/112) -- 0x5a8d500 con 0x4acb000 2026-02-11T23:31:27.079+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.079+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.179+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 29 ==== pg_query2(1.0 1.0 query(info 0'0 epoch_sent 113) e113/113) ==== 158+0+0 (unknown 0 0 0) 0x4fca960 con 0x4acb000 2026-02-11T23:31:27.179+0000 7f415aefe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 28 ==== osd_map(113..113 src has 1..113) ==== 651+0+0 (unknown 0 0 0) 0x4aa21c0 con 0x4acb000 2026-02-11T23:31:27.179+0000 7f415aefe640 20 osd.0 112 OSD::ms_dispatch: osd_map(113..113 src has 1..113) 2026-02-11T23:31:27.179+0000 7f415aefe640 20 osd.0 112 _dispatch 0x4aa21c0 osd_map(113..113 src has 1..113) 2026-02-11T23:31:27.179+0000 7f416abfa640 15 osd.0 112 enqueue_peering_evt 1.0 epoch_sent: 113 epoch_requested: 113 MQuery 1.0 from 1 query_epoch 113 query: query(info 0'0 epoch_sent 113) 2026-02-11T23:31:27.179+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 113 epoch_requested: 113 MQuery 1.0 from 1 query_epoch 113 query: query(info 0'0 epoch_sent 113)) class_id 2 prio 255 cost 10 e113) 2026-02-11T23:31:27.179+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.179+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 113 epoch_requested: 113 MQuery 1.0 from 1 query_epoch 113 query: query(info 0'0 epoch_sent 113)) class_id 2immediate prio 255 cost 10 e113) queued 2026-02-11T23:31:27.179+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.179+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 113 epoch_requested: 113 MQuery 1.0 from 1 query_epoch 113 query: query(info 0'0 epoch_sent 113)) class_id 2immediate prio 255 cost 10 e113) pg 0 2026-02-11T23:31:27.179+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 map 113 > 112 2026-02-11T23:31:27.179+0000 7f414f993640 20 osd.0 op_wq _add_slot_waiter 1.0 peering, item epoch is 113, will wait on OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 113 epoch_requested: 113 MQuery 1.0 from 1 query_epoch 113 query: query(info 0'0 epoch_sent 113)) class_id 2immediate prio 255 cost 10 e113) 2026-02-11T23:31:27.179+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.179+0000 7f415aefe640 3 osd.0 112 handle_osd_map epochs [113,113], i have 112, src has [1,113] 2026-02-11T23:31:27.179+0000 7f415aefe640 10 osd.0 112 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:31:27.179+0000 7f415aefe640 10 osd.0 112 handle_osd_map got inc map for epoch 113 2026-02-11T23:31:27.179+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.179+0000 7f415aefe640 20 osd.0 112 got_full_map 113, nothing requested 2026-02-11T23:31:27.179+0000 7f415aefe640 20 osd.0 112 trim_maps: min=1 oldest_map=1 2026-02-11T23:31:27.179+0000 7f415aefe640 20 osd.0 112 handle_osd_map pg_num_history pg_num_history(e113 pg_nums {1={27=1},2={34=1},3={42=1},4={49=1},5={56=1},6={64=1},7={70=1},8={77=1},9={85=1},10={92=1}} deleted_pools 99,2,100,3,101,5,102,6,103,7,104,8,105,9,106,10) 2026-02-11T23:31:27.179+0000 7f415aefe640 10 snap_mapper record_purged_snaps purged_snaps {113={}} 2026-02-11T23:31:27.179+0000 7f415aefe640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:31:27.179+0000 7f415aefe640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e113 maps [1~113] lci=[5,113] tlb=1) 2026-02-11T23:31:27.179+0000 7f41591a6640 10 osd.0 112 _committed_osd_maps 113..113 2026-02-11T23:31:27.179+0000 7f41591a6640 10 osd.0 112 advance to epoch 113 (<= last 113 <= newest_map 113) 2026-02-11T23:31:27.179+0000 7f41591a6640 20 osd.0 113 consume_map version 113 2026-02-11T23:31:27.179+0000 7f41591a6640 20 osd.0 112 await_reserved_maps epoch:113 2026-02-11T23:31:27.179+0000 7f41591a6640 20 osd.0 112 await_reserved_maps done 0 2026-02-11T23:31:27.179+0000 7f41591a6640 20 osd.0 113 consume_map 113 -- publish done 2026-02-11T23:31:27.179+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 1 slots 2026-02-11T23:31:27.179+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3701340 2026-02-11T23:31:27.179+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot 1.0 has no pg and waiting_for_split 2026-02-11T23:31:27.179+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:27.179+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:27.179+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:27.179+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:27.179+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:27.179+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:27.179+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:27.179+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:27.179+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:27.179+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:27.179+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:27.179+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:27.179+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:27.179+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:27.179+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:27.179+0000 7f41591a6640 20 osd.0 113 prune_pg_created 2026-02-11T23:31:27.179+0000 7f41591a6640 10 osd.0:0.consume_map 113 (was 112) 2026-02-11T23:31:27.179+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:31:27.179+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 pending_peering first epoch 113 <= 113, requeueing 2026-02-11T23:31:27.179+0000 7f41591a6640 20 osd.0:0._wake_pg_slot _wake_pg_slot 1.0 to_process <> waiting <> waiting_peering {113=} 2026-02-11T23:31:27.179+0000 7f41591a6640 10 osd.0:1.consume_map 113 (was 112) 2026-02-11T23:31:27.179+0000 7f41591a6640 10 osd.0:2.consume_map 113 (was 112) 2026-02-11T23:31:27.179+0000 7f41591a6640 10 osd.0:3.consume_map 113 (was 112) 2026-02-11T23:31:27.179+0000 7f41591a6640 10 osd.0:4.consume_map 113 (was 112) 2026-02-11T23:31:27.179+0000 7f41591a6640 10 osd.0:5.consume_map 113 (was 112) 2026-02-11T23:31:27.179+0000 7f41591a6640 10 osd.0:6.consume_map 113 (was 112) 2026-02-11T23:31:27.179+0000 7f41591a6640 10 osd.0:7.consume_map 113 (was 112) 2026-02-11T23:31:27.179+0000 7f41591a6640 10 osd.0 113 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:31:27.179+0000 7f41591a6640 7 osd.0 113 activate_map version 113 2026-02-11T23:31:27.179+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.179+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 113 epoch_requested: 113 MQuery 1.0 from 1 query_epoch 113 query: query(info 0'0 epoch_sent 113)) class_id 2immediate prio 255 cost 10 e113) queued 2026-02-11T23:31:27.179+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.179+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 113 epoch_requested: 113 MQuery 1.0 from 1 query_epoch 113 query: query(info 0'0 epoch_sent 113)) class_id 2immediate prio 255 cost 10 e113) pg 0 2026-02-11T23:31:27.179+0000 7f414f993640 10 osd.0 113 handle_pg_query_nopg 1.0 2026-02-11T23:31:27.179+0000 7f414f993640 10 osd.0 113 pg 1.0 dne 2026-02-11T23:31:27.179+0000 7f4150194640 20 osd.0 113 agent_entry empty queue 2026-02-11T23:31:27.179+0000 7f414f993640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:27.179+0000 7f414f993640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:27.179+0000 7f414f993640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:27.179+0000 7f414f993640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/3043114013 map epoch 112 -> 113 (shared) 2026-02-11T23:31:27.179+0000 7f414f993640 10 osd.0 113 send_incremental_map epoch range: (113, 113] to v2:127.0.0.1:6812/3043114013 2026-02-11T23:31:27.179+0000 7f414f993640 10 osd.0 113 add_map_inc_bl 113 607 bytes 2026-02-11T23:31:27.179+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- osd_map(113..113 src has 1..113) -- 0x4a64000 con 0x4acb000 2026-02-11T23:31:27.179+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_notify2(1.0 1.0 (query:113 sent:113 1.0( DNE empty local-lis/les=0/0 n=0 ec=0/0 lis/c=0/0 les/c/f=0/0/0 sis=0) ([0,0] all_participants={} intervals=[])) e113/113) -- 0x5b24300 con 0x4acb000 2026-02-11T23:31:27.179+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.230+0000 7f4147983640 5 osd.0 113 heartbeat osd_stat(store_statfs(0x384000/0x0/0x384000, data 0x7f242/0x120000, compress 0x0/0x0/0x0, omap 0xa403, meta 0x1a45bfd), peers [1,2,3,4] op hist []) 2026-02-11T23:31:27.230+0000 7f4147983640 20 osd.0 113 check_full_status cur ratio 0, physical ratio 0, new state none 2026-02-11T23:31:27.230+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6816/3043114013,v1:127.0.0.1:6817/3043114013] -- osd_ping(ping e113 up_from 5 ping_stamp 2026-02-11T23:31:27.231622+0000/69.907028198s send_stamp 69.907028198s delta_ub 2.011764050s) -- 0x4ecc000 con 0x48c4400 2026-02-11T23:31:27.230+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6814/3043114013,v1:127.0.0.1:6815/3043114013] -- osd_ping(ping e113 up_from 5 ping_stamp 2026-02-11T23:31:27.231622+0000/69.907028198s send_stamp 69.907028198s delta_ub 2.011764050s) -- 0x4ecc180 con 0x4640c00 2026-02-11T23:31:27.230+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6824/2365263801,v1:127.0.0.1:6825/2365263801] -- osd_ping(ping e113 up_from 5 ping_stamp 2026-02-11T23:31:27.231622+0000/69.907028198s send_stamp 69.907028198s delta_ub 4.566924572s) -- 0x4ecc300 con 0x48c4000 2026-02-11T23:31:27.230+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6822/2365263801,v1:127.0.0.1:6823/2365263801] -- osd_ping(ping e113 up_from 5 ping_stamp 2026-02-11T23:31:27.231622+0000/69.907028198s send_stamp 69.907028198s delta_ub 4.566924572s) -- 0x4ecc480 con 0x48cd000 2026-02-11T23:31:27.230+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6832/2699377081,v1:127.0.0.1:6833/2699377081] -- osd_ping(ping e113 up_from 5 ping_stamp 2026-02-11T23:31:27.231622+0000/69.907028198s send_stamp 69.907028198s delta_ub 7.219584942s) -- 0x4ecc600 con 0x48cd400 2026-02-11T23:31:27.230+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6830/2699377081,v1:127.0.0.1:6831/2699377081] -- osd_ping(ping e113 up_from 5 ping_stamp 2026-02-11T23:31:27.231622+0000/69.907028198s send_stamp 69.907028198s delta_ub 7.219584942s) -- 0x4ecc780 con 0x48cd800 2026-02-11T23:31:27.230+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6840/3331151400,v1:127.0.0.1:6841/3331151400] -- osd_ping(ping e113 up_from 5 ping_stamp 2026-02-11T23:31:27.231622+0000/69.907028198s send_stamp 69.907028198s delta_ub 9.836735725s) -- 0x4ecc900 con 0x507bc00 2026-02-11T23:31:27.230+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6838/3331151400,v1:127.0.0.1:6839/3331151400] -- osd_ping(ping e113 up_from 5 ping_stamp 2026-02-11T23:31:27.231622+0000/69.907028198s send_stamp 69.907028198s delta_ub 9.836735725s) -- 0x4ecca80 con 0x5282000 2026-02-11T23:31:27.230+0000 7f416cbfe640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6824/2365263801 20 ==== osd_ping(ping_reply e113 up_from 14 ping_stamp 2026-02-11T23:31:27.231622+0000/69.907028198s send_stamp 65.340332031s delta_ub -4.566695213s) ==== 2033+0+0 (crc 0 0 0) 0x4ecc300 con 0x48c4000 2026-02-11T23:31:27.230+0000 7f416cbfe640 20 osd.0 113 get_con_osd_cluster to osd.2 from_epoch 113 2026-02-11T23:31:27.230+0000 7f416cbfe640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:27.230+0000 7f416cbfe640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:27.230+0000 7f416cbfe640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6820/2365263801 updating session's projected_epoch from 111 to ping map epoch of 113 2026-02-11T23:31:27.230+0000 7f416cbfe640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:27.230+0000 7f416cbfe640 20 osd.0 113 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566864491s,-4.566695213s]) 2026-02-11T23:31:27.230+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6832/2699377081 20 ==== osd_ping(ping_reply e112 up_from 19 ping_stamp 2026-02-11T23:31:27.231622+0000/69.907028198s send_stamp 62.687679291s delta_ub -7.219347477s) ==== 2033+0+0 (crc 0 0 0) 0x4ecc900 con 0x48cd400 2026-02-11T23:31:27.230+0000 7f416abfa640 20 osd.0 113 get_con_osd_cluster to osd.3 from_epoch 113 2026-02-11T23:31:27.230+0000 7f416abfa640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:27.230+0000 7f416abfa640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:27.230+0000 7f416abfa640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6828/2699377081 updating session's projected_epoch from 111 to ping map epoch of 112 2026-02-11T23:31:27.230+0000 7f416abfa640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6828/2699377081 map epoch 112 -> 113 (shared) 2026-02-11T23:31:27.230+0000 7f416abfa640 10 osd.0 113 send_incremental_map epoch range: (113, 113] to v2:127.0.0.1:6828/2699377081 2026-02-11T23:31:27.230+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6828/2699377081,v1:127.0.0.1:6829/2699377081] -- osd_map(113..113 src has 1..113) -- 0x4a64000 con 0x48ccc00 2026-02-11T23:31:27.230+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6830/2699377081 20 ==== osd_ping(ping_reply e112 up_from 19 ping_stamp 2026-02-11T23:31:27.231622+0000/69.907028198s send_stamp 62.687698364s delta_ub -7.219326973s) ==== 2033+0+0 (crc 0 0 0) 0x4ecca80 con 0x48cd800 2026-02-11T23:31:27.230+0000 7f416abfa640 20 osd.0 113 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219526768s,-7.219347477s]) 2026-02-11T23:31:27.230+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6816/3043114013 21 ==== osd_ping(ping_reply e113 up_from 10 ping_stamp 2026-02-11T23:31:27.231622+0000/69.907028198s send_stamp 67.895469666s delta_ub -2.011555910s) ==== 2033+0+0 (crc 0 0 0) 0x4ecc900 con 0x48c4400 2026-02-11T23:31:27.230+0000 7f416c3fd640 20 osd.0 113 get_con_osd_cluster to osd.3 from_epoch 113 2026-02-11T23:31:27.230+0000 7f416c3fd640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:27.230+0000 7f416c3fd640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:27.230+0000 7f416c3fd640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:27.230+0000 7f416c3fd640 20 osd.0 113 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219535828s,-7.219326973s]) 2026-02-11T23:31:27.230+0000 7f416abfa640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:27.230+0000 7f416abfa640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:27.230+0000 7f416abfa640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:27.230+0000 7f416abfa640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:27.230+0000 7f416abfa640 20 osd.0 113 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011788130s,-2.011555910s]) 2026-02-11T23:31:27.230+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6814/3043114013 21 ==== osd_ping(ping_reply e113 up_from 10 ping_stamp 2026-02-11T23:31:27.231622+0000/69.907028198s send_stamp 67.895500183s delta_ub -2.011524916s) ==== 2033+0+0 (crc 0 0 0) 0x4ecca80 con 0x4640c00 2026-02-11T23:31:27.230+0000 7f416c3fd640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:27.230+0000 7f416c3fd640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:27.230+0000 7f416c3fd640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:27.230+0000 7f416c3fd640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:27.230+0000 7f416c3fd640 20 osd.0 113 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011766434s,-2.011524916s]) 2026-02-11T23:31:27.230+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6822/2365263801 20 ==== osd_ping(ping_reply e113 up_from 14 ping_stamp 2026-02-11T23:31:27.231622+0000/69.907028198s send_stamp 65.340347290s delta_ub -4.566675663s) ==== 2033+0+0 (crc 0 0 0) 0x4ecc900 con 0x48cd000 2026-02-11T23:31:27.230+0000 7f416abfa640 20 osd.0 113 get_con_osd_cluster to osd.2 from_epoch 113 2026-02-11T23:31:27.230+0000 7f416abfa640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:27.230+0000 7f416abfa640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:27.230+0000 7f416abfa640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:27.230+0000 7f416abfa640 20 osd.0 113 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566926003s,-4.566675663s]) 2026-02-11T23:31:27.230+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6838/3331151400 19 ==== osd_ping(ping_reply e112 up_from 23 ping_stamp 2026-02-11T23:31:27.231622+0000/69.907028198s send_stamp 60.070545197s delta_ub -9.836483955s) ==== 2033+0+0 (crc 0 0 0) 0x4ecca80 con 0x5282000 2026-02-11T23:31:27.230+0000 7f416c3fd640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:27.230+0000 7f416c3fd640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:27.230+0000 7f416c3fd640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:27.230+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6840/3331151400 19 ==== osd_ping(ping_reply e112 up_from 23 ping_stamp 2026-02-11T23:31:27.231622+0000/69.907028198s send_stamp 60.070522308s delta_ub -9.836503029s) ==== 2033+0+0 (crc 0 0 0) 0x4ecc900 con 0x507bc00 2026-02-11T23:31:27.230+0000 7f416c3fd640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/3331151400 updating session's projected_epoch from 111 to ping map epoch of 112 2026-02-11T23:31:27.230+0000 7f416c3fd640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/3331151400 map epoch 112 -> 113 (shared) 2026-02-11T23:31:27.230+0000 7f416c3fd640 10 osd.0 113 send_incremental_map epoch range: (113, 113] to v2:127.0.0.1:6836/3331151400 2026-02-11T23:31:27.230+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- osd_map(113..113 src has 1..113) -- 0x4a31a40 con 0x507ac00 2026-02-11T23:31:27.230+0000 7f416c3fd640 20 osd.0 113 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836747169s,-9.836483955s]) 2026-02-11T23:31:27.230+0000 7f416abfa640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:27.230+0000 7f416abfa640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:27.230+0000 7f416abfa640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:27.230+0000 7f416abfa640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:27.230+0000 7f416abfa640 20 osd.0 113 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836797714s,-9.836503029s]) 2026-02-11T23:31:27.233+0000 7f41633eb640 10 osd.0 113 tick 2026-02-11T23:31:27.233+0000 7f41633eb640 20 osd.0 113 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:27.280+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 30 ==== PGlog(1.0 log log((111'590,111'600], crt=111'600) pi ([107,112] all_participants={0, 1} intervals=[([107,111] acting={1})]) pg_lease(ru 83.945602417s ub 83.945602417s int 16.000000000s) e114/113) ==== 39009+0+0 (unknown 0 0 0) 0x5a94800 con 0x4acb000 2026-02-11T23:31:27.280+0000 7f416abfa640 15 osd.0 113 enqueue_peering_evt 1.0 epoch_sent: 114 epoch_requested: 113 MLogRec from 1 log log((111'590,111'600], crt=111'600) pi ([107,112] all_participants={0, 1} intervals=[([107,111] acting={1})]) pg_lease(ru 83.945602417s ub 83.945602417s int 16.000000000s) +create_info 2026-02-11T23:31:27.280+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 114 epoch_requested: 113 MLogRec from 1 log log((111'590,111'600], crt=111'600) pi ([107,112] all_participants={0, 1} intervals=[([107,111] acting={1})]) pg_lease(ru 83.945602417s ub 83.945602417s int 16.000000000s) +create_info) class_id 2 prio 255 cost 10 e114) 2026-02-11T23:31:27.280+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.280+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 114 epoch_requested: 113 MLogRec from 1 log log((111'590,111'600], crt=111'600) pi ([107,112] all_participants={0, 1} intervals=[([107,111] acting={1})]) pg_lease(ru 83.945602417s ub 83.945602417s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e114) queued 2026-02-11T23:31:27.280+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.280+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 114 epoch_requested: 113 MLogRec from 1 log log((111'590,111'600], crt=111'600) pi ([107,112] all_participants={0, 1} intervals=[([107,111] acting={1})]) pg_lease(ru 83.945602417s ub 83.945602417s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e114) pg 0 2026-02-11T23:31:27.281+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 map 114 > 113 2026-02-11T23:31:27.281+0000 7f414b98b640 20 osd.0 op_wq _add_slot_waiter 1.0 peering, item epoch is 114, will wait on OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 114 epoch_requested: 113 MLogRec from 1 log log((111'590,111'600], crt=111'600) pi ([107,112] all_participants={0, 1} intervals=[([107,111] acting={1})]) pg_lease(ru 83.945602417s ub 83.945602417s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e114) 2026-02-11T23:31:27.281+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.281+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.329+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:27.329+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:27.329+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:57.330731+0000) 2026-02-11T23:31:27.475+0000 7f4162bea640 10 osd.0 113 tick_without_osd_lock 2026-02-11T23:31:27.475+0000 7f4162bea640 0 osd.0 113 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:31:27.475+0000 7f4162bea640 20 osd.0 113 tick_without_osd_lock max_waiting_epoch 114, requesting new map 2026-02-11T23:31:27.475+0000 7f4162bea640 10 monclient: _renew_subs 2026-02-11T23:31:27.475+0000 7f4162bea640 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:31:27.475+0000 7f4162bea640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> v2:127.0.0.1:7180/0 -- mon_subscribe({osdmap=114}) -- 0x3814680 con 0x4acbc00 2026-02-11T23:31:27.475+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:27.475+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:27.475+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:27.475+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.236 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:27.475+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:27.476, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:27.475+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:27.475+0000 7f4162bea640 10 osd.0 113 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:27.475+0000 7f4162bea640 20 osd.0 113 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:27.475+0000 7f4162bea640 10 osd.0 113 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:27.475+0000 7f4162bea640 20 osd.0 113 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:27.475+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 52 ==== osd_map(114..115 src has 1..115) ==== 878+0+0 (crc 0 0 0) 0x4a64000 con 0x4acbc00 2026-02-11T23:31:27.475+0000 7f415cf02640 20 osd.0 113 OSD::ms_dispatch: osd_map(114..115 src has 1..115) 2026-02-11T23:31:27.475+0000 7f415cf02640 20 osd.0 113 _dispatch 0x4a64000 osd_map(114..115 src has 1..115) 2026-02-11T23:31:27.475+0000 7f415cf02640 3 osd.0 113 handle_osd_map epochs [114,115], i have 113, src has [1,115] 2026-02-11T23:31:27.475+0000 7f415cf02640 10 osd.0 113 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:31:27.475+0000 7f415cf02640 10 osd.0 113 handle_osd_map got inc map for epoch 114 2026-02-11T23:31:27.475+0000 7f415cf02640 20 osd.0 113 got_full_map 114, nothing requested 2026-02-11T23:31:27.475+0000 7f415cf02640 10 osd.0 113 handle_osd_map got inc map for epoch 115 2026-02-11T23:31:27.475+0000 7f415cf02640 20 osd.0 113 got_full_map 115, nothing requested 2026-02-11T23:31:27.475+0000 7f415cf02640 20 osd.0 113 trim_maps: min=1 oldest_map=1 2026-02-11T23:31:27.475+0000 7f415cf02640 20 osd.0 113 handle_osd_map pg_num_history pg_num_history(e115 pg_nums {1={27=1},2={34=1},3={42=1},4={49=1},5={56=1},6={64=1},7={70=1},8={77=1},9={85=1},10={92=1}} deleted_pools 99,2,100,3,101,5,102,6,103,7,104,8,105,9,106,10) 2026-02-11T23:31:27.475+0000 7f415cf02640 10 snap_mapper record_purged_snaps purged_snaps {114={},115={}} 2026-02-11T23:31:27.475+0000 7f415cf02640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:31:27.475+0000 7f415cf02640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e115 maps [1~115] lci=[5,115] tlb=1) 2026-02-11T23:31:27.475+0000 7f41591a6640 10 osd.0 113 _committed_osd_maps 114..115 2026-02-11T23:31:27.475+0000 7f41591a6640 10 osd.0 113 advance to epoch 114 (<= last 115 <= newest_map 115) 2026-02-11T23:31:27.475+0000 7f41591a6640 10 osd.0 114 advance to epoch 115 (<= last 115 <= newest_map 115) 2026-02-11T23:31:27.475+0000 7f41591a6640 20 osd.0 115 consume_map version 115 2026-02-11T23:31:27.475+0000 7f41591a6640 20 osd.0 113 await_reserved_maps epoch:115 2026-02-11T23:31:27.475+0000 7f41591a6640 20 osd.0 113 await_reserved_maps done 0 2026-02-11T23:31:27.475+0000 7f41591a6640 20 osd.0 115 consume_map 115 -- publish done 2026-02-11T23:31:27.475+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 1 slots 2026-02-11T23:31:27.475+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3701340 2026-02-11T23:31:27.475+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot 1.0 has no pg and waiting_for_split 2026-02-11T23:31:27.475+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:27.475+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:27.475+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:27.475+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:27.475+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:27.475+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:27.475+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:27.475+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:27.475+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:27.475+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:27.475+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:27.475+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:27.475+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:27.475+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:27.475+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:27.475+0000 7f41591a6640 20 osd.0 115 prune_pg_created 2026-02-11T23:31:27.475+0000 7f41591a6640 10 osd.0:0.consume_map 115 (was 113) 2026-02-11T23:31:27.475+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:31:27.475+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 pending_peering first epoch 114 <= 115, requeueing 2026-02-11T23:31:27.475+0000 7f41591a6640 20 osd.0:0._wake_pg_slot _wake_pg_slot 1.0 to_process <> waiting <> waiting_peering {114=} 2026-02-11T23:31:27.475+0000 7f41591a6640 10 osd.0:1.consume_map 115 (was 113) 2026-02-11T23:31:27.475+0000 7f41591a6640 10 osd.0:2.consume_map 115 (was 113) 2026-02-11T23:31:27.475+0000 7f41591a6640 10 osd.0:3.consume_map 115 (was 113) 2026-02-11T23:31:27.475+0000 7f41591a6640 10 osd.0:4.consume_map 115 (was 113) 2026-02-11T23:31:27.475+0000 7f41591a6640 10 osd.0:5.consume_map 115 (was 113) 2026-02-11T23:31:27.475+0000 7f41591a6640 10 osd.0:6.consume_map 115 (was 113) 2026-02-11T23:31:27.475+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.475+0000 7f41591a6640 10 osd.0:7.consume_map 115 (was 113) 2026-02-11T23:31:27.475+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 114 epoch_requested: 113 MLogRec from 1 log log((111'590,111'600], crt=111'600) pi ([107,112] all_participants={0, 1} intervals=[([107,111] acting={1})]) pg_lease(ru 83.945602417s ub 83.945602417s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e114) queued 2026-02-11T23:31:27.475+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.475+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 114 epoch_requested: 113 MLogRec from 1 log log((111'590,111'600], crt=111'600) pi ([107,112] all_participants={0, 1} intervals=[([107,111] acting={1})]) pg_lease(ru 83.945602417s ub 83.945602417s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e114) pg 0 2026-02-11T23:31:27.475+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 no pg, should create on OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 114 epoch_requested: 113 MLogRec from 1 log log((111'590,111'600], crt=111'600) pi ([107,112] all_participants={0, 1} intervals=[([107,111] acting={1})]) pg_lease(ru 83.945602417s ub 83.945602417s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e114) 2026-02-11T23:31:27.475+0000 7f414b98b640 10 osd.0 115 _make_pg 1.0 2026-02-11T23:31:27.475+0000 7f414b98b640 10 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:1, shard:-1, shard_prefix: , prefixes: {}] SnapMapper 2026-02-11T23:31:27.475+0000 7f414b98b640 20 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:1, shard:-1, shard_prefix: , prefixes: {}] update_bits new_bits: 0 2026-02-11T23:31:27.475+0000 7f414b98b640 20 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:1, shard:-1, shard_prefix: , prefixes: {"0000000000000001."}] update_bits prefix updated 2026-02-11T23:31:27.475+0000 7f414b98b640 10 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:1, shard:-1, shard_prefix: , prefixes: {"0000000000000001."}] reset_prefix_itr::from <0> to ::update_bits 2026-02-11T23:31:27.475+0000 7f414b98b640 5 osd.0 pg_epoch: 113 pg[1.0(unlocked)] enter Initial 2026-02-11T23:31:27.475+0000 7f414b98b640 20 osd.0 pg_epoch: 113 pg[1.0(unlocked)] enter NotTrimming 2026-02-11T23:31:27.475+0000 7f414b98b640 10 osd.0 pg_epoch: 113 pg[1.0(unlocked)] scrubber: FSM: -- state -->> NotActive 2026-02-11T23:31:27.475+0000 7f414b98b640 10 osd.0 pg_epoch: 113 pg[1.0( DNE empty local-lis/les=0/0 n=0 ec=0/0 lis/c=0/0 les/c/f=0/0/0 sis=0) [] r=-1 lpr=0 crt=0'0 unknown mbc={}] PeeringState::init init role -1 up [1,0] acting [1] history ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113 past_intervals ([107,112] all_participants={0, 1} intervals=[([107,111] acting={1})]) 2026-02-11T23:31:27.475+0000 7f414b98b640 20 osd.0 pg_epoch: 113 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) [1,0]/[1] r=-1 lpr=0 pi=[107,113)/1 crt=0'0 unknown mbc={}] PeeringState::on_new_interval 2026-02-11T23:31:27.475+0000 7f414b98b640 20 osd.0 pg_epoch: 113 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) [1,0]/[1] r=-1 lpr=0 pi=[107,113)/1 crt=0'0 unknown mbc={}] PeeringState::on_new_interval upacting_features 0x3f0ffffffffdffff from [1]+[1,0] 2026-02-11T23:31:27.475+0000 7f414b98b640 20 osd.0 pg_epoch: 113 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) [1,0]/[1] r=-1 lpr=0 pi=[107,113)/1 crt=0'0 unknown mbc={}] PeeringState::on_new_interval checking missing set deletes flag. missing = missing(0 may_include_deletes = 1) 2026-02-11T23:31:27.475+0000 7f414b98b640 10 osd.0 pg_epoch: 113 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) [1,0]/[1] r=-1 lpr=0 pi=[107,113)/1 crt=0'0 unknown mbc={}] PeeringState::init_hb_stamps now [] 2026-02-11T23:31:27.475+0000 7f414b98b640 10 osd.0 pg_epoch: 113 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) [1,0]/[1] r=-1 lpr=0 pi=[107,113)/1 crt=0'0 unknown mbc={}] PeeringState::on_new_interval prior_readable_until_ub 0.000000000s (mnow 70.152511597s + 0.000000000s) 2026-02-11T23:31:27.475+0000 7f414b98b640 10 osd.0 pg_epoch: 113 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) [1,0]/[1] r=-1 lpr=0 pi=[107,113)/1 crt=0'0 unknown mbc={}] cancel_recovery 2026-02-11T23:31:27.475+0000 7f414b98b640 10 osd.0 pg_epoch: 113 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) [1,0]/[1] r=-1 lpr=0 pi=[107,113)/1 crt=0'0 unknown mbc={}] clear_recovery_state 2026-02-11T23:31:27.475+0000 7f414b98b640 15 osd.0 pg_epoch: 113 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) [1,0]/[1] r=-1 lpr=0 pi=[107,113)/1 crt=0'0 unknown mbc={}] _clear_recovery_state 2026-02-11T23:31:27.475+0000 7f414b98b640 10 osd.0 pg_epoch: 113 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) [1,0]/[1] r=-1 lpr=0 pi=[107,113)/1 crt=0'0 unknown mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:31:27.475+0000 7f414b98b640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:31:27.475+0000 7f414b98b640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:31:27.475+0000 7f414b98b640 6 write_log_and_missing with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, writeout_from: 4294967295'18446744073709551615, trimmed: , trimmed_dups: , clear_divergent_priors: 0 2026-02-11T23:31:27.475+0000 7f414b98b640 10 _write_log_and_missing clearing up to 0'0 dirty_to_dups=0'0 dirty_from_dups=4294967295'18446744073709551615 write_from_dups=4294967295'18446744073709551615 trimmed_dups.size()=0 2026-02-11T23:31:27.475+0000 7f414b98b640 10 _write_log_and_missing going to encode log.dups.size()=0 2026-02-11T23:31:27.475+0000 7f414b98b640 10 _write_log_and_missing 1st round encoded log.dups.size()=0 2026-02-11T23:31:27.475+0000 7f414b98b640 10 _write_log_and_missing 2st round encoded log.dups.size()=0 2026-02-11T23:31:27.475+0000 7f414b98b640 10 end of _write_log_and_missing 2026-02-11T23:31:27.475+0000 7f414b98b640 10 osd.0 pg_epoch: 113 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) [1,0]/[1] r=-1 lpr=0 pi=[107,113)/1 crt=0'0 unknown mbc={}] handle_initialize 2026-02-11T23:31:27.475+0000 7f414b98b640 5 osd.0 pg_epoch: 113 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) [1,0]/[1] r=-1 lpr=0 pi=[107,113)/1 crt=0'0 unknown mbc={}] exit Initial 0.000090 0 0.000000 2026-02-11T23:31:27.475+0000 7f414b98b640 5 osd.0 pg_epoch: 113 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) [1,0]/[1] r=-1 lpr=0 pi=[107,113)/1 crt=0'0 unknown mbc={}] enter Reset 2026-02-11T23:31:27.475+0000 7f414b98b640 20 osd.0 pg_epoch: 113 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) [1,0]/[1] r=-1 lpr=0 pi=[107,113)/1 crt=0'0 unknown mbc={}] PeeringState::set_last_peering_reset set_last_peering_reset 113 2026-02-11T23:31:27.475+0000 7f414b98b640 10 osd.0 pg_epoch: 113 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 crt=0'0 unknown mbc={}] PeeringState::set_last_peering_reset Clearing blocked outgoing recovery messages 2026-02-11T23:31:27.475+0000 7f414b98b640 10 osd.0 pg_epoch: 113 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 crt=0'0 unknown mbc={}] PeeringState::set_last_peering_reset Not blocking outgoing recovery messages 2026-02-11T23:31:27.475+0000 7f414b98b640 10 osd.0 pg_epoch: 113 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 crt=0'0 unknown mbc={}] handle_activate_map: epoch range: 113..113 2026-02-11T23:31:27.475+0000 7f414b98b640 10 osd.0 pg_epoch: 113 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 crt=0'0 unknown mbc={}] PeeringState::activate_map 2026-02-11T23:31:27.475+0000 7f414b98b640 5 osd.0 pg_epoch: 113 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 crt=0'0 unknown mbc={}] exit Reset 0.000024 1 0.000023 2026-02-11T23:31:27.475+0000 7f414b98b640 5 osd.0 pg_epoch: 113 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 crt=0'0 unknown mbc={}] enter Started 2026-02-11T23:31:27.475+0000 7f414b98b640 5 osd.0 pg_epoch: 113 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 crt=0'0 unknown mbc={}] enter Start 2026-02-11T23:31:27.475+0000 7f414b98b640 1 osd.0 pg_epoch: 113 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 crt=0'0 unknown mbc={}] state: transitioning to Stray 2026-02-11T23:31:27.475+0000 7f414b98b640 5 osd.0 pg_epoch: 113 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 crt=0'0 unknown mbc={}] exit Start 0.000003 0 0.000000 2026-02-11T23:31:27.475+0000 7f414b98b640 5 osd.0 pg_epoch: 113 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 crt=0'0 unknown mbc={}] enter Started/Stray 2026-02-11T23:31:27.475+0000 7f414b98b640 20 osd.0 pg_epoch: 113 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 crt=0'0 unknown mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 113 while current is 113 2026-02-11T23:31:27.475+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.475+0000 7f414b98b640 10 osd.0 pg_epoch: 113 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 crt=0'0 unknown mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:31:27.475+0000 7f414b98b640 10 osd.0 115 handle_pg_create_info new pg pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 crt=0'0 unknown mbc={}] 2026-02-11T23:31:27.475+0000 7f414b98b640 10 osd.0:0._attach_pg 1.0 0x4ef0000 2026-02-11T23:31:27.475+0000 7f414b98b640 20 osd.0:0._wake_pg_slot _wake_pg_slot 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.475+0000 7f414b98b640 20 osd.0 115 identify_splits_and_merges 1.0 e113 to e115 2026-02-11T23:31:27.475+0000 7f414b98b640 20 osd.0 115 identify_splits_and_merges 1.0 e113 to e115 pg_nums {27=1} 2026-02-11T23:31:27.475+0000 7f414b98b640 10 osd.0:0._prime_splits 2026-02-11T23:31:27.475+0000 7f414b98b640 10 osd.0 pg_epoch: 113 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 crt=0'0 unknown mbc={}] handle_advance_map: 114 2026-02-11T23:31:27.475+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 1.0 113 -> 114 2026-02-11T23:31:27.475+0000 7f41591a6640 10 osd.0 115 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:31:27.475+0000 7f414b98b640 10 osd.0 pg_epoch: 113 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 crt=0'0 unknown mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1] -- 1/1 2026-02-11T23:31:27.475+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.475+0000 7f414b98b640 10 osd.0 pg_epoch: 114 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 crt=0'0 unknown mbc={}] state: Started advmap 2026-02-11T23:31:27.475+0000 7f41591a6640 15 osd.0 115 enqueue_peering_evt 1.0 epoch_sent: 115 epoch_requested: 115 NullEvt 2026-02-11T23:31:27.475+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 115 epoch_requested: 115 NullEvt) class_id 2 prio 255 cost 10 e115) 2026-02-11T23:31:27.475+0000 7f41591a6640 7 osd.0 115 activate_map version 115 2026-02-11T23:31:27.475+0000 7f414b98b640 10 osd.0 pg_epoch: 114 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 crt=0'0 unknown mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:31:27.475+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.475+0000 7f414b98b640 20 osd.0 115 advance_pg new pool opts old pool opts 2026-02-11T23:31:27.475+0000 7f4150194640 20 osd.0 115 agent_entry empty queue 2026-02-11T23:31:27.475+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 115 epoch_requested: 115 NullEvt) class_id 2immediate prio 255 cost 10 e115) queued 2026-02-11T23:31:27.475+0000 7f414b98b640 10 osd.0 pg_epoch: 114 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 crt=0'0 unknown mbc={}] handle_advance_map: 115 2026-02-11T23:31:27.475+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 1.0 114 -> 115 2026-02-11T23:31:27.475+0000 7f414b98b640 10 osd.0 pg_epoch: 114 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 crt=0'0 unknown mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1] -- 1/1 2026-02-11T23:31:27.475+0000 7f414b98b640 10 osd.0 pg_epoch: 115 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 crt=0'0 unknown mbc={}] state: Started advmap 2026-02-11T23:31:27.475+0000 7f414b98b640 10 osd.0 pg_epoch: 115 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 crt=0'0 unknown mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:31:27.475+0000 7f414b98b640 20 osd.0 115 advance_pg new pool opts old pool opts 2026-02-11T23:31:27.475+0000 7f414b98b640 10 osd.0 pg_epoch: 115 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 crt=0'0 unknown mbc={}] handle_activate_map: epoch range: 114..115 2026-02-11T23:31:27.475+0000 7f414b98b640 10 osd.0 pg_epoch: 115 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 crt=0'0 unknown mbc={}] PeeringState::activate_map 2026-02-11T23:31:27.475+0000 7f414b98b640 20 osd.0 pg_epoch: 115 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 crt=0'0 unknown mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 113 while current is 115 2026-02-11T23:31:27.475+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.475+0000 7f414b98b640 10 osd.0 pg_epoch: 115 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 crt=0'0 unknown mbc={}] do_peering_event: epoch_sent: 114 epoch_requested: 113 MLogRec from 1 log log((111'590,111'600], crt=111'600) pi ([107,112] all_participants={0, 1} intervals=[([107,111] acting={1})]) pg_lease(ru 83.945602417s ub 83.945602417s int 16.000000000s) +create_info 2026-02-11T23:31:27.475+0000 7f414b98b640 10 osd.0 pg_epoch: 115 pg[1.0( empty local-lis/les=0/0 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 crt=0'0 unknown mbc={}] state: got info+log from osd.1 1.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=113/114 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) log((111'590,111'600], crt=111'600) 2026-02-11T23:31:27.476+0000 7f414b98b640 20 osd.0 pg_epoch: 115 pg[1.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=113/114 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 crt=111'600 lcod 0'0 unknown mbc={}] PeeringState::proc_lease no-op, !nonprimary 2026-02-11T23:31:27.476+0000 7f414b98b640 5 osd.0 pg_epoch: 115 pg[1.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=113/114 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 crt=111'600 lcod 0'0 unknown mbc={}] exit Started/Stray 0.000310 4 0.000031 2026-02-11T23:31:27.476+0000 7f414b98b640 5 osd.0 pg_epoch: 115 pg[1.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=113/114 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 crt=111'600 lcod 0'0 unknown mbc={}] enter Started/ReplicaActive 2026-02-11T23:31:27.476+0000 7f414b98b640 5 osd.0 pg_epoch: 115 pg[1.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=113/114 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 crt=111'600 lcod 0'0 unknown mbc={}] enter Started/ReplicaActive/RepNotRecovering 2026-02-11T23:31:27.476+0000 7f414b98b640 10 osd.0 pg_epoch: 115 pg[1.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=113/114 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 crt=111'600 lcod 0'0 unknown mbc={}] state: In ReplicaActive, about to call activate 2026-02-11T23:31:27.476+0000 7f414b98b640 10 osd.0 pg_epoch: 115 pg[1.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=113/114 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 crt=111'600 lcod 0'0 unknown mbc={}] PeeringState::activate activate - no missing, moving last_complete 111'600 -> 111'600 2026-02-11T23:31:27.476+0000 7f414b98b640 10 osd.0 pg_epoch: 115 pg[1.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=113/114 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 crt=111'600 lcod 0'0 unknown mbc={}] scrubber: on_replica_activate 2026-02-11T23:31:27.476+0000 7f414b98b640 20 scrubberFSM event: --vvvv---- ReplicaActivate 2026-02-11T23:31:27.476+0000 7f414b98b640 10 osd.0 pg_epoch: 115 pg[1.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=113/114 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 crt=111'600 lcod 0'0 unknown mbc={}] scrubber: FSM: -- state -->> ReplicaActive 2026-02-11T23:31:27.476+0000 7f414b98b640 10 osd.0 pg_epoch: 115 pg[1.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=113/114 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 crt=111'600 lcod 0'0 unknown mbc={}] scrubber: FSM: -- state -->> ReplicaActive/ReplicaIdle 2026-02-11T23:31:27.476+0000 7f414b98b640 20 scrubberFSM event: --^^^^---- ReplicaActivate 2026-02-11T23:31:27.476+0000 7f414b98b640 10 osd.0 pg_epoch: 115 pg[1.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=113/114 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 crt=111'600 lcod 0'0 unknown mbc={}] state: Activate Finished 2026-02-11T23:31:27.476+0000 7f414b98b640 6 write_log_and_missing with: dirty_to: 4294967295'18446744073709551615, dirty_from: 4294967295'18446744073709551615, writeout_from: 4294967295'18446744073709551615, trimmed: , trimmed_dups: , clear_divergent_priors: 0 2026-02-11T23:31:27.476+0000 7f414b98b640 10 _write_log_and_missing clearing up to 4294967295'18446744073709551615 dirty_to_dups=4294967295'18446744073709551615 dirty_from_dups=4294967295'18446744073709551615 write_from_dups=4294967295'18446744073709551615 trimmed_dups.size()=0 2026-02-11T23:31:27.476+0000 7f414b98b640 10 _write_log_and_missing remove dups min=dup_0000000000.00000000000000000000 to dirty_to_dup=dup_4294967295.18446744073709551615 2026-02-11T23:31:27.476+0000 7f414b98b640 10 _write_log_and_missing going to encode log.dups.size()=590 2026-02-11T23:31:27.477+0000 7f414b98b640 10 _write_log_and_missing 1st round encoded log.dups.size()=590 2026-02-11T23:31:27.477+0000 7f414b98b640 10 _write_log_and_missing 2st round encoded log.dups.size()=590 2026-02-11T23:31:27.477+0000 7f414b98b640 10 end of _write_log_and_missing 2026-02-11T23:31:27.477+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.477+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 115 epoch_requested: 115 NullEvt) class_id 2immediate prio 255 cost 10 e115) pg 0x4ef0000 2026-02-11T23:31:27.477+0000 7f414f993640 10 osd.0 pg_epoch: 115 pg[1.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=113/114 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 crt=111'600 lcod 0'0 unknown mbc={}] do_peering_event: epoch_sent: 115 epoch_requested: 115 NullEvt 2026-02-11T23:31:27.477+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.477+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.477+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.477+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.477+0000 7f414f993640 15 osd.0 115 enqueue_peering_evt 1.0 epoch_sent: 115 epoch_requested: 115 ActivateCommitted from 114 processed at 115 2026-02-11T23:31:27.477+0000 7f414f993640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 115 epoch_requested: 115 ActivateCommitted from 114 processed at 115) class_id 2 prio 255 cost 10 e115) 2026-02-11T23:31:27.477+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.477+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 115 epoch_requested: 115 ActivateCommitted from 114 processed at 115) class_id 2immediate prio 255 cost 10 e115) queued 2026-02-11T23:31:27.477+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.477+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 115 epoch_requested: 115 ActivateCommitted from 114 processed at 115) class_id 2immediate prio 255 cost 10 e115) pg 0x4ef0000 2026-02-11T23:31:27.478+0000 7f414f993640 10 osd.0 pg_epoch: 115 pg[1.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=113/114 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 crt=111'600 lcod 0'0 unknown mbc={}] do_peering_event: epoch_sent: 115 epoch_requested: 115 ActivateCommitted from 114 processed at 115 2026-02-11T23:31:27.478+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.478+0000 7f414f993640 10 osd.0 pg_epoch: 115 pg[1.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=113/114 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 crt=111'600 lcod 0'0 unknown mbc={}] state: react 115 telling primary 2026-02-11T23:31:27.478+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.478+0000 7f414f993640 20 osd.0 115 get_con_osd_cluster to osd.1 from_epoch 115 2026-02-11T23:31:27.478+0000 7f414f993640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:27.478+0000 7f414f993640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:27.478+0000 7f414f993640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6812/3043114013 map epoch 113 -> 115 (shared) 2026-02-11T23:31:27.478+0000 7f414f993640 10 osd.0 115 send_incremental_map epoch range: (114, 115] to v2:127.0.0.1:6812/3043114013 2026-02-11T23:31:27.478+0000 7f414f993640 10 osd.0 115 add_map_inc_bl 114 252 bytes 2026-02-11T23:31:27.478+0000 7f414f993640 10 osd.0 115 add_map_inc_bl 115 574 bytes 2026-02-11T23:31:27.478+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- osd_map(114..115 src has 1..115) -- 0x4a31dc0 con 0x4acb000 2026-02-11T23:31:27.478+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_info2(1.0 1.0 1.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=113/114 n=0 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) e115/115) -- 0x5b22700 con 0x4acb000 2026-02-11T23:31:27.478+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.478+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 31 ==== pg_info2(1.0 1.0 1.0( v 111'600 (111'500,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) e115/115) ==== 1111+0+0 (unknown 0 0 0) 0x5b22700 con 0x4acb000 2026-02-11T23:31:27.478+0000 7f416abfa640 15 osd.0 115 enqueue_peering_evt 1.0 epoch_sent: 115 epoch_requested: 115 MInfoRec from 1 info: 1.0( v 111'600 (111'500,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) pg_lease(ru 84.143806458s ub 84.143806458s int 16.000000000s) 2026-02-11T23:31:27.478+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 115 epoch_requested: 115 MInfoRec from 1 info: 1.0( v 111'600 (111'500,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) pg_lease(ru 84.143806458s ub 84.143806458s int 16.000000000s)) class_id 2 prio 255 cost 10 e115) 2026-02-11T23:31:27.478+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.478+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 115 epoch_requested: 115 MInfoRec from 1 info: 1.0( v 111'600 (111'500,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) pg_lease(ru 84.143806458s ub 84.143806458s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e115) queued 2026-02-11T23:31:27.478+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.478+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 115 epoch_requested: 115 MInfoRec from 1 info: 1.0( v 111'600 (111'500,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) pg_lease(ru 84.143806458s ub 84.143806458s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e115) pg 0x4ef0000 2026-02-11T23:31:27.478+0000 7f414b98b640 10 osd.0 pg_epoch: 115 pg[1.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=113/114 n=0 ec=27/27 lis/c=107/107 les/c/f=109/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 lcod 0'0 active mbc={}] do_peering_event: epoch_sent: 115 epoch_requested: 115 MInfoRec from 1 info: 1.0( v 111'600 (111'500,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) pg_lease(ru 84.143806458s ub 84.143806458s int 16.000000000s) 2026-02-11T23:31:27.478+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.478+0000 7f414b98b640 20 osd.0 pg_epoch: 115 pg[1.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=113/114 n=0 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 lcod 0'0 active mbc={}] PeeringState::update_history advanced history from ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113 2026-02-11T23:31:27.478+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.478+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.478+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.478+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.479+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 32 ==== MBackfillReserve(1.0 REQUEST prio: 151 e115/115) ==== 46+0+0 (unknown 0 0 0) 0x4ecc900 con 0x4acb000 2026-02-11T23:31:27.479+0000 7f416abfa640 15 osd.0 115 enqueue_peering_evt 1.0 epoch_sent: 115 epoch_requested: 115 RequestBackfillPrio: priority 151 primary bytes 2457600 local bytes 0 2026-02-11T23:31:27.479+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 115 epoch_requested: 115 RequestBackfillPrio: priority 151 primary bytes 2457600 local bytes 0) class_id 2 prio 255 cost 10 e115) 2026-02-11T23:31:27.479+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.479+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 115 epoch_requested: 115 RequestBackfillPrio: priority 151 primary bytes 2457600 local bytes 0) class_id 2immediate prio 255 cost 10 e115) queued 2026-02-11T23:31:27.479+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.479+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 115 epoch_requested: 115 RequestBackfillPrio: priority 151 primary bytes 2457600 local bytes 0) class_id 2immediate prio 255 cost 10 e115) pg 0x4ef0000 2026-02-11T23:31:27.479+0000 7f414b98b640 10 osd.0 pg_epoch: 115 pg[1.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=113/114 n=0 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 lcod 0'0 active mbc={}] do_peering_event: epoch_sent: 115 epoch_requested: 115 RequestBackfillPrio: priority 151 primary bytes 2457600 local bytes 0 2026-02-11T23:31:27.479+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.479+0000 7f414b98b640 10 osd.0 pg_epoch: 115 pg[1.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=113/114 n=0 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 lcod 0'0 active mbc={}] try_reserve_recovery_space primary_bytes 2400KiB local 0KiB pending_adjustments 2400KiB 2026-02-11T23:31:27.479+0000 7f414b98b640 20 osd.0 pg_epoch: 115 pg[1.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=113/114 n=0 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 lcod 0'0 active mbc={}] _tentative_full type backfillfull adjust_used 2400KiB 2026-02-11T23:31:27.479+0000 7f414b98b640 20 osd.0 115 compute_adjusted_ratio Before kb_used() 0 2026-02-11T23:31:27.479+0000 7f414b98b640 20 osd.0 115 compute_adjusted_ratio After kb_used() 2400 2026-02-11T23:31:27.479+0000 7f414b98b640 5 osd.0 pg_epoch: 115 pg[1.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=113/114 n=0 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 lcod 0'0 active mbc={}] exit Started/ReplicaActive/RepNotRecovering 0.002854 4 0.000247 2026-02-11T23:31:27.479+0000 7f414b98b640 5 osd.0 pg_epoch: 115 pg[1.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=113/114 n=0 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 lcod 0'0 active mbc={}] enter Started/ReplicaActive/RepWaitBackfillReserved 2026-02-11T23:31:27.479+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.479+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.479+0000 7f4152999640 15 osd.0 115 enqueue_peering_evt 1.0 epoch_sent: 115 epoch_requested: 115 RemoteBackfillReserved 2026-02-11T23:31:27.479+0000 7f4152999640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 115 epoch_requested: 115 RemoteBackfillReserved) class_id 2 prio 255 cost 10 e115) 2026-02-11T23:31:27.479+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.479+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 115 epoch_requested: 115 RemoteBackfillReserved) class_id 2immediate prio 255 cost 10 e115) queued 2026-02-11T23:31:27.479+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.479+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 115 epoch_requested: 115 RemoteBackfillReserved) class_id 2immediate prio 255 cost 10 e115) pg 0x4ef0000 2026-02-11T23:31:27.479+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.479+0000 7f414b98b640 10 osd.0 pg_epoch: 115 pg[1.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=113/114 n=0 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 lcod 0'0 active mbc={}] do_peering_event: epoch_sent: 115 epoch_requested: 115 RemoteBackfillReserved 2026-02-11T23:31:27.479+0000 7f414b98b640 20 osd.0 115 get_con_osd_cluster to osd.1 from_epoch 115 2026-02-11T23:31:27.479+0000 7f414b98b640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:27.479+0000 7f414b98b640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:27.479+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MBackfillReserve(1.0 GRANT e115/115) -- 0x4ec2780 con 0x4acb000 2026-02-11T23:31:27.479+0000 7f414b98b640 5 osd.0 pg_epoch: 115 pg[1.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=113/114 n=0 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 lcod 0'0 active mbc={}] exit Started/ReplicaActive/RepWaitBackfillReserved 0.000026 1 0.000022 2026-02-11T23:31:27.479+0000 7f414b98b640 5 osd.0 pg_epoch: 115 pg[1.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=113/114 n=0 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 lcod 0'0 active mbc={}] enter Started/ReplicaActive/RepRecovering 2026-02-11T23:31:27.479+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.479+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.481+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 34 ==== pg_scan(get_digest 1.0 MIN-MIN e 115/113) ==== 119+0+0 (unknown 0 0 0) 0x4a7ac80 con 0x4acb000 2026-02-11T23:31:27.481+0000 7f415aefe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 33 ==== osd_map(114..115 src has 1..115) ==== 878+0+0 (unknown 0 0 0) 0x4a31dc0 con 0x4acb000 2026-02-11T23:31:27.481+0000 7f415aefe640 20 osd.0 115 OSD::ms_dispatch: osd_map(114..115 src has 1..115) 2026-02-11T23:31:27.481+0000 7f415aefe640 20 osd.0 115 _dispatch 0x4a31dc0 osd_map(114..115 src has 1..115) 2026-02-11T23:31:27.481+0000 7f416abfa640 15 osd.0 115 enqueue_op pg_scan(get_digest 1.0 MIN-MIN e 115/113) v2 prio 15 type 94 cost 0 latency 0.000010 epoch 115 2026-02-11T23:31:27.481+0000 7f415aefe640 3 osd.0 115 handle_osd_map epochs [114,115], i have 115, src has [1,115] 2026-02-11T23:31:27.481+0000 7f415aefe640 10 osd.0 115 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:31:27.481+0000 7f415aefe640 10 osd.0 115 no new maps here, dropping 2026-02-11T23:31:27.481+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_scan(get_digest 1.0 MIN-MIN e 115/113) v2) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:27.481+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.481+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_scan(get_digest 1.0 MIN-MIN e 115/113) v2) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e115) queued 2026-02-11T23:31:27.481+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.481+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_scan(get_digest 1.0 MIN-MIN e 115/113) v2) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e115) pg 0x4ef0000 2026-02-11T23:31:27.481+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.481+0000 7f414f993640 10 osd.0 115 dequeue_op pg_scan(get_digest 1.0 MIN-MIN e 115/113) v2 prio 15 cost 0 latency 0.000037 pg_scan(get_digest 1.0 MIN-MIN e 115/113) v2 pg pg[1.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=113/114 n=0 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 lcod 0'0 active mbc={}] 2026-02-11T23:31:27.481+0000 7f414f993640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:27.481+0000 7f414f993640 10 osd.0 pg_epoch: 115 pg[1.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=113/114 n=0 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 lcod 0'0 active mbc={}] _handle_message: pg_scan(get_digest 1.0 MIN-MIN e 115/113) v2 2026-02-11T23:31:27.481+0000 7f414f993640 10 osd.0 pg_epoch: 115 pg[1.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=113/114 n=0 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 lcod 0'0 active mbc={}] do_scan pg_scan(get_digest 1.0 MIN-MIN e 115/113) 2026-02-11T23:31:27.481+0000 7f414f993640 10 osd.0 pg_epoch: 115 pg[1.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=113/114 n=0 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 lcod 0'0 active mbc={}] scan_range_replica from MIN 2026-02-11T23:31:27.481+0000 7f414f993640 10 osd.0 pg_epoch: 115 pg[1.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=113/114 n=0 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 lcod 0'0 active mbc={}] got 0 items, next MAX 2026-02-11T23:31:27.481+0000 7f414f993640 20 osd.0 pg_epoch: 115 pg[1.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=113/114 n=0 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 lcod 0'0 active mbc={}] [] 2026-02-11T23:31:27.481+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_scan(digest 1.0 MIN-MAX e 115/113) -- 0x4a7af00 con 0x4acb000 2026-02-11T23:31:27.481+0000 7f414f993640 10 osd.0 115 dequeue_op pg_scan(get_digest 1.0 MIN-MIN e 115/113) v2 finish latency 0.000050 2026-02-11T23:31:27.481+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.482+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 35 ==== MOSDPGPush(1.0 115/113 [PushOp(1:006bea6a:::obj172:head, version: 111'172, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:006bea6a:::obj172:head@111'172, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3814340 con 0x4acb000 2026-02-11T23:31:27.482+0000 7f416abfa640 15 osd.0 115 enqueue_op MOSDPGPush(1.0 115/113 [PushOp(1:006bea6a:::obj172:head, version: 111'172, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:006bea6a:::obj172:head@111'172, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000015 epoch 115 2026-02-11T23:31:27.482+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 115/113 [PushOp(1:006bea6a:::obj172:head, version: 111'172, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:006bea6a:::obj172:head@111'172, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:27.482+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.482+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 115/113 [PushOp(1:006bea6a:::obj172:head, version: 111'172, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:006bea6a:::obj172:head@111'172, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e115) queued 2026-02-11T23:31:27.482+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.482+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 115/113 [PushOp(1:006bea6a:::obj172:head, version: 111'172, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:006bea6a:::obj172:head@111'172, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e115) pg 0x4ef0000 2026-02-11T23:31:27.482+0000 7f414f993640 10 osd.0 115 dequeue_op MOSDPGPush(1.0 115/113 [PushOp(1:006bea6a:::obj172:head, version: 111'172, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:006bea6a:::obj172:head@111'172, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000047 MOSDPGPush(1.0 115/113 [PushOp(1:006bea6a:::obj172:head, version: 111'172, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:006bea6a:::obj172:head@111'172, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=113/114 n=0 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 lcod 0'0 active mbc={}] 2026-02-11T23:31:27.482+0000 7f414f993640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:27.482+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.482+0000 7f414f993640 10 osd.0 pg_epoch: 115 pg[1.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=113/114 n=0 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 lcod 0'0 active mbc={}] _handle_message: MOSDPGPush(1.0 115/113 [PushOp(1:006bea6a:::obj172:head, version: 111'172, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:006bea6a:::obj172:head@111'172, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.482+0000 7f414f993640 10 osd.0 pg_epoch: 115 pg[1.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=113/114 n=0 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 lcod 0'0 active mbc={}] handle_push ObjectRecoveryInfo(1:006bea6a:::obj172:head@111'172, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.482+0000 7f414f993640 10 osd.0 pg_epoch: 115 pg[1.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=113/114 n=0 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 lcod 0'0 active mbc={}] submit_push_data 1:006bea6a:::obj172:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.482+0000 7f414f993640 20 osd.0 pg_epoch: 115 pg[1.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=113/114 n=0 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 lcod 0'0 active mbc={}] submit_push_data recovering object 1:006bea6a:::obj172:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.482+0000 7f414f993640 10 osd.0 pg_epoch: 115 pg[1.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=113/114 n=0 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 lcod 0'0 active mbc={}] on_local_recover: 1:006bea6a:::obj172:head 2026-02-11T23:31:27.482+0000 7f414f993640 10 osd.0 pg_epoch: 115 pg[1.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=113/114 n=0 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 lcod 0'0 active mbc={}] PeeringState::recover_got got missing 1:006bea6a:::obj172:head v 111'172 2026-02-11T23:31:27.482+0000 7f414f993640 10 osd.0 pg_epoch: 115 pg[1.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=113/114 n=0 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 lcod 0'0 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.482+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.482+0000 7f414f993640 10 osd.0 115 dequeue_op MOSDPGPush(1.0 115/113 [PushOp(1:006bea6a:::obj172:head, version: 111'172, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:006bea6a:::obj172:head@111'172, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000083 2026-02-11T23:31:27.482+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.482+0000 7f414f993640 20 osd.0 pg_epoch: 115 pg[1.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=113/114 n=0 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 lcod 0'0 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.482+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.482+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.482+0000 7f414f993640 10 osd.0 pg_epoch: 115 pg[1.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=113/114 n=0 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 lcod 0'0 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.482+0000 7f414f993640 20 osd.0 115 get_con_osd_cluster to osd.1 from_epoch 115 2026-02-11T23:31:27.482+0000 7f414f993640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:27.482+0000 7f414f993640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:27.482+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e115/115) -- 0x4ec2f00 con 0x4acb000 2026-02-11T23:31:27.482+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 115/113 [PushReplyOp(1:006bea6a:::obj172:head)]) -- 0x4ec2480 con 0x4acb000 2026-02-11T23:31:27.482+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.482+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 36 ==== MOSDPGPush(1.0 115/113 [PushOp(1:00833431:::obj76:head, version: 111'76, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:00833431:::obj76:head@111'76, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x3814d00 con 0x4acb000 2026-02-11T23:31:27.482+0000 7f416abfa640 15 osd.0 115 enqueue_op MOSDPGPush(1.0 115/113 [PushOp(1:00833431:::obj76:head, version: 111'76, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:00833431:::obj76:head@111'76, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000018 epoch 115 2026-02-11T23:31:27.482+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 115/113 [PushOp(1:00833431:::obj76:head, version: 111'76, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:00833431:::obj76:head@111'76, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:27.482+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.482+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 37 ==== pg_backfill(progress 1.0 e 115/115 lb 1:006bea6a:::obj172:head) ==== 1062+0+0 (unknown 0 0 0) 0x4a2cb00 con 0x4acb000 2026-02-11T23:31:27.482+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 115/113 [PushOp(1:00833431:::obj76:head, version: 111'76, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:00833431:::obj76:head@111'76, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e115) queued 2026-02-11T23:31:27.482+0000 7f416abfa640 15 osd.0 115 enqueue_op pg_backfill(progress 1.0 e 115/115 lb 1:006bea6a:::obj172:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 115 2026-02-11T23:31:27.482+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.482+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 115/113 [PushOp(1:00833431:::obj76:head, version: 111'76, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:00833431:::obj76:head@111'76, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e115) pg 0x4ef0000 2026-02-11T23:31:27.482+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 115/115 lb 1:006bea6a:::obj172:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:27.482+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.482+0000 7f414b98b640 10 osd.0 115 dequeue_op MOSDPGPush(1.0 115/113 [PushOp(1:00833431:::obj76:head, version: 111'76, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:00833431:::obj76:head@111'76, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000046 MOSDPGPush(1.0 115/113 [PushOp(1:00833431:::obj76:head, version: 111'76, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:00833431:::obj76:head@111'76, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=113/114 n=0 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.482+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 115/115 lb 1:006bea6a:::obj172:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e115) queued 2026-02-11T23:31:27.482+0000 7f414b98b640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:27.482+0000 7f414b98b640 10 osd.0 pg_epoch: 115 pg[1.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=113/114 n=0 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 115/113 [PushOp(1:00833431:::obj76:head, version: 111'76, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:00833431:::obj76:head@111'76, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.482+0000 7f414b98b640 10 osd.0 pg_epoch: 115 pg[1.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=113/114 n=0 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:00833431:::obj76:head@111'76, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.482+0000 7f414b98b640 10 osd.0 pg_epoch: 115 pg[1.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=113/114 n=0 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:00833431:::obj76:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.482+0000 7f414b98b640 20 osd.0 pg_epoch: 115 pg[1.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=113/114 n=0 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:00833431:::obj76:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.482+0000 7f414b98b640 10 osd.0 pg_epoch: 115 pg[1.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=113/114 n=0 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:00833431:::obj76:head 2026-02-11T23:31:27.482+0000 7f414b98b640 10 osd.0 pg_epoch: 115 pg[1.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=113/114 n=0 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:00833431:::obj76:head v 111'76 2026-02-11T23:31:27.482+0000 7f414b98b640 10 osd.0 pg_epoch: 115 pg[1.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=113/114 n=0 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.482+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.482+0000 7f414b98b640 10 osd.0 115 dequeue_op MOSDPGPush(1.0 115/113 [PushOp(1:00833431:::obj76:head, version: 111'76, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:00833431:::obj76:head@111'76, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000068 2026-02-11T23:31:27.483+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.483+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.483+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 115/115 lb 1:006bea6a:::obj172:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e115) pg 0x4ef0000 2026-02-11T23:31:27.483+0000 7f414f993640 10 osd.0 115 dequeue_op pg_backfill(progress 1.0 e 115/115 lb 1:006bea6a:::obj172:head) v3 prio 15 cost 0 latency 0.000089 pg_backfill(progress 1.0 e 115/115 lb 1:006bea6a:::obj172:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=113/114 n=0 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.483+0000 7f414f993640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:27.483+0000 7f414f993640 10 osd.0 pg_epoch: 115 pg[1.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=113/114 n=0 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 115/115 lb 1:006bea6a:::obj172:head) v3 2026-02-11T23:31:27.483+0000 7f414f993640 10 osd.0 pg_epoch: 115 pg[1.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=113/114 n=0 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 115/115 lb 1:006bea6a:::obj172:head) 2026-02-11T23:31:27.483+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.483+0000 7f414f993640 10 osd.0 115 dequeue_op pg_backfill(progress 1.0 e 115/115 lb 1:006bea6a:::obj172:head) v3 finish latency 0.000015 2026-02-11T23:31:27.483+0000 7f414f993640 20 osd.0 pg_epoch: 115 pg[1.0( v 111'600 (111'590,111'600] lb 1:006bea6a:::obj172:head local-lis/les=113/114 n=1 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.483+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.483+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.483+0000 7f414f993640 10 osd.0 pg_epoch: 115 pg[1.0( v 111'600 (111'590,111'600] lb 1:006bea6a:::obj172:head local-lis/les=113/114 n=1 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.483+0000 7f414f993640 20 osd.0 115 get_con_osd_cluster to osd.1 from_epoch 115 2026-02-11T23:31:27.483+0000 7f414f993640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:27.483+0000 7f414f993640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:27.483+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e115/115) -- 0x4ec3680 con 0x4acb000 2026-02-11T23:31:27.483+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 115/113 [PushReplyOp(1:00833431:::obj76:head)]) -- 0x4ec3080 con 0x4acb000 2026-02-11T23:31:27.483+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.483+0000 7f415aefe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 38 ==== osd_map(116..116 src has 1..116) ==== 618+0+0 (unknown 0 0 0) 0x4a31a40 con 0x4acb000 2026-02-11T23:31:27.483+0000 7f415aefe640 20 osd.0 115 OSD::ms_dispatch: osd_map(116..116 src has 1..116) 2026-02-11T23:31:27.483+0000 7f415aefe640 20 osd.0 115 _dispatch 0x4a31a40 osd_map(116..116 src has 1..116) 2026-02-11T23:31:27.483+0000 7f415aefe640 3 osd.0 115 handle_osd_map epochs [116,116], i have 115, src has [1,116] 2026-02-11T23:31:27.483+0000 7f415aefe640 10 osd.0 115 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:31:27.483+0000 7f415aefe640 10 osd.0 115 handle_osd_map got inc map for epoch 116 2026-02-11T23:31:27.483+0000 7f415aefe640 20 osd.0 115 got_full_map 116, nothing requested 2026-02-11T23:31:27.483+0000 7f415aefe640 20 osd.0 115 trim_maps: min=1 oldest_map=1 2026-02-11T23:31:27.483+0000 7f415aefe640 20 osd.0 115 handle_osd_map pg_num_history pg_num_history(e116 pg_nums {1={27=1},2={34=1},3={42=1},4={49=1},5={56=1},6={64=1},7={70=1},8={77=1},9={85=1},10={92=1}} deleted_pools 99,2,100,3,101,5,102,6,103,7,104,8,105,9,106,10) 2026-02-11T23:31:27.483+0000 7f415aefe640 10 snap_mapper record_purged_snaps purged_snaps {116={}} 2026-02-11T23:31:27.483+0000 7f415aefe640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:31:27.483+0000 7f415aefe640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e116 maps [1~116] lci=[5,116] tlb=1) 2026-02-11T23:31:27.483+0000 7f41591a6640 10 osd.0 115 _committed_osd_maps 116..116 2026-02-11T23:31:27.483+0000 7f41591a6640 10 osd.0 115 advance to epoch 116 (<= last 116 <= newest_map 116) 2026-02-11T23:31:27.483+0000 7f41591a6640 20 osd.0 116 consume_map version 116 2026-02-11T23:31:27.483+0000 7f41591a6640 20 osd.0 115 await_reserved_maps epoch:116 2026-02-11T23:31:27.483+0000 7f41591a6640 20 osd.0 115 await_reserved_maps done 0 2026-02-11T23:31:27.483+0000 7f41591a6640 20 osd.0 116 consume_map 116 -- publish done 2026-02-11T23:31:27.483+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 1 slots 2026-02-11T23:31:27.483+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3701340 2026-02-11T23:31:27.483+0000 7f41591a6640 20 osd.0 116 identify_splits_and_merges 1.0 e115 to e116 2026-02-11T23:31:27.483+0000 7f41591a6640 20 osd.0 116 identify_splits_and_merges 1.0 e115 to e116 pg_nums {27=1} 2026-02-11T23:31:27.483+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:27.483+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:27.483+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:27.483+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:27.483+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:27.483+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:27.483+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:27.483+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:27.483+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:27.483+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:27.483+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:27.483+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:27.483+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:27.483+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:27.483+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:27.483+0000 7f41591a6640 20 osd.0 116 prune_pg_created 2026-02-11T23:31:27.483+0000 7f41591a6640 10 osd.0:0.consume_map 116 (was 115) 2026-02-11T23:31:27.483+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:31:27.483+0000 7f41591a6640 10 osd.0:1.consume_map 116 (was 115) 2026-02-11T23:31:27.483+0000 7f41591a6640 10 osd.0:2.consume_map 116 (was 115) 2026-02-11T23:31:27.483+0000 7f41591a6640 10 osd.0:3.consume_map 116 (was 115) 2026-02-11T23:31:27.483+0000 7f41591a6640 10 osd.0:4.consume_map 116 (was 115) 2026-02-11T23:31:27.483+0000 7f41591a6640 10 osd.0:5.consume_map 116 (was 115) 2026-02-11T23:31:27.483+0000 7f41591a6640 10 osd.0:6.consume_map 116 (was 115) 2026-02-11T23:31:27.483+0000 7f41591a6640 10 osd.0:7.consume_map 116 (was 115) 2026-02-11T23:31:27.483+0000 7f41591a6640 10 osd.0 116 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:31:27.483+0000 7f41591a6640 15 osd.0 116 enqueue_peering_evt 1.0 epoch_sent: 116 epoch_requested: 116 NullEvt 2026-02-11T23:31:27.483+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 116 epoch_requested: 116 NullEvt) class_id 2 prio 255 cost 10 e116) 2026-02-11T23:31:27.483+0000 7f41591a6640 7 osd.0 116 activate_map version 116 2026-02-11T23:31:27.483+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.483+0000 7f4150194640 20 osd.0 116 agent_entry empty queue 2026-02-11T23:31:27.483+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 116 epoch_requested: 116 NullEvt) class_id 2immediate prio 255 cost 10 e116) queued 2026-02-11T23:31:27.483+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.483+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 116 epoch_requested: 116 NullEvt) class_id 2immediate prio 255 cost 10 e116) pg 0x4ef0000 2026-02-11T23:31:27.483+0000 7f414b98b640 10 osd.0 pg_epoch: 115 pg[1.0( v 111'600 (111'590,111'600] lb 1:006bea6a:::obj172:head local-lis/les=113/114 n=1 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_advance_map: 116 2026-02-11T23:31:27.483+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.483+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 1.0 115 -> 116 2026-02-11T23:31:27.483+0000 7f414b98b640 10 osd.0 pg_epoch: 115 pg[1.0( v 111'600 (111'590,111'600] lb 1:006bea6a:::obj172:head local-lis/les=113/114 n=1 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1] -- 1/1 2026-02-11T23:31:27.483+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:006bea6a:::obj172:head local-lis/les=113/114 n=1 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] state: Started advmap 2026-02-11T23:31:27.483+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:006bea6a:::obj172:head local-lis/les=113/114 n=1 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:31:27.483+0000 7f414b98b640 20 osd.0 116 advance_pg new pool opts old pool opts 2026-02-11T23:31:27.483+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:006bea6a:::obj172:head local-lis/les=113/114 n=1 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_activate_map: epoch range: 116..116 2026-02-11T23:31:27.483+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:006bea6a:::obj172:head local-lis/les=113/114 n=1 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::activate_map 2026-02-11T23:31:27.483+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:006bea6a:::obj172:head local-lis/les=113/114 n=1 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 115 while current is 116 2026-02-11T23:31:27.483+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.483+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:006bea6a:::obj172:head local-lis/les=113/114 n=1 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_peering_event: epoch_sent: 116 epoch_requested: 116 NullEvt 2026-02-11T23:31:27.483+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.483+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.483+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 39 ==== MOSDPGPush(1.0 116/113 [PushOp(1:00b31b6c:::obj19:head, version: 111'19, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:00b31b6c:::obj19:head@111'19, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x3815380 con 0x4acb000 2026-02-11T23:31:27.483+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:00b31b6c:::obj19:head, version: 111'19, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:00b31b6c:::obj19:head@111'19, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 116 2026-02-11T23:31:27.483+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:00b31b6c:::obj19:head, version: 111'19, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:00b31b6c:::obj19:head@111'19, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.483+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.483+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:00b31b6c:::obj19:head, version: 111'19, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:00b31b6c:::obj19:head@111'19, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.483+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 40 ==== pg_backfill(progress 1.0 e 116/116 lb 1:00833431:::obj76:head) ==== 1061+0+0 (unknown 0 0 0) 0x4a2c000 con 0x4acb000 2026-02-11T23:31:27.483+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:00833431:::obj76:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.483+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.483+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:00833431:::obj76:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.483+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:00b31b6c:::obj19:head, version: 111'19, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:00b31b6c:::obj19:head@111'19, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.483+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.483+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:00b31b6c:::obj19:head, version: 111'19, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:00b31b6c:::obj19:head@111'19, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000041 MOSDPGPush(1.0 116/113 [PushOp(1:00b31b6c:::obj19:head, version: 111'19, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:00b31b6c:::obj19:head@111'19, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:006bea6a:::obj172:head local-lis/les=113/114 n=1 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.483+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 updating session's projected_epoch from 115 to ping map epoch of 116 2026-02-11T23:31:27.483+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.483+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:006bea6a:::obj172:head local-lis/les=113/114 n=1 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:00b31b6c:::obj19:head, version: 111'19, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:00b31b6c:::obj19:head@111'19, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.483+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.483+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:006bea6a:::obj172:head local-lis/les=113/114 n=1 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:00b31b6c:::obj19:head@111'19, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.483+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:00833431:::obj76:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.483+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:006bea6a:::obj172:head local-lis/les=113/114 n=1 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:00b31b6c:::obj19:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.483+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:006bea6a:::obj172:head local-lis/les=113/114 n=1 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:00b31b6c:::obj19:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.483+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:006bea6a:::obj172:head local-lis/les=113/114 n=1 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:00b31b6c:::obj19:head 2026-02-11T23:31:27.483+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:006bea6a:::obj172:head local-lis/les=113/114 n=1 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:00b31b6c:::obj19:head v 111'19 2026-02-11T23:31:27.483+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:006bea6a:::obj172:head local-lis/les=113/114 n=1 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.483+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.483+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:00b31b6c:::obj19:head, version: 111'19, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:00b31b6c:::obj19:head@111'19, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000074 2026-02-11T23:31:27.483+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.483+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.483+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:00833431:::obj76:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.483+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:00833431:::obj76:head) v3 prio 15 cost 0 latency 0.000093 pg_backfill(progress 1.0 e 116/116 lb 1:00833431:::obj76:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:006bea6a:::obj172:head local-lis/les=113/114 n=1 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.483+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.483+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:006bea6a:::obj172:head local-lis/les=113/114 n=1 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:00833431:::obj76:head) v3 2026-02-11T23:31:27.483+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:006bea6a:::obj172:head local-lis/les=113/114 n=1 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:00833431:::obj76:head) 2026-02-11T23:31:27.483+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.484+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:00833431:::obj76:head) v3 finish latency 0.000021 2026-02-11T23:31:27.484+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:00833431:::obj76:head local-lis/les=113/114 n=2 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.484+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.484+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.484+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:00833431:::obj76:head local-lis/les=113/114 n=2 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.484+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.484+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.484+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.484+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x4ec2d80 con 0x4acb000 2026-02-11T23:31:27.484+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:00b31b6c:::obj19:head)]) -- 0x4ec3380 con 0x4acb000 2026-02-11T23:31:27.484+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.484+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 41 ==== MOSDPGPush(1.0 116/113 [PushOp(1:010e1e72:::obj351:head, version: 111'351, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:010e1e72:::obj351:head@111'351, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3815860 con 0x4acb000 2026-02-11T23:31:27.484+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:010e1e72:::obj351:head, version: 111'351, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:010e1e72:::obj351:head@111'351, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 116 2026-02-11T23:31:27.484+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:010e1e72:::obj351:head, version: 111'351, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:010e1e72:::obj351:head@111'351, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.484+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.484+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 42 ==== pg_backfill(progress 1.0 e 116/116 lb 1:00b31b6c:::obj19:head) ==== 1061+0+0 (unknown 0 0 0) 0x5985700 con 0x4acb000 2026-02-11T23:31:27.484+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:010e1e72:::obj351:head, version: 111'351, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:010e1e72:::obj351:head@111'351, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.484+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:00b31b6c:::obj19:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 116 2026-02-11T23:31:27.484+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.484+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:00b31b6c:::obj19:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.484+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:010e1e72:::obj351:head, version: 111'351, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:010e1e72:::obj351:head@111'351, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.484+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.484+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:010e1e72:::obj351:head, version: 111'351, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:010e1e72:::obj351:head@111'351, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000043 MOSDPGPush(1.0 116/113 [PushOp(1:010e1e72:::obj351:head, version: 111'351, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:010e1e72:::obj351:head@111'351, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:00833431:::obj76:head local-lis/les=113/114 n=2 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.484+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.484+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:00833431:::obj76:head local-lis/les=113/114 n=2 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:010e1e72:::obj351:head, version: 111'351, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:010e1e72:::obj351:head@111'351, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.484+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:00833431:::obj76:head local-lis/les=113/114 n=2 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:010e1e72:::obj351:head@111'351, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.484+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.484+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:00b31b6c:::obj19:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.484+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:00833431:::obj76:head local-lis/les=113/114 n=2 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:010e1e72:::obj351:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.484+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:00833431:::obj76:head local-lis/les=113/114 n=2 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:010e1e72:::obj351:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.484+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:00833431:::obj76:head local-lis/les=113/114 n=2 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:010e1e72:::obj351:head 2026-02-11T23:31:27.484+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:00833431:::obj76:head local-lis/les=113/114 n=2 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:010e1e72:::obj351:head v 111'351 2026-02-11T23:31:27.484+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:00833431:::obj76:head local-lis/les=113/114 n=2 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.484+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.484+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:010e1e72:::obj351:head, version: 111'351, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:010e1e72:::obj351:head@111'351, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000069 2026-02-11T23:31:27.484+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:00833431:::obj76:head local-lis/les=113/114 n=2 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.484+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.484+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.484+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:00b31b6c:::obj19:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.484+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:00b31b6c:::obj19:head) v3 prio 15 cost 0 latency 0.000097 pg_backfill(progress 1.0 e 116/116 lb 1:00b31b6c:::obj19:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:00833431:::obj76:head local-lis/les=113/114 n=2 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.484+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.484+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:00833431:::obj76:head local-lis/les=113/114 n=2 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:00b31b6c:::obj19:head) v3 2026-02-11T23:31:27.484+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:00833431:::obj76:head local-lis/les=113/114 n=2 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:00b31b6c:::obj19:head) 2026-02-11T23:31:27.484+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.484+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:00b31b6c:::obj19:head) v3 finish latency 0.000020 2026-02-11T23:31:27.484+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.484+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.484+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:00b31b6c:::obj19:head local-lis/les=113/114 n=3 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.484+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.484+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.484+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.484+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x4a89980 con 0x4acb000 2026-02-11T23:31:27.484+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:010e1e72:::obj351:head)]) -- 0x4ec2000 con 0x4acb000 2026-02-11T23:31:27.484+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.485+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 43 ==== MOSDPGPush(1.0 116/113 [PushOp(1:015e5cd2:::obj354:head, version: 111'354, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:015e5cd2:::obj354:head@111'354, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3815d40 con 0x4acb000 2026-02-11T23:31:27.485+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:015e5cd2:::obj354:head, version: 111'354, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:015e5cd2:::obj354:head@111'354, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.485+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:015e5cd2:::obj354:head, version: 111'354, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:015e5cd2:::obj354:head@111'354, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.485+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.485+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 44 ==== pg_backfill(progress 1.0 e 116/116 lb 1:010e1e72:::obj351:head) ==== 1062+0+0 (unknown 0 0 0) 0x5982b00 con 0x4acb000 2026-02-11T23:31:27.485+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:015e5cd2:::obj354:head, version: 111'354, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:015e5cd2:::obj354:head@111'354, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.485+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:010e1e72:::obj351:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.485+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.485+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:010e1e72:::obj351:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.485+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:015e5cd2:::obj354:head, version: 111'354, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:015e5cd2:::obj354:head@111'354, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.485+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.485+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:015e5cd2:::obj354:head, version: 111'354, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:015e5cd2:::obj354:head@111'354, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(1.0 116/113 [PushOp(1:015e5cd2:::obj354:head, version: 111'354, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:015e5cd2:::obj354:head@111'354, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:00b31b6c:::obj19:head local-lis/les=113/114 n=3 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.485+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.485+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:00b31b6c:::obj19:head local-lis/les=113/114 n=3 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:015e5cd2:::obj354:head, version: 111'354, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:015e5cd2:::obj354:head@111'354, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.485+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:00b31b6c:::obj19:head local-lis/les=113/114 n=3 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:015e5cd2:::obj354:head@111'354, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.485+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.485+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:010e1e72:::obj351:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.485+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:00b31b6c:::obj19:head local-lis/les=113/114 n=3 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:015e5cd2:::obj354:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.485+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:00b31b6c:::obj19:head local-lis/les=113/114 n=3 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:015e5cd2:::obj354:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.485+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:00b31b6c:::obj19:head local-lis/les=113/114 n=3 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:015e5cd2:::obj354:head 2026-02-11T23:31:27.485+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:00b31b6c:::obj19:head local-lis/les=113/114 n=3 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:015e5cd2:::obj354:head v 111'354 2026-02-11T23:31:27.485+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:00b31b6c:::obj19:head local-lis/les=113/114 n=3 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.485+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.485+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:015e5cd2:::obj354:head, version: 111'354, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:015e5cd2:::obj354:head@111'354, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000065 2026-02-11T23:31:27.485+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:00b31b6c:::obj19:head local-lis/les=113/114 n=3 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.485+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.485+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.485+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:010e1e72:::obj351:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.485+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:010e1e72:::obj351:head) v3 prio 15 cost 0 latency 0.000091 pg_backfill(progress 1.0 e 116/116 lb 1:010e1e72:::obj351:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:00b31b6c:::obj19:head local-lis/les=113/114 n=3 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.485+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.485+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:00b31b6c:::obj19:head local-lis/les=113/114 n=3 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:010e1e72:::obj351:head) v3 2026-02-11T23:31:27.485+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:00b31b6c:::obj19:head local-lis/les=113/114 n=3 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:010e1e72:::obj351:head) 2026-02-11T23:31:27.485+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.485+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:010e1e72:::obj351:head) v3 finish latency 0.000018 2026-02-11T23:31:27.485+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.485+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.485+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:010e1e72:::obj351:head local-lis/les=113/114 n=4 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.485+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.485+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.485+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.485+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x4ecc000 con 0x4acb000 2026-02-11T23:31:27.485+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:015e5cd2:::obj354:head)]) -- 0x4ec3980 con 0x4acb000 2026-02-11T23:31:27.485+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.486+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 45 ==== MOSDPGPush(1.0 116/113 [PushOp(1:022ea2cc:::obj392:head, version: 111'392, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:022ea2cc:::obj392:head@111'392, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e26340 con 0x4acb000 2026-02-11T23:31:27.486+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:022ea2cc:::obj392:head, version: 111'392, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:022ea2cc:::obj392:head@111'392, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.486+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:022ea2cc:::obj392:head, version: 111'392, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:022ea2cc:::obj392:head@111'392, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.486+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.486+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 46 ==== pg_backfill(progress 1.0 e 116/116 lb 1:015e5cd2:::obj354:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2c000 con 0x4acb000 2026-02-11T23:31:27.486+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:022ea2cc:::obj392:head, version: 111'392, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:022ea2cc:::obj392:head@111'392, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.486+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:015e5cd2:::obj354:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 116 2026-02-11T23:31:27.486+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.486+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:015e5cd2:::obj354:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.486+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:022ea2cc:::obj392:head, version: 111'392, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:022ea2cc:::obj392:head@111'392, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.486+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.486+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:022ea2cc:::obj392:head, version: 111'392, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:022ea2cc:::obj392:head@111'392, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(1.0 116/113 [PushOp(1:022ea2cc:::obj392:head, version: 111'392, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:022ea2cc:::obj392:head@111'392, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:010e1e72:::obj351:head local-lis/les=113/114 n=4 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.486+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.486+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:010e1e72:::obj351:head local-lis/les=113/114 n=4 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:022ea2cc:::obj392:head, version: 111'392, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:022ea2cc:::obj392:head@111'392, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.486+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:010e1e72:::obj351:head local-lis/les=113/114 n=4 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:022ea2cc:::obj392:head@111'392, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.486+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.486+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:015e5cd2:::obj354:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.486+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:010e1e72:::obj351:head local-lis/les=113/114 n=4 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:022ea2cc:::obj392:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.486+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:010e1e72:::obj351:head local-lis/les=113/114 n=4 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:022ea2cc:::obj392:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.486+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:010e1e72:::obj351:head local-lis/les=113/114 n=4 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:022ea2cc:::obj392:head 2026-02-11T23:31:27.486+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:010e1e72:::obj351:head local-lis/les=113/114 n=4 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:022ea2cc:::obj392:head v 111'392 2026-02-11T23:31:27.486+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:010e1e72:::obj351:head local-lis/les=113/114 n=4 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.486+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.486+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:022ea2cc:::obj392:head, version: 111'392, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:022ea2cc:::obj392:head@111'392, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:27.486+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.486+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.486+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:015e5cd2:::obj354:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.486+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:015e5cd2:::obj354:head) v3 prio 15 cost 0 latency 0.000081 pg_backfill(progress 1.0 e 116/116 lb 1:015e5cd2:::obj354:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:010e1e72:::obj351:head local-lis/les=113/114 n=4 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.486+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.486+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:010e1e72:::obj351:head local-lis/les=113/114 n=4 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:015e5cd2:::obj354:head) v3 2026-02-11T23:31:27.486+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:010e1e72:::obj351:head local-lis/les=113/114 n=4 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:015e5cd2:::obj354:head) 2026-02-11T23:31:27.486+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.486+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:015e5cd2:::obj354:head) v3 finish latency 0.000029 2026-02-11T23:31:27.486+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:015e5cd2:::obj354:head local-lis/les=113/114 n=5 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.486+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.486+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.486+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:015e5cd2:::obj354:head local-lis/les=113/114 n=5 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.486+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.486+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.486+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.486+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x4ecc480 con 0x4acb000 2026-02-11T23:31:27.486+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:022ea2cc:::obj392:head)]) -- 0x4ecde00 con 0x4acb000 2026-02-11T23:31:27.486+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.486+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.486+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.486+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 47 ==== MOSDPGPush(1.0 116/113 [PushOp(1:02455f6c:::obj462:head, version: 111'462, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:02455f6c:::obj462:head@111'462, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e26820 con 0x4acb000 2026-02-11T23:31:27.486+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:02455f6c:::obj462:head, version: 111'462, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:02455f6c:::obj462:head@111'462, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 116 2026-02-11T23:31:27.486+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:02455f6c:::obj462:head, version: 111'462, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:02455f6c:::obj462:head@111'462, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.486+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.486+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 48 ==== pg_backfill(progress 1.0 e 116/116 lb 1:022ea2cc:::obj392:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2c580 con 0x4acb000 2026-02-11T23:31:27.486+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:022ea2cc:::obj392:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.486+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:02455f6c:::obj462:head, version: 111'462, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:02455f6c:::obj462:head@111'462, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.486+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:022ea2cc:::obj392:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.486+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.486+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:02455f6c:::obj462:head, version: 111'462, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:02455f6c:::obj462:head@111'462, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.486+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:02455f6c:::obj462:head, version: 111'462, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:02455f6c:::obj462:head@111'462, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000043 MOSDPGPush(1.0 116/113 [PushOp(1:02455f6c:::obj462:head, version: 111'462, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:02455f6c:::obj462:head@111'462, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:015e5cd2:::obj354:head local-lis/les=113/114 n=5 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.486+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.486+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.486+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:022ea2cc:::obj392:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.486+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:015e5cd2:::obj354:head local-lis/les=113/114 n=5 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:02455f6c:::obj462:head, version: 111'462, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:02455f6c:::obj462:head@111'462, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.486+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:015e5cd2:::obj354:head local-lis/les=113/114 n=5 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:02455f6c:::obj462:head@111'462, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.486+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:015e5cd2:::obj354:head local-lis/les=113/114 n=5 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:02455f6c:::obj462:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.486+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:015e5cd2:::obj354:head local-lis/les=113/114 n=5 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:02455f6c:::obj462:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.486+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:015e5cd2:::obj354:head local-lis/les=113/114 n=5 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:02455f6c:::obj462:head 2026-02-11T23:31:27.486+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:015e5cd2:::obj354:head local-lis/les=113/114 n=5 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:02455f6c:::obj462:head v 111'462 2026-02-11T23:31:27.486+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:015e5cd2:::obj354:head local-lis/les=113/114 n=5 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.486+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.486+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:02455f6c:::obj462:head, version: 111'462, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:02455f6c:::obj462:head@111'462, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000067 2026-02-11T23:31:27.486+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:015e5cd2:::obj354:head local-lis/les=113/114 n=5 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.486+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.486+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.486+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:022ea2cc:::obj392:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.487+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:022ea2cc:::obj392:head) v3 prio 15 cost 0 latency 0.000093 pg_backfill(progress 1.0 e 116/116 lb 1:022ea2cc:::obj392:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:015e5cd2:::obj354:head local-lis/les=113/114 n=5 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.487+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.487+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:015e5cd2:::obj354:head local-lis/les=113/114 n=5 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:022ea2cc:::obj392:head) v3 2026-02-11T23:31:27.487+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:015e5cd2:::obj354:head local-lis/les=113/114 n=5 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:022ea2cc:::obj392:head) 2026-02-11T23:31:27.487+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.487+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:022ea2cc:::obj392:head) v3 finish latency 0.000028 2026-02-11T23:31:27.487+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.487+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.487+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:022ea2cc:::obj392:head local-lis/les=113/114 n=6 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.487+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.487+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.487+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.487+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x4c9af00 con 0x4acb000 2026-02-11T23:31:27.487+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:02455f6c:::obj462:head)]) -- 0x4ecc600 con 0x4acb000 2026-02-11T23:31:27.487+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.487+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 49 ==== MOSDPGPush(1.0 116/113 [PushOp(1:024707f1:::obj490:head, version: 111'490, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:024707f1:::obj490:head@111'490, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e26d00 con 0x4acb000 2026-02-11T23:31:27.487+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:024707f1:::obj490:head, version: 111'490, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:024707f1:::obj490:head@111'490, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.487+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:024707f1:::obj490:head, version: 111'490, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:024707f1:::obj490:head@111'490, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.487+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.487+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 50 ==== pg_backfill(progress 1.0 e 116/116 lb 1:02455f6c:::obj462:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2cb00 con 0x4acb000 2026-02-11T23:31:27.487+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:024707f1:::obj490:head, version: 111'490, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:024707f1:::obj490:head@111'490, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.487+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:02455f6c:::obj462:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 116 2026-02-11T23:31:27.487+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.487+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:02455f6c:::obj462:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.487+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:024707f1:::obj490:head, version: 111'490, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:024707f1:::obj490:head@111'490, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.487+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.487+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:024707f1:::obj490:head, version: 111'490, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:024707f1:::obj490:head@111'490, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 116/113 [PushOp(1:024707f1:::obj490:head, version: 111'490, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:024707f1:::obj490:head@111'490, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:022ea2cc:::obj392:head local-lis/les=113/114 n=6 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.487+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.487+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:022ea2cc:::obj392:head local-lis/les=113/114 n=6 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:024707f1:::obj490:head, version: 111'490, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:024707f1:::obj490:head@111'490, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.487+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:022ea2cc:::obj392:head local-lis/les=113/114 n=6 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:024707f1:::obj490:head@111'490, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.487+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.487+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:02455f6c:::obj462:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.487+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:022ea2cc:::obj392:head local-lis/les=113/114 n=6 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:024707f1:::obj490:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.487+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:022ea2cc:::obj392:head local-lis/les=113/114 n=6 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:024707f1:::obj490:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.487+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:022ea2cc:::obj392:head local-lis/les=113/114 n=6 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:024707f1:::obj490:head 2026-02-11T23:31:27.487+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:022ea2cc:::obj392:head local-lis/les=113/114 n=6 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:024707f1:::obj490:head v 111'490 2026-02-11T23:31:27.487+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:022ea2cc:::obj392:head local-lis/les=113/114 n=6 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.487+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.487+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:024707f1:::obj490:head, version: 111'490, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:024707f1:::obj490:head@111'490, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:27.487+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:022ea2cc:::obj392:head local-lis/les=113/114 n=6 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.487+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.487+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.487+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:02455f6c:::obj462:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.487+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:02455f6c:::obj462:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 1.0 e 116/116 lb 1:02455f6c:::obj462:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:022ea2cc:::obj392:head local-lis/les=113/114 n=6 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.487+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.487+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:022ea2cc:::obj392:head local-lis/les=113/114 n=6 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:02455f6c:::obj462:head) v3 2026-02-11T23:31:27.487+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:022ea2cc:::obj392:head local-lis/les=113/114 n=6 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:02455f6c:::obj462:head) 2026-02-11T23:31:27.487+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.487+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:02455f6c:::obj462:head) v3 finish latency 0.000028 2026-02-11T23:31:27.487+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.487+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.487+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:02455f6c:::obj462:head local-lis/les=113/114 n=7 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.487+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.487+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.487+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.487+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e3a900 con 0x4acb000 2026-02-11T23:31:27.487+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:024707f1:::obj490:head)]) -- 0x5e3a600 con 0x4acb000 2026-02-11T23:31:27.487+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.488+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 51 ==== MOSDPGPush(1.0 116/113 [PushOp(1:02817247:::obj101:head, version: 111'101, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:02817247:::obj101:head@111'101, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e271e0 con 0x4acb000 2026-02-11T23:31:27.488+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:02817247:::obj101:head, version: 111'101, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:02817247:::obj101:head@111'101, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 116 2026-02-11T23:31:27.488+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:02817247:::obj101:head, version: 111'101, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:02817247:::obj101:head@111'101, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.488+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.488+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 52 ==== pg_backfill(progress 1.0 e 116/116 lb 1:024707f1:::obj490:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2d080 con 0x4acb000 2026-02-11T23:31:27.488+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:024707f1:::obj490:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.488+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:02817247:::obj101:head, version: 111'101, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:02817247:::obj101:head@111'101, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.488+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:024707f1:::obj490:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.488+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.488+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:02817247:::obj101:head, version: 111'101, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:02817247:::obj101:head@111'101, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.488+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:02817247:::obj101:head, version: 111'101, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:02817247:::obj101:head@111'101, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 116/113 [PushOp(1:02817247:::obj101:head, version: 111'101, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:02817247:::obj101:head@111'101, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:02455f6c:::obj462:head local-lis/les=113/114 n=7 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.488+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.488+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:02455f6c:::obj462:head local-lis/les=113/114 n=7 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:02817247:::obj101:head, version: 111'101, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:02817247:::obj101:head@111'101, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.488+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.488+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:02455f6c:::obj462:head local-lis/les=113/114 n=7 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:02817247:::obj101:head@111'101, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.488+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:024707f1:::obj490:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.488+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:02455f6c:::obj462:head local-lis/les=113/114 n=7 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:02817247:::obj101:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.488+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:02455f6c:::obj462:head local-lis/les=113/114 n=7 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:02817247:::obj101:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.488+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:02455f6c:::obj462:head local-lis/les=113/114 n=7 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:02817247:::obj101:head 2026-02-11T23:31:27.488+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:02455f6c:::obj462:head local-lis/les=113/114 n=7 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:02817247:::obj101:head v 111'101 2026-02-11T23:31:27.488+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:02455f6c:::obj462:head local-lis/les=113/114 n=7 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.488+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.488+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:02817247:::obj101:head, version: 111'101, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:02817247:::obj101:head@111'101, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:27.488+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:02455f6c:::obj462:head local-lis/les=113/114 n=7 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.488+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.488+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.488+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:024707f1:::obj490:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.488+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:024707f1:::obj490:head) v3 prio 15 cost 0 latency 0.000082 pg_backfill(progress 1.0 e 116/116 lb 1:024707f1:::obj490:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:02455f6c:::obj462:head local-lis/les=113/114 n=7 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.488+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.488+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:02455f6c:::obj462:head local-lis/les=113/114 n=7 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:024707f1:::obj490:head) v3 2026-02-11T23:31:27.488+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:02455f6c:::obj462:head local-lis/les=113/114 n=7 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:024707f1:::obj490:head) 2026-02-11T23:31:27.488+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.488+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:024707f1:::obj490:head) v3 finish latency 0.000016 2026-02-11T23:31:27.488+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.488+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.488+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:024707f1:::obj490:head local-lis/les=113/114 n=8 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.488+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.488+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.488+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.488+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.488+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e3ad80 con 0x4acb000 2026-02-11T23:31:27.488+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:02817247:::obj101:head)]) -- 0x5e3aa80 con 0x4acb000 2026-02-11T23:31:27.488+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.489+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 53 ==== MOSDPGPush(1.0 116/113 [PushOp(1:03553543:::obj227:head, version: 111'227, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:03553543:::obj227:head@111'227, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e276c0 con 0x4acb000 2026-02-11T23:31:27.489+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:03553543:::obj227:head, version: 111'227, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:03553543:::obj227:head@111'227, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.489+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:03553543:::obj227:head, version: 111'227, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:03553543:::obj227:head@111'227, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.489+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.489+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 54 ==== pg_backfill(progress 1.0 e 116/116 lb 1:02817247:::obj101:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2d600 con 0x4acb000 2026-02-11T23:31:27.489+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:02817247:::obj101:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.489+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:03553543:::obj227:head, version: 111'227, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:03553543:::obj227:head@111'227, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.489+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:02817247:::obj101:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.489+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.489+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:03553543:::obj227:head, version: 111'227, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:03553543:::obj227:head@111'227, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.489+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.489+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:03553543:::obj227:head, version: 111'227, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:03553543:::obj227:head@111'227, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:03553543:::obj227:head, version: 111'227, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:03553543:::obj227:head@111'227, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:024707f1:::obj490:head local-lis/les=113/114 n=8 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.489+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.489+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:024707f1:::obj490:head local-lis/les=113/114 n=8 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:03553543:::obj227:head, version: 111'227, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:03553543:::obj227:head@111'227, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.489+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:024707f1:::obj490:head local-lis/les=113/114 n=8 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:03553543:::obj227:head@111'227, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.489+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.489+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:02817247:::obj101:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.489+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:024707f1:::obj490:head local-lis/les=113/114 n=8 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:03553543:::obj227:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.489+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:024707f1:::obj490:head local-lis/les=113/114 n=8 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:03553543:::obj227:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.489+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:024707f1:::obj490:head local-lis/les=113/114 n=8 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:03553543:::obj227:head 2026-02-11T23:31:27.489+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:024707f1:::obj490:head local-lis/les=113/114 n=8 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:03553543:::obj227:head v 111'227 2026-02-11T23:31:27.489+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:024707f1:::obj490:head local-lis/les=113/114 n=8 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.489+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.489+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:03553543:::obj227:head, version: 111'227, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:03553543:::obj227:head@111'227, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:27.489+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:024707f1:::obj490:head local-lis/les=113/114 n=8 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.489+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.489+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.489+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:02817247:::obj101:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.489+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:02817247:::obj101:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 1.0 e 116/116 lb 1:02817247:::obj101:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:024707f1:::obj490:head local-lis/les=113/114 n=8 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.489+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.489+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:024707f1:::obj490:head local-lis/les=113/114 n=8 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:02817247:::obj101:head) v3 2026-02-11T23:31:27.489+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:024707f1:::obj490:head local-lis/les=113/114 n=8 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:02817247:::obj101:head) 2026-02-11T23:31:27.489+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.489+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:02817247:::obj101:head) v3 finish latency 0.000013 2026-02-11T23:31:27.489+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.489+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.489+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:02817247:::obj101:head local-lis/les=113/114 n=9 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.489+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.489+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.489+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.489+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x4ec2480 con 0x4acb000 2026-02-11T23:31:27.489+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:03553543:::obj227:head)]) -- 0x4ec2780 con 0x4acb000 2026-02-11T23:31:27.489+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.489+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 55 ==== MOSDPGPush(1.0 116/113 [PushOp(1:0366f813:::obj557:head, version: 111'557, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0366f813:::obj557:head@111'557, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e27ba0 con 0x4acb000 2026-02-11T23:31:27.489+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:0366f813:::obj557:head, version: 111'557, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0366f813:::obj557:head@111'557, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.489+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0366f813:::obj557:head, version: 111'557, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0366f813:::obj557:head@111'557, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.490+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.490+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 56 ==== pg_backfill(progress 1.0 e 116/116 lb 1:03553543:::obj227:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2db80 con 0x4acb000 2026-02-11T23:31:27.490+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0366f813:::obj557:head, version: 111'557, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0366f813:::obj557:head@111'557, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.490+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:03553543:::obj227:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.490+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:03553543:::obj227:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.490+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.490+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0366f813:::obj557:head, version: 111'557, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0366f813:::obj557:head@111'557, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.490+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.490+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:0366f813:::obj557:head, version: 111'557, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0366f813:::obj557:head@111'557, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 116/113 [PushOp(1:0366f813:::obj557:head, version: 111'557, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0366f813:::obj557:head@111'557, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:02817247:::obj101:head local-lis/les=113/114 n=9 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.490+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:03553543:::obj227:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.490+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.490+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:02817247:::obj101:head local-lis/les=113/114 n=9 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:0366f813:::obj557:head, version: 111'557, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0366f813:::obj557:head@111'557, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.490+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:02817247:::obj101:head local-lis/les=113/114 n=9 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:0366f813:::obj557:head@111'557, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.490+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:02817247:::obj101:head local-lis/les=113/114 n=9 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:0366f813:::obj557:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.490+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:02817247:::obj101:head local-lis/les=113/114 n=9 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:0366f813:::obj557:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.490+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:02817247:::obj101:head local-lis/les=113/114 n=9 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:0366f813:::obj557:head 2026-02-11T23:31:27.490+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:02817247:::obj101:head local-lis/les=113/114 n=9 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:0366f813:::obj557:head v 111'557 2026-02-11T23:31:27.490+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:02817247:::obj101:head local-lis/les=113/114 n=9 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.490+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.490+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:0366f813:::obj557:head, version: 111'557, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0366f813:::obj557:head@111'557, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:27.490+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:02817247:::obj101:head local-lis/les=113/114 n=9 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.490+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.490+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.490+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:03553543:::obj227:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.490+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:03553543:::obj227:head) v3 prio 15 cost 0 latency 0.000079 pg_backfill(progress 1.0 e 116/116 lb 1:03553543:::obj227:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:02817247:::obj101:head local-lis/les=113/114 n=9 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.490+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.490+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:02817247:::obj101:head local-lis/les=113/114 n=9 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:03553543:::obj227:head) v3 2026-02-11T23:31:27.490+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:02817247:::obj101:head local-lis/les=113/114 n=9 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:03553543:::obj227:head) 2026-02-11T23:31:27.490+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.490+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:03553543:::obj227:head) v3 finish latency 0.000013 2026-02-11T23:31:27.490+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.490+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.490+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:03553543:::obj227:head local-lis/les=113/114 n=10 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.490+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.490+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.490+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.490+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x4ec2d80 con 0x4acb000 2026-02-11T23:31:27.490+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:0366f813:::obj557:head)]) -- 0x4ec3680 con 0x4acb000 2026-02-11T23:31:27.490+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.490+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 57 ==== MOSDPGPush(1.0 116/113 [PushOp(1:03b9100b:::obj283:head, version: 111'283, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:03b9100b:::obj283:head@111'283, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db01a0 con 0x4acb000 2026-02-11T23:31:27.490+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:03b9100b:::obj283:head, version: 111'283, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:03b9100b:::obj283:head@111'283, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 116 2026-02-11T23:31:27.490+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:03b9100b:::obj283:head, version: 111'283, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:03b9100b:::obj283:head@111'283, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.490+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.490+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 58 ==== pg_backfill(progress 1.0 e 116/116 lb 1:0366f813:::obj557:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e100 con 0x4acb000 2026-02-11T23:31:27.490+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:0366f813:::obj557:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.490+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0366f813:::obj557:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.490+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:03b9100b:::obj283:head, version: 111'283, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:03b9100b:::obj283:head@111'283, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.490+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.490+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:03b9100b:::obj283:head, version: 111'283, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:03b9100b:::obj283:head@111'283, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.490+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:03b9100b:::obj283:head, version: 111'283, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:03b9100b:::obj283:head@111'283, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(1.0 116/113 [PushOp(1:03b9100b:::obj283:head, version: 111'283, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:03b9100b:::obj283:head@111'283, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:03553543:::obj227:head local-lis/les=113/114 n=10 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.490+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.490+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.490+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:03553543:::obj227:head local-lis/les=113/114 n=10 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:03b9100b:::obj283:head, version: 111'283, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:03b9100b:::obj283:head@111'283, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.490+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0366f813:::obj557:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.490+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:03553543:::obj227:head local-lis/les=113/114 n=10 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:03b9100b:::obj283:head@111'283, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.490+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:03553543:::obj227:head local-lis/les=113/114 n=10 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:03b9100b:::obj283:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.490+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:03553543:::obj227:head local-lis/les=113/114 n=10 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:03b9100b:::obj283:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.490+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:03553543:::obj227:head local-lis/les=113/114 n=10 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:03b9100b:::obj283:head 2026-02-11T23:31:27.490+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:03553543:::obj227:head local-lis/les=113/114 n=10 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:03b9100b:::obj283:head v 111'283 2026-02-11T23:31:27.490+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:03553543:::obj227:head local-lis/les=113/114 n=10 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.490+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.490+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:03b9100b:::obj283:head, version: 111'283, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:03b9100b:::obj283:head@111'283, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:27.490+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:03553543:::obj227:head local-lis/les=113/114 n=10 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.490+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.490+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.490+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0366f813:::obj557:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.490+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:0366f813:::obj557:head) v3 prio 15 cost 0 latency 0.000082 pg_backfill(progress 1.0 e 116/116 lb 1:0366f813:::obj557:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:03553543:::obj227:head local-lis/les=113/114 n=10 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.490+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.490+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:03553543:::obj227:head local-lis/les=113/114 n=10 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:0366f813:::obj557:head) v3 2026-02-11T23:31:27.490+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:03553543:::obj227:head local-lis/les=113/114 n=10 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:0366f813:::obj557:head) 2026-02-11T23:31:27.490+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.490+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:0366f813:::obj557:head) v3 finish latency 0.000012 2026-02-11T23:31:27.490+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.490+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.490+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0366f813:::obj557:head local-lis/les=113/114 n=11 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.490+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.490+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.490+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.490+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x4ecc000 con 0x4acb000 2026-02-11T23:31:27.490+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:03b9100b:::obj283:head)]) -- 0x4a89980 con 0x4acb000 2026-02-11T23:31:27.490+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.491+0000 7f41683f5640 10 monclient: _renew_subs 2026-02-11T23:31:27.491+0000 7f41683f5640 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:31:27.491+0000 7f41683f5640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> v2:127.0.0.1:7180/0 -- mon_subscribe({osdmap=117}) -- 0x5db0ea0 con 0x4acbc00 2026-02-11T23:31:27.491+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 59 ==== MOSDPGPush(1.0 116/113 [PushOp(1:040c73f9:::obj237:head, version: 111'237, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:040c73f9:::obj237:head@111'237, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db0680 con 0x4acb000 2026-02-11T23:31:27.491+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:040c73f9:::obj237:head, version: 111'237, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:040c73f9:::obj237:head@111'237, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 116 2026-02-11T23:31:27.491+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:040c73f9:::obj237:head, version: 111'237, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:040c73f9:::obj237:head@111'237, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.491+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.491+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 60 ==== pg_backfill(progress 1.0 e 116/116 lb 1:03b9100b:::obj283:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e680 con 0x4acb000 2026-02-11T23:31:27.491+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:040c73f9:::obj237:head, version: 111'237, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:040c73f9:::obj237:head@111'237, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.491+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:03b9100b:::obj283:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.491+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:03b9100b:::obj283:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.491+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.491+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:040c73f9:::obj237:head, version: 111'237, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:040c73f9:::obj237:head@111'237, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.491+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.491+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:040c73f9:::obj237:head, version: 111'237, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:040c73f9:::obj237:head@111'237, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000044 MOSDPGPush(1.0 116/113 [PushOp(1:040c73f9:::obj237:head, version: 111'237, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:040c73f9:::obj237:head@111'237, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:0366f813:::obj557:head local-lis/les=113/114 n=11 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.491+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.491+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0366f813:::obj557:head local-lis/les=113/114 n=11 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:040c73f9:::obj237:head, version: 111'237, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:040c73f9:::obj237:head@111'237, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.491+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0366f813:::obj557:head local-lis/les=113/114 n=11 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:040c73f9:::obj237:head@111'237, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.491+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.491+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:03b9100b:::obj283:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.491+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0366f813:::obj557:head local-lis/les=113/114 n=11 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:040c73f9:::obj237:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.491+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0366f813:::obj557:head local-lis/les=113/114 n=11 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:040c73f9:::obj237:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.491+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0366f813:::obj557:head local-lis/les=113/114 n=11 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:040c73f9:::obj237:head 2026-02-11T23:31:27.491+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0366f813:::obj557:head local-lis/les=113/114 n=11 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:040c73f9:::obj237:head v 111'237 2026-02-11T23:31:27.491+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0366f813:::obj557:head local-lis/les=113/114 n=11 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.491+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.491+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:040c73f9:::obj237:head, version: 111'237, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:040c73f9:::obj237:head@111'237, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000067 2026-02-11T23:31:27.491+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.491+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.491+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:03b9100b:::obj283:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.491+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:03b9100b:::obj283:head) v3 prio 15 cost 0 latency 0.000091 pg_backfill(progress 1.0 e 116/116 lb 1:03b9100b:::obj283:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:0366f813:::obj557:head local-lis/les=113/114 n=11 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.491+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.491+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0366f813:::obj557:head local-lis/les=113/114 n=11 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:03b9100b:::obj283:head) v3 2026-02-11T23:31:27.491+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0366f813:::obj557:head local-lis/les=113/114 n=11 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:03b9100b:::obj283:head) 2026-02-11T23:31:27.491+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.491+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:03b9100b:::obj283:head) v3 finish latency 0.000013 2026-02-11T23:31:27.491+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:03b9100b:::obj283:head local-lis/les=113/114 n=12 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.491+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.491+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.491+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:03b9100b:::obj283:head local-lis/les=113/114 n=12 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.491+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.491+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.491+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.491+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x4ecc480 con 0x4acb000 2026-02-11T23:31:27.491+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:040c73f9:::obj237:head)]) -- 0x5e3a180 con 0x4acb000 2026-02-11T23:31:27.491+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.492+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 61 ==== MOSDPGPush(1.0 116/113 [PushOp(1:04510f17:::obj73:head, version: 111'73, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04510f17:::obj73:head@111'73, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5db0b60 con 0x4acb000 2026-02-11T23:31:27.492+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:04510f17:::obj73:head, version: 111'73, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04510f17:::obj73:head@111'73, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.492+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:04510f17:::obj73:head, version: 111'73, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04510f17:::obj73:head@111'73, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.492+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.492+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 62 ==== pg_backfill(progress 1.0 e 116/116 lb 1:040c73f9:::obj237:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2ec00 con 0x4acb000 2026-02-11T23:31:27.492+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:04510f17:::obj73:head, version: 111'73, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04510f17:::obj73:head@111'73, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.492+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:040c73f9:::obj237:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.492+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:040c73f9:::obj237:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.492+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.492+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:04510f17:::obj73:head, version: 111'73, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04510f17:::obj73:head@111'73, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.492+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:04510f17:::obj73:head, version: 111'73, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04510f17:::obj73:head@111'73, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 116/113 [PushOp(1:04510f17:::obj73:head, version: 111'73, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04510f17:::obj73:head@111'73, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:03b9100b:::obj283:head local-lis/les=113/114 n=12 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.492+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.492+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.492+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:03b9100b:::obj283:head local-lis/les=113/114 n=12 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:04510f17:::obj73:head, version: 111'73, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04510f17:::obj73:head@111'73, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.492+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:040c73f9:::obj237:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.492+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:03b9100b:::obj283:head local-lis/les=113/114 n=12 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:04510f17:::obj73:head@111'73, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.492+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:03b9100b:::obj283:head local-lis/les=113/114 n=12 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:04510f17:::obj73:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.492+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:03b9100b:::obj283:head local-lis/les=113/114 n=12 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:04510f17:::obj73:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.492+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:03b9100b:::obj283:head local-lis/les=113/114 n=12 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:04510f17:::obj73:head 2026-02-11T23:31:27.492+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:03b9100b:::obj283:head local-lis/les=113/114 n=12 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:04510f17:::obj73:head v 111'73 2026-02-11T23:31:27.492+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:03b9100b:::obj283:head local-lis/les=113/114 n=12 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.492+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.492+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:04510f17:::obj73:head, version: 111'73, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04510f17:::obj73:head@111'73, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:27.492+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.492+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.492+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:040c73f9:::obj237:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.492+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:040c73f9:::obj237:head) v3 prio 15 cost 0 latency 0.000082 pg_backfill(progress 1.0 e 116/116 lb 1:040c73f9:::obj237:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:03b9100b:::obj283:head local-lis/les=113/114 n=12 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.492+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.492+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:03b9100b:::obj283:head local-lis/les=113/114 n=12 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:040c73f9:::obj237:head) v3 2026-02-11T23:31:27.492+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:03b9100b:::obj283:head local-lis/les=113/114 n=12 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:040c73f9:::obj237:head) 2026-02-11T23:31:27.492+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.492+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:040c73f9:::obj237:head) v3 finish latency 0.000040 2026-02-11T23:31:27.492+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:040c73f9:::obj237:head local-lis/les=113/114 n=13 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.492+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.492+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.492+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:040c73f9:::obj237:head local-lis/les=113/114 n=13 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.492+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.492+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.492+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.492+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x4ecde00 con 0x4acb000 2026-02-11T23:31:27.492+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:04510f17:::obj73:head)]) -- 0x5e3a480 con 0x4acb000 2026-02-11T23:31:27.492+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.492+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.492+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.492+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 63 ==== MOSDPGPush(1.0 116/113 [PushOp(1:0485ab12:::obj244:head, version: 111'244, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0485ab12:::obj244:head@111'244, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db1040 con 0x4acb000 2026-02-11T23:31:27.492+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:0485ab12:::obj244:head, version: 111'244, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0485ab12:::obj244:head@111'244, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.492+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0485ab12:::obj244:head, version: 111'244, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0485ab12:::obj244:head@111'244, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.492+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.492+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 64 ==== pg_backfill(progress 1.0 e 116/116 lb 1:04510f17:::obj73:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d2f180 con 0x4acb000 2026-02-11T23:31:27.492+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:04510f17:::obj73:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 116 2026-02-11T23:31:27.492+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:04510f17:::obj73:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.492+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0485ab12:::obj244:head, version: 111'244, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0485ab12:::obj244:head@111'244, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.492+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.492+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0485ab12:::obj244:head, version: 111'244, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0485ab12:::obj244:head@111'244, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.492+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:0485ab12:::obj244:head, version: 111'244, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0485ab12:::obj244:head@111'244, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000040 MOSDPGPush(1.0 116/113 [PushOp(1:0485ab12:::obj244:head, version: 111'244, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0485ab12:::obj244:head@111'244, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:040c73f9:::obj237:head local-lis/les=113/114 n=13 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.492+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.492+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.492+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:04510f17:::obj73:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.492+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:040c73f9:::obj237:head local-lis/les=113/114 n=13 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:0485ab12:::obj244:head, version: 111'244, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0485ab12:::obj244:head@111'244, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.492+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:040c73f9:::obj237:head local-lis/les=113/114 n=13 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:0485ab12:::obj244:head@111'244, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.492+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:040c73f9:::obj237:head local-lis/les=113/114 n=13 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:0485ab12:::obj244:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.492+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:040c73f9:::obj237:head local-lis/les=113/114 n=13 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:0485ab12:::obj244:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.492+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:040c73f9:::obj237:head local-lis/les=113/114 n=13 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:0485ab12:::obj244:head 2026-02-11T23:31:27.492+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:040c73f9:::obj237:head local-lis/les=113/114 n=13 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:0485ab12:::obj244:head v 111'244 2026-02-11T23:31:27.492+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:040c73f9:::obj237:head local-lis/les=113/114 n=13 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.492+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.492+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:0485ab12:::obj244:head, version: 111'244, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0485ab12:::obj244:head@111'244, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000091 2026-02-11T23:31:27.492+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:040c73f9:::obj237:head local-lis/les=113/114 n=13 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.492+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.492+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.492+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:04510f17:::obj73:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.492+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:04510f17:::obj73:head) v3 prio 15 cost 0 latency 0.000125 pg_backfill(progress 1.0 e 116/116 lb 1:04510f17:::obj73:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:040c73f9:::obj237:head local-lis/les=113/114 n=13 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.492+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.492+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:040c73f9:::obj237:head local-lis/les=113/114 n=13 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:04510f17:::obj73:head) v3 2026-02-11T23:31:27.492+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:040c73f9:::obj237:head local-lis/les=113/114 n=13 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:04510f17:::obj73:head) 2026-02-11T23:31:27.492+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.492+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:04510f17:::obj73:head) v3 finish latency 0.000019 2026-02-11T23:31:27.492+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.492+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.492+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:04510f17:::obj73:head local-lis/les=113/114 n=14 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.492+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.492+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.492+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.492+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x4ec2000 con 0x4acb000 2026-02-11T23:31:27.492+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:0485ab12:::obj244:head)]) -- 0x4c9af00 con 0x4acb000 2026-02-11T23:31:27.492+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.493+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 65 ==== MOSDPGPush(1.0 116/113 [PushOp(1:04865490:::obj600:head, version: 111'600, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04865490:::obj600:head@111'600, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e27d40 con 0x4acb000 2026-02-11T23:31:27.493+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:04865490:::obj600:head, version: 111'600, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04865490:::obj600:head@111'600, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.493+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:04865490:::obj600:head, version: 111'600, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04865490:::obj600:head@111'600, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.493+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.493+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 66 ==== pg_backfill(progress 1.0 e 116/116 lb 1:0485ab12:::obj244:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2f700 con 0x4acb000 2026-02-11T23:31:27.493+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:0485ab12:::obj244:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.493+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:04865490:::obj600:head, version: 111'600, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04865490:::obj600:head@111'600, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.493+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0485ab12:::obj244:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.493+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.493+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0485ab12:::obj244:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.493+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.493+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:04865490:::obj600:head, version: 111'600, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04865490:::obj600:head@111'600, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.493+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:04865490:::obj600:head, version: 111'600, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04865490:::obj600:head@111'600, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000045 MOSDPGPush(1.0 116/113 [PushOp(1:04865490:::obj600:head, version: 111'600, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04865490:::obj600:head@111'600, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:04510f17:::obj73:head local-lis/les=113/114 n=14 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.493+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.493+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:04510f17:::obj73:head local-lis/les=113/114 n=14 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:04865490:::obj600:head, version: 111'600, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04865490:::obj600:head@111'600, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.493+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:04510f17:::obj73:head local-lis/les=113/114 n=14 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:04865490:::obj600:head@111'600, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.493+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:04510f17:::obj73:head local-lis/les=113/114 n=14 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:04865490:::obj600:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.493+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:04510f17:::obj73:head local-lis/les=113/114 n=14 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:04865490:::obj600:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.493+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:04510f17:::obj73:head local-lis/les=113/114 n=14 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:04865490:::obj600:head 2026-02-11T23:31:27.493+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:04510f17:::obj73:head local-lis/les=113/114 n=14 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:04865490:::obj600:head v 111'600 2026-02-11T23:31:27.493+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:04510f17:::obj73:head local-lis/les=113/114 n=14 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.493+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.493+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:04865490:::obj600:head, version: 111'600, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04865490:::obj600:head@111'600, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:27.493+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.493+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.493+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0485ab12:::obj244:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.493+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:0485ab12:::obj244:head) v3 prio 15 cost 0 latency 0.000090 pg_backfill(progress 1.0 e 116/116 lb 1:0485ab12:::obj244:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:04510f17:::obj73:head local-lis/les=113/114 n=14 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.493+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.493+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:04510f17:::obj73:head local-lis/les=113/114 n=14 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:0485ab12:::obj244:head) v3 2026-02-11T23:31:27.493+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:04510f17:::obj73:head local-lis/les=113/114 n=14 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:0485ab12:::obj244:head) 2026-02-11T23:31:27.493+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.493+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:0485ab12:::obj244:head) v3 finish latency 0.000022 2026-02-11T23:31:27.493+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0485ab12:::obj244:head local-lis/les=113/114 n=15 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.493+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.493+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.493+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0485ab12:::obj244:head local-lis/les=113/114 n=15 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.493+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.493+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.493+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.493+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x4ec3980 con 0x4acb000 2026-02-11T23:31:27.493+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:04865490:::obj600:head)]) -- 0x5e3a900 con 0x4acb000 2026-02-11T23:31:27.493+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.494+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 67 ==== MOSDPGPush(1.0 116/113 [PushOp(1:048f8fd3:::obj518:head, version: 111'518, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:048f8fd3:::obj518:head@111'518, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e27a00 con 0x4acb000 2026-02-11T23:31:27.494+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:048f8fd3:::obj518:head, version: 111'518, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:048f8fd3:::obj518:head@111'518, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.494+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:048f8fd3:::obj518:head, version: 111'518, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:048f8fd3:::obj518:head@111'518, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.494+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.494+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 68 ==== pg_backfill(progress 1.0 e 116/116 lb 1:04865490:::obj600:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2d600 con 0x4acb000 2026-02-11T23:31:27.494+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:04865490:::obj600:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.494+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:048f8fd3:::obj518:head, version: 111'518, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:048f8fd3:::obj518:head@111'518, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.494+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:04865490:::obj600:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.494+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.494+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:048f8fd3:::obj518:head, version: 111'518, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:048f8fd3:::obj518:head@111'518, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.494+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:048f8fd3:::obj518:head, version: 111'518, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:048f8fd3:::obj518:head@111'518, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:048f8fd3:::obj518:head, version: 111'518, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:048f8fd3:::obj518:head@111'518, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:0485ab12:::obj244:head local-lis/les=113/114 n=15 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.494+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.494+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.494+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:04865490:::obj600:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.494+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0485ab12:::obj244:head local-lis/les=113/114 n=15 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:048f8fd3:::obj518:head, version: 111'518, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:048f8fd3:::obj518:head@111'518, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.494+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0485ab12:::obj244:head local-lis/les=113/114 n=15 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:048f8fd3:::obj518:head@111'518, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.494+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0485ab12:::obj244:head local-lis/les=113/114 n=15 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:048f8fd3:::obj518:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.494+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0485ab12:::obj244:head local-lis/les=113/114 n=15 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:048f8fd3:::obj518:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.494+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0485ab12:::obj244:head local-lis/les=113/114 n=15 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:048f8fd3:::obj518:head 2026-02-11T23:31:27.494+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0485ab12:::obj244:head local-lis/les=113/114 n=15 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:048f8fd3:::obj518:head v 111'518 2026-02-11T23:31:27.494+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0485ab12:::obj244:head local-lis/les=113/114 n=15 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.494+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.494+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:048f8fd3:::obj518:head, version: 111'518, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:048f8fd3:::obj518:head@111'518, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000072 2026-02-11T23:31:27.494+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0485ab12:::obj244:head local-lis/les=113/114 n=15 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.494+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.494+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:04865490:::obj600:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.494+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.494+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:04865490:::obj600:head) v3 prio 15 cost 0 latency 0.000094 pg_backfill(progress 1.0 e 116/116 lb 1:04865490:::obj600:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:0485ab12:::obj244:head local-lis/les=113/114 n=15 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.494+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.494+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0485ab12:::obj244:head local-lis/les=113/114 n=15 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:04865490:::obj600:head) v3 2026-02-11T23:31:27.494+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0485ab12:::obj244:head local-lis/les=113/114 n=15 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:04865490:::obj600:head) 2026-02-11T23:31:27.494+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.494+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:04865490:::obj600:head) v3 finish latency 0.000024 2026-02-11T23:31:27.494+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.494+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.494+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:04865490:::obj600:head local-lis/les=113/114 n=16 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.494+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.494+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.494+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.494+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e3b500 con 0x4acb000 2026-02-11T23:31:27.494+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:048f8fd3:::obj518:head)]) -- 0x5e3b200 con 0x4acb000 2026-02-11T23:31:27.494+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.494+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 69 ==== MOSDPGPush(1.0 116/113 [PushOp(1:04b71422:::obj226:head, version: 111'226, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04b71422:::obj226:head@111'226, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db01a0 con 0x4acb000 2026-02-11T23:31:27.494+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:04b71422:::obj226:head, version: 111'226, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04b71422:::obj226:head@111'226, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.494+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:04b71422:::obj226:head, version: 111'226, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04b71422:::obj226:head@111'226, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.494+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.494+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 70 ==== pg_backfill(progress 1.0 e 116/116 lb 1:048f8fd3:::obj518:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2db80 con 0x4acb000 2026-02-11T23:31:27.494+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:04b71422:::obj226:head, version: 111'226, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04b71422:::obj226:head@111'226, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.494+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:048f8fd3:::obj518:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 116 2026-02-11T23:31:27.494+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:048f8fd3:::obj518:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.494+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.494+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:04b71422:::obj226:head, version: 111'226, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04b71422:::obj226:head@111'226, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.494+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:04b71422:::obj226:head, version: 111'226, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04b71422:::obj226:head@111'226, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 116/113 [PushOp(1:04b71422:::obj226:head, version: 111'226, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04b71422:::obj226:head@111'226, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:04865490:::obj600:head local-lis/les=113/114 n=16 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.494+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.494+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.494+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:048f8fd3:::obj518:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.494+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:04865490:::obj600:head local-lis/les=113/114 n=16 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:04b71422:::obj226:head, version: 111'226, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04b71422:::obj226:head@111'226, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.494+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:04865490:::obj600:head local-lis/les=113/114 n=16 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:04b71422:::obj226:head@111'226, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.494+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:04865490:::obj600:head local-lis/les=113/114 n=16 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:04b71422:::obj226:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.494+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:04865490:::obj600:head local-lis/les=113/114 n=16 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:04b71422:::obj226:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.494+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:04865490:::obj600:head local-lis/les=113/114 n=16 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:04b71422:::obj226:head 2026-02-11T23:31:27.494+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:04865490:::obj600:head local-lis/les=113/114 n=16 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:04b71422:::obj226:head v 111'226 2026-02-11T23:31:27.494+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:04865490:::obj600:head local-lis/les=113/114 n=16 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.494+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.494+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:04b71422:::obj226:head, version: 111'226, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04b71422:::obj226:head@111'226, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000069 2026-02-11T23:31:27.494+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:04865490:::obj600:head local-lis/les=113/114 n=16 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.494+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.494+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.494+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:048f8fd3:::obj518:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.495+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:048f8fd3:::obj518:head) v3 prio 15 cost 0 latency 0.000098 pg_backfill(progress 1.0 e 116/116 lb 1:048f8fd3:::obj518:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:04865490:::obj600:head local-lis/les=113/114 n=16 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.495+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.495+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:04865490:::obj600:head local-lis/les=113/114 n=16 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:048f8fd3:::obj518:head) v3 2026-02-11T23:31:27.495+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:04865490:::obj600:head local-lis/les=113/114 n=16 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:048f8fd3:::obj518:head) 2026-02-11T23:31:27.495+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.495+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:048f8fd3:::obj518:head) v3 finish latency 0.000019 2026-02-11T23:31:27.495+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.495+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.495+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:048f8fd3:::obj518:head local-lis/les=113/114 n=17 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.495+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.495+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.495+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.495+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e3b980 con 0x4acb000 2026-02-11T23:31:27.495+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:04b71422:::obj226:head)]) -- 0x5e3b680 con 0x4acb000 2026-02-11T23:31:27.495+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.495+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.495+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.495+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 71 ==== MOSDPGPush(1.0 116/113 [PushOp(1:05651d4b:::obj461:head, version: 111'461, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05651d4b:::obj461:head@111'461, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db0d00 con 0x4acb000 2026-02-11T23:31:27.495+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:05651d4b:::obj461:head, version: 111'461, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05651d4b:::obj461:head@111'461, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.495+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:05651d4b:::obj461:head, version: 111'461, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05651d4b:::obj461:head@111'461, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.495+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.495+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 72 ==== pg_backfill(progress 1.0 e 116/116 lb 1:04b71422:::obj226:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc2000 con 0x4acb000 2026-02-11T23:31:27.495+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:04b71422:::obj226:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.495+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:05651d4b:::obj461:head, version: 111'461, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05651d4b:::obj461:head@111'461, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.495+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:04b71422:::obj226:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.495+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.495+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:05651d4b:::obj461:head, version: 111'461, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05651d4b:::obj461:head@111'461, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.495+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:05651d4b:::obj461:head, version: 111'461, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05651d4b:::obj461:head@111'461, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 116/113 [PushOp(1:05651d4b:::obj461:head, version: 111'461, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05651d4b:::obj461:head@111'461, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:048f8fd3:::obj518:head local-lis/les=113/114 n=17 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.495+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.495+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.495+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:048f8fd3:::obj518:head local-lis/les=113/114 n=17 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:05651d4b:::obj461:head, version: 111'461, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05651d4b:::obj461:head@111'461, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.495+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:04b71422:::obj226:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.495+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:048f8fd3:::obj518:head local-lis/les=113/114 n=17 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:05651d4b:::obj461:head@111'461, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.495+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:048f8fd3:::obj518:head local-lis/les=113/114 n=17 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:05651d4b:::obj461:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.495+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:048f8fd3:::obj518:head local-lis/les=113/114 n=17 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:05651d4b:::obj461:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.495+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:048f8fd3:::obj518:head local-lis/les=113/114 n=17 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:05651d4b:::obj461:head 2026-02-11T23:31:27.495+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:048f8fd3:::obj518:head local-lis/les=113/114 n=17 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:05651d4b:::obj461:head v 111'461 2026-02-11T23:31:27.495+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:048f8fd3:::obj518:head local-lis/les=113/114 n=17 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.495+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.495+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:05651d4b:::obj461:head, version: 111'461, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05651d4b:::obj461:head@111'461, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:27.495+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.495+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.495+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:04b71422:::obj226:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.495+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:04b71422:::obj226:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 1.0 e 116/116 lb 1:04b71422:::obj226:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:048f8fd3:::obj518:head local-lis/les=113/114 n=17 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.495+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.495+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:048f8fd3:::obj518:head local-lis/les=113/114 n=17 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:04b71422:::obj226:head) v3 2026-02-11T23:31:27.495+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:048f8fd3:::obj518:head local-lis/les=113/114 n=17 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:04b71422:::obj226:head) 2026-02-11T23:31:27.495+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.495+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:04b71422:::obj226:head) v3 finish latency 0.000019 2026-02-11T23:31:27.495+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:04b71422:::obj226:head local-lis/les=113/114 n=18 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.495+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.495+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.495+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:04b71422:::obj226:head local-lis/les=113/114 n=18 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.495+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.495+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.495+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.495+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e3bb00 con 0x4acb000 2026-02-11T23:31:27.495+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:05651d4b:::obj461:head)]) -- 0x5e3af00 con 0x4acb000 2026-02-11T23:31:27.495+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.496+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 73 ==== MOSDPGPush(1.0 116/113 [PushOp(1:05793669:::obj415:head, version: 111'415, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05793669:::obj415:head@111'415, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e26340 con 0x4acb000 2026-02-11T23:31:27.496+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:05793669:::obj415:head, version: 111'415, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05793669:::obj415:head@111'415, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.496+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:05793669:::obj415:head, version: 111'415, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05793669:::obj415:head@111'415, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.496+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.496+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 74 ==== pg_backfill(progress 1.0 e 116/116 lb 1:05651d4b:::obj461:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc2580 con 0x4acb000 2026-02-11T23:31:27.496+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:05793669:::obj415:head, version: 111'415, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05793669:::obj415:head@111'415, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.496+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:05651d4b:::obj461:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.496+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:05651d4b:::obj461:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.496+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.496+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:05793669:::obj415:head, version: 111'415, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05793669:::obj415:head@111'415, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.496+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:05793669:::obj415:head, version: 111'415, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05793669:::obj415:head@111'415, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(1.0 116/113 [PushOp(1:05793669:::obj415:head, version: 111'415, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05793669:::obj415:head@111'415, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:04b71422:::obj226:head local-lis/les=113/114 n=18 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.496+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.496+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.496+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:04b71422:::obj226:head local-lis/les=113/114 n=18 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:05793669:::obj415:head, version: 111'415, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05793669:::obj415:head@111'415, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.496+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:05651d4b:::obj461:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.496+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:04b71422:::obj226:head local-lis/les=113/114 n=18 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:05793669:::obj415:head@111'415, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.496+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:04b71422:::obj226:head local-lis/les=113/114 n=18 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:05793669:::obj415:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.496+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:04b71422:::obj226:head local-lis/les=113/114 n=18 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:05793669:::obj415:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.496+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:04b71422:::obj226:head local-lis/les=113/114 n=18 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:05793669:::obj415:head 2026-02-11T23:31:27.496+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:04b71422:::obj226:head local-lis/les=113/114 n=18 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:05793669:::obj415:head v 111'415 2026-02-11T23:31:27.496+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:04b71422:::obj226:head local-lis/les=113/114 n=18 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.496+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.496+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:05793669:::obj415:head, version: 111'415, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05793669:::obj415:head@111'415, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:27.496+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:04b71422:::obj226:head local-lis/les=113/114 n=18 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.496+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.496+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.496+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:05651d4b:::obj461:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.496+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:05651d4b:::obj461:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 1.0 e 116/116 lb 1:05651d4b:::obj461:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:04b71422:::obj226:head local-lis/les=113/114 n=18 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.496+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.496+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:04b71422:::obj226:head local-lis/les=113/114 n=18 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:05651d4b:::obj461:head) v3 2026-02-11T23:31:27.496+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:04b71422:::obj226:head local-lis/les=113/114 n=18 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:05651d4b:::obj461:head) 2026-02-11T23:31:27.496+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.496+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:05651d4b:::obj461:head) v3 finish latency 0.000017 2026-02-11T23:31:27.496+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.496+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.496+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:05651d4b:::obj461:head local-lis/les=113/114 n=19 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.496+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.496+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.496+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.496+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x4ec2480 con 0x4acb000 2026-02-11T23:31:27.496+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:05793669:::obj415:head)]) -- 0x5e3bc80 con 0x4acb000 2026-02-11T23:31:27.496+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.496+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 75 ==== MOSDPGPush(1.0 116/113 [PushOp(1:05e4a07c:::obj265:head, version: 111'265, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05e4a07c:::obj265:head@111'265, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e271e0 con 0x4acb000 2026-02-11T23:31:27.496+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:05e4a07c:::obj265:head, version: 111'265, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05e4a07c:::obj265:head@111'265, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.496+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:05e4a07c:::obj265:head, version: 111'265, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05e4a07c:::obj265:head@111'265, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.496+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.496+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 76 ==== pg_backfill(progress 1.0 e 116/116 lb 1:05793669:::obj415:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc2b00 con 0x4acb000 2026-02-11T23:31:27.496+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:05e4a07c:::obj265:head, version: 111'265, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05e4a07c:::obj265:head@111'265, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.496+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:05793669:::obj415:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.496+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:05793669:::obj415:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.496+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.496+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:05e4a07c:::obj265:head, version: 111'265, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05e4a07c:::obj265:head@111'265, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.496+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:05e4a07c:::obj265:head, version: 111'265, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05e4a07c:::obj265:head@111'265, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 116/113 [PushOp(1:05e4a07c:::obj265:head, version: 111'265, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05e4a07c:::obj265:head@111'265, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:05651d4b:::obj461:head local-lis/les=113/114 n=19 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.496+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.496+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.496+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:05793669:::obj415:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.496+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:05651d4b:::obj461:head local-lis/les=113/114 n=19 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:05e4a07c:::obj265:head, version: 111'265, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05e4a07c:::obj265:head@111'265, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.496+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:05651d4b:::obj461:head local-lis/les=113/114 n=19 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:05e4a07c:::obj265:head@111'265, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.496+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:05651d4b:::obj461:head local-lis/les=113/114 n=19 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:05e4a07c:::obj265:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.496+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:05651d4b:::obj461:head local-lis/les=113/114 n=19 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:05e4a07c:::obj265:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.496+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:05651d4b:::obj461:head local-lis/les=113/114 n=19 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:05e4a07c:::obj265:head 2026-02-11T23:31:27.496+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:05651d4b:::obj461:head local-lis/les=113/114 n=19 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:05e4a07c:::obj265:head v 111'265 2026-02-11T23:31:27.496+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:05651d4b:::obj461:head local-lis/les=113/114 n=19 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.496+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.496+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:05e4a07c:::obj265:head, version: 111'265, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05e4a07c:::obj265:head@111'265, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:27.496+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:05651d4b:::obj461:head local-lis/les=113/114 n=19 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.496+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.496+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:05793669:::obj415:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.496+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:05793669:::obj415:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 1.0 e 116/116 lb 1:05793669:::obj415:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:05651d4b:::obj461:head local-lis/les=113/114 n=19 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.496+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.496+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:05651d4b:::obj461:head local-lis/les=113/114 n=19 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:05793669:::obj415:head) v3 2026-02-11T23:31:27.496+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.496+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:05651d4b:::obj461:head local-lis/les=113/114 n=19 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:05793669:::obj415:head) 2026-02-11T23:31:27.496+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.496+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:05793669:::obj415:head) v3 finish latency 0.000017 2026-02-11T23:31:27.496+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.497+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.497+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:05793669:::obj415:head local-lis/les=113/114 n=20 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.497+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.497+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.497+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.497+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x4ec3680 con 0x4acb000 2026-02-11T23:31:27.497+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:05e4a07c:::obj265:head)]) -- 0x4ec2780 con 0x4acb000 2026-02-11T23:31:27.497+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.497+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 77 ==== MOSDPGPush(1.0 116/113 [PushOp(1:05eb3c5d:::obj514:head, version: 111'514, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05eb3c5d:::obj514:head@111'514, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db0340 con 0x4acb000 2026-02-11T23:31:27.497+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:05eb3c5d:::obj514:head, version: 111'514, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05eb3c5d:::obj514:head@111'514, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.497+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:05eb3c5d:::obj514:head, version: 111'514, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05eb3c5d:::obj514:head@111'514, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.497+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.497+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 78 ==== pg_backfill(progress 1.0 e 116/116 lb 1:05e4a07c:::obj265:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc3080 con 0x4acb000 2026-02-11T23:31:27.497+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:05e4a07c:::obj265:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.497+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:05eb3c5d:::obj514:head, version: 111'514, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05eb3c5d:::obj514:head@111'514, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.497+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:05e4a07c:::obj265:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.497+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.497+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:05eb3c5d:::obj514:head, version: 111'514, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05eb3c5d:::obj514:head@111'514, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.497+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:05eb3c5d:::obj514:head, version: 111'514, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05eb3c5d:::obj514:head@111'514, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 116/113 [PushOp(1:05eb3c5d:::obj514:head, version: 111'514, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05eb3c5d:::obj514:head@111'514, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:05793669:::obj415:head local-lis/les=113/114 n=20 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.497+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.497+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.497+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:05e4a07c:::obj265:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.497+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:05793669:::obj415:head local-lis/les=113/114 n=20 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:05eb3c5d:::obj514:head, version: 111'514, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05eb3c5d:::obj514:head@111'514, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.497+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:05793669:::obj415:head local-lis/les=113/114 n=20 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:05eb3c5d:::obj514:head@111'514, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.497+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:05793669:::obj415:head local-lis/les=113/114 n=20 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:05eb3c5d:::obj514:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.497+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:05793669:::obj415:head local-lis/les=113/114 n=20 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:05eb3c5d:::obj514:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.497+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:05793669:::obj415:head local-lis/les=113/114 n=20 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:05eb3c5d:::obj514:head 2026-02-11T23:31:27.497+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:05793669:::obj415:head local-lis/les=113/114 n=20 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:05eb3c5d:::obj514:head v 111'514 2026-02-11T23:31:27.497+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:05793669:::obj415:head local-lis/les=113/114 n=20 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.497+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.497+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:05eb3c5d:::obj514:head, version: 111'514, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05eb3c5d:::obj514:head@111'514, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:27.497+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:05793669:::obj415:head local-lis/les=113/114 n=20 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.497+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.497+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.497+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:05e4a07c:::obj265:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.497+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:05e4a07c:::obj265:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 1.0 e 116/116 lb 1:05e4a07c:::obj265:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:05793669:::obj415:head local-lis/les=113/114 n=20 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.497+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.497+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:05793669:::obj415:head local-lis/les=113/114 n=20 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:05e4a07c:::obj265:head) v3 2026-02-11T23:31:27.497+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:05793669:::obj415:head local-lis/les=113/114 n=20 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:05e4a07c:::obj265:head) 2026-02-11T23:31:27.497+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.497+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:05e4a07c:::obj265:head) v3 finish latency 0.000014 2026-02-11T23:31:27.497+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.497+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.497+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:05e4a07c:::obj265:head local-lis/les=113/114 n=21 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.497+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.497+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.497+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.497+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x4ecc000 con 0x4acb000 2026-02-11T23:31:27.497+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:05eb3c5d:::obj514:head)]) -- 0x4ec2000 con 0x4acb000 2026-02-11T23:31:27.497+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.498+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 79 ==== MOSDPGPush(1.0 116/113 [PushOp(1:0698d46e:::obj173:head, version: 111'173, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0698d46e:::obj173:head@111'173, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db1860 con 0x4acb000 2026-02-11T23:31:27.498+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:0698d46e:::obj173:head, version: 111'173, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0698d46e:::obj173:head@111'173, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.498+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0698d46e:::obj173:head, version: 111'173, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0698d46e:::obj173:head@111'173, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.498+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.498+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 80 ==== pg_backfill(progress 1.0 e 116/116 lb 1:05eb3c5d:::obj514:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc3600 con 0x4acb000 2026-02-11T23:31:27.498+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:05eb3c5d:::obj514:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.498+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0698d46e:::obj173:head, version: 111'173, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0698d46e:::obj173:head@111'173, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.498+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:05eb3c5d:::obj514:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.498+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.498+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0698d46e:::obj173:head, version: 111'173, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0698d46e:::obj173:head@111'173, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.498+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.498+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:05eb3c5d:::obj514:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.498+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:0698d46e:::obj173:head, version: 111'173, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0698d46e:::obj173:head@111'173, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 116/113 [PushOp(1:0698d46e:::obj173:head, version: 111'173, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0698d46e:::obj173:head@111'173, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:05e4a07c:::obj265:head local-lis/les=113/114 n=21 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.498+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.498+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:05e4a07c:::obj265:head local-lis/les=113/114 n=21 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:0698d46e:::obj173:head, version: 111'173, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0698d46e:::obj173:head@111'173, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.498+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:05e4a07c:::obj265:head local-lis/les=113/114 n=21 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:0698d46e:::obj173:head@111'173, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.498+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:05e4a07c:::obj265:head local-lis/les=113/114 n=21 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:0698d46e:::obj173:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.498+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:05e4a07c:::obj265:head local-lis/les=113/114 n=21 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:0698d46e:::obj173:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.498+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:05e4a07c:::obj265:head local-lis/les=113/114 n=21 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:0698d46e:::obj173:head 2026-02-11T23:31:27.498+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:05e4a07c:::obj265:head local-lis/les=113/114 n=21 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:0698d46e:::obj173:head v 111'173 2026-02-11T23:31:27.498+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:05e4a07c:::obj265:head local-lis/les=113/114 n=21 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.498+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.498+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:0698d46e:::obj173:head, version: 111'173, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0698d46e:::obj173:head@111'173, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000067 2026-02-11T23:31:27.498+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:05e4a07c:::obj265:head local-lis/les=113/114 n=21 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.498+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.498+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:05eb3c5d:::obj514:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.498+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:05eb3c5d:::obj514:head) v3 prio 15 cost 0 latency 0.000086 pg_backfill(progress 1.0 e 116/116 lb 1:05eb3c5d:::obj514:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:05e4a07c:::obj265:head local-lis/les=113/114 n=21 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.498+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.498+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.498+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:05e4a07c:::obj265:head local-lis/les=113/114 n=21 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:05eb3c5d:::obj514:head) v3 2026-02-11T23:31:27.498+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:05e4a07c:::obj265:head local-lis/les=113/114 n=21 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:05eb3c5d:::obj514:head) 2026-02-11T23:31:27.498+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.498+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:05eb3c5d:::obj514:head) v3 finish latency 0.000016 2026-02-11T23:31:27.498+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.498+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:05eb3c5d:::obj514:head local-lis/les=113/114 n=22 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.498+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.498+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.498+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.498+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.498+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x4a89980 con 0x4acb000 2026-02-11T23:31:27.498+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:0698d46e:::obj173:head)]) -- 0x4ecc480 con 0x4acb000 2026-02-11T23:31:27.498+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.498+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 81 ==== MOSDPGPush(1.0 116/113 [PushOp(1:06c5d68c:::obj504:head, version: 111'504, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:06c5d68c:::obj504:head@111'504, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e26d00 con 0x4acb000 2026-02-11T23:31:27.498+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:06c5d68c:::obj504:head, version: 111'504, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:06c5d68c:::obj504:head@111'504, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.498+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:06c5d68c:::obj504:head, version: 111'504, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:06c5d68c:::obj504:head@111'504, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.498+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.498+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 82 ==== pg_backfill(progress 1.0 e 116/116 lb 1:0698d46e:::obj173:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc3b80 con 0x4acb000 2026-02-11T23:31:27.498+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:0698d46e:::obj173:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.498+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:06c5d68c:::obj504:head, version: 111'504, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:06c5d68c:::obj504:head@111'504, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.498+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0698d46e:::obj173:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.498+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.498+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:06c5d68c:::obj504:head, version: 111'504, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:06c5d68c:::obj504:head@111'504, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.498+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.498+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0698d46e:::obj173:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.498+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:06c5d68c:::obj504:head, version: 111'504, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:06c5d68c:::obj504:head@111'504, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:06c5d68c:::obj504:head, version: 111'504, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:06c5d68c:::obj504:head@111'504, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:05eb3c5d:::obj514:head local-lis/les=113/114 n=22 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.498+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.498+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:05eb3c5d:::obj514:head local-lis/les=113/114 n=22 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:06c5d68c:::obj504:head, version: 111'504, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:06c5d68c:::obj504:head@111'504, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.498+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:05eb3c5d:::obj514:head local-lis/les=113/114 n=22 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:06c5d68c:::obj504:head@111'504, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.498+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:05eb3c5d:::obj514:head local-lis/les=113/114 n=22 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:06c5d68c:::obj504:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.498+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:05eb3c5d:::obj514:head local-lis/les=113/114 n=22 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:06c5d68c:::obj504:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.498+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:05eb3c5d:::obj514:head local-lis/les=113/114 n=22 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:06c5d68c:::obj504:head 2026-02-11T23:31:27.498+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:05eb3c5d:::obj514:head local-lis/les=113/114 n=22 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:06c5d68c:::obj504:head v 111'504 2026-02-11T23:31:27.498+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:05eb3c5d:::obj514:head local-lis/les=113/114 n=22 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.498+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.498+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:06c5d68c:::obj504:head, version: 111'504, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:06c5d68c:::obj504:head@111'504, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:27.498+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:05eb3c5d:::obj514:head local-lis/les=113/114 n=22 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.498+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.498+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.498+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0698d46e:::obj173:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.498+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:0698d46e:::obj173:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 1.0 e 116/116 lb 1:0698d46e:::obj173:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:05eb3c5d:::obj514:head local-lis/les=113/114 n=22 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.498+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.498+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:05eb3c5d:::obj514:head local-lis/les=113/114 n=22 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:0698d46e:::obj173:head) v3 2026-02-11T23:31:27.498+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:05eb3c5d:::obj514:head local-lis/les=113/114 n=22 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:0698d46e:::obj173:head) 2026-02-11T23:31:27.498+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.498+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:0698d46e:::obj173:head) v3 finish latency 0.000015 2026-02-11T23:31:27.498+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.499+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.499+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0698d46e:::obj173:head local-lis/les=113/114 n=23 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.499+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.499+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.499+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.499+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e3a180 con 0x4acb000 2026-02-11T23:31:27.499+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:06c5d68c:::obj504:head)]) -- 0x5e3ad80 con 0x4acb000 2026-02-11T23:31:27.499+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.499+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 83 ==== MOSDPGPush(1.0 116/113 [PushOp(1:06d56074:::obj162:head, version: 111'162, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:06d56074:::obj162:head@111'162, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e26000 con 0x4acb000 2026-02-11T23:31:27.499+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:06d56074:::obj162:head, version: 111'162, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:06d56074:::obj162:head@111'162, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 116 2026-02-11T23:31:27.499+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:06d56074:::obj162:head, version: 111'162, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:06d56074:::obj162:head@111'162, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.499+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.499+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 84 ==== pg_backfill(progress 1.0 e 116/116 lb 1:06c5d68c:::obj504:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc4100 con 0x4acb000 2026-02-11T23:31:27.499+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:06d56074:::obj162:head, version: 111'162, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:06d56074:::obj162:head@111'162, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.499+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:06c5d68c:::obj504:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.499+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:06c5d68c:::obj504:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.499+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.499+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:06d56074:::obj162:head, version: 111'162, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:06d56074:::obj162:head@111'162, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.499+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:06d56074:::obj162:head, version: 111'162, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:06d56074:::obj162:head@111'162, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(1.0 116/113 [PushOp(1:06d56074:::obj162:head, version: 111'162, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:06d56074:::obj162:head@111'162, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:0698d46e:::obj173:head local-lis/les=113/114 n=23 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.499+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.499+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.499+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0698d46e:::obj173:head local-lis/les=113/114 n=23 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:06d56074:::obj162:head, version: 111'162, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:06d56074:::obj162:head@111'162, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.499+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:06c5d68c:::obj504:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.499+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0698d46e:::obj173:head local-lis/les=113/114 n=23 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:06d56074:::obj162:head@111'162, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.499+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0698d46e:::obj173:head local-lis/les=113/114 n=23 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:06d56074:::obj162:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.499+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0698d46e:::obj173:head local-lis/les=113/114 n=23 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:06d56074:::obj162:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.499+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0698d46e:::obj173:head local-lis/les=113/114 n=23 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:06d56074:::obj162:head 2026-02-11T23:31:27.499+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0698d46e:::obj173:head local-lis/les=113/114 n=23 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:06d56074:::obj162:head v 111'162 2026-02-11T23:31:27.499+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0698d46e:::obj173:head local-lis/les=113/114 n=23 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.499+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.499+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:06d56074:::obj162:head, version: 111'162, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:06d56074:::obj162:head@111'162, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:27.499+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0698d46e:::obj173:head local-lis/les=113/114 n=23 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.499+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.499+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.499+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:06c5d68c:::obj504:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.499+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:06c5d68c:::obj504:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 1.0 e 116/116 lb 1:06c5d68c:::obj504:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:0698d46e:::obj173:head local-lis/les=113/114 n=23 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.499+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.499+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0698d46e:::obj173:head local-lis/les=113/114 n=23 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:06c5d68c:::obj504:head) v3 2026-02-11T23:31:27.499+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0698d46e:::obj173:head local-lis/les=113/114 n=23 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:06c5d68c:::obj504:head) 2026-02-11T23:31:27.499+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.499+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:06c5d68c:::obj504:head) v3 finish latency 0.000014 2026-02-11T23:31:27.499+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.499+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.499+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:06c5d68c:::obj504:head local-lis/les=113/114 n=24 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.499+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.499+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.499+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.499+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e3b500 con 0x4acb000 2026-02-11T23:31:27.499+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:06d56074:::obj162:head)]) -- 0x5e3a480 con 0x4acb000 2026-02-11T23:31:27.499+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.500+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 85 ==== MOSDPGPush(1.0 116/113 [PushOp(1:07c44a4d:::obj181:head, version: 111'181, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:07c44a4d:::obj181:head@111'181, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db0340 con 0x4acb000 2026-02-11T23:31:27.500+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:07c44a4d:::obj181:head, version: 111'181, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:07c44a4d:::obj181:head@111'181, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.500+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:07c44a4d:::obj181:head, version: 111'181, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:07c44a4d:::obj181:head@111'181, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.500+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.500+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:07c44a4d:::obj181:head, version: 111'181, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:07c44a4d:::obj181:head@111'181, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.500+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 86 ==== pg_backfill(progress 1.0 e 116/116 lb 1:06d56074:::obj162:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc4680 con 0x4acb000 2026-02-11T23:31:27.500+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:06d56074:::obj162:head) v3 prio 15 type 95 cost 0 latency 0.000010 epoch 116 2026-02-11T23:31:27.500+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.500+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:06d56074:::obj162:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.500+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:07c44a4d:::obj181:head, version: 111'181, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:07c44a4d:::obj181:head@111'181, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.500+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.500+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:07c44a4d:::obj181:head, version: 111'181, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:07c44a4d:::obj181:head@111'181, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 116/113 [PushOp(1:07c44a4d:::obj181:head, version: 111'181, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:07c44a4d:::obj181:head@111'181, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:06c5d68c:::obj504:head local-lis/les=113/114 n=24 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.500+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.500+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:06c5d68c:::obj504:head local-lis/les=113/114 n=24 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:07c44a4d:::obj181:head, version: 111'181, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:07c44a4d:::obj181:head@111'181, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.500+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.500+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:06d56074:::obj162:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.500+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:06c5d68c:::obj504:head local-lis/les=113/114 n=24 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:07c44a4d:::obj181:head@111'181, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.500+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:06c5d68c:::obj504:head local-lis/les=113/114 n=24 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:07c44a4d:::obj181:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.500+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:06c5d68c:::obj504:head local-lis/les=113/114 n=24 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:07c44a4d:::obj181:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.500+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:06c5d68c:::obj504:head local-lis/les=113/114 n=24 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:07c44a4d:::obj181:head 2026-02-11T23:31:27.500+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:06c5d68c:::obj504:head local-lis/les=113/114 n=24 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:07c44a4d:::obj181:head v 111'181 2026-02-11T23:31:27.500+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:06c5d68c:::obj504:head local-lis/les=113/114 n=24 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.500+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.500+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:07c44a4d:::obj181:head, version: 111'181, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:07c44a4d:::obj181:head@111'181, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000065 2026-02-11T23:31:27.500+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.500+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.500+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:06d56074:::obj162:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.500+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:06d56074:::obj162:head) v3 prio 15 cost 0 latency 0.000089 pg_backfill(progress 1.0 e 116/116 lb 1:06d56074:::obj162:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:06c5d68c:::obj504:head local-lis/les=113/114 n=24 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.500+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.500+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:06c5d68c:::obj504:head local-lis/les=113/114 n=24 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:06d56074:::obj162:head) v3 2026-02-11T23:31:27.500+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:06c5d68c:::obj504:head local-lis/les=113/114 n=24 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:06d56074:::obj162:head) 2026-02-11T23:31:27.500+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.500+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:06d56074:::obj162:head) v3 finish latency 0.000019 2026-02-11T23:31:27.500+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:06d56074:::obj162:head local-lis/les=113/114 n=25 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.500+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.500+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.500+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:06d56074:::obj162:head local-lis/les=113/114 n=25 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.500+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.500+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.500+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.500+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e3b200 con 0x4acb000 2026-02-11T23:31:27.500+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:07c44a4d:::obj181:head)]) -- 0x4ecc600 con 0x4acb000 2026-02-11T23:31:27.500+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.500+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 87 ==== MOSDPGPush(1.0 116/113 [PushOp(1:080cd0cf:::obj146:head, version: 111'146, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:080cd0cf:::obj146:head@111'146, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db1860 con 0x4acb000 2026-02-11T23:31:27.500+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:080cd0cf:::obj146:head, version: 111'146, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:080cd0cf:::obj146:head@111'146, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 116 2026-02-11T23:31:27.500+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:080cd0cf:::obj146:head, version: 111'146, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:080cd0cf:::obj146:head@111'146, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.500+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.500+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:080cd0cf:::obj146:head, version: 111'146, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:080cd0cf:::obj146:head@111'146, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.500+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 88 ==== pg_backfill(progress 1.0 e 116/116 lb 1:07c44a4d:::obj181:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc2580 con 0x4acb000 2026-02-11T23:31:27.500+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:07c44a4d:::obj181:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.500+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.500+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:07c44a4d:::obj181:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.500+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:080cd0cf:::obj146:head, version: 111'146, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:080cd0cf:::obj146:head@111'146, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.500+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:080cd0cf:::obj146:head, version: 111'146, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:080cd0cf:::obj146:head@111'146, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(1.0 116/113 [PushOp(1:080cd0cf:::obj146:head, version: 111'146, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:080cd0cf:::obj146:head@111'146, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:06d56074:::obj162:head local-lis/les=113/114 n=25 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.500+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.500+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.500+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:06d56074:::obj162:head local-lis/les=113/114 n=25 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:080cd0cf:::obj146:head, version: 111'146, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:080cd0cf:::obj146:head@111'146, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.500+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:06d56074:::obj162:head local-lis/les=113/114 n=25 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:080cd0cf:::obj146:head@111'146, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.500+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.500+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:07c44a4d:::obj181:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.500+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:06d56074:::obj162:head local-lis/les=113/114 n=25 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:080cd0cf:::obj146:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.500+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:06d56074:::obj162:head local-lis/les=113/114 n=25 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:080cd0cf:::obj146:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.500+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:06d56074:::obj162:head local-lis/les=113/114 n=25 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:080cd0cf:::obj146:head 2026-02-11T23:31:27.500+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:06d56074:::obj162:head local-lis/les=113/114 n=25 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:080cd0cf:::obj146:head v 111'146 2026-02-11T23:31:27.500+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:06d56074:::obj162:head local-lis/les=113/114 n=25 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.500+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.500+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:080cd0cf:::obj146:head, version: 111'146, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:080cd0cf:::obj146:head@111'146, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:27.500+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:06d56074:::obj162:head local-lis/les=113/114 n=25 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.500+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.500+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.500+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:07c44a4d:::obj181:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.500+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:07c44a4d:::obj181:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 1.0 e 116/116 lb 1:07c44a4d:::obj181:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:06d56074:::obj162:head local-lis/les=113/114 n=25 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.500+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.500+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:06d56074:::obj162:head local-lis/les=113/114 n=25 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:07c44a4d:::obj181:head) v3 2026-02-11T23:31:27.500+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:06d56074:::obj162:head local-lis/les=113/114 n=25 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:07c44a4d:::obj181:head) 2026-02-11T23:31:27.500+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.500+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:07c44a4d:::obj181:head) v3 finish latency 0.000016 2026-02-11T23:31:27.500+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.500+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.500+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:07c44a4d:::obj181:head local-lis/les=113/114 n=26 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.500+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.500+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.500+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.500+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e6a900 con 0x4acb000 2026-02-11T23:31:27.500+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:080cd0cf:::obj146:head)]) -- 0x5e3b980 con 0x4acb000 2026-02-11T23:31:27.500+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.501+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 89 ==== MOSDPGPush(1.0 116/113 [PushOp(1:0877fdce:::obj530:head, version: 111'530, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0877fdce:::obj530:head@111'530, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e27520 con 0x4acb000 2026-02-11T23:31:27.501+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:0877fdce:::obj530:head, version: 111'530, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0877fdce:::obj530:head@111'530, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.501+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0877fdce:::obj530:head, version: 111'530, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0877fdce:::obj530:head@111'530, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.501+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 90 ==== pg_backfill(progress 1.0 e 116/116 lb 1:080cd0cf:::obj146:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc2b00 con 0x4acb000 2026-02-11T23:31:27.501+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.501+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:080cd0cf:::obj146:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.501+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:080cd0cf:::obj146:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.501+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0877fdce:::obj530:head, version: 111'530, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0877fdce:::obj530:head@111'530, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.501+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.501+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0877fdce:::obj530:head, version: 111'530, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0877fdce:::obj530:head@111'530, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.501+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:0877fdce:::obj530:head, version: 111'530, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0877fdce:::obj530:head@111'530, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 116/113 [PushOp(1:0877fdce:::obj530:head, version: 111'530, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0877fdce:::obj530:head@111'530, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:07c44a4d:::obj181:head local-lis/les=113/114 n=26 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.501+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.501+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.501+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:07c44a4d:::obj181:head local-lis/les=113/114 n=26 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:0877fdce:::obj530:head, version: 111'530, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0877fdce:::obj530:head@111'530, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.501+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:080cd0cf:::obj146:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.501+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:07c44a4d:::obj181:head local-lis/les=113/114 n=26 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:0877fdce:::obj530:head@111'530, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.501+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:07c44a4d:::obj181:head local-lis/les=113/114 n=26 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:0877fdce:::obj530:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.501+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:07c44a4d:::obj181:head local-lis/les=113/114 n=26 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:0877fdce:::obj530:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.501+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:07c44a4d:::obj181:head local-lis/les=113/114 n=26 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:0877fdce:::obj530:head 2026-02-11T23:31:27.501+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:07c44a4d:::obj181:head local-lis/les=113/114 n=26 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:0877fdce:::obj530:head v 111'530 2026-02-11T23:31:27.501+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:07c44a4d:::obj181:head local-lis/les=113/114 n=26 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.501+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.501+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:0877fdce:::obj530:head, version: 111'530, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0877fdce:::obj530:head@111'530, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:27.501+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:07c44a4d:::obj181:head local-lis/les=113/114 n=26 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.501+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.501+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.501+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:080cd0cf:::obj146:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.501+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:080cd0cf:::obj146:head) v3 prio 15 cost 0 latency 0.000084 pg_backfill(progress 1.0 e 116/116 lb 1:080cd0cf:::obj146:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:07c44a4d:::obj181:head local-lis/les=113/114 n=26 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.501+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.501+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:07c44a4d:::obj181:head local-lis/les=113/114 n=26 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:080cd0cf:::obj146:head) v3 2026-02-11T23:31:27.501+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:07c44a4d:::obj181:head local-lis/les=113/114 n=26 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:080cd0cf:::obj146:head) 2026-02-11T23:31:27.501+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.501+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:080cd0cf:::obj146:head) v3 finish latency 0.000014 2026-02-11T23:31:27.501+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.501+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.501+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:080cd0cf:::obj146:head local-lis/les=113/114 n=27 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.501+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.501+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.501+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.501+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e6ad80 con 0x4acb000 2026-02-11T23:31:27.501+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:0877fdce:::obj530:head)]) -- 0x5e6aa80 con 0x4acb000 2026-02-11T23:31:27.501+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.502+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 91 ==== MOSDPGPush(1.0 116/113 [PushOp(1:0881dee6:::obj412:head, version: 111'412, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0881dee6:::obj412:head@111'412, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e27d40 con 0x4acb000 2026-02-11T23:31:27.502+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:0881dee6:::obj412:head, version: 111'412, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0881dee6:::obj412:head@111'412, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.502+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0881dee6:::obj412:head, version: 111'412, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0881dee6:::obj412:head@111'412, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.502+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.502+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 92 ==== pg_backfill(progress 1.0 e 116/116 lb 1:0877fdce:::obj530:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc3080 con 0x4acb000 2026-02-11T23:31:27.502+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0881dee6:::obj412:head, version: 111'412, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0881dee6:::obj412:head@111'412, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.502+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:0877fdce:::obj530:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 116 2026-02-11T23:31:27.502+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.502+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0877fdce:::obj530:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.502+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0881dee6:::obj412:head, version: 111'412, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0881dee6:::obj412:head@111'412, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.502+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:0881dee6:::obj412:head, version: 111'412, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0881dee6:::obj412:head@111'412, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 116/113 [PushOp(1:0881dee6:::obj412:head, version: 111'412, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0881dee6:::obj412:head@111'412, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:080cd0cf:::obj146:head local-lis/les=113/114 n=27 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.502+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.502+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.502+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0877fdce:::obj530:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.502+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:080cd0cf:::obj146:head local-lis/les=113/114 n=27 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:0881dee6:::obj412:head, version: 111'412, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0881dee6:::obj412:head@111'412, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.502+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:080cd0cf:::obj146:head local-lis/les=113/114 n=27 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:0881dee6:::obj412:head@111'412, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.502+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:080cd0cf:::obj146:head local-lis/les=113/114 n=27 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:0881dee6:::obj412:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.502+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:080cd0cf:::obj146:head local-lis/les=113/114 n=27 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:0881dee6:::obj412:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.502+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:080cd0cf:::obj146:head local-lis/les=113/114 n=27 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:0881dee6:::obj412:head 2026-02-11T23:31:27.502+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:080cd0cf:::obj146:head local-lis/les=113/114 n=27 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:0881dee6:::obj412:head v 111'412 2026-02-11T23:31:27.502+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:080cd0cf:::obj146:head local-lis/les=113/114 n=27 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.502+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.502+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:0881dee6:::obj412:head, version: 111'412, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0881dee6:::obj412:head@111'412, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:27.502+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:080cd0cf:::obj146:head local-lis/les=113/114 n=27 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.502+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.502+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.502+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0877fdce:::obj530:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.502+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:0877fdce:::obj530:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 1.0 e 116/116 lb 1:0877fdce:::obj530:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:080cd0cf:::obj146:head local-lis/les=113/114 n=27 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.502+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.502+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:080cd0cf:::obj146:head local-lis/les=113/114 n=27 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:0877fdce:::obj530:head) v3 2026-02-11T23:31:27.502+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:080cd0cf:::obj146:head local-lis/les=113/114 n=27 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:0877fdce:::obj530:head) 2026-02-11T23:31:27.502+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.502+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:0877fdce:::obj530:head) v3 finish latency 0.000014 2026-02-11T23:31:27.502+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.502+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.502+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0877fdce:::obj530:head local-lis/les=113/114 n=28 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.502+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.502+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.502+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.502+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e6b200 con 0x4acb000 2026-02-11T23:31:27.502+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:0881dee6:::obj412:head)]) -- 0x5e6af00 con 0x4acb000 2026-02-11T23:31:27.502+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.502+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 93 ==== MOSDPGPush(1.0 116/113 [PushOp(1:08a5234b:::obj62:head, version: 111'62, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:08a5234b:::obj62:head@111'62, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5db01a0 con 0x4acb000 2026-02-11T23:31:27.502+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:08a5234b:::obj62:head, version: 111'62, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:08a5234b:::obj62:head@111'62, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.502+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:08a5234b:::obj62:head, version: 111'62, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:08a5234b:::obj62:head@111'62, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.502+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.502+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 94 ==== pg_backfill(progress 1.0 e 116/116 lb 1:0881dee6:::obj412:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc3600 con 0x4acb000 2026-02-11T23:31:27.502+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:08a5234b:::obj62:head, version: 111'62, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:08a5234b:::obj62:head@111'62, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.502+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:0881dee6:::obj412:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.502+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0881dee6:::obj412:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.502+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.502+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:08a5234b:::obj62:head, version: 111'62, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:08a5234b:::obj62:head@111'62, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.502+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:08a5234b:::obj62:head, version: 111'62, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:08a5234b:::obj62:head@111'62, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 116/113 [PushOp(1:08a5234b:::obj62:head, version: 111'62, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:08a5234b:::obj62:head@111'62, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:0877fdce:::obj530:head local-lis/les=113/114 n=28 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.502+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.502+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.502+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0877fdce:::obj530:head local-lis/les=113/114 n=28 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:08a5234b:::obj62:head, version: 111'62, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:08a5234b:::obj62:head@111'62, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.502+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0881dee6:::obj412:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.502+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0877fdce:::obj530:head local-lis/les=113/114 n=28 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:08a5234b:::obj62:head@111'62, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.502+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0877fdce:::obj530:head local-lis/les=113/114 n=28 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:08a5234b:::obj62:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.502+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0877fdce:::obj530:head local-lis/les=113/114 n=28 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:08a5234b:::obj62:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.502+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0877fdce:::obj530:head local-lis/les=113/114 n=28 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:08a5234b:::obj62:head 2026-02-11T23:31:27.502+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0877fdce:::obj530:head local-lis/les=113/114 n=28 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:08a5234b:::obj62:head v 111'62 2026-02-11T23:31:27.502+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0877fdce:::obj530:head local-lis/les=113/114 n=28 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.502+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.502+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:08a5234b:::obj62:head, version: 111'62, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:08a5234b:::obj62:head@111'62, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:27.502+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0877fdce:::obj530:head local-lis/les=113/114 n=28 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.502+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.502+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.502+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0881dee6:::obj412:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.502+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:0881dee6:::obj412:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 1.0 e 116/116 lb 1:0881dee6:::obj412:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:0877fdce:::obj530:head local-lis/les=113/114 n=28 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.502+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.502+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0877fdce:::obj530:head local-lis/les=113/114 n=28 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:0881dee6:::obj412:head) v3 2026-02-11T23:31:27.502+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0877fdce:::obj530:head local-lis/les=113/114 n=28 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:0881dee6:::obj412:head) 2026-02-11T23:31:27.502+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.502+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:0881dee6:::obj412:head) v3 finish latency 0.000016 2026-02-11T23:31:27.502+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.502+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.502+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0881dee6:::obj412:head local-lis/les=113/114 n=29 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.502+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.502+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.502+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.502+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e6b500 con 0x4acb000 2026-02-11T23:31:27.502+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:08a5234b:::obj62:head)]) -- 0x5e6b380 con 0x4acb000 2026-02-11T23:31:27.502+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.503+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 95 ==== MOSDPGPush(1.0 116/113 [PushOp(1:08fb0692:::obj136:head, version: 111'136, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:08fb0692:::obj136:head@111'136, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db1860 con 0x4acb000 2026-02-11T23:31:27.503+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:08fb0692:::obj136:head, version: 111'136, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:08fb0692:::obj136:head@111'136, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.503+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:08fb0692:::obj136:head, version: 111'136, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:08fb0692:::obj136:head@111'136, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.503+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.503+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 96 ==== pg_backfill(progress 1.0 e 116/116 lb 1:08a5234b:::obj62:head) ==== 1061+0+0 (unknown 0 0 0) 0x5dc2000 con 0x4acb000 2026-02-11T23:31:27.503+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:08a5234b:::obj62:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 116 2026-02-11T23:31:27.503+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:08fb0692:::obj136:head, version: 111'136, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:08fb0692:::obj136:head@111'136, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.503+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:08a5234b:::obj62:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.503+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.503+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:08fb0692:::obj136:head, version: 111'136, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:08fb0692:::obj136:head@111'136, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.503+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.503+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:08a5234b:::obj62:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.503+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:08fb0692:::obj136:head, version: 111'136, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:08fb0692:::obj136:head@111'136, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 116/113 [PushOp(1:08fb0692:::obj136:head, version: 111'136, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:08fb0692:::obj136:head@111'136, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:0881dee6:::obj412:head local-lis/les=113/114 n=29 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.503+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.503+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0881dee6:::obj412:head local-lis/les=113/114 n=29 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:08fb0692:::obj136:head, version: 111'136, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:08fb0692:::obj136:head@111'136, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.503+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0881dee6:::obj412:head local-lis/les=113/114 n=29 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:08fb0692:::obj136:head@111'136, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.503+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0881dee6:::obj412:head local-lis/les=113/114 n=29 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:08fb0692:::obj136:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.503+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0881dee6:::obj412:head local-lis/les=113/114 n=29 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:08fb0692:::obj136:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.503+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0881dee6:::obj412:head local-lis/les=113/114 n=29 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:08fb0692:::obj136:head 2026-02-11T23:31:27.503+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0881dee6:::obj412:head local-lis/les=113/114 n=29 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:08fb0692:::obj136:head v 111'136 2026-02-11T23:31:27.503+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0881dee6:::obj412:head local-lis/les=113/114 n=29 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.503+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.503+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:08fb0692:::obj136:head, version: 111'136, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:08fb0692:::obj136:head@111'136, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:27.503+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0881dee6:::obj412:head local-lis/les=113/114 n=29 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.503+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.503+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:08a5234b:::obj62:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.503+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:08a5234b:::obj62:head) v3 prio 15 cost 0 latency 0.000090 pg_backfill(progress 1.0 e 116/116 lb 1:08a5234b:::obj62:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:0881dee6:::obj412:head local-lis/les=113/114 n=29 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.503+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.503+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.503+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0881dee6:::obj412:head local-lis/les=113/114 n=29 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:08a5234b:::obj62:head) v3 2026-02-11T23:31:27.503+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0881dee6:::obj412:head local-lis/les=113/114 n=29 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:08a5234b:::obj62:head) 2026-02-11T23:31:27.503+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.503+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:08a5234b:::obj62:head) v3 finish latency 0.000017 2026-02-11T23:31:27.503+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.503+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.503+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:08a5234b:::obj62:head local-lis/les=113/114 n=30 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.503+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.503+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.503+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.503+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e6b980 con 0x4acb000 2026-02-11T23:31:27.503+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:08fb0692:::obj136:head)]) -- 0x5e6b680 con 0x4acb000 2026-02-11T23:31:27.503+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.504+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 97 ==== MOSDPGPush(1.0 116/113 [PushOp(1:095c6b7a:::obj79:head, version: 111'79, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:095c6b7a:::obj79:head@111'79, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5db0d00 con 0x4acb000 2026-02-11T23:31:27.504+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:095c6b7a:::obj79:head, version: 111'79, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:095c6b7a:::obj79:head@111'79, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 116 2026-02-11T23:31:27.504+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:095c6b7a:::obj79:head, version: 111'79, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:095c6b7a:::obj79:head@111'79, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.504+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.504+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 98 ==== pg_backfill(progress 1.0 e 116/116 lb 1:08fb0692:::obj136:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc4c00 con 0x4acb000 2026-02-11T23:31:27.504+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:095c6b7a:::obj79:head, version: 111'79, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:095c6b7a:::obj79:head@111'79, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.504+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:08fb0692:::obj136:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 116 2026-02-11T23:31:27.504+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.504+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:08fb0692:::obj136:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.504+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:095c6b7a:::obj79:head, version: 111'79, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:095c6b7a:::obj79:head@111'79, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.504+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.504+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:095c6b7a:::obj79:head, version: 111'79, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:095c6b7a:::obj79:head@111'79, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:095c6b7a:::obj79:head, version: 111'79, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:095c6b7a:::obj79:head@111'79, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:08a5234b:::obj62:head local-lis/les=113/114 n=30 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.504+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.504+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:08a5234b:::obj62:head local-lis/les=113/114 n=30 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:095c6b7a:::obj79:head, version: 111'79, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:095c6b7a:::obj79:head@111'79, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.504+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.504+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:08fb0692:::obj136:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.504+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:08a5234b:::obj62:head local-lis/les=113/114 n=30 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:095c6b7a:::obj79:head@111'79, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.504+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:08a5234b:::obj62:head local-lis/les=113/114 n=30 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:095c6b7a:::obj79:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.504+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:08a5234b:::obj62:head local-lis/les=113/114 n=30 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:095c6b7a:::obj79:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.504+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:08a5234b:::obj62:head local-lis/les=113/114 n=30 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:095c6b7a:::obj79:head 2026-02-11T23:31:27.504+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:08a5234b:::obj62:head local-lis/les=113/114 n=30 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:095c6b7a:::obj79:head v 111'79 2026-02-11T23:31:27.504+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:08a5234b:::obj62:head local-lis/les=113/114 n=30 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.504+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.504+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:095c6b7a:::obj79:head, version: 111'79, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:095c6b7a:::obj79:head@111'79, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:27.504+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.504+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.504+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:08fb0692:::obj136:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.504+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:08fb0692:::obj136:head) v3 prio 15 cost 0 latency 0.000079 pg_backfill(progress 1.0 e 116/116 lb 1:08fb0692:::obj136:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:08a5234b:::obj62:head local-lis/les=113/114 n=30 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.504+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.504+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:08a5234b:::obj62:head local-lis/les=113/114 n=30 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:08fb0692:::obj136:head) v3 2026-02-11T23:31:27.504+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:08a5234b:::obj62:head local-lis/les=113/114 n=30 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:08fb0692:::obj136:head) 2026-02-11T23:31:27.504+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.504+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:08fb0692:::obj136:head) v3 finish latency 0.000019 2026-02-11T23:31:27.504+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:08fb0692:::obj136:head local-lis/les=113/114 n=31 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.504+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.504+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.504+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:08fb0692:::obj136:head local-lis/les=113/114 n=31 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.504+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.504+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.504+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.504+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e3b680 con 0x4acb000 2026-02-11T23:31:27.504+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:095c6b7a:::obj79:head)]) -- 0x5e6a180 con 0x4acb000 2026-02-11T23:31:27.504+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.504+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 99 ==== MOSDPGPush(1.0 116/113 [PushOp(1:097a0595:::obj169:head, version: 111'169, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:097a0595:::obj169:head@111'169, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e26000 con 0x4acb000 2026-02-11T23:31:27.504+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:097a0595:::obj169:head, version: 111'169, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:097a0595:::obj169:head@111'169, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 116 2026-02-11T23:31:27.504+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:097a0595:::obj169:head, version: 111'169, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:097a0595:::obj169:head@111'169, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.504+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.504+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 100 ==== pg_backfill(progress 1.0 e 116/116 lb 1:095c6b7a:::obj79:head) ==== 1061+0+0 (unknown 0 0 0) 0x5dc5180 con 0x4acb000 2026-02-11T23:31:27.504+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:095c6b7a:::obj79:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.504+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:097a0595:::obj169:head, version: 111'169, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:097a0595:::obj169:head@111'169, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.504+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:095c6b7a:::obj79:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.504+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.504+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:095c6b7a:::obj79:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.504+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.504+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:097a0595:::obj169:head, version: 111'169, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:097a0595:::obj169:head@111'169, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.504+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:097a0595:::obj169:head, version: 111'169, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:097a0595:::obj169:head@111'169, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(1.0 116/113 [PushOp(1:097a0595:::obj169:head, version: 111'169, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:097a0595:::obj169:head@111'169, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:08fb0692:::obj136:head local-lis/les=113/114 n=31 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.504+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.504+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:08fb0692:::obj136:head local-lis/les=113/114 n=31 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:097a0595:::obj169:head, version: 111'169, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:097a0595:::obj169:head@111'169, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.504+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:08fb0692:::obj136:head local-lis/les=113/114 n=31 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:097a0595:::obj169:head@111'169, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.504+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:08fb0692:::obj136:head local-lis/les=113/114 n=31 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:097a0595:::obj169:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.504+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:08fb0692:::obj136:head local-lis/les=113/114 n=31 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:097a0595:::obj169:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.504+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:08fb0692:::obj136:head local-lis/les=113/114 n=31 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:097a0595:::obj169:head 2026-02-11T23:31:27.504+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:08fb0692:::obj136:head local-lis/les=113/114 n=31 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:097a0595:::obj169:head v 111'169 2026-02-11T23:31:27.504+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:08fb0692:::obj136:head local-lis/les=113/114 n=31 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.504+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.504+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:097a0595:::obj169:head, version: 111'169, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:097a0595:::obj169:head@111'169, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:27.504+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:08fb0692:::obj136:head local-lis/les=113/114 n=31 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.504+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.504+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.504+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:095c6b7a:::obj79:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.504+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:095c6b7a:::obj79:head) v3 prio 15 cost 0 latency 0.000091 pg_backfill(progress 1.0 e 116/116 lb 1:095c6b7a:::obj79:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:08fb0692:::obj136:head local-lis/les=113/114 n=31 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.504+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.504+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:08fb0692:::obj136:head local-lis/les=113/114 n=31 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:095c6b7a:::obj79:head) v3 2026-02-11T23:31:27.504+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:08fb0692:::obj136:head local-lis/les=113/114 n=31 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:095c6b7a:::obj79:head) 2026-02-11T23:31:27.504+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.504+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:095c6b7a:::obj79:head) v3 finish latency 0.000018 2026-02-11T23:31:27.504+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.504+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.504+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:095c6b7a:::obj79:head local-lis/les=113/114 n=32 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.504+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.504+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.504+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.504+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e3bc80 con 0x4acb000 2026-02-11T23:31:27.504+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:097a0595:::obj169:head)]) -- 0x5e3bb00 con 0x4acb000 2026-02-11T23:31:27.504+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.505+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 101 ==== MOSDPGPush(1.0 116/113 [PushOp(1:09d0cce6:::obj481:head, version: 111'481, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:09d0cce6:::obj481:head@111'481, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3815d40 con 0x4acb000 2026-02-11T23:31:27.505+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:09d0cce6:::obj481:head, version: 111'481, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:09d0cce6:::obj481:head@111'481, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.505+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:09d0cce6:::obj481:head, version: 111'481, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:09d0cce6:::obj481:head@111'481, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.505+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.505+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 102 ==== pg_backfill(progress 1.0 e 116/116 lb 1:097a0595:::obj169:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e100 con 0x4acb000 2026-02-11T23:31:27.505+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:09d0cce6:::obj481:head, version: 111'481, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:09d0cce6:::obj481:head@111'481, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.505+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:097a0595:::obj169:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.505+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:097a0595:::obj169:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.505+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.505+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:09d0cce6:::obj481:head, version: 111'481, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:09d0cce6:::obj481:head@111'481, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.505+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:09d0cce6:::obj481:head, version: 111'481, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:09d0cce6:::obj481:head@111'481, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 116/113 [PushOp(1:09d0cce6:::obj481:head, version: 111'481, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:09d0cce6:::obj481:head@111'481, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:095c6b7a:::obj79:head local-lis/les=113/114 n=32 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.505+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.505+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.505+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:095c6b7a:::obj79:head local-lis/les=113/114 n=32 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:09d0cce6:::obj481:head, version: 111'481, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:09d0cce6:::obj481:head@111'481, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.505+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:097a0595:::obj169:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.505+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:095c6b7a:::obj79:head local-lis/les=113/114 n=32 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:09d0cce6:::obj481:head@111'481, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.505+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:095c6b7a:::obj79:head local-lis/les=113/114 n=32 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:09d0cce6:::obj481:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.505+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:095c6b7a:::obj79:head local-lis/les=113/114 n=32 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:09d0cce6:::obj481:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.505+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:095c6b7a:::obj79:head local-lis/les=113/114 n=32 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:09d0cce6:::obj481:head 2026-02-11T23:31:27.505+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:095c6b7a:::obj79:head local-lis/les=113/114 n=32 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:09d0cce6:::obj481:head v 111'481 2026-02-11T23:31:27.505+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:095c6b7a:::obj79:head local-lis/les=113/114 n=32 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.505+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.505+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:09d0cce6:::obj481:head, version: 111'481, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:09d0cce6:::obj481:head@111'481, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:27.505+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:095c6b7a:::obj79:head local-lis/les=113/114 n=32 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.505+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.505+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.505+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:097a0595:::obj169:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.505+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:097a0595:::obj169:head) v3 prio 15 cost 0 latency 0.000079 pg_backfill(progress 1.0 e 116/116 lb 1:097a0595:::obj169:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:095c6b7a:::obj79:head local-lis/les=113/114 n=32 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.505+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.505+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:095c6b7a:::obj79:head local-lis/les=113/114 n=32 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:097a0595:::obj169:head) v3 2026-02-11T23:31:27.505+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:095c6b7a:::obj79:head local-lis/les=113/114 n=32 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:097a0595:::obj169:head) 2026-02-11T23:31:27.505+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.505+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:097a0595:::obj169:head) v3 finish latency 0.000015 2026-02-11T23:31:27.505+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.505+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.505+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:097a0595:::obj169:head local-lis/les=113/114 n=33 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.505+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.505+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.505+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.505+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x4ecc480 con 0x4acb000 2026-02-11T23:31:27.505+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:09d0cce6:::obj481:head)]) -- 0x5e3a180 con 0x4acb000 2026-02-11T23:31:27.505+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.505+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 103 ==== MOSDPGPush(1.0 116/113 [PushOp(1:09eb3833:::obj488:head, version: 111'488, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:09eb3833:::obj488:head@111'488, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db0340 con 0x4acb000 2026-02-11T23:31:27.505+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:09eb3833:::obj488:head, version: 111'488, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:09eb3833:::obj488:head@111'488, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.505+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:09eb3833:::obj488:head, version: 111'488, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:09eb3833:::obj488:head@111'488, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.505+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.505+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 104 ==== pg_backfill(progress 1.0 e 116/116 lb 1:09d0cce6:::obj481:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2d080 con 0x4acb000 2026-02-11T23:31:27.505+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:09d0cce6:::obj481:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.505+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:09eb3833:::obj488:head, version: 111'488, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:09eb3833:::obj488:head@111'488, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.505+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:09d0cce6:::obj481:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.505+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.505+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:09eb3833:::obj488:head, version: 111'488, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:09eb3833:::obj488:head@111'488, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.505+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:09eb3833:::obj488:head, version: 111'488, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:09eb3833:::obj488:head@111'488, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(1.0 116/113 [PushOp(1:09eb3833:::obj488:head, version: 111'488, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:09eb3833:::obj488:head@111'488, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:097a0595:::obj169:head local-lis/les=113/114 n=33 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.505+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.505+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.505+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:097a0595:::obj169:head local-lis/les=113/114 n=33 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:09eb3833:::obj488:head, version: 111'488, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:09eb3833:::obj488:head@111'488, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.505+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:09d0cce6:::obj481:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.505+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:097a0595:::obj169:head local-lis/les=113/114 n=33 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:09eb3833:::obj488:head@111'488, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.505+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:097a0595:::obj169:head local-lis/les=113/114 n=33 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:09eb3833:::obj488:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.505+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:097a0595:::obj169:head local-lis/les=113/114 n=33 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:09eb3833:::obj488:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.505+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:097a0595:::obj169:head local-lis/les=113/114 n=33 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:09eb3833:::obj488:head 2026-02-11T23:31:27.505+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:097a0595:::obj169:head local-lis/les=113/114 n=33 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:09eb3833:::obj488:head v 111'488 2026-02-11T23:31:27.505+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:097a0595:::obj169:head local-lis/les=113/114 n=33 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.505+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.505+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:09eb3833:::obj488:head, version: 111'488, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:09eb3833:::obj488:head@111'488, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:27.505+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:097a0595:::obj169:head local-lis/les=113/114 n=33 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.505+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.505+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.505+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:09d0cce6:::obj481:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.505+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:09d0cce6:::obj481:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 1.0 e 116/116 lb 1:09d0cce6:::obj481:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:097a0595:::obj169:head local-lis/les=113/114 n=33 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.505+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.505+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:097a0595:::obj169:head local-lis/les=113/114 n=33 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:09d0cce6:::obj481:head) v3 2026-02-11T23:31:27.505+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:097a0595:::obj169:head local-lis/les=113/114 n=33 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:09d0cce6:::obj481:head) 2026-02-11T23:31:27.506+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.506+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:09d0cce6:::obj481:head) v3 finish latency 0.000018 2026-02-11T23:31:27.506+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.506+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.506+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:09d0cce6:::obj481:head local-lis/les=113/114 n=34 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.506+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.506+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.506+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.506+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x4ec2480 con 0x4acb000 2026-02-11T23:31:27.506+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:09eb3833:::obj488:head)]) -- 0x4ecc600 con 0x4acb000 2026-02-11T23:31:27.506+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.506+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 105 ==== MOSDPGPush(1.0 116/113 [PushOp(1:0a4fcbe7:::obj507:head, version: 111'507, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0a4fcbe7:::obj507:head@111'507, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db1a00 con 0x4acb000 2026-02-11T23:31:27.506+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:0a4fcbe7:::obj507:head, version: 111'507, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0a4fcbe7:::obj507:head@111'507, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.506+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0a4fcbe7:::obj507:head, version: 111'507, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0a4fcbe7:::obj507:head@111'507, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.506+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.506+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 106 ==== pg_backfill(progress 1.0 e 116/116 lb 1:09eb3833:::obj488:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc3080 con 0x4acb000 2026-02-11T23:31:27.506+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:09eb3833:::obj488:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.506+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0a4fcbe7:::obj507:head, version: 111'507, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0a4fcbe7:::obj507:head@111'507, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.506+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:09eb3833:::obj488:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.506+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.506+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0a4fcbe7:::obj507:head, version: 111'507, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0a4fcbe7:::obj507:head@111'507, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.506+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.506+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:09eb3833:::obj488:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.506+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:0a4fcbe7:::obj507:head, version: 111'507, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0a4fcbe7:::obj507:head@111'507, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 116/113 [PushOp(1:0a4fcbe7:::obj507:head, version: 111'507, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0a4fcbe7:::obj507:head@111'507, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:09d0cce6:::obj481:head local-lis/les=113/114 n=34 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.506+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.506+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:09d0cce6:::obj481:head local-lis/les=113/114 n=34 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:0a4fcbe7:::obj507:head, version: 111'507, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0a4fcbe7:::obj507:head@111'507, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.506+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:09d0cce6:::obj481:head local-lis/les=113/114 n=34 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:0a4fcbe7:::obj507:head@111'507, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.506+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:09d0cce6:::obj481:head local-lis/les=113/114 n=34 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:0a4fcbe7:::obj507:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.506+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:09d0cce6:::obj481:head local-lis/les=113/114 n=34 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:0a4fcbe7:::obj507:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.506+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:09d0cce6:::obj481:head local-lis/les=113/114 n=34 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:0a4fcbe7:::obj507:head 2026-02-11T23:31:27.506+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:09d0cce6:::obj481:head local-lis/les=113/114 n=34 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:0a4fcbe7:::obj507:head v 111'507 2026-02-11T23:31:27.506+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:09d0cce6:::obj481:head local-lis/les=113/114 n=34 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.506+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.506+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:0a4fcbe7:::obj507:head, version: 111'507, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0a4fcbe7:::obj507:head@111'507, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:27.506+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.506+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.506+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:09eb3833:::obj488:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.506+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:09eb3833:::obj488:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 1.0 e 116/116 lb 1:09eb3833:::obj488:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:09d0cce6:::obj481:head local-lis/les=113/114 n=34 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.506+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.506+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:09d0cce6:::obj481:head local-lis/les=113/114 n=34 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:09eb3833:::obj488:head) v3 2026-02-11T23:31:27.506+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:09d0cce6:::obj481:head local-lis/les=113/114 n=34 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:09eb3833:::obj488:head) 2026-02-11T23:31:27.506+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.506+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:09eb3833:::obj488:head) v3 finish latency 0.000023 2026-02-11T23:31:27.506+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:09eb3833:::obj488:head local-lis/les=113/114 n=35 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.506+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.506+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.506+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:09eb3833:::obj488:head local-lis/les=113/114 n=35 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.506+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.506+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.506+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.506+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x4ec3680 con 0x4acb000 2026-02-11T23:31:27.506+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:0a4fcbe7:::obj507:head)]) -- 0x5e6a300 con 0x4acb000 2026-02-11T23:31:27.506+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.507+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 107 ==== MOSDPGPush(1.0 116/113 [PushOp(1:0b3f05c8:::obj589:head, version: 111'589, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0b3f05c8:::obj589:head@111'589, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e26340 con 0x4acb000 2026-02-11T23:31:27.507+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:0b3f05c8:::obj589:head, version: 111'589, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0b3f05c8:::obj589:head@111'589, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.507+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0b3f05c8:::obj589:head, version: 111'589, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0b3f05c8:::obj589:head@111'589, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.507+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.507+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 108 ==== pg_backfill(progress 1.0 e 116/116 lb 1:0a4fcbe7:::obj507:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc3600 con 0x4acb000 2026-02-11T23:31:27.507+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0b3f05c8:::obj589:head, version: 111'589, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0b3f05c8:::obj589:head@111'589, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.507+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:0a4fcbe7:::obj507:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.507+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0a4fcbe7:::obj507:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.507+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.507+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0b3f05c8:::obj589:head, version: 111'589, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0b3f05c8:::obj589:head@111'589, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.507+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:0b3f05c8:::obj589:head, version: 111'589, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0b3f05c8:::obj589:head@111'589, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(1.0 116/113 [PushOp(1:0b3f05c8:::obj589:head, version: 111'589, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0b3f05c8:::obj589:head@111'589, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:09eb3833:::obj488:head local-lis/les=113/114 n=35 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.507+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.507+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:09eb3833:::obj488:head local-lis/les=113/114 n=35 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:0b3f05c8:::obj589:head, version: 111'589, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0b3f05c8:::obj589:head@111'589, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.507+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.507+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:09eb3833:::obj488:head local-lis/les=113/114 n=35 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:0b3f05c8:::obj589:head@111'589, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.507+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0a4fcbe7:::obj507:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.507+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:09eb3833:::obj488:head local-lis/les=113/114 n=35 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:0b3f05c8:::obj589:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.507+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:09eb3833:::obj488:head local-lis/les=113/114 n=35 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:0b3f05c8:::obj589:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.507+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:09eb3833:::obj488:head local-lis/les=113/114 n=35 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:0b3f05c8:::obj589:head 2026-02-11T23:31:27.507+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:09eb3833:::obj488:head local-lis/les=113/114 n=35 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:0b3f05c8:::obj589:head v 111'589 2026-02-11T23:31:27.507+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:09eb3833:::obj488:head local-lis/les=113/114 n=35 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.507+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.507+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:0b3f05c8:::obj589:head, version: 111'589, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0b3f05c8:::obj589:head@111'589, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:27.507+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:09eb3833:::obj488:head local-lis/les=113/114 n=35 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.507+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.507+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.507+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0a4fcbe7:::obj507:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.507+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:0a4fcbe7:::obj507:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 1.0 e 116/116 lb 1:0a4fcbe7:::obj507:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:09eb3833:::obj488:head local-lis/les=113/114 n=35 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.507+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.507+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:09eb3833:::obj488:head local-lis/les=113/114 n=35 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:0a4fcbe7:::obj507:head) v3 2026-02-11T23:31:27.507+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:09eb3833:::obj488:head local-lis/les=113/114 n=35 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:0a4fcbe7:::obj507:head) 2026-02-11T23:31:27.507+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.507+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:0a4fcbe7:::obj507:head) v3 finish latency 0.000014 2026-02-11T23:31:27.507+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.507+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.507+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0a4fcbe7:::obj507:head local-lis/les=113/114 n=36 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.507+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.507+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.507+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.507+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e6a900 con 0x4acb000 2026-02-11T23:31:27.507+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:0b3f05c8:::obj589:head)]) -- 0x4ec2780 con 0x4acb000 2026-02-11T23:31:27.507+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.507+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 109 ==== MOSDPGPush(1.0 116/113 [PushOp(1:0bdade2e:::obj246:head, version: 111'246, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0bdade2e:::obj246:head@111'246, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e271e0 con 0x4acb000 2026-02-11T23:31:27.507+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:0bdade2e:::obj246:head, version: 111'246, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0bdade2e:::obj246:head@111'246, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.507+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0bdade2e:::obj246:head, version: 111'246, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0bdade2e:::obj246:head@111'246, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.507+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.507+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 110 ==== pg_backfill(progress 1.0 e 116/116 lb 1:0b3f05c8:::obj589:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc4c00 con 0x4acb000 2026-02-11T23:31:27.507+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0bdade2e:::obj246:head, version: 111'246, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0bdade2e:::obj246:head@111'246, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.507+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:0b3f05c8:::obj589:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.507+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0b3f05c8:::obj589:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.507+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.507+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0bdade2e:::obj246:head, version: 111'246, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0bdade2e:::obj246:head@111'246, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.507+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:0bdade2e:::obj246:head, version: 111'246, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0bdade2e:::obj246:head@111'246, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 116/113 [PushOp(1:0bdade2e:::obj246:head, version: 111'246, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0bdade2e:::obj246:head@111'246, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:0a4fcbe7:::obj507:head local-lis/les=113/114 n=36 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.507+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.507+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.507+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0b3f05c8:::obj589:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.507+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0a4fcbe7:::obj507:head local-lis/les=113/114 n=36 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:0bdade2e:::obj246:head, version: 111'246, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0bdade2e:::obj246:head@111'246, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.507+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0a4fcbe7:::obj507:head local-lis/les=113/114 n=36 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:0bdade2e:::obj246:head@111'246, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.507+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0a4fcbe7:::obj507:head local-lis/les=113/114 n=36 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:0bdade2e:::obj246:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.507+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0a4fcbe7:::obj507:head local-lis/les=113/114 n=36 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:0bdade2e:::obj246:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.507+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0a4fcbe7:::obj507:head local-lis/les=113/114 n=36 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:0bdade2e:::obj246:head 2026-02-11T23:31:27.507+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0a4fcbe7:::obj507:head local-lis/les=113/114 n=36 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:0bdade2e:::obj246:head v 111'246 2026-02-11T23:31:27.507+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0a4fcbe7:::obj507:head local-lis/les=113/114 n=36 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.507+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.507+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:0bdade2e:::obj246:head, version: 111'246, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0bdade2e:::obj246:head@111'246, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:27.507+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0a4fcbe7:::obj507:head local-lis/les=113/114 n=36 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.507+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.507+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0b3f05c8:::obj589:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.507+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:0b3f05c8:::obj589:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 1.0 e 116/116 lb 1:0b3f05c8:::obj589:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:0a4fcbe7:::obj507:head local-lis/les=113/114 n=36 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.507+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.507+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.507+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0a4fcbe7:::obj507:head local-lis/les=113/114 n=36 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:0b3f05c8:::obj589:head) v3 2026-02-11T23:31:27.507+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0a4fcbe7:::obj507:head local-lis/les=113/114 n=36 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:0b3f05c8:::obj589:head) 2026-02-11T23:31:27.507+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.507+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:0b3f05c8:::obj589:head) v3 finish latency 0.000016 2026-02-11T23:31:27.507+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.508+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0b3f05c8:::obj589:head local-lis/les=113/114 n=37 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.508+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.508+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.508+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.508+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.508+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e6bc80 con 0x4acb000 2026-02-11T23:31:27.508+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:0bdade2e:::obj246:head)]) -- 0x5e6ad80 con 0x4acb000 2026-02-11T23:31:27.508+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.508+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 111 ==== MOSDPGPush(1.0 116/113 [PushOp(1:0bde2796:::obj413:head, version: 111'413, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0bde2796:::obj413:head@111'413, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db0d00 con 0x4acb000 2026-02-11T23:31:27.508+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:0bde2796:::obj413:head, version: 111'413, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0bde2796:::obj413:head@111'413, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.508+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0bde2796:::obj413:head, version: 111'413, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0bde2796:::obj413:head@111'413, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.508+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.508+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 112 ==== pg_backfill(progress 1.0 e 116/116 lb 1:0bdade2e:::obj246:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc2000 con 0x4acb000 2026-02-11T23:31:27.508+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0bde2796:::obj413:head, version: 111'413, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0bde2796:::obj413:head@111'413, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.508+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:0bdade2e:::obj246:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 116 2026-02-11T23:31:27.508+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0bdade2e:::obj246:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.508+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.508+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0bde2796:::obj413:head, version: 111'413, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0bde2796:::obj413:head@111'413, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.508+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:0bde2796:::obj413:head, version: 111'413, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0bde2796:::obj413:head@111'413, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 116/113 [PushOp(1:0bde2796:::obj413:head, version: 111'413, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0bde2796:::obj413:head@111'413, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:0b3f05c8:::obj589:head local-lis/les=113/114 n=37 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.508+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.508+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.508+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0b3f05c8:::obj589:head local-lis/les=113/114 n=37 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:0bde2796:::obj413:head, version: 111'413, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0bde2796:::obj413:head@111'413, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.508+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0bdade2e:::obj246:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.508+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0b3f05c8:::obj589:head local-lis/les=113/114 n=37 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:0bde2796:::obj413:head@111'413, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.508+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0b3f05c8:::obj589:head local-lis/les=113/114 n=37 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:0bde2796:::obj413:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.508+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0b3f05c8:::obj589:head local-lis/les=113/114 n=37 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:0bde2796:::obj413:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.508+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0b3f05c8:::obj589:head local-lis/les=113/114 n=37 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:0bde2796:::obj413:head 2026-02-11T23:31:27.508+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0b3f05c8:::obj589:head local-lis/les=113/114 n=37 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:0bde2796:::obj413:head v 111'413 2026-02-11T23:31:27.508+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0b3f05c8:::obj589:head local-lis/les=113/114 n=37 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.508+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.508+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:0bde2796:::obj413:head, version: 111'413, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0bde2796:::obj413:head@111'413, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:27.508+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.508+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.508+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0bdade2e:::obj246:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.508+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:0bdade2e:::obj246:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 1.0 e 116/116 lb 1:0bdade2e:::obj246:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:0b3f05c8:::obj589:head local-lis/les=113/114 n=37 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.508+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.508+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0b3f05c8:::obj589:head local-lis/les=113/114 n=37 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:0bdade2e:::obj246:head) v3 2026-02-11T23:31:27.508+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0b3f05c8:::obj589:head local-lis/les=113/114 n=37 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:0bdade2e:::obj246:head) 2026-02-11T23:31:27.508+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.508+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:0bdade2e:::obj246:head) v3 finish latency 0.000019 2026-02-11T23:31:27.508+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0bdade2e:::obj246:head local-lis/les=113/114 n=38 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.508+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.508+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.508+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0bdade2e:::obj246:head local-lis/les=113/114 n=38 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.508+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.508+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.508+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.508+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e6be00 con 0x4acb000 2026-02-11T23:31:27.508+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:0bde2796:::obj413:head)]) -- 0x5e6a600 con 0x4acb000 2026-02-11T23:31:27.508+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.509+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 113 ==== MOSDPGPush(1.0 116/113 [PushOp(1:0c03281d:::obj596:head, version: 111'596, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0c03281d:::obj596:head@111'596, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db01a0 con 0x4acb000 2026-02-11T23:31:27.509+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:0c03281d:::obj596:head, version: 111'596, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0c03281d:::obj596:head@111'596, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.509+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0c03281d:::obj596:head, version: 111'596, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0c03281d:::obj596:head@111'596, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.509+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.509+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 114 ==== pg_backfill(progress 1.0 e 116/116 lb 1:0bde2796:::obj413:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc3b80 con 0x4acb000 2026-02-11T23:31:27.509+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0c03281d:::obj596:head, version: 111'596, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0c03281d:::obj596:head@111'596, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.509+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:0bde2796:::obj413:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 116 2026-02-11T23:31:27.509+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0bde2796:::obj413:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.509+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.509+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0c03281d:::obj596:head, version: 111'596, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0c03281d:::obj596:head@111'596, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.509+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:0c03281d:::obj596:head, version: 111'596, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0c03281d:::obj596:head@111'596, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:0c03281d:::obj596:head, version: 111'596, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0c03281d:::obj596:head@111'596, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:0bdade2e:::obj246:head local-lis/les=113/114 n=38 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.509+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.509+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.509+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0bdade2e:::obj246:head local-lis/les=113/114 n=38 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:0c03281d:::obj596:head, version: 111'596, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0c03281d:::obj596:head@111'596, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.509+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0bde2796:::obj413:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.509+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0bdade2e:::obj246:head local-lis/les=113/114 n=38 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:0c03281d:::obj596:head@111'596, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.509+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0bdade2e:::obj246:head local-lis/les=113/114 n=38 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:0c03281d:::obj596:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.509+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0bdade2e:::obj246:head local-lis/les=113/114 n=38 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:0c03281d:::obj596:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.509+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0bdade2e:::obj246:head local-lis/les=113/114 n=38 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:0c03281d:::obj596:head 2026-02-11T23:31:27.509+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0bdade2e:::obj246:head local-lis/les=113/114 n=38 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:0c03281d:::obj596:head v 111'596 2026-02-11T23:31:27.509+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0bdade2e:::obj246:head local-lis/les=113/114 n=38 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.509+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.509+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:0c03281d:::obj596:head, version: 111'596, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0c03281d:::obj596:head@111'596, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:27.509+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0bdade2e:::obj246:head local-lis/les=113/114 n=38 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.509+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.509+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.509+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0bde2796:::obj413:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.509+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:0bde2796:::obj413:head) v3 prio 15 cost 0 latency 0.000079 pg_backfill(progress 1.0 e 116/116 lb 1:0bde2796:::obj413:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:0bdade2e:::obj246:head local-lis/les=113/114 n=38 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.509+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.509+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0bdade2e:::obj246:head local-lis/les=113/114 n=38 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:0bde2796:::obj413:head) v3 2026-02-11T23:31:27.509+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0bdade2e:::obj246:head local-lis/les=113/114 n=38 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:0bde2796:::obj413:head) 2026-02-11T23:31:27.509+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.509+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:0bde2796:::obj413:head) v3 finish latency 0.000020 2026-02-11T23:31:27.509+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.509+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.509+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0bde2796:::obj413:head local-lis/les=113/114 n=39 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.509+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.509+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.509+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.509+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e3a480 con 0x4acb000 2026-02-11T23:31:27.509+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:0c03281d:::obj596:head)]) -- 0x5e3ad80 con 0x4acb000 2026-02-11T23:31:27.509+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.509+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 115 ==== MOSDPGPush(1.0 116/113 [PushOp(1:0cfeb9e4:::obj357:head, version: 111'357, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0cfeb9e4:::obj357:head@111'357, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3815d40 con 0x4acb000 2026-02-11T23:31:27.509+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:0cfeb9e4:::obj357:head, version: 111'357, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0cfeb9e4:::obj357:head@111'357, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.509+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0cfeb9e4:::obj357:head, version: 111'357, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0cfeb9e4:::obj357:head@111'357, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.509+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.509+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0cfeb9e4:::obj357:head, version: 111'357, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0cfeb9e4:::obj357:head@111'357, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.509+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 116 ==== pg_backfill(progress 1.0 e 116/116 lb 1:0c03281d:::obj596:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc4100 con 0x4acb000 2026-02-11T23:31:27.509+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:0c03281d:::obj596:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 116 2026-02-11T23:31:27.509+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.509+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0c03281d:::obj596:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.509+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0cfeb9e4:::obj357:head, version: 111'357, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0cfeb9e4:::obj357:head@111'357, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.509+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.509+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:0cfeb9e4:::obj357:head, version: 111'357, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0cfeb9e4:::obj357:head@111'357, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:0cfeb9e4:::obj357:head, version: 111'357, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0cfeb9e4:::obj357:head@111'357, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:0bde2796:::obj413:head local-lis/les=113/114 n=39 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.509+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.509+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0bde2796:::obj413:head local-lis/les=113/114 n=39 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:0cfeb9e4:::obj357:head, version: 111'357, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0cfeb9e4:::obj357:head@111'357, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.509+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0bde2796:::obj413:head local-lis/les=113/114 n=39 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:0cfeb9e4:::obj357:head@111'357, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.509+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.509+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0c03281d:::obj596:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.509+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0bde2796:::obj413:head local-lis/les=113/114 n=39 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:0cfeb9e4:::obj357:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.509+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0bde2796:::obj413:head local-lis/les=113/114 n=39 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:0cfeb9e4:::obj357:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.509+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0bde2796:::obj413:head local-lis/les=113/114 n=39 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:0cfeb9e4:::obj357:head 2026-02-11T23:31:27.509+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0bde2796:::obj413:head local-lis/les=113/114 n=39 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:0cfeb9e4:::obj357:head v 111'357 2026-02-11T23:31:27.509+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0bde2796:::obj413:head local-lis/les=113/114 n=39 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.509+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.510+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:0cfeb9e4:::obj357:head, version: 111'357, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0cfeb9e4:::obj357:head@111'357, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000062 2026-02-11T23:31:27.510+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0bde2796:::obj413:head local-lis/les=113/114 n=39 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.510+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.510+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0c03281d:::obj596:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.510+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:0c03281d:::obj596:head) v3 prio 15 cost 0 latency 0.000084 pg_backfill(progress 1.0 e 116/116 lb 1:0c03281d:::obj596:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:0bde2796:::obj413:head local-lis/les=113/114 n=39 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.510+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.510+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.510+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0bde2796:::obj413:head local-lis/les=113/114 n=39 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:0c03281d:::obj596:head) v3 2026-02-11T23:31:27.510+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0bde2796:::obj413:head local-lis/les=113/114 n=39 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:0c03281d:::obj596:head) 2026-02-11T23:31:27.510+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.510+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:0c03281d:::obj596:head) v3 finish latency 0.000015 2026-02-11T23:31:27.510+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.510+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.510+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0c03281d:::obj596:head local-lis/les=113/114 n=40 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.510+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.510+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.510+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.510+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e6aa80 con 0x4acb000 2026-02-11T23:31:27.510+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:0cfeb9e4:::obj357:head)]) -- 0x5e3b200 con 0x4acb000 2026-02-11T23:31:27.510+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.510+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 117 ==== MOSDPGPush(1.0 116/113 [PushOp(1:0dc4a75f:::obj282:head, version: 111'282, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0dc4a75f:::obj282:head@111'282, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e26000 con 0x4acb000 2026-02-11T23:31:27.510+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:0dc4a75f:::obj282:head, version: 111'282, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0dc4a75f:::obj282:head@111'282, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.510+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0dc4a75f:::obj282:head, version: 111'282, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0dc4a75f:::obj282:head@111'282, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.510+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.510+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 118 ==== pg_backfill(progress 1.0 e 116/116 lb 1:0cfeb9e4:::obj357:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc2580 con 0x4acb000 2026-02-11T23:31:27.510+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0dc4a75f:::obj282:head, version: 111'282, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0dc4a75f:::obj282:head@111'282, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.510+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:0cfeb9e4:::obj357:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 116 2026-02-11T23:31:27.510+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.510+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0cfeb9e4:::obj357:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.510+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0dc4a75f:::obj282:head, version: 111'282, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0dc4a75f:::obj282:head@111'282, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.510+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.510+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:0dc4a75f:::obj282:head, version: 111'282, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0dc4a75f:::obj282:head@111'282, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 116/113 [PushOp(1:0dc4a75f:::obj282:head, version: 111'282, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0dc4a75f:::obj282:head@111'282, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:0c03281d:::obj596:head local-lis/les=113/114 n=40 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.510+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.510+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0c03281d:::obj596:head local-lis/les=113/114 n=40 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:0dc4a75f:::obj282:head, version: 111'282, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0dc4a75f:::obj282:head@111'282, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.510+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.510+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0cfeb9e4:::obj357:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.510+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0c03281d:::obj596:head local-lis/les=113/114 n=40 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:0dc4a75f:::obj282:head@111'282, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.510+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0c03281d:::obj596:head local-lis/les=113/114 n=40 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:0dc4a75f:::obj282:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.510+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0c03281d:::obj596:head local-lis/les=113/114 n=40 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:0dc4a75f:::obj282:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.510+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0c03281d:::obj596:head local-lis/les=113/114 n=40 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:0dc4a75f:::obj282:head 2026-02-11T23:31:27.510+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0c03281d:::obj596:head local-lis/les=113/114 n=40 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:0dc4a75f:::obj282:head v 111'282 2026-02-11T23:31:27.510+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0c03281d:::obj596:head local-lis/les=113/114 n=40 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.510+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.510+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:0dc4a75f:::obj282:head, version: 111'282, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0dc4a75f:::obj282:head@111'282, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:27.510+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.510+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.510+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0cfeb9e4:::obj357:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.510+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:0cfeb9e4:::obj357:head) v3 prio 15 cost 0 latency 0.000084 pg_backfill(progress 1.0 e 116/116 lb 1:0cfeb9e4:::obj357:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:0c03281d:::obj596:head local-lis/les=113/114 n=40 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.510+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.510+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0c03281d:::obj596:head local-lis/les=113/114 n=40 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:0cfeb9e4:::obj357:head) v3 2026-02-11T23:31:27.510+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0c03281d:::obj596:head local-lis/les=113/114 n=40 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:0cfeb9e4:::obj357:head) 2026-02-11T23:31:27.510+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.510+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:0cfeb9e4:::obj357:head) v3 finish latency 0.000018 2026-02-11T23:31:27.510+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0cfeb9e4:::obj357:head local-lis/les=113/114 n=41 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.510+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.510+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.510+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0cfeb9e4:::obj357:head local-lis/les=113/114 n=41 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.510+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.510+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.510+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.510+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e6b200 con 0x4acb000 2026-02-11T23:31:27.510+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:0dc4a75f:::obj282:head)]) -- 0x5e6b680 con 0x4acb000 2026-02-11T23:31:27.510+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.511+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 119 ==== MOSDPGPush(1.0 116/113 [PushOp(1:0e39285d:::obj525:head, version: 111'525, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e39285d:::obj525:head@111'525, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db1a00 con 0x4acb000 2026-02-11T23:31:27.511+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:0e39285d:::obj525:head, version: 111'525, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e39285d:::obj525:head@111'525, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.511+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0e39285d:::obj525:head, version: 111'525, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e39285d:::obj525:head@111'525, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.511+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.511+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 120 ==== pg_backfill(progress 1.0 e 116/116 lb 1:0dc4a75f:::obj282:head) ==== 1062+0+0 (unknown 0 0 0) 0x4a2cb00 con 0x4acb000 2026-02-11T23:31:27.511+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0e39285d:::obj525:head, version: 111'525, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e39285d:::obj525:head@111'525, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.511+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:0dc4a75f:::obj282:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.511+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0dc4a75f:::obj282:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.511+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.511+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0e39285d:::obj525:head, version: 111'525, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e39285d:::obj525:head@111'525, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.511+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:0e39285d:::obj525:head, version: 111'525, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e39285d:::obj525:head@111'525, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(1.0 116/113 [PushOp(1:0e39285d:::obj525:head, version: 111'525, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e39285d:::obj525:head@111'525, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:0cfeb9e4:::obj357:head local-lis/les=113/114 n=41 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.511+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.511+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.511+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0cfeb9e4:::obj357:head local-lis/les=113/114 n=41 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:0e39285d:::obj525:head, version: 111'525, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e39285d:::obj525:head@111'525, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.511+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0dc4a75f:::obj282:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.511+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0cfeb9e4:::obj357:head local-lis/les=113/114 n=41 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:0e39285d:::obj525:head@111'525, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.511+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0cfeb9e4:::obj357:head local-lis/les=113/114 n=41 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:0e39285d:::obj525:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.511+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0cfeb9e4:::obj357:head local-lis/les=113/114 n=41 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:0e39285d:::obj525:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.511+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0cfeb9e4:::obj357:head local-lis/les=113/114 n=41 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:0e39285d:::obj525:head 2026-02-11T23:31:27.511+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0cfeb9e4:::obj357:head local-lis/les=113/114 n=41 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:0e39285d:::obj525:head v 111'525 2026-02-11T23:31:27.511+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0cfeb9e4:::obj357:head local-lis/les=113/114 n=41 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.511+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.511+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:0e39285d:::obj525:head, version: 111'525, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e39285d:::obj525:head@111'525, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:27.511+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0cfeb9e4:::obj357:head local-lis/les=113/114 n=41 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.511+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.511+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.511+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0dc4a75f:::obj282:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.511+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:0dc4a75f:::obj282:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 1.0 e 116/116 lb 1:0dc4a75f:::obj282:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:0cfeb9e4:::obj357:head local-lis/les=113/114 n=41 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.511+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.511+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0cfeb9e4:::obj357:head local-lis/les=113/114 n=41 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:0dc4a75f:::obj282:head) v3 2026-02-11T23:31:27.511+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0cfeb9e4:::obj357:head local-lis/les=113/114 n=41 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:0dc4a75f:::obj282:head) 2026-02-11T23:31:27.511+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.511+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:0dc4a75f:::obj282:head) v3 finish latency 0.000018 2026-02-11T23:31:27.511+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.511+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.511+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0dc4a75f:::obj282:head local-lis/les=113/114 n=42 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.511+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.511+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.511+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.511+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e6b380 con 0x4acb000 2026-02-11T23:31:27.511+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:0e39285d:::obj525:head)]) -- 0x5e6af00 con 0x4acb000 2026-02-11T23:31:27.511+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.511+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 121 ==== MOSDPGPush(1.0 116/113 [PushOp(1:0e3dc695:::obj78:head, version: 111'78, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e3dc695:::obj78:head@111'78, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5db0d00 con 0x4acb000 2026-02-11T23:31:27.511+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:0e3dc695:::obj78:head, version: 111'78, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e3dc695:::obj78:head@111'78, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.511+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0e3dc695:::obj78:head, version: 111'78, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e3dc695:::obj78:head@111'78, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.511+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.511+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 122 ==== pg_backfill(progress 1.0 e 116/116 lb 1:0e39285d:::obj525:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e680 con 0x4acb000 2026-02-11T23:31:27.511+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:0e39285d:::obj525:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.511+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0e3dc695:::obj78:head, version: 111'78, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e3dc695:::obj78:head@111'78, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.511+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0e39285d:::obj525:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.511+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.511+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0e3dc695:::obj78:head, version: 111'78, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e3dc695:::obj78:head@111'78, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.511+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.511+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0e39285d:::obj525:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.511+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:0e3dc695:::obj78:head, version: 111'78, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e3dc695:::obj78:head@111'78, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:0e3dc695:::obj78:head, version: 111'78, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e3dc695:::obj78:head@111'78, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:0dc4a75f:::obj282:head local-lis/les=113/114 n=42 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.511+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.511+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0dc4a75f:::obj282:head local-lis/les=113/114 n=42 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:0e3dc695:::obj78:head, version: 111'78, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e3dc695:::obj78:head@111'78, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.511+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0dc4a75f:::obj282:head local-lis/les=113/114 n=42 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:0e3dc695:::obj78:head@111'78, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.511+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0dc4a75f:::obj282:head local-lis/les=113/114 n=42 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:0e3dc695:::obj78:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.511+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0dc4a75f:::obj282:head local-lis/les=113/114 n=42 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:0e3dc695:::obj78:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.511+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0dc4a75f:::obj282:head local-lis/les=113/114 n=42 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:0e3dc695:::obj78:head 2026-02-11T23:31:27.511+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0dc4a75f:::obj282:head local-lis/les=113/114 n=42 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:0e3dc695:::obj78:head v 111'78 2026-02-11T23:31:27.511+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0dc4a75f:::obj282:head local-lis/les=113/114 n=42 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.511+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.512+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:0e3dc695:::obj78:head, version: 111'78, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e3dc695:::obj78:head@111'78, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:27.512+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.512+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.512+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0e39285d:::obj525:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.512+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:0e39285d:::obj525:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 1.0 e 116/116 lb 1:0e39285d:::obj525:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:0dc4a75f:::obj282:head local-lis/les=113/114 n=42 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.512+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.512+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0dc4a75f:::obj282:head local-lis/les=113/114 n=42 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:0e39285d:::obj525:head) v3 2026-02-11T23:31:27.512+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0dc4a75f:::obj282:head local-lis/les=113/114 n=42 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:0e39285d:::obj525:head) 2026-02-11T23:31:27.512+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.512+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:0e39285d:::obj525:head) v3 finish latency 0.000019 2026-02-11T23:31:27.512+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0e39285d:::obj525:head local-lis/les=113/114 n=43 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.512+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.512+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.512+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0e39285d:::obj525:head local-lis/les=113/114 n=43 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.512+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.512+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.512+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.512+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e6b980 con 0x4acb000 2026-02-11T23:31:27.512+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:0e3dc695:::obj78:head)]) -- 0x5e6a300 con 0x4acb000 2026-02-11T23:31:27.512+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.512+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 123 ==== MOSDPGPush(1.0 116/113 [PushOp(1:0e5b0695:::obj117:head, version: 111'117, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e5b0695:::obj117:head@111'117, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e27520 con 0x4acb000 2026-02-11T23:31:27.512+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:0e5b0695:::obj117:head, version: 111'117, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e5b0695:::obj117:head@111'117, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.512+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0e5b0695:::obj117:head, version: 111'117, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e5b0695:::obj117:head@111'117, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.512+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.512+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 124 ==== pg_backfill(progress 1.0 e 116/116 lb 1:0e3dc695:::obj78:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d2e100 con 0x4acb000 2026-02-11T23:31:27.512+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0e5b0695:::obj117:head, version: 111'117, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e5b0695:::obj117:head@111'117, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.512+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:0e3dc695:::obj78:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.512+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0e3dc695:::obj78:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.512+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.512+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0e5b0695:::obj117:head, version: 111'117, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e5b0695:::obj117:head@111'117, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.512+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:0e5b0695:::obj117:head, version: 111'117, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e5b0695:::obj117:head@111'117, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(1.0 116/113 [PushOp(1:0e5b0695:::obj117:head, version: 111'117, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e5b0695:::obj117:head@111'117, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:0e39285d:::obj525:head local-lis/les=113/114 n=43 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.512+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.512+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.512+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0e39285d:::obj525:head local-lis/les=113/114 n=43 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:0e5b0695:::obj117:head, version: 111'117, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e5b0695:::obj117:head@111'117, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.512+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0e3dc695:::obj78:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.512+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0e39285d:::obj525:head local-lis/les=113/114 n=43 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:0e5b0695:::obj117:head@111'117, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.512+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0e39285d:::obj525:head local-lis/les=113/114 n=43 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:0e5b0695:::obj117:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.512+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0e39285d:::obj525:head local-lis/les=113/114 n=43 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:0e5b0695:::obj117:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.512+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0e39285d:::obj525:head local-lis/les=113/114 n=43 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:0e5b0695:::obj117:head 2026-02-11T23:31:27.512+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0e39285d:::obj525:head local-lis/les=113/114 n=43 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:0e5b0695:::obj117:head v 111'117 2026-02-11T23:31:27.512+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0e39285d:::obj525:head local-lis/les=113/114 n=43 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.512+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.512+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:0e5b0695:::obj117:head, version: 111'117, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e5b0695:::obj117:head@111'117, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:27.512+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0e39285d:::obj525:head local-lis/les=113/114 n=43 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.512+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.512+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.512+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0e3dc695:::obj78:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.512+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:0e3dc695:::obj78:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 1.0 e 116/116 lb 1:0e3dc695:::obj78:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:0e39285d:::obj525:head local-lis/les=113/114 n=43 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.512+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.512+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0e39285d:::obj525:head local-lis/les=113/114 n=43 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:0e3dc695:::obj78:head) v3 2026-02-11T23:31:27.512+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0e39285d:::obj525:head local-lis/les=113/114 n=43 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:0e3dc695:::obj78:head) 2026-02-11T23:31:27.512+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.512+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:0e3dc695:::obj78:head) v3 finish latency 0.000017 2026-02-11T23:31:27.512+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.512+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.512+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0e3dc695:::obj78:head local-lis/les=113/114 n=44 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.512+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.512+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.512+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.512+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x4ec2780 con 0x4acb000 2026-02-11T23:31:27.512+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:0e5b0695:::obj117:head)]) -- 0x4ec2480 con 0x4acb000 2026-02-11T23:31:27.512+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.513+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 125 ==== MOSDPGPush(1.0 116/113 [PushOp(1:0e5e5c87:::obj521:head, version: 111'521, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e5e5c87:::obj521:head@111'521, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e27d40 con 0x4acb000 2026-02-11T23:31:27.513+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:0e5e5c87:::obj521:head, version: 111'521, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e5e5c87:::obj521:head@111'521, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 116 2026-02-11T23:31:27.513+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0e5e5c87:::obj521:head, version: 111'521, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e5e5c87:::obj521:head@111'521, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.513+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.513+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 126 ==== pg_backfill(progress 1.0 e 116/116 lb 1:0e5b0695:::obj117:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc3080 con 0x4acb000 2026-02-11T23:31:27.513+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:0e5b0695:::obj117:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.513+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0e5e5c87:::obj521:head, version: 111'521, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e5e5c87:::obj521:head@111'521, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.513+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0e5b0695:::obj117:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.513+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.513+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0e5e5c87:::obj521:head, version: 111'521, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e5e5c87:::obj521:head@111'521, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.513+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.513+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0e5b0695:::obj117:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.513+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:0e5e5c87:::obj521:head, version: 111'521, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e5e5c87:::obj521:head@111'521, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:0e5e5c87:::obj521:head, version: 111'521, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e5e5c87:::obj521:head@111'521, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:0e3dc695:::obj78:head local-lis/les=113/114 n=44 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.513+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.513+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0e3dc695:::obj78:head local-lis/les=113/114 n=44 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:0e5e5c87:::obj521:head, version: 111'521, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e5e5c87:::obj521:head@111'521, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.513+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0e3dc695:::obj78:head local-lis/les=113/114 n=44 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:0e5e5c87:::obj521:head@111'521, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.513+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0e3dc695:::obj78:head local-lis/les=113/114 n=44 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:0e5e5c87:::obj521:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.513+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0e3dc695:::obj78:head local-lis/les=113/114 n=44 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:0e5e5c87:::obj521:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.513+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0e3dc695:::obj78:head local-lis/les=113/114 n=44 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:0e5e5c87:::obj521:head 2026-02-11T23:31:27.513+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0e3dc695:::obj78:head local-lis/les=113/114 n=44 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:0e5e5c87:::obj521:head v 111'521 2026-02-11T23:31:27.513+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0e3dc695:::obj78:head local-lis/les=113/114 n=44 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.513+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.513+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:0e5e5c87:::obj521:head, version: 111'521, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e5e5c87:::obj521:head@111'521, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:27.513+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0e3dc695:::obj78:head local-lis/les=113/114 n=44 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.513+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.513+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.513+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0e5b0695:::obj117:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.513+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:0e5b0695:::obj117:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 1.0 e 116/116 lb 1:0e5b0695:::obj117:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:0e3dc695:::obj78:head local-lis/les=113/114 n=44 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.513+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.513+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0e3dc695:::obj78:head local-lis/les=113/114 n=44 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:0e5b0695:::obj117:head) v3 2026-02-11T23:31:27.513+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0e3dc695:::obj78:head local-lis/les=113/114 n=44 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:0e5b0695:::obj117:head) 2026-02-11T23:31:27.513+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.513+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:0e5b0695:::obj117:head) v3 finish latency 0.000015 2026-02-11T23:31:27.513+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.513+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.513+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0e5b0695:::obj117:head local-lis/les=113/114 n=45 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.513+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.513+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.513+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.513+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e3b680 con 0x4acb000 2026-02-11T23:31:27.513+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:0e5e5c87:::obj521:head)]) -- 0x4ecc480 con 0x4acb000 2026-02-11T23:31:27.513+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.513+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 127 ==== MOSDPGPush(1.0 116/113 [PushOp(1:0ed26813:::obj311:head, version: 111'311, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ed26813:::obj311:head@111'311, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db01a0 con 0x4acb000 2026-02-11T23:31:27.513+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:0ed26813:::obj311:head, version: 111'311, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ed26813:::obj311:head@111'311, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.513+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0ed26813:::obj311:head, version: 111'311, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ed26813:::obj311:head@111'311, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.513+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.513+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 128 ==== pg_backfill(progress 1.0 e 116/116 lb 1:0e5e5c87:::obj521:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc3600 con 0x4acb000 2026-02-11T23:31:27.513+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0ed26813:::obj311:head, version: 111'311, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ed26813:::obj311:head@111'311, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.513+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:0e5e5c87:::obj521:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.513+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0e5e5c87:::obj521:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.513+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.513+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0ed26813:::obj311:head, version: 111'311, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ed26813:::obj311:head@111'311, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.513+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:0ed26813:::obj311:head, version: 111'311, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ed26813:::obj311:head@111'311, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 116/113 [PushOp(1:0ed26813:::obj311:head, version: 111'311, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ed26813:::obj311:head@111'311, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:0e5b0695:::obj117:head local-lis/les=113/114 n=45 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.513+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.513+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.513+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0e5b0695:::obj117:head local-lis/les=113/114 n=45 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:0ed26813:::obj311:head, version: 111'311, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ed26813:::obj311:head@111'311, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.513+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0e5e5c87:::obj521:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.513+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0e5b0695:::obj117:head local-lis/les=113/114 n=45 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:0ed26813:::obj311:head@111'311, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.513+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0e5b0695:::obj117:head local-lis/les=113/114 n=45 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:0ed26813:::obj311:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.513+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0e5b0695:::obj117:head local-lis/les=113/114 n=45 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:0ed26813:::obj311:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.513+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0e5b0695:::obj117:head local-lis/les=113/114 n=45 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:0ed26813:::obj311:head 2026-02-11T23:31:27.513+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0e5b0695:::obj117:head local-lis/les=113/114 n=45 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:0ed26813:::obj311:head v 111'311 2026-02-11T23:31:27.513+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0e5b0695:::obj117:head local-lis/les=113/114 n=45 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.513+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.513+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:0ed26813:::obj311:head, version: 111'311, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ed26813:::obj311:head@111'311, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:27.513+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0e5b0695:::obj117:head local-lis/les=113/114 n=45 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.513+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.513+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.513+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0e5e5c87:::obj521:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.513+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:0e5e5c87:::obj521:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 1.0 e 116/116 lb 1:0e5e5c87:::obj521:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:0e5b0695:::obj117:head local-lis/les=113/114 n=45 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.514+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.514+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0e5b0695:::obj117:head local-lis/les=113/114 n=45 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:0e5e5c87:::obj521:head) v3 2026-02-11T23:31:27.514+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0e5b0695:::obj117:head local-lis/les=113/114 n=45 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:0e5e5c87:::obj521:head) 2026-02-11T23:31:27.514+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.514+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:0e5e5c87:::obj521:head) v3 finish latency 0.000017 2026-02-11T23:31:27.514+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.514+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.514+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0e5e5c87:::obj521:head local-lis/les=113/114 n=46 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.514+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.514+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.514+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.514+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e3a180 con 0x4acb000 2026-02-11T23:31:27.514+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:0ed26813:::obj311:head)]) -- 0x5e3bc80 con 0x4acb000 2026-02-11T23:31:27.514+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.514+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 129 ==== MOSDPGPush(1.0 116/113 [PushOp(1:0ed3c684:::obj270:head, version: 111'270, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ed3c684:::obj270:head@111'270, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db1860 con 0x4acb000 2026-02-11T23:31:27.514+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:0ed3c684:::obj270:head, version: 111'270, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ed3c684:::obj270:head@111'270, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.514+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0ed3c684:::obj270:head, version: 111'270, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ed3c684:::obj270:head@111'270, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.514+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.514+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 130 ==== pg_backfill(progress 1.0 e 116/116 lb 1:0ed26813:::obj311:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc4c00 con 0x4acb000 2026-02-11T23:31:27.514+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:0ed26813:::obj311:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.514+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0ed3c684:::obj270:head, version: 111'270, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ed3c684:::obj270:head@111'270, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.514+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0ed26813:::obj311:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.514+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.514+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0ed3c684:::obj270:head, version: 111'270, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ed3c684:::obj270:head@111'270, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.514+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.514+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0ed26813:::obj311:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.514+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:0ed3c684:::obj270:head, version: 111'270, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ed3c684:::obj270:head@111'270, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 116/113 [PushOp(1:0ed3c684:::obj270:head, version: 111'270, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ed3c684:::obj270:head@111'270, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:0e5e5c87:::obj521:head local-lis/les=113/114 n=46 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.514+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.514+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0e5e5c87:::obj521:head local-lis/les=113/114 n=46 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:0ed3c684:::obj270:head, version: 111'270, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ed3c684:::obj270:head@111'270, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.514+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0e5e5c87:::obj521:head local-lis/les=113/114 n=46 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:0ed3c684:::obj270:head@111'270, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.514+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0e5e5c87:::obj521:head local-lis/les=113/114 n=46 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:0ed3c684:::obj270:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.514+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0e5e5c87:::obj521:head local-lis/les=113/114 n=46 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:0ed3c684:::obj270:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.514+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0e5e5c87:::obj521:head local-lis/les=113/114 n=46 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:0ed3c684:::obj270:head 2026-02-11T23:31:27.514+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0e5e5c87:::obj521:head local-lis/les=113/114 n=46 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:0ed3c684:::obj270:head v 111'270 2026-02-11T23:31:27.514+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0e5e5c87:::obj521:head local-lis/les=113/114 n=46 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.514+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.514+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:0ed3c684:::obj270:head, version: 111'270, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ed3c684:::obj270:head@111'270, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:27.514+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0e5e5c87:::obj521:head local-lis/les=113/114 n=46 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.514+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.514+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.514+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0ed26813:::obj311:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.514+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:0ed26813:::obj311:head) v3 prio 15 cost 0 latency 0.000080 pg_backfill(progress 1.0 e 116/116 lb 1:0ed26813:::obj311:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:0e5e5c87:::obj521:head local-lis/les=113/114 n=46 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.514+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.514+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0e5e5c87:::obj521:head local-lis/les=113/114 n=46 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:0ed26813:::obj311:head) v3 2026-02-11T23:31:27.514+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0e5e5c87:::obj521:head local-lis/les=113/114 n=46 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:0ed26813:::obj311:head) 2026-02-11T23:31:27.514+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.514+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:0ed26813:::obj311:head) v3 finish latency 0.000015 2026-02-11T23:31:27.514+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.514+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.514+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0ed26813:::obj311:head local-lis/les=113/114 n=47 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.514+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.514+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.514+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.514+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d78300 con 0x4acb000 2026-02-11T23:31:27.514+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:0ed3c684:::obj270:head)]) -- 0x5d78000 con 0x4acb000 2026-02-11T23:31:27.514+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.515+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 131 ==== MOSDPGPush(1.0 116/113 [PushOp(1:0ee9ae15:::obj4:head, version: 111'4, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ee9ae15:::obj4:head@111'4, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 4999+0+0 (unknown 0 0 0) 0x5db09c0 con 0x4acb000 2026-02-11T23:31:27.515+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:0ee9ae15:::obj4:head, version: 111'4, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ee9ae15:::obj4:head@111'4, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.515+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0ee9ae15:::obj4:head, version: 111'4, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ee9ae15:::obj4:head@111'4, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.515+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.515+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 132 ==== pg_backfill(progress 1.0 e 116/116 lb 1:0ed3c684:::obj270:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc2000 con 0x4acb000 2026-02-11T23:31:27.515+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:0ed3c684:::obj270:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.515+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0ee9ae15:::obj4:head, version: 111'4, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ee9ae15:::obj4:head@111'4, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.515+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0ed3c684:::obj270:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.515+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.515+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0ee9ae15:::obj4:head, version: 111'4, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ee9ae15:::obj4:head@111'4, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.515+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.515+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0ed3c684:::obj270:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.515+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:0ee9ae15:::obj4:head, version: 111'4, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ee9ae15:::obj4:head@111'4, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 116/113 [PushOp(1:0ee9ae15:::obj4:head, version: 111'4, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ee9ae15:::obj4:head@111'4, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:0ed26813:::obj311:head local-lis/les=113/114 n=47 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.515+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.515+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0ed26813:::obj311:head local-lis/les=113/114 n=47 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:0ee9ae15:::obj4:head, version: 111'4, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ee9ae15:::obj4:head@111'4, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.515+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0ed26813:::obj311:head local-lis/les=113/114 n=47 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:0ee9ae15:::obj4:head@111'4, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.515+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0ed26813:::obj311:head local-lis/les=113/114 n=47 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:0ee9ae15:::obj4:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.515+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0ed26813:::obj311:head local-lis/les=113/114 n=47 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:0ee9ae15:::obj4:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.515+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0ed26813:::obj311:head local-lis/les=113/114 n=47 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:0ee9ae15:::obj4:head 2026-02-11T23:31:27.515+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0ed26813:::obj311:head local-lis/les=113/114 n=47 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:0ee9ae15:::obj4:head v 111'4 2026-02-11T23:31:27.515+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0ed26813:::obj311:head local-lis/les=113/114 n=47 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.515+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.515+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:0ee9ae15:::obj4:head, version: 111'4, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ee9ae15:::obj4:head@111'4, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000073 2026-02-11T23:31:27.515+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0ed26813:::obj311:head local-lis/les=113/114 n=47 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.515+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.515+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0ed3c684:::obj270:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.515+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:0ed3c684:::obj270:head) v3 prio 15 cost 0 latency 0.000093 pg_backfill(progress 1.0 e 116/116 lb 1:0ed3c684:::obj270:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:0ed26813:::obj311:head local-lis/les=113/114 n=47 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.515+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.515+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.515+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0ed26813:::obj311:head local-lis/les=113/114 n=47 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:0ed3c684:::obj270:head) v3 2026-02-11T23:31:27.515+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0ed26813:::obj311:head local-lis/les=113/114 n=47 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:0ed3c684:::obj270:head) 2026-02-11T23:31:27.515+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.515+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:0ed3c684:::obj270:head) v3 finish latency 0.000015 2026-02-11T23:31:27.515+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.515+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.515+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0ed3c684:::obj270:head local-lis/les=113/114 n=48 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.515+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.515+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.515+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.515+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d78600 con 0x4acb000 2026-02-11T23:31:27.515+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:0ee9ae15:::obj4:head)]) -- 0x5d78480 con 0x4acb000 2026-02-11T23:31:27.515+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.515+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 133 ==== MOSDPGPush(1.0 116/113 [PushOp(1:0efd4012:::obj445:head, version: 111'445, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0efd4012:::obj445:head@111'445, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e27ba0 con 0x4acb000 2026-02-11T23:31:27.515+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:0efd4012:::obj445:head, version: 111'445, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0efd4012:::obj445:head@111'445, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 116 2026-02-11T23:31:27.515+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0efd4012:::obj445:head, version: 111'445, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0efd4012:::obj445:head@111'445, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.515+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.515+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 134 ==== pg_backfill(progress 1.0 e 116/116 lb 1:0ee9ae15:::obj4:head) ==== 1060+0+0 (unknown 0 0 0) 0x5dc3b80 con 0x4acb000 2026-02-11T23:31:27.515+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:0ee9ae15:::obj4:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.515+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0efd4012:::obj445:head, version: 111'445, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0efd4012:::obj445:head@111'445, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.515+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0ee9ae15:::obj4:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.515+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.515+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0efd4012:::obj445:head, version: 111'445, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0efd4012:::obj445:head@111'445, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.515+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.515+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:0efd4012:::obj445:head, version: 111'445, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0efd4012:::obj445:head@111'445, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(1.0 116/113 [PushOp(1:0efd4012:::obj445:head, version: 111'445, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0efd4012:::obj445:head@111'445, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:0ed3c684:::obj270:head local-lis/les=113/114 n=48 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.515+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0ee9ae15:::obj4:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.515+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.515+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0ed3c684:::obj270:head local-lis/les=113/114 n=48 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:0efd4012:::obj445:head, version: 111'445, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0efd4012:::obj445:head@111'445, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.515+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0ed3c684:::obj270:head local-lis/les=113/114 n=48 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:0efd4012:::obj445:head@111'445, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.515+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0ed3c684:::obj270:head local-lis/les=113/114 n=48 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:0efd4012:::obj445:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.515+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0ed3c684:::obj270:head local-lis/les=113/114 n=48 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:0efd4012:::obj445:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.515+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0ed3c684:::obj270:head local-lis/les=113/114 n=48 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:0efd4012:::obj445:head 2026-02-11T23:31:27.515+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0ed3c684:::obj270:head local-lis/les=113/114 n=48 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:0efd4012:::obj445:head v 111'445 2026-02-11T23:31:27.515+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0ed3c684:::obj270:head local-lis/les=113/114 n=48 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.515+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.515+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:0efd4012:::obj445:head, version: 111'445, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0efd4012:::obj445:head@111'445, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:27.515+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0ed3c684:::obj270:head local-lis/les=113/114 n=48 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.515+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.515+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.515+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0ee9ae15:::obj4:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.515+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:0ee9ae15:::obj4:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 1.0 e 116/116 lb 1:0ee9ae15:::obj4:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:0ed3c684:::obj270:head local-lis/les=113/114 n=48 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.515+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.515+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0ed3c684:::obj270:head local-lis/les=113/114 n=48 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:0ee9ae15:::obj4:head) v3 2026-02-11T23:31:27.515+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0ed3c684:::obj270:head local-lis/les=113/114 n=48 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:0ee9ae15:::obj4:head) 2026-02-11T23:31:27.515+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.515+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:0ee9ae15:::obj4:head) v3 finish latency 0.000013 2026-02-11T23:31:27.515+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.516+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.516+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0ee9ae15:::obj4:head local-lis/les=113/114 n=49 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.516+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.516+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.516+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.516+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d78a80 con 0x4acb000 2026-02-11T23:31:27.516+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:0efd4012:::obj445:head)]) -- 0x5d78780 con 0x4acb000 2026-02-11T23:31:27.516+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.516+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 135 ==== MOSDPGPush(1.0 116/113 [PushOp(1:0f33f807:::obj273:head, version: 111'273, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0f33f807:::obj273:head@111'273, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3815d40 con 0x4acb000 2026-02-11T23:31:27.516+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:0f33f807:::obj273:head, version: 111'273, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0f33f807:::obj273:head@111'273, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 116 2026-02-11T23:31:27.516+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0f33f807:::obj273:head, version: 111'273, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0f33f807:::obj273:head@111'273, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.516+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.516+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 136 ==== pg_backfill(progress 1.0 e 116/116 lb 1:0efd4012:::obj445:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc4100 con 0x4acb000 2026-02-11T23:31:27.516+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0f33f807:::obj273:head, version: 111'273, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0f33f807:::obj273:head@111'273, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.516+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:0efd4012:::obj445:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 116 2026-02-11T23:31:27.516+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0efd4012:::obj445:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.516+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.516+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0f33f807:::obj273:head, version: 111'273, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0f33f807:::obj273:head@111'273, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.516+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:0f33f807:::obj273:head, version: 111'273, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0f33f807:::obj273:head@111'273, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 116/113 [PushOp(1:0f33f807:::obj273:head, version: 111'273, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0f33f807:::obj273:head@111'273, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:0ee9ae15:::obj4:head local-lis/les=113/114 n=49 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.516+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.516+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.516+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0ee9ae15:::obj4:head local-lis/les=113/114 n=49 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:0f33f807:::obj273:head, version: 111'273, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0f33f807:::obj273:head@111'273, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.516+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0efd4012:::obj445:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.516+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0ee9ae15:::obj4:head local-lis/les=113/114 n=49 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:0f33f807:::obj273:head@111'273, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.516+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0ee9ae15:::obj4:head local-lis/les=113/114 n=49 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:0f33f807:::obj273:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.516+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0ee9ae15:::obj4:head local-lis/les=113/114 n=49 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:0f33f807:::obj273:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.516+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0ee9ae15:::obj4:head local-lis/les=113/114 n=49 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:0f33f807:::obj273:head 2026-02-11T23:31:27.516+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0ee9ae15:::obj4:head local-lis/les=113/114 n=49 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:0f33f807:::obj273:head v 111'273 2026-02-11T23:31:27.516+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0ee9ae15:::obj4:head local-lis/les=113/114 n=49 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.516+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.516+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:0f33f807:::obj273:head, version: 111'273, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0f33f807:::obj273:head@111'273, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:27.516+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.516+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.516+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0efd4012:::obj445:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.516+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:0efd4012:::obj445:head) v3 prio 15 cost 0 latency 0.000080 pg_backfill(progress 1.0 e 116/116 lb 1:0efd4012:::obj445:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:0ee9ae15:::obj4:head local-lis/les=113/114 n=49 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.516+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.516+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0ee9ae15:::obj4:head local-lis/les=113/114 n=49 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:0efd4012:::obj445:head) v3 2026-02-11T23:31:27.516+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0ee9ae15:::obj4:head local-lis/les=113/114 n=49 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:0efd4012:::obj445:head) 2026-02-11T23:31:27.516+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.516+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:0efd4012:::obj445:head) v3 finish latency 0.000023 2026-02-11T23:31:27.516+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0efd4012:::obj445:head local-lis/les=113/114 n=50 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.516+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.516+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.516+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0efd4012:::obj445:head local-lis/les=113/114 n=50 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.516+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.516+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.516+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.516+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d78c00 con 0x4acb000 2026-02-11T23:31:27.516+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:0f33f807:::obj273:head)]) -- 0x5e6a900 con 0x4acb000 2026-02-11T23:31:27.516+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.516+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.516+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.517+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 137 ==== MOSDPGPush(1.0 116/113 [PushOp(1:0f45d4cb:::obj171:head, version: 111'171, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0f45d4cb:::obj171:head@111'171, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db01a0 con 0x4acb000 2026-02-11T23:31:27.517+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:0f45d4cb:::obj171:head, version: 111'171, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0f45d4cb:::obj171:head@111'171, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.517+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0f45d4cb:::obj171:head, version: 111'171, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0f45d4cb:::obj171:head@111'171, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.517+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.517+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 138 ==== pg_backfill(progress 1.0 e 116/116 lb 1:0f33f807:::obj273:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc2b00 con 0x4acb000 2026-02-11T23:31:27.517+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0f45d4cb:::obj171:head, version: 111'171, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0f45d4cb:::obj171:head@111'171, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.517+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:0f33f807:::obj273:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.517+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0f33f807:::obj273:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.517+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.517+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:0f45d4cb:::obj171:head, version: 111'171, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0f45d4cb:::obj171:head@111'171, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.517+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:0f45d4cb:::obj171:head, version: 111'171, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0f45d4cb:::obj171:head@111'171, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(1.0 116/113 [PushOp(1:0f45d4cb:::obj171:head, version: 111'171, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0f45d4cb:::obj171:head@111'171, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:0efd4012:::obj445:head local-lis/les=113/114 n=50 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.517+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.517+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0efd4012:::obj445:head local-lis/les=113/114 n=50 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:0f45d4cb:::obj171:head, version: 111'171, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0f45d4cb:::obj171:head@111'171, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.517+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.517+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0efd4012:::obj445:head local-lis/les=113/114 n=50 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:0f45d4cb:::obj171:head@111'171, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.517+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0f33f807:::obj273:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.517+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0efd4012:::obj445:head local-lis/les=113/114 n=50 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:0f45d4cb:::obj171:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.517+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0efd4012:::obj445:head local-lis/les=113/114 n=50 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:0f45d4cb:::obj171:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.517+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0efd4012:::obj445:head local-lis/les=113/114 n=50 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:0f45d4cb:::obj171:head 2026-02-11T23:31:27.517+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0efd4012:::obj445:head local-lis/les=113/114 n=50 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:0f45d4cb:::obj171:head v 111'171 2026-02-11T23:31:27.517+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0efd4012:::obj445:head local-lis/les=113/114 n=50 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.517+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.517+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:0f45d4cb:::obj171:head, version: 111'171, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0f45d4cb:::obj171:head@111'171, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:27.517+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0efd4012:::obj445:head local-lis/les=113/114 n=50 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.517+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.517+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.517+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0f33f807:::obj273:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.517+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:0f33f807:::obj273:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 1.0 e 116/116 lb 1:0f33f807:::obj273:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:0efd4012:::obj445:head local-lis/les=113/114 n=50 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.517+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.517+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0efd4012:::obj445:head local-lis/les=113/114 n=50 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:0f33f807:::obj273:head) v3 2026-02-11T23:31:27.517+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0efd4012:::obj445:head local-lis/les=113/114 n=50 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:0f33f807:::obj273:head) 2026-02-11T23:31:27.517+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.517+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:0f33f807:::obj273:head) v3 finish latency 0.000017 2026-02-11T23:31:27.517+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.517+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.517+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0f33f807:::obj273:head local-lis/les=113/114 n=51 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.517+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.517+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.517+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.517+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d79080 con 0x4acb000 2026-02-11T23:31:27.517+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:0f45d4cb:::obj171:head)]) -- 0x5d78d80 con 0x4acb000 2026-02-11T23:31:27.517+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.517+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 139 ==== MOSDPGPush(1.0 116/113 [PushOp(1:104778fc:::obj2:head, version: 111'2, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:104778fc:::obj2:head@111'2, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 4999+0+0 (unknown 0 0 0) 0x5db1860 con 0x4acb000 2026-02-11T23:31:27.517+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:104778fc:::obj2:head, version: 111'2, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:104778fc:::obj2:head@111'2, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.517+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:104778fc:::obj2:head, version: 111'2, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:104778fc:::obj2:head@111'2, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.517+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.517+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 140 ==== pg_backfill(progress 1.0 e 116/116 lb 1:0f45d4cb:::obj171:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc5700 con 0x4acb000 2026-02-11T23:31:27.517+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:0f45d4cb:::obj171:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.517+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:104778fc:::obj2:head, version: 111'2, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:104778fc:::obj2:head@111'2, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.517+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0f45d4cb:::obj171:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.517+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.517+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0f45d4cb:::obj171:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.517+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.517+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:104778fc:::obj2:head, version: 111'2, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:104778fc:::obj2:head@111'2, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.517+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:104778fc:::obj2:head, version: 111'2, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:104778fc:::obj2:head@111'2, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000043 MOSDPGPush(1.0 116/113 [PushOp(1:104778fc:::obj2:head, version: 111'2, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:104778fc:::obj2:head@111'2, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:0f33f807:::obj273:head local-lis/les=113/114 n=51 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.517+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.517+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0f33f807:::obj273:head local-lis/les=113/114 n=51 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:104778fc:::obj2:head, version: 111'2, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:104778fc:::obj2:head@111'2, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.517+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0f33f807:::obj273:head local-lis/les=113/114 n=51 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:104778fc:::obj2:head@111'2, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.517+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0f33f807:::obj273:head local-lis/les=113/114 n=51 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:104778fc:::obj2:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.517+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0f33f807:::obj273:head local-lis/les=113/114 n=51 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:104778fc:::obj2:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.517+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0f33f807:::obj273:head local-lis/les=113/114 n=51 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:104778fc:::obj2:head 2026-02-11T23:31:27.517+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0f33f807:::obj273:head local-lis/les=113/114 n=51 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:104778fc:::obj2:head v 111'2 2026-02-11T23:31:27.517+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0f33f807:::obj273:head local-lis/les=113/114 n=51 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.517+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.517+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:104778fc:::obj2:head, version: 111'2, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:104778fc:::obj2:head@111'2, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:27.517+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0f33f807:::obj273:head local-lis/les=113/114 n=51 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.517+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.517+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.517+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:0f45d4cb:::obj171:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.517+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:0f45d4cb:::obj171:head) v3 prio 15 cost 0 latency 0.000090 pg_backfill(progress 1.0 e 116/116 lb 1:0f45d4cb:::obj171:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:0f33f807:::obj273:head local-lis/les=113/114 n=51 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.517+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.517+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0f33f807:::obj273:head local-lis/les=113/114 n=51 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:0f45d4cb:::obj171:head) v3 2026-02-11T23:31:27.517+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0f33f807:::obj273:head local-lis/les=113/114 n=51 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:0f45d4cb:::obj171:head) 2026-02-11T23:31:27.517+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.517+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:0f45d4cb:::obj171:head) v3 finish latency 0.000014 2026-02-11T23:31:27.517+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.517+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.517+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0f45d4cb:::obj171:head local-lis/les=113/114 n=52 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.517+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.517+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.517+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.517+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d79380 con 0x4acb000 2026-02-11T23:31:27.517+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:104778fc:::obj2:head)]) -- 0x5d79200 con 0x4acb000 2026-02-11T23:31:27.517+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.518+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 141 ==== MOSDPGPush(1.0 116/113 [PushOp(1:105983a6:::obj88:head, version: 111'88, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:105983a6:::obj88:head@111'88, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5db09c0 con 0x4acb000 2026-02-11T23:31:27.518+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:105983a6:::obj88:head, version: 111'88, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:105983a6:::obj88:head@111'88, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.518+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:105983a6:::obj88:head, version: 111'88, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:105983a6:::obj88:head@111'88, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.518+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.518+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 142 ==== pg_backfill(progress 1.0 e 116/116 lb 1:104778fc:::obj2:head) ==== 1060+0+0 (unknown 0 0 0) 0x5d2d080 con 0x4acb000 2026-02-11T23:31:27.518+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:105983a6:::obj88:head, version: 111'88, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:105983a6:::obj88:head@111'88, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.518+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:104778fc:::obj2:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.518+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:104778fc:::obj2:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.518+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.518+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:105983a6:::obj88:head, version: 111'88, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:105983a6:::obj88:head@111'88, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.518+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:105983a6:::obj88:head, version: 111'88, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:105983a6:::obj88:head@111'88, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(1.0 116/113 [PushOp(1:105983a6:::obj88:head, version: 111'88, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:105983a6:::obj88:head@111'88, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:0f45d4cb:::obj171:head local-lis/les=113/114 n=52 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.518+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.518+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.518+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0f45d4cb:::obj171:head local-lis/les=113/114 n=52 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:105983a6:::obj88:head, version: 111'88, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:105983a6:::obj88:head@111'88, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.518+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:104778fc:::obj2:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.518+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0f45d4cb:::obj171:head local-lis/les=113/114 n=52 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:105983a6:::obj88:head@111'88, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.518+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0f45d4cb:::obj171:head local-lis/les=113/114 n=52 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:105983a6:::obj88:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.518+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0f45d4cb:::obj171:head local-lis/les=113/114 n=52 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:105983a6:::obj88:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.518+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0f45d4cb:::obj171:head local-lis/les=113/114 n=52 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:105983a6:::obj88:head 2026-02-11T23:31:27.518+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0f45d4cb:::obj171:head local-lis/les=113/114 n=52 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:105983a6:::obj88:head v 111'88 2026-02-11T23:31:27.518+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0f45d4cb:::obj171:head local-lis/les=113/114 n=52 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.518+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.518+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:105983a6:::obj88:head, version: 111'88, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:105983a6:::obj88:head@111'88, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:27.518+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0f45d4cb:::obj171:head local-lis/les=113/114 n=52 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.518+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.518+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.518+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:104778fc:::obj2:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.518+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:104778fc:::obj2:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 1.0 e 116/116 lb 1:104778fc:::obj2:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:0f45d4cb:::obj171:head local-lis/les=113/114 n=52 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.518+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.518+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0f45d4cb:::obj171:head local-lis/les=113/114 n=52 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:104778fc:::obj2:head) v3 2026-02-11T23:31:27.518+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:0f45d4cb:::obj171:head local-lis/les=113/114 n=52 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:104778fc:::obj2:head) 2026-02-11T23:31:27.518+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.518+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:104778fc:::obj2:head) v3 finish latency 0.000016 2026-02-11T23:31:27.518+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.518+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.518+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:104778fc:::obj2:head local-lis/les=113/114 n=53 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.518+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.518+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.518+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.518+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d79680 con 0x4acb000 2026-02-11T23:31:27.518+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:105983a6:::obj88:head)]) -- 0x5d79500 con 0x4acb000 2026-02-11T23:31:27.518+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.518+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 143 ==== MOSDPGPush(1.0 116/113 [PushOp(1:10754df1:::obj159:head, version: 111'159, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10754df1:::obj159:head@111'159, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e26340 con 0x4acb000 2026-02-11T23:31:27.518+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:10754df1:::obj159:head, version: 111'159, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10754df1:::obj159:head@111'159, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.518+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:10754df1:::obj159:head, version: 111'159, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10754df1:::obj159:head@111'159, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.519+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.519+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 144 ==== pg_backfill(progress 1.0 e 116/116 lb 1:105983a6:::obj88:head) ==== 1061+0+0 (unknown 0 0 0) 0x5dc2580 con 0x4acb000 2026-02-11T23:31:27.519+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:105983a6:::obj88:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.519+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:10754df1:::obj159:head, version: 111'159, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10754df1:::obj159:head@111'159, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.519+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:105983a6:::obj88:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.519+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.519+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:105983a6:::obj88:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.519+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.519+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:10754df1:::obj159:head, version: 111'159, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10754df1:::obj159:head@111'159, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.519+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:10754df1:::obj159:head, version: 111'159, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10754df1:::obj159:head@111'159, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000049 MOSDPGPush(1.0 116/113 [PushOp(1:10754df1:::obj159:head, version: 111'159, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10754df1:::obj159:head@111'159, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:104778fc:::obj2:head local-lis/les=113/114 n=53 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.519+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.519+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:104778fc:::obj2:head local-lis/les=113/114 n=53 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:10754df1:::obj159:head, version: 111'159, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10754df1:::obj159:head@111'159, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.519+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:104778fc:::obj2:head local-lis/les=113/114 n=53 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:10754df1:::obj159:head@111'159, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.519+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:104778fc:::obj2:head local-lis/les=113/114 n=53 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:10754df1:::obj159:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.519+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:104778fc:::obj2:head local-lis/les=113/114 n=53 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:10754df1:::obj159:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.519+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:104778fc:::obj2:head local-lis/les=113/114 n=53 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:10754df1:::obj159:head 2026-02-11T23:31:27.519+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:104778fc:::obj2:head local-lis/les=113/114 n=53 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:10754df1:::obj159:head v 111'159 2026-02-11T23:31:27.519+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:104778fc:::obj2:head local-lis/les=113/114 n=53 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.519+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.519+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:10754df1:::obj159:head, version: 111'159, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10754df1:::obj159:head@111'159, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:27.519+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.519+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.519+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:105983a6:::obj88:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.519+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:105983a6:::obj88:head) v3 prio 15 cost 0 latency 0.000086 pg_backfill(progress 1.0 e 116/116 lb 1:105983a6:::obj88:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:104778fc:::obj2:head local-lis/les=113/114 n=53 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.519+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.519+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:104778fc:::obj2:head local-lis/les=113/114 n=53 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:105983a6:::obj88:head) v3 2026-02-11T23:31:27.519+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:104778fc:::obj2:head local-lis/les=113/114 n=53 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:105983a6:::obj88:head) 2026-02-11T23:31:27.519+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.519+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:105983a6:::obj88:head) v3 finish latency 0.000018 2026-02-11T23:31:27.519+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:105983a6:::obj88:head local-lis/les=113/114 n=54 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.519+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.519+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.519+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:105983a6:::obj88:head local-lis/les=113/114 n=54 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.519+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.519+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.519+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.519+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d78300 con 0x4acb000 2026-02-11T23:31:27.519+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:10754df1:::obj159:head)]) -- 0x5e6ad80 con 0x4acb000 2026-02-11T23:31:27.519+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.519+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 145 ==== MOSDPGPush(1.0 116/113 [PushOp(1:10b422cc:::obj197:head, version: 111'197, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10b422cc:::obj197:head@111'197, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e271e0 con 0x4acb000 2026-02-11T23:31:27.519+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:10b422cc:::obj197:head, version: 111'197, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10b422cc:::obj197:head@111'197, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.519+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:10b422cc:::obj197:head, version: 111'197, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10b422cc:::obj197:head@111'197, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.519+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.519+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 146 ==== pg_backfill(progress 1.0 e 116/116 lb 1:10754df1:::obj159:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc3080 con 0x4acb000 2026-02-11T23:31:27.519+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:10b422cc:::obj197:head, version: 111'197, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10b422cc:::obj197:head@111'197, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.519+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:10754df1:::obj159:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.519+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.519+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:10754df1:::obj159:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.519+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:10b422cc:::obj197:head, version: 111'197, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10b422cc:::obj197:head@111'197, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.519+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:10b422cc:::obj197:head, version: 111'197, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10b422cc:::obj197:head@111'197, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 116/113 [PushOp(1:10b422cc:::obj197:head, version: 111'197, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10b422cc:::obj197:head@111'197, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:105983a6:::obj88:head local-lis/les=113/114 n=54 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.519+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.519+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.519+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:105983a6:::obj88:head local-lis/les=113/114 n=54 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:10b422cc:::obj197:head, version: 111'197, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10b422cc:::obj197:head@111'197, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.519+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:105983a6:::obj88:head local-lis/les=113/114 n=54 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:10b422cc:::obj197:head@111'197, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.519+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:105983a6:::obj88:head local-lis/les=113/114 n=54 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:10b422cc:::obj197:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.519+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.519+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:105983a6:::obj88:head local-lis/les=113/114 n=54 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:10b422cc:::obj197:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.519+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:10754df1:::obj159:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.519+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:105983a6:::obj88:head local-lis/les=113/114 n=54 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:10b422cc:::obj197:head 2026-02-11T23:31:27.519+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:105983a6:::obj88:head local-lis/les=113/114 n=54 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:10b422cc:::obj197:head v 111'197 2026-02-11T23:31:27.519+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:105983a6:::obj88:head local-lis/les=113/114 n=54 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.519+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.519+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:10b422cc:::obj197:head, version: 111'197, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10b422cc:::obj197:head@111'197, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:27.519+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:105983a6:::obj88:head local-lis/les=113/114 n=54 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.519+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.519+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.519+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:10754df1:::obj159:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.519+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:10754df1:::obj159:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 1.0 e 116/116 lb 1:10754df1:::obj159:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:105983a6:::obj88:head local-lis/les=113/114 n=54 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.519+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.519+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:105983a6:::obj88:head local-lis/les=113/114 n=54 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:10754df1:::obj159:head) v3 2026-02-11T23:31:27.519+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:105983a6:::obj88:head local-lis/les=113/114 n=54 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:10754df1:::obj159:head) 2026-02-11T23:31:27.519+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.519+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:10754df1:::obj159:head) v3 finish latency 0.000015 2026-02-11T23:31:27.519+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.519+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.519+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:10754df1:::obj159:head local-lis/les=113/114 n=55 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.519+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.519+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.519+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.519+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d78480 con 0x4acb000 2026-02-11T23:31:27.519+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:10b422cc:::obj197:head)]) -- 0x5d78000 con 0x4acb000 2026-02-11T23:31:27.519+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.520+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 147 ==== MOSDPGPush(1.0 116/113 [PushOp(1:10c2d3a1:::obj448:head, version: 111'448, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10c2d3a1:::obj448:head@111'448, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e26000 con 0x4acb000 2026-02-11T23:31:27.520+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:10c2d3a1:::obj448:head, version: 111'448, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10c2d3a1:::obj448:head@111'448, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.520+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:10c2d3a1:::obj448:head, version: 111'448, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10c2d3a1:::obj448:head@111'448, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.520+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.520+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:10c2d3a1:::obj448:head, version: 111'448, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10c2d3a1:::obj448:head@111'448, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.520+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 148 ==== pg_backfill(progress 1.0 e 116/116 lb 1:10b422cc:::obj197:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc3600 con 0x4acb000 2026-02-11T23:31:27.520+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.520+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:10c2d3a1:::obj448:head, version: 111'448, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10c2d3a1:::obj448:head@111'448, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.520+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:10b422cc:::obj197:head) v3 prio 15 type 95 cost 0 latency 0.000010 epoch 116 2026-02-11T23:31:27.520+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:10b422cc:::obj197:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.520+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.520+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:10c2d3a1:::obj448:head, version: 111'448, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10c2d3a1:::obj448:head@111'448, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:10c2d3a1:::obj448:head, version: 111'448, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10c2d3a1:::obj448:head@111'448, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:10754df1:::obj159:head local-lis/les=113/114 n=55 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.520+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:10b422cc:::obj197:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.520+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.520+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:10754df1:::obj159:head local-lis/les=113/114 n=55 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:10c2d3a1:::obj448:head, version: 111'448, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10c2d3a1:::obj448:head@111'448, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.520+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:10754df1:::obj159:head local-lis/les=113/114 n=55 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:10c2d3a1:::obj448:head@111'448, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.520+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:10754df1:::obj159:head local-lis/les=113/114 n=55 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:10c2d3a1:::obj448:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.520+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:10754df1:::obj159:head local-lis/les=113/114 n=55 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:10c2d3a1:::obj448:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.520+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:10754df1:::obj159:head local-lis/les=113/114 n=55 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:10c2d3a1:::obj448:head 2026-02-11T23:31:27.520+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:10754df1:::obj159:head local-lis/les=113/114 n=55 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:10c2d3a1:::obj448:head v 111'448 2026-02-11T23:31:27.520+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:10754df1:::obj159:head local-lis/les=113/114 n=55 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.520+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.520+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:10c2d3a1:::obj448:head, version: 111'448, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10c2d3a1:::obj448:head@111'448, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:27.520+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.520+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.520+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:10b422cc:::obj197:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.520+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:10b422cc:::obj197:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 1.0 e 116/116 lb 1:10b422cc:::obj197:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:10754df1:::obj159:head local-lis/les=113/114 n=55 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.520+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.520+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:10754df1:::obj159:head local-lis/les=113/114 n=55 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:10b422cc:::obj197:head) v3 2026-02-11T23:31:27.520+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:10754df1:::obj159:head local-lis/les=113/114 n=55 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:10b422cc:::obj197:head) 2026-02-11T23:31:27.520+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.520+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:10b422cc:::obj197:head) v3 finish latency 0.000017 2026-02-11T23:31:27.520+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:10b422cc:::obj197:head local-lis/les=113/114 n=56 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.520+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.520+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.520+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:10b422cc:::obj197:head local-lis/les=113/114 n=56 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.520+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.520+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.520+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.520+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d79800 con 0x4acb000 2026-02-11T23:31:27.520+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:10c2d3a1:::obj448:head)]) -- 0x5e6a600 con 0x4acb000 2026-02-11T23:31:27.520+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.520+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 149 ==== MOSDPGPush(1.0 116/113 [PushOp(1:10f4e680:::obj204:head, version: 111'204, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10f4e680:::obj204:head@111'204, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db16c0 con 0x4acb000 2026-02-11T23:31:27.520+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:10f4e680:::obj204:head, version: 111'204, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10f4e680:::obj204:head@111'204, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.520+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:10f4e680:::obj204:head, version: 111'204, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10f4e680:::obj204:head@111'204, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.520+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.520+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 150 ==== pg_backfill(progress 1.0 e 116/116 lb 1:10c2d3a1:::obj448:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc4c00 con 0x4acb000 2026-02-11T23:31:27.520+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:10f4e680:::obj204:head, version: 111'204, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10f4e680:::obj204:head@111'204, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.520+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:10c2d3a1:::obj448:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 116 2026-02-11T23:31:27.520+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:10c2d3a1:::obj448:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.520+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.520+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:10f4e680:::obj204:head, version: 111'204, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10f4e680:::obj204:head@111'204, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.520+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:10f4e680:::obj204:head, version: 111'204, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10f4e680:::obj204:head@111'204, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:10f4e680:::obj204:head, version: 111'204, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10f4e680:::obj204:head@111'204, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:10b422cc:::obj197:head local-lis/les=113/114 n=56 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.520+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.520+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.520+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:10b422cc:::obj197:head local-lis/les=113/114 n=56 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:10f4e680:::obj204:head, version: 111'204, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10f4e680:::obj204:head@111'204, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.520+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:10c2d3a1:::obj448:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.520+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:10b422cc:::obj197:head local-lis/les=113/114 n=56 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:10f4e680:::obj204:head@111'204, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.520+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:10b422cc:::obj197:head local-lis/les=113/114 n=56 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:10f4e680:::obj204:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.520+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:10b422cc:::obj197:head local-lis/les=113/114 n=56 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:10f4e680:::obj204:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.520+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:10b422cc:::obj197:head local-lis/les=113/114 n=56 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:10f4e680:::obj204:head 2026-02-11T23:31:27.520+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:10b422cc:::obj197:head local-lis/les=113/114 n=56 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:10f4e680:::obj204:head v 111'204 2026-02-11T23:31:27.520+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:10b422cc:::obj197:head local-lis/les=113/114 n=56 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.520+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.520+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:10f4e680:::obj204:head, version: 111'204, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10f4e680:::obj204:head@111'204, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:27.520+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:10b422cc:::obj197:head local-lis/les=113/114 n=56 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.520+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.520+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.520+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:10c2d3a1:::obj448:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.520+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:10c2d3a1:::obj448:head) v3 prio 15 cost 0 latency 0.000079 pg_backfill(progress 1.0 e 116/116 lb 1:10c2d3a1:::obj448:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:10b422cc:::obj197:head local-lis/les=113/114 n=56 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.520+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.520+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:10b422cc:::obj197:head local-lis/les=113/114 n=56 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:10c2d3a1:::obj448:head) v3 2026-02-11T23:31:27.520+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:10b422cc:::obj197:head local-lis/les=113/114 n=56 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:10c2d3a1:::obj448:head) 2026-02-11T23:31:27.520+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.520+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:10c2d3a1:::obj448:head) v3 finish latency 0.000020 2026-02-11T23:31:27.520+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.521+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.521+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:10c2d3a1:::obj448:head local-lis/les=113/114 n=57 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.521+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.521+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.521+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.521+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e3a480 con 0x4acb000 2026-02-11T23:31:27.521+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:10f4e680:::obj204:head)]) -- 0x5d79980 con 0x4acb000 2026-02-11T23:31:27.521+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.521+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 151 ==== MOSDPGPush(1.0 116/113 [PushOp(1:11426d25:::obj314:head, version: 111'314, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11426d25:::obj314:head@111'314, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db0340 con 0x4acb000 2026-02-11T23:31:27.521+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:11426d25:::obj314:head, version: 111'314, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11426d25:::obj314:head@111'314, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.521+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:11426d25:::obj314:head, version: 111'314, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11426d25:::obj314:head@111'314, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.521+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.521+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 152 ==== pg_backfill(progress 1.0 e 116/116 lb 1:10f4e680:::obj204:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc4680 con 0x4acb000 2026-02-11T23:31:27.521+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:10f4e680:::obj204:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.521+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:11426d25:::obj314:head, version: 111'314, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11426d25:::obj314:head@111'314, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.521+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:10f4e680:::obj204:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.521+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.521+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:11426d25:::obj314:head, version: 111'314, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11426d25:::obj314:head@111'314, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.521+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:11426d25:::obj314:head, version: 111'314, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11426d25:::obj314:head@111'314, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 116/113 [PushOp(1:11426d25:::obj314:head, version: 111'314, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11426d25:::obj314:head@111'314, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:10c2d3a1:::obj448:head local-lis/les=113/114 n=57 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.521+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.521+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:10c2d3a1:::obj448:head local-lis/les=113/114 n=57 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:11426d25:::obj314:head, version: 111'314, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11426d25:::obj314:head@111'314, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.521+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.521+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:10c2d3a1:::obj448:head local-lis/les=113/114 n=57 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:11426d25:::obj314:head@111'314, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.521+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:10f4e680:::obj204:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.521+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:10c2d3a1:::obj448:head local-lis/les=113/114 n=57 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:11426d25:::obj314:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.521+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:10c2d3a1:::obj448:head local-lis/les=113/114 n=57 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:11426d25:::obj314:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.521+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:10c2d3a1:::obj448:head local-lis/les=113/114 n=57 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:11426d25:::obj314:head 2026-02-11T23:31:27.521+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:10c2d3a1:::obj448:head local-lis/les=113/114 n=57 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:11426d25:::obj314:head v 111'314 2026-02-11T23:31:27.521+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:10c2d3a1:::obj448:head local-lis/les=113/114 n=57 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.521+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.521+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:11426d25:::obj314:head, version: 111'314, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11426d25:::obj314:head@111'314, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:27.521+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:10c2d3a1:::obj448:head local-lis/les=113/114 n=57 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.521+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.521+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.521+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:10f4e680:::obj204:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.521+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:10f4e680:::obj204:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 1.0 e 116/116 lb 1:10f4e680:::obj204:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:10c2d3a1:::obj448:head local-lis/les=113/114 n=57 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.521+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.521+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:10c2d3a1:::obj448:head local-lis/les=113/114 n=57 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:10f4e680:::obj204:head) v3 2026-02-11T23:31:27.521+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:10c2d3a1:::obj448:head local-lis/les=113/114 n=57 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:10f4e680:::obj204:head) 2026-02-11T23:31:27.521+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.521+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:10f4e680:::obj204:head) v3 finish latency 0.000014 2026-02-11T23:31:27.521+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.521+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.521+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:10f4e680:::obj204:head local-lis/les=113/114 n=58 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.521+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.521+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.521+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.521+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e3b680 con 0x4acb000 2026-02-11T23:31:27.521+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:11426d25:::obj314:head)]) -- 0x5e3ad80 con 0x4acb000 2026-02-11T23:31:27.521+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.522+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 153 ==== MOSDPGPush(1.0 116/113 [PushOp(1:11a85aa0:::obj43:head, version: 111'43, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11a85aa0:::obj43:head@111'43, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5db1a00 con 0x4acb000 2026-02-11T23:31:27.522+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:11a85aa0:::obj43:head, version: 111'43, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11a85aa0:::obj43:head@111'43, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.522+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:11a85aa0:::obj43:head, version: 111'43, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11a85aa0:::obj43:head@111'43, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.522+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.522+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 154 ==== pg_backfill(progress 1.0 e 116/116 lb 1:11426d25:::obj314:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e680 con 0x4acb000 2026-02-11T23:31:27.522+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:11426d25:::obj314:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.522+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:11a85aa0:::obj43:head, version: 111'43, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11a85aa0:::obj43:head@111'43, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.522+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:11426d25:::obj314:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.522+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.522+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:11a85aa0:::obj43:head, version: 111'43, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11a85aa0:::obj43:head@111'43, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.522+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:11a85aa0:::obj43:head, version: 111'43, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11a85aa0:::obj43:head@111'43, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:11a85aa0:::obj43:head, version: 111'43, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11a85aa0:::obj43:head@111'43, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:10f4e680:::obj204:head local-lis/les=113/114 n=58 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.522+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.522+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.522+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:11426d25:::obj314:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.522+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:10f4e680:::obj204:head local-lis/les=113/114 n=58 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:11a85aa0:::obj43:head, version: 111'43, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11a85aa0:::obj43:head@111'43, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.522+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:10f4e680:::obj204:head local-lis/les=113/114 n=58 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:11a85aa0:::obj43:head@111'43, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.522+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:10f4e680:::obj204:head local-lis/les=113/114 n=58 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:11a85aa0:::obj43:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.522+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:10f4e680:::obj204:head local-lis/les=113/114 n=58 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:11a85aa0:::obj43:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.522+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:10f4e680:::obj204:head local-lis/les=113/114 n=58 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:11a85aa0:::obj43:head 2026-02-11T23:31:27.522+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:10f4e680:::obj204:head local-lis/les=113/114 n=58 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:11a85aa0:::obj43:head v 111'43 2026-02-11T23:31:27.522+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:10f4e680:::obj204:head local-lis/les=113/114 n=58 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.522+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.522+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:11a85aa0:::obj43:head, version: 111'43, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11a85aa0:::obj43:head@111'43, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:27.522+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.522+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.522+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:11426d25:::obj314:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.522+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:11426d25:::obj314:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 1.0 e 116/116 lb 1:11426d25:::obj314:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:10f4e680:::obj204:head local-lis/les=113/114 n=58 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.522+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.522+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:10f4e680:::obj204:head local-lis/les=113/114 n=58 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:11426d25:::obj314:head) v3 2026-02-11T23:31:27.522+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:10f4e680:::obj204:head local-lis/les=113/114 n=58 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:11426d25:::obj314:head) 2026-02-11T23:31:27.522+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.522+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:11426d25:::obj314:head) v3 finish latency 0.000016 2026-02-11T23:31:27.522+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:11426d25:::obj314:head local-lis/les=113/114 n=59 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.522+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.522+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.522+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:11426d25:::obj314:head local-lis/les=113/114 n=59 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.522+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.522+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.522+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.522+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e3a180 con 0x4acb000 2026-02-11T23:31:27.522+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:11a85aa0:::obj43:head)]) -- 0x5e6b200 con 0x4acb000 2026-02-11T23:31:27.522+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.522+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 155 ==== MOSDPGPush(1.0 116/113 [PushOp(1:11bee109:::obj319:head, version: 111'319, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11bee109:::obj319:head@111'319, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db0d00 con 0x4acb000 2026-02-11T23:31:27.522+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:11bee109:::obj319:head, version: 111'319, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11bee109:::obj319:head@111'319, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.522+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:11bee109:::obj319:head, version: 111'319, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11bee109:::obj319:head@111'319, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.522+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.522+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 156 ==== pg_backfill(progress 1.0 e 116/116 lb 1:11a85aa0:::obj43:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d2e100 con 0x4acb000 2026-02-11T23:31:27.522+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:11bee109:::obj319:head, version: 111'319, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11bee109:::obj319:head@111'319, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.522+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:11a85aa0:::obj43:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.522+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:11a85aa0:::obj43:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.522+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.522+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:11bee109:::obj319:head, version: 111'319, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11bee109:::obj319:head@111'319, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.522+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:11bee109:::obj319:head, version: 111'319, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11bee109:::obj319:head@111'319, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(1.0 116/113 [PushOp(1:11bee109:::obj319:head, version: 111'319, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11bee109:::obj319:head@111'319, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:11426d25:::obj314:head local-lis/les=113/114 n=59 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.522+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.522+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.522+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:11426d25:::obj314:head local-lis/les=113/114 n=59 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:11bee109:::obj319:head, version: 111'319, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11bee109:::obj319:head@111'319, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.522+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:11a85aa0:::obj43:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.522+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:11426d25:::obj314:head local-lis/les=113/114 n=59 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:11bee109:::obj319:head@111'319, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.522+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:11426d25:::obj314:head local-lis/les=113/114 n=59 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:11bee109:::obj319:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.522+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:11426d25:::obj314:head local-lis/les=113/114 n=59 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:11bee109:::obj319:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.522+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:11426d25:::obj314:head local-lis/les=113/114 n=59 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:11bee109:::obj319:head 2026-02-11T23:31:27.522+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:11426d25:::obj314:head local-lis/les=113/114 n=59 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:11bee109:::obj319:head v 111'319 2026-02-11T23:31:27.522+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:11426d25:::obj314:head local-lis/les=113/114 n=59 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.522+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.522+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:11bee109:::obj319:head, version: 111'319, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11bee109:::obj319:head@111'319, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:27.522+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:11426d25:::obj314:head local-lis/les=113/114 n=59 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.522+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.522+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:11a85aa0:::obj43:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.522+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.522+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:11a85aa0:::obj43:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 1.0 e 116/116 lb 1:11a85aa0:::obj43:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:11426d25:::obj314:head local-lis/les=113/114 n=59 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.522+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.522+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:11426d25:::obj314:head local-lis/les=113/114 n=59 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:11a85aa0:::obj43:head) v3 2026-02-11T23:31:27.522+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:11426d25:::obj314:head local-lis/les=113/114 n=59 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:11a85aa0:::obj43:head) 2026-02-11T23:31:27.522+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.522+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:11a85aa0:::obj43:head) v3 finish latency 0.000021 2026-02-11T23:31:27.522+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.522+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.522+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:11a85aa0:::obj43:head local-lis/les=113/114 n=60 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.522+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.522+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.522+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.522+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x4ec2780 con 0x4acb000 2026-02-11T23:31:27.522+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:11bee109:::obj319:head)]) -- 0x5e3bc80 con 0x4acb000 2026-02-11T23:31:27.522+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.523+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 157 ==== MOSDPGPush(1.0 116/113 [PushOp(1:121cd428:::obj443:head, version: 111'443, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:121cd428:::obj443:head@111'443, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db16c0 con 0x4acb000 2026-02-11T23:31:27.523+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:121cd428:::obj443:head, version: 111'443, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:121cd428:::obj443:head@111'443, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.523+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:121cd428:::obj443:head, version: 111'443, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:121cd428:::obj443:head@111'443, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.523+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.523+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 158 ==== pg_backfill(progress 1.0 e 116/116 lb 1:11bee109:::obj319:head) ==== 1062+0+0 (unknown 0 0 0) 0x4a2cb00 con 0x4acb000 2026-02-11T23:31:27.523+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:121cd428:::obj443:head, version: 111'443, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:121cd428:::obj443:head@111'443, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.523+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:11bee109:::obj319:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.523+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:11bee109:::obj319:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.523+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.523+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:121cd428:::obj443:head, version: 111'443, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:121cd428:::obj443:head@111'443, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.523+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:121cd428:::obj443:head, version: 111'443, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:121cd428:::obj443:head@111'443, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 116/113 [PushOp(1:121cd428:::obj443:head, version: 111'443, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:121cd428:::obj443:head@111'443, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:11a85aa0:::obj43:head local-lis/les=113/114 n=60 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.523+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.523+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.523+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:11bee109:::obj319:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.523+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:11a85aa0:::obj43:head local-lis/les=113/114 n=60 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:121cd428:::obj443:head, version: 111'443, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:121cd428:::obj443:head@111'443, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.523+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:11a85aa0:::obj43:head local-lis/les=113/114 n=60 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:121cd428:::obj443:head@111'443, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.523+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:11a85aa0:::obj43:head local-lis/les=113/114 n=60 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:121cd428:::obj443:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.523+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:11a85aa0:::obj43:head local-lis/les=113/114 n=60 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:121cd428:::obj443:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.523+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:11a85aa0:::obj43:head local-lis/les=113/114 n=60 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:121cd428:::obj443:head 2026-02-11T23:31:27.523+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:11a85aa0:::obj43:head local-lis/les=113/114 n=60 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:121cd428:::obj443:head v 111'443 2026-02-11T23:31:27.523+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:11a85aa0:::obj43:head local-lis/les=113/114 n=60 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.523+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.523+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:121cd428:::obj443:head, version: 111'443, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:121cd428:::obj443:head@111'443, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:27.523+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:11a85aa0:::obj43:head local-lis/les=113/114 n=60 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.523+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.523+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:11bee109:::obj319:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.523+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:11bee109:::obj319:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 1.0 e 116/116 lb 1:11bee109:::obj319:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:11a85aa0:::obj43:head local-lis/les=113/114 n=60 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.523+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.523+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.523+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:11a85aa0:::obj43:head local-lis/les=113/114 n=60 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:11bee109:::obj319:head) v3 2026-02-11T23:31:27.523+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:11a85aa0:::obj43:head local-lis/les=113/114 n=60 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:11bee109:::obj319:head) 2026-02-11T23:31:27.523+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.523+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:11bee109:::obj319:head) v3 finish latency 0.000013 2026-02-11T23:31:27.523+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.523+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.523+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:11bee109:::obj319:head local-lis/les=113/114 n=61 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.523+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.523+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.523+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.523+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d78a80 con 0x4acb000 2026-02-11T23:31:27.523+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:121cd428:::obj443:head)]) -- 0x4ec2480 con 0x4acb000 2026-02-11T23:31:27.523+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.524+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 159 ==== MOSDPGPush(1.0 116/113 [PushOp(1:1243fe1a:::obj21:head, version: 111'21, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1243fe1a:::obj21:head@111'21, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5db0340 con 0x4acb000 2026-02-11T23:31:27.524+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:1243fe1a:::obj21:head, version: 111'21, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1243fe1a:::obj21:head@111'21, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 116 2026-02-11T23:31:27.524+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1243fe1a:::obj21:head, version: 111'21, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1243fe1a:::obj21:head@111'21, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.524+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.524+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 160 ==== pg_backfill(progress 1.0 e 116/116 lb 1:121cd428:::obj443:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc3b80 con 0x4acb000 2026-02-11T23:31:27.524+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:121cd428:::obj443:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.524+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1243fe1a:::obj21:head, version: 111'21, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1243fe1a:::obj21:head@111'21, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.524+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:121cd428:::obj443:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.524+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.524+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1243fe1a:::obj21:head, version: 111'21, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1243fe1a:::obj21:head@111'21, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.524+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:1243fe1a:::obj21:head, version: 111'21, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1243fe1a:::obj21:head@111'21, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 116/113 [PushOp(1:1243fe1a:::obj21:head, version: 111'21, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1243fe1a:::obj21:head@111'21, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:11bee109:::obj319:head local-lis/les=113/114 n=61 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.524+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.524+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.524+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:11bee109:::obj319:head local-lis/les=113/114 n=61 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:1243fe1a:::obj21:head, version: 111'21, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1243fe1a:::obj21:head@111'21, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.524+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:121cd428:::obj443:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.524+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:11bee109:::obj319:head local-lis/les=113/114 n=61 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:1243fe1a:::obj21:head@111'21, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.524+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:11bee109:::obj319:head local-lis/les=113/114 n=61 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:1243fe1a:::obj21:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.524+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:11bee109:::obj319:head local-lis/les=113/114 n=61 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:1243fe1a:::obj21:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.524+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:11bee109:::obj319:head local-lis/les=113/114 n=61 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:1243fe1a:::obj21:head 2026-02-11T23:31:27.524+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:11bee109:::obj319:head local-lis/les=113/114 n=61 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:1243fe1a:::obj21:head v 111'21 2026-02-11T23:31:27.524+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:11bee109:::obj319:head local-lis/les=113/114 n=61 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.524+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.524+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:1243fe1a:::obj21:head, version: 111'21, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1243fe1a:::obj21:head@111'21, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:27.524+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.524+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.524+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:121cd428:::obj443:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.524+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:121cd428:::obj443:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 1.0 e 116/116 lb 1:121cd428:::obj443:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:11bee109:::obj319:head local-lis/les=113/114 n=61 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.524+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.524+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:11bee109:::obj319:head local-lis/les=113/114 n=61 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:121cd428:::obj443:head) v3 2026-02-11T23:31:27.524+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:11bee109:::obj319:head local-lis/les=113/114 n=61 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:121cd428:::obj443:head) 2026-02-11T23:31:27.524+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.524+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:121cd428:::obj443:head) v3 finish latency 0.000024 2026-02-11T23:31:27.524+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:121cd428:::obj443:head local-lis/les=113/114 n=62 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.524+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.524+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.524+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:121cd428:::obj443:head local-lis/les=113/114 n=62 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.524+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.524+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.524+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.524+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d78780 con 0x4acb000 2026-02-11T23:31:27.524+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:1243fe1a:::obj21:head)]) -- 0x5e6b680 con 0x4acb000 2026-02-11T23:31:27.524+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.524+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 161 ==== MOSDPGPush(1.0 116/113 [PushOp(1:124a99b2:::obj215:head, version: 111'215, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:124a99b2:::obj215:head@111'215, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db1a00 con 0x4acb000 2026-02-11T23:31:27.524+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:124a99b2:::obj215:head, version: 111'215, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:124a99b2:::obj215:head@111'215, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.524+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:124a99b2:::obj215:head, version: 111'215, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:124a99b2:::obj215:head@111'215, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.524+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.524+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 162 ==== pg_backfill(progress 1.0 e 116/116 lb 1:1243fe1a:::obj21:head) ==== 1061+0+0 (unknown 0 0 0) 0x5dc4100 con 0x4acb000 2026-02-11T23:31:27.524+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:1243fe1a:::obj21:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.524+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:124a99b2:::obj215:head, version: 111'215, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:124a99b2:::obj215:head@111'215, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.524+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1243fe1a:::obj21:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.524+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.524+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:124a99b2:::obj215:head, version: 111'215, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:124a99b2:::obj215:head@111'215, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.524+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:124a99b2:::obj215:head, version: 111'215, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:124a99b2:::obj215:head@111'215, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 116/113 [PushOp(1:124a99b2:::obj215:head, version: 111'215, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:124a99b2:::obj215:head@111'215, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:121cd428:::obj443:head local-lis/les=113/114 n=62 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.524+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.524+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.524+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1243fe1a:::obj21:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.524+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:121cd428:::obj443:head local-lis/les=113/114 n=62 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:124a99b2:::obj215:head, version: 111'215, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:124a99b2:::obj215:head@111'215, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.524+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:121cd428:::obj443:head local-lis/les=113/114 n=62 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:124a99b2:::obj215:head@111'215, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.524+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:121cd428:::obj443:head local-lis/les=113/114 n=62 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:124a99b2:::obj215:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.524+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:121cd428:::obj443:head local-lis/les=113/114 n=62 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:124a99b2:::obj215:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.524+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:121cd428:::obj443:head local-lis/les=113/114 n=62 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:124a99b2:::obj215:head 2026-02-11T23:31:27.524+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:121cd428:::obj443:head local-lis/les=113/114 n=62 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:124a99b2:::obj215:head v 111'215 2026-02-11T23:31:27.524+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:121cd428:::obj443:head local-lis/les=113/114 n=62 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.524+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.524+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:124a99b2:::obj215:head, version: 111'215, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:124a99b2:::obj215:head@111'215, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:27.524+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:121cd428:::obj443:head local-lis/les=113/114 n=62 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.524+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.524+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.524+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1243fe1a:::obj21:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.524+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:1243fe1a:::obj21:head) v3 prio 15 cost 0 latency 0.000079 pg_backfill(progress 1.0 e 116/116 lb 1:1243fe1a:::obj21:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:121cd428:::obj443:head local-lis/les=113/114 n=62 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.524+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.524+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:121cd428:::obj443:head local-lis/les=113/114 n=62 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:1243fe1a:::obj21:head) v3 2026-02-11T23:31:27.524+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:121cd428:::obj443:head local-lis/les=113/114 n=62 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:1243fe1a:::obj21:head) 2026-02-11T23:31:27.524+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.524+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:1243fe1a:::obj21:head) v3 finish latency 0.000016 2026-02-11T23:31:27.524+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.524+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.524+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1243fe1a:::obj21:head local-lis/les=113/114 n=63 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.524+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.524+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.524+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.524+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d78d80 con 0x4acb000 2026-02-11T23:31:27.524+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:124a99b2:::obj215:head)]) -- 0x5d78c00 con 0x4acb000 2026-02-11T23:31:27.524+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.525+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 163 ==== MOSDPGPush(1.0 116/113 [PushOp(1:1405054b:::obj129:head, version: 111'129, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1405054b:::obj129:head@111'129, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e264e0 con 0x4acb000 2026-02-11T23:31:27.525+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:1405054b:::obj129:head, version: 111'129, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1405054b:::obj129:head@111'129, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 116 2026-02-11T23:31:27.525+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1405054b:::obj129:head, version: 111'129, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1405054b:::obj129:head@111'129, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.525+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.525+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 164 ==== pg_backfill(progress 1.0 e 116/116 lb 1:124a99b2:::obj215:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc2b00 con 0x4acb000 2026-02-11T23:31:27.525+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1405054b:::obj129:head, version: 111'129, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1405054b:::obj129:head@111'129, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.525+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:124a99b2:::obj215:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.525+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:124a99b2:::obj215:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.525+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.525+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1405054b:::obj129:head, version: 111'129, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1405054b:::obj129:head@111'129, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.525+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:1405054b:::obj129:head, version: 111'129, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1405054b:::obj129:head@111'129, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 116/113 [PushOp(1:1405054b:::obj129:head, version: 111'129, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1405054b:::obj129:head@111'129, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:1243fe1a:::obj21:head local-lis/les=113/114 n=63 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.525+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.525+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.525+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1243fe1a:::obj21:head local-lis/les=113/114 n=63 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:1405054b:::obj129:head, version: 111'129, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1405054b:::obj129:head@111'129, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.525+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:124a99b2:::obj215:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.525+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1243fe1a:::obj21:head local-lis/les=113/114 n=63 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:1405054b:::obj129:head@111'129, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.525+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1243fe1a:::obj21:head local-lis/les=113/114 n=63 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:1405054b:::obj129:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.525+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1243fe1a:::obj21:head local-lis/les=113/114 n=63 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:1405054b:::obj129:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.525+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1243fe1a:::obj21:head local-lis/les=113/114 n=63 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:1405054b:::obj129:head 2026-02-11T23:31:27.525+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1243fe1a:::obj21:head local-lis/les=113/114 n=63 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:1405054b:::obj129:head v 111'129 2026-02-11T23:31:27.525+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1243fe1a:::obj21:head local-lis/les=113/114 n=63 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.525+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.525+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:1405054b:::obj129:head, version: 111'129, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1405054b:::obj129:head@111'129, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:27.525+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.525+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.525+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:124a99b2:::obj215:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.525+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:124a99b2:::obj215:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 1.0 e 116/116 lb 1:124a99b2:::obj215:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:1243fe1a:::obj21:head local-lis/les=113/114 n=63 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.525+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.525+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1243fe1a:::obj21:head local-lis/les=113/114 n=63 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:124a99b2:::obj215:head) v3 2026-02-11T23:31:27.525+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1243fe1a:::obj21:head local-lis/les=113/114 n=63 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:124a99b2:::obj215:head) 2026-02-11T23:31:27.525+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.525+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:124a99b2:::obj215:head) v3 finish latency 0.000018 2026-02-11T23:31:27.525+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:124a99b2:::obj215:head local-lis/les=113/114 n=64 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.525+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.525+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.525+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:124a99b2:::obj215:head local-lis/les=113/114 n=64 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.525+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.525+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.525+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.525+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d79380 con 0x4acb000 2026-02-11T23:31:27.525+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:1405054b:::obj129:head)]) -- 0x5e6b380 con 0x4acb000 2026-02-11T23:31:27.525+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.526+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 165 ==== MOSDPGPush(1.0 116/113 [PushOp(1:147354e0:::obj436:head, version: 111'436, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:147354e0:::obj436:head@111'436, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e27380 con 0x4acb000 2026-02-11T23:31:27.526+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:147354e0:::obj436:head, version: 111'436, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:147354e0:::obj436:head@111'436, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.526+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:147354e0:::obj436:head, version: 111'436, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:147354e0:::obj436:head@111'436, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.526+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.526+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 166 ==== pg_backfill(progress 1.0 e 116/116 lb 1:1405054b:::obj129:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc5700 con 0x4acb000 2026-02-11T23:31:27.526+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:1405054b:::obj129:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.526+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:147354e0:::obj436:head, version: 111'436, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:147354e0:::obj436:head@111'436, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.526+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1405054b:::obj129:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.526+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.526+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:147354e0:::obj436:head, version: 111'436, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:147354e0:::obj436:head@111'436, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.526+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.526+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1405054b:::obj129:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.526+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:147354e0:::obj436:head, version: 111'436, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:147354e0:::obj436:head@111'436, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:147354e0:::obj436:head, version: 111'436, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:147354e0:::obj436:head@111'436, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:124a99b2:::obj215:head local-lis/les=113/114 n=64 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.526+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.526+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:124a99b2:::obj215:head local-lis/les=113/114 n=64 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:147354e0:::obj436:head, version: 111'436, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:147354e0:::obj436:head@111'436, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.526+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:124a99b2:::obj215:head local-lis/les=113/114 n=64 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:147354e0:::obj436:head@111'436, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.526+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:124a99b2:::obj215:head local-lis/les=113/114 n=64 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:147354e0:::obj436:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.526+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:124a99b2:::obj215:head local-lis/les=113/114 n=64 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:147354e0:::obj436:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.526+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:124a99b2:::obj215:head local-lis/les=113/114 n=64 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:147354e0:::obj436:head 2026-02-11T23:31:27.526+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:124a99b2:::obj215:head local-lis/les=113/114 n=64 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:147354e0:::obj436:head v 111'436 2026-02-11T23:31:27.526+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:124a99b2:::obj215:head local-lis/les=113/114 n=64 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.526+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.526+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:147354e0:::obj436:head, version: 111'436, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:147354e0:::obj436:head@111'436, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:27.526+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:124a99b2:::obj215:head local-lis/les=113/114 n=64 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.526+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.526+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.526+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1405054b:::obj129:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.526+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:1405054b:::obj129:head) v3 prio 15 cost 0 latency 0.000081 pg_backfill(progress 1.0 e 116/116 lb 1:1405054b:::obj129:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:124a99b2:::obj215:head local-lis/les=113/114 n=64 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.526+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.526+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:124a99b2:::obj215:head local-lis/les=113/114 n=64 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:1405054b:::obj129:head) v3 2026-02-11T23:31:27.526+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:124a99b2:::obj215:head local-lis/les=113/114 n=64 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:1405054b:::obj129:head) 2026-02-11T23:31:27.526+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.526+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:1405054b:::obj129:head) v3 finish latency 0.000016 2026-02-11T23:31:27.526+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.526+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.526+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1405054b:::obj129:head local-lis/les=113/114 n=65 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.526+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.526+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.526+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.526+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d79500 con 0x4acb000 2026-02-11T23:31:27.526+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:147354e0:::obj436:head)]) -- 0x5d79200 con 0x4acb000 2026-02-11T23:31:27.526+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.526+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 167 ==== MOSDPGPush(1.0 116/113 [PushOp(1:147b5010:::obj178:head, version: 111'178, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:147b5010:::obj178:head@111'178, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e26d00 con 0x4acb000 2026-02-11T23:31:27.526+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:147b5010:::obj178:head, version: 111'178, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:147b5010:::obj178:head@111'178, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.526+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:147b5010:::obj178:head, version: 111'178, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:147b5010:::obj178:head@111'178, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.526+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.526+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 168 ==== pg_backfill(progress 1.0 e 116/116 lb 1:147354e0:::obj436:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc2580 con 0x4acb000 2026-02-11T23:31:27.526+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:147b5010:::obj178:head, version: 111'178, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:147b5010:::obj178:head@111'178, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.526+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:147354e0:::obj436:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 116 2026-02-11T23:31:27.526+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:147354e0:::obj436:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.526+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.526+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:147b5010:::obj178:head, version: 111'178, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:147b5010:::obj178:head@111'178, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.526+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:147b5010:::obj178:head, version: 111'178, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:147b5010:::obj178:head@111'178, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 116/113 [PushOp(1:147b5010:::obj178:head, version: 111'178, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:147b5010:::obj178:head@111'178, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:1405054b:::obj129:head local-lis/les=113/114 n=65 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.526+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.526+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.526+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1405054b:::obj129:head local-lis/les=113/114 n=65 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:147b5010:::obj178:head, version: 111'178, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:147b5010:::obj178:head@111'178, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.526+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:147354e0:::obj436:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.526+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1405054b:::obj129:head local-lis/les=113/114 n=65 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:147b5010:::obj178:head@111'178, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.526+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1405054b:::obj129:head local-lis/les=113/114 n=65 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:147b5010:::obj178:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.526+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1405054b:::obj129:head local-lis/les=113/114 n=65 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:147b5010:::obj178:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.526+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1405054b:::obj129:head local-lis/les=113/114 n=65 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:147b5010:::obj178:head 2026-02-11T23:31:27.526+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1405054b:::obj129:head local-lis/les=113/114 n=65 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:147b5010:::obj178:head v 111'178 2026-02-11T23:31:27.526+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1405054b:::obj129:head local-lis/les=113/114 n=65 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.526+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.526+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:147b5010:::obj178:head, version: 111'178, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:147b5010:::obj178:head@111'178, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:27.526+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1405054b:::obj129:head local-lis/les=113/114 n=65 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.526+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.526+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.526+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:147354e0:::obj436:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.526+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:147354e0:::obj436:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 1.0 e 116/116 lb 1:147354e0:::obj436:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:1405054b:::obj129:head local-lis/les=113/114 n=65 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.526+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.526+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1405054b:::obj129:head local-lis/les=113/114 n=65 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:147354e0:::obj436:head) v3 2026-02-11T23:31:27.526+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1405054b:::obj129:head local-lis/les=113/114 n=65 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:147354e0:::obj436:head) 2026-02-11T23:31:27.526+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.526+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:147354e0:::obj436:head) v3 finish latency 0.000012 2026-02-11T23:31:27.526+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.526+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.526+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:147354e0:::obj436:head local-lis/les=113/114 n=66 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.526+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.526+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.526+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.526+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e6ad80 con 0x4acb000 2026-02-11T23:31:27.526+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:147b5010:::obj178:head)]) -- 0x4ec2780 con 0x4acb000 2026-02-11T23:31:27.526+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.527+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 169 ==== MOSDPGPush(1.0 116/113 [PushOp(1:1503467e:::obj118:head, version: 111'118, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1503467e:::obj118:head@111'118, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e264e0 con 0x4acb000 2026-02-11T23:31:27.527+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:1503467e:::obj118:head, version: 111'118, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1503467e:::obj118:head@111'118, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.527+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1503467e:::obj118:head, version: 111'118, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1503467e:::obj118:head@111'118, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.527+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.527+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 170 ==== pg_backfill(progress 1.0 e 116/116 lb 1:147b5010:::obj178:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc3080 con 0x4acb000 2026-02-11T23:31:27.527+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1503467e:::obj118:head, version: 111'118, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1503467e:::obj118:head@111'118, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.527+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:147b5010:::obj178:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 116 2026-02-11T23:31:27.527+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:147b5010:::obj178:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.527+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.527+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1503467e:::obj118:head, version: 111'118, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1503467e:::obj118:head@111'118, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.527+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:1503467e:::obj118:head, version: 111'118, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1503467e:::obj118:head@111'118, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 116/113 [PushOp(1:1503467e:::obj118:head, version: 111'118, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1503467e:::obj118:head@111'118, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:147354e0:::obj436:head local-lis/les=113/114 n=66 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.527+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.527+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.527+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:147b5010:::obj178:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.527+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:147354e0:::obj436:head local-lis/les=113/114 n=66 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:1503467e:::obj118:head, version: 111'118, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1503467e:::obj118:head@111'118, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.527+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:147354e0:::obj436:head local-lis/les=113/114 n=66 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:1503467e:::obj118:head@111'118, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.527+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:147354e0:::obj436:head local-lis/les=113/114 n=66 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:1503467e:::obj118:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.527+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:147354e0:::obj436:head local-lis/les=113/114 n=66 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:1503467e:::obj118:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.527+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:147354e0:::obj436:head local-lis/les=113/114 n=66 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:1503467e:::obj118:head 2026-02-11T23:31:27.527+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:147354e0:::obj436:head local-lis/les=113/114 n=66 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:1503467e:::obj118:head v 111'118 2026-02-11T23:31:27.527+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:147354e0:::obj436:head local-lis/les=113/114 n=66 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.527+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.527+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:1503467e:::obj118:head, version: 111'118, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1503467e:::obj118:head@111'118, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:27.527+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:147354e0:::obj436:head local-lis/les=113/114 n=66 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.527+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.527+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.527+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:147b5010:::obj178:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.527+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:147b5010:::obj178:head) v3 prio 15 cost 0 latency 0.000081 pg_backfill(progress 1.0 e 116/116 lb 1:147b5010:::obj178:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:147354e0:::obj436:head local-lis/les=113/114 n=66 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.527+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.527+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:147354e0:::obj436:head local-lis/les=113/114 n=66 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:147b5010:::obj178:head) v3 2026-02-11T23:31:27.527+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:147354e0:::obj436:head local-lis/les=113/114 n=66 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:147b5010:::obj178:head) 2026-02-11T23:31:27.527+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.527+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:147b5010:::obj178:head) v3 finish latency 0.000014 2026-02-11T23:31:27.527+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.527+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.527+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:147b5010:::obj178:head local-lis/les=113/114 n=67 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.527+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.527+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.527+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.527+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e3a480 con 0x4acb000 2026-02-11T23:31:27.527+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:1503467e:::obj118:head)]) -- 0x5e6a600 con 0x4acb000 2026-02-11T23:31:27.527+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.527+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 171 ==== MOSDPGPush(1.0 116/113 [PushOp(1:1547e7aa:::obj466:head, version: 111'466, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1547e7aa:::obj466:head@111'466, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e27380 con 0x4acb000 2026-02-11T23:31:27.527+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:1547e7aa:::obj466:head, version: 111'466, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1547e7aa:::obj466:head@111'466, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.527+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1547e7aa:::obj466:head, version: 111'466, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1547e7aa:::obj466:head@111'466, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.527+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.527+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 172 ==== pg_backfill(progress 1.0 e 116/116 lb 1:1503467e:::obj118:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc3600 con 0x4acb000 2026-02-11T23:31:27.527+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1547e7aa:::obj466:head, version: 111'466, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1547e7aa:::obj466:head@111'466, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.527+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:1503467e:::obj118:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 116 2026-02-11T23:31:27.527+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1503467e:::obj118:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.527+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.527+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1547e7aa:::obj466:head, version: 111'466, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1547e7aa:::obj466:head@111'466, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.527+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:1547e7aa:::obj466:head, version: 111'466, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1547e7aa:::obj466:head@111'466, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 116/113 [PushOp(1:1547e7aa:::obj466:head, version: 111'466, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1547e7aa:::obj466:head@111'466, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:147b5010:::obj178:head local-lis/les=113/114 n=67 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.527+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.527+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.527+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1503467e:::obj118:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.527+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:147b5010:::obj178:head local-lis/les=113/114 n=67 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:1547e7aa:::obj466:head, version: 111'466, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1547e7aa:::obj466:head@111'466, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.528+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:147b5010:::obj178:head local-lis/les=113/114 n=67 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:1547e7aa:::obj466:head@111'466, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.528+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:147b5010:::obj178:head local-lis/les=113/114 n=67 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:1547e7aa:::obj466:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.528+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:147b5010:::obj178:head local-lis/les=113/114 n=67 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:1547e7aa:::obj466:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.528+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:147b5010:::obj178:head local-lis/les=113/114 n=67 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:1547e7aa:::obj466:head 2026-02-11T23:31:27.528+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:147b5010:::obj178:head local-lis/les=113/114 n=67 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:1547e7aa:::obj466:head v 111'466 2026-02-11T23:31:27.528+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:147b5010:::obj178:head local-lis/les=113/114 n=67 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.528+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.528+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:1547e7aa:::obj466:head, version: 111'466, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1547e7aa:::obj466:head@111'466, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:27.528+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.528+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.528+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1503467e:::obj118:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.528+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:1503467e:::obj118:head) v3 prio 15 cost 0 latency 0.000082 pg_backfill(progress 1.0 e 116/116 lb 1:1503467e:::obj118:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:147b5010:::obj178:head local-lis/les=113/114 n=67 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.528+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.528+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:147b5010:::obj178:head local-lis/les=113/114 n=67 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:1503467e:::obj118:head) v3 2026-02-11T23:31:27.528+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:147b5010:::obj178:head local-lis/les=113/114 n=67 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:1503467e:::obj118:head) 2026-02-11T23:31:27.528+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.528+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:1503467e:::obj118:head) v3 finish latency 0.000018 2026-02-11T23:31:27.528+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1503467e:::obj118:head local-lis/les=113/114 n=68 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.528+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.528+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.528+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1503467e:::obj118:head local-lis/les=113/114 n=68 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.528+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.528+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.528+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.528+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e3b680 con 0x4acb000 2026-02-11T23:31:27.528+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:1547e7aa:::obj466:head)]) -- 0x5e6b980 con 0x4acb000 2026-02-11T23:31:27.528+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.528+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 173 ==== MOSDPGPush(1.0 116/113 [PushOp(1:170a838d:::obj199:head, version: 111'199, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:170a838d:::obj199:head@111'199, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db0d00 con 0x4acb000 2026-02-11T23:31:27.528+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:170a838d:::obj199:head, version: 111'199, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:170a838d:::obj199:head@111'199, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 116 2026-02-11T23:31:27.528+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:170a838d:::obj199:head, version: 111'199, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:170a838d:::obj199:head@111'199, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.528+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.528+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 174 ==== pg_backfill(progress 1.0 e 116/116 lb 1:1547e7aa:::obj466:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc4c00 con 0x4acb000 2026-02-11T23:31:27.528+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:1547e7aa:::obj466:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.528+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:170a838d:::obj199:head, version: 111'199, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:170a838d:::obj199:head@111'199, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.528+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1547e7aa:::obj466:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.528+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.528+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:170a838d:::obj199:head, version: 111'199, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:170a838d:::obj199:head@111'199, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.528+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:170a838d:::obj199:head, version: 111'199, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:170a838d:::obj199:head@111'199, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:170a838d:::obj199:head, version: 111'199, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:170a838d:::obj199:head@111'199, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:1503467e:::obj118:head local-lis/les=113/114 n=68 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.528+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.528+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.528+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1503467e:::obj118:head local-lis/les=113/114 n=68 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:170a838d:::obj199:head, version: 111'199, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:170a838d:::obj199:head@111'199, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.528+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1547e7aa:::obj466:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.528+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1503467e:::obj118:head local-lis/les=113/114 n=68 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:170a838d:::obj199:head@111'199, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.528+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1503467e:::obj118:head local-lis/les=113/114 n=68 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:170a838d:::obj199:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.528+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1503467e:::obj118:head local-lis/les=113/114 n=68 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:170a838d:::obj199:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.528+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1503467e:::obj118:head local-lis/les=113/114 n=68 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:170a838d:::obj199:head 2026-02-11T23:31:27.528+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1503467e:::obj118:head local-lis/les=113/114 n=68 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:170a838d:::obj199:head v 111'199 2026-02-11T23:31:27.528+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1503467e:::obj118:head local-lis/les=113/114 n=68 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.528+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.528+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:170a838d:::obj199:head, version: 111'199, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:170a838d:::obj199:head@111'199, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:27.528+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1503467e:::obj118:head local-lis/les=113/114 n=68 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.528+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.528+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.528+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1547e7aa:::obj466:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.528+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:1547e7aa:::obj466:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 1.0 e 116/116 lb 1:1547e7aa:::obj466:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:1503467e:::obj118:head local-lis/les=113/114 n=68 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.528+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.528+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1503467e:::obj118:head local-lis/les=113/114 n=68 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:1547e7aa:::obj466:head) v3 2026-02-11T23:31:27.528+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1503467e:::obj118:head local-lis/les=113/114 n=68 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:1547e7aa:::obj466:head) 2026-02-11T23:31:27.528+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.528+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:1547e7aa:::obj466:head) v3 finish latency 0.000019 2026-02-11T23:31:27.528+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.528+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.528+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1547e7aa:::obj466:head local-lis/les=113/114 n=69 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.528+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.528+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.528+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.528+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e7e000 con 0x4acb000 2026-02-11T23:31:27.528+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:170a838d:::obj199:head)]) -- 0x5e3ad80 con 0x4acb000 2026-02-11T23:31:27.528+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.529+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 175 ==== MOSDPGPush(1.0 116/113 [PushOp(1:171b715e:::obj570:head, version: 111'570, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:171b715e:::obj570:head@111'570, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db01a0 con 0x4acb000 2026-02-11T23:31:27.529+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:171b715e:::obj570:head, version: 111'570, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:171b715e:::obj570:head@111'570, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.529+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:171b715e:::obj570:head, version: 111'570, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:171b715e:::obj570:head@111'570, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.529+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.529+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 176 ==== pg_backfill(progress 1.0 e 116/116 lb 1:170a838d:::obj199:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2d080 con 0x4acb000 2026-02-11T23:31:27.529+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:171b715e:::obj570:head, version: 111'570, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:171b715e:::obj570:head@111'570, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.529+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:170a838d:::obj199:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 116 2026-02-11T23:31:27.529+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:170a838d:::obj199:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.529+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.529+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:171b715e:::obj570:head, version: 111'570, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:171b715e:::obj570:head@111'570, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.529+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:171b715e:::obj570:head, version: 111'570, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:171b715e:::obj570:head@111'570, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:171b715e:::obj570:head, version: 111'570, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:171b715e:::obj570:head@111'570, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:1547e7aa:::obj466:head local-lis/les=113/114 n=69 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.529+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.529+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.529+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:170a838d:::obj199:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.529+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1547e7aa:::obj466:head local-lis/les=113/114 n=69 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:171b715e:::obj570:head, version: 111'570, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:171b715e:::obj570:head@111'570, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.529+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1547e7aa:::obj466:head local-lis/les=113/114 n=69 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:171b715e:::obj570:head@111'570, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.529+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1547e7aa:::obj466:head local-lis/les=113/114 n=69 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:171b715e:::obj570:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.529+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1547e7aa:::obj466:head local-lis/les=113/114 n=69 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:171b715e:::obj570:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.529+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1547e7aa:::obj466:head local-lis/les=113/114 n=69 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:171b715e:::obj570:head 2026-02-11T23:31:27.529+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1547e7aa:::obj466:head local-lis/les=113/114 n=69 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:171b715e:::obj570:head v 111'570 2026-02-11T23:31:27.529+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1547e7aa:::obj466:head local-lis/les=113/114 n=69 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.529+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.529+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:171b715e:::obj570:head, version: 111'570, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:171b715e:::obj570:head@111'570, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:27.529+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1547e7aa:::obj466:head local-lis/les=113/114 n=69 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.529+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.529+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:170a838d:::obj199:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.529+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:170a838d:::obj199:head) v3 prio 15 cost 0 latency 0.000080 pg_backfill(progress 1.0 e 116/116 lb 1:170a838d:::obj199:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:1547e7aa:::obj466:head local-lis/les=113/114 n=69 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.529+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.529+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.529+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1547e7aa:::obj466:head local-lis/les=113/114 n=69 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:170a838d:::obj199:head) v3 2026-02-11T23:31:27.529+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1547e7aa:::obj466:head local-lis/les=113/114 n=69 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:170a838d:::obj199:head) 2026-02-11T23:31:27.529+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.529+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:170a838d:::obj199:head) v3 finish latency 0.000015 2026-02-11T23:31:27.529+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.529+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.529+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:170a838d:::obj199:head local-lis/les=113/114 n=70 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.529+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.529+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.529+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.529+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e7e480 con 0x4acb000 2026-02-11T23:31:27.529+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:171b715e:::obj570:head)]) -- 0x5e7e180 con 0x4acb000 2026-02-11T23:31:27.529+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.529+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 177 ==== MOSDPGPush(1.0 116/113 [PushOp(1:17371e94:::obj362:head, version: 111'362, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:17371e94:::obj362:head@111'362, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db1860 con 0x4acb000 2026-02-11T23:31:27.529+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:17371e94:::obj362:head, version: 111'362, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:17371e94:::obj362:head@111'362, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.529+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:17371e94:::obj362:head, version: 111'362, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:17371e94:::obj362:head@111'362, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.529+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.529+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 178 ==== pg_backfill(progress 1.0 e 116/116 lb 1:171b715e:::obj570:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc4680 con 0x4acb000 2026-02-11T23:31:27.529+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:171b715e:::obj570:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.529+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:17371e94:::obj362:head, version: 111'362, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:17371e94:::obj362:head@111'362, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.529+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:171b715e:::obj570:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.529+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.529+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:17371e94:::obj362:head, version: 111'362, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:17371e94:::obj362:head@111'362, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.529+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:17371e94:::obj362:head, version: 111'362, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:17371e94:::obj362:head@111'362, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:17371e94:::obj362:head, version: 111'362, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:17371e94:::obj362:head@111'362, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:170a838d:::obj199:head local-lis/les=113/114 n=70 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.529+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.529+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.529+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:170a838d:::obj199:head local-lis/les=113/114 n=70 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:17371e94:::obj362:head, version: 111'362, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:17371e94:::obj362:head@111'362, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.529+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:171b715e:::obj570:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.529+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:170a838d:::obj199:head local-lis/les=113/114 n=70 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:17371e94:::obj362:head@111'362, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.529+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:170a838d:::obj199:head local-lis/les=113/114 n=70 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:17371e94:::obj362:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.529+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:170a838d:::obj199:head local-lis/les=113/114 n=70 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:17371e94:::obj362:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.529+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:170a838d:::obj199:head local-lis/les=113/114 n=70 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:17371e94:::obj362:head 2026-02-11T23:31:27.529+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:170a838d:::obj199:head local-lis/les=113/114 n=70 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:17371e94:::obj362:head v 111'362 2026-02-11T23:31:27.529+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:170a838d:::obj199:head local-lis/les=113/114 n=70 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.529+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.529+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:17371e94:::obj362:head, version: 111'362, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:17371e94:::obj362:head@111'362, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:27.529+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:170a838d:::obj199:head local-lis/les=113/114 n=70 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.529+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.529+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.529+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:171b715e:::obj570:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.529+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:171b715e:::obj570:head) v3 prio 15 cost 0 latency 0.000080 pg_backfill(progress 1.0 e 116/116 lb 1:171b715e:::obj570:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:170a838d:::obj199:head local-lis/les=113/114 n=70 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.529+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.529+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:170a838d:::obj199:head local-lis/les=113/114 n=70 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:171b715e:::obj570:head) v3 2026-02-11T23:31:27.529+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:170a838d:::obj199:head local-lis/les=113/114 n=70 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:171b715e:::obj570:head) 2026-02-11T23:31:27.529+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.530+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:171b715e:::obj570:head) v3 finish latency 0.000017 2026-02-11T23:31:27.530+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.530+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.530+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:171b715e:::obj570:head local-lis/les=113/114 n=71 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.530+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.530+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.530+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.530+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e7e900 con 0x4acb000 2026-02-11T23:31:27.530+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:17371e94:::obj362:head)]) -- 0x5e7e600 con 0x4acb000 2026-02-11T23:31:27.530+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.530+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 179 ==== MOSDPGPush(1.0 116/113 [PushOp(1:18100969:::obj255:head, version: 111'255, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18100969:::obj255:head@111'255, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3815d40 con 0x4acb000 2026-02-11T23:31:27.530+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:18100969:::obj255:head, version: 111'255, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18100969:::obj255:head@111'255, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.530+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:18100969:::obj255:head, version: 111'255, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18100969:::obj255:head@111'255, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.530+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.530+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 180 ==== pg_backfill(progress 1.0 e 116/116 lb 1:17371e94:::obj362:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc3b80 con 0x4acb000 2026-02-11T23:31:27.530+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:18100969:::obj255:head, version: 111'255, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18100969:::obj255:head@111'255, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.530+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:17371e94:::obj362:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 116 2026-02-11T23:31:27.530+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.530+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:17371e94:::obj362:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.530+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:18100969:::obj255:head, version: 111'255, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18100969:::obj255:head@111'255, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.530+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.530+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:18100969:::obj255:head, version: 111'255, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18100969:::obj255:head@111'255, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:18100969:::obj255:head, version: 111'255, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18100969:::obj255:head@111'255, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:171b715e:::obj570:head local-lis/les=113/114 n=71 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.530+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.530+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:171b715e:::obj570:head local-lis/les=113/114 n=71 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:18100969:::obj255:head, version: 111'255, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18100969:::obj255:head@111'255, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.530+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.530+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:171b715e:::obj570:head local-lis/les=113/114 n=71 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:18100969:::obj255:head@111'255, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.530+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:17371e94:::obj362:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.530+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:171b715e:::obj570:head local-lis/les=113/114 n=71 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:18100969:::obj255:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.530+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:171b715e:::obj570:head local-lis/les=113/114 n=71 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:18100969:::obj255:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.530+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:171b715e:::obj570:head local-lis/les=113/114 n=71 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:18100969:::obj255:head 2026-02-11T23:31:27.530+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:171b715e:::obj570:head local-lis/les=113/114 n=71 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:18100969:::obj255:head v 111'255 2026-02-11T23:31:27.530+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:171b715e:::obj570:head local-lis/les=113/114 n=71 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.530+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.530+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:18100969:::obj255:head, version: 111'255, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18100969:::obj255:head@111'255, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:27.530+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:171b715e:::obj570:head local-lis/les=113/114 n=71 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.530+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.530+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.530+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:17371e94:::obj362:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.530+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:17371e94:::obj362:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 1.0 e 116/116 lb 1:17371e94:::obj362:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:171b715e:::obj570:head local-lis/les=113/114 n=71 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.530+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.530+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:171b715e:::obj570:head local-lis/les=113/114 n=71 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:17371e94:::obj362:head) v3 2026-02-11T23:31:27.530+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:171b715e:::obj570:head local-lis/les=113/114 n=71 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:17371e94:::obj362:head) 2026-02-11T23:31:27.530+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.530+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:17371e94:::obj362:head) v3 finish latency 0.000015 2026-02-11T23:31:27.530+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.530+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.530+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:17371e94:::obj362:head local-lis/les=113/114 n=72 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.530+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.530+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.530+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.530+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e7ed80 con 0x4acb000 2026-02-11T23:31:27.530+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:18100969:::obj255:head)]) -- 0x5e7ea80 con 0x4acb000 2026-02-11T23:31:27.530+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.531+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 181 ==== MOSDPGPush(1.0 116/113 [PushOp(1:1842fa44:::obj284:head, version: 111'284, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1842fa44:::obj284:head@111'284, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5d7e1a0 con 0x4acb000 2026-02-11T23:31:27.531+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:1842fa44:::obj284:head, version: 111'284, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1842fa44:::obj284:head@111'284, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.531+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1842fa44:::obj284:head, version: 111'284, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1842fa44:::obj284:head@111'284, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.531+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.531+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 182 ==== pg_backfill(progress 1.0 e 116/116 lb 1:18100969:::obj255:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc4100 con 0x4acb000 2026-02-11T23:31:27.531+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1842fa44:::obj284:head, version: 111'284, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1842fa44:::obj284:head@111'284, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.531+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:18100969:::obj255:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.531+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:18100969:::obj255:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.531+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.531+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1842fa44:::obj284:head, version: 111'284, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1842fa44:::obj284:head@111'284, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.531+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:1842fa44:::obj284:head, version: 111'284, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1842fa44:::obj284:head@111'284, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 116/113 [PushOp(1:1842fa44:::obj284:head, version: 111'284, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1842fa44:::obj284:head@111'284, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:17371e94:::obj362:head local-lis/les=113/114 n=72 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.531+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.531+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:17371e94:::obj362:head local-lis/les=113/114 n=72 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:1842fa44:::obj284:head, version: 111'284, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1842fa44:::obj284:head@111'284, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.531+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.531+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:17371e94:::obj362:head local-lis/les=113/114 n=72 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:1842fa44:::obj284:head@111'284, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.531+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:18100969:::obj255:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.531+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:17371e94:::obj362:head local-lis/les=113/114 n=72 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:1842fa44:::obj284:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.531+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:17371e94:::obj362:head local-lis/les=113/114 n=72 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:1842fa44:::obj284:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.531+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:17371e94:::obj362:head local-lis/les=113/114 n=72 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:1842fa44:::obj284:head 2026-02-11T23:31:27.531+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:17371e94:::obj362:head local-lis/les=113/114 n=72 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:1842fa44:::obj284:head v 111'284 2026-02-11T23:31:27.531+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:17371e94:::obj362:head local-lis/les=113/114 n=72 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.531+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.531+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:1842fa44:::obj284:head, version: 111'284, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1842fa44:::obj284:head@111'284, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:27.531+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:17371e94:::obj362:head local-lis/les=113/114 n=72 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.531+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.531+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.531+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:18100969:::obj255:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.531+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:18100969:::obj255:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 1.0 e 116/116 lb 1:18100969:::obj255:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:17371e94:::obj362:head local-lis/les=113/114 n=72 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.531+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.531+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:17371e94:::obj362:head local-lis/les=113/114 n=72 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:18100969:::obj255:head) v3 2026-02-11T23:31:27.531+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:17371e94:::obj362:head local-lis/les=113/114 n=72 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:18100969:::obj255:head) 2026-02-11T23:31:27.531+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.531+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:18100969:::obj255:head) v3 finish latency 0.000013 2026-02-11T23:31:27.531+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.531+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.531+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:18100969:::obj255:head local-lis/les=113/114 n=73 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.531+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.531+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.531+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.531+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e7f200 con 0x4acb000 2026-02-11T23:31:27.531+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:1842fa44:::obj284:head)]) -- 0x5e7ef00 con 0x4acb000 2026-02-11T23:31:27.531+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.531+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 183 ==== MOSDPGPush(1.0 116/113 [PushOp(1:18b572c6:::obj440:head, version: 111'440, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18b572c6:::obj440:head@111'440, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5d7e680 con 0x4acb000 2026-02-11T23:31:27.531+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:18b572c6:::obj440:head, version: 111'440, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18b572c6:::obj440:head@111'440, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.531+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:18b572c6:::obj440:head, version: 111'440, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18b572c6:::obj440:head@111'440, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.531+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.531+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 184 ==== pg_backfill(progress 1.0 e 116/116 lb 1:1842fa44:::obj284:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc2b00 con 0x4acb000 2026-02-11T23:31:27.531+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:18b572c6:::obj440:head, version: 111'440, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18b572c6:::obj440:head@111'440, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.531+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:1842fa44:::obj284:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.531+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1842fa44:::obj284:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.531+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.531+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:18b572c6:::obj440:head, version: 111'440, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18b572c6:::obj440:head@111'440, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.531+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:18b572c6:::obj440:head, version: 111'440, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18b572c6:::obj440:head@111'440, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(1.0 116/113 [PushOp(1:18b572c6:::obj440:head, version: 111'440, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18b572c6:::obj440:head@111'440, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:18100969:::obj255:head local-lis/les=113/114 n=73 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.531+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.531+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.531+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:18100969:::obj255:head local-lis/les=113/114 n=73 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:18b572c6:::obj440:head, version: 111'440, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18b572c6:::obj440:head@111'440, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.531+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1842fa44:::obj284:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.531+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:18100969:::obj255:head local-lis/les=113/114 n=73 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:18b572c6:::obj440:head@111'440, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.531+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:18100969:::obj255:head local-lis/les=113/114 n=73 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:18b572c6:::obj440:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.531+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:18100969:::obj255:head local-lis/les=113/114 n=73 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:18b572c6:::obj440:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.531+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:18100969:::obj255:head local-lis/les=113/114 n=73 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:18b572c6:::obj440:head 2026-02-11T23:31:27.531+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:18100969:::obj255:head local-lis/les=113/114 n=73 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:18b572c6:::obj440:head v 111'440 2026-02-11T23:31:27.531+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:18100969:::obj255:head local-lis/les=113/114 n=73 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.532+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.532+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:18b572c6:::obj440:head, version: 111'440, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18b572c6:::obj440:head@111'440, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:27.532+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:18100969:::obj255:head local-lis/les=113/114 n=73 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.532+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.532+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.532+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1842fa44:::obj284:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.532+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:1842fa44:::obj284:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 1.0 e 116/116 lb 1:1842fa44:::obj284:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:18100969:::obj255:head local-lis/les=113/114 n=73 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.532+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.532+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:18100969:::obj255:head local-lis/les=113/114 n=73 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:1842fa44:::obj284:head) v3 2026-02-11T23:31:27.532+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:18100969:::obj255:head local-lis/les=113/114 n=73 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:1842fa44:::obj284:head) 2026-02-11T23:31:27.532+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.532+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:1842fa44:::obj284:head) v3 finish latency 0.000015 2026-02-11T23:31:27.532+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.532+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.532+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1842fa44:::obj284:head local-lis/les=113/114 n=74 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.532+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.532+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.532+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.532+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e7f680 con 0x4acb000 2026-02-11T23:31:27.532+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:18b572c6:::obj440:head)]) -- 0x5e7f380 con 0x4acb000 2026-02-11T23:31:27.532+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.532+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 185 ==== MOSDPGPush(1.0 116/113 [PushOp(1:18c50d58:::obj63:head, version: 111'63, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18c50d58:::obj63:head@111'63, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5d7eb60 con 0x4acb000 2026-02-11T23:31:27.532+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:18c50d58:::obj63:head, version: 111'63, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18c50d58:::obj63:head@111'63, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.532+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:18c50d58:::obj63:head, version: 111'63, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18c50d58:::obj63:head@111'63, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.532+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.532+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 186 ==== pg_backfill(progress 1.0 e 116/116 lb 1:18b572c6:::obj440:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc5700 con 0x4acb000 2026-02-11T23:31:27.532+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:18b572c6:::obj440:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.532+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:18c50d58:::obj63:head, version: 111'63, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18c50d58:::obj63:head@111'63, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.532+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:18b572c6:::obj440:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.532+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.532+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:18c50d58:::obj63:head, version: 111'63, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18c50d58:::obj63:head@111'63, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.532+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:18c50d58:::obj63:head, version: 111'63, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18c50d58:::obj63:head@111'63, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 116/113 [PushOp(1:18c50d58:::obj63:head, version: 111'63, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18c50d58:::obj63:head@111'63, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:1842fa44:::obj284:head local-lis/les=113/114 n=74 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.532+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.532+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.532+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1842fa44:::obj284:head local-lis/les=113/114 n=74 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:18c50d58:::obj63:head, version: 111'63, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18c50d58:::obj63:head@111'63, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.532+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:18b572c6:::obj440:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.532+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1842fa44:::obj284:head local-lis/les=113/114 n=74 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:18c50d58:::obj63:head@111'63, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.532+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1842fa44:::obj284:head local-lis/les=113/114 n=74 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:18c50d58:::obj63:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.532+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1842fa44:::obj284:head local-lis/les=113/114 n=74 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:18c50d58:::obj63:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.532+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1842fa44:::obj284:head local-lis/les=113/114 n=74 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:18c50d58:::obj63:head 2026-02-11T23:31:27.532+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1842fa44:::obj284:head local-lis/les=113/114 n=74 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:18c50d58:::obj63:head v 111'63 2026-02-11T23:31:27.532+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1842fa44:::obj284:head local-lis/les=113/114 n=74 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.532+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.532+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:18c50d58:::obj63:head, version: 111'63, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18c50d58:::obj63:head@111'63, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:27.532+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.532+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.532+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:18b572c6:::obj440:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.532+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:18b572c6:::obj440:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 1.0 e 116/116 lb 1:18b572c6:::obj440:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:1842fa44:::obj284:head local-lis/les=113/114 n=74 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.532+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.532+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1842fa44:::obj284:head local-lis/les=113/114 n=74 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:18b572c6:::obj440:head) v3 2026-02-11T23:31:27.532+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1842fa44:::obj284:head local-lis/les=113/114 n=74 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:18b572c6:::obj440:head) 2026-02-11T23:31:27.532+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.532+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:18b572c6:::obj440:head) v3 finish latency 0.000018 2026-02-11T23:31:27.532+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:18b572c6:::obj440:head local-lis/les=113/114 n=75 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.532+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.532+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.532+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:18b572c6:::obj440:head local-lis/les=113/114 n=75 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.532+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.532+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.532+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.532+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e7f800 con 0x4acb000 2026-02-11T23:31:27.532+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:18c50d58:::obj63:head)]) -- 0x5d78480 con 0x4acb000 2026-02-11T23:31:27.532+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.533+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 187 ==== MOSDPGPush(1.0 116/113 [PushOp(1:19299046:::obj541:head, version: 111'541, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:19299046:::obj541:head@111'541, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5d7f040 con 0x4acb000 2026-02-11T23:31:27.533+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:19299046:::obj541:head, version: 111'541, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:19299046:::obj541:head@111'541, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.533+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:19299046:::obj541:head, version: 111'541, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:19299046:::obj541:head@111'541, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.533+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.533+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 188 ==== pg_backfill(progress 1.0 e 116/116 lb 1:18c50d58:::obj63:head) ==== 1061+0+0 (unknown 0 0 0) 0x5dc2580 con 0x4acb000 2026-02-11T23:31:27.533+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:19299046:::obj541:head, version: 111'541, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:19299046:::obj541:head@111'541, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.533+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:18c50d58:::obj63:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.533+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:18c50d58:::obj63:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.533+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.533+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:19299046:::obj541:head, version: 111'541, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:19299046:::obj541:head@111'541, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.533+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:19299046:::obj541:head, version: 111'541, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:19299046:::obj541:head@111'541, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(1.0 116/113 [PushOp(1:19299046:::obj541:head, version: 111'541, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:19299046:::obj541:head@111'541, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:18b572c6:::obj440:head local-lis/les=113/114 n=75 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.533+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.533+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.533+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:18b572c6:::obj440:head local-lis/les=113/114 n=75 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:19299046:::obj541:head, version: 111'541, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:19299046:::obj541:head@111'541, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.533+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:18c50d58:::obj63:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.533+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:18b572c6:::obj440:head local-lis/les=113/114 n=75 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:19299046:::obj541:head@111'541, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.533+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:18b572c6:::obj440:head local-lis/les=113/114 n=75 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:19299046:::obj541:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.533+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:18b572c6:::obj440:head local-lis/les=113/114 n=75 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:19299046:::obj541:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.533+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:18b572c6:::obj440:head local-lis/les=113/114 n=75 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:19299046:::obj541:head 2026-02-11T23:31:27.533+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:18b572c6:::obj440:head local-lis/les=113/114 n=75 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:19299046:::obj541:head v 111'541 2026-02-11T23:31:27.533+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:18b572c6:::obj440:head local-lis/les=113/114 n=75 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.533+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.533+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:19299046:::obj541:head, version: 111'541, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:19299046:::obj541:head@111'541, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:27.533+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:18b572c6:::obj440:head local-lis/les=113/114 n=75 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.533+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.533+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.533+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:18c50d58:::obj63:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.533+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:18c50d58:::obj63:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 1.0 e 116/116 lb 1:18c50d58:::obj63:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:18b572c6:::obj440:head local-lis/les=113/114 n=75 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.533+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.533+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:18b572c6:::obj440:head local-lis/les=113/114 n=75 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:18c50d58:::obj63:head) v3 2026-02-11T23:31:27.533+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:18b572c6:::obj440:head local-lis/les=113/114 n=75 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:18c50d58:::obj63:head) 2026-02-11T23:31:27.533+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.533+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:18c50d58:::obj63:head) v3 finish latency 0.000016 2026-02-11T23:31:27.533+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.533+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.533+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:18c50d58:::obj63:head local-lis/les=113/114 n=76 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.533+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.533+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.533+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.533+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e7e000 con 0x4acb000 2026-02-11T23:31:27.533+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:19299046:::obj541:head)]) -- 0x5e7f980 con 0x4acb000 2026-02-11T23:31:27.533+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.533+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 189 ==== MOSDPGPush(1.0 116/113 [PushOp(1:1a1150ce:::obj60:head, version: 111'60, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a1150ce:::obj60:head@111'60, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5d7f520 con 0x4acb000 2026-02-11T23:31:27.533+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:1a1150ce:::obj60:head, version: 111'60, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a1150ce:::obj60:head@111'60, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.533+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1a1150ce:::obj60:head, version: 111'60, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a1150ce:::obj60:head@111'60, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.533+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.533+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 190 ==== pg_backfill(progress 1.0 e 116/116 lb 1:19299046:::obj541:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc3080 con 0x4acb000 2026-02-11T23:31:27.533+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1a1150ce:::obj60:head, version: 111'60, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a1150ce:::obj60:head@111'60, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.533+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:19299046:::obj541:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 116 2026-02-11T23:31:27.533+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.533+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:19299046:::obj541:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.533+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1a1150ce:::obj60:head, version: 111'60, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a1150ce:::obj60:head@111'60, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.533+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:1a1150ce:::obj60:head, version: 111'60, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a1150ce:::obj60:head@111'60, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:1a1150ce:::obj60:head, version: 111'60, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a1150ce:::obj60:head@111'60, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:18c50d58:::obj63:head local-lis/les=113/114 n=76 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.533+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.533+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.533+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:19299046:::obj541:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.533+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:18c50d58:::obj63:head local-lis/les=113/114 n=76 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:1a1150ce:::obj60:head, version: 111'60, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a1150ce:::obj60:head@111'60, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.533+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:18c50d58:::obj63:head local-lis/les=113/114 n=76 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:1a1150ce:::obj60:head@111'60, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.533+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:18c50d58:::obj63:head local-lis/les=113/114 n=76 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:1a1150ce:::obj60:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.533+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:18c50d58:::obj63:head local-lis/les=113/114 n=76 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:1a1150ce:::obj60:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.533+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:18c50d58:::obj63:head local-lis/les=113/114 n=76 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:1a1150ce:::obj60:head 2026-02-11T23:31:27.533+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:18c50d58:::obj63:head local-lis/les=113/114 n=76 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:1a1150ce:::obj60:head v 111'60 2026-02-11T23:31:27.533+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:18c50d58:::obj63:head local-lis/les=113/114 n=76 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.533+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.534+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:1a1150ce:::obj60:head, version: 111'60, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a1150ce:::obj60:head@111'60, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000064 2026-02-11T23:31:27.534+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:18c50d58:::obj63:head local-lis/les=113/114 n=76 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.534+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.534+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.534+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:19299046:::obj541:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.534+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:19299046:::obj541:head) v3 prio 15 cost 0 latency 0.000084 pg_backfill(progress 1.0 e 116/116 lb 1:19299046:::obj541:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:18c50d58:::obj63:head local-lis/les=113/114 n=76 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.534+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.534+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:18c50d58:::obj63:head local-lis/les=113/114 n=76 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:19299046:::obj541:head) v3 2026-02-11T23:31:27.534+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:18c50d58:::obj63:head local-lis/les=113/114 n=76 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:19299046:::obj541:head) 2026-02-11T23:31:27.534+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.534+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:19299046:::obj541:head) v3 finish latency 0.000014 2026-02-11T23:31:27.534+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.534+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.534+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:19299046:::obj541:head local-lis/les=113/114 n=77 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.534+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.534+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.534+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.534+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e3bc80 con 0x4acb000 2026-02-11T23:31:27.534+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:1a1150ce:::obj60:head)]) -- 0x5e7e480 con 0x4acb000 2026-02-11T23:31:27.534+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.534+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 191 ==== MOSDPGPush(1.0 116/113 [PushOp(1:1a20990d:::obj103:head, version: 111'103, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a20990d:::obj103:head@111'103, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5d7fa00 con 0x4acb000 2026-02-11T23:31:27.534+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:1a20990d:::obj103:head, version: 111'103, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a20990d:::obj103:head@111'103, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.534+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1a20990d:::obj103:head, version: 111'103, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a20990d:::obj103:head@111'103, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.534+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.534+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 192 ==== pg_backfill(progress 1.0 e 116/116 lb 1:1a1150ce:::obj60:head) ==== 1061+0+0 (unknown 0 0 0) 0x4a2cb00 con 0x4acb000 2026-02-11T23:31:27.534+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1a20990d:::obj103:head, version: 111'103, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a20990d:::obj103:head@111'103, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.534+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:1a1150ce:::obj60:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 116 2026-02-11T23:31:27.534+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.534+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1a1150ce:::obj60:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.534+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1a20990d:::obj103:head, version: 111'103, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a20990d:::obj103:head@111'103, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.534+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.534+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:1a20990d:::obj103:head, version: 111'103, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a20990d:::obj103:head@111'103, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:1a20990d:::obj103:head, version: 111'103, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a20990d:::obj103:head@111'103, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:19299046:::obj541:head local-lis/les=113/114 n=77 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.534+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.534+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:19299046:::obj541:head local-lis/les=113/114 n=77 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:1a20990d:::obj103:head, version: 111'103, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a20990d:::obj103:head@111'103, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.534+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.534+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1a1150ce:::obj60:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.534+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:19299046:::obj541:head local-lis/les=113/114 n=77 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:1a20990d:::obj103:head@111'103, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.534+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:19299046:::obj541:head local-lis/les=113/114 n=77 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:1a20990d:::obj103:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.534+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:19299046:::obj541:head local-lis/les=113/114 n=77 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:1a20990d:::obj103:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.534+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:19299046:::obj541:head local-lis/les=113/114 n=77 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:1a20990d:::obj103:head 2026-02-11T23:31:27.534+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:19299046:::obj541:head local-lis/les=113/114 n=77 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:1a20990d:::obj103:head v 111'103 2026-02-11T23:31:27.534+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:19299046:::obj541:head local-lis/les=113/114 n=77 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.534+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.534+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:1a20990d:::obj103:head, version: 111'103, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a20990d:::obj103:head@111'103, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:27.534+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.534+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.534+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1a1150ce:::obj60:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.534+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:1a1150ce:::obj60:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 1.0 e 116/116 lb 1:1a1150ce:::obj60:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:19299046:::obj541:head local-lis/les=113/114 n=77 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.534+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.534+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:19299046:::obj541:head local-lis/les=113/114 n=77 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:1a1150ce:::obj60:head) v3 2026-02-11T23:31:27.534+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:19299046:::obj541:head local-lis/les=113/114 n=77 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:1a1150ce:::obj60:head) 2026-02-11T23:31:27.534+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.534+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:1a1150ce:::obj60:head) v3 finish latency 0.000017 2026-02-11T23:31:27.534+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1a1150ce:::obj60:head local-lis/les=113/114 n=78 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.534+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.534+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.534+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1a1150ce:::obj60:head local-lis/les=113/114 n=78 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.534+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.534+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.534+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.534+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e3a480 con 0x4acb000 2026-02-11T23:31:27.534+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:1a20990d:::obj103:head)]) -- 0x5d78000 con 0x4acb000 2026-02-11T23:31:27.534+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.535+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 193 ==== MOSDPGPush(1.0 116/113 [PushOp(1:1a2eef70:::obj126:head, version: 111'126, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a2eef70:::obj126:head@111'126, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e26000 con 0x4acb000 2026-02-11T23:31:27.535+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:1a2eef70:::obj126:head, version: 111'126, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a2eef70:::obj126:head@111'126, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 116 2026-02-11T23:31:27.535+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1a2eef70:::obj126:head, version: 111'126, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a2eef70:::obj126:head@111'126, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.535+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.535+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 194 ==== pg_backfill(progress 1.0 e 116/116 lb 1:1a20990d:::obj103:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e680 con 0x4acb000 2026-02-11T23:31:27.535+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:1a20990d:::obj103:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.535+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1a2eef70:::obj126:head, version: 111'126, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a2eef70:::obj126:head@111'126, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.535+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1a20990d:::obj103:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.535+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.535+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1a20990d:::obj103:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.535+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.535+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1a2eef70:::obj126:head, version: 111'126, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a2eef70:::obj126:head@111'126, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.535+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:1a2eef70:::obj126:head, version: 111'126, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a2eef70:::obj126:head@111'126, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000044 MOSDPGPush(1.0 116/113 [PushOp(1:1a2eef70:::obj126:head, version: 111'126, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a2eef70:::obj126:head@111'126, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:1a1150ce:::obj60:head local-lis/les=113/114 n=78 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.535+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.535+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1a1150ce:::obj60:head local-lis/les=113/114 n=78 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:1a2eef70:::obj126:head, version: 111'126, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a2eef70:::obj126:head@111'126, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.535+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1a1150ce:::obj60:head local-lis/les=113/114 n=78 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:1a2eef70:::obj126:head@111'126, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.535+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1a1150ce:::obj60:head local-lis/les=113/114 n=78 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:1a2eef70:::obj126:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.535+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1a1150ce:::obj60:head local-lis/les=113/114 n=78 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:1a2eef70:::obj126:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.535+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1a1150ce:::obj60:head local-lis/les=113/114 n=78 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:1a2eef70:::obj126:head 2026-02-11T23:31:27.535+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1a1150ce:::obj60:head local-lis/les=113/114 n=78 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:1a2eef70:::obj126:head v 111'126 2026-02-11T23:31:27.535+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1a1150ce:::obj60:head local-lis/les=113/114 n=78 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.535+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.535+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:1a2eef70:::obj126:head, version: 111'126, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a2eef70:::obj126:head@111'126, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:27.535+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1a1150ce:::obj60:head local-lis/les=113/114 n=78 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.535+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.535+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1a20990d:::obj103:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.535+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:1a20990d:::obj103:head) v3 prio 15 cost 0 latency 0.000088 pg_backfill(progress 1.0 e 116/116 lb 1:1a20990d:::obj103:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:1a1150ce:::obj60:head local-lis/les=113/114 n=78 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.535+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.535+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.535+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1a1150ce:::obj60:head local-lis/les=113/114 n=78 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:1a20990d:::obj103:head) v3 2026-02-11T23:31:27.535+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1a1150ce:::obj60:head local-lis/les=113/114 n=78 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:1a20990d:::obj103:head) 2026-02-11T23:31:27.535+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.535+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:1a20990d:::obj103:head) v3 finish latency 0.000017 2026-02-11T23:31:27.535+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.535+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.535+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1a20990d:::obj103:head local-lis/les=113/114 n=79 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.535+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.535+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.535+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.535+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e6b680 con 0x4acb000 2026-02-11T23:31:27.535+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:1a2eef70:::obj126:head)]) -- 0x5e3b680 con 0x4acb000 2026-02-11T23:31:27.535+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.535+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 195 ==== MOSDPGPush(1.0 116/113 [PushOp(1:1a783fd9:::obj328:head, version: 111'328, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a783fd9:::obj328:head@111'328, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e26340 con 0x4acb000 2026-02-11T23:31:27.535+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:1a783fd9:::obj328:head, version: 111'328, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a783fd9:::obj328:head@111'328, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.535+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1a783fd9:::obj328:head, version: 111'328, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a783fd9:::obj328:head@111'328, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.535+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.535+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 196 ==== pg_backfill(progress 1.0 e 116/116 lb 1:1a2eef70:::obj126:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e100 con 0x4acb000 2026-02-11T23:31:27.535+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1a783fd9:::obj328:head, version: 111'328, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a783fd9:::obj328:head@111'328, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.535+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:1a2eef70:::obj126:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.535+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1a2eef70:::obj126:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.535+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.535+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1a783fd9:::obj328:head, version: 111'328, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a783fd9:::obj328:head@111'328, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.535+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:1a783fd9:::obj328:head, version: 111'328, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a783fd9:::obj328:head@111'328, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(1.0 116/113 [PushOp(1:1a783fd9:::obj328:head, version: 111'328, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a783fd9:::obj328:head@111'328, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:1a20990d:::obj103:head local-lis/les=113/114 n=79 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.535+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.535+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.535+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1a20990d:::obj103:head local-lis/les=113/114 n=79 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:1a783fd9:::obj328:head, version: 111'328, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a783fd9:::obj328:head@111'328, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.535+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1a2eef70:::obj126:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.535+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1a20990d:::obj103:head local-lis/les=113/114 n=79 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:1a783fd9:::obj328:head@111'328, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.535+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1a20990d:::obj103:head local-lis/les=113/114 n=79 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:1a783fd9:::obj328:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.535+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1a20990d:::obj103:head local-lis/les=113/114 n=79 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:1a783fd9:::obj328:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.535+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1a20990d:::obj103:head local-lis/les=113/114 n=79 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:1a783fd9:::obj328:head 2026-02-11T23:31:27.535+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1a20990d:::obj103:head local-lis/les=113/114 n=79 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:1a783fd9:::obj328:head v 111'328 2026-02-11T23:31:27.535+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1a20990d:::obj103:head local-lis/les=113/114 n=79 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.535+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.535+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:1a783fd9:::obj328:head, version: 111'328, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a783fd9:::obj328:head@111'328, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:27.535+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1a20990d:::obj103:head local-lis/les=113/114 n=79 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.535+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.535+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.535+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1a2eef70:::obj126:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.535+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:1a2eef70:::obj126:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 116/116 lb 1:1a2eef70:::obj126:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:1a20990d:::obj103:head local-lis/les=113/114 n=79 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.535+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.535+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1a20990d:::obj103:head local-lis/les=113/114 n=79 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:1a2eef70:::obj126:head) v3 2026-02-11T23:31:27.535+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1a20990d:::obj103:head local-lis/les=113/114 n=79 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:1a2eef70:::obj126:head) 2026-02-11T23:31:27.535+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.536+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:1a2eef70:::obj126:head) v3 finish latency 0.000016 2026-02-11T23:31:27.536+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.536+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.536+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1a2eef70:::obj126:head local-lis/les=113/114 n=80 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.536+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.536+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.536+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.536+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e6a600 con 0x4acb000 2026-02-11T23:31:27.536+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:1a783fd9:::obj328:head)]) -- 0x5e6b380 con 0x4acb000 2026-02-11T23:31:27.536+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.536+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 197 ==== MOSDPGPush(1.0 116/113 [PushOp(1:1abd4411:::obj66:head, version: 111'66, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1abd4411:::obj66:head@111'66, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5d7e340 con 0x4acb000 2026-02-11T23:31:27.536+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:1abd4411:::obj66:head, version: 111'66, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1abd4411:::obj66:head@111'66, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.536+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1abd4411:::obj66:head, version: 111'66, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1abd4411:::obj66:head@111'66, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.536+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.536+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1abd4411:::obj66:head, version: 111'66, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1abd4411:::obj66:head@111'66, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.536+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 198 ==== pg_backfill(progress 1.0 e 116/116 lb 1:1a783fd9:::obj328:head) ==== 1062+0+0 (unknown 0 0 0) 0x4a2cb00 con 0x4acb000 2026-02-11T23:31:27.536+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:1a783fd9:::obj328:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.536+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.536+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1a783fd9:::obj328:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.536+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1abd4411:::obj66:head, version: 111'66, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1abd4411:::obj66:head@111'66, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.536+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.536+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:1abd4411:::obj66:head, version: 111'66, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1abd4411:::obj66:head@111'66, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(1.0 116/113 [PushOp(1:1abd4411:::obj66:head, version: 111'66, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1abd4411:::obj66:head@111'66, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:1a2eef70:::obj126:head local-lis/les=113/114 n=80 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.536+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1a783fd9:::obj328:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.536+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.536+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1a2eef70:::obj126:head local-lis/les=113/114 n=80 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:1abd4411:::obj66:head, version: 111'66, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1abd4411:::obj66:head@111'66, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.536+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1a2eef70:::obj126:head local-lis/les=113/114 n=80 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:1abd4411:::obj66:head@111'66, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.536+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1a2eef70:::obj126:head local-lis/les=113/114 n=80 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:1abd4411:::obj66:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.536+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1a2eef70:::obj126:head local-lis/les=113/114 n=80 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:1abd4411:::obj66:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.536+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1a2eef70:::obj126:head local-lis/les=113/114 n=80 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:1abd4411:::obj66:head 2026-02-11T23:31:27.536+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1a2eef70:::obj126:head local-lis/les=113/114 n=80 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:1abd4411:::obj66:head v 111'66 2026-02-11T23:31:27.536+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1a2eef70:::obj126:head local-lis/les=113/114 n=80 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.536+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.536+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:1abd4411:::obj66:head, version: 111'66, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1abd4411:::obj66:head@111'66, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.536+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1a2eef70:::obj126:head local-lis/les=113/114 n=80 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.536+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.536+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.536+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1a783fd9:::obj328:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.536+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:1a783fd9:::obj328:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 1.0 e 116/116 lb 1:1a783fd9:::obj328:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:1a2eef70:::obj126:head local-lis/les=113/114 n=80 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.536+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.536+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1a2eef70:::obj126:head local-lis/les=113/114 n=80 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:1a783fd9:::obj328:head) v3 2026-02-11T23:31:27.536+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1a2eef70:::obj126:head local-lis/les=113/114 n=80 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:1a783fd9:::obj328:head) 2026-02-11T23:31:27.536+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.536+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:1a783fd9:::obj328:head) v3 finish latency 0.000015 2026-02-11T23:31:27.536+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.536+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.536+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1a783fd9:::obj328:head local-lis/les=113/114 n=81 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.536+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.536+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.536+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.536+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x4ec2480 con 0x4acb000 2026-02-11T23:31:27.536+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:1abd4411:::obj66:head)]) -- 0x5e6b980 con 0x4acb000 2026-02-11T23:31:27.536+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.537+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 199 ==== MOSDPGPush(1.0 116/113 [PushOp(1:1b644371:::obj48:head, version: 111'48, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b644371:::obj48:head@111'48, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5d7e820 con 0x4acb000 2026-02-11T23:31:27.537+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:1b644371:::obj48:head, version: 111'48, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b644371:::obj48:head@111'48, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.537+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1b644371:::obj48:head, version: 111'48, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b644371:::obj48:head@111'48, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.537+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.537+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 200 ==== pg_backfill(progress 1.0 e 116/116 lb 1:1abd4411:::obj66:head) ==== 1061+0+0 (unknown 0 0 0) 0x5dc3600 con 0x4acb000 2026-02-11T23:31:27.537+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:1abd4411:::obj66:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.537+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1b644371:::obj48:head, version: 111'48, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b644371:::obj48:head@111'48, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.537+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1abd4411:::obj66:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.537+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.537+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1abd4411:::obj66:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.537+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.537+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1b644371:::obj48:head, version: 111'48, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b644371:::obj48:head@111'48, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.537+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:1b644371:::obj48:head, version: 111'48, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b644371:::obj48:head@111'48, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000041 MOSDPGPush(1.0 116/113 [PushOp(1:1b644371:::obj48:head, version: 111'48, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b644371:::obj48:head@111'48, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:1a783fd9:::obj328:head local-lis/les=113/114 n=81 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.537+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.537+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1a783fd9:::obj328:head local-lis/les=113/114 n=81 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:1b644371:::obj48:head, version: 111'48, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b644371:::obj48:head@111'48, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.537+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1a783fd9:::obj328:head local-lis/les=113/114 n=81 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:1b644371:::obj48:head@111'48, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.537+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1a783fd9:::obj328:head local-lis/les=113/114 n=81 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:1b644371:::obj48:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.537+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1a783fd9:::obj328:head local-lis/les=113/114 n=81 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:1b644371:::obj48:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.537+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1a783fd9:::obj328:head local-lis/les=113/114 n=81 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:1b644371:::obj48:head 2026-02-11T23:31:27.537+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1a783fd9:::obj328:head local-lis/les=113/114 n=81 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:1b644371:::obj48:head v 111'48 2026-02-11T23:31:27.537+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1a783fd9:::obj328:head local-lis/les=113/114 n=81 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.537+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.537+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:1b644371:::obj48:head, version: 111'48, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b644371:::obj48:head@111'48, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:27.537+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.537+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.537+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1abd4411:::obj66:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.537+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:1abd4411:::obj66:head) v3 prio 15 cost 0 latency 0.000087 pg_backfill(progress 1.0 e 116/116 lb 1:1abd4411:::obj66:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:1a783fd9:::obj328:head local-lis/les=113/114 n=81 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.537+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.537+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1a783fd9:::obj328:head local-lis/les=113/114 n=81 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:1abd4411:::obj66:head) v3 2026-02-11T23:31:27.537+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1a783fd9:::obj328:head local-lis/les=113/114 n=81 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:1abd4411:::obj66:head) 2026-02-11T23:31:27.537+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.537+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:1abd4411:::obj66:head) v3 finish latency 0.000017 2026-02-11T23:31:27.537+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1abd4411:::obj66:head local-lis/les=113/114 n=82 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.537+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.537+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.537+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1abd4411:::obj66:head local-lis/les=113/114 n=82 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.537+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.537+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.537+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.537+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x4ec2780 con 0x4acb000 2026-02-11T23:31:27.537+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:1b644371:::obj48:head)]) -- 0x5d79980 con 0x4acb000 2026-02-11T23:31:27.537+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.537+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 201 ==== MOSDPGPush(1.0 116/113 [PushOp(1:1b6bffcc:::obj61:head, version: 111'61, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b6bffcc:::obj61:head@111'61, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5d7ed00 con 0x4acb000 2026-02-11T23:31:27.537+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:1b6bffcc:::obj61:head, version: 111'61, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b6bffcc:::obj61:head@111'61, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 116 2026-02-11T23:31:27.537+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1b6bffcc:::obj61:head, version: 111'61, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b6bffcc:::obj61:head@111'61, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.537+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.537+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 202 ==== pg_backfill(progress 1.0 e 116/116 lb 1:1b644371:::obj48:head) ==== 1061+0+0 (unknown 0 0 0) 0x5dc4c00 con 0x4acb000 2026-02-11T23:31:27.537+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1b6bffcc:::obj61:head, version: 111'61, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b6bffcc:::obj61:head@111'61, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.537+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:1b644371:::obj48:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.537+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1b644371:::obj48:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.537+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.537+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1b6bffcc:::obj61:head, version: 111'61, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b6bffcc:::obj61:head@111'61, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.537+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:1b6bffcc:::obj61:head, version: 111'61, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b6bffcc:::obj61:head@111'61, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(1.0 116/113 [PushOp(1:1b6bffcc:::obj61:head, version: 111'61, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b6bffcc:::obj61:head@111'61, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:1abd4411:::obj66:head local-lis/les=113/114 n=82 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.537+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.537+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.537+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1b644371:::obj48:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.537+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1abd4411:::obj66:head local-lis/les=113/114 n=82 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:1b6bffcc:::obj61:head, version: 111'61, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b6bffcc:::obj61:head@111'61, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.537+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1abd4411:::obj66:head local-lis/les=113/114 n=82 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:1b6bffcc:::obj61:head@111'61, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.537+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1abd4411:::obj66:head local-lis/les=113/114 n=82 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:1b6bffcc:::obj61:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.537+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1abd4411:::obj66:head local-lis/les=113/114 n=82 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:1b6bffcc:::obj61:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.537+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1abd4411:::obj66:head local-lis/les=113/114 n=82 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:1b6bffcc:::obj61:head 2026-02-11T23:31:27.537+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1abd4411:::obj66:head local-lis/les=113/114 n=82 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:1b6bffcc:::obj61:head v 111'61 2026-02-11T23:31:27.537+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1abd4411:::obj66:head local-lis/les=113/114 n=82 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.537+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.537+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:1b6bffcc:::obj61:head, version: 111'61, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b6bffcc:::obj61:head@111'61, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:27.537+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.537+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.537+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1b644371:::obj48:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.537+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:1b644371:::obj48:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 1.0 e 116/116 lb 1:1b644371:::obj48:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:1abd4411:::obj66:head local-lis/les=113/114 n=82 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.537+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.537+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1abd4411:::obj66:head local-lis/les=113/114 n=82 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:1b644371:::obj48:head) v3 2026-02-11T23:31:27.537+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1abd4411:::obj66:head local-lis/les=113/114 n=82 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:1b644371:::obj48:head) 2026-02-11T23:31:27.537+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.537+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:1b644371:::obj48:head) v3 finish latency 0.000015 2026-02-11T23:31:27.537+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1b644371:::obj48:head local-lis/les=113/114 n=83 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.537+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.538+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.538+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1b644371:::obj48:head local-lis/les=113/114 n=83 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.538+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.538+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.538+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.538+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d78a80 con 0x4acb000 2026-02-11T23:31:27.538+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:1b6bffcc:::obj61:head)]) -- 0x5d79c80 con 0x4acb000 2026-02-11T23:31:27.538+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.538+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 203 ==== MOSDPGPush(1.0 116/113 [PushOp(1:1b6ee6bf:::obj501:head, version: 111'501, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b6ee6bf:::obj501:head@111'501, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5d7f1e0 con 0x4acb000 2026-02-11T23:31:27.538+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:1b6ee6bf:::obj501:head, version: 111'501, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b6ee6bf:::obj501:head@111'501, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.538+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1b6ee6bf:::obj501:head, version: 111'501, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b6ee6bf:::obj501:head@111'501, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.538+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.538+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 204 ==== pg_backfill(progress 1.0 e 116/116 lb 1:1b6bffcc:::obj61:head) ==== 1061+0+0 (unknown 0 0 0) 0x5dc4680 con 0x4acb000 2026-02-11T23:31:27.538+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1b6ee6bf:::obj501:head, version: 111'501, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b6ee6bf:::obj501:head@111'501, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.538+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:1b6bffcc:::obj61:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.538+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1b6bffcc:::obj61:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.538+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.538+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1b6ee6bf:::obj501:head, version: 111'501, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b6ee6bf:::obj501:head@111'501, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.538+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:1b6ee6bf:::obj501:head, version: 111'501, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b6ee6bf:::obj501:head@111'501, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 116/113 [PushOp(1:1b6ee6bf:::obj501:head, version: 111'501, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b6ee6bf:::obj501:head@111'501, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:1b644371:::obj48:head local-lis/les=113/114 n=83 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.538+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.538+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.538+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1b6bffcc:::obj61:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.538+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1b644371:::obj48:head local-lis/les=113/114 n=83 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:1b6ee6bf:::obj501:head, version: 111'501, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b6ee6bf:::obj501:head@111'501, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.538+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1b644371:::obj48:head local-lis/les=113/114 n=83 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:1b6ee6bf:::obj501:head@111'501, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.538+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1b644371:::obj48:head local-lis/les=113/114 n=83 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:1b6ee6bf:::obj501:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.538+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1b644371:::obj48:head local-lis/les=113/114 n=83 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:1b6ee6bf:::obj501:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.538+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1b644371:::obj48:head local-lis/les=113/114 n=83 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:1b6ee6bf:::obj501:head 2026-02-11T23:31:27.538+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1b644371:::obj48:head local-lis/les=113/114 n=83 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:1b6ee6bf:::obj501:head v 111'501 2026-02-11T23:31:27.538+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1b644371:::obj48:head local-lis/les=113/114 n=83 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.538+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.538+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:1b6ee6bf:::obj501:head, version: 111'501, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b6ee6bf:::obj501:head@111'501, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:27.538+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.538+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.538+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1b6bffcc:::obj61:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.538+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:1b6bffcc:::obj61:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 1.0 e 116/116 lb 1:1b6bffcc:::obj61:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:1b644371:::obj48:head local-lis/les=113/114 n=83 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.538+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.538+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1b644371:::obj48:head local-lis/les=113/114 n=83 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:1b6bffcc:::obj61:head) v3 2026-02-11T23:31:27.538+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1b644371:::obj48:head local-lis/les=113/114 n=83 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:1b6bffcc:::obj61:head) 2026-02-11T23:31:27.538+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.538+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:1b6bffcc:::obj61:head) v3 finish latency 0.000015 2026-02-11T23:31:27.538+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1b6bffcc:::obj61:head local-lis/les=113/114 n=84 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.538+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.538+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.538+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1b6bffcc:::obj61:head local-lis/les=113/114 n=84 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.538+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.538+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.538+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.538+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d78780 con 0x4acb000 2026-02-11T23:31:27.538+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:1b6ee6bf:::obj501:head)]) -- 0x5d79e00 con 0x4acb000 2026-02-11T23:31:27.538+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.539+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 205 ==== MOSDPGPush(1.0 116/113 [PushOp(1:1bad8e21:::obj122:head, version: 111'122, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1bad8e21:::obj122:head@111'122, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5d7f6c0 con 0x4acb000 2026-02-11T23:31:27.539+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:1bad8e21:::obj122:head, version: 111'122, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1bad8e21:::obj122:head@111'122, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.539+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1bad8e21:::obj122:head, version: 111'122, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1bad8e21:::obj122:head@111'122, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.539+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.539+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 206 ==== pg_backfill(progress 1.0 e 116/116 lb 1:1b6ee6bf:::obj501:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc3b80 con 0x4acb000 2026-02-11T23:31:27.539+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:1b6ee6bf:::obj501:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.539+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1bad8e21:::obj122:head, version: 111'122, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1bad8e21:::obj122:head@111'122, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.539+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1b6ee6bf:::obj501:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.539+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.539+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1bad8e21:::obj122:head, version: 111'122, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1bad8e21:::obj122:head@111'122, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.539+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.539+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:1bad8e21:::obj122:head, version: 111'122, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1bad8e21:::obj122:head@111'122, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 116/113 [PushOp(1:1bad8e21:::obj122:head, version: 111'122, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1bad8e21:::obj122:head@111'122, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:1b6bffcc:::obj61:head local-lis/les=113/114 n=84 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.539+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1b6ee6bf:::obj501:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.539+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.539+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1b6bffcc:::obj61:head local-lis/les=113/114 n=84 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:1bad8e21:::obj122:head, version: 111'122, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1bad8e21:::obj122:head@111'122, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.539+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1b6bffcc:::obj61:head local-lis/les=113/114 n=84 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:1bad8e21:::obj122:head@111'122, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.539+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1b6bffcc:::obj61:head local-lis/les=113/114 n=84 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:1bad8e21:::obj122:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.539+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1b6bffcc:::obj61:head local-lis/les=113/114 n=84 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:1bad8e21:::obj122:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.539+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1b6bffcc:::obj61:head local-lis/les=113/114 n=84 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:1bad8e21:::obj122:head 2026-02-11T23:31:27.539+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1b6bffcc:::obj61:head local-lis/les=113/114 n=84 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:1bad8e21:::obj122:head v 111'122 2026-02-11T23:31:27.539+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1b6bffcc:::obj61:head local-lis/les=113/114 n=84 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.539+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.539+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:1bad8e21:::obj122:head, version: 111'122, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1bad8e21:::obj122:head@111'122, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:27.539+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.539+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.539+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1b6ee6bf:::obj501:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.539+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:1b6ee6bf:::obj501:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 1.0 e 116/116 lb 1:1b6ee6bf:::obj501:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:1b6bffcc:::obj61:head local-lis/les=113/114 n=84 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.539+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.539+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1b6bffcc:::obj61:head local-lis/les=113/114 n=84 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:1b6ee6bf:::obj501:head) v3 2026-02-11T23:31:27.539+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1b6bffcc:::obj61:head local-lis/les=113/114 n=84 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:1b6ee6bf:::obj501:head) 2026-02-11T23:31:27.539+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.539+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:1b6ee6bf:::obj501:head) v3 finish latency 0.000017 2026-02-11T23:31:27.539+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1b6ee6bf:::obj501:head local-lis/les=113/114 n=85 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.539+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.539+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.539+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1b6ee6bf:::obj501:head local-lis/les=113/114 n=85 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.539+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.539+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.539+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.539+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d78d80 con 0x4acb000 2026-02-11T23:31:27.539+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:1bad8e21:::obj122:head)]) -- 0x5e7e900 con 0x4acb000 2026-02-11T23:31:27.539+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.539+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 207 ==== MOSDPGPush(1.0 116/113 [PushOp(1:1bb1e69e:::obj390:head, version: 111'390, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1bb1e69e:::obj390:head@111'390, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5d7fba0 con 0x4acb000 2026-02-11T23:31:27.539+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:1bb1e69e:::obj390:head, version: 111'390, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1bb1e69e:::obj390:head@111'390, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 116 2026-02-11T23:31:27.539+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1bb1e69e:::obj390:head, version: 111'390, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1bb1e69e:::obj390:head@111'390, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.539+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.539+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 208 ==== pg_backfill(progress 1.0 e 116/116 lb 1:1bad8e21:::obj122:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc4100 con 0x4acb000 2026-02-11T23:31:27.539+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1bb1e69e:::obj390:head, version: 111'390, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1bb1e69e:::obj390:head@111'390, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.539+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:1bad8e21:::obj122:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.539+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1bad8e21:::obj122:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.539+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.539+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1bb1e69e:::obj390:head, version: 111'390, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1bb1e69e:::obj390:head@111'390, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.539+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.539+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:1bb1e69e:::obj390:head, version: 111'390, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1bb1e69e:::obj390:head@111'390, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 116/113 [PushOp(1:1bb1e69e:::obj390:head, version: 111'390, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1bb1e69e:::obj390:head@111'390, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:1b6ee6bf:::obj501:head local-lis/les=113/114 n=85 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.539+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.539+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1b6ee6bf:::obj501:head local-lis/les=113/114 n=85 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:1bb1e69e:::obj390:head, version: 111'390, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1bb1e69e:::obj390:head@111'390, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.539+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.539+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1bad8e21:::obj122:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.539+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1b6ee6bf:::obj501:head local-lis/les=113/114 n=85 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:1bb1e69e:::obj390:head@111'390, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.539+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1b6ee6bf:::obj501:head local-lis/les=113/114 n=85 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:1bb1e69e:::obj390:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.539+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1b6ee6bf:::obj501:head local-lis/les=113/114 n=85 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:1bb1e69e:::obj390:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.539+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1b6ee6bf:::obj501:head local-lis/les=113/114 n=85 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:1bb1e69e:::obj390:head 2026-02-11T23:31:27.539+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1b6ee6bf:::obj501:head local-lis/les=113/114 n=85 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:1bb1e69e:::obj390:head v 111'390 2026-02-11T23:31:27.539+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1b6ee6bf:::obj501:head local-lis/les=113/114 n=85 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.539+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.539+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:1bb1e69e:::obj390:head, version: 111'390, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1bb1e69e:::obj390:head@111'390, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:27.539+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.539+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.539+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1bad8e21:::obj122:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.539+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:1bad8e21:::obj122:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 1.0 e 116/116 lb 1:1bad8e21:::obj122:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:1b6ee6bf:::obj501:head local-lis/les=113/114 n=85 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.539+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.539+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1b6ee6bf:::obj501:head local-lis/les=113/114 n=85 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:1bad8e21:::obj122:head) v3 2026-02-11T23:31:27.539+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1b6ee6bf:::obj501:head local-lis/les=113/114 n=85 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:1bad8e21:::obj122:head) 2026-02-11T23:31:27.539+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.539+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:1bad8e21:::obj122:head) v3 finish latency 0.000016 2026-02-11T23:31:27.539+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1bad8e21:::obj122:head local-lis/les=113/114 n=86 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.539+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.539+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.539+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1bad8e21:::obj122:head local-lis/les=113/114 n=86 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.539+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.539+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.539+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.539+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d78c00 con 0x4acb000 2026-02-11T23:31:27.539+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:1bb1e69e:::obj390:head)]) -- 0x5e7ed80 con 0x4acb000 2026-02-11T23:31:27.539+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.540+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 209 ==== MOSDPGPush(1.0 116/113 [PushOp(1:1c47bf8e:::obj558:head, version: 111'558, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1c47bf8e:::obj558:head@111'558, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e26000 con 0x4acb000 2026-02-11T23:31:27.540+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:1c47bf8e:::obj558:head, version: 111'558, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1c47bf8e:::obj558:head@111'558, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.540+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1c47bf8e:::obj558:head, version: 111'558, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1c47bf8e:::obj558:head@111'558, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.540+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.540+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 210 ==== pg_backfill(progress 1.0 e 116/116 lb 1:1bb1e69e:::obj390:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc2b00 con 0x4acb000 2026-02-11T23:31:27.540+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:1bb1e69e:::obj390:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.540+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1c47bf8e:::obj558:head, version: 111'558, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1c47bf8e:::obj558:head@111'558, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.540+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1bb1e69e:::obj390:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.540+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.540+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1c47bf8e:::obj558:head, version: 111'558, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1c47bf8e:::obj558:head@111'558, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.540+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.540+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:1c47bf8e:::obj558:head, version: 111'558, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1c47bf8e:::obj558:head@111'558, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 116/113 [PushOp(1:1c47bf8e:::obj558:head, version: 111'558, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1c47bf8e:::obj558:head@111'558, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:1bad8e21:::obj122:head local-lis/les=113/114 n=86 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.540+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1bb1e69e:::obj390:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.540+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.540+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1bad8e21:::obj122:head local-lis/les=113/114 n=86 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:1c47bf8e:::obj558:head, version: 111'558, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1c47bf8e:::obj558:head@111'558, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.540+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1bad8e21:::obj122:head local-lis/les=113/114 n=86 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:1c47bf8e:::obj558:head@111'558, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.540+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1bad8e21:::obj122:head local-lis/les=113/114 n=86 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:1c47bf8e:::obj558:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.540+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1bad8e21:::obj122:head local-lis/les=113/114 n=86 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:1c47bf8e:::obj558:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.540+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1bad8e21:::obj122:head local-lis/les=113/114 n=86 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:1c47bf8e:::obj558:head 2026-02-11T23:31:27.540+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1bad8e21:::obj122:head local-lis/les=113/114 n=86 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:1c47bf8e:::obj558:head v 111'558 2026-02-11T23:31:27.540+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1bad8e21:::obj122:head local-lis/les=113/114 n=86 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.540+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.540+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:1c47bf8e:::obj558:head, version: 111'558, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1c47bf8e:::obj558:head@111'558, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:27.540+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1bad8e21:::obj122:head local-lis/les=113/114 n=86 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.540+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.540+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1bb1e69e:::obj390:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.540+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:1bb1e69e:::obj390:head) v3 prio 15 cost 0 latency 0.000082 pg_backfill(progress 1.0 e 116/116 lb 1:1bb1e69e:::obj390:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:1bad8e21:::obj122:head local-lis/les=113/114 n=86 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.540+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.540+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.540+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1bad8e21:::obj122:head local-lis/les=113/114 n=86 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:1bb1e69e:::obj390:head) v3 2026-02-11T23:31:27.540+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1bad8e21:::obj122:head local-lis/les=113/114 n=86 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:1bb1e69e:::obj390:head) 2026-02-11T23:31:27.540+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.540+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:1bb1e69e:::obj390:head) v3 finish latency 0.000020 2026-02-11T23:31:27.540+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.540+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.540+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1bb1e69e:::obj390:head local-lis/les=113/114 n=87 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.540+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.540+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.540+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.540+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d79200 con 0x4acb000 2026-02-11T23:31:27.540+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:1c47bf8e:::obj558:head)]) -- 0x5d79380 con 0x4acb000 2026-02-11T23:31:27.540+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.540+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 211 ==== MOSDPGPush(1.0 116/113 [PushOp(1:1cb33d95:::obj190:head, version: 111'190, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cb33d95:::obj190:head@111'190, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e26340 con 0x4acb000 2026-02-11T23:31:27.540+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:1cb33d95:::obj190:head, version: 111'190, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cb33d95:::obj190:head@111'190, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.540+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1cb33d95:::obj190:head, version: 111'190, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cb33d95:::obj190:head@111'190, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.540+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.540+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 212 ==== pg_backfill(progress 1.0 e 116/116 lb 1:1c47bf8e:::obj558:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc5700 con 0x4acb000 2026-02-11T23:31:27.540+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:1c47bf8e:::obj558:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.540+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1cb33d95:::obj190:head, version: 111'190, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cb33d95:::obj190:head@111'190, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.540+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1c47bf8e:::obj558:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.540+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.540+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1cb33d95:::obj190:head, version: 111'190, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cb33d95:::obj190:head@111'190, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.540+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.540+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:1cb33d95:::obj190:head, version: 111'190, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cb33d95:::obj190:head@111'190, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:1cb33d95:::obj190:head, version: 111'190, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cb33d95:::obj190:head@111'190, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:1bb1e69e:::obj390:head local-lis/les=113/114 n=87 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.540+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1c47bf8e:::obj558:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.540+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.540+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1bb1e69e:::obj390:head local-lis/les=113/114 n=87 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:1cb33d95:::obj190:head, version: 111'190, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cb33d95:::obj190:head@111'190, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.540+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1bb1e69e:::obj390:head local-lis/les=113/114 n=87 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:1cb33d95:::obj190:head@111'190, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.540+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1bb1e69e:::obj390:head local-lis/les=113/114 n=87 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:1cb33d95:::obj190:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.540+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1bb1e69e:::obj390:head local-lis/les=113/114 n=87 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:1cb33d95:::obj190:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.540+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1bb1e69e:::obj390:head local-lis/les=113/114 n=87 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:1cb33d95:::obj190:head 2026-02-11T23:31:27.540+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1bb1e69e:::obj390:head local-lis/les=113/114 n=87 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:1cb33d95:::obj190:head v 111'190 2026-02-11T23:31:27.540+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1bb1e69e:::obj390:head local-lis/les=113/114 n=87 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.540+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.541+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:1cb33d95:::obj190:head, version: 111'190, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cb33d95:::obj190:head@111'190, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:27.541+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.541+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.541+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1c47bf8e:::obj558:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.541+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:1c47bf8e:::obj558:head) v3 prio 15 cost 0 latency 0.000081 pg_backfill(progress 1.0 e 116/116 lb 1:1c47bf8e:::obj558:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:1bb1e69e:::obj390:head local-lis/les=113/114 n=87 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.541+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.541+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1bb1e69e:::obj390:head local-lis/les=113/114 n=87 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:1c47bf8e:::obj558:head) v3 2026-02-11T23:31:27.541+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1bb1e69e:::obj390:head local-lis/les=113/114 n=87 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:1c47bf8e:::obj558:head) 2026-02-11T23:31:27.541+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.541+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:1c47bf8e:::obj558:head) v3 finish latency 0.000016 2026-02-11T23:31:27.541+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1c47bf8e:::obj558:head local-lis/les=113/114 n=88 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.541+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.541+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.541+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1c47bf8e:::obj558:head local-lis/les=113/114 n=88 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.541+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.541+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.541+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.541+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d78480 con 0x4acb000 2026-02-11T23:31:27.541+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:1cb33d95:::obj190:head)]) -- 0x5e7fe00 con 0x4acb000 2026-02-11T23:31:27.541+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.541+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 213 ==== MOSDPGPush(1.0 116/113 [PushOp(1:1cd4c3d5:::obj142:head, version: 111'142, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cd4c3d5:::obj142:head@111'142, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e271e0 con 0x4acb000 2026-02-11T23:31:27.541+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:1cd4c3d5:::obj142:head, version: 111'142, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cd4c3d5:::obj142:head@111'142, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 116 2026-02-11T23:31:27.541+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1cd4c3d5:::obj142:head, version: 111'142, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cd4c3d5:::obj142:head@111'142, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.541+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.541+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 214 ==== pg_backfill(progress 1.0 e 116/116 lb 1:1cb33d95:::obj190:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc2580 con 0x4acb000 2026-02-11T23:31:27.541+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1cd4c3d5:::obj142:head, version: 111'142, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cd4c3d5:::obj142:head@111'142, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.541+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:1cb33d95:::obj190:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 116 2026-02-11T23:31:27.541+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1cb33d95:::obj190:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.541+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.541+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1cd4c3d5:::obj142:head, version: 111'142, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cd4c3d5:::obj142:head@111'142, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.541+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:1cd4c3d5:::obj142:head, version: 111'142, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cd4c3d5:::obj142:head@111'142, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:1cd4c3d5:::obj142:head, version: 111'142, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cd4c3d5:::obj142:head@111'142, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:1c47bf8e:::obj558:head local-lis/les=113/114 n=88 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.541+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.541+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.541+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1cb33d95:::obj190:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.541+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1c47bf8e:::obj558:head local-lis/les=113/114 n=88 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:1cd4c3d5:::obj142:head, version: 111'142, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cd4c3d5:::obj142:head@111'142, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.541+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1c47bf8e:::obj558:head local-lis/les=113/114 n=88 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:1cd4c3d5:::obj142:head@111'142, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.541+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1c47bf8e:::obj558:head local-lis/les=113/114 n=88 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:1cd4c3d5:::obj142:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.541+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1c47bf8e:::obj558:head local-lis/les=113/114 n=88 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:1cd4c3d5:::obj142:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.541+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1c47bf8e:::obj558:head local-lis/les=113/114 n=88 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:1cd4c3d5:::obj142:head 2026-02-11T23:31:27.541+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1c47bf8e:::obj558:head local-lis/les=113/114 n=88 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:1cd4c3d5:::obj142:head v 111'142 2026-02-11T23:31:27.541+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1c47bf8e:::obj558:head local-lis/les=113/114 n=88 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.541+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.541+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:1cd4c3d5:::obj142:head, version: 111'142, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cd4c3d5:::obj142:head@111'142, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:27.541+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.541+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.541+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1cb33d95:::obj190:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.541+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:1cb33d95:::obj190:head) v3 prio 15 cost 0 latency 0.000079 pg_backfill(progress 1.0 e 116/116 lb 1:1cb33d95:::obj190:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:1c47bf8e:::obj558:head local-lis/les=113/114 n=88 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.541+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.541+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1c47bf8e:::obj558:head local-lis/les=113/114 n=88 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:1cb33d95:::obj190:head) v3 2026-02-11T23:31:27.541+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1c47bf8e:::obj558:head local-lis/les=113/114 n=88 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:1cb33d95:::obj190:head) 2026-02-11T23:31:27.541+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.541+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:1cb33d95:::obj190:head) v3 finish latency 0.000014 2026-02-11T23:31:27.541+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1cb33d95:::obj190:head local-lis/les=113/114 n=89 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.541+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.541+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.541+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1cb33d95:::obj190:head local-lis/les=113/114 n=89 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.541+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.541+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.541+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.541+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d78000 con 0x4acb000 2026-02-11T23:31:27.541+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:1cd4c3d5:::obj142:head)]) -- 0x5e3a480 con 0x4acb000 2026-02-11T23:31:27.541+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.542+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 215 ==== MOSDPGPush(1.0 116/113 [PushOp(1:1cf1ad59:::obj156:head, version: 111'156, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cf1ad59:::obj156:head@111'156, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5d7e000 con 0x4acb000 2026-02-11T23:31:27.542+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:1cf1ad59:::obj156:head, version: 111'156, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cf1ad59:::obj156:head@111'156, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.542+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1cf1ad59:::obj156:head, version: 111'156, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cf1ad59:::obj156:head@111'156, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.542+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.542+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 216 ==== pg_backfill(progress 1.0 e 116/116 lb 1:1cd4c3d5:::obj142:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc3080 con 0x4acb000 2026-02-11T23:31:27.542+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:1cd4c3d5:::obj142:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.542+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1cf1ad59:::obj156:head, version: 111'156, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cf1ad59:::obj156:head@111'156, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.542+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1cd4c3d5:::obj142:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.542+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.542+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1cf1ad59:::obj156:head, version: 111'156, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cf1ad59:::obj156:head@111'156, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.542+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.542+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1cd4c3d5:::obj142:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.542+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:1cf1ad59:::obj156:head, version: 111'156, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cf1ad59:::obj156:head@111'156, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 116/113 [PushOp(1:1cf1ad59:::obj156:head, version: 111'156, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cf1ad59:::obj156:head@111'156, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:1cb33d95:::obj190:head local-lis/les=113/114 n=89 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.542+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.542+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1cb33d95:::obj190:head local-lis/les=113/114 n=89 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:1cf1ad59:::obj156:head, version: 111'156, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cf1ad59:::obj156:head@111'156, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.542+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1cb33d95:::obj190:head local-lis/les=113/114 n=89 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:1cf1ad59:::obj156:head@111'156, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.542+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1cb33d95:::obj190:head local-lis/les=113/114 n=89 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:1cf1ad59:::obj156:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.542+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1cb33d95:::obj190:head local-lis/les=113/114 n=89 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:1cf1ad59:::obj156:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.542+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1cb33d95:::obj190:head local-lis/les=113/114 n=89 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:1cf1ad59:::obj156:head 2026-02-11T23:31:27.542+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1cb33d95:::obj190:head local-lis/les=113/114 n=89 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:1cf1ad59:::obj156:head v 111'156 2026-02-11T23:31:27.542+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1cb33d95:::obj190:head local-lis/les=113/114 n=89 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.542+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.542+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:1cf1ad59:::obj156:head, version: 111'156, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cf1ad59:::obj156:head@111'156, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000062 2026-02-11T23:31:27.542+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1cb33d95:::obj190:head local-lis/les=113/114 n=89 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.542+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.542+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1cd4c3d5:::obj142:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.542+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:1cd4c3d5:::obj142:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 1.0 e 116/116 lb 1:1cd4c3d5:::obj142:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:1cb33d95:::obj190:head local-lis/les=113/114 n=89 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.542+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.542+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.542+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1cb33d95:::obj190:head local-lis/les=113/114 n=89 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:1cd4c3d5:::obj142:head) v3 2026-02-11T23:31:27.542+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1cb33d95:::obj190:head local-lis/les=113/114 n=89 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:1cd4c3d5:::obj142:head) 2026-02-11T23:31:27.542+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.542+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:1cd4c3d5:::obj142:head) v3 finish latency 0.000015 2026-02-11T23:31:27.542+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.542+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.542+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1cd4c3d5:::obj142:head local-lis/les=113/114 n=90 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.542+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.542+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.542+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.542+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d79c80 con 0x4acb000 2026-02-11T23:31:27.542+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:1cf1ad59:::obj156:head)]) -- 0x5d79980 con 0x4acb000 2026-02-11T23:31:27.542+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.542+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 217 ==== MOSDPGPush(1.0 116/113 [PushOp(1:1cff0951:::obj258:head, version: 111'258, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cff0951:::obj258:head@111'258, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5d7e4e0 con 0x4acb000 2026-02-11T23:31:27.542+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:1cff0951:::obj258:head, version: 111'258, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cff0951:::obj258:head@111'258, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.542+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1cff0951:::obj258:head, version: 111'258, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cff0951:::obj258:head@111'258, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.542+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.542+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 218 ==== pg_backfill(progress 1.0 e 116/116 lb 1:1cf1ad59:::obj156:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2d080 con 0x4acb000 2026-02-11T23:31:27.542+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:1cf1ad59:::obj156:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.542+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1cff0951:::obj258:head, version: 111'258, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cff0951:::obj258:head@111'258, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.542+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1cf1ad59:::obj156:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.542+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.542+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1cf1ad59:::obj156:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.542+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.542+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1cff0951:::obj258:head, version: 111'258, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cff0951:::obj258:head@111'258, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.542+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:1cff0951:::obj258:head, version: 111'258, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cff0951:::obj258:head@111'258, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000043 MOSDPGPush(1.0 116/113 [PushOp(1:1cff0951:::obj258:head, version: 111'258, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cff0951:::obj258:head@111'258, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:1cd4c3d5:::obj142:head local-lis/les=113/114 n=90 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.542+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.542+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1cd4c3d5:::obj142:head local-lis/les=113/114 n=90 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:1cff0951:::obj258:head, version: 111'258, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cff0951:::obj258:head@111'258, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.542+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1cd4c3d5:::obj142:head local-lis/les=113/114 n=90 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:1cff0951:::obj258:head@111'258, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.542+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1cd4c3d5:::obj142:head local-lis/les=113/114 n=90 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:1cff0951:::obj258:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.542+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1cd4c3d5:::obj142:head local-lis/les=113/114 n=90 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:1cff0951:::obj258:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.542+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1cd4c3d5:::obj142:head local-lis/les=113/114 n=90 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:1cff0951:::obj258:head 2026-02-11T23:31:27.542+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1cd4c3d5:::obj142:head local-lis/les=113/114 n=90 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:1cff0951:::obj258:head v 111'258 2026-02-11T23:31:27.542+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1cd4c3d5:::obj142:head local-lis/les=113/114 n=90 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.542+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.542+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:1cff0951:::obj258:head, version: 111'258, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cff0951:::obj258:head@111'258, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:27.542+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.542+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.542+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1cf1ad59:::obj156:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.542+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:1cf1ad59:::obj156:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 1.0 e 116/116 lb 1:1cf1ad59:::obj156:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:1cd4c3d5:::obj142:head local-lis/les=113/114 n=90 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.542+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.542+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1cd4c3d5:::obj142:head local-lis/les=113/114 n=90 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:1cf1ad59:::obj156:head) v3 2026-02-11T23:31:27.542+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1cd4c3d5:::obj142:head local-lis/les=113/114 n=90 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:1cf1ad59:::obj156:head) 2026-02-11T23:31:27.542+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.542+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:1cf1ad59:::obj156:head) v3 finish latency 0.000017 2026-02-11T23:31:27.542+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1cf1ad59:::obj156:head local-lis/les=113/114 n=91 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.542+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.542+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.542+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1cf1ad59:::obj156:head local-lis/les=113/114 n=91 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.542+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.542+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.542+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.542+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d78780 con 0x4acb000 2026-02-11T23:31:27.542+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:1cff0951:::obj258:head)]) -- 0x5e7f200 con 0x4acb000 2026-02-11T23:31:27.542+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.543+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 219 ==== MOSDPGPush(1.0 116/113 [PushOp(1:1d2e941a:::obj186:head, version: 111'186, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1d2e941a:::obj186:head@111'186, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5d7e9c0 con 0x4acb000 2026-02-11T23:31:27.543+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:1d2e941a:::obj186:head, version: 111'186, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1d2e941a:::obj186:head@111'186, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.543+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1d2e941a:::obj186:head, version: 111'186, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1d2e941a:::obj186:head@111'186, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.543+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.543+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1d2e941a:::obj186:head, version: 111'186, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1d2e941a:::obj186:head@111'186, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.543+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 220 ==== pg_backfill(progress 1.0 e 116/116 lb 1:1cff0951:::obj258:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc3600 con 0x4acb000 2026-02-11T23:31:27.543+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:1cff0951:::obj258:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.543+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1cff0951:::obj258:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.543+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.543+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1d2e941a:::obj186:head, version: 111'186, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1d2e941a:::obj186:head@111'186, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.543+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:1d2e941a:::obj186:head, version: 111'186, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1d2e941a:::obj186:head@111'186, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 116/113 [PushOp(1:1d2e941a:::obj186:head, version: 111'186, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1d2e941a:::obj186:head@111'186, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:1cf1ad59:::obj156:head local-lis/les=113/114 n=91 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.543+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.543+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.543+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1cf1ad59:::obj156:head local-lis/les=113/114 n=91 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:1d2e941a:::obj186:head, version: 111'186, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1d2e941a:::obj186:head@111'186, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.543+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1cff0951:::obj258:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.543+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1cf1ad59:::obj156:head local-lis/les=113/114 n=91 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:1d2e941a:::obj186:head@111'186, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.543+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1cf1ad59:::obj156:head local-lis/les=113/114 n=91 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:1d2e941a:::obj186:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.543+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1cf1ad59:::obj156:head local-lis/les=113/114 n=91 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:1d2e941a:::obj186:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.543+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1cf1ad59:::obj156:head local-lis/les=113/114 n=91 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:1d2e941a:::obj186:head 2026-02-11T23:31:27.543+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1cf1ad59:::obj156:head local-lis/les=113/114 n=91 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:1d2e941a:::obj186:head v 111'186 2026-02-11T23:31:27.543+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1cf1ad59:::obj156:head local-lis/les=113/114 n=91 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.543+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.543+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:1d2e941a:::obj186:head, version: 111'186, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1d2e941a:::obj186:head@111'186, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000062 2026-02-11T23:31:27.543+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1cf1ad59:::obj156:head local-lis/les=113/114 n=91 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.543+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.543+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.543+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1cff0951:::obj258:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.543+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:1cff0951:::obj258:head) v3 prio 15 cost 0 latency 0.000085 pg_backfill(progress 1.0 e 116/116 lb 1:1cff0951:::obj258:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:1cf1ad59:::obj156:head local-lis/les=113/114 n=91 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.543+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.543+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1cf1ad59:::obj156:head local-lis/les=113/114 n=91 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:1cff0951:::obj258:head) v3 2026-02-11T23:31:27.543+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1cf1ad59:::obj156:head local-lis/les=113/114 n=91 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:1cff0951:::obj258:head) 2026-02-11T23:31:27.543+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.543+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:1cff0951:::obj258:head) v3 finish latency 0.000017 2026-02-11T23:31:27.543+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.543+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.543+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1cff0951:::obj258:head local-lis/les=113/114 n=92 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.543+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.543+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.543+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.543+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x4ec2780 con 0x4acb000 2026-02-11T23:31:27.543+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:1d2e941a:::obj186:head)]) -- 0x5d79e00 con 0x4acb000 2026-02-11T23:31:27.543+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.544+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 221 ==== MOSDPGPush(1.0 116/113 [PushOp(1:1dafe4f0:::obj47:head, version: 111'47, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1dafe4f0:::obj47:head@111'47, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5d7eea0 con 0x4acb000 2026-02-11T23:31:27.544+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:1dafe4f0:::obj47:head, version: 111'47, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1dafe4f0:::obj47:head@111'47, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 116 2026-02-11T23:31:27.544+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1dafe4f0:::obj47:head, version: 111'47, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1dafe4f0:::obj47:head@111'47, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.544+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.544+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 222 ==== pg_backfill(progress 1.0 e 116/116 lb 1:1d2e941a:::obj186:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc4c00 con 0x4acb000 2026-02-11T23:31:27.544+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1dafe4f0:::obj47:head, version: 111'47, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1dafe4f0:::obj47:head@111'47, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.544+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:1d2e941a:::obj186:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.544+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1d2e941a:::obj186:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.544+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.544+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1dafe4f0:::obj47:head, version: 111'47, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1dafe4f0:::obj47:head@111'47, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.544+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:1dafe4f0:::obj47:head, version: 111'47, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1dafe4f0:::obj47:head@111'47, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 116/113 [PushOp(1:1dafe4f0:::obj47:head, version: 111'47, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1dafe4f0:::obj47:head@111'47, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:1cff0951:::obj258:head local-lis/les=113/114 n=92 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.544+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.544+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.544+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1cff0951:::obj258:head local-lis/les=113/114 n=92 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:1dafe4f0:::obj47:head, version: 111'47, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1dafe4f0:::obj47:head@111'47, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.544+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1d2e941a:::obj186:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.544+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1cff0951:::obj258:head local-lis/les=113/114 n=92 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:1dafe4f0:::obj47:head@111'47, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.544+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1cff0951:::obj258:head local-lis/les=113/114 n=92 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:1dafe4f0:::obj47:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.544+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1cff0951:::obj258:head local-lis/les=113/114 n=92 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:1dafe4f0:::obj47:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.544+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1cff0951:::obj258:head local-lis/les=113/114 n=92 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:1dafe4f0:::obj47:head 2026-02-11T23:31:27.544+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1cff0951:::obj258:head local-lis/les=113/114 n=92 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:1dafe4f0:::obj47:head v 111'47 2026-02-11T23:31:27.544+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1cff0951:::obj258:head local-lis/les=113/114 n=92 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.544+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.544+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:1dafe4f0:::obj47:head, version: 111'47, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1dafe4f0:::obj47:head@111'47, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:27.544+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.544+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.544+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1d2e941a:::obj186:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.544+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:1d2e941a:::obj186:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 1.0 e 116/116 lb 1:1d2e941a:::obj186:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:1cff0951:::obj258:head local-lis/les=113/114 n=92 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.544+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.544+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1cff0951:::obj258:head local-lis/les=113/114 n=92 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:1d2e941a:::obj186:head) v3 2026-02-11T23:31:27.544+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1cff0951:::obj258:head local-lis/les=113/114 n=92 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:1d2e941a:::obj186:head) 2026-02-11T23:31:27.544+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.544+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:1d2e941a:::obj186:head) v3 finish latency 0.000018 2026-02-11T23:31:27.544+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1d2e941a:::obj186:head local-lis/les=113/114 n=93 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.544+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.544+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.544+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1d2e941a:::obj186:head local-lis/les=113/114 n=93 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.544+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.544+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.544+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.544+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e6b680 con 0x4acb000 2026-02-11T23:31:27.544+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:1dafe4f0:::obj47:head)]) -- 0x5e7f680 con 0x4acb000 2026-02-11T23:31:27.544+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.544+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 223 ==== MOSDPGPush(1.0 116/113 [PushOp(1:1e53d0d0:::obj151:head, version: 111'151, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1e53d0d0:::obj151:head@111'151, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5d7f380 con 0x4acb000 2026-02-11T23:31:27.544+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:1e53d0d0:::obj151:head, version: 111'151, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1e53d0d0:::obj151:head@111'151, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.544+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1e53d0d0:::obj151:head, version: 111'151, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1e53d0d0:::obj151:head@111'151, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.544+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.544+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 224 ==== pg_backfill(progress 1.0 e 116/116 lb 1:1dafe4f0:::obj47:head) ==== 1061+0+0 (unknown 0 0 0) 0x5dc4680 con 0x4acb000 2026-02-11T23:31:27.544+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1e53d0d0:::obj151:head, version: 111'151, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1e53d0d0:::obj151:head@111'151, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.544+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:1dafe4f0:::obj47:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.544+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1dafe4f0:::obj47:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.544+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.544+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1e53d0d0:::obj151:head, version: 111'151, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1e53d0d0:::obj151:head@111'151, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.544+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:1e53d0d0:::obj151:head, version: 111'151, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1e53d0d0:::obj151:head@111'151, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(1.0 116/113 [PushOp(1:1e53d0d0:::obj151:head, version: 111'151, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1e53d0d0:::obj151:head@111'151, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:1d2e941a:::obj186:head local-lis/les=113/114 n=93 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.544+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.544+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.544+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1d2e941a:::obj186:head local-lis/les=113/114 n=93 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:1e53d0d0:::obj151:head, version: 111'151, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1e53d0d0:::obj151:head@111'151, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.544+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1dafe4f0:::obj47:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.544+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1d2e941a:::obj186:head local-lis/les=113/114 n=93 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:1e53d0d0:::obj151:head@111'151, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.544+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1d2e941a:::obj186:head local-lis/les=113/114 n=93 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:1e53d0d0:::obj151:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.544+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1d2e941a:::obj186:head local-lis/les=113/114 n=93 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:1e53d0d0:::obj151:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.544+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1d2e941a:::obj186:head local-lis/les=113/114 n=93 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:1e53d0d0:::obj151:head 2026-02-11T23:31:27.544+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1d2e941a:::obj186:head local-lis/les=113/114 n=93 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:1e53d0d0:::obj151:head v 111'151 2026-02-11T23:31:27.544+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1d2e941a:::obj186:head local-lis/les=113/114 n=93 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.544+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.544+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:1e53d0d0:::obj151:head, version: 111'151, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1e53d0d0:::obj151:head@111'151, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:27.544+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1d2e941a:::obj186:head local-lis/les=113/114 n=93 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.544+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.544+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.544+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1dafe4f0:::obj47:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.544+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:1dafe4f0:::obj47:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 1.0 e 116/116 lb 1:1dafe4f0:::obj47:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:1d2e941a:::obj186:head local-lis/les=113/114 n=93 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.544+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.544+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1d2e941a:::obj186:head local-lis/les=113/114 n=93 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:1dafe4f0:::obj47:head) v3 2026-02-11T23:31:27.544+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1d2e941a:::obj186:head local-lis/les=113/114 n=93 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:1dafe4f0:::obj47:head) 2026-02-11T23:31:27.544+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.544+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:1dafe4f0:::obj47:head) v3 finish latency 0.000017 2026-02-11T23:31:27.544+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.544+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.544+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1dafe4f0:::obj47:head local-lis/les=113/114 n=94 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.544+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.544+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.544+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.544+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e6b980 con 0x4acb000 2026-02-11T23:31:27.544+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:1e53d0d0:::obj151:head)]) -- 0x5e6a600 con 0x4acb000 2026-02-11T23:31:27.544+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.545+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 225 ==== MOSDPGPush(1.0 116/113 [PushOp(1:1efaad2b:::obj120:head, version: 111'120, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1efaad2b:::obj120:head@111'120, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5d7f860 con 0x4acb000 2026-02-11T23:31:27.545+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:1efaad2b:::obj120:head, version: 111'120, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1efaad2b:::obj120:head@111'120, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.545+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1efaad2b:::obj120:head, version: 111'120, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1efaad2b:::obj120:head@111'120, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.545+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.545+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 226 ==== pg_backfill(progress 1.0 e 116/116 lb 1:1e53d0d0:::obj151:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc3b80 con 0x4acb000 2026-02-11T23:31:27.545+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1efaad2b:::obj120:head, version: 111'120, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1efaad2b:::obj120:head@111'120, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.545+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:1e53d0d0:::obj151:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 116 2026-02-11T23:31:27.545+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.545+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1e53d0d0:::obj151:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.545+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:1efaad2b:::obj120:head, version: 111'120, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1efaad2b:::obj120:head@111'120, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.545+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.545+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:1efaad2b:::obj120:head, version: 111'120, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1efaad2b:::obj120:head@111'120, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 116/113 [PushOp(1:1efaad2b:::obj120:head, version: 111'120, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1efaad2b:::obj120:head@111'120, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:1dafe4f0:::obj47:head local-lis/les=113/114 n=94 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.545+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.545+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1dafe4f0:::obj47:head local-lis/les=113/114 n=94 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:1efaad2b:::obj120:head, version: 111'120, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1efaad2b:::obj120:head@111'120, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.545+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.545+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1e53d0d0:::obj151:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.545+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1dafe4f0:::obj47:head local-lis/les=113/114 n=94 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:1efaad2b:::obj120:head@111'120, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.545+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1dafe4f0:::obj47:head local-lis/les=113/114 n=94 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:1efaad2b:::obj120:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.545+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1dafe4f0:::obj47:head local-lis/les=113/114 n=94 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:1efaad2b:::obj120:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.545+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1dafe4f0:::obj47:head local-lis/les=113/114 n=94 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:1efaad2b:::obj120:head 2026-02-11T23:31:27.545+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1dafe4f0:::obj47:head local-lis/les=113/114 n=94 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:1efaad2b:::obj120:head v 111'120 2026-02-11T23:31:27.545+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1dafe4f0:::obj47:head local-lis/les=113/114 n=94 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.545+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.545+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:1efaad2b:::obj120:head, version: 111'120, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1efaad2b:::obj120:head@111'120, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:27.545+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.545+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.545+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1e53d0d0:::obj151:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.545+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:1e53d0d0:::obj151:head) v3 prio 15 cost 0 latency 0.000084 pg_backfill(progress 1.0 e 116/116 lb 1:1e53d0d0:::obj151:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:1dafe4f0:::obj47:head local-lis/les=113/114 n=94 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.545+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.545+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1dafe4f0:::obj47:head local-lis/les=113/114 n=94 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:1e53d0d0:::obj151:head) v3 2026-02-11T23:31:27.545+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1dafe4f0:::obj47:head local-lis/les=113/114 n=94 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:1e53d0d0:::obj151:head) 2026-02-11T23:31:27.545+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.545+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:1e53d0d0:::obj151:head) v3 finish latency 0.000017 2026-02-11T23:31:27.545+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1e53d0d0:::obj151:head local-lis/les=113/114 n=95 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.545+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.545+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.545+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1e53d0d0:::obj151:head local-lis/les=113/114 n=95 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.545+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.545+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.545+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.545+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e3b680 con 0x4acb000 2026-02-11T23:31:27.545+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:1efaad2b:::obj120:head)]) -- 0x5e7f380 con 0x4acb000 2026-02-11T23:31:27.545+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.545+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 227 ==== MOSDPGPush(1.0 116/113 [PushOp(1:20b87b8b:::obj369:head, version: 111'369, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:20b87b8b:::obj369:head@111'369, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db16c0 con 0x4acb000 2026-02-11T23:31:27.545+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:20b87b8b:::obj369:head, version: 111'369, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:20b87b8b:::obj369:head@111'369, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 116 2026-02-11T23:31:27.545+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:20b87b8b:::obj369:head, version: 111'369, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:20b87b8b:::obj369:head@111'369, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.545+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.545+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:20b87b8b:::obj369:head, version: 111'369, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:20b87b8b:::obj369:head@111'369, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.545+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 228 ==== pg_backfill(progress 1.0 e 116/116 lb 1:1efaad2b:::obj120:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc4100 con 0x4acb000 2026-02-11T23:31:27.545+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:1efaad2b:::obj120:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 116 2026-02-11T23:31:27.545+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.545+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1efaad2b:::obj120:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.545+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:20b87b8b:::obj369:head, version: 111'369, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:20b87b8b:::obj369:head@111'369, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.545+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.545+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:20b87b8b:::obj369:head, version: 111'369, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:20b87b8b:::obj369:head@111'369, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:20b87b8b:::obj369:head, version: 111'369, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:20b87b8b:::obj369:head@111'369, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:1e53d0d0:::obj151:head local-lis/les=113/114 n=95 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.545+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.545+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1e53d0d0:::obj151:head local-lis/les=113/114 n=95 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:20b87b8b:::obj369:head, version: 111'369, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:20b87b8b:::obj369:head@111'369, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.545+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.545+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1efaad2b:::obj120:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.545+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1e53d0d0:::obj151:head local-lis/les=113/114 n=95 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:20b87b8b:::obj369:head@111'369, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.546+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1e53d0d0:::obj151:head local-lis/les=113/114 n=95 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:20b87b8b:::obj369:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.546+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1e53d0d0:::obj151:head local-lis/les=113/114 n=95 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:20b87b8b:::obj369:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.546+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1e53d0d0:::obj151:head local-lis/les=113/114 n=95 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:20b87b8b:::obj369:head 2026-02-11T23:31:27.546+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1e53d0d0:::obj151:head local-lis/les=113/114 n=95 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:20b87b8b:::obj369:head v 111'369 2026-02-11T23:31:27.546+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1e53d0d0:::obj151:head local-lis/les=113/114 n=95 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.546+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.546+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:20b87b8b:::obj369:head, version: 111'369, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:20b87b8b:::obj369:head@111'369, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:27.546+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.546+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.546+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:1efaad2b:::obj120:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.546+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:1efaad2b:::obj120:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 1.0 e 116/116 lb 1:1efaad2b:::obj120:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:1e53d0d0:::obj151:head local-lis/les=113/114 n=95 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.546+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.546+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1e53d0d0:::obj151:head local-lis/les=113/114 n=95 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:1efaad2b:::obj120:head) v3 2026-02-11T23:31:27.546+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1e53d0d0:::obj151:head local-lis/les=113/114 n=95 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:1efaad2b:::obj120:head) 2026-02-11T23:31:27.546+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.546+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:1efaad2b:::obj120:head) v3 finish latency 0.000015 2026-02-11T23:31:27.546+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1efaad2b:::obj120:head local-lis/les=113/114 n=96 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.546+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.546+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.546+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1efaad2b:::obj120:head local-lis/les=113/114 n=96 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.546+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.546+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.546+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.546+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5df8000 con 0x4acb000 2026-02-11T23:31:27.546+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:20b87b8b:::obj369:head)]) -- 0x5e7e000 con 0x4acb000 2026-02-11T23:31:27.546+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.546+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 229 ==== MOSDPGPush(1.0 116/113 [PushOp(1:212e6483:::obj209:head, version: 111'209, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:212e6483:::obj209:head@111'209, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e27520 con 0x4acb000 2026-02-11T23:31:27.546+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:212e6483:::obj209:head, version: 111'209, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:212e6483:::obj209:head@111'209, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.546+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:212e6483:::obj209:head, version: 111'209, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:212e6483:::obj209:head@111'209, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.546+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.546+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 230 ==== pg_backfill(progress 1.0 e 116/116 lb 1:20b87b8b:::obj369:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc2b00 con 0x4acb000 2026-02-11T23:31:27.546+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:20b87b8b:::obj369:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.546+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:212e6483:::obj209:head, version: 111'209, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:212e6483:::obj209:head@111'209, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.546+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:20b87b8b:::obj369:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.546+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.546+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:212e6483:::obj209:head, version: 111'209, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:212e6483:::obj209:head@111'209, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.546+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.546+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:212e6483:::obj209:head, version: 111'209, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:212e6483:::obj209:head@111'209, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:212e6483:::obj209:head, version: 111'209, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:212e6483:::obj209:head@111'209, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:1efaad2b:::obj120:head local-lis/les=113/114 n=96 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.546+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:20b87b8b:::obj369:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.546+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.546+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1efaad2b:::obj120:head local-lis/les=113/114 n=96 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:212e6483:::obj209:head, version: 111'209, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:212e6483:::obj209:head@111'209, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.546+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1efaad2b:::obj120:head local-lis/les=113/114 n=96 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:212e6483:::obj209:head@111'209, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.546+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1efaad2b:::obj120:head local-lis/les=113/114 n=96 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:212e6483:::obj209:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.546+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1efaad2b:::obj120:head local-lis/les=113/114 n=96 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:212e6483:::obj209:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.546+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1efaad2b:::obj120:head local-lis/les=113/114 n=96 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:212e6483:::obj209:head 2026-02-11T23:31:27.546+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1efaad2b:::obj120:head local-lis/les=113/114 n=96 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:212e6483:::obj209:head v 111'209 2026-02-11T23:31:27.546+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1efaad2b:::obj120:head local-lis/les=113/114 n=96 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.546+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.546+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:212e6483:::obj209:head, version: 111'209, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:212e6483:::obj209:head@111'209, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:27.546+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.546+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.546+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:20b87b8b:::obj369:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.546+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:20b87b8b:::obj369:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 1.0 e 116/116 lb 1:20b87b8b:::obj369:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:1efaad2b:::obj120:head local-lis/les=113/114 n=96 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.546+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.546+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1efaad2b:::obj120:head local-lis/les=113/114 n=96 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:20b87b8b:::obj369:head) v3 2026-02-11T23:31:27.546+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:1efaad2b:::obj120:head local-lis/les=113/114 n=96 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:20b87b8b:::obj369:head) 2026-02-11T23:31:27.546+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.546+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:20b87b8b:::obj369:head) v3 finish latency 0.000015 2026-02-11T23:31:27.546+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:20b87b8b:::obj369:head local-lis/les=113/114 n=97 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.546+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.546+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.546+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:20b87b8b:::obj369:head local-lis/les=113/114 n=97 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.546+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.546+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.546+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.546+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5df8180 con 0x4acb000 2026-02-11T23:31:27.546+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:212e6483:::obj209:head)]) -- 0x5e7e480 con 0x4acb000 2026-02-11T23:31:27.546+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.547+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 231 ==== MOSDPGPush(1.0 116/113 [PushOp(1:2222704f:::obj161:head, version: 111'161, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2222704f:::obj161:head@111'161, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e27d40 con 0x4acb000 2026-02-11T23:31:27.547+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:2222704f:::obj161:head, version: 111'161, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2222704f:::obj161:head@111'161, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.547+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:2222704f:::obj161:head, version: 111'161, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2222704f:::obj161:head@111'161, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.547+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.547+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 232 ==== pg_backfill(progress 1.0 e 116/116 lb 1:212e6483:::obj209:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc5700 con 0x4acb000 2026-02-11T23:31:27.547+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:2222704f:::obj161:head, version: 111'161, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2222704f:::obj161:head@111'161, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.547+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:212e6483:::obj209:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.547+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:212e6483:::obj209:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.547+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.547+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:2222704f:::obj161:head, version: 111'161, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2222704f:::obj161:head@111'161, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.547+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:2222704f:::obj161:head, version: 111'161, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2222704f:::obj161:head@111'161, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 116/113 [PushOp(1:2222704f:::obj161:head, version: 111'161, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2222704f:::obj161:head@111'161, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:20b87b8b:::obj369:head local-lis/les=113/114 n=97 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.547+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.547+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.547+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:20b87b8b:::obj369:head local-lis/les=113/114 n=97 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:2222704f:::obj161:head, version: 111'161, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2222704f:::obj161:head@111'161, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.547+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:212e6483:::obj209:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.547+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:20b87b8b:::obj369:head local-lis/les=113/114 n=97 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:2222704f:::obj161:head@111'161, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.547+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:20b87b8b:::obj369:head local-lis/les=113/114 n=97 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:2222704f:::obj161:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.547+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:20b87b8b:::obj369:head local-lis/les=113/114 n=97 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:2222704f:::obj161:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.547+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:20b87b8b:::obj369:head local-lis/les=113/114 n=97 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:2222704f:::obj161:head 2026-02-11T23:31:27.547+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:20b87b8b:::obj369:head local-lis/les=113/114 n=97 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:2222704f:::obj161:head v 111'161 2026-02-11T23:31:27.547+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:20b87b8b:::obj369:head local-lis/les=113/114 n=97 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.547+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.547+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:2222704f:::obj161:head, version: 111'161, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2222704f:::obj161:head@111'161, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:27.547+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:20b87b8b:::obj369:head local-lis/les=113/114 n=97 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.547+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.547+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:212e6483:::obj209:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.547+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:212e6483:::obj209:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 1.0 e 116/116 lb 1:212e6483:::obj209:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:20b87b8b:::obj369:head local-lis/les=113/114 n=97 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.547+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.547+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.547+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:20b87b8b:::obj369:head local-lis/les=113/114 n=97 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:212e6483:::obj209:head) v3 2026-02-11T23:31:27.547+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:20b87b8b:::obj369:head local-lis/les=113/114 n=97 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:212e6483:::obj209:head) 2026-02-11T23:31:27.547+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.547+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:212e6483:::obj209:head) v3 finish latency 0.000018 2026-02-11T23:31:27.547+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.547+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.547+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:212e6483:::obj209:head local-lis/les=113/114 n=98 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.547+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.547+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.547+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.547+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5df8600 con 0x4acb000 2026-02-11T23:31:27.547+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:2222704f:::obj161:head)]) -- 0x5df8300 con 0x4acb000 2026-02-11T23:31:27.547+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.547+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 233 ==== MOSDPGPush(1.0 116/113 [PushOp(1:2260a46a:::obj167:head, version: 111'167, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2260a46a:::obj167:head@111'167, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e27ba0 con 0x4acb000 2026-02-11T23:31:27.547+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:2260a46a:::obj167:head, version: 111'167, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2260a46a:::obj167:head@111'167, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 116 2026-02-11T23:31:27.547+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:2260a46a:::obj167:head, version: 111'167, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2260a46a:::obj167:head@111'167, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.547+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.547+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 234 ==== pg_backfill(progress 1.0 e 116/116 lb 1:2222704f:::obj161:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc2580 con 0x4acb000 2026-02-11T23:31:27.547+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:2222704f:::obj161:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.547+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:2260a46a:::obj167:head, version: 111'167, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2260a46a:::obj167:head@111'167, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.547+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:2222704f:::obj161:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.547+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.547+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:2222704f:::obj161:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.547+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.547+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:2260a46a:::obj167:head, version: 111'167, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2260a46a:::obj167:head@111'167, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.547+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:2260a46a:::obj167:head, version: 111'167, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2260a46a:::obj167:head@111'167, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000040 MOSDPGPush(1.0 116/113 [PushOp(1:2260a46a:::obj167:head, version: 111'167, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2260a46a:::obj167:head@111'167, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:212e6483:::obj209:head local-lis/les=113/114 n=98 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.547+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.547+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:212e6483:::obj209:head local-lis/les=113/114 n=98 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:2260a46a:::obj167:head, version: 111'167, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2260a46a:::obj167:head@111'167, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.547+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:212e6483:::obj209:head local-lis/les=113/114 n=98 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:2260a46a:::obj167:head@111'167, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.547+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:212e6483:::obj209:head local-lis/les=113/114 n=98 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:2260a46a:::obj167:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.547+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:212e6483:::obj209:head local-lis/les=113/114 n=98 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:2260a46a:::obj167:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.547+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:212e6483:::obj209:head local-lis/les=113/114 n=98 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:2260a46a:::obj167:head 2026-02-11T23:31:27.547+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:212e6483:::obj209:head local-lis/les=113/114 n=98 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:2260a46a:::obj167:head v 111'167 2026-02-11T23:31:27.547+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:212e6483:::obj209:head local-lis/les=113/114 n=98 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.547+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.547+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:2260a46a:::obj167:head, version: 111'167, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2260a46a:::obj167:head@111'167, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:27.547+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.547+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.547+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:2222704f:::obj161:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.547+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:2222704f:::obj161:head) v3 prio 15 cost 0 latency 0.000082 pg_backfill(progress 1.0 e 116/116 lb 1:2222704f:::obj161:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:212e6483:::obj209:head local-lis/les=113/114 n=98 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.547+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.547+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:212e6483:::obj209:head local-lis/les=113/114 n=98 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:2222704f:::obj161:head) v3 2026-02-11T23:31:27.547+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:212e6483:::obj209:head local-lis/les=113/114 n=98 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:2222704f:::obj161:head) 2026-02-11T23:31:27.547+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.548+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:2222704f:::obj161:head) v3 finish latency 0.000021 2026-02-11T23:31:27.548+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2222704f:::obj161:head local-lis/les=113/114 n=99 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.548+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.548+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.548+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2222704f:::obj161:head local-lis/les=113/114 n=99 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.548+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.548+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.548+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.548+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5df8780 con 0x4acb000 2026-02-11T23:31:27.548+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:2260a46a:::obj167:head)]) -- 0x5d8e180 con 0x4acb000 2026-02-11T23:31:27.548+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.548+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 235 ==== MOSDPGPush(1.0 116/113 [PushOp(1:233a42c1:::obj6:head, version: 111'6, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:233a42c1:::obj6:head@111'6, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 4999+0+0 (unknown 0 0 0) 0x5d7fd40 con 0x4acb000 2026-02-11T23:31:27.548+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:233a42c1:::obj6:head, version: 111'6, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:233a42c1:::obj6:head@111'6, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 116 2026-02-11T23:31:27.548+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:233a42c1:::obj6:head, version: 111'6, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:233a42c1:::obj6:head@111'6, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.548+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.548+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 236 ==== pg_backfill(progress 1.0 e 116/116 lb 1:2260a46a:::obj167:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc3080 con 0x4acb000 2026-02-11T23:31:27.548+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:233a42c1:::obj6:head, version: 111'6, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:233a42c1:::obj6:head@111'6, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.548+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:2260a46a:::obj167:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 116 2026-02-11T23:31:27.548+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.548+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:2260a46a:::obj167:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.548+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:233a42c1:::obj6:head, version: 111'6, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:233a42c1:::obj6:head@111'6, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.548+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.548+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:233a42c1:::obj6:head, version: 111'6, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:233a42c1:::obj6:head@111'6, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 116/113 [PushOp(1:233a42c1:::obj6:head, version: 111'6, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:233a42c1:::obj6:head@111'6, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:2222704f:::obj161:head local-lis/les=113/114 n=99 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.548+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.548+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2222704f:::obj161:head local-lis/les=113/114 n=99 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:233a42c1:::obj6:head, version: 111'6, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:233a42c1:::obj6:head@111'6, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.548+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.548+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:2260a46a:::obj167:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.548+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2222704f:::obj161:head local-lis/les=113/114 n=99 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:233a42c1:::obj6:head@111'6, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.548+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2222704f:::obj161:head local-lis/les=113/114 n=99 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:233a42c1:::obj6:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.548+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2222704f:::obj161:head local-lis/les=113/114 n=99 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:233a42c1:::obj6:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.548+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2222704f:::obj161:head local-lis/les=113/114 n=99 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:233a42c1:::obj6:head 2026-02-11T23:31:27.548+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2222704f:::obj161:head local-lis/les=113/114 n=99 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:233a42c1:::obj6:head v 111'6 2026-02-11T23:31:27.548+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2222704f:::obj161:head local-lis/les=113/114 n=99 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.548+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.548+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:233a42c1:::obj6:head, version: 111'6, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:233a42c1:::obj6:head@111'6, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:27.548+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.548+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.548+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:2260a46a:::obj167:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.548+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:2260a46a:::obj167:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 1.0 e 116/116 lb 1:2260a46a:::obj167:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:2222704f:::obj161:head local-lis/les=113/114 n=99 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.548+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.548+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2222704f:::obj161:head local-lis/les=113/114 n=99 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:2260a46a:::obj167:head) v3 2026-02-11T23:31:27.548+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2222704f:::obj161:head local-lis/les=113/114 n=99 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:2260a46a:::obj167:head) 2026-02-11T23:31:27.548+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.548+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:2260a46a:::obj167:head) v3 finish latency 0.000014 2026-02-11T23:31:27.548+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2260a46a:::obj167:head local-lis/les=113/114 n=100 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.548+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.548+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.548+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2260a46a:::obj167:head local-lis/les=113/114 n=100 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.548+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.548+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.548+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.548+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5df8900 con 0x4acb000 2026-02-11T23:31:27.548+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:233a42c1:::obj6:head)]) -- 0x5d8e480 con 0x4acb000 2026-02-11T23:31:27.548+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.549+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 237 ==== MOSDPGPush(1.0 116/113 [PushOp(1:239f544e:::obj128:head, version: 111'128, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:239f544e:::obj128:head@111'128, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5d7e340 con 0x4acb000 2026-02-11T23:31:27.549+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:239f544e:::obj128:head, version: 111'128, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:239f544e:::obj128:head@111'128, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.549+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:239f544e:::obj128:head, version: 111'128, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:239f544e:::obj128:head@111'128, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.549+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.549+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 238 ==== pg_backfill(progress 1.0 e 116/116 lb 1:233a42c1:::obj6:head) ==== 1060+0+0 (unknown 0 0 0) 0x4a2cb00 con 0x4acb000 2026-02-11T23:31:27.549+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:239f544e:::obj128:head, version: 111'128, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:239f544e:::obj128:head@111'128, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.549+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:233a42c1:::obj6:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.549+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:233a42c1:::obj6:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.549+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.549+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:239f544e:::obj128:head, version: 111'128, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:239f544e:::obj128:head@111'128, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.549+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.549+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:239f544e:::obj128:head, version: 111'128, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:239f544e:::obj128:head@111'128, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 116/113 [PushOp(1:239f544e:::obj128:head, version: 111'128, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:239f544e:::obj128:head@111'128, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:2260a46a:::obj167:head local-lis/les=113/114 n=100 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.549+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.549+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2260a46a:::obj167:head local-lis/les=113/114 n=100 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:239f544e:::obj128:head, version: 111'128, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:239f544e:::obj128:head@111'128, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.549+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.549+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2260a46a:::obj167:head local-lis/les=113/114 n=100 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:239f544e:::obj128:head@111'128, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.549+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:233a42c1:::obj6:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.549+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2260a46a:::obj167:head local-lis/les=113/114 n=100 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:239f544e:::obj128:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.549+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2260a46a:::obj167:head local-lis/les=113/114 n=100 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:239f544e:::obj128:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.549+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2260a46a:::obj167:head local-lis/les=113/114 n=100 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:239f544e:::obj128:head 2026-02-11T23:31:27.549+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2260a46a:::obj167:head local-lis/les=113/114 n=100 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:239f544e:::obj128:head v 111'128 2026-02-11T23:31:27.549+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2260a46a:::obj167:head local-lis/les=113/114 n=100 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.549+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.549+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:239f544e:::obj128:head, version: 111'128, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:239f544e:::obj128:head@111'128, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:27.549+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.549+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.549+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:233a42c1:::obj6:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.549+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:233a42c1:::obj6:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 1.0 e 116/116 lb 1:233a42c1:::obj6:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:2260a46a:::obj167:head local-lis/les=113/114 n=100 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.549+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.549+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2260a46a:::obj167:head local-lis/les=113/114 n=100 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:233a42c1:::obj6:head) v3 2026-02-11T23:31:27.549+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2260a46a:::obj167:head local-lis/les=113/114 n=100 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:233a42c1:::obj6:head) 2026-02-11T23:31:27.549+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.549+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:233a42c1:::obj6:head) v3 finish latency 0.000017 2026-02-11T23:31:27.549+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:233a42c1:::obj6:head local-lis/les=113/114 n=101 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.549+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.549+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.549+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:233a42c1:::obj6:head local-lis/les=113/114 n=101 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.549+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.549+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.549+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.549+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5df8a80 con 0x4acb000 2026-02-11T23:31:27.549+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:239f544e:::obj128:head)]) -- 0x5d8e600 con 0x4acb000 2026-02-11T23:31:27.549+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.549+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 239 ==== MOSDPGPush(1.0 116/113 [PushOp(1:242c40e0:::obj23:head, version: 111'23, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:242c40e0:::obj23:head@111'23, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5d7e820 con 0x4acb000 2026-02-11T23:31:27.549+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:242c40e0:::obj23:head, version: 111'23, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:242c40e0:::obj23:head@111'23, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.549+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:242c40e0:::obj23:head, version: 111'23, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:242c40e0:::obj23:head@111'23, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.549+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.549+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 240 ==== pg_backfill(progress 1.0 e 116/116 lb 1:239f544e:::obj128:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e680 con 0x4acb000 2026-02-11T23:31:27.549+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:239f544e:::obj128:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.549+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:242c40e0:::obj23:head, version: 111'23, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:242c40e0:::obj23:head@111'23, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.549+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:239f544e:::obj128:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.549+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.549+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:242c40e0:::obj23:head, version: 111'23, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:242c40e0:::obj23:head@111'23, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.549+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.549+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:239f544e:::obj128:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.549+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:242c40e0:::obj23:head, version: 111'23, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:242c40e0:::obj23:head@111'23, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:242c40e0:::obj23:head, version: 111'23, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:242c40e0:::obj23:head@111'23, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:233a42c1:::obj6:head local-lis/les=113/114 n=101 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.549+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.549+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:233a42c1:::obj6:head local-lis/les=113/114 n=101 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:242c40e0:::obj23:head, version: 111'23, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:242c40e0:::obj23:head@111'23, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.549+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:233a42c1:::obj6:head local-lis/les=113/114 n=101 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:242c40e0:::obj23:head@111'23, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.549+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:233a42c1:::obj6:head local-lis/les=113/114 n=101 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:242c40e0:::obj23:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.549+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:233a42c1:::obj6:head local-lis/les=113/114 n=101 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:242c40e0:::obj23:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.549+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:233a42c1:::obj6:head local-lis/les=113/114 n=101 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:242c40e0:::obj23:head 2026-02-11T23:31:27.549+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:233a42c1:::obj6:head local-lis/les=113/114 n=101 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:242c40e0:::obj23:head v 111'23 2026-02-11T23:31:27.549+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:233a42c1:::obj6:head local-lis/les=113/114 n=101 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.549+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.549+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:242c40e0:::obj23:head, version: 111'23, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:242c40e0:::obj23:head@111'23, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.549+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.549+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.549+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:239f544e:::obj128:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.549+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:239f544e:::obj128:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 1.0 e 116/116 lb 1:239f544e:::obj128:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:233a42c1:::obj6:head local-lis/les=113/114 n=101 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.549+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.549+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:233a42c1:::obj6:head local-lis/les=113/114 n=101 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:239f544e:::obj128:head) v3 2026-02-11T23:31:27.549+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:233a42c1:::obj6:head local-lis/les=113/114 n=101 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:239f544e:::obj128:head) 2026-02-11T23:31:27.549+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.549+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:239f544e:::obj128:head) v3 finish latency 0.000016 2026-02-11T23:31:27.549+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:239f544e:::obj128:head local-lis/les=113/114 n=102 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.549+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.549+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.549+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:239f544e:::obj128:head local-lis/les=113/114 n=102 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.549+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.549+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.549+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.549+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5df8c00 con 0x4acb000 2026-02-11T23:31:27.549+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:242c40e0:::obj23:head)]) -- 0x5d8e900 con 0x4acb000 2026-02-11T23:31:27.549+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.550+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 241 ==== MOSDPGPush(1.0 116/113 [PushOp(1:249a23dc:::obj422:head, version: 111'422, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:249a23dc:::obj422:head@111'422, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5d7ed00 con 0x4acb000 2026-02-11T23:31:27.550+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:249a23dc:::obj422:head, version: 111'422, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:249a23dc:::obj422:head@111'422, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.550+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:249a23dc:::obj422:head, version: 111'422, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:249a23dc:::obj422:head@111'422, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.550+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.550+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 242 ==== pg_backfill(progress 1.0 e 116/116 lb 1:242c40e0:::obj23:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d2e100 con 0x4acb000 2026-02-11T23:31:27.550+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:249a23dc:::obj422:head, version: 111'422, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:249a23dc:::obj422:head@111'422, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.550+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:242c40e0:::obj23:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.550+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:242c40e0:::obj23:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.550+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.550+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:249a23dc:::obj422:head, version: 111'422, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:249a23dc:::obj422:head@111'422, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.550+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:249a23dc:::obj422:head, version: 111'422, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:249a23dc:::obj422:head@111'422, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(1.0 116/113 [PushOp(1:249a23dc:::obj422:head, version: 111'422, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:249a23dc:::obj422:head@111'422, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:239f544e:::obj128:head local-lis/les=113/114 n=102 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.550+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.550+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.550+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:242c40e0:::obj23:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.550+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:239f544e:::obj128:head local-lis/les=113/114 n=102 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:249a23dc:::obj422:head, version: 111'422, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:249a23dc:::obj422:head@111'422, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.550+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:239f544e:::obj128:head local-lis/les=113/114 n=102 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:249a23dc:::obj422:head@111'422, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.550+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:239f544e:::obj128:head local-lis/les=113/114 n=102 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:249a23dc:::obj422:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.550+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:239f544e:::obj128:head local-lis/les=113/114 n=102 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:249a23dc:::obj422:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.550+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:239f544e:::obj128:head local-lis/les=113/114 n=102 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:249a23dc:::obj422:head 2026-02-11T23:31:27.550+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:239f544e:::obj128:head local-lis/les=113/114 n=102 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:249a23dc:::obj422:head v 111'422 2026-02-11T23:31:27.550+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:239f544e:::obj128:head local-lis/les=113/114 n=102 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.550+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.550+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:249a23dc:::obj422:head, version: 111'422, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:249a23dc:::obj422:head@111'422, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:27.550+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:239f544e:::obj128:head local-lis/les=113/114 n=102 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.550+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.550+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.550+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:242c40e0:::obj23:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.550+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:242c40e0:::obj23:head) v3 prio 15 cost 0 latency 0.000080 pg_backfill(progress 1.0 e 116/116 lb 1:242c40e0:::obj23:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:239f544e:::obj128:head local-lis/les=113/114 n=102 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.550+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.550+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:239f544e:::obj128:head local-lis/les=113/114 n=102 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:242c40e0:::obj23:head) v3 2026-02-11T23:31:27.550+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:239f544e:::obj128:head local-lis/les=113/114 n=102 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:242c40e0:::obj23:head) 2026-02-11T23:31:27.550+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.550+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:242c40e0:::obj23:head) v3 finish latency 0.000017 2026-02-11T23:31:27.550+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.550+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.550+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:242c40e0:::obj23:head local-lis/les=113/114 n=103 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.550+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.550+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.550+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.550+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5df9080 con 0x4acb000 2026-02-11T23:31:27.550+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:249a23dc:::obj422:head)]) -- 0x5df8d80 con 0x4acb000 2026-02-11T23:31:27.550+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.550+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 243 ==== MOSDPGPush(1.0 116/113 [PushOp(1:24b5611b:::obj20:head, version: 111'20, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:24b5611b:::obj20:head@111'20, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5db09c0 con 0x4acb000 2026-02-11T23:31:27.550+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:24b5611b:::obj20:head, version: 111'20, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:24b5611b:::obj20:head@111'20, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.550+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:24b5611b:::obj20:head, version: 111'20, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:24b5611b:::obj20:head@111'20, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.550+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.550+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 244 ==== pg_backfill(progress 1.0 e 116/116 lb 1:249a23dc:::obj422:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2d080 con 0x4acb000 2026-02-11T23:31:27.550+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:24b5611b:::obj20:head, version: 111'20, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:24b5611b:::obj20:head@111'20, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.550+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:249a23dc:::obj422:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.550+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:249a23dc:::obj422:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.550+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.550+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:24b5611b:::obj20:head, version: 111'20, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:24b5611b:::obj20:head@111'20, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.550+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:24b5611b:::obj20:head, version: 111'20, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:24b5611b:::obj20:head@111'20, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:24b5611b:::obj20:head, version: 111'20, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:24b5611b:::obj20:head@111'20, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:242c40e0:::obj23:head local-lis/les=113/114 n=103 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.550+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.550+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.550+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:249a23dc:::obj422:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.550+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:242c40e0:::obj23:head local-lis/les=113/114 n=103 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:24b5611b:::obj20:head, version: 111'20, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:24b5611b:::obj20:head@111'20, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.550+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:242c40e0:::obj23:head local-lis/les=113/114 n=103 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:24b5611b:::obj20:head@111'20, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.551+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:242c40e0:::obj23:head local-lis/les=113/114 n=103 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:24b5611b:::obj20:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.551+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:242c40e0:::obj23:head local-lis/les=113/114 n=103 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:24b5611b:::obj20:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.551+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:242c40e0:::obj23:head local-lis/les=113/114 n=103 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:24b5611b:::obj20:head 2026-02-11T23:31:27.551+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:242c40e0:::obj23:head local-lis/les=113/114 n=103 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:24b5611b:::obj20:head v 111'20 2026-02-11T23:31:27.551+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:242c40e0:::obj23:head local-lis/les=113/114 n=103 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.551+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.551+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:24b5611b:::obj20:head, version: 111'20, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:24b5611b:::obj20:head@111'20, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000067 2026-02-11T23:31:27.551+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:242c40e0:::obj23:head local-lis/les=113/114 n=103 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.551+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.551+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.551+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:249a23dc:::obj422:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.551+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:249a23dc:::obj422:head) v3 prio 15 cost 0 latency 0.000091 pg_backfill(progress 1.0 e 116/116 lb 1:249a23dc:::obj422:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:242c40e0:::obj23:head local-lis/les=113/114 n=103 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.551+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.551+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:242c40e0:::obj23:head local-lis/les=113/114 n=103 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:249a23dc:::obj422:head) v3 2026-02-11T23:31:27.551+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:242c40e0:::obj23:head local-lis/les=113/114 n=103 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:249a23dc:::obj422:head) 2026-02-11T23:31:27.551+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.551+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:249a23dc:::obj422:head) v3 finish latency 0.000014 2026-02-11T23:31:27.551+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.551+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.551+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:249a23dc:::obj422:head local-lis/les=113/114 n=104 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.551+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.551+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.551+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.551+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5df9380 con 0x4acb000 2026-02-11T23:31:27.551+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:24b5611b:::obj20:head)]) -- 0x5df9200 con 0x4acb000 2026-02-11T23:31:27.551+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.551+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 245 ==== MOSDPGPush(1.0 116/113 [PushOp(1:24fc6e92:::obj17:head, version: 111'17, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:24fc6e92:::obj17:head@111'17, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5db1ba0 con 0x4acb000 2026-02-11T23:31:27.551+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:24fc6e92:::obj17:head, version: 111'17, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:24fc6e92:::obj17:head@111'17, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.551+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:24fc6e92:::obj17:head, version: 111'17, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:24fc6e92:::obj17:head@111'17, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.551+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.551+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 246 ==== pg_backfill(progress 1.0 e 116/116 lb 1:24b5611b:::obj20:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d2e680 con 0x4acb000 2026-02-11T23:31:27.551+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:24b5611b:::obj20:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.551+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:24fc6e92:::obj17:head, version: 111'17, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:24fc6e92:::obj17:head@111'17, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.551+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:24b5611b:::obj20:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.551+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.551+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:24fc6e92:::obj17:head, version: 111'17, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:24fc6e92:::obj17:head@111'17, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.551+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:24fc6e92:::obj17:head, version: 111'17, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:24fc6e92:::obj17:head@111'17, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:24fc6e92:::obj17:head, version: 111'17, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:24fc6e92:::obj17:head@111'17, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:249a23dc:::obj422:head local-lis/les=113/114 n=104 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.551+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.551+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.551+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:249a23dc:::obj422:head local-lis/les=113/114 n=104 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:24fc6e92:::obj17:head, version: 111'17, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:24fc6e92:::obj17:head@111'17, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.551+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:24b5611b:::obj20:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.551+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:249a23dc:::obj422:head local-lis/les=113/114 n=104 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:24fc6e92:::obj17:head@111'17, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.551+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:249a23dc:::obj422:head local-lis/les=113/114 n=104 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:24fc6e92:::obj17:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.551+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:249a23dc:::obj422:head local-lis/les=113/114 n=104 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:24fc6e92:::obj17:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.551+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:249a23dc:::obj422:head local-lis/les=113/114 n=104 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:24fc6e92:::obj17:head 2026-02-11T23:31:27.551+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:249a23dc:::obj422:head local-lis/les=113/114 n=104 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:24fc6e92:::obj17:head v 111'17 2026-02-11T23:31:27.551+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:249a23dc:::obj422:head local-lis/les=113/114 n=104 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.551+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.551+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:24fc6e92:::obj17:head, version: 111'17, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:24fc6e92:::obj17:head@111'17, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:27.551+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.551+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.551+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:24b5611b:::obj20:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.551+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:24b5611b:::obj20:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 1.0 e 116/116 lb 1:24b5611b:::obj20:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:249a23dc:::obj422:head local-lis/les=113/114 n=104 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.551+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.551+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:249a23dc:::obj422:head local-lis/les=113/114 n=104 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:24b5611b:::obj20:head) v3 2026-02-11T23:31:27.551+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:249a23dc:::obj422:head local-lis/les=113/114 n=104 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:24b5611b:::obj20:head) 2026-02-11T23:31:27.551+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.551+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:24b5611b:::obj20:head) v3 finish latency 0.000024 2026-02-11T23:31:27.551+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:24b5611b:::obj20:head local-lis/les=113/114 n=105 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.551+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.551+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.551+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:24b5611b:::obj20:head local-lis/les=113/114 n=105 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.551+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.551+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.551+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.551+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5df9500 con 0x4acb000 2026-02-11T23:31:27.551+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:24fc6e92:::obj17:head)]) -- 0x5d8ea80 con 0x4acb000 2026-02-11T23:31:27.551+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.551+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.551+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.552+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 247 ==== MOSDPGPush(1.0 116/113 [PushOp(1:2540f982:::obj403:head, version: 111'403, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2540f982:::obj403:head@111'403, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db1520 con 0x4acb000 2026-02-11T23:31:27.552+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:2540f982:::obj403:head, version: 111'403, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2540f982:::obj403:head@111'403, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.552+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:2540f982:::obj403:head, version: 111'403, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2540f982:::obj403:head@111'403, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.552+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.552+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 248 ==== pg_backfill(progress 1.0 e 116/116 lb 1:24fc6e92:::obj17:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d2e100 con 0x4acb000 2026-02-11T23:31:27.552+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:2540f982:::obj403:head, version: 111'403, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2540f982:::obj403:head@111'403, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.552+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:24fc6e92:::obj17:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.552+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.552+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:24fc6e92:::obj17:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.552+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:2540f982:::obj403:head, version: 111'403, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2540f982:::obj403:head@111'403, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.552+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.552+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:2540f982:::obj403:head, version: 111'403, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2540f982:::obj403:head@111'403, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(1.0 116/113 [PushOp(1:2540f982:::obj403:head, version: 111'403, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2540f982:::obj403:head@111'403, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:24b5611b:::obj20:head local-lis/les=113/114 n=105 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.552+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.552+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:24b5611b:::obj20:head local-lis/les=113/114 n=105 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:2540f982:::obj403:head, version: 111'403, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2540f982:::obj403:head@111'403, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.552+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:24b5611b:::obj20:head local-lis/les=113/114 n=105 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:2540f982:::obj403:head@111'403, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.552+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.552+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:24fc6e92:::obj17:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.552+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:24b5611b:::obj20:head local-lis/les=113/114 n=105 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:2540f982:::obj403:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.552+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:24b5611b:::obj20:head local-lis/les=113/114 n=105 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:2540f982:::obj403:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.552+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:24b5611b:::obj20:head local-lis/les=113/114 n=105 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:2540f982:::obj403:head 2026-02-11T23:31:27.552+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:24b5611b:::obj20:head local-lis/les=113/114 n=105 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:2540f982:::obj403:head v 111'403 2026-02-11T23:31:27.552+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:24b5611b:::obj20:head local-lis/les=113/114 n=105 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.552+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.552+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:2540f982:::obj403:head, version: 111'403, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2540f982:::obj403:head@111'403, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:27.552+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.552+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.552+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:24fc6e92:::obj17:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.552+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:24fc6e92:::obj17:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 1.0 e 116/116 lb 1:24fc6e92:::obj17:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:24b5611b:::obj20:head local-lis/les=113/114 n=105 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.552+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.552+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:24b5611b:::obj20:head local-lis/les=113/114 n=105 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:24fc6e92:::obj17:head) v3 2026-02-11T23:31:27.552+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:24b5611b:::obj20:head local-lis/les=113/114 n=105 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:24fc6e92:::obj17:head) 2026-02-11T23:31:27.552+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.552+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:24fc6e92:::obj17:head) v3 finish latency 0.000014 2026-02-11T23:31:27.552+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:24fc6e92:::obj17:head local-lis/les=113/114 n=106 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.552+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.552+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.552+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:24fc6e92:::obj17:head local-lis/les=113/114 n=106 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.552+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.552+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.552+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.552+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5df9680 con 0x4acb000 2026-02-11T23:31:27.552+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:2540f982:::obj403:head)]) -- 0x5d8ec00 con 0x4acb000 2026-02-11T23:31:27.552+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.552+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 249 ==== MOSDPGPush(1.0 116/113 [PushOp(1:2551b5e8:::obj231:head, version: 111'231, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2551b5e8:::obj231:head@111'231, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5d7f1e0 con 0x4acb000 2026-02-11T23:31:27.552+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:2551b5e8:::obj231:head, version: 111'231, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2551b5e8:::obj231:head@111'231, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.552+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:2551b5e8:::obj231:head, version: 111'231, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2551b5e8:::obj231:head@111'231, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.552+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.552+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 250 ==== pg_backfill(progress 1.0 e 116/116 lb 1:2540f982:::obj403:head) ==== 1062+0+0 (unknown 0 0 0) 0x4a2cb00 con 0x4acb000 2026-02-11T23:31:27.552+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:2551b5e8:::obj231:head, version: 111'231, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2551b5e8:::obj231:head@111'231, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.552+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:2540f982:::obj403:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 116 2026-02-11T23:31:27.552+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.552+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:2540f982:::obj403:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.552+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:2551b5e8:::obj231:head, version: 111'231, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2551b5e8:::obj231:head@111'231, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.552+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.552+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:2551b5e8:::obj231:head, version: 111'231, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2551b5e8:::obj231:head@111'231, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 116/113 [PushOp(1:2551b5e8:::obj231:head, version: 111'231, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2551b5e8:::obj231:head@111'231, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:24fc6e92:::obj17:head local-lis/les=113/114 n=106 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.552+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.552+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:24fc6e92:::obj17:head local-lis/les=113/114 n=106 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:2551b5e8:::obj231:head, version: 111'231, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2551b5e8:::obj231:head@111'231, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.552+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:24fc6e92:::obj17:head local-lis/les=113/114 n=106 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:2551b5e8:::obj231:head@111'231, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.552+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.552+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:2540f982:::obj403:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.552+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:24fc6e92:::obj17:head local-lis/les=113/114 n=106 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:2551b5e8:::obj231:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.552+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:24fc6e92:::obj17:head local-lis/les=113/114 n=106 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:2551b5e8:::obj231:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.552+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:24fc6e92:::obj17:head local-lis/les=113/114 n=106 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:2551b5e8:::obj231:head 2026-02-11T23:31:27.552+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:24fc6e92:::obj17:head local-lis/les=113/114 n=106 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:2551b5e8:::obj231:head v 111'231 2026-02-11T23:31:27.552+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:24fc6e92:::obj17:head local-lis/les=113/114 n=106 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.552+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.552+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:2551b5e8:::obj231:head, version: 111'231, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2551b5e8:::obj231:head@111'231, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:27.552+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.552+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.552+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:2540f982:::obj403:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.552+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:2540f982:::obj403:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 1.0 e 116/116 lb 1:2540f982:::obj403:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:24fc6e92:::obj17:head local-lis/les=113/114 n=106 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.552+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.552+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:24fc6e92:::obj17:head local-lis/les=113/114 n=106 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:2540f982:::obj403:head) v3 2026-02-11T23:31:27.552+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:24fc6e92:::obj17:head local-lis/les=113/114 n=106 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:2540f982:::obj403:head) 2026-02-11T23:31:27.552+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.552+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:2540f982:::obj403:head) v3 finish latency 0.000015 2026-02-11T23:31:27.552+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2540f982:::obj403:head local-lis/les=113/114 n=107 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.552+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.552+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.552+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2540f982:::obj403:head local-lis/les=113/114 n=107 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.552+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.552+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.552+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.552+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5df9800 con 0x4acb000 2026-02-11T23:31:27.552+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:2551b5e8:::obj231:head)]) -- 0x5d8ef00 con 0x4acb000 2026-02-11T23:31:27.552+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.553+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 251 ==== MOSDPGPush(1.0 116/113 [PushOp(1:26b5008c:::obj336:head, version: 111'336, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:26b5008c:::obj336:head@111'336, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5d7f6c0 con 0x4acb000 2026-02-11T23:31:27.553+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:26b5008c:::obj336:head, version: 111'336, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:26b5008c:::obj336:head@111'336, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.553+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:26b5008c:::obj336:head, version: 111'336, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:26b5008c:::obj336:head@111'336, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.553+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.553+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:26b5008c:::obj336:head, version: 111'336, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:26b5008c:::obj336:head@111'336, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.553+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 252 ==== pg_backfill(progress 1.0 e 116/116 lb 1:2551b5e8:::obj231:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc3600 con 0x4acb000 2026-02-11T23:31:27.553+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:2551b5e8:::obj231:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.553+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.553+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:26b5008c:::obj336:head, version: 111'336, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:26b5008c:::obj336:head@111'336, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.553+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:2551b5e8:::obj231:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.553+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.553+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:26b5008c:::obj336:head, version: 111'336, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:26b5008c:::obj336:head@111'336, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 116/113 [PushOp(1:26b5008c:::obj336:head, version: 111'336, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:26b5008c:::obj336:head@111'336, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:2540f982:::obj403:head local-lis/les=113/114 n=107 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.553+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:2551b5e8:::obj231:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.553+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.553+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2540f982:::obj403:head local-lis/les=113/114 n=107 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:26b5008c:::obj336:head, version: 111'336, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:26b5008c:::obj336:head@111'336, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.553+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2540f982:::obj403:head local-lis/les=113/114 n=107 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:26b5008c:::obj336:head@111'336, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.553+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2540f982:::obj403:head local-lis/les=113/114 n=107 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:26b5008c:::obj336:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.553+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2540f982:::obj403:head local-lis/les=113/114 n=107 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:26b5008c:::obj336:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.553+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2540f982:::obj403:head local-lis/les=113/114 n=107 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:26b5008c:::obj336:head 2026-02-11T23:31:27.553+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2540f982:::obj403:head local-lis/les=113/114 n=107 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:26b5008c:::obj336:head v 111'336 2026-02-11T23:31:27.553+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2540f982:::obj403:head local-lis/les=113/114 n=107 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.553+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.553+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:26b5008c:::obj336:head, version: 111'336, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:26b5008c:::obj336:head@111'336, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:27.553+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2540f982:::obj403:head local-lis/les=113/114 n=107 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.553+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.553+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:2551b5e8:::obj231:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.553+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:2551b5e8:::obj231:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 1.0 e 116/116 lb 1:2551b5e8:::obj231:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:2540f982:::obj403:head local-lis/les=113/114 n=107 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.553+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.553+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2540f982:::obj403:head local-lis/les=113/114 n=107 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:2551b5e8:::obj231:head) v3 2026-02-11T23:31:27.553+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.553+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2540f982:::obj403:head local-lis/les=113/114 n=107 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:2551b5e8:::obj231:head) 2026-02-11T23:31:27.553+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.553+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:2551b5e8:::obj231:head) v3 finish latency 0.000024 2026-02-11T23:31:27.553+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.553+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.553+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2551b5e8:::obj231:head local-lis/les=113/114 n=108 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.553+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.553+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.553+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.553+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5df9c80 con 0x4acb000 2026-02-11T23:31:27.553+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:26b5008c:::obj336:head)]) -- 0x5df9980 con 0x4acb000 2026-02-11T23:31:27.553+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.553+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.553+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.554+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 253 ==== MOSDPGPush(1.0 116/113 [PushOp(1:26cf9b2d:::obj546:head, version: 111'546, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:26cf9b2d:::obj546:head@111'546, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5d7fba0 con 0x4acb000 2026-02-11T23:31:27.554+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:26cf9b2d:::obj546:head, version: 111'546, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:26cf9b2d:::obj546:head@111'546, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 116 2026-02-11T23:31:27.554+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:26cf9b2d:::obj546:head, version: 111'546, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:26cf9b2d:::obj546:head@111'546, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.554+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.554+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:26cf9b2d:::obj546:head, version: 111'546, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:26cf9b2d:::obj546:head@111'546, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.554+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 254 ==== pg_backfill(progress 1.0 e 116/116 lb 1:26b5008c:::obj336:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc4c00 con 0x4acb000 2026-02-11T23:31:27.554+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:26b5008c:::obj336:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.554+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.554+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:26b5008c:::obj336:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.554+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:26cf9b2d:::obj546:head, version: 111'546, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:26cf9b2d:::obj546:head@111'546, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.554+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.554+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:26cf9b2d:::obj546:head, version: 111'546, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:26cf9b2d:::obj546:head@111'546, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 116/113 [PushOp(1:26cf9b2d:::obj546:head, version: 111'546, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:26cf9b2d:::obj546:head@111'546, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:2551b5e8:::obj231:head local-lis/les=113/114 n=108 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.554+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.554+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2551b5e8:::obj231:head local-lis/les=113/114 n=108 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:26cf9b2d:::obj546:head, version: 111'546, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:26cf9b2d:::obj546:head@111'546, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.554+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2551b5e8:::obj231:head local-lis/les=113/114 n=108 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:26cf9b2d:::obj546:head@111'546, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.554+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.554+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:26b5008c:::obj336:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.554+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2551b5e8:::obj231:head local-lis/les=113/114 n=108 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:26cf9b2d:::obj546:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.554+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2551b5e8:::obj231:head local-lis/les=113/114 n=108 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:26cf9b2d:::obj546:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.554+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2551b5e8:::obj231:head local-lis/les=113/114 n=108 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:26cf9b2d:::obj546:head 2026-02-11T23:31:27.554+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2551b5e8:::obj231:head local-lis/les=113/114 n=108 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:26cf9b2d:::obj546:head v 111'546 2026-02-11T23:31:27.554+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2551b5e8:::obj231:head local-lis/les=113/114 n=108 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.554+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.554+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:26cf9b2d:::obj546:head, version: 111'546, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:26cf9b2d:::obj546:head@111'546, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:27.554+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2551b5e8:::obj231:head local-lis/les=113/114 n=108 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.554+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.554+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.554+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:26b5008c:::obj336:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.554+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:26b5008c:::obj336:head) v3 prio 15 cost 0 latency 0.000085 pg_backfill(progress 1.0 e 116/116 lb 1:26b5008c:::obj336:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:2551b5e8:::obj231:head local-lis/les=113/114 n=108 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.554+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.554+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2551b5e8:::obj231:head local-lis/les=113/114 n=108 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:26b5008c:::obj336:head) v3 2026-02-11T23:31:27.554+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2551b5e8:::obj231:head local-lis/les=113/114 n=108 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:26b5008c:::obj336:head) 2026-02-11T23:31:27.554+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.554+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:26b5008c:::obj336:head) v3 finish latency 0.000013 2026-02-11T23:31:27.554+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.554+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.554+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:26b5008c:::obj336:head local-lis/les=113/114 n=109 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.554+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.554+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.554+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.554+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e7ed80 con 0x4acb000 2026-02-11T23:31:27.554+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:26cf9b2d:::obj546:head)]) -- 0x5df9e00 con 0x4acb000 2026-02-11T23:31:27.554+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.554+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 255 ==== MOSDPGPush(1.0 116/113 [PushOp(1:27794e2d:::obj115:head, version: 111'115, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:27794e2d:::obj115:head@111'115, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e26000 con 0x4acb000 2026-02-11T23:31:27.554+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:27794e2d:::obj115:head, version: 111'115, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:27794e2d:::obj115:head@111'115, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.554+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:27794e2d:::obj115:head, version: 111'115, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:27794e2d:::obj115:head@111'115, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.554+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.554+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 256 ==== pg_backfill(progress 1.0 e 116/116 lb 1:26cf9b2d:::obj546:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc4680 con 0x4acb000 2026-02-11T23:31:27.554+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:27794e2d:::obj115:head, version: 111'115, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:27794e2d:::obj115:head@111'115, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.554+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:26cf9b2d:::obj546:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 116 2026-02-11T23:31:27.554+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.554+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:26cf9b2d:::obj546:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.554+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:27794e2d:::obj115:head, version: 111'115, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:27794e2d:::obj115:head@111'115, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.554+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.554+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:27794e2d:::obj115:head, version: 111'115, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:27794e2d:::obj115:head@111'115, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 116/113 [PushOp(1:27794e2d:::obj115:head, version: 111'115, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:27794e2d:::obj115:head@111'115, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:26b5008c:::obj336:head local-lis/les=113/114 n=109 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.554+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.554+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:26b5008c:::obj336:head local-lis/les=113/114 n=109 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:27794e2d:::obj115:head, version: 111'115, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:27794e2d:::obj115:head@111'115, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.554+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:26b5008c:::obj336:head local-lis/les=113/114 n=109 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:27794e2d:::obj115:head@111'115, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.554+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.554+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:26cf9b2d:::obj546:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.554+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:26b5008c:::obj336:head local-lis/les=113/114 n=109 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:27794e2d:::obj115:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.554+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:26b5008c:::obj336:head local-lis/les=113/114 n=109 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:27794e2d:::obj115:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.554+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:26b5008c:::obj336:head local-lis/les=113/114 n=109 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:27794e2d:::obj115:head 2026-02-11T23:31:27.554+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:26b5008c:::obj336:head local-lis/les=113/114 n=109 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:27794e2d:::obj115:head v 111'115 2026-02-11T23:31:27.554+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:26b5008c:::obj336:head local-lis/les=113/114 n=109 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.554+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.554+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:27794e2d:::obj115:head, version: 111'115, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:27794e2d:::obj115:head@111'115, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:27.554+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.554+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.554+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:26cf9b2d:::obj546:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.554+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:26cf9b2d:::obj546:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 1.0 e 116/116 lb 1:26cf9b2d:::obj546:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:26b5008c:::obj336:head local-lis/les=113/114 n=109 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.554+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.554+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:26b5008c:::obj336:head local-lis/les=113/114 n=109 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:26cf9b2d:::obj546:head) v3 2026-02-11T23:31:27.554+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:26b5008c:::obj336:head local-lis/les=113/114 n=109 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:26cf9b2d:::obj546:head) 2026-02-11T23:31:27.554+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.554+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:26cf9b2d:::obj546:head) v3 finish latency 0.000017 2026-02-11T23:31:27.554+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:26cf9b2d:::obj546:head local-lis/les=113/114 n=110 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.554+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.554+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.554+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:26cf9b2d:::obj546:head local-lis/les=113/114 n=110 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.554+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.554+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.554+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.554+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e7fe00 con 0x4acb000 2026-02-11T23:31:27.554+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:27794e2d:::obj115:head)]) -- 0x5d8f200 con 0x4acb000 2026-02-11T23:31:27.554+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.555+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 257 ==== MOSDPGPush(1.0 116/113 [PushOp(1:27952899:::obj201:head, version: 111'201, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:27952899:::obj201:head@111'201, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e26340 con 0x4acb000 2026-02-11T23:31:27.555+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:27952899:::obj201:head, version: 111'201, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:27952899:::obj201:head@111'201, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.555+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:27952899:::obj201:head, version: 111'201, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:27952899:::obj201:head@111'201, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.555+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.555+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:27952899:::obj201:head, version: 111'201, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:27952899:::obj201:head@111'201, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.555+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 258 ==== pg_backfill(progress 1.0 e 116/116 lb 1:27794e2d:::obj115:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc3b80 con 0x4acb000 2026-02-11T23:31:27.555+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.555+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:27794e2d:::obj115:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.555+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:27952899:::obj201:head, version: 111'201, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:27952899:::obj201:head@111'201, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.555+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:27794e2d:::obj115:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.555+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.555+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:27794e2d:::obj115:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.555+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:27952899:::obj201:head, version: 111'201, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:27952899:::obj201:head@111'201, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 116/113 [PushOp(1:27952899:::obj201:head, version: 111'201, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:27952899:::obj201:head@111'201, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:26cf9b2d:::obj546:head local-lis/les=113/114 n=110 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.555+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.555+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:26cf9b2d:::obj546:head local-lis/les=113/114 n=110 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:27952899:::obj201:head, version: 111'201, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:27952899:::obj201:head@111'201, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.555+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:26cf9b2d:::obj546:head local-lis/les=113/114 n=110 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:27952899:::obj201:head@111'201, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.555+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:26cf9b2d:::obj546:head local-lis/les=113/114 n=110 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:27952899:::obj201:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.555+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:26cf9b2d:::obj546:head local-lis/les=113/114 n=110 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:27952899:::obj201:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.555+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:26cf9b2d:::obj546:head local-lis/les=113/114 n=110 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:27952899:::obj201:head 2026-02-11T23:31:27.555+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:26cf9b2d:::obj546:head local-lis/les=113/114 n=110 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:27952899:::obj201:head v 111'201 2026-02-11T23:31:27.555+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:26cf9b2d:::obj546:head local-lis/les=113/114 n=110 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.555+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.555+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:27952899:::obj201:head, version: 111'201, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:27952899:::obj201:head@111'201, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:27.555+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.555+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.555+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:27794e2d:::obj115:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.555+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:27794e2d:::obj115:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 1.0 e 116/116 lb 1:27794e2d:::obj115:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:26cf9b2d:::obj546:head local-lis/les=113/114 n=110 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.555+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.555+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:26cf9b2d:::obj546:head local-lis/les=113/114 n=110 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:27794e2d:::obj115:head) v3 2026-02-11T23:31:27.555+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:26cf9b2d:::obj546:head local-lis/les=113/114 n=110 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:27794e2d:::obj115:head) 2026-02-11T23:31:27.555+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.555+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:27794e2d:::obj115:head) v3 finish latency 0.000015 2026-02-11T23:31:27.555+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:27794e2d:::obj115:head local-lis/les=113/114 n=111 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.555+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.555+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.555+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:27794e2d:::obj115:head local-lis/les=113/114 n=111 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.555+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.555+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.555+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.555+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e7f200 con 0x4acb000 2026-02-11T23:31:27.555+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:27952899:::obj201:head)]) -- 0x5d8f500 con 0x4acb000 2026-02-11T23:31:27.555+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.555+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 259 ==== MOSDPGPush(1.0 116/113 [PushOp(1:29cd40a5:::obj549:head, version: 111'549, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:29cd40a5:::obj549:head@111'549, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e271e0 con 0x4acb000 2026-02-11T23:31:27.555+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:29cd40a5:::obj549:head, version: 111'549, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:29cd40a5:::obj549:head@111'549, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.555+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:29cd40a5:::obj549:head, version: 111'549, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:29cd40a5:::obj549:head@111'549, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.555+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.555+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:29cd40a5:::obj549:head, version: 111'549, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:29cd40a5:::obj549:head@111'549, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.555+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 260 ==== pg_backfill(progress 1.0 e 116/116 lb 1:27952899:::obj201:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc4100 con 0x4acb000 2026-02-11T23:31:27.555+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.555+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:27952899:::obj201:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 116 2026-02-11T23:31:27.555+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:29cd40a5:::obj549:head, version: 111'549, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:29cd40a5:::obj549:head@111'549, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.555+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:27952899:::obj201:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.555+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.555+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:27952899:::obj201:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.555+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:29cd40a5:::obj549:head, version: 111'549, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:29cd40a5:::obj549:head@111'549, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(1.0 116/113 [PushOp(1:29cd40a5:::obj549:head, version: 111'549, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:29cd40a5:::obj549:head@111'549, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:27794e2d:::obj115:head local-lis/les=113/114 n=111 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.555+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.555+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:27794e2d:::obj115:head local-lis/les=113/114 n=111 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:29cd40a5:::obj549:head, version: 111'549, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:29cd40a5:::obj549:head@111'549, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.555+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:27794e2d:::obj115:head local-lis/les=113/114 n=111 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:29cd40a5:::obj549:head@111'549, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.555+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:27794e2d:::obj115:head local-lis/les=113/114 n=111 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:29cd40a5:::obj549:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.555+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:27794e2d:::obj115:head local-lis/les=113/114 n=111 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:29cd40a5:::obj549:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.555+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:27794e2d:::obj115:head local-lis/les=113/114 n=111 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:29cd40a5:::obj549:head 2026-02-11T23:31:27.555+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:27794e2d:::obj115:head local-lis/les=113/114 n=111 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:29cd40a5:::obj549:head v 111'549 2026-02-11T23:31:27.555+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:27794e2d:::obj115:head local-lis/les=113/114 n=111 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.555+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.555+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:29cd40a5:::obj549:head, version: 111'549, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:29cd40a5:::obj549:head@111'549, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.555+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.555+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.555+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:27952899:::obj201:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.555+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:27952899:::obj201:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 1.0 e 116/116 lb 1:27952899:::obj201:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:27794e2d:::obj115:head local-lis/les=113/114 n=111 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.555+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.555+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:27794e2d:::obj115:head local-lis/les=113/114 n=111 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:27952899:::obj201:head) v3 2026-02-11T23:31:27.555+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:27794e2d:::obj115:head local-lis/les=113/114 n=111 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:27952899:::obj201:head) 2026-02-11T23:31:27.555+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.555+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:27952899:::obj201:head) v3 finish latency 0.000014 2026-02-11T23:31:27.555+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:27952899:::obj201:head local-lis/les=113/114 n=112 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.555+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.556+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.556+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:27952899:::obj201:head local-lis/les=113/114 n=112 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.556+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.556+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.556+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.556+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e7f680 con 0x4acb000 2026-02-11T23:31:27.556+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:29cd40a5:::obj549:head)]) -- 0x5d8f800 con 0x4acb000 2026-02-11T23:31:27.556+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.556+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 261 ==== MOSDPGPush(1.0 116/113 [PushOp(1:2a627378:::obj100:head, version: 111'100, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2a627378:::obj100:head@111'100, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db09c0 con 0x4acb000 2026-02-11T23:31:27.556+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:2a627378:::obj100:head, version: 111'100, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2a627378:::obj100:head@111'100, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.556+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:2a627378:::obj100:head, version: 111'100, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2a627378:::obj100:head@111'100, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.556+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.556+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 262 ==== pg_backfill(progress 1.0 e 116/116 lb 1:29cd40a5:::obj549:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc2b00 con 0x4acb000 2026-02-11T23:31:27.556+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:2a627378:::obj100:head, version: 111'100, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2a627378:::obj100:head@111'100, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.556+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:29cd40a5:::obj549:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.556+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.556+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:29cd40a5:::obj549:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.556+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:2a627378:::obj100:head, version: 111'100, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2a627378:::obj100:head@111'100, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.556+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.556+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:2a627378:::obj100:head, version: 111'100, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2a627378:::obj100:head@111'100, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(1.0 116/113 [PushOp(1:2a627378:::obj100:head, version: 111'100, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2a627378:::obj100:head@111'100, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:27952899:::obj201:head local-lis/les=113/114 n=112 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.556+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.556+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:27952899:::obj201:head local-lis/les=113/114 n=112 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:2a627378:::obj100:head, version: 111'100, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2a627378:::obj100:head@111'100, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.556+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:27952899:::obj201:head local-lis/les=113/114 n=112 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:2a627378:::obj100:head@111'100, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.556+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.556+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:29cd40a5:::obj549:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.556+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:27952899:::obj201:head local-lis/les=113/114 n=112 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:2a627378:::obj100:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.556+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:27952899:::obj201:head local-lis/les=113/114 n=112 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:2a627378:::obj100:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.556+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:27952899:::obj201:head local-lis/les=113/114 n=112 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:2a627378:::obj100:head 2026-02-11T23:31:27.556+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:27952899:::obj201:head local-lis/les=113/114 n=112 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:2a627378:::obj100:head v 111'100 2026-02-11T23:31:27.556+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:27952899:::obj201:head local-lis/les=113/114 n=112 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.556+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.556+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:2a627378:::obj100:head, version: 111'100, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2a627378:::obj100:head@111'100, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:27.556+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.556+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.556+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:29cd40a5:::obj549:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.556+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:29cd40a5:::obj549:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 1.0 e 116/116 lb 1:29cd40a5:::obj549:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:27952899:::obj201:head local-lis/les=113/114 n=112 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.556+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.556+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:27952899:::obj201:head local-lis/les=113/114 n=112 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:29cd40a5:::obj549:head) v3 2026-02-11T23:31:27.556+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:27952899:::obj201:head local-lis/les=113/114 n=112 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:29cd40a5:::obj549:head) 2026-02-11T23:31:27.556+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.556+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:29cd40a5:::obj549:head) v3 finish latency 0.000013 2026-02-11T23:31:27.556+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:29cd40a5:::obj549:head local-lis/les=113/114 n=113 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.556+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.556+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.556+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:29cd40a5:::obj549:head local-lis/les=113/114 n=113 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.556+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.556+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.556+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.556+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5df8000 con 0x4acb000 2026-02-11T23:31:27.556+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:2a627378:::obj100:head)]) -- 0x5d8fb00 con 0x4acb000 2026-02-11T23:31:27.556+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.557+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 263 ==== MOSDPGPush(1.0 116/113 [PushOp(1:2acce685:::obj583:head, version: 111'583, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2acce685:::obj583:head@111'583, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db1ba0 con 0x4acb000 2026-02-11T23:31:27.557+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:2acce685:::obj583:head, version: 111'583, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2acce685:::obj583:head@111'583, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.557+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:2acce685:::obj583:head, version: 111'583, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2acce685:::obj583:head@111'583, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.557+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.557+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 264 ==== pg_backfill(progress 1.0 e 116/116 lb 1:2a627378:::obj100:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc5700 con 0x4acb000 2026-02-11T23:31:27.557+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:2acce685:::obj583:head, version: 111'583, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2acce685:::obj583:head@111'583, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.557+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:2a627378:::obj100:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.557+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.557+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:2a627378:::obj100:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.557+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:2acce685:::obj583:head, version: 111'583, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2acce685:::obj583:head@111'583, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.557+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.557+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:2acce685:::obj583:head, version: 111'583, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2acce685:::obj583:head@111'583, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(1.0 116/113 [PushOp(1:2acce685:::obj583:head, version: 111'583, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2acce685:::obj583:head@111'583, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:29cd40a5:::obj549:head local-lis/les=113/114 n=113 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.557+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.557+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:29cd40a5:::obj549:head local-lis/les=113/114 n=113 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:2acce685:::obj583:head, version: 111'583, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2acce685:::obj583:head@111'583, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.557+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:29cd40a5:::obj549:head local-lis/les=113/114 n=113 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:2acce685:::obj583:head@111'583, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.557+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.557+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:2a627378:::obj100:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.557+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:29cd40a5:::obj549:head local-lis/les=113/114 n=113 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:2acce685:::obj583:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.557+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:29cd40a5:::obj549:head local-lis/les=113/114 n=113 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:2acce685:::obj583:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.557+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:29cd40a5:::obj549:head local-lis/les=113/114 n=113 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:2acce685:::obj583:head 2026-02-11T23:31:27.557+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:29cd40a5:::obj549:head local-lis/les=113/114 n=113 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:2acce685:::obj583:head v 111'583 2026-02-11T23:31:27.557+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:29cd40a5:::obj549:head local-lis/les=113/114 n=113 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.557+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.557+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:2acce685:::obj583:head, version: 111'583, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2acce685:::obj583:head@111'583, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:27.557+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.557+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.557+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:2a627378:::obj100:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.557+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:2a627378:::obj100:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 1.0 e 116/116 lb 1:2a627378:::obj100:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:29cd40a5:::obj549:head local-lis/les=113/114 n=113 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.557+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.557+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:29cd40a5:::obj549:head local-lis/les=113/114 n=113 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:2a627378:::obj100:head) v3 2026-02-11T23:31:27.557+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:29cd40a5:::obj549:head local-lis/les=113/114 n=113 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:2a627378:::obj100:head) 2026-02-11T23:31:27.557+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.557+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:2a627378:::obj100:head) v3 finish latency 0.000014 2026-02-11T23:31:27.557+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2a627378:::obj100:head local-lis/les=113/114 n=114 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.557+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.557+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.557+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2a627378:::obj100:head local-lis/les=113/114 n=114 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.557+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.557+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.557+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.557+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5df8180 con 0x4acb000 2026-02-11T23:31:27.557+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:2acce685:::obj583:head)]) -- 0x5d8fe00 con 0x4acb000 2026-02-11T23:31:27.557+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.557+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 265 ==== MOSDPGPush(1.0 116/113 [PushOp(1:2b184b8d:::obj560:head, version: 111'560, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2b184b8d:::obj560:head@111'560, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db1520 con 0x4acb000 2026-02-11T23:31:27.557+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:2b184b8d:::obj560:head, version: 111'560, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2b184b8d:::obj560:head@111'560, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.557+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:2b184b8d:::obj560:head, version: 111'560, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2b184b8d:::obj560:head@111'560, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.557+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.557+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 266 ==== pg_backfill(progress 1.0 e 116/116 lb 1:2acce685:::obj583:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc2580 con 0x4acb000 2026-02-11T23:31:27.557+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:2b184b8d:::obj560:head, version: 111'560, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2b184b8d:::obj560:head@111'560, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.557+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:2acce685:::obj583:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.557+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.557+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:2acce685:::obj583:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.557+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:2b184b8d:::obj560:head, version: 111'560, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2b184b8d:::obj560:head@111'560, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.557+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.557+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:2b184b8d:::obj560:head, version: 111'560, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2b184b8d:::obj560:head@111'560, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(1.0 116/113 [PushOp(1:2b184b8d:::obj560:head, version: 111'560, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2b184b8d:::obj560:head@111'560, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:2a627378:::obj100:head local-lis/les=113/114 n=114 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.557+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.557+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2a627378:::obj100:head local-lis/les=113/114 n=114 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:2b184b8d:::obj560:head, version: 111'560, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2b184b8d:::obj560:head@111'560, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.557+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2a627378:::obj100:head local-lis/les=113/114 n=114 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:2b184b8d:::obj560:head@111'560, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.557+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.557+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:2acce685:::obj583:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.557+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2a627378:::obj100:head local-lis/les=113/114 n=114 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:2b184b8d:::obj560:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.557+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2a627378:::obj100:head local-lis/les=113/114 n=114 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:2b184b8d:::obj560:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.557+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2a627378:::obj100:head local-lis/les=113/114 n=114 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:2b184b8d:::obj560:head 2026-02-11T23:31:27.557+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2a627378:::obj100:head local-lis/les=113/114 n=114 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:2b184b8d:::obj560:head v 111'560 2026-02-11T23:31:27.557+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2a627378:::obj100:head local-lis/les=113/114 n=114 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.557+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.557+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:2b184b8d:::obj560:head, version: 111'560, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2b184b8d:::obj560:head@111'560, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:27.557+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.557+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.557+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:2acce685:::obj583:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.557+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:2acce685:::obj583:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 1.0 e 116/116 lb 1:2acce685:::obj583:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:2a627378:::obj100:head local-lis/les=113/114 n=114 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.557+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.557+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2a627378:::obj100:head local-lis/les=113/114 n=114 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:2acce685:::obj583:head) v3 2026-02-11T23:31:27.557+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2a627378:::obj100:head local-lis/les=113/114 n=114 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:2acce685:::obj583:head) 2026-02-11T23:31:27.557+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.557+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:2acce685:::obj583:head) v3 finish latency 0.000015 2026-02-11T23:31:27.557+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2acce685:::obj583:head local-lis/les=113/114 n=115 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.557+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.557+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.557+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2acce685:::obj583:head local-lis/les=113/114 n=115 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.557+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.557+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.557+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.557+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e3a480 con 0x4acb000 2026-02-11T23:31:27.557+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:2b184b8d:::obj560:head)]) -- 0x5e7e000 con 0x4acb000 2026-02-11T23:31:27.557+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.558+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 267 ==== MOSDPGPush(1.0 116/113 [PushOp(1:2c8d8a76:::obj467:head, version: 111'467, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2c8d8a76:::obj467:head@111'467, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db1d40 con 0x4acb000 2026-02-11T23:31:27.558+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:2c8d8a76:::obj467:head, version: 111'467, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2c8d8a76:::obj467:head@111'467, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.558+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:2c8d8a76:::obj467:head, version: 111'467, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2c8d8a76:::obj467:head@111'467, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.558+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.558+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:2c8d8a76:::obj467:head, version: 111'467, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2c8d8a76:::obj467:head@111'467, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.558+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 268 ==== pg_backfill(progress 1.0 e 116/116 lb 1:2b184b8d:::obj560:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc3080 con 0x4acb000 2026-02-11T23:31:27.558+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:2b184b8d:::obj560:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.558+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.558+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:2b184b8d:::obj560:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.558+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:2c8d8a76:::obj467:head, version: 111'467, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2c8d8a76:::obj467:head@111'467, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.558+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.558+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:2c8d8a76:::obj467:head, version: 111'467, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2c8d8a76:::obj467:head@111'467, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 116/113 [PushOp(1:2c8d8a76:::obj467:head, version: 111'467, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2c8d8a76:::obj467:head@111'467, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:2acce685:::obj583:head local-lis/les=113/114 n=115 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.558+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:2b184b8d:::obj560:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.558+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.558+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2acce685:::obj583:head local-lis/les=113/114 n=115 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:2c8d8a76:::obj467:head, version: 111'467, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2c8d8a76:::obj467:head@111'467, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.558+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2acce685:::obj583:head local-lis/les=113/114 n=115 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:2c8d8a76:::obj467:head@111'467, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.558+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2acce685:::obj583:head local-lis/les=113/114 n=115 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:2c8d8a76:::obj467:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.558+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2acce685:::obj583:head local-lis/les=113/114 n=115 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:2c8d8a76:::obj467:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.558+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2acce685:::obj583:head local-lis/les=113/114 n=115 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:2c8d8a76:::obj467:head 2026-02-11T23:31:27.558+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2acce685:::obj583:head local-lis/les=113/114 n=115 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:2c8d8a76:::obj467:head v 111'467 2026-02-11T23:31:27.558+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2acce685:::obj583:head local-lis/les=113/114 n=115 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.558+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.558+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:2c8d8a76:::obj467:head, version: 111'467, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2c8d8a76:::obj467:head@111'467, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:27.558+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2acce685:::obj583:head local-lis/les=113/114 n=115 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.558+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.558+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:2b184b8d:::obj560:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.558+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:2b184b8d:::obj560:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 1.0 e 116/116 lb 1:2b184b8d:::obj560:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:2acce685:::obj583:head local-lis/les=113/114 n=115 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.558+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.558+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.558+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2acce685:::obj583:head local-lis/les=113/114 n=115 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:2b184b8d:::obj560:head) v3 2026-02-11T23:31:27.558+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2acce685:::obj583:head local-lis/les=113/114 n=115 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:2b184b8d:::obj560:head) 2026-02-11T23:31:27.558+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.558+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:2b184b8d:::obj560:head) v3 finish latency 0.000016 2026-02-11T23:31:27.558+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.558+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.558+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2b184b8d:::obj560:head local-lis/les=113/114 n=116 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.558+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.558+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.558+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.558+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e6b980 con 0x4acb000 2026-02-11T23:31:27.558+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:2c8d8a76:::obj467:head)]) -- 0x5e3b680 con 0x4acb000 2026-02-11T23:31:27.558+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.558+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 269 ==== MOSDPGPush(1.0 116/113 [PushOp(1:2caf4871:::obj405:head, version: 111'405, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2caf4871:::obj405:head@111'405, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5d7e4e0 con 0x4acb000 2026-02-11T23:31:27.558+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:2caf4871:::obj405:head, version: 111'405, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2caf4871:::obj405:head@111'405, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.558+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:2caf4871:::obj405:head, version: 111'405, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2caf4871:::obj405:head@111'405, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.558+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.558+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:2caf4871:::obj405:head, version: 111'405, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2caf4871:::obj405:head@111'405, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.558+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 270 ==== pg_backfill(progress 1.0 e 116/116 lb 1:2c8d8a76:::obj467:head) ==== 1062+0+0 (unknown 0 0 0) 0x5e16000 con 0x4acb000 2026-02-11T23:31:27.558+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:2c8d8a76:::obj467:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 116 2026-02-11T23:31:27.558+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.558+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:2caf4871:::obj405:head, version: 111'405, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2caf4871:::obj405:head@111'405, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.558+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:2c8d8a76:::obj467:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.558+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.558+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:2caf4871:::obj405:head, version: 111'405, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2caf4871:::obj405:head@111'405, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 116/113 [PushOp(1:2caf4871:::obj405:head, version: 111'405, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2caf4871:::obj405:head@111'405, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:2b184b8d:::obj560:head local-lis/les=113/114 n=116 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.558+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:2c8d8a76:::obj467:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.558+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.558+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2b184b8d:::obj560:head local-lis/les=113/114 n=116 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:2caf4871:::obj405:head, version: 111'405, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2caf4871:::obj405:head@111'405, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.558+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2b184b8d:::obj560:head local-lis/les=113/114 n=116 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:2caf4871:::obj405:head@111'405, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.558+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2b184b8d:::obj560:head local-lis/les=113/114 n=116 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:2caf4871:::obj405:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.558+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2b184b8d:::obj560:head local-lis/les=113/114 n=116 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:2caf4871:::obj405:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.558+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2b184b8d:::obj560:head local-lis/les=113/114 n=116 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:2caf4871:::obj405:head 2026-02-11T23:31:27.558+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2b184b8d:::obj560:head local-lis/les=113/114 n=116 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:2caf4871:::obj405:head v 111'405 2026-02-11T23:31:27.558+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2b184b8d:::obj560:head local-lis/les=113/114 n=116 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.558+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.559+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:2caf4871:::obj405:head, version: 111'405, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2caf4871:::obj405:head@111'405, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:27.559+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.559+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.559+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:2c8d8a76:::obj467:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.559+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:2c8d8a76:::obj467:head) v3 prio 15 cost 0 latency 0.000080 pg_backfill(progress 1.0 e 116/116 lb 1:2c8d8a76:::obj467:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:2b184b8d:::obj560:head local-lis/les=113/114 n=116 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.559+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.559+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2b184b8d:::obj560:head local-lis/les=113/114 n=116 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:2c8d8a76:::obj467:head) v3 2026-02-11T23:31:27.559+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2b184b8d:::obj560:head local-lis/les=113/114 n=116 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:2c8d8a76:::obj467:head) 2026-02-11T23:31:27.559+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.559+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:2c8d8a76:::obj467:head) v3 finish latency 0.000017 2026-02-11T23:31:27.559+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2c8d8a76:::obj467:head local-lis/les=113/114 n=117 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.559+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.559+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.559+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2c8d8a76:::obj467:head local-lis/les=113/114 n=117 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.559+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.559+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.559+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.559+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e6a600 con 0x4acb000 2026-02-11T23:31:27.559+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:2caf4871:::obj405:head)]) -- 0x5df8600 con 0x4acb000 2026-02-11T23:31:27.559+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.559+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 271 ==== MOSDPGPush(1.0 116/113 [PushOp(1:2d59679a:::obj179:head, version: 111'179, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2d59679a:::obj179:head@111'179, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5d7e9c0 con 0x4acb000 2026-02-11T23:31:27.559+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:2d59679a:::obj179:head, version: 111'179, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2d59679a:::obj179:head@111'179, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.559+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:2d59679a:::obj179:head, version: 111'179, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2d59679a:::obj179:head@111'179, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.559+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.559+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:2d59679a:::obj179:head, version: 111'179, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2d59679a:::obj179:head@111'179, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.559+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 272 ==== pg_backfill(progress 1.0 e 116/116 lb 1:2caf4871:::obj405:head) ==== 1062+0+0 (unknown 0 0 0) 0x5e16580 con 0x4acb000 2026-02-11T23:31:27.559+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.559+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:2caf4871:::obj405:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 116 2026-02-11T23:31:27.559+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:2d59679a:::obj179:head, version: 111'179, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2d59679a:::obj179:head@111'179, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.559+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:2caf4871:::obj405:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.559+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.559+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:2caf4871:::obj405:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.559+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:2d59679a:::obj179:head, version: 111'179, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2d59679a:::obj179:head@111'179, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 116/113 [PushOp(1:2d59679a:::obj179:head, version: 111'179, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2d59679a:::obj179:head@111'179, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:2c8d8a76:::obj467:head local-lis/les=113/114 n=117 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.559+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.559+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2c8d8a76:::obj467:head local-lis/les=113/114 n=117 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:2d59679a:::obj179:head, version: 111'179, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2d59679a:::obj179:head@111'179, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.559+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2c8d8a76:::obj467:head local-lis/les=113/114 n=117 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:2d59679a:::obj179:head@111'179, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.559+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2c8d8a76:::obj467:head local-lis/les=113/114 n=117 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:2d59679a:::obj179:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.559+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2c8d8a76:::obj467:head local-lis/les=113/114 n=117 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:2d59679a:::obj179:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.559+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2c8d8a76:::obj467:head local-lis/les=113/114 n=117 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:2d59679a:::obj179:head 2026-02-11T23:31:27.559+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2c8d8a76:::obj467:head local-lis/les=113/114 n=117 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:2d59679a:::obj179:head v 111'179 2026-02-11T23:31:27.559+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2c8d8a76:::obj467:head local-lis/les=113/114 n=117 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.559+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.559+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:2d59679a:::obj179:head, version: 111'179, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2d59679a:::obj179:head@111'179, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:27.559+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.559+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.559+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:2caf4871:::obj405:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.559+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:2caf4871:::obj405:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 1.0 e 116/116 lb 1:2caf4871:::obj405:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:2c8d8a76:::obj467:head local-lis/les=113/114 n=117 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.559+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.559+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2c8d8a76:::obj467:head local-lis/les=113/114 n=117 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:2caf4871:::obj405:head) v3 2026-02-11T23:31:27.559+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2c8d8a76:::obj467:head local-lis/les=113/114 n=117 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:2caf4871:::obj405:head) 2026-02-11T23:31:27.559+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.559+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:2caf4871:::obj405:head) v3 finish latency 0.000016 2026-02-11T23:31:27.559+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2caf4871:::obj405:head local-lis/les=113/114 n=118 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.559+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.559+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.559+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2caf4871:::obj405:head local-lis/les=113/114 n=118 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.559+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.559+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.559+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.559+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d78d80 con 0x4acb000 2026-02-11T23:31:27.559+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:2d59679a:::obj179:head)]) -- 0x5df8780 con 0x4acb000 2026-02-11T23:31:27.559+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.560+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 273 ==== MOSDPGPush(1.0 116/113 [PushOp(1:2df8f997:::obj267:head, version: 111'267, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2df8f997:::obj267:head@111'267, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5d7eea0 con 0x4acb000 2026-02-11T23:31:27.560+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:2df8f997:::obj267:head, version: 111'267, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2df8f997:::obj267:head@111'267, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.560+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:2df8f997:::obj267:head, version: 111'267, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2df8f997:::obj267:head@111'267, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.560+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.560+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:2df8f997:::obj267:head, version: 111'267, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2df8f997:::obj267:head@111'267, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.560+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 274 ==== pg_backfill(progress 1.0 e 116/116 lb 1:2d59679a:::obj179:head) ==== 1062+0+0 (unknown 0 0 0) 0x5e16b00 con 0x4acb000 2026-02-11T23:31:27.560+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.560+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:2d59679a:::obj179:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 116 2026-02-11T23:31:27.560+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:2df8f997:::obj267:head, version: 111'267, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2df8f997:::obj267:head@111'267, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.560+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:2d59679a:::obj179:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.560+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.560+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:2d59679a:::obj179:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.560+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:2df8f997:::obj267:head, version: 111'267, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2df8f997:::obj267:head@111'267, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(1.0 116/113 [PushOp(1:2df8f997:::obj267:head, version: 111'267, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2df8f997:::obj267:head@111'267, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:2caf4871:::obj405:head local-lis/les=113/114 n=118 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.560+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.560+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2caf4871:::obj405:head local-lis/les=113/114 n=118 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:2df8f997:::obj267:head, version: 111'267, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2df8f997:::obj267:head@111'267, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.560+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2caf4871:::obj405:head local-lis/les=113/114 n=118 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:2df8f997:::obj267:head@111'267, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.560+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2caf4871:::obj405:head local-lis/les=113/114 n=118 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:2df8f997:::obj267:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.560+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2caf4871:::obj405:head local-lis/les=113/114 n=118 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:2df8f997:::obj267:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.560+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2caf4871:::obj405:head local-lis/les=113/114 n=118 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:2df8f997:::obj267:head 2026-02-11T23:31:27.560+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2caf4871:::obj405:head local-lis/les=113/114 n=118 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:2df8f997:::obj267:head v 111'267 2026-02-11T23:31:27.560+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2caf4871:::obj405:head local-lis/les=113/114 n=118 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.560+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.560+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:2df8f997:::obj267:head, version: 111'267, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2df8f997:::obj267:head@111'267, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000070 2026-02-11T23:31:27.560+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2caf4871:::obj405:head local-lis/les=113/114 n=118 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.560+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.560+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.560+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:2d59679a:::obj179:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.560+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:2d59679a:::obj179:head) v3 prio 15 cost 0 latency 0.000093 pg_backfill(progress 1.0 e 116/116 lb 1:2d59679a:::obj179:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:2caf4871:::obj405:head local-lis/les=113/114 n=118 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.560+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.560+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2caf4871:::obj405:head local-lis/les=113/114 n=118 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:2d59679a:::obj179:head) v3 2026-02-11T23:31:27.560+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2caf4871:::obj405:head local-lis/les=113/114 n=118 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:2d59679a:::obj179:head) 2026-02-11T23:31:27.560+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.560+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:2d59679a:::obj179:head) v3 finish latency 0.000015 2026-02-11T23:31:27.560+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.560+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.560+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2d59679a:::obj179:head local-lis/les=113/114 n=119 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.560+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.560+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.560+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.560+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d79380 con 0x4acb000 2026-02-11T23:31:27.560+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:2df8f997:::obj267:head)]) -- 0x5d78c00 con 0x4acb000 2026-02-11T23:31:27.560+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.560+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 275 ==== MOSDPGPush(1.0 116/113 [PushOp(1:2e5f3a17:::obj520:head, version: 111'520, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e5f3a17:::obj520:head@111'520, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5d7f380 con 0x4acb000 2026-02-11T23:31:27.560+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:2e5f3a17:::obj520:head, version: 111'520, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e5f3a17:::obj520:head@111'520, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.560+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:2e5f3a17:::obj520:head, version: 111'520, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e5f3a17:::obj520:head@111'520, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.560+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.560+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:2e5f3a17:::obj520:head, version: 111'520, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e5f3a17:::obj520:head@111'520, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.560+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 276 ==== pg_backfill(progress 1.0 e 116/116 lb 1:2df8f997:::obj267:head) ==== 1062+0+0 (unknown 0 0 0) 0x5e17080 con 0x4acb000 2026-02-11T23:31:27.560+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.560+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:2df8f997:::obj267:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 116 2026-02-11T23:31:27.560+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:2e5f3a17:::obj520:head, version: 111'520, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e5f3a17:::obj520:head@111'520, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.560+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:2df8f997:::obj267:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.560+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.560+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:2e5f3a17:::obj520:head, version: 111'520, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e5f3a17:::obj520:head@111'520, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000043 MOSDPGPush(1.0 116/113 [PushOp(1:2e5f3a17:::obj520:head, version: 111'520, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e5f3a17:::obj520:head@111'520, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:2d59679a:::obj179:head local-lis/les=113/114 n=119 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.560+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:2df8f997:::obj267:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.560+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.560+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2d59679a:::obj179:head local-lis/les=113/114 n=119 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:2e5f3a17:::obj520:head, version: 111'520, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e5f3a17:::obj520:head@111'520, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.560+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2d59679a:::obj179:head local-lis/les=113/114 n=119 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:2e5f3a17:::obj520:head@111'520, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.560+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2d59679a:::obj179:head local-lis/les=113/114 n=119 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:2e5f3a17:::obj520:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.560+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2d59679a:::obj179:head local-lis/les=113/114 n=119 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:2e5f3a17:::obj520:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.560+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2d59679a:::obj179:head local-lis/les=113/114 n=119 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:2e5f3a17:::obj520:head 2026-02-11T23:31:27.560+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2d59679a:::obj179:head local-lis/les=113/114 n=119 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:2e5f3a17:::obj520:head v 111'520 2026-02-11T23:31:27.560+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2d59679a:::obj179:head local-lis/les=113/114 n=119 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.560+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.560+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:2e5f3a17:::obj520:head, version: 111'520, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e5f3a17:::obj520:head@111'520, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:27.560+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.560+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.560+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:2df8f997:::obj267:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.560+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:2df8f997:::obj267:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 1.0 e 116/116 lb 1:2df8f997:::obj267:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:2d59679a:::obj179:head local-lis/les=113/114 n=119 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.560+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.560+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2d59679a:::obj179:head local-lis/les=113/114 n=119 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:2df8f997:::obj267:head) v3 2026-02-11T23:31:27.560+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2d59679a:::obj179:head local-lis/les=113/114 n=119 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:2df8f997:::obj267:head) 2026-02-11T23:31:27.560+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.560+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:2df8f997:::obj267:head) v3 finish latency 0.000017 2026-02-11T23:31:27.560+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2df8f997:::obj267:head local-lis/les=113/114 n=120 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.560+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.560+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.560+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2df8f997:::obj267:head local-lis/les=113/114 n=120 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.560+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.560+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.560+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.560+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d78480 con 0x4acb000 2026-02-11T23:31:27.560+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:2e5f3a17:::obj520:head)]) -- 0x5df8a80 con 0x4acb000 2026-02-11T23:31:27.560+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.561+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 277 ==== MOSDPGPush(1.0 116/113 [PushOp(1:2e69a8fc:::obj598:head, version: 111'598, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e69a8fc:::obj598:head@111'598, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5d7f860 con 0x4acb000 2026-02-11T23:31:27.561+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:2e69a8fc:::obj598:head, version: 111'598, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e69a8fc:::obj598:head@111'598, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.561+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:2e69a8fc:::obj598:head, version: 111'598, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e69a8fc:::obj598:head@111'598, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.561+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.561+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:2e69a8fc:::obj598:head, version: 111'598, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e69a8fc:::obj598:head@111'598, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.561+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 278 ==== pg_backfill(progress 1.0 e 116/116 lb 1:2e5f3a17:::obj520:head) ==== 1062+0+0 (unknown 0 0 0) 0x5e17600 con 0x4acb000 2026-02-11T23:31:27.561+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.561+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:2e5f3a17:::obj520:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 116 2026-02-11T23:31:27.561+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:2e69a8fc:::obj598:head, version: 111'598, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e69a8fc:::obj598:head@111'598, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.561+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:2e5f3a17:::obj520:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.561+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.561+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:2e69a8fc:::obj598:head, version: 111'598, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e69a8fc:::obj598:head@111'598, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 116/113 [PushOp(1:2e69a8fc:::obj598:head, version: 111'598, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e69a8fc:::obj598:head@111'598, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:2df8f997:::obj267:head local-lis/les=113/114 n=120 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.561+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:2e5f3a17:::obj520:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.561+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.561+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2df8f997:::obj267:head local-lis/les=113/114 n=120 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:2e69a8fc:::obj598:head, version: 111'598, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e69a8fc:::obj598:head@111'598, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.561+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2df8f997:::obj267:head local-lis/les=113/114 n=120 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:2e69a8fc:::obj598:head@111'598, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.561+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2df8f997:::obj267:head local-lis/les=113/114 n=120 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:2e69a8fc:::obj598:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.561+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2df8f997:::obj267:head local-lis/les=113/114 n=120 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:2e69a8fc:::obj598:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.561+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2df8f997:::obj267:head local-lis/les=113/114 n=120 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:2e69a8fc:::obj598:head 2026-02-11T23:31:27.561+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2df8f997:::obj267:head local-lis/les=113/114 n=120 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:2e69a8fc:::obj598:head v 111'598 2026-02-11T23:31:27.561+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2df8f997:::obj267:head local-lis/les=113/114 n=120 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.561+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.561+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:2e69a8fc:::obj598:head, version: 111'598, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e69a8fc:::obj598:head@111'598, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:27.561+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2df8f997:::obj267:head local-lis/les=113/114 n=120 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.561+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.561+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:2e5f3a17:::obj520:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.561+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:2e5f3a17:::obj520:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 1.0 e 116/116 lb 1:2e5f3a17:::obj520:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:2df8f997:::obj267:head local-lis/les=113/114 n=120 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.561+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.561+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.561+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2df8f997:::obj267:head local-lis/les=113/114 n=120 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:2e5f3a17:::obj520:head) v3 2026-02-11T23:31:27.561+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2df8f997:::obj267:head local-lis/les=113/114 n=120 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:2e5f3a17:::obj520:head) 2026-02-11T23:31:27.561+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.561+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:2e5f3a17:::obj520:head) v3 finish latency 0.000016 2026-02-11T23:31:27.561+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.561+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.561+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2e5f3a17:::obj520:head local-lis/les=113/114 n=121 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.561+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.561+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.561+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.561+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d79980 con 0x4acb000 2026-02-11T23:31:27.561+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:2e69a8fc:::obj598:head)]) -- 0x5d78000 con 0x4acb000 2026-02-11T23:31:27.561+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.562+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 279 ==== MOSDPGPush(1.0 116/113 [PushOp(1:2e72dfbc:::obj232:head, version: 111'232, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e72dfbc:::obj232:head@111'232, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5d7fa00 con 0x4acb000 2026-02-11T23:31:27.562+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:2e72dfbc:::obj232:head, version: 111'232, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e72dfbc:::obj232:head@111'232, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 116 2026-02-11T23:31:27.562+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:2e72dfbc:::obj232:head, version: 111'232, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e72dfbc:::obj232:head@111'232, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.562+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.562+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:2e72dfbc:::obj232:head, version: 111'232, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e72dfbc:::obj232:head@111'232, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.562+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 280 ==== pg_backfill(progress 1.0 e 116/116 lb 1:2e69a8fc:::obj598:head) ==== 1062+0+0 (unknown 0 0 0) 0x5e17b80 con 0x4acb000 2026-02-11T23:31:27.562+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.562+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:2e69a8fc:::obj598:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 116 2026-02-11T23:31:27.562+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:2e72dfbc:::obj232:head, version: 111'232, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e72dfbc:::obj232:head@111'232, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.562+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:2e69a8fc:::obj598:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.562+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.562+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:2e72dfbc:::obj232:head, version: 111'232, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e72dfbc:::obj232:head@111'232, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(1.0 116/113 [PushOp(1:2e72dfbc:::obj232:head, version: 111'232, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e72dfbc:::obj232:head@111'232, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:2e5f3a17:::obj520:head local-lis/les=113/114 n=121 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.562+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:2e69a8fc:::obj598:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.562+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.562+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2e5f3a17:::obj520:head local-lis/les=113/114 n=121 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:2e72dfbc:::obj232:head, version: 111'232, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e72dfbc:::obj232:head@111'232, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.562+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2e5f3a17:::obj520:head local-lis/les=113/114 n=121 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:2e72dfbc:::obj232:head@111'232, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.562+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2e5f3a17:::obj520:head local-lis/les=113/114 n=121 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:2e72dfbc:::obj232:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.562+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2e5f3a17:::obj520:head local-lis/les=113/114 n=121 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:2e72dfbc:::obj232:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.562+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2e5f3a17:::obj520:head local-lis/les=113/114 n=121 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:2e72dfbc:::obj232:head 2026-02-11T23:31:27.562+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2e5f3a17:::obj520:head local-lis/les=113/114 n=121 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:2e72dfbc:::obj232:head v 111'232 2026-02-11T23:31:27.562+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2e5f3a17:::obj520:head local-lis/les=113/114 n=121 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.562+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.562+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:2e72dfbc:::obj232:head, version: 111'232, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e72dfbc:::obj232:head@111'232, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:27.562+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.562+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.562+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:2e69a8fc:::obj598:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.562+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:2e69a8fc:::obj598:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 1.0 e 116/116 lb 1:2e69a8fc:::obj598:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:2e5f3a17:::obj520:head local-lis/les=113/114 n=121 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.562+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.562+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2e5f3a17:::obj520:head local-lis/les=113/114 n=121 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:2e69a8fc:::obj598:head) v3 2026-02-11T23:31:27.562+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2e5f3a17:::obj520:head local-lis/les=113/114 n=121 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:2e69a8fc:::obj598:head) 2026-02-11T23:31:27.562+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.562+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:2e69a8fc:::obj598:head) v3 finish latency 0.000017 2026-02-11T23:31:27.562+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2e69a8fc:::obj598:head local-lis/les=113/114 n=122 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.562+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.562+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.562+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2e69a8fc:::obj598:head local-lis/les=113/114 n=122 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.562+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.562+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.562+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.562+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d78780 con 0x4acb000 2026-02-11T23:31:27.562+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:2e72dfbc:::obj232:head)]) -- 0x5df9080 con 0x4acb000 2026-02-11T23:31:27.562+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.562+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 281 ==== MOSDPGPush(1.0 116/113 [PushOp(1:2eeda62f:::obj154:head, version: 111'154, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2eeda62f:::obj154:head@111'154, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5d7e000 con 0x4acb000 2026-02-11T23:31:27.562+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:2eeda62f:::obj154:head, version: 111'154, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2eeda62f:::obj154:head@111'154, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.562+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:2eeda62f:::obj154:head, version: 111'154, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2eeda62f:::obj154:head@111'154, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.562+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.562+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 282 ==== pg_backfill(progress 1.0 e 116/116 lb 1:2e72dfbc:::obj232:head) ==== 1062+0+0 (unknown 0 0 0) 0x5e18100 con 0x4acb000 2026-02-11T23:31:27.562+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:2eeda62f:::obj154:head, version: 111'154, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2eeda62f:::obj154:head@111'154, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.562+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:2e72dfbc:::obj232:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.562+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.562+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:2e72dfbc:::obj232:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.562+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:2eeda62f:::obj154:head, version: 111'154, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2eeda62f:::obj154:head@111'154, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.562+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.562+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:2eeda62f:::obj154:head, version: 111'154, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2eeda62f:::obj154:head@111'154, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 116/113 [PushOp(1:2eeda62f:::obj154:head, version: 111'154, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2eeda62f:::obj154:head@111'154, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:2e69a8fc:::obj598:head local-lis/les=113/114 n=122 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.562+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.562+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2e69a8fc:::obj598:head local-lis/les=113/114 n=122 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:2eeda62f:::obj154:head, version: 111'154, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2eeda62f:::obj154:head@111'154, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.562+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.562+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:2e72dfbc:::obj232:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.562+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2e69a8fc:::obj598:head local-lis/les=113/114 n=122 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:2eeda62f:::obj154:head@111'154, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.562+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2e69a8fc:::obj598:head local-lis/les=113/114 n=122 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:2eeda62f:::obj154:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.562+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2e69a8fc:::obj598:head local-lis/les=113/114 n=122 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:2eeda62f:::obj154:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.562+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2e69a8fc:::obj598:head local-lis/les=113/114 n=122 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:2eeda62f:::obj154:head 2026-02-11T23:31:27.562+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2e69a8fc:::obj598:head local-lis/les=113/114 n=122 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:2eeda62f:::obj154:head v 111'154 2026-02-11T23:31:27.562+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2e69a8fc:::obj598:head local-lis/les=113/114 n=122 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.562+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.562+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:2eeda62f:::obj154:head, version: 111'154, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2eeda62f:::obj154:head@111'154, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000067 2026-02-11T23:31:27.562+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.562+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.562+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:2e72dfbc:::obj232:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.562+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:2e72dfbc:::obj232:head) v3 prio 15 cost 0 latency 0.000088 pg_backfill(progress 1.0 e 116/116 lb 1:2e72dfbc:::obj232:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:2e69a8fc:::obj598:head local-lis/les=113/114 n=122 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.562+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.562+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2e69a8fc:::obj598:head local-lis/les=113/114 n=122 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:2e72dfbc:::obj232:head) v3 2026-02-11T23:31:27.562+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2e69a8fc:::obj598:head local-lis/les=113/114 n=122 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:2e72dfbc:::obj232:head) 2026-02-11T23:31:27.562+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.562+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:2e72dfbc:::obj232:head) v3 finish latency 0.000014 2026-02-11T23:31:27.562+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2e72dfbc:::obj232:head local-lis/les=113/114 n=123 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.562+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.562+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.562+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2e72dfbc:::obj232:head local-lis/les=113/114 n=123 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.562+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.562+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.562+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.562+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d79e00 con 0x4acb000 2026-02-11T23:31:27.562+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:2eeda62f:::obj154:head)]) -- 0x5df9380 con 0x4acb000 2026-02-11T23:31:27.562+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.563+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 283 ==== MOSDPGPush(1.0 116/113 [PushOp(1:2f478301:::obj479:head, version: 111'479, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2f478301:::obj479:head@111'479, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db09c0 con 0x4acb000 2026-02-11T23:31:27.563+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:2f478301:::obj479:head, version: 111'479, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2f478301:::obj479:head@111'479, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.563+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:2f478301:::obj479:head, version: 111'479, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2f478301:::obj479:head@111'479, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.563+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.563+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:2f478301:::obj479:head, version: 111'479, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2f478301:::obj479:head@111'479, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.563+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 284 ==== pg_backfill(progress 1.0 e 116/116 lb 1:2eeda62f:::obj154:head) ==== 1062+0+0 (unknown 0 0 0) 0x5e18680 con 0x4acb000 2026-02-11T23:31:27.563+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.563+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:2f478301:::obj479:head, version: 111'479, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2f478301:::obj479:head@111'479, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.563+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:2eeda62f:::obj154:head) v3 prio 15 type 95 cost 0 latency 0.000011 epoch 116 2026-02-11T23:31:27.563+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:2eeda62f:::obj154:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.563+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.563+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:2eeda62f:::obj154:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.563+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:2f478301:::obj479:head, version: 111'479, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2f478301:::obj479:head@111'479, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 116/113 [PushOp(1:2f478301:::obj479:head, version: 111'479, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2f478301:::obj479:head@111'479, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:2e72dfbc:::obj232:head local-lis/les=113/114 n=123 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.563+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.563+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2e72dfbc:::obj232:head local-lis/les=113/114 n=123 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:2f478301:::obj479:head, version: 111'479, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2f478301:::obj479:head@111'479, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.563+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2e72dfbc:::obj232:head local-lis/les=113/114 n=123 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:2f478301:::obj479:head@111'479, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.563+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2e72dfbc:::obj232:head local-lis/les=113/114 n=123 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:2f478301:::obj479:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.563+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2e72dfbc:::obj232:head local-lis/les=113/114 n=123 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:2f478301:::obj479:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.563+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2e72dfbc:::obj232:head local-lis/les=113/114 n=123 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:2f478301:::obj479:head 2026-02-11T23:31:27.563+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2e72dfbc:::obj232:head local-lis/les=113/114 n=123 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:2f478301:::obj479:head v 111'479 2026-02-11T23:31:27.563+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2e72dfbc:::obj232:head local-lis/les=113/114 n=123 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.563+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.563+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:2f478301:::obj479:head, version: 111'479, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2f478301:::obj479:head@111'479, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:27.563+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.563+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.563+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:2eeda62f:::obj154:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.563+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:2eeda62f:::obj154:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 1.0 e 116/116 lb 1:2eeda62f:::obj154:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:2e72dfbc:::obj232:head local-lis/les=113/114 n=123 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.563+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.563+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2e72dfbc:::obj232:head local-lis/les=113/114 n=123 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:2eeda62f:::obj154:head) v3 2026-02-11T23:31:27.563+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2e72dfbc:::obj232:head local-lis/les=113/114 n=123 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:2eeda62f:::obj154:head) 2026-02-11T23:31:27.563+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.563+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:2eeda62f:::obj154:head) v3 finish latency 0.000013 2026-02-11T23:31:27.563+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2eeda62f:::obj154:head local-lis/les=113/114 n=124 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.563+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.563+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.563+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2eeda62f:::obj154:head local-lis/les=113/114 n=124 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.563+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.563+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.563+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.563+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x4ec2780 con 0x4acb000 2026-02-11T23:31:27.563+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:2f478301:::obj479:head)]) -- 0x5df9500 con 0x4acb000 2026-02-11T23:31:27.563+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.563+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 285 ==== MOSDPGPush(1.0 116/113 [PushOp(1:303427ce:::obj485:head, version: 111'485, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:303427ce:::obj485:head@111'485, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db1ba0 con 0x4acb000 2026-02-11T23:31:27.563+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:303427ce:::obj485:head, version: 111'485, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:303427ce:::obj485:head@111'485, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.563+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:303427ce:::obj485:head, version: 111'485, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:303427ce:::obj485:head@111'485, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.563+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.563+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 286 ==== pg_backfill(progress 1.0 e 116/116 lb 1:2f478301:::obj479:head) ==== 1062+0+0 (unknown 0 0 0) 0x5e18c00 con 0x4acb000 2026-02-11T23:31:27.563+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:303427ce:::obj485:head, version: 111'485, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:303427ce:::obj485:head@111'485, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.563+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:2f478301:::obj479:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.563+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.563+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:303427ce:::obj485:head, version: 111'485, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:303427ce:::obj485:head@111'485, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.563+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:2f478301:::obj479:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.563+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.563+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:2f478301:::obj479:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.563+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:303427ce:::obj485:head, version: 111'485, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:303427ce:::obj485:head@111'485, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 116/113 [PushOp(1:303427ce:::obj485:head, version: 111'485, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:303427ce:::obj485:head@111'485, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:2eeda62f:::obj154:head local-lis/les=113/114 n=124 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.563+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.563+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2eeda62f:::obj154:head local-lis/les=113/114 n=124 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:303427ce:::obj485:head, version: 111'485, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:303427ce:::obj485:head@111'485, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.563+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2eeda62f:::obj154:head local-lis/les=113/114 n=124 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:303427ce:::obj485:head@111'485, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.563+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2eeda62f:::obj154:head local-lis/les=113/114 n=124 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:303427ce:::obj485:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.563+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2eeda62f:::obj154:head local-lis/les=113/114 n=124 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:303427ce:::obj485:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.563+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2eeda62f:::obj154:head local-lis/les=113/114 n=124 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:303427ce:::obj485:head 2026-02-11T23:31:27.563+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2eeda62f:::obj154:head local-lis/les=113/114 n=124 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:303427ce:::obj485:head v 111'485 2026-02-11T23:31:27.563+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2eeda62f:::obj154:head local-lis/les=113/114 n=124 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.563+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.563+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:303427ce:::obj485:head, version: 111'485, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:303427ce:::obj485:head@111'485, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:27.563+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.563+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.563+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:2f478301:::obj479:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.563+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:2f478301:::obj479:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 1.0 e 116/116 lb 1:2f478301:::obj479:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:2eeda62f:::obj154:head local-lis/les=113/114 n=124 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.563+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.563+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2eeda62f:::obj154:head local-lis/les=113/114 n=124 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:2f478301:::obj479:head) v3 2026-02-11T23:31:27.563+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2eeda62f:::obj154:head local-lis/les=113/114 n=124 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:2f478301:::obj479:head) 2026-02-11T23:31:27.563+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.563+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:2f478301:::obj479:head) v3 finish latency 0.000016 2026-02-11T23:31:27.563+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2f478301:::obj479:head local-lis/les=113/114 n=125 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.563+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.564+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.564+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2f478301:::obj479:head local-lis/les=113/114 n=125 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.564+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.564+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.564+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.564+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e7ed80 con 0x4acb000 2026-02-11T23:31:27.564+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:303427ce:::obj485:head)]) -- 0x5df9800 con 0x4acb000 2026-02-11T23:31:27.564+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.564+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 287 ==== MOSDPGPush(1.0 116/113 [PushOp(1:320b4068:::obj138:head, version: 111'138, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:320b4068:::obj138:head@111'138, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db1520 con 0x4acb000 2026-02-11T23:31:27.564+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:320b4068:::obj138:head, version: 111'138, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:320b4068:::obj138:head@111'138, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.564+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:320b4068:::obj138:head, version: 111'138, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:320b4068:::obj138:head@111'138, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.564+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.564+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:320b4068:::obj138:head, version: 111'138, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:320b4068:::obj138:head@111'138, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.564+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 288 ==== pg_backfill(progress 1.0 e 116/116 lb 1:303427ce:::obj485:head) ==== 1062+0+0 (unknown 0 0 0) 0x5e19180 con 0x4acb000 2026-02-11T23:31:27.564+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:303427ce:::obj485:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.564+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.564+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:320b4068:::obj138:head, version: 111'138, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:320b4068:::obj138:head@111'138, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.564+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:303427ce:::obj485:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.564+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.564+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:320b4068:::obj138:head, version: 111'138, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:320b4068:::obj138:head@111'138, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 116/113 [PushOp(1:320b4068:::obj138:head, version: 111'138, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:320b4068:::obj138:head@111'138, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:2f478301:::obj479:head local-lis/les=113/114 n=125 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.564+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:303427ce:::obj485:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.564+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.564+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2f478301:::obj479:head local-lis/les=113/114 n=125 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:320b4068:::obj138:head, version: 111'138, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:320b4068:::obj138:head@111'138, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.564+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2f478301:::obj479:head local-lis/les=113/114 n=125 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:320b4068:::obj138:head@111'138, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.564+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2f478301:::obj479:head local-lis/les=113/114 n=125 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:320b4068:::obj138:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.564+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2f478301:::obj479:head local-lis/les=113/114 n=125 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:320b4068:::obj138:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.564+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2f478301:::obj479:head local-lis/les=113/114 n=125 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:320b4068:::obj138:head 2026-02-11T23:31:27.564+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2f478301:::obj479:head local-lis/les=113/114 n=125 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:320b4068:::obj138:head v 111'138 2026-02-11T23:31:27.564+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2f478301:::obj479:head local-lis/les=113/114 n=125 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.564+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.564+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:320b4068:::obj138:head, version: 111'138, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:320b4068:::obj138:head@111'138, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:27.564+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2f478301:::obj479:head local-lis/les=113/114 n=125 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.564+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.564+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:303427ce:::obj485:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.564+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:303427ce:::obj485:head) v3 prio 15 cost 0 latency 0.000084 pg_backfill(progress 1.0 e 116/116 lb 1:303427ce:::obj485:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:2f478301:::obj479:head local-lis/les=113/114 n=125 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.564+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.564+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.564+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2f478301:::obj479:head local-lis/les=113/114 n=125 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:303427ce:::obj485:head) v3 2026-02-11T23:31:27.564+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:2f478301:::obj479:head local-lis/les=113/114 n=125 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:303427ce:::obj485:head) 2026-02-11T23:31:27.564+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.564+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:303427ce:::obj485:head) v3 finish latency 0.000018 2026-02-11T23:31:27.564+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.564+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.564+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:303427ce:::obj485:head local-lis/les=113/114 n=126 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.564+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.564+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.564+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.564+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d8e180 con 0x4acb000 2026-02-11T23:31:27.564+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:320b4068:::obj138:head)]) -- 0x5e7fe00 con 0x4acb000 2026-02-11T23:31:27.564+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.565+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 289 ==== MOSDPGPush(1.0 116/113 [PushOp(1:325eabfb:::obj310:head, version: 111'310, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:325eabfb:::obj310:head@111'310, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db1d40 con 0x4acb000 2026-02-11T23:31:27.565+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:325eabfb:::obj310:head, version: 111'310, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:325eabfb:::obj310:head@111'310, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.565+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:325eabfb:::obj310:head, version: 111'310, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:325eabfb:::obj310:head@111'310, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.565+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.565+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:325eabfb:::obj310:head, version: 111'310, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:325eabfb:::obj310:head@111'310, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.565+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 290 ==== pg_backfill(progress 1.0 e 116/116 lb 1:320b4068:::obj138:head) ==== 1062+0+0 (unknown 0 0 0) 0x5e19700 con 0x4acb000 2026-02-11T23:31:27.565+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:320b4068:::obj138:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 116 2026-02-11T23:31:27.565+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.565+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:325eabfb:::obj310:head, version: 111'310, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:325eabfb:::obj310:head@111'310, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.565+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:320b4068:::obj138:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.565+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.565+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:320b4068:::obj138:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.565+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:325eabfb:::obj310:head, version: 111'310, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:325eabfb:::obj310:head@111'310, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 116/113 [PushOp(1:325eabfb:::obj310:head, version: 111'310, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:325eabfb:::obj310:head@111'310, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:303427ce:::obj485:head local-lis/les=113/114 n=126 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.565+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.565+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:303427ce:::obj485:head local-lis/les=113/114 n=126 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:325eabfb:::obj310:head, version: 111'310, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:325eabfb:::obj310:head@111'310, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.565+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:303427ce:::obj485:head local-lis/les=113/114 n=126 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:325eabfb:::obj310:head@111'310, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.565+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:303427ce:::obj485:head local-lis/les=113/114 n=126 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:325eabfb:::obj310:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.565+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:303427ce:::obj485:head local-lis/les=113/114 n=126 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:325eabfb:::obj310:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.565+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:303427ce:::obj485:head local-lis/les=113/114 n=126 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:325eabfb:::obj310:head 2026-02-11T23:31:27.565+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:303427ce:::obj485:head local-lis/les=113/114 n=126 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:325eabfb:::obj310:head v 111'310 2026-02-11T23:31:27.565+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:303427ce:::obj485:head local-lis/les=113/114 n=126 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.565+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.565+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:325eabfb:::obj310:head, version: 111'310, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:325eabfb:::obj310:head@111'310, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:27.565+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:303427ce:::obj485:head local-lis/les=113/114 n=126 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.565+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.565+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.565+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:320b4068:::obj138:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.565+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:320b4068:::obj138:head) v3 prio 15 cost 0 latency 0.000079 pg_backfill(progress 1.0 e 116/116 lb 1:320b4068:::obj138:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:303427ce:::obj485:head local-lis/les=113/114 n=126 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.565+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.565+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:303427ce:::obj485:head local-lis/les=113/114 n=126 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:320b4068:::obj138:head) v3 2026-02-11T23:31:27.565+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:303427ce:::obj485:head local-lis/les=113/114 n=126 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:320b4068:::obj138:head) 2026-02-11T23:31:27.565+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.565+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:320b4068:::obj138:head) v3 finish latency 0.000013 2026-02-11T23:31:27.565+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.565+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.565+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:320b4068:::obj138:head local-lis/les=113/114 n=127 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.565+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.565+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.565+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.565+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d8e900 con 0x4acb000 2026-02-11T23:31:27.565+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:325eabfb:::obj310:head)]) -- 0x5d8e480 con 0x4acb000 2026-02-11T23:31:27.565+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.565+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 291 ==== MOSDPGPush(1.0 116/113 [PushOp(1:3263b485:::obj202:head, version: 111'202, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3263b485:::obj202:head@111'202, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e27520 con 0x4acb000 2026-02-11T23:31:27.565+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:3263b485:::obj202:head, version: 111'202, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3263b485:::obj202:head@111'202, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.565+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3263b485:::obj202:head, version: 111'202, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3263b485:::obj202:head@111'202, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.565+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.565+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3263b485:::obj202:head, version: 111'202, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3263b485:::obj202:head@111'202, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.565+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 292 ==== pg_backfill(progress 1.0 e 116/116 lb 1:325eabfb:::obj310:head) ==== 1062+0+0 (unknown 0 0 0) 0x5e16000 con 0x4acb000 2026-02-11T23:31:27.565+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.565+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:325eabfb:::obj310:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 116 2026-02-11T23:31:27.565+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3263b485:::obj202:head, version: 111'202, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3263b485:::obj202:head@111'202, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.565+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:325eabfb:::obj310:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.565+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.565+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:3263b485:::obj202:head, version: 111'202, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3263b485:::obj202:head@111'202, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(1.0 116/113 [PushOp(1:3263b485:::obj202:head, version: 111'202, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3263b485:::obj202:head@111'202, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:320b4068:::obj138:head local-lis/les=113/114 n=127 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.565+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:325eabfb:::obj310:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.565+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.565+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:320b4068:::obj138:head local-lis/les=113/114 n=127 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:3263b485:::obj202:head, version: 111'202, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3263b485:::obj202:head@111'202, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.565+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:320b4068:::obj138:head local-lis/les=113/114 n=127 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:3263b485:::obj202:head@111'202, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.565+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:320b4068:::obj138:head local-lis/les=113/114 n=127 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:3263b485:::obj202:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.565+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:320b4068:::obj138:head local-lis/les=113/114 n=127 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:3263b485:::obj202:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.565+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:320b4068:::obj138:head local-lis/les=113/114 n=127 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:3263b485:::obj202:head 2026-02-11T23:31:27.565+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:320b4068:::obj138:head local-lis/les=113/114 n=127 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:3263b485:::obj202:head v 111'202 2026-02-11T23:31:27.565+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:320b4068:::obj138:head local-lis/les=113/114 n=127 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.565+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.565+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:3263b485:::obj202:head, version: 111'202, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3263b485:::obj202:head@111'202, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:27.565+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:320b4068:::obj138:head local-lis/les=113/114 n=127 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.565+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.565+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:325eabfb:::obj310:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.565+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:325eabfb:::obj310:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 1.0 e 116/116 lb 1:325eabfb:::obj310:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:320b4068:::obj138:head local-lis/les=113/114 n=127 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.565+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.565+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.565+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:320b4068:::obj138:head local-lis/les=113/114 n=127 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:325eabfb:::obj310:head) v3 2026-02-11T23:31:27.565+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:320b4068:::obj138:head local-lis/les=113/114 n=127 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:325eabfb:::obj310:head) 2026-02-11T23:31:27.565+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.565+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:325eabfb:::obj310:head) v3 finish latency 0.000018 2026-02-11T23:31:27.565+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.565+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.565+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:325eabfb:::obj310:head local-lis/les=113/114 n=128 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.565+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.566+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.566+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.566+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d8ef00 con 0x4acb000 2026-02-11T23:31:27.566+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:3263b485:::obj202:head)]) -- 0x5d8ea80 con 0x4acb000 2026-02-11T23:31:27.566+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.566+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 293 ==== MOSDPGPush(1.0 116/113 [PushOp(1:326c82ab:::obj527:head, version: 111'527, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:326c82ab:::obj527:head@111'527, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e27d40 con 0x4acb000 2026-02-11T23:31:27.566+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:326c82ab:::obj527:head, version: 111'527, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:326c82ab:::obj527:head@111'527, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.566+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:326c82ab:::obj527:head, version: 111'527, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:326c82ab:::obj527:head@111'527, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.566+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.566+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:326c82ab:::obj527:head, version: 111'527, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:326c82ab:::obj527:head@111'527, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.566+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 294 ==== pg_backfill(progress 1.0 e 116/116 lb 1:3263b485:::obj202:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc3600 con 0x4acb000 2026-02-11T23:31:27.566+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.566+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:3263b485:::obj202:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 116 2026-02-11T23:31:27.566+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:326c82ab:::obj527:head, version: 111'527, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:326c82ab:::obj527:head@111'527, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.566+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3263b485:::obj202:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.566+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.566+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:326c82ab:::obj527:head, version: 111'527, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:326c82ab:::obj527:head@111'527, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(1.0 116/113 [PushOp(1:326c82ab:::obj527:head, version: 111'527, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:326c82ab:::obj527:head@111'527, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:325eabfb:::obj310:head local-lis/les=113/114 n=128 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.566+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3263b485:::obj202:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.566+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.566+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:325eabfb:::obj310:head local-lis/les=113/114 n=128 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:326c82ab:::obj527:head, version: 111'527, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:326c82ab:::obj527:head@111'527, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.566+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:325eabfb:::obj310:head local-lis/les=113/114 n=128 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:326c82ab:::obj527:head@111'527, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.566+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:325eabfb:::obj310:head local-lis/les=113/114 n=128 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:326c82ab:::obj527:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.566+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:325eabfb:::obj310:head local-lis/les=113/114 n=128 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:326c82ab:::obj527:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.566+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:325eabfb:::obj310:head local-lis/les=113/114 n=128 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:326c82ab:::obj527:head 2026-02-11T23:31:27.566+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:325eabfb:::obj310:head local-lis/les=113/114 n=128 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:326c82ab:::obj527:head v 111'527 2026-02-11T23:31:27.566+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:325eabfb:::obj310:head local-lis/les=113/114 n=128 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.566+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.566+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:326c82ab:::obj527:head, version: 111'527, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:326c82ab:::obj527:head@111'527, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:27.566+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.566+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.566+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3263b485:::obj202:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.566+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:3263b485:::obj202:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 1.0 e 116/116 lb 1:3263b485:::obj202:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:325eabfb:::obj310:head local-lis/les=113/114 n=128 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.566+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.566+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:325eabfb:::obj310:head local-lis/les=113/114 n=128 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:3263b485:::obj202:head) v3 2026-02-11T23:31:27.566+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:325eabfb:::obj310:head local-lis/les=113/114 n=128 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:3263b485:::obj202:head) 2026-02-11T23:31:27.566+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.566+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:3263b485:::obj202:head) v3 finish latency 0.000016 2026-02-11T23:31:27.566+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3263b485:::obj202:head local-lis/les=113/114 n=129 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.566+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.566+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.566+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3263b485:::obj202:head local-lis/les=113/114 n=129 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.566+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.566+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.566+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.566+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d8f200 con 0x4acb000 2026-02-11T23:31:27.566+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:326c82ab:::obj527:head)]) -- 0x5d78d80 con 0x4acb000 2026-02-11T23:31:27.566+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.566+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 295 ==== MOSDPGPush(1.0 116/113 [PushOp(1:3286c571:::obj350:head, version: 111'350, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3286c571:::obj350:head@111'350, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e27ba0 con 0x4acb000 2026-02-11T23:31:27.566+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:3286c571:::obj350:head, version: 111'350, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3286c571:::obj350:head@111'350, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 116 2026-02-11T23:31:27.566+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3286c571:::obj350:head, version: 111'350, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3286c571:::obj350:head@111'350, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.566+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.566+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3286c571:::obj350:head, version: 111'350, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3286c571:::obj350:head@111'350, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.566+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 296 ==== pg_backfill(progress 1.0 e 116/116 lb 1:326c82ab:::obj527:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc4c00 con 0x4acb000 2026-02-11T23:31:27.566+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.566+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:326c82ab:::obj527:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 116 2026-02-11T23:31:27.566+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3286c571:::obj350:head, version: 111'350, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3286c571:::obj350:head@111'350, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.566+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:326c82ab:::obj527:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.566+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.566+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:3286c571:::obj350:head, version: 111'350, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3286c571:::obj350:head@111'350, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(1.0 116/113 [PushOp(1:3286c571:::obj350:head, version: 111'350, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3286c571:::obj350:head@111'350, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:3263b485:::obj202:head local-lis/les=113/114 n=129 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.566+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:326c82ab:::obj527:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.566+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.566+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3263b485:::obj202:head local-lis/les=113/114 n=129 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:3286c571:::obj350:head, version: 111'350, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3286c571:::obj350:head@111'350, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.566+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3263b485:::obj202:head local-lis/les=113/114 n=129 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:3286c571:::obj350:head@111'350, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.566+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3263b485:::obj202:head local-lis/les=113/114 n=129 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:3286c571:::obj350:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.566+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3263b485:::obj202:head local-lis/les=113/114 n=129 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:3286c571:::obj350:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.566+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3263b485:::obj202:head local-lis/les=113/114 n=129 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:3286c571:::obj350:head 2026-02-11T23:31:27.566+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3263b485:::obj202:head local-lis/les=113/114 n=129 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:3286c571:::obj350:head v 111'350 2026-02-11T23:31:27.566+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3263b485:::obj202:head local-lis/les=113/114 n=129 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.566+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.566+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:3286c571:::obj350:head, version: 111'350, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3286c571:::obj350:head@111'350, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:27.567+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.567+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.567+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:326c82ab:::obj527:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.567+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:326c82ab:::obj527:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 1.0 e 116/116 lb 1:326c82ab:::obj527:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:3263b485:::obj202:head local-lis/les=113/114 n=129 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.567+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.567+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3263b485:::obj202:head local-lis/les=113/114 n=129 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:326c82ab:::obj527:head) v3 2026-02-11T23:31:27.567+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3263b485:::obj202:head local-lis/les=113/114 n=129 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:326c82ab:::obj527:head) 2026-02-11T23:31:27.567+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.567+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:326c82ab:::obj527:head) v3 finish latency 0.000014 2026-02-11T23:31:27.567+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:326c82ab:::obj527:head local-lis/les=113/114 n=130 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.567+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.567+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.567+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:326c82ab:::obj527:head local-lis/les=113/114 n=130 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.567+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.567+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.567+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.567+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5df9980 con 0x4acb000 2026-02-11T23:31:27.567+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:3286c571:::obj350:head)]) -- 0x5d78c00 con 0x4acb000 2026-02-11T23:31:27.567+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.567+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 297 ==== MOSDPGPush(1.0 116/113 [PushOp(1:32db648a:::obj568:head, version: 111'568, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:32db648a:::obj568:head@111'568, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db16c0 con 0x4acb000 2026-02-11T23:31:27.567+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:32db648a:::obj568:head, version: 111'568, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:32db648a:::obj568:head@111'568, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.567+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:32db648a:::obj568:head, version: 111'568, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:32db648a:::obj568:head@111'568, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.567+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.567+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:32db648a:::obj568:head, version: 111'568, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:32db648a:::obj568:head@111'568, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.567+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 298 ==== pg_backfill(progress 1.0 e 116/116 lb 1:3286c571:::obj350:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc4680 con 0x4acb000 2026-02-11T23:31:27.567+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.567+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:32db648a:::obj568:head, version: 111'568, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:32db648a:::obj568:head@111'568, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.567+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:3286c571:::obj350:head) v3 prio 15 type 95 cost 0 latency 0.000010 epoch 116 2026-02-11T23:31:27.567+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3286c571:::obj350:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.567+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:32db648a:::obj568:head, version: 111'568, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:32db648a:::obj568:head@111'568, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 116/113 [PushOp(1:32db648a:::obj568:head, version: 111'568, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:32db648a:::obj568:head@111'568, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:326c82ab:::obj527:head local-lis/les=113/114 n=130 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.567+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.567+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.567+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:326c82ab:::obj527:head local-lis/les=113/114 n=130 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:32db648a:::obj568:head, version: 111'568, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:32db648a:::obj568:head@111'568, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.567+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3286c571:::obj350:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.567+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:326c82ab:::obj527:head local-lis/les=113/114 n=130 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:32db648a:::obj568:head@111'568, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.567+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:326c82ab:::obj527:head local-lis/les=113/114 n=130 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:32db648a:::obj568:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.567+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:326c82ab:::obj527:head local-lis/les=113/114 n=130 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:32db648a:::obj568:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.567+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:326c82ab:::obj527:head local-lis/les=113/114 n=130 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:32db648a:::obj568:head 2026-02-11T23:31:27.567+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:326c82ab:::obj527:head local-lis/les=113/114 n=130 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:32db648a:::obj568:head v 111'568 2026-02-11T23:31:27.567+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:326c82ab:::obj527:head local-lis/les=113/114 n=130 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.567+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.567+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:32db648a:::obj568:head, version: 111'568, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:32db648a:::obj568:head@111'568, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:27.567+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.567+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.567+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3286c571:::obj350:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.567+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:3286c571:::obj350:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 1.0 e 116/116 lb 1:3286c571:::obj350:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:326c82ab:::obj527:head local-lis/les=113/114 n=130 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.567+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.567+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:326c82ab:::obj527:head local-lis/les=113/114 n=130 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:3286c571:::obj350:head) v3 2026-02-11T23:31:27.567+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:326c82ab:::obj527:head local-lis/les=113/114 n=130 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:3286c571:::obj350:head) 2026-02-11T23:31:27.567+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.567+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:3286c571:::obj350:head) v3 finish latency 0.000014 2026-02-11T23:31:27.567+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3286c571:::obj350:head local-lis/les=113/114 n=131 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.567+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.567+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.567+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3286c571:::obj350:head local-lis/les=113/114 n=131 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.567+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.567+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.567+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.567+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5df9e00 con 0x4acb000 2026-02-11T23:31:27.567+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:32db648a:::obj568:head)]) -- 0x5d79980 con 0x4acb000 2026-02-11T23:31:27.567+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.568+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 299 ==== MOSDPGPush(1.0 116/113 [PushOp(1:3372ec01:::obj225:head, version: 111'225, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3372ec01:::obj225:head@111'225, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3815d40 con 0x4acb000 2026-02-11T23:31:27.568+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:3372ec01:::obj225:head, version: 111'225, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3372ec01:::obj225:head@111'225, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.568+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3372ec01:::obj225:head, version: 111'225, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3372ec01:::obj225:head@111'225, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.568+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.568+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3372ec01:::obj225:head, version: 111'225, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3372ec01:::obj225:head@111'225, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.568+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 300 ==== pg_backfill(progress 1.0 e 116/116 lb 1:32db648a:::obj568:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc3b80 con 0x4acb000 2026-02-11T23:31:27.568+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.568+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:32db648a:::obj568:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 116 2026-02-11T23:31:27.568+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3372ec01:::obj225:head, version: 111'225, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3372ec01:::obj225:head@111'225, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.568+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:32db648a:::obj568:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.568+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.568+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:3372ec01:::obj225:head, version: 111'225, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3372ec01:::obj225:head@111'225, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 116/113 [PushOp(1:3372ec01:::obj225:head, version: 111'225, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3372ec01:::obj225:head@111'225, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:3286c571:::obj350:head local-lis/les=113/114 n=131 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.568+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:32db648a:::obj568:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.568+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.568+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3286c571:::obj350:head local-lis/les=113/114 n=131 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:3372ec01:::obj225:head, version: 111'225, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3372ec01:::obj225:head@111'225, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.568+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3286c571:::obj350:head local-lis/les=113/114 n=131 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:3372ec01:::obj225:head@111'225, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.568+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3286c571:::obj350:head local-lis/les=113/114 n=131 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:3372ec01:::obj225:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.568+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3286c571:::obj350:head local-lis/les=113/114 n=131 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:3372ec01:::obj225:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.568+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3286c571:::obj350:head local-lis/les=113/114 n=131 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:3372ec01:::obj225:head 2026-02-11T23:31:27.568+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3286c571:::obj350:head local-lis/les=113/114 n=131 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:3372ec01:::obj225:head v 111'225 2026-02-11T23:31:27.568+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3286c571:::obj350:head local-lis/les=113/114 n=131 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.568+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.568+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:3372ec01:::obj225:head, version: 111'225, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3372ec01:::obj225:head@111'225, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:27.568+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3286c571:::obj350:head local-lis/les=113/114 n=131 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.568+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.568+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.568+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:32db648a:::obj568:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.568+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:32db648a:::obj568:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 1.0 e 116/116 lb 1:32db648a:::obj568:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:3286c571:::obj350:head local-lis/les=113/114 n=131 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.568+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.568+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3286c571:::obj350:head local-lis/les=113/114 n=131 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:32db648a:::obj568:head) v3 2026-02-11T23:31:27.568+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3286c571:::obj350:head local-lis/les=113/114 n=131 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:32db648a:::obj568:head) 2026-02-11T23:31:27.568+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.568+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:32db648a:::obj568:head) v3 finish latency 0.000017 2026-02-11T23:31:27.568+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.568+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.568+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:32db648a:::obj568:head local-lis/les=113/114 n=132 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.568+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.568+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.568+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.568+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e7ed80 con 0x4acb000 2026-02-11T23:31:27.568+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:3372ec01:::obj225:head)]) -- 0x5e7f680 con 0x4acb000 2026-02-11T23:31:27.568+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.568+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 301 ==== MOSDPGPush(1.0 116/113 [PushOp(1:339fd60b:::obj93:head, version: 111'93, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:339fd60b:::obj93:head@111'93, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5e0a1a0 con 0x4acb000 2026-02-11T23:31:27.568+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:339fd60b:::obj93:head, version: 111'93, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:339fd60b:::obj93:head@111'93, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.568+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:339fd60b:::obj93:head, version: 111'93, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:339fd60b:::obj93:head@111'93, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.568+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.568+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:339fd60b:::obj93:head, version: 111'93, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:339fd60b:::obj93:head@111'93, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.568+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 302 ==== pg_backfill(progress 1.0 e 116/116 lb 1:3372ec01:::obj225:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc4100 con 0x4acb000 2026-02-11T23:31:27.568+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.568+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:3372ec01:::obj225:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.568+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:339fd60b:::obj93:head, version: 111'93, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:339fd60b:::obj93:head@111'93, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.568+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3372ec01:::obj225:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.568+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:339fd60b:::obj93:head, version: 111'93, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:339fd60b:::obj93:head@111'93, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 116/113 [PushOp(1:339fd60b:::obj93:head, version: 111'93, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:339fd60b:::obj93:head@111'93, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:32db648a:::obj568:head local-lis/les=113/114 n=132 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.568+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.568+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.568+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3372ec01:::obj225:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.568+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:32db648a:::obj568:head local-lis/les=113/114 n=132 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:339fd60b:::obj93:head, version: 111'93, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:339fd60b:::obj93:head@111'93, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.568+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:32db648a:::obj568:head local-lis/les=113/114 n=132 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:339fd60b:::obj93:head@111'93, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.568+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:32db648a:::obj568:head local-lis/les=113/114 n=132 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:339fd60b:::obj93:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.568+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:32db648a:::obj568:head local-lis/les=113/114 n=132 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:339fd60b:::obj93:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.568+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:32db648a:::obj568:head local-lis/les=113/114 n=132 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:339fd60b:::obj93:head 2026-02-11T23:31:27.568+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:32db648a:::obj568:head local-lis/les=113/114 n=132 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:339fd60b:::obj93:head v 111'93 2026-02-11T23:31:27.568+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:32db648a:::obj568:head local-lis/les=113/114 n=132 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.568+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.568+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:339fd60b:::obj93:head, version: 111'93, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:339fd60b:::obj93:head@111'93, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:27.568+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.568+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.568+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3372ec01:::obj225:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.568+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:3372ec01:::obj225:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 1.0 e 116/116 lb 1:3372ec01:::obj225:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:32db648a:::obj568:head local-lis/les=113/114 n=132 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.568+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.568+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:32db648a:::obj568:head local-lis/les=113/114 n=132 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:3372ec01:::obj225:head) v3 2026-02-11T23:31:27.568+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:32db648a:::obj568:head local-lis/les=113/114 n=132 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:3372ec01:::obj225:head) 2026-02-11T23:31:27.568+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.568+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:3372ec01:::obj225:head) v3 finish latency 0.000016 2026-02-11T23:31:27.568+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3372ec01:::obj225:head local-lis/les=113/114 n=133 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.568+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.568+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.568+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3372ec01:::obj225:head local-lis/les=113/114 n=133 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.568+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.568+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.569+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.569+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5df8000 con 0x4acb000 2026-02-11T23:31:27.569+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:339fd60b:::obj93:head)]) -- 0x5d78780 con 0x4acb000 2026-02-11T23:31:27.569+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.569+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 303 ==== MOSDPGPush(1.0 116/113 [PushOp(1:33a4b6a3:::obj360:head, version: 111'360, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:33a4b6a3:::obj360:head@111'360, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e0a680 con 0x4acb000 2026-02-11T23:31:27.569+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:33a4b6a3:::obj360:head, version: 111'360, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:33a4b6a3:::obj360:head@111'360, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.569+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:33a4b6a3:::obj360:head, version: 111'360, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:33a4b6a3:::obj360:head@111'360, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.569+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.569+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:33a4b6a3:::obj360:head, version: 111'360, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:33a4b6a3:::obj360:head@111'360, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.569+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 304 ==== pg_backfill(progress 1.0 e 116/116 lb 1:339fd60b:::obj93:head) ==== 1061+0+0 (unknown 0 0 0) 0x5dc2b00 con 0x4acb000 2026-02-11T23:31:27.569+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.569+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:33a4b6a3:::obj360:head, version: 111'360, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:33a4b6a3:::obj360:head@111'360, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.569+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:339fd60b:::obj93:head) v3 prio 15 type 95 cost 0 latency 0.000011 epoch 116 2026-02-11T23:31:27.569+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.569+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:33a4b6a3:::obj360:head, version: 111'360, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:33a4b6a3:::obj360:head@111'360, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(1.0 116/113 [PushOp(1:33a4b6a3:::obj360:head, version: 111'360, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:33a4b6a3:::obj360:head@111'360, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:3372ec01:::obj225:head local-lis/les=113/114 n=133 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.569+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.569+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3372ec01:::obj225:head local-lis/les=113/114 n=133 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:33a4b6a3:::obj360:head, version: 111'360, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:33a4b6a3:::obj360:head@111'360, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.569+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:339fd60b:::obj93:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.569+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3372ec01:::obj225:head local-lis/les=113/114 n=133 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:33a4b6a3:::obj360:head@111'360, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.569+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.569+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3372ec01:::obj225:head local-lis/les=113/114 n=133 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:33a4b6a3:::obj360:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.569+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:339fd60b:::obj93:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.569+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3372ec01:::obj225:head local-lis/les=113/114 n=133 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:33a4b6a3:::obj360:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.569+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3372ec01:::obj225:head local-lis/les=113/114 n=133 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:33a4b6a3:::obj360:head 2026-02-11T23:31:27.569+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3372ec01:::obj225:head local-lis/les=113/114 n=133 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:33a4b6a3:::obj360:head v 111'360 2026-02-11T23:31:27.569+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3372ec01:::obj225:head local-lis/les=113/114 n=133 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.569+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.569+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:33a4b6a3:::obj360:head, version: 111'360, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:33a4b6a3:::obj360:head@111'360, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:27.569+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3372ec01:::obj225:head local-lis/les=113/114 n=133 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.569+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.569+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.569+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:339fd60b:::obj93:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.569+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:339fd60b:::obj93:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 1.0 e 116/116 lb 1:339fd60b:::obj93:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:3372ec01:::obj225:head local-lis/les=113/114 n=133 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.569+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.569+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3372ec01:::obj225:head local-lis/les=113/114 n=133 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:339fd60b:::obj93:head) v3 2026-02-11T23:31:27.569+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3372ec01:::obj225:head local-lis/les=113/114 n=133 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:339fd60b:::obj93:head) 2026-02-11T23:31:27.569+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.569+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:339fd60b:::obj93:head) v3 finish latency 0.000017 2026-02-11T23:31:27.569+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.569+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.569+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:339fd60b:::obj93:head local-lis/les=113/114 n=134 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.569+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.569+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.569+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.569+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5df8780 con 0x4acb000 2026-02-11T23:31:27.569+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:33a4b6a3:::obj360:head)]) -- 0x5df8180 con 0x4acb000 2026-02-11T23:31:27.569+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.569+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 305 ==== MOSDPGPush(1.0 116/113 [PushOp(1:3421eea3:::obj166:head, version: 111'166, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3421eea3:::obj166:head@111'166, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e0ab60 con 0x4acb000 2026-02-11T23:31:27.569+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:3421eea3:::obj166:head, version: 111'166, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3421eea3:::obj166:head@111'166, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000016 epoch 116 2026-02-11T23:31:27.569+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3421eea3:::obj166:head, version: 111'166, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3421eea3:::obj166:head@111'166, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.569+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.569+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3421eea3:::obj166:head, version: 111'166, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3421eea3:::obj166:head@111'166, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.569+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 306 ==== pg_backfill(progress 1.0 e 116/116 lb 1:33a4b6a3:::obj360:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc5700 con 0x4acb000 2026-02-11T23:31:27.569+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.569+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3421eea3:::obj166:head, version: 111'166, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3421eea3:::obj166:head@111'166, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.569+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:33a4b6a3:::obj360:head) v3 prio 15 type 95 cost 0 latency 0.000011 epoch 116 2026-02-11T23:31:27.569+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:33a4b6a3:::obj360:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.569+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:3421eea3:::obj166:head, version: 111'166, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3421eea3:::obj166:head@111'166, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000044 MOSDPGPush(1.0 116/113 [PushOp(1:3421eea3:::obj166:head, version: 111'166, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3421eea3:::obj166:head@111'166, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:339fd60b:::obj93:head local-lis/les=113/114 n=134 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.569+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.569+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.569+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:33a4b6a3:::obj360:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.569+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:339fd60b:::obj93:head local-lis/les=113/114 n=134 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:3421eea3:::obj166:head, version: 111'166, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3421eea3:::obj166:head@111'166, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.569+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:339fd60b:::obj93:head local-lis/les=113/114 n=134 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:3421eea3:::obj166:head@111'166, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.569+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:339fd60b:::obj93:head local-lis/les=113/114 n=134 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:3421eea3:::obj166:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.569+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:339fd60b:::obj93:head local-lis/les=113/114 n=134 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:3421eea3:::obj166:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.569+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:339fd60b:::obj93:head local-lis/les=113/114 n=134 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:3421eea3:::obj166:head 2026-02-11T23:31:27.569+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:339fd60b:::obj93:head local-lis/les=113/114 n=134 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:3421eea3:::obj166:head v 111'166 2026-02-11T23:31:27.569+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:339fd60b:::obj93:head local-lis/les=113/114 n=134 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.570+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.570+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:3421eea3:::obj166:head, version: 111'166, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3421eea3:::obj166:head@111'166, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:27.570+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.570+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.570+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:33a4b6a3:::obj360:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.570+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:33a4b6a3:::obj360:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 1.0 e 116/116 lb 1:33a4b6a3:::obj360:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:339fd60b:::obj93:head local-lis/les=113/114 n=134 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.570+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.570+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:339fd60b:::obj93:head local-lis/les=113/114 n=134 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:33a4b6a3:::obj360:head) v3 2026-02-11T23:31:27.570+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:339fd60b:::obj93:head local-lis/les=113/114 n=134 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:33a4b6a3:::obj360:head) 2026-02-11T23:31:27.570+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.570+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:33a4b6a3:::obj360:head) v3 finish latency 0.000017 2026-02-11T23:31:27.570+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:33a4b6a3:::obj360:head local-lis/les=113/114 n=135 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.570+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.570+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.570+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:33a4b6a3:::obj360:head local-lis/les=113/114 n=135 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.570+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.570+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.570+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.570+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5df8a80 con 0x4acb000 2026-02-11T23:31:27.570+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:3421eea3:::obj166:head)]) -- 0x5e6b980 con 0x4acb000 2026-02-11T23:31:27.570+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.570+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 307 ==== MOSDPGPush(1.0 116/113 [PushOp(1:360ee05a:::obj519:head, version: 111'519, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:360ee05a:::obj519:head@111'519, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e0b040 con 0x4acb000 2026-02-11T23:31:27.570+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:360ee05a:::obj519:head, version: 111'519, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:360ee05a:::obj519:head@111'519, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.570+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:360ee05a:::obj519:head, version: 111'519, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:360ee05a:::obj519:head@111'519, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.570+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.570+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:360ee05a:::obj519:head, version: 111'519, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:360ee05a:::obj519:head@111'519, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.570+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 308 ==== pg_backfill(progress 1.0 e 116/116 lb 1:3421eea3:::obj166:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc2580 con 0x4acb000 2026-02-11T23:31:27.570+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.570+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:3421eea3:::obj166:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 116 2026-02-11T23:31:27.570+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:360ee05a:::obj519:head, version: 111'519, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:360ee05a:::obj519:head@111'519, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.570+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3421eea3:::obj166:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.570+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.570+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3421eea3:::obj166:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.570+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:360ee05a:::obj519:head, version: 111'519, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:360ee05a:::obj519:head@111'519, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 116/113 [PushOp(1:360ee05a:::obj519:head, version: 111'519, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:360ee05a:::obj519:head@111'519, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:33a4b6a3:::obj360:head local-lis/les=113/114 n=135 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.570+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.570+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:33a4b6a3:::obj360:head local-lis/les=113/114 n=135 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:360ee05a:::obj519:head, version: 111'519, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:360ee05a:::obj519:head@111'519, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.570+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:33a4b6a3:::obj360:head local-lis/les=113/114 n=135 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:360ee05a:::obj519:head@111'519, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.570+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:33a4b6a3:::obj360:head local-lis/les=113/114 n=135 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:360ee05a:::obj519:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.570+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:33a4b6a3:::obj360:head local-lis/les=113/114 n=135 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:360ee05a:::obj519:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.570+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:33a4b6a3:::obj360:head local-lis/les=113/114 n=135 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:360ee05a:::obj519:head 2026-02-11T23:31:27.570+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:33a4b6a3:::obj360:head local-lis/les=113/114 n=135 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:360ee05a:::obj519:head v 111'519 2026-02-11T23:31:27.570+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:33a4b6a3:::obj360:head local-lis/les=113/114 n=135 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.570+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.570+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:360ee05a:::obj519:head, version: 111'519, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:360ee05a:::obj519:head@111'519, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:27.570+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.570+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.570+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3421eea3:::obj166:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.570+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:3421eea3:::obj166:head) v3 prio 15 cost 0 latency 0.000080 pg_backfill(progress 1.0 e 116/116 lb 1:3421eea3:::obj166:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:33a4b6a3:::obj360:head local-lis/les=113/114 n=135 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.570+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.570+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:33a4b6a3:::obj360:head local-lis/les=113/114 n=135 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:3421eea3:::obj166:head) v3 2026-02-11T23:31:27.570+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:33a4b6a3:::obj360:head local-lis/les=113/114 n=135 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:3421eea3:::obj166:head) 2026-02-11T23:31:27.570+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.570+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:3421eea3:::obj166:head) v3 finish latency 0.000013 2026-02-11T23:31:27.570+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3421eea3:::obj166:head local-lis/les=113/114 n=136 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.570+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.570+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.570+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3421eea3:::obj166:head local-lis/les=113/114 n=136 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.570+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.570+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.570+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.570+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5df9080 con 0x4acb000 2026-02-11T23:31:27.570+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:360ee05a:::obj519:head)]) -- 0x5e3a480 con 0x4acb000 2026-02-11T23:31:27.570+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.571+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 309 ==== MOSDPGPush(1.0 116/113 [PushOp(1:3655ac21:::obj70:head, version: 111'70, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3655ac21:::obj70:head@111'70, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5e0b520 con 0x4acb000 2026-02-11T23:31:27.571+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:3655ac21:::obj70:head, version: 111'70, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3655ac21:::obj70:head@111'70, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.571+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3655ac21:::obj70:head, version: 111'70, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3655ac21:::obj70:head@111'70, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.571+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.571+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3655ac21:::obj70:head, version: 111'70, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3655ac21:::obj70:head@111'70, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.571+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 310 ==== pg_backfill(progress 1.0 e 116/116 lb 1:360ee05a:::obj519:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc3080 con 0x4acb000 2026-02-11T23:31:27.571+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:360ee05a:::obj519:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.571+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.571+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:360ee05a:::obj519:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.571+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3655ac21:::obj70:head, version: 111'70, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3655ac21:::obj70:head@111'70, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.571+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.571+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:360ee05a:::obj519:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.571+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:3655ac21:::obj70:head, version: 111'70, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3655ac21:::obj70:head@111'70, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 116/113 [PushOp(1:3655ac21:::obj70:head, version: 111'70, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3655ac21:::obj70:head@111'70, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:3421eea3:::obj166:head local-lis/les=113/114 n=136 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.571+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.571+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3421eea3:::obj166:head local-lis/les=113/114 n=136 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:3655ac21:::obj70:head, version: 111'70, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3655ac21:::obj70:head@111'70, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.571+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3421eea3:::obj166:head local-lis/les=113/114 n=136 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:3655ac21:::obj70:head@111'70, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.571+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3421eea3:::obj166:head local-lis/les=113/114 n=136 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:3655ac21:::obj70:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.571+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3421eea3:::obj166:head local-lis/les=113/114 n=136 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:3655ac21:::obj70:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.571+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3421eea3:::obj166:head local-lis/les=113/114 n=136 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:3655ac21:::obj70:head 2026-02-11T23:31:27.571+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3421eea3:::obj166:head local-lis/les=113/114 n=136 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:3655ac21:::obj70:head v 111'70 2026-02-11T23:31:27.571+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3421eea3:::obj166:head local-lis/les=113/114 n=136 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.571+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.571+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:3655ac21:::obj70:head, version: 111'70, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3655ac21:::obj70:head@111'70, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:27.571+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3421eea3:::obj166:head local-lis/les=113/114 n=136 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.571+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.571+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.571+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:360ee05a:::obj519:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.571+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:360ee05a:::obj519:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 1.0 e 116/116 lb 1:360ee05a:::obj519:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:3421eea3:::obj166:head local-lis/les=113/114 n=136 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.571+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.571+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3421eea3:::obj166:head local-lis/les=113/114 n=136 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:360ee05a:::obj519:head) v3 2026-02-11T23:31:27.571+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3421eea3:::obj166:head local-lis/les=113/114 n=136 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:360ee05a:::obj519:head) 2026-02-11T23:31:27.571+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.571+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:360ee05a:::obj519:head) v3 finish latency 0.000016 2026-02-11T23:31:27.571+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.571+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.571+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:360ee05a:::obj519:head local-lis/les=113/114 n=137 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.571+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.571+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.571+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.571+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5df9500 con 0x4acb000 2026-02-11T23:31:27.571+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:3655ac21:::obj70:head)]) -- 0x5df9380 con 0x4acb000 2026-02-11T23:31:27.571+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.571+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 311 ==== MOSDPGPush(1.0 116/113 [PushOp(1:372fd24d:::obj442:head, version: 111'442, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:372fd24d:::obj442:head@111'442, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e0ba00 con 0x4acb000 2026-02-11T23:31:27.571+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:372fd24d:::obj442:head, version: 111'442, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:372fd24d:::obj442:head@111'442, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.571+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:372fd24d:::obj442:head, version: 111'442, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:372fd24d:::obj442:head@111'442, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.571+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.571+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:372fd24d:::obj442:head, version: 111'442, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:372fd24d:::obj442:head@111'442, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.571+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 312 ==== pg_backfill(progress 1.0 e 116/116 lb 1:3655ac21:::obj70:head) ==== 1061+0+0 (unknown 0 0 0) 0x4a2cb00 con 0x4acb000 2026-02-11T23:31:27.571+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:3655ac21:::obj70:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.571+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.571+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3655ac21:::obj70:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.571+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:372fd24d:::obj442:head, version: 111'442, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:372fd24d:::obj442:head@111'442, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.571+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.571+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:372fd24d:::obj442:head, version: 111'442, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:372fd24d:::obj442:head@111'442, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 116/113 [PushOp(1:372fd24d:::obj442:head, version: 111'442, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:372fd24d:::obj442:head@111'442, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:360ee05a:::obj519:head local-lis/les=113/114 n=137 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.571+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.571+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:360ee05a:::obj519:head local-lis/les=113/114 n=137 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:372fd24d:::obj442:head, version: 111'442, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:372fd24d:::obj442:head@111'442, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.571+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:360ee05a:::obj519:head local-lis/les=113/114 n=137 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:372fd24d:::obj442:head@111'442, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.571+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.571+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3655ac21:::obj70:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.571+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:360ee05a:::obj519:head local-lis/les=113/114 n=137 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:372fd24d:::obj442:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.571+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:360ee05a:::obj519:head local-lis/les=113/114 n=137 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:372fd24d:::obj442:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.571+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:360ee05a:::obj519:head local-lis/les=113/114 n=137 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:372fd24d:::obj442:head 2026-02-11T23:31:27.571+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:360ee05a:::obj519:head local-lis/les=113/114 n=137 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:372fd24d:::obj442:head v 111'442 2026-02-11T23:31:27.571+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:360ee05a:::obj519:head local-lis/les=113/114 n=137 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.571+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.571+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:372fd24d:::obj442:head, version: 111'442, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:372fd24d:::obj442:head@111'442, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.571+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.571+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.571+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3655ac21:::obj70:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.571+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:3655ac21:::obj70:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 1.0 e 116/116 lb 1:3655ac21:::obj70:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:360ee05a:::obj519:head local-lis/les=113/114 n=137 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.571+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.571+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:360ee05a:::obj519:head local-lis/les=113/114 n=137 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:3655ac21:::obj70:head) v3 2026-02-11T23:31:27.571+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:360ee05a:::obj519:head local-lis/les=113/114 n=137 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:3655ac21:::obj70:head) 2026-02-11T23:31:27.571+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.571+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:3655ac21:::obj70:head) v3 finish latency 0.000019 2026-02-11T23:31:27.571+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3655ac21:::obj70:head local-lis/les=113/114 n=138 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.571+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.572+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.572+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3655ac21:::obj70:head local-lis/les=113/114 n=138 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.572+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.572+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.572+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.572+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x4ec2780 con 0x4acb000 2026-02-11T23:31:27.572+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:372fd24d:::obj442:head)]) -- 0x5d8f500 con 0x4acb000 2026-02-11T23:31:27.572+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.572+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 313 ==== MOSDPGPush(1.0 116/113 [PushOp(1:37ff9db7:::obj453:head, version: 111'453, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:37ff9db7:::obj453:head@111'453, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55e64e0 con 0x4acb000 2026-02-11T23:31:27.572+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:37ff9db7:::obj453:head, version: 111'453, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:37ff9db7:::obj453:head@111'453, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 116 2026-02-11T23:31:27.572+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:37ff9db7:::obj453:head, version: 111'453, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:37ff9db7:::obj453:head@111'453, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.572+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.572+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 314 ==== pg_backfill(progress 1.0 e 116/116 lb 1:372fd24d:::obj442:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2d080 con 0x4acb000 2026-02-11T23:31:27.572+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:37ff9db7:::obj453:head, version: 111'453, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:37ff9db7:::obj453:head@111'453, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.572+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:372fd24d:::obj442:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 116 2026-02-11T23:31:27.572+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.572+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:372fd24d:::obj442:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.572+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:37ff9db7:::obj453:head, version: 111'453, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:37ff9db7:::obj453:head@111'453, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.572+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.572+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:37ff9db7:::obj453:head, version: 111'453, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:37ff9db7:::obj453:head@111'453, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(1.0 116/113 [PushOp(1:37ff9db7:::obj453:head, version: 111'453, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:37ff9db7:::obj453:head@111'453, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:3655ac21:::obj70:head local-lis/les=113/114 n=138 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.572+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.572+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3655ac21:::obj70:head local-lis/les=113/114 n=138 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:37ff9db7:::obj453:head, version: 111'453, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:37ff9db7:::obj453:head@111'453, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.572+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3655ac21:::obj70:head local-lis/les=113/114 n=138 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:37ff9db7:::obj453:head@111'453, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.572+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.572+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:372fd24d:::obj442:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.572+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3655ac21:::obj70:head local-lis/les=113/114 n=138 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:37ff9db7:::obj453:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.572+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3655ac21:::obj70:head local-lis/les=113/114 n=138 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:37ff9db7:::obj453:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.572+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3655ac21:::obj70:head local-lis/les=113/114 n=138 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:37ff9db7:::obj453:head 2026-02-11T23:31:27.572+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3655ac21:::obj70:head local-lis/les=113/114 n=138 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:37ff9db7:::obj453:head v 111'453 2026-02-11T23:31:27.572+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3655ac21:::obj70:head local-lis/les=113/114 n=138 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.572+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.572+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:37ff9db7:::obj453:head, version: 111'453, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:37ff9db7:::obj453:head@111'453, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:27.572+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.572+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.572+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:372fd24d:::obj442:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.572+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:372fd24d:::obj442:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 1.0 e 116/116 lb 1:372fd24d:::obj442:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:3655ac21:::obj70:head local-lis/les=113/114 n=138 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.572+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.572+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3655ac21:::obj70:head local-lis/les=113/114 n=138 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:372fd24d:::obj442:head) v3 2026-02-11T23:31:27.572+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3655ac21:::obj70:head local-lis/les=113/114 n=138 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:372fd24d:::obj442:head) 2026-02-11T23:31:27.572+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.572+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:372fd24d:::obj442:head) v3 finish latency 0.000015 2026-02-11T23:31:27.572+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:372fd24d:::obj442:head local-lis/les=113/114 n=139 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.572+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.572+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.572+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:372fd24d:::obj442:head local-lis/les=113/114 n=139 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.572+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.572+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.572+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.572+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d79e00 con 0x4acb000 2026-02-11T23:31:27.572+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:37ff9db7:::obj453:head)]) -- 0x5d8fb00 con 0x4acb000 2026-02-11T23:31:27.572+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.573+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 315 ==== MOSDPGPush(1.0 116/113 [PushOp(1:384eca0c:::obj484:head, version: 111'484, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:384eca0c:::obj484:head@111'484, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55e6820 con 0x4acb000 2026-02-11T23:31:27.573+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:384eca0c:::obj484:head, version: 111'484, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:384eca0c:::obj484:head@111'484, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.573+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:384eca0c:::obj484:head, version: 111'484, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:384eca0c:::obj484:head@111'484, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.573+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.573+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:384eca0c:::obj484:head, version: 111'484, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:384eca0c:::obj484:head@111'484, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.573+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 316 ==== pg_backfill(progress 1.0 e 116/116 lb 1:37ff9db7:::obj453:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e680 con 0x4acb000 2026-02-11T23:31:27.573+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:37ff9db7:::obj453:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.573+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.573+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:37ff9db7:::obj453:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.573+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:384eca0c:::obj484:head, version: 111'484, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:384eca0c:::obj484:head@111'484, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.573+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.573+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:384eca0c:::obj484:head, version: 111'484, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:384eca0c:::obj484:head@111'484, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 116/113 [PushOp(1:384eca0c:::obj484:head, version: 111'484, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:384eca0c:::obj484:head@111'484, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:372fd24d:::obj442:head local-lis/les=113/114 n=139 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.573+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.573+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:372fd24d:::obj442:head local-lis/les=113/114 n=139 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:384eca0c:::obj484:head, version: 111'484, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:384eca0c:::obj484:head@111'484, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.573+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.573+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:372fd24d:::obj442:head local-lis/les=113/114 n=139 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:384eca0c:::obj484:head@111'484, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.573+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:37ff9db7:::obj453:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.573+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:372fd24d:::obj442:head local-lis/les=113/114 n=139 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:384eca0c:::obj484:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.573+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:372fd24d:::obj442:head local-lis/les=113/114 n=139 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:384eca0c:::obj484:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.573+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:372fd24d:::obj442:head local-lis/les=113/114 n=139 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:384eca0c:::obj484:head 2026-02-11T23:31:27.573+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:372fd24d:::obj442:head local-lis/les=113/114 n=139 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:384eca0c:::obj484:head v 111'484 2026-02-11T23:31:27.573+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:372fd24d:::obj442:head local-lis/les=113/114 n=139 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.573+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.573+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:384eca0c:::obj484:head, version: 111'484, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:384eca0c:::obj484:head@111'484, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:27.573+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.573+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.573+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:37ff9db7:::obj453:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.573+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:37ff9db7:::obj453:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 1.0 e 116/116 lb 1:37ff9db7:::obj453:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:372fd24d:::obj442:head local-lis/les=113/114 n=139 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.573+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.573+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:372fd24d:::obj442:head local-lis/les=113/114 n=139 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:37ff9db7:::obj453:head) v3 2026-02-11T23:31:27.573+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:372fd24d:::obj442:head local-lis/les=113/114 n=139 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:37ff9db7:::obj453:head) 2026-02-11T23:31:27.573+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.573+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:37ff9db7:::obj453:head) v3 finish latency 0.000016 2026-02-11T23:31:27.573+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:37ff9db7:::obj453:head local-lis/les=113/114 n=140 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.573+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.573+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.573+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:37ff9db7:::obj453:head local-lis/les=113/114 n=140 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.573+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.573+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.573+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.573+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d8e180 con 0x4acb000 2026-02-11T23:31:27.573+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:384eca0c:::obj484:head)]) -- 0x4ec2780 con 0x4acb000 2026-02-11T23:31:27.573+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.573+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 317 ==== MOSDPGPush(1.0 116/113 [PushOp(1:385733f0:::obj251:head, version: 111'251, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:385733f0:::obj251:head@111'251, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56aa4e0 con 0x4acb000 2026-02-11T23:31:27.573+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:385733f0:::obj251:head, version: 111'251, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:385733f0:::obj251:head@111'251, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.573+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:385733f0:::obj251:head, version: 111'251, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:385733f0:::obj251:head@111'251, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.573+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.573+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:385733f0:::obj251:head, version: 111'251, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:385733f0:::obj251:head@111'251, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.573+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 318 ==== pg_backfill(progress 1.0 e 116/116 lb 1:384eca0c:::obj484:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e100 con 0x4acb000 2026-02-11T23:31:27.573+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.573+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:384eca0c:::obj484:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.573+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:385733f0:::obj251:head, version: 111'251, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:385733f0:::obj251:head@111'251, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.573+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:384eca0c:::obj484:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.573+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.573+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:384eca0c:::obj484:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.573+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:385733f0:::obj251:head, version: 111'251, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:385733f0:::obj251:head@111'251, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 116/113 [PushOp(1:385733f0:::obj251:head, version: 111'251, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:385733f0:::obj251:head@111'251, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:37ff9db7:::obj453:head local-lis/les=113/114 n=140 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.573+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.573+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:37ff9db7:::obj453:head local-lis/les=113/114 n=140 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:385733f0:::obj251:head, version: 111'251, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:385733f0:::obj251:head@111'251, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.573+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:37ff9db7:::obj453:head local-lis/les=113/114 n=140 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:385733f0:::obj251:head@111'251, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.573+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:37ff9db7:::obj453:head local-lis/les=113/114 n=140 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:385733f0:::obj251:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.573+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:37ff9db7:::obj453:head local-lis/les=113/114 n=140 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:385733f0:::obj251:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.573+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:37ff9db7:::obj453:head local-lis/les=113/114 n=140 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:385733f0:::obj251:head 2026-02-11T23:31:27.573+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:37ff9db7:::obj453:head local-lis/les=113/114 n=140 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:385733f0:::obj251:head v 111'251 2026-02-11T23:31:27.573+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:37ff9db7:::obj453:head local-lis/les=113/114 n=140 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.573+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.573+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:385733f0:::obj251:head, version: 111'251, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:385733f0:::obj251:head@111'251, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:27.573+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.573+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.573+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:384eca0c:::obj484:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.573+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:384eca0c:::obj484:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 1.0 e 116/116 lb 1:384eca0c:::obj484:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:37ff9db7:::obj453:head local-lis/les=113/114 n=140 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.573+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.573+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:37ff9db7:::obj453:head local-lis/les=113/114 n=140 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:384eca0c:::obj484:head) v3 2026-02-11T23:31:27.573+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:37ff9db7:::obj453:head local-lis/les=113/114 n=140 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:384eca0c:::obj484:head) 2026-02-11T23:31:27.573+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.573+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:384eca0c:::obj484:head) v3 finish latency 0.000016 2026-02-11T23:31:27.573+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:384eca0c:::obj484:head local-lis/les=113/114 n=141 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.573+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.573+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.573+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:384eca0c:::obj484:head local-lis/les=113/114 n=141 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.573+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.573+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.573+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.573+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d8e900 con 0x4acb000 2026-02-11T23:31:27.573+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:385733f0:::obj251:head)]) -- 0x5df9980 con 0x4acb000 2026-02-11T23:31:27.573+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.574+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 319 ==== MOSDPGPush(1.0 116/113 [PushOp(1:3945af93:::obj271:head, version: 111'271, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3945af93:::obj271:head@111'271, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56aa9c0 con 0x4acb000 2026-02-11T23:31:27.574+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:3945af93:::obj271:head, version: 111'271, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3945af93:::obj271:head@111'271, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.574+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3945af93:::obj271:head, version: 111'271, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3945af93:::obj271:head@111'271, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.574+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.574+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3945af93:::obj271:head, version: 111'271, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3945af93:::obj271:head@111'271, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.574+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 320 ==== pg_backfill(progress 1.0 e 116/116 lb 1:385733f0:::obj251:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc3600 con 0x4acb000 2026-02-11T23:31:27.574+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:385733f0:::obj251:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 116 2026-02-11T23:31:27.574+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.574+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3945af93:::obj271:head, version: 111'271, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3945af93:::obj271:head@111'271, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.574+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:385733f0:::obj251:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.574+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.574+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:3945af93:::obj271:head, version: 111'271, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3945af93:::obj271:head@111'271, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 116/113 [PushOp(1:3945af93:::obj271:head, version: 111'271, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3945af93:::obj271:head@111'271, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:384eca0c:::obj484:head local-lis/les=113/114 n=141 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.574+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:385733f0:::obj251:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.574+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.574+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:384eca0c:::obj484:head local-lis/les=113/114 n=141 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:3945af93:::obj271:head, version: 111'271, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3945af93:::obj271:head@111'271, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.574+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:384eca0c:::obj484:head local-lis/les=113/114 n=141 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:3945af93:::obj271:head@111'271, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.574+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:384eca0c:::obj484:head local-lis/les=113/114 n=141 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:3945af93:::obj271:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.574+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:384eca0c:::obj484:head local-lis/les=113/114 n=141 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:3945af93:::obj271:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.574+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:384eca0c:::obj484:head local-lis/les=113/114 n=141 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:3945af93:::obj271:head 2026-02-11T23:31:27.574+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:384eca0c:::obj484:head local-lis/les=113/114 n=141 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:3945af93:::obj271:head v 111'271 2026-02-11T23:31:27.574+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:384eca0c:::obj484:head local-lis/les=113/114 n=141 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.574+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.574+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:3945af93:::obj271:head, version: 111'271, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3945af93:::obj271:head@111'271, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:27.574+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:384eca0c:::obj484:head local-lis/les=113/114 n=141 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.574+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.574+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.574+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:385733f0:::obj251:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.574+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:385733f0:::obj251:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 1.0 e 116/116 lb 1:385733f0:::obj251:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:384eca0c:::obj484:head local-lis/les=113/114 n=141 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.574+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.574+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:384eca0c:::obj484:head local-lis/les=113/114 n=141 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:385733f0:::obj251:head) v3 2026-02-11T23:31:27.574+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:384eca0c:::obj484:head local-lis/les=113/114 n=141 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:385733f0:::obj251:head) 2026-02-11T23:31:27.574+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.574+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:385733f0:::obj251:head) v3 finish latency 0.000016 2026-02-11T23:31:27.574+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.574+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.574+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:385733f0:::obj251:head local-lis/les=113/114 n=142 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.574+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.574+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.574+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.574+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d8ea80 con 0x4acb000 2026-02-11T23:31:27.574+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:3945af93:::obj271:head)]) -- 0x5d8e480 con 0x4acb000 2026-02-11T23:31:27.574+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.574+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 321 ==== MOSDPGPush(1.0 116/113 [PushOp(1:3a764ab4:::obj37:head, version: 111'37, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3a764ab4:::obj37:head@111'37, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x56aaea0 con 0x4acb000 2026-02-11T23:31:27.574+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:3a764ab4:::obj37:head, version: 111'37, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3a764ab4:::obj37:head@111'37, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.574+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3a764ab4:::obj37:head, version: 111'37, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3a764ab4:::obj37:head@111'37, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.574+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.574+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3a764ab4:::obj37:head, version: 111'37, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3a764ab4:::obj37:head@111'37, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.574+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 322 ==== pg_backfill(progress 1.0 e 116/116 lb 1:3945af93:::obj271:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc4c00 con 0x4acb000 2026-02-11T23:31:27.574+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.574+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:3945af93:::obj271:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.574+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3a764ab4:::obj37:head, version: 111'37, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3a764ab4:::obj37:head@111'37, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.574+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3945af93:::obj271:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.574+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.574+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:3a764ab4:::obj37:head, version: 111'37, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3a764ab4:::obj37:head@111'37, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 116/113 [PushOp(1:3a764ab4:::obj37:head, version: 111'37, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3a764ab4:::obj37:head@111'37, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:385733f0:::obj251:head local-lis/les=113/114 n=142 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.574+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.574+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3945af93:::obj271:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.574+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:385733f0:::obj251:head local-lis/les=113/114 n=142 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:3a764ab4:::obj37:head, version: 111'37, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3a764ab4:::obj37:head@111'37, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.574+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:385733f0:::obj251:head local-lis/les=113/114 n=142 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:3a764ab4:::obj37:head@111'37, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.574+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:385733f0:::obj251:head local-lis/les=113/114 n=142 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:3a764ab4:::obj37:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.574+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:385733f0:::obj251:head local-lis/les=113/114 n=142 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:3a764ab4:::obj37:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.574+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:385733f0:::obj251:head local-lis/les=113/114 n=142 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:3a764ab4:::obj37:head 2026-02-11T23:31:27.574+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:385733f0:::obj251:head local-lis/les=113/114 n=142 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:3a764ab4:::obj37:head v 111'37 2026-02-11T23:31:27.574+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:385733f0:::obj251:head local-lis/les=113/114 n=142 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.574+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.574+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:3a764ab4:::obj37:head, version: 111'37, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3a764ab4:::obj37:head@111'37, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:27.574+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.574+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.574+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3945af93:::obj271:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.574+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:3945af93:::obj271:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 1.0 e 116/116 lb 1:3945af93:::obj271:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:385733f0:::obj251:head local-lis/les=113/114 n=142 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.574+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.574+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:385733f0:::obj251:head local-lis/les=113/114 n=142 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:3945af93:::obj271:head) v3 2026-02-11T23:31:27.574+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:385733f0:::obj251:head local-lis/les=113/114 n=142 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:3945af93:::obj271:head) 2026-02-11T23:31:27.574+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.574+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:3945af93:::obj271:head) v3 finish latency 0.000018 2026-02-11T23:31:27.574+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3945af93:::obj271:head local-lis/les=113/114 n=143 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.574+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.575+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.575+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3945af93:::obj271:head local-lis/les=113/114 n=143 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.575+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.575+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.575+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.575+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d8f200 con 0x4acb000 2026-02-11T23:31:27.575+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:3a764ab4:::obj37:head)]) -- 0x5df8000 con 0x4acb000 2026-02-11T23:31:27.575+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.575+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 323 ==== MOSDPGPush(1.0 116/113 [PushOp(1:3b0aa4c9:::obj511:head, version: 111'511, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3b0aa4c9:::obj511:head@111'511, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56ab380 con 0x4acb000 2026-02-11T23:31:27.575+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:3b0aa4c9:::obj511:head, version: 111'511, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3b0aa4c9:::obj511:head@111'511, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.575+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3b0aa4c9:::obj511:head, version: 111'511, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3b0aa4c9:::obj511:head@111'511, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.575+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.575+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 324 ==== pg_backfill(progress 1.0 e 116/116 lb 1:3a764ab4:::obj37:head) ==== 1061+0+0 (unknown 0 0 0) 0x5dc4680 con 0x4acb000 2026-02-11T23:31:27.575+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3b0aa4c9:::obj511:head, version: 111'511, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3b0aa4c9:::obj511:head@111'511, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.575+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:3a764ab4:::obj37:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 116 2026-02-11T23:31:27.575+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.575+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3a764ab4:::obj37:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.575+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3b0aa4c9:::obj511:head, version: 111'511, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3b0aa4c9:::obj511:head@111'511, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.575+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.575+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:3b0aa4c9:::obj511:head, version: 111'511, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3b0aa4c9:::obj511:head@111'511, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 116/113 [PushOp(1:3b0aa4c9:::obj511:head, version: 111'511, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3b0aa4c9:::obj511:head@111'511, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:3945af93:::obj271:head local-lis/les=113/114 n=143 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.575+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.575+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3945af93:::obj271:head local-lis/les=113/114 n=143 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:3b0aa4c9:::obj511:head, version: 111'511, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3b0aa4c9:::obj511:head@111'511, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.575+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.575+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3945af93:::obj271:head local-lis/les=113/114 n=143 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:3b0aa4c9:::obj511:head@111'511, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.575+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3a764ab4:::obj37:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.575+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3945af93:::obj271:head local-lis/les=113/114 n=143 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:3b0aa4c9:::obj511:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.575+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3945af93:::obj271:head local-lis/les=113/114 n=143 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:3b0aa4c9:::obj511:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.575+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3945af93:::obj271:head local-lis/les=113/114 n=143 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:3b0aa4c9:::obj511:head 2026-02-11T23:31:27.575+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3945af93:::obj271:head local-lis/les=113/114 n=143 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:3b0aa4c9:::obj511:head v 111'511 2026-02-11T23:31:27.575+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3945af93:::obj271:head local-lis/les=113/114 n=143 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.575+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.575+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:3b0aa4c9:::obj511:head, version: 111'511, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3b0aa4c9:::obj511:head@111'511, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:27.575+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.575+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.575+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3a764ab4:::obj37:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.575+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:3a764ab4:::obj37:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 1.0 e 116/116 lb 1:3a764ab4:::obj37:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:3945af93:::obj271:head local-lis/les=113/114 n=143 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.575+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.575+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3945af93:::obj271:head local-lis/les=113/114 n=143 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:3a764ab4:::obj37:head) v3 2026-02-11T23:31:27.575+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3945af93:::obj271:head local-lis/les=113/114 n=143 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:3a764ab4:::obj37:head) 2026-02-11T23:31:27.575+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.575+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:3a764ab4:::obj37:head) v3 finish latency 0.000016 2026-02-11T23:31:27.575+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3a764ab4:::obj37:head local-lis/les=113/114 n=144 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.575+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.575+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.575+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3a764ab4:::obj37:head local-lis/les=113/114 n=144 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.575+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.575+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.575+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.575+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d8f500 con 0x4acb000 2026-02-11T23:31:27.575+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:3b0aa4c9:::obj511:head)]) -- 0x5df8780 con 0x4acb000 2026-02-11T23:31:27.575+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.576+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 325 ==== MOSDPGPush(1.0 116/113 [PushOp(1:3bc05e5f:::obj343:head, version: 111'343, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3bc05e5f:::obj343:head@111'343, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56ab860 con 0x4acb000 2026-02-11T23:31:27.576+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:3bc05e5f:::obj343:head, version: 111'343, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3bc05e5f:::obj343:head@111'343, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 116 2026-02-11T23:31:27.576+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3bc05e5f:::obj343:head, version: 111'343, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3bc05e5f:::obj343:head@111'343, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.576+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.576+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3bc05e5f:::obj343:head, version: 111'343, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3bc05e5f:::obj343:head@111'343, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.576+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 326 ==== pg_backfill(progress 1.0 e 116/116 lb 1:3b0aa4c9:::obj511:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc3b80 con 0x4acb000 2026-02-11T23:31:27.576+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.576+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:3b0aa4c9:::obj511:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 116 2026-02-11T23:31:27.576+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3bc05e5f:::obj343:head, version: 111'343, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3bc05e5f:::obj343:head@111'343, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.576+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3b0aa4c9:::obj511:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.576+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.576+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3b0aa4c9:::obj511:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.576+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:3bc05e5f:::obj343:head, version: 111'343, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3bc05e5f:::obj343:head@111'343, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(1.0 116/113 [PushOp(1:3bc05e5f:::obj343:head, version: 111'343, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3bc05e5f:::obj343:head@111'343, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:3a764ab4:::obj37:head local-lis/les=113/114 n=144 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.576+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.576+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3a764ab4:::obj37:head local-lis/les=113/114 n=144 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:3bc05e5f:::obj343:head, version: 111'343, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3bc05e5f:::obj343:head@111'343, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.576+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3a764ab4:::obj37:head local-lis/les=113/114 n=144 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:3bc05e5f:::obj343:head@111'343, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.576+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3a764ab4:::obj37:head local-lis/les=113/114 n=144 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:3bc05e5f:::obj343:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.576+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3a764ab4:::obj37:head local-lis/les=113/114 n=144 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:3bc05e5f:::obj343:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.576+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3a764ab4:::obj37:head local-lis/les=113/114 n=144 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:3bc05e5f:::obj343:head 2026-02-11T23:31:27.576+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3a764ab4:::obj37:head local-lis/les=113/114 n=144 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:3bc05e5f:::obj343:head v 111'343 2026-02-11T23:31:27.576+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3a764ab4:::obj37:head local-lis/les=113/114 n=144 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.576+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.576+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:3bc05e5f:::obj343:head, version: 111'343, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3bc05e5f:::obj343:head@111'343, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:27.576+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.576+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.576+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3b0aa4c9:::obj511:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.576+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:3b0aa4c9:::obj511:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 1.0 e 116/116 lb 1:3b0aa4c9:::obj511:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:3a764ab4:::obj37:head local-lis/les=113/114 n=144 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.576+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.576+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3a764ab4:::obj37:head local-lis/les=113/114 n=144 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:3b0aa4c9:::obj511:head) v3 2026-02-11T23:31:27.576+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3a764ab4:::obj37:head local-lis/les=113/114 n=144 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:3b0aa4c9:::obj511:head) 2026-02-11T23:31:27.576+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.576+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:3b0aa4c9:::obj511:head) v3 finish latency 0.000015 2026-02-11T23:31:27.576+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3b0aa4c9:::obj511:head local-lis/les=113/114 n=145 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.576+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.576+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.576+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3b0aa4c9:::obj511:head local-lis/les=113/114 n=145 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.576+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.576+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.576+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.576+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d8fb00 con 0x4acb000 2026-02-11T23:31:27.576+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:3bc05e5f:::obj343:head)]) -- 0x5df8a80 con 0x4acb000 2026-02-11T23:31:27.576+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.576+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 327 ==== MOSDPGPush(1.0 116/113 [PushOp(1:3cb06661:::obj109:head, version: 111'109, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cb06661:::obj109:head@111'109, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x555d860 con 0x4acb000 2026-02-11T23:31:27.576+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:3cb06661:::obj109:head, version: 111'109, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cb06661:::obj109:head@111'109, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.576+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3cb06661:::obj109:head, version: 111'109, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cb06661:::obj109:head@111'109, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.576+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.576+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3cb06661:::obj109:head, version: 111'109, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cb06661:::obj109:head@111'109, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.576+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.576+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 328 ==== pg_backfill(progress 1.0 e 116/116 lb 1:3bc05e5f:::obj343:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98000 con 0x4acb000 2026-02-11T23:31:27.576+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3cb06661:::obj109:head, version: 111'109, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cb06661:::obj109:head@111'109, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.576+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:3bc05e5f:::obj343:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 116 2026-02-11T23:31:27.576+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3bc05e5f:::obj343:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.576+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:3cb06661:::obj109:head, version: 111'109, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cb06661:::obj109:head@111'109, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 116/113 [PushOp(1:3cb06661:::obj109:head, version: 111'109, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cb06661:::obj109:head@111'109, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:3b0aa4c9:::obj511:head local-lis/les=113/114 n=145 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.576+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.576+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.576+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3b0aa4c9:::obj511:head local-lis/les=113/114 n=145 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:3cb06661:::obj109:head, version: 111'109, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cb06661:::obj109:head@111'109, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.576+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3bc05e5f:::obj343:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.576+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3b0aa4c9:::obj511:head local-lis/les=113/114 n=145 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:3cb06661:::obj109:head@111'109, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.576+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3b0aa4c9:::obj511:head local-lis/les=113/114 n=145 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:3cb06661:::obj109:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.576+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3b0aa4c9:::obj511:head local-lis/les=113/114 n=145 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:3cb06661:::obj109:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.576+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3b0aa4c9:::obj511:head local-lis/les=113/114 n=145 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:3cb06661:::obj109:head 2026-02-11T23:31:27.576+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3b0aa4c9:::obj511:head local-lis/les=113/114 n=145 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:3cb06661:::obj109:head v 111'109 2026-02-11T23:31:27.576+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3b0aa4c9:::obj511:head local-lis/les=113/114 n=145 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.576+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.576+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:3cb06661:::obj109:head, version: 111'109, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cb06661:::obj109:head@111'109, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:27.576+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3b0aa4c9:::obj511:head local-lis/les=113/114 n=145 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.576+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.576+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3bc05e5f:::obj343:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.576+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:3bc05e5f:::obj343:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 1.0 e 116/116 lb 1:3bc05e5f:::obj343:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:3b0aa4c9:::obj511:head local-lis/les=113/114 n=145 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.576+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.576+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.576+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3b0aa4c9:::obj511:head local-lis/les=113/114 n=145 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:3bc05e5f:::obj343:head) v3 2026-02-11T23:31:27.576+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3b0aa4c9:::obj511:head local-lis/les=113/114 n=145 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:3bc05e5f:::obj343:head) 2026-02-11T23:31:27.576+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.576+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:3bc05e5f:::obj343:head) v3 finish latency 0.000019 2026-02-11T23:31:27.576+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.576+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.576+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3bc05e5f:::obj343:head local-lis/les=113/114 n=146 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.576+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.576+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.576+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.576+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d79980 con 0x4acb000 2026-02-11T23:31:27.576+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:3cb06661:::obj109:head)]) -- 0x5d78d80 con 0x4acb000 2026-02-11T23:31:27.576+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.577+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 329 ==== MOSDPGPush(1.0 116/113 [PushOp(1:3cb2c87d:::obj96:head, version: 111'96, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cb2c87d:::obj96:head@111'96, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x555dd40 con 0x4acb000 2026-02-11T23:31:27.577+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:3cb2c87d:::obj96:head, version: 111'96, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cb2c87d:::obj96:head@111'96, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.577+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3cb2c87d:::obj96:head, version: 111'96, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cb2c87d:::obj96:head@111'96, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.577+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.577+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3cb2c87d:::obj96:head, version: 111'96, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cb2c87d:::obj96:head@111'96, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.577+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 330 ==== pg_backfill(progress 1.0 e 116/116 lb 1:3cb06661:::obj109:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98580 con 0x4acb000 2026-02-11T23:31:27.577+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.577+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:3cb06661:::obj109:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.577+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3cb2c87d:::obj96:head, version: 111'96, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cb2c87d:::obj96:head@111'96, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.577+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3cb06661:::obj109:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.577+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.577+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:3cb2c87d:::obj96:head, version: 111'96, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cb2c87d:::obj96:head@111'96, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 116/113 [PushOp(1:3cb2c87d:::obj96:head, version: 111'96, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cb2c87d:::obj96:head@111'96, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:3bc05e5f:::obj343:head local-lis/les=113/114 n=146 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.577+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3cb06661:::obj109:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.577+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.577+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3bc05e5f:::obj343:head local-lis/les=113/114 n=146 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:3cb2c87d:::obj96:head, version: 111'96, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cb2c87d:::obj96:head@111'96, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.577+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3bc05e5f:::obj343:head local-lis/les=113/114 n=146 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:3cb2c87d:::obj96:head@111'96, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.577+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3bc05e5f:::obj343:head local-lis/les=113/114 n=146 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:3cb2c87d:::obj96:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.577+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3bc05e5f:::obj343:head local-lis/les=113/114 n=146 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:3cb2c87d:::obj96:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.577+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3bc05e5f:::obj343:head local-lis/les=113/114 n=146 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:3cb2c87d:::obj96:head 2026-02-11T23:31:27.577+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3bc05e5f:::obj343:head local-lis/les=113/114 n=146 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:3cb2c87d:::obj96:head v 111'96 2026-02-11T23:31:27.577+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3bc05e5f:::obj343:head local-lis/les=113/114 n=146 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.577+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.577+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:3cb2c87d:::obj96:head, version: 111'96, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cb2c87d:::obj96:head@111'96, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:27.577+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.577+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.577+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3cb06661:::obj109:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.577+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:3cb06661:::obj109:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 1.0 e 116/116 lb 1:3cb06661:::obj109:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:3bc05e5f:::obj343:head local-lis/les=113/114 n=146 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.577+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.577+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3bc05e5f:::obj343:head local-lis/les=113/114 n=146 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:3cb06661:::obj109:head) v3 2026-02-11T23:31:27.577+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3bc05e5f:::obj343:head local-lis/les=113/114 n=146 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:3cb06661:::obj109:head) 2026-02-11T23:31:27.577+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.577+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:3cb06661:::obj109:head) v3 finish latency 0.000016 2026-02-11T23:31:27.577+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3cb06661:::obj109:head local-lis/les=113/114 n=147 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.577+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.577+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.577+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3cb06661:::obj109:head local-lis/les=113/114 n=147 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.577+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.577+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.577+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.577+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d78780 con 0x4acb000 2026-02-11T23:31:27.577+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:3cb2c87d:::obj96:head)]) -- 0x5df9500 con 0x4acb000 2026-02-11T23:31:27.577+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.577+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 331 ==== MOSDPGPush(1.0 116/113 [PushOp(1:3cc20580:::obj177:head, version: 111'177, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cc20580:::obj177:head@111'177, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x555cea0 con 0x4acb000 2026-02-11T23:31:27.577+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:3cc20580:::obj177:head, version: 111'177, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cc20580:::obj177:head@111'177, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.577+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3cc20580:::obj177:head, version: 111'177, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cc20580:::obj177:head@111'177, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.577+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.577+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3cc20580:::obj177:head, version: 111'177, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cc20580:::obj177:head@111'177, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.577+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 332 ==== pg_backfill(progress 1.0 e 116/116 lb 1:3cb2c87d:::obj96:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d98b00 con 0x4acb000 2026-02-11T23:31:27.577+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:3cb2c87d:::obj96:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.577+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.577+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3cc20580:::obj177:head, version: 111'177, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cc20580:::obj177:head@111'177, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.577+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3cb2c87d:::obj96:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.578+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.578+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3cb2c87d:::obj96:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.577+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:3cc20580:::obj177:head, version: 111'177, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cc20580:::obj177:head@111'177, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000043 MOSDPGPush(1.0 116/113 [PushOp(1:3cc20580:::obj177:head, version: 111'177, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cc20580:::obj177:head@111'177, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:3cb06661:::obj109:head local-lis/les=113/114 n=147 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.578+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.578+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3cb06661:::obj109:head local-lis/les=113/114 n=147 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:3cc20580:::obj177:head, version: 111'177, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cc20580:::obj177:head@111'177, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.578+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3cb06661:::obj109:head local-lis/les=113/114 n=147 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:3cc20580:::obj177:head@111'177, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.578+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3cb06661:::obj109:head local-lis/les=113/114 n=147 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:3cc20580:::obj177:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.578+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3cb06661:::obj109:head local-lis/les=113/114 n=147 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:3cc20580:::obj177:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.578+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3cb06661:::obj109:head local-lis/les=113/114 n=147 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:3cc20580:::obj177:head 2026-02-11T23:31:27.578+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3cb06661:::obj109:head local-lis/les=113/114 n=147 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:3cc20580:::obj177:head v 111'177 2026-02-11T23:31:27.578+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3cb06661:::obj109:head local-lis/les=113/114 n=147 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.578+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.578+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:3cc20580:::obj177:head, version: 111'177, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cc20580:::obj177:head@111'177, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000068 2026-02-11T23:31:27.578+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3cb06661:::obj109:head local-lis/les=113/114 n=147 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.578+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.578+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3cb2c87d:::obj96:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.578+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:3cb2c87d:::obj96:head) v3 prio 15 cost 0 latency 0.000087 pg_backfill(progress 1.0 e 116/116 lb 1:3cb2c87d:::obj96:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:3cb06661:::obj109:head local-lis/les=113/114 n=147 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.578+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.578+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.578+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3cb06661:::obj109:head local-lis/les=113/114 n=147 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:3cb2c87d:::obj96:head) v3 2026-02-11T23:31:27.578+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3cb06661:::obj109:head local-lis/les=113/114 n=147 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:3cb2c87d:::obj96:head) 2026-02-11T23:31:27.578+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.578+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:3cb2c87d:::obj96:head) v3 finish latency 0.000017 2026-02-11T23:31:27.578+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.578+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.578+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3cb2c87d:::obj96:head local-lis/les=113/114 n=148 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.578+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.578+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.578+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.578+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e0a180 con 0x4acb000 2026-02-11T23:31:27.578+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:3cc20580:::obj177:head)]) -- 0x5d79e00 con 0x4acb000 2026-02-11T23:31:27.578+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.578+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 333 ==== MOSDPGPush(1.0 116/113 [PushOp(1:3d7e2b4d:::obj143:head, version: 111'143, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d7e2b4d:::obj143:head@111'143, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x555d380 con 0x4acb000 2026-02-11T23:31:27.578+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:3d7e2b4d:::obj143:head, version: 111'143, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d7e2b4d:::obj143:head@111'143, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.578+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3d7e2b4d:::obj143:head, version: 111'143, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d7e2b4d:::obj143:head@111'143, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.578+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.578+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3d7e2b4d:::obj143:head, version: 111'143, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d7e2b4d:::obj143:head@111'143, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.578+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 334 ==== pg_backfill(progress 1.0 e 116/116 lb 1:3cc20580:::obj177:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99080 con 0x4acb000 2026-02-11T23:31:27.578+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.578+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:3cc20580:::obj177:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.578+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3d7e2b4d:::obj143:head, version: 111'143, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d7e2b4d:::obj143:head@111'143, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.578+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3cc20580:::obj177:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.578+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:3d7e2b4d:::obj143:head, version: 111'143, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d7e2b4d:::obj143:head@111'143, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 116/113 [PushOp(1:3d7e2b4d:::obj143:head, version: 111'143, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d7e2b4d:::obj143:head@111'143, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:3cb2c87d:::obj96:head local-lis/les=113/114 n=148 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.578+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.578+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.578+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3cb2c87d:::obj96:head local-lis/les=113/114 n=148 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:3d7e2b4d:::obj143:head, version: 111'143, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d7e2b4d:::obj143:head@111'143, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.578+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3cc20580:::obj177:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.578+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3cb2c87d:::obj96:head local-lis/les=113/114 n=148 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:3d7e2b4d:::obj143:head@111'143, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.578+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3cb2c87d:::obj96:head local-lis/les=113/114 n=148 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:3d7e2b4d:::obj143:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.578+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3cb2c87d:::obj96:head local-lis/les=113/114 n=148 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:3d7e2b4d:::obj143:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.578+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3cb2c87d:::obj96:head local-lis/les=113/114 n=148 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:3d7e2b4d:::obj143:head 2026-02-11T23:31:27.578+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3cb2c87d:::obj96:head local-lis/les=113/114 n=148 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:3d7e2b4d:::obj143:head v 111'143 2026-02-11T23:31:27.578+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3cb2c87d:::obj96:head local-lis/les=113/114 n=148 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.578+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.578+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:3d7e2b4d:::obj143:head, version: 111'143, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d7e2b4d:::obj143:head@111'143, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:27.578+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.578+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.578+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3cc20580:::obj177:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.578+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:3cc20580:::obj177:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 1.0 e 116/116 lb 1:3cc20580:::obj177:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:3cb2c87d:::obj96:head local-lis/les=113/114 n=148 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.578+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.578+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3cb2c87d:::obj96:head local-lis/les=113/114 n=148 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:3cc20580:::obj177:head) v3 2026-02-11T23:31:27.578+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3cb2c87d:::obj96:head local-lis/les=113/114 n=148 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:3cc20580:::obj177:head) 2026-02-11T23:31:27.578+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.578+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:3cc20580:::obj177:head) v3 finish latency 0.000018 2026-02-11T23:31:27.578+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3cc20580:::obj177:head local-lis/les=113/114 n=149 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.578+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.578+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.578+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3cc20580:::obj177:head local-lis/les=113/114 n=149 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.578+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.578+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.578+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.578+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e0a300 con 0x4acb000 2026-02-11T23:31:27.578+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:3d7e2b4d:::obj143:head)]) -- 0x5df9380 con 0x4acb000 2026-02-11T23:31:27.578+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.579+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 335 ==== MOSDPGPush(1.0 116/113 [PushOp(1:3d8750d8:::obj471:head, version: 111'471, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d8750d8:::obj471:head@111'471, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55e7040 con 0x4acb000 2026-02-11T23:31:27.579+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:3d8750d8:::obj471:head, version: 111'471, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d8750d8:::obj471:head@111'471, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.579+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3d8750d8:::obj471:head, version: 111'471, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d8750d8:::obj471:head@111'471, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.579+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.579+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 336 ==== pg_backfill(progress 1.0 e 116/116 lb 1:3d7e2b4d:::obj143:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99600 con 0x4acb000 2026-02-11T23:31:27.579+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3d8750d8:::obj471:head, version: 111'471, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d8750d8:::obj471:head@111'471, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.579+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:3d7e2b4d:::obj143:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 116 2026-02-11T23:31:27.579+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.579+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3d7e2b4d:::obj143:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.579+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3d8750d8:::obj471:head, version: 111'471, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d8750d8:::obj471:head@111'471, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.579+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.579+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:3d8750d8:::obj471:head, version: 111'471, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d8750d8:::obj471:head@111'471, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000041 MOSDPGPush(1.0 116/113 [PushOp(1:3d8750d8:::obj471:head, version: 111'471, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d8750d8:::obj471:head@111'471, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:3cc20580:::obj177:head local-lis/les=113/114 n=149 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.579+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.579+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3cc20580:::obj177:head local-lis/les=113/114 n=149 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:3d8750d8:::obj471:head, version: 111'471, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d8750d8:::obj471:head@111'471, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.579+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.579+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3d7e2b4d:::obj143:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.579+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3cc20580:::obj177:head local-lis/les=113/114 n=149 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:3d8750d8:::obj471:head@111'471, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.579+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3cc20580:::obj177:head local-lis/les=113/114 n=149 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:3d8750d8:::obj471:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.579+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3cc20580:::obj177:head local-lis/les=113/114 n=149 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:3d8750d8:::obj471:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.579+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3cc20580:::obj177:head local-lis/les=113/114 n=149 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:3d8750d8:::obj471:head 2026-02-11T23:31:27.579+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3cc20580:::obj177:head local-lis/les=113/114 n=149 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:3d8750d8:::obj471:head v 111'471 2026-02-11T23:31:27.579+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3cc20580:::obj177:head local-lis/les=113/114 n=149 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.579+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.579+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:3d8750d8:::obj471:head, version: 111'471, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d8750d8:::obj471:head@111'471, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:27.579+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.579+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.579+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3d7e2b4d:::obj143:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.579+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:3d7e2b4d:::obj143:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 1.0 e 116/116 lb 1:3d7e2b4d:::obj143:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:3cc20580:::obj177:head local-lis/les=113/114 n=149 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.579+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.579+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3cc20580:::obj177:head local-lis/les=113/114 n=149 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:3d7e2b4d:::obj143:head) v3 2026-02-11T23:31:27.579+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3cc20580:::obj177:head local-lis/les=113/114 n=149 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:3d7e2b4d:::obj143:head) 2026-02-11T23:31:27.579+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.579+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:3d7e2b4d:::obj143:head) v3 finish latency 0.000015 2026-02-11T23:31:27.579+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3d7e2b4d:::obj143:head local-lis/les=113/114 n=150 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.579+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.579+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.579+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3d7e2b4d:::obj143:head local-lis/les=113/114 n=150 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.579+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.579+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.579+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.579+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e0a480 con 0x4acb000 2026-02-11T23:31:27.579+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:3d8750d8:::obj471:head)]) -- 0x5e6b980 con 0x4acb000 2026-02-11T23:31:27.579+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.579+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 337 ==== MOSDPGPush(1.0 116/113 [PushOp(1:3d88a4f9:::obj551:head, version: 111'551, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d88a4f9:::obj551:head@111'551, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55e7860 con 0x4acb000 2026-02-11T23:31:27.579+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:3d88a4f9:::obj551:head, version: 111'551, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d88a4f9:::obj551:head@111'551, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.579+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3d88a4f9:::obj551:head, version: 111'551, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d88a4f9:::obj551:head@111'551, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.579+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.579+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3d88a4f9:::obj551:head, version: 111'551, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d88a4f9:::obj551:head@111'551, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.579+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 338 ==== pg_backfill(progress 1.0 e 116/116 lb 1:3d8750d8:::obj471:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99b80 con 0x4acb000 2026-02-11T23:31:27.579+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:3d8750d8:::obj471:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.579+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.579+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3d8750d8:::obj471:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.579+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3d88a4f9:::obj551:head, version: 111'551, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d88a4f9:::obj551:head@111'551, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.579+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.579+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:3d88a4f9:::obj551:head, version: 111'551, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d88a4f9:::obj551:head@111'551, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 116/113 [PushOp(1:3d88a4f9:::obj551:head, version: 111'551, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d88a4f9:::obj551:head@111'551, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:3d7e2b4d:::obj143:head local-lis/les=113/114 n=150 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.579+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.579+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3d7e2b4d:::obj143:head local-lis/les=113/114 n=150 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:3d88a4f9:::obj551:head, version: 111'551, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d88a4f9:::obj551:head@111'551, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.579+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3d7e2b4d:::obj143:head local-lis/les=113/114 n=150 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:3d88a4f9:::obj551:head@111'551, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.579+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.579+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3d8750d8:::obj471:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.579+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3d7e2b4d:::obj143:head local-lis/les=113/114 n=150 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:3d88a4f9:::obj551:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.579+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3d7e2b4d:::obj143:head local-lis/les=113/114 n=150 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:3d88a4f9:::obj551:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.579+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3d7e2b4d:::obj143:head local-lis/les=113/114 n=150 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:3d88a4f9:::obj551:head 2026-02-11T23:31:27.579+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3d7e2b4d:::obj143:head local-lis/les=113/114 n=150 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:3d88a4f9:::obj551:head v 111'551 2026-02-11T23:31:27.579+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3d7e2b4d:::obj143:head local-lis/les=113/114 n=150 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.579+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.579+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:3d88a4f9:::obj551:head, version: 111'551, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d88a4f9:::obj551:head@111'551, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:27.579+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3d7e2b4d:::obj143:head local-lis/les=113/114 n=150 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.579+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.579+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.579+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3d8750d8:::obj471:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.579+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:3d8750d8:::obj471:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 1.0 e 116/116 lb 1:3d8750d8:::obj471:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:3d7e2b4d:::obj143:head local-lis/les=113/114 n=150 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.579+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.579+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3d7e2b4d:::obj143:head local-lis/les=113/114 n=150 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:3d8750d8:::obj471:head) v3 2026-02-11T23:31:27.579+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3d7e2b4d:::obj143:head local-lis/les=113/114 n=150 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:3d8750d8:::obj471:head) 2026-02-11T23:31:27.579+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.579+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:3d8750d8:::obj471:head) v3 finish latency 0.000015 2026-02-11T23:31:27.579+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.580+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.580+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3d8750d8:::obj471:head local-lis/les=113/114 n=151 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.580+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.580+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.580+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.580+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e0a900 con 0x4acb000 2026-02-11T23:31:27.580+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:3d88a4f9:::obj551:head)]) -- 0x5e0a600 con 0x4acb000 2026-02-11T23:31:27.580+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.580+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 339 ==== MOSDPGPush(1.0 116/113 [PushOp(1:3e07a120:::obj540:head, version: 111'540, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3e07a120:::obj540:head@111'540, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55e7d40 con 0x4acb000 2026-02-11T23:31:27.580+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:3e07a120:::obj540:head, version: 111'540, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3e07a120:::obj540:head@111'540, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.580+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3e07a120:::obj540:head, version: 111'540, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3e07a120:::obj540:head@111'540, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.580+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.580+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 340 ==== pg_backfill(progress 1.0 e 116/116 lb 1:3d88a4f9:::obj551:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9a100 con 0x4acb000 2026-02-11T23:31:27.580+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3e07a120:::obj540:head, version: 111'540, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3e07a120:::obj540:head@111'540, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.580+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:3d88a4f9:::obj551:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.580+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.580+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3d88a4f9:::obj551:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.580+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3e07a120:::obj540:head, version: 111'540, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3e07a120:::obj540:head@111'540, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.580+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.580+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:3e07a120:::obj540:head, version: 111'540, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3e07a120:::obj540:head@111'540, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(1.0 116/113 [PushOp(1:3e07a120:::obj540:head, version: 111'540, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3e07a120:::obj540:head@111'540, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:3d8750d8:::obj471:head local-lis/les=113/114 n=151 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.580+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.580+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3d8750d8:::obj471:head local-lis/les=113/114 n=151 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:3e07a120:::obj540:head, version: 111'540, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3e07a120:::obj540:head@111'540, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.580+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3d8750d8:::obj471:head local-lis/les=113/114 n=151 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:3e07a120:::obj540:head@111'540, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.580+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.580+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3d88a4f9:::obj551:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.580+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3d8750d8:::obj471:head local-lis/les=113/114 n=151 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:3e07a120:::obj540:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.580+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3d8750d8:::obj471:head local-lis/les=113/114 n=151 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:3e07a120:::obj540:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.580+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3d8750d8:::obj471:head local-lis/les=113/114 n=151 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:3e07a120:::obj540:head 2026-02-11T23:31:27.580+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3d8750d8:::obj471:head local-lis/les=113/114 n=151 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:3e07a120:::obj540:head v 111'540 2026-02-11T23:31:27.580+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3d8750d8:::obj471:head local-lis/les=113/114 n=151 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.580+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.580+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:3e07a120:::obj540:head, version: 111'540, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3e07a120:::obj540:head@111'540, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:27.580+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.580+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.580+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3d88a4f9:::obj551:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.580+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:3d88a4f9:::obj551:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 1.0 e 116/116 lb 1:3d88a4f9:::obj551:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:3d8750d8:::obj471:head local-lis/les=113/114 n=151 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.580+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.580+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3d8750d8:::obj471:head local-lis/les=113/114 n=151 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:3d88a4f9:::obj551:head) v3 2026-02-11T23:31:27.580+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3d8750d8:::obj471:head local-lis/les=113/114 n=151 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:3d88a4f9:::obj551:head) 2026-02-11T23:31:27.580+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.580+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:3d88a4f9:::obj551:head) v3 finish latency 0.000015 2026-02-11T23:31:27.580+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3d88a4f9:::obj551:head local-lis/les=113/114 n=152 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.580+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.580+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.580+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3d88a4f9:::obj551:head local-lis/les=113/114 n=152 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.580+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.580+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.580+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.580+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e0aa80 con 0x4acb000 2026-02-11T23:31:27.580+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:3e07a120:::obj540:head)]) -- 0x5e7ed80 con 0x4acb000 2026-02-11T23:31:27.580+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.581+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 341 ==== MOSDPGPush(1.0 116/113 [PushOp(1:3ed3e33d:::obj376:head, version: 111'376, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3ed3e33d:::obj376:head@111'376, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55e6b60 con 0x4acb000 2026-02-11T23:31:27.581+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:3ed3e33d:::obj376:head, version: 111'376, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3ed3e33d:::obj376:head@111'376, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 116 2026-02-11T23:31:27.581+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3ed3e33d:::obj376:head, version: 111'376, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3ed3e33d:::obj376:head@111'376, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.581+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.581+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3ed3e33d:::obj376:head, version: 111'376, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3ed3e33d:::obj376:head@111'376, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.581+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 342 ==== pg_backfill(progress 1.0 e 116/116 lb 1:3e07a120:::obj540:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9a680 con 0x4acb000 2026-02-11T23:31:27.581+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.581+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:3e07a120:::obj540:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.581+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3ed3e33d:::obj376:head, version: 111'376, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3ed3e33d:::obj376:head@111'376, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.581+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3e07a120:::obj540:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.581+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.581+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:3ed3e33d:::obj376:head, version: 111'376, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3ed3e33d:::obj376:head@111'376, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 116/113 [PushOp(1:3ed3e33d:::obj376:head, version: 111'376, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3ed3e33d:::obj376:head@111'376, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:3d88a4f9:::obj551:head local-lis/les=113/114 n=152 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.581+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3e07a120:::obj540:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.581+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.581+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3d88a4f9:::obj551:head local-lis/les=113/114 n=152 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:3ed3e33d:::obj376:head, version: 111'376, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3ed3e33d:::obj376:head@111'376, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.581+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3d88a4f9:::obj551:head local-lis/les=113/114 n=152 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:3ed3e33d:::obj376:head@111'376, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.581+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3d88a4f9:::obj551:head local-lis/les=113/114 n=152 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:3ed3e33d:::obj376:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.581+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3d88a4f9:::obj551:head local-lis/les=113/114 n=152 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:3ed3e33d:::obj376:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.581+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3d88a4f9:::obj551:head local-lis/les=113/114 n=152 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:3ed3e33d:::obj376:head 2026-02-11T23:31:27.581+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3d88a4f9:::obj551:head local-lis/les=113/114 n=152 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:3ed3e33d:::obj376:head v 111'376 2026-02-11T23:31:27.581+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3d88a4f9:::obj551:head local-lis/les=113/114 n=152 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.581+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.581+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:3ed3e33d:::obj376:head, version: 111'376, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3ed3e33d:::obj376:head@111'376, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:27.581+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3d88a4f9:::obj551:head local-lis/les=113/114 n=152 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.581+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.581+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3e07a120:::obj540:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.581+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:3e07a120:::obj540:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 1.0 e 116/116 lb 1:3e07a120:::obj540:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:3d88a4f9:::obj551:head local-lis/les=113/114 n=152 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.581+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.581+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.581+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3d88a4f9:::obj551:head local-lis/les=113/114 n=152 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:3e07a120:::obj540:head) v3 2026-02-11T23:31:27.581+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3d88a4f9:::obj551:head local-lis/les=113/114 n=152 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:3e07a120:::obj540:head) 2026-02-11T23:31:27.581+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.581+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:3e07a120:::obj540:head) v3 finish latency 0.000017 2026-02-11T23:31:27.581+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.581+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.581+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3e07a120:::obj540:head local-lis/les=113/114 n=153 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.581+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.581+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.581+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.581+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e0af00 con 0x4acb000 2026-02-11T23:31:27.581+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:3ed3e33d:::obj376:head)]) -- 0x5e0ac00 con 0x4acb000 2026-02-11T23:31:27.581+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.581+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 343 ==== MOSDPGPush(1.0 116/113 [PushOp(1:3efbd745:::obj84:head, version: 111'84, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3efbd745:::obj84:head@111'84, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5625380 con 0x4acb000 2026-02-11T23:31:27.581+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:3efbd745:::obj84:head, version: 111'84, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3efbd745:::obj84:head@111'84, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.581+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3efbd745:::obj84:head, version: 111'84, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3efbd745:::obj84:head@111'84, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.581+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.581+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 344 ==== pg_backfill(progress 1.0 e 116/116 lb 1:3ed3e33d:::obj376:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9ac00 con 0x4acb000 2026-02-11T23:31:27.581+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3efbd745:::obj84:head, version: 111'84, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3efbd745:::obj84:head@111'84, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.581+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:3ed3e33d:::obj376:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.581+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.581+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3ed3e33d:::obj376:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.581+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3efbd745:::obj84:head, version: 111'84, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3efbd745:::obj84:head@111'84, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.581+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.581+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:3efbd745:::obj84:head, version: 111'84, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3efbd745:::obj84:head@111'84, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 116/113 [PushOp(1:3efbd745:::obj84:head, version: 111'84, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3efbd745:::obj84:head@111'84, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:3e07a120:::obj540:head local-lis/les=113/114 n=153 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.581+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.581+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3e07a120:::obj540:head local-lis/les=113/114 n=153 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:3efbd745:::obj84:head, version: 111'84, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3efbd745:::obj84:head@111'84, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.581+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3e07a120:::obj540:head local-lis/les=113/114 n=153 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:3efbd745:::obj84:head@111'84, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.581+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.581+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3ed3e33d:::obj376:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.581+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3e07a120:::obj540:head local-lis/les=113/114 n=153 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:3efbd745:::obj84:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.581+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3e07a120:::obj540:head local-lis/les=113/114 n=153 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:3efbd745:::obj84:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.581+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3e07a120:::obj540:head local-lis/les=113/114 n=153 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:3efbd745:::obj84:head 2026-02-11T23:31:27.581+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3e07a120:::obj540:head local-lis/les=113/114 n=153 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:3efbd745:::obj84:head v 111'84 2026-02-11T23:31:27.581+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3e07a120:::obj540:head local-lis/les=113/114 n=153 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.581+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.581+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:3efbd745:::obj84:head, version: 111'84, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3efbd745:::obj84:head@111'84, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:27.581+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.581+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.581+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3ed3e33d:::obj376:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.581+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:3ed3e33d:::obj376:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 1.0 e 116/116 lb 1:3ed3e33d:::obj376:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:3e07a120:::obj540:head local-lis/les=113/114 n=153 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.581+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.581+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3e07a120:::obj540:head local-lis/les=113/114 n=153 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:3ed3e33d:::obj376:head) v3 2026-02-11T23:31:27.581+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3e07a120:::obj540:head local-lis/les=113/114 n=153 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:3ed3e33d:::obj376:head) 2026-02-11T23:31:27.581+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.581+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:3ed3e33d:::obj376:head) v3 finish latency 0.000015 2026-02-11T23:31:27.581+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3ed3e33d:::obj376:head local-lis/les=113/114 n=154 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.581+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.581+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.581+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3ed3e33d:::obj376:head local-lis/les=113/114 n=154 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.581+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.581+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.581+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.581+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e0b080 con 0x4acb000 2026-02-11T23:31:27.581+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:3efbd745:::obj84:head)]) -- 0x5daa000 con 0x4acb000 2026-02-11T23:31:27.581+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.582+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 345 ==== MOSDPGPush(1.0 116/113 [PushOp(1:3fc284d5:::obj140:head, version: 111'140, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3fc284d5:::obj140:head@111'140, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5625860 con 0x4acb000 2026-02-11T23:31:27.582+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:3fc284d5:::obj140:head, version: 111'140, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3fc284d5:::obj140:head@111'140, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.582+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3fc284d5:::obj140:head, version: 111'140, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3fc284d5:::obj140:head@111'140, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.582+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.582+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3fc284d5:::obj140:head, version: 111'140, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3fc284d5:::obj140:head@111'140, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.582+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 346 ==== pg_backfill(progress 1.0 e 116/116 lb 1:3efbd745:::obj84:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d9b180 con 0x4acb000 2026-02-11T23:31:27.582+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.582+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:3efbd745:::obj84:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.582+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3fc284d5:::obj140:head, version: 111'140, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3fc284d5:::obj140:head@111'140, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.582+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3efbd745:::obj84:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.582+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.582+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:3fc284d5:::obj140:head, version: 111'140, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3fc284d5:::obj140:head@111'140, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:3fc284d5:::obj140:head, version: 111'140, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3fc284d5:::obj140:head@111'140, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:3ed3e33d:::obj376:head local-lis/les=113/114 n=154 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.582+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3efbd745:::obj84:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.582+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.582+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3ed3e33d:::obj376:head local-lis/les=113/114 n=154 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:3fc284d5:::obj140:head, version: 111'140, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3fc284d5:::obj140:head@111'140, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.582+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3ed3e33d:::obj376:head local-lis/les=113/114 n=154 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:3fc284d5:::obj140:head@111'140, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.582+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3ed3e33d:::obj376:head local-lis/les=113/114 n=154 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:3fc284d5:::obj140:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.582+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3ed3e33d:::obj376:head local-lis/les=113/114 n=154 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:3fc284d5:::obj140:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.582+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3ed3e33d:::obj376:head local-lis/les=113/114 n=154 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:3fc284d5:::obj140:head 2026-02-11T23:31:27.582+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3ed3e33d:::obj376:head local-lis/les=113/114 n=154 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:3fc284d5:::obj140:head v 111'140 2026-02-11T23:31:27.582+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3ed3e33d:::obj376:head local-lis/les=113/114 n=154 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.582+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.582+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:3fc284d5:::obj140:head, version: 111'140, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3fc284d5:::obj140:head@111'140, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:27.582+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3ed3e33d:::obj376:head local-lis/les=113/114 n=154 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.582+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.582+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3efbd745:::obj84:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.582+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:3efbd745:::obj84:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 1.0 e 116/116 lb 1:3efbd745:::obj84:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:3ed3e33d:::obj376:head local-lis/les=113/114 n=154 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.582+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.582+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.582+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3ed3e33d:::obj376:head local-lis/les=113/114 n=154 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:3efbd745:::obj84:head) v3 2026-02-11T23:31:27.582+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3ed3e33d:::obj376:head local-lis/les=113/114 n=154 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:3efbd745:::obj84:head) 2026-02-11T23:31:27.582+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.582+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:3efbd745:::obj84:head) v3 finish latency 0.000020 2026-02-11T23:31:27.582+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.582+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.582+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3efbd745:::obj84:head local-lis/les=113/114 n=155 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.582+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.582+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.582+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.582+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e0b500 con 0x4acb000 2026-02-11T23:31:27.582+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:3fc284d5:::obj140:head)]) -- 0x5e0b200 con 0x4acb000 2026-02-11T23:31:27.582+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.582+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 347 ==== MOSDPGPush(1.0 116/113 [PushOp(1:3fe0a5f5:::obj46:head, version: 111'46, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3fe0a5f5:::obj46:head@111'46, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5697860 con 0x4acb000 2026-02-11T23:31:27.582+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:3fe0a5f5:::obj46:head, version: 111'46, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3fe0a5f5:::obj46:head@111'46, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 116 2026-02-11T23:31:27.582+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3fe0a5f5:::obj46:head, version: 111'46, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3fe0a5f5:::obj46:head@111'46, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.582+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.582+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 348 ==== pg_backfill(progress 1.0 e 116/116 lb 1:3fc284d5:::obj140:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9b700 con 0x4acb000 2026-02-11T23:31:27.582+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3fe0a5f5:::obj46:head, version: 111'46, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3fe0a5f5:::obj46:head@111'46, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.582+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:3fc284d5:::obj140:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 116 2026-02-11T23:31:27.582+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3fc284d5:::obj140:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.582+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.582+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:3fe0a5f5:::obj46:head, version: 111'46, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3fe0a5f5:::obj46:head@111'46, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.582+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:3fe0a5f5:::obj46:head, version: 111'46, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3fe0a5f5:::obj46:head@111'46, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000042 MOSDPGPush(1.0 116/113 [PushOp(1:3fe0a5f5:::obj46:head, version: 111'46, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3fe0a5f5:::obj46:head@111'46, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:3efbd745:::obj84:head local-lis/les=113/114 n=155 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.582+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.582+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.582+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3efbd745:::obj84:head local-lis/les=113/114 n=155 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:3fe0a5f5:::obj46:head, version: 111'46, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3fe0a5f5:::obj46:head@111'46, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.582+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3fc284d5:::obj140:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.582+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3efbd745:::obj84:head local-lis/les=113/114 n=155 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:3fe0a5f5:::obj46:head@111'46, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.582+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3efbd745:::obj84:head local-lis/les=113/114 n=155 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:3fe0a5f5:::obj46:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.582+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3efbd745:::obj84:head local-lis/les=113/114 n=155 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:3fe0a5f5:::obj46:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.582+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3efbd745:::obj84:head local-lis/les=113/114 n=155 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:3fe0a5f5:::obj46:head 2026-02-11T23:31:27.582+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3efbd745:::obj84:head local-lis/les=113/114 n=155 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:3fe0a5f5:::obj46:head v 111'46 2026-02-11T23:31:27.582+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3efbd745:::obj84:head local-lis/les=113/114 n=155 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.582+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.582+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:3fe0a5f5:::obj46:head, version: 111'46, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3fe0a5f5:::obj46:head@111'46, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:27.582+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.582+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.582+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3fc284d5:::obj140:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.582+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:3fc284d5:::obj140:head) v3 prio 15 cost 0 latency 0.000082 pg_backfill(progress 1.0 e 116/116 lb 1:3fc284d5:::obj140:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:3efbd745:::obj84:head local-lis/les=113/114 n=155 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.583+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.583+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3efbd745:::obj84:head local-lis/les=113/114 n=155 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:3fc284d5:::obj140:head) v3 2026-02-11T23:31:27.583+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3efbd745:::obj84:head local-lis/les=113/114 n=155 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:3fc284d5:::obj140:head) 2026-02-11T23:31:27.583+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.583+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:3fc284d5:::obj140:head) v3 finish latency 0.000040 2026-02-11T23:31:27.583+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3fc284d5:::obj140:head local-lis/les=113/114 n=156 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.583+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.583+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.583+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3fc284d5:::obj140:head local-lis/les=113/114 n=156 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.583+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.583+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.583+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.583+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e0b680 con 0x4acb000 2026-02-11T23:31:27.583+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:3fe0a5f5:::obj46:head)]) -- 0x5daa180 con 0x4acb000 2026-02-11T23:31:27.583+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.583+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.583+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.583+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 349 ==== MOSDPGPush(1.0 116/113 [PushOp(1:401e0147:::obj213:head, version: 111'213, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:401e0147:::obj213:head@111'213, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5697d40 con 0x4acb000 2026-02-11T23:31:27.583+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:401e0147:::obj213:head, version: 111'213, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:401e0147:::obj213:head@111'213, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 116 2026-02-11T23:31:27.583+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:401e0147:::obj213:head, version: 111'213, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:401e0147:::obj213:head@111'213, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.583+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.583+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:401e0147:::obj213:head, version: 111'213, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:401e0147:::obj213:head@111'213, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.583+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 350 ==== pg_backfill(progress 1.0 e 116/116 lb 1:3fe0a5f5:::obj46:head) ==== 1061+0+0 (unknown 0 0 0) 0x5dc4100 con 0x4acb000 2026-02-11T23:31:27.583+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.583+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:3fe0a5f5:::obj46:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 116 2026-02-11T23:31:27.583+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:401e0147:::obj213:head, version: 111'213, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:401e0147:::obj213:head@111'213, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.583+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3fe0a5f5:::obj46:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.583+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:401e0147:::obj213:head, version: 111'213, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:401e0147:::obj213:head@111'213, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(1.0 116/113 [PushOp(1:401e0147:::obj213:head, version: 111'213, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:401e0147:::obj213:head@111'213, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:3fc284d5:::obj140:head local-lis/les=113/114 n=156 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.583+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.583+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.583+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3fe0a5f5:::obj46:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.583+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3fc284d5:::obj140:head local-lis/les=113/114 n=156 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:401e0147:::obj213:head, version: 111'213, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:401e0147:::obj213:head@111'213, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.583+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3fc284d5:::obj140:head local-lis/les=113/114 n=156 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:401e0147:::obj213:head@111'213, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.583+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3fc284d5:::obj140:head local-lis/les=113/114 n=156 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:401e0147:::obj213:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.583+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3fc284d5:::obj140:head local-lis/les=113/114 n=156 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:401e0147:::obj213:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.583+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3fc284d5:::obj140:head local-lis/les=113/114 n=156 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:401e0147:::obj213:head 2026-02-11T23:31:27.583+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3fc284d5:::obj140:head local-lis/les=113/114 n=156 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:401e0147:::obj213:head v 111'213 2026-02-11T23:31:27.583+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3fc284d5:::obj140:head local-lis/les=113/114 n=156 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.583+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.583+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:401e0147:::obj213:head, version: 111'213, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:401e0147:::obj213:head@111'213, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:27.583+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.583+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.583+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:3fe0a5f5:::obj46:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.583+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:3fe0a5f5:::obj46:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 1.0 e 116/116 lb 1:3fe0a5f5:::obj46:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:3fc284d5:::obj140:head local-lis/les=113/114 n=156 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.583+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.583+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3fc284d5:::obj140:head local-lis/les=113/114 n=156 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:3fe0a5f5:::obj46:head) v3 2026-02-11T23:31:27.583+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3fc284d5:::obj140:head local-lis/les=113/114 n=156 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:3fe0a5f5:::obj46:head) 2026-02-11T23:31:27.583+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.583+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:3fe0a5f5:::obj46:head) v3 finish latency 0.000033 2026-02-11T23:31:27.583+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3fe0a5f5:::obj46:head local-lis/les=113/114 n=157 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.583+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.583+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.583+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3fe0a5f5:::obj46:head local-lis/les=113/114 n=157 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.583+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.583+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.583+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.583+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e0b800 con 0x4acb000 2026-02-11T23:31:27.583+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:401e0147:::obj213:head)]) -- 0x5daa300 con 0x4acb000 2026-02-11T23:31:27.583+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.583+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.583+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.584+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 351 ==== MOSDPGPush(1.0 116/113 [PushOp(1:401e941b:::obj398:head, version: 111'398, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:401e941b:::obj398:head@111'398, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x49d0d00 con 0x4acb000 2026-02-11T23:31:27.584+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:401e941b:::obj398:head, version: 111'398, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:401e941b:::obj398:head@111'398, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.584+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:401e941b:::obj398:head, version: 111'398, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:401e941b:::obj398:head@111'398, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.584+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.584+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 352 ==== pg_backfill(progress 1.0 e 116/116 lb 1:401e0147:::obj213:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc2b00 con 0x4acb000 2026-02-11T23:31:27.584+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:401e0147:::obj213:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.584+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:401e941b:::obj398:head, version: 111'398, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:401e941b:::obj398:head@111'398, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.584+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:401e0147:::obj213:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.584+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.584+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:401e941b:::obj398:head, version: 111'398, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:401e941b:::obj398:head@111'398, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.584+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:401e941b:::obj398:head, version: 111'398, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:401e941b:::obj398:head@111'398, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 116/113 [PushOp(1:401e941b:::obj398:head, version: 111'398, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:401e941b:::obj398:head@111'398, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:3fe0a5f5:::obj46:head local-lis/les=113/114 n=157 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.584+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.584+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.584+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3fe0a5f5:::obj46:head local-lis/les=113/114 n=157 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:401e941b:::obj398:head, version: 111'398, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:401e941b:::obj398:head@111'398, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.584+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:401e0147:::obj213:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.584+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3fe0a5f5:::obj46:head local-lis/les=113/114 n=157 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:401e941b:::obj398:head@111'398, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.584+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3fe0a5f5:::obj46:head local-lis/les=113/114 n=157 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:401e941b:::obj398:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.584+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3fe0a5f5:::obj46:head local-lis/les=113/114 n=157 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:401e941b:::obj398:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.584+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3fe0a5f5:::obj46:head local-lis/les=113/114 n=157 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:401e941b:::obj398:head 2026-02-11T23:31:27.584+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3fe0a5f5:::obj46:head local-lis/les=113/114 n=157 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:401e941b:::obj398:head v 111'398 2026-02-11T23:31:27.584+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3fe0a5f5:::obj46:head local-lis/les=113/114 n=157 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.584+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.584+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:401e941b:::obj398:head, version: 111'398, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:401e941b:::obj398:head@111'398, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:27.584+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.584+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.584+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:401e0147:::obj213:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.584+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:401e0147:::obj213:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 1.0 e 116/116 lb 1:401e0147:::obj213:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:3fe0a5f5:::obj46:head local-lis/les=113/114 n=157 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.584+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.584+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3fe0a5f5:::obj46:head local-lis/les=113/114 n=157 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:401e0147:::obj213:head) v3 2026-02-11T23:31:27.584+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:3fe0a5f5:::obj46:head local-lis/les=113/114 n=157 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:401e0147:::obj213:head) 2026-02-11T23:31:27.584+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.584+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:401e0147:::obj213:head) v3 finish latency 0.000011 2026-02-11T23:31:27.584+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:401e0147:::obj213:head local-lis/les=113/114 n=158 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.584+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.584+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.584+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:401e0147:::obj213:head local-lis/les=113/114 n=158 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.584+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.584+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.584+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.584+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e0b980 con 0x4acb000 2026-02-11T23:31:27.584+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:401e941b:::obj398:head)]) -- 0x5daa600 con 0x4acb000 2026-02-11T23:31:27.584+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.584+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 353 ==== MOSDPGPush(1.0 116/113 [PushOp(1:40d1abdf:::obj254:head, version: 111'254, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:40d1abdf:::obj254:head@111'254, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x49d1a00 con 0x4acb000 2026-02-11T23:31:27.584+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:40d1abdf:::obj254:head, version: 111'254, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:40d1abdf:::obj254:head@111'254, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.584+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:40d1abdf:::obj254:head, version: 111'254, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:40d1abdf:::obj254:head@111'254, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.584+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.584+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 354 ==== pg_backfill(progress 1.0 e 116/116 lb 1:401e941b:::obj398:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc5700 con 0x4acb000 2026-02-11T23:31:27.584+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:401e941b:::obj398:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.584+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:40d1abdf:::obj254:head, version: 111'254, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:40d1abdf:::obj254:head@111'254, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.584+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:401e941b:::obj398:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.584+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.584+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:40d1abdf:::obj254:head, version: 111'254, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:40d1abdf:::obj254:head@111'254, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.584+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:40d1abdf:::obj254:head, version: 111'254, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:40d1abdf:::obj254:head@111'254, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 116/113 [PushOp(1:40d1abdf:::obj254:head, version: 111'254, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:40d1abdf:::obj254:head@111'254, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:401e0147:::obj213:head local-lis/les=113/114 n=158 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.584+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.584+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.584+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:401e0147:::obj213:head local-lis/les=113/114 n=158 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:40d1abdf:::obj254:head, version: 111'254, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:40d1abdf:::obj254:head@111'254, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.584+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:401e941b:::obj398:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.584+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:401e0147:::obj213:head local-lis/les=113/114 n=158 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:40d1abdf:::obj254:head@111'254, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.584+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:401e0147:::obj213:head local-lis/les=113/114 n=158 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:40d1abdf:::obj254:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.584+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:401e0147:::obj213:head local-lis/les=113/114 n=158 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:40d1abdf:::obj254:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.584+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:401e0147:::obj213:head local-lis/les=113/114 n=158 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:40d1abdf:::obj254:head 2026-02-11T23:31:27.584+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:401e0147:::obj213:head local-lis/les=113/114 n=158 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:40d1abdf:::obj254:head v 111'254 2026-02-11T23:31:27.584+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:401e0147:::obj213:head local-lis/les=113/114 n=158 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.584+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.584+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:40d1abdf:::obj254:head, version: 111'254, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:40d1abdf:::obj254:head@111'254, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:27.584+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.584+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.584+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:401e941b:::obj398:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.584+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:401e941b:::obj398:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 1.0 e 116/116 lb 1:401e941b:::obj398:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:401e0147:::obj213:head local-lis/les=113/114 n=158 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.584+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.584+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:401e0147:::obj213:head local-lis/les=113/114 n=158 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:401e941b:::obj398:head) v3 2026-02-11T23:31:27.584+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:401e0147:::obj213:head local-lis/les=113/114 n=158 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:401e941b:::obj398:head) 2026-02-11T23:31:27.584+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.584+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:401e941b:::obj398:head) v3 finish latency 0.000009 2026-02-11T23:31:27.584+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:401e941b:::obj398:head local-lis/les=113/114 n=159 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.584+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.585+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.585+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:401e941b:::obj398:head local-lis/les=113/114 n=159 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.585+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.585+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.585+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.585+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e0bb00 con 0x4acb000 2026-02-11T23:31:27.585+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:40d1abdf:::obj254:head)]) -- 0x5daa900 con 0x4acb000 2026-02-11T23:31:27.585+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.585+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 355 ==== MOSDPGPush(1.0 116/113 [PushOp(1:41892471:::obj524:head, version: 111'524, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:41892471:::obj524:head@111'524, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x49d1520 con 0x4acb000 2026-02-11T23:31:27.585+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:41892471:::obj524:head, version: 111'524, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:41892471:::obj524:head@111'524, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.585+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:41892471:::obj524:head, version: 111'524, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:41892471:::obj524:head@111'524, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.585+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.585+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 356 ==== pg_backfill(progress 1.0 e 116/116 lb 1:40d1abdf:::obj254:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc2580 con 0x4acb000 2026-02-11T23:31:27.585+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:40d1abdf:::obj254:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.585+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:41892471:::obj524:head, version: 111'524, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:41892471:::obj524:head@111'524, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.585+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:40d1abdf:::obj254:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.585+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.585+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:41892471:::obj524:head, version: 111'524, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:41892471:::obj524:head@111'524, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.585+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.585+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:41892471:::obj524:head, version: 111'524, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:41892471:::obj524:head@111'524, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:41892471:::obj524:head, version: 111'524, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:41892471:::obj524:head@111'524, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:401e941b:::obj398:head local-lis/les=113/114 n=159 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.585+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.585+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:401e941b:::obj398:head local-lis/les=113/114 n=159 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:41892471:::obj524:head, version: 111'524, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:41892471:::obj524:head@111'524, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.585+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:401e941b:::obj398:head local-lis/les=113/114 n=159 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:41892471:::obj524:head@111'524, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.585+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.585+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:40d1abdf:::obj254:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.585+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:401e941b:::obj398:head local-lis/les=113/114 n=159 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:41892471:::obj524:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.585+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:401e941b:::obj398:head local-lis/les=113/114 n=159 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:41892471:::obj524:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.585+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:401e941b:::obj398:head local-lis/les=113/114 n=159 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:41892471:::obj524:head 2026-02-11T23:31:27.585+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:401e941b:::obj398:head local-lis/les=113/114 n=159 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:41892471:::obj524:head v 111'524 2026-02-11T23:31:27.585+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:401e941b:::obj398:head local-lis/les=113/114 n=159 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.585+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.585+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:41892471:::obj524:head, version: 111'524, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:41892471:::obj524:head@111'524, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000044 2026-02-11T23:31:27.585+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.585+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.585+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:40d1abdf:::obj254:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.585+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:40d1abdf:::obj254:head) v3 prio 15 cost 0 latency 0.000060 pg_backfill(progress 1.0 e 116/116 lb 1:40d1abdf:::obj254:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:401e941b:::obj398:head local-lis/les=113/114 n=159 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.585+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.585+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:401e941b:::obj398:head local-lis/les=113/114 n=159 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:40d1abdf:::obj254:head) v3 2026-02-11T23:31:27.585+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:401e941b:::obj398:head local-lis/les=113/114 n=159 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:40d1abdf:::obj254:head) 2026-02-11T23:31:27.585+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.585+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:40d1abdf:::obj254:head) v3 finish latency 0.000009 2026-02-11T23:31:27.585+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:40d1abdf:::obj254:head local-lis/les=113/114 n=160 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.585+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.585+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.585+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:40d1abdf:::obj254:head local-lis/les=113/114 n=160 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.585+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.585+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.585+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.585+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e0bc80 con 0x4acb000 2026-02-11T23:31:27.585+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:41892471:::obj524:head)]) -- 0x5daac00 con 0x4acb000 2026-02-11T23:31:27.585+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.586+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 357 ==== MOSDPGPush(1.0 116/113 [PushOp(1:41ea52ba:::obj210:head, version: 111'210, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:41ea52ba:::obj210:head@111'210, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x49d0340 con 0x4acb000 2026-02-11T23:31:27.586+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:41ea52ba:::obj210:head, version: 111'210, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:41ea52ba:::obj210:head@111'210, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.586+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:41ea52ba:::obj210:head, version: 111'210, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:41ea52ba:::obj210:head@111'210, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.586+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.586+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 358 ==== pg_backfill(progress 1.0 e 116/116 lb 1:41892471:::obj524:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc3080 con 0x4acb000 2026-02-11T23:31:27.586+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:41892471:::obj524:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.586+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:41ea52ba:::obj210:head, version: 111'210, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:41ea52ba:::obj210:head@111'210, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.586+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:41892471:::obj524:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.586+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.586+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:41ea52ba:::obj210:head, version: 111'210, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:41ea52ba:::obj210:head@111'210, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.586+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:41ea52ba:::obj210:head, version: 111'210, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:41ea52ba:::obj210:head@111'210, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 116/113 [PushOp(1:41ea52ba:::obj210:head, version: 111'210, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:41ea52ba:::obj210:head@111'210, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:40d1abdf:::obj254:head local-lis/les=113/114 n=160 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.586+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.586+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.586+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:40d1abdf:::obj254:head local-lis/les=113/114 n=160 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:41ea52ba:::obj210:head, version: 111'210, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:41ea52ba:::obj210:head@111'210, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.586+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:41892471:::obj524:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.586+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:40d1abdf:::obj254:head local-lis/les=113/114 n=160 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:41ea52ba:::obj210:head@111'210, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.586+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:40d1abdf:::obj254:head local-lis/les=113/114 n=160 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:41ea52ba:::obj210:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.586+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:40d1abdf:::obj254:head local-lis/les=113/114 n=160 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:41ea52ba:::obj210:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.586+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:40d1abdf:::obj254:head local-lis/les=113/114 n=160 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:41ea52ba:::obj210:head 2026-02-11T23:31:27.586+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:40d1abdf:::obj254:head local-lis/les=113/114 n=160 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:41ea52ba:::obj210:head v 111'210 2026-02-11T23:31:27.586+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:40d1abdf:::obj254:head local-lis/les=113/114 n=160 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.586+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.586+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:41ea52ba:::obj210:head, version: 111'210, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:41ea52ba:::obj210:head@111'210, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000062 2026-02-11T23:31:27.586+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:40d1abdf:::obj254:head local-lis/les=113/114 n=160 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.586+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.586+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.586+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:41892471:::obj524:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.586+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:41892471:::obj524:head) v3 prio 15 cost 0 latency 0.000086 pg_backfill(progress 1.0 e 116/116 lb 1:41892471:::obj524:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:40d1abdf:::obj254:head local-lis/les=113/114 n=160 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.586+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.586+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:40d1abdf:::obj254:head local-lis/les=113/114 n=160 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:41892471:::obj524:head) v3 2026-02-11T23:31:27.586+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:40d1abdf:::obj254:head local-lis/les=113/114 n=160 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:41892471:::obj524:head) 2026-02-11T23:31:27.586+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.586+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:41892471:::obj524:head) v3 finish latency 0.000010 2026-02-11T23:31:27.586+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.586+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.586+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:41892471:::obj524:head local-lis/les=113/114 n=161 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.586+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.586+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.586+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.586+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e0a300 con 0x4acb000 2026-02-11T23:31:27.586+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:41ea52ba:::obj210:head)]) -- 0x5e0be00 con 0x4acb000 2026-02-11T23:31:27.586+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.586+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 359 ==== MOSDPGPush(1.0 116/113 [PushOp(1:424ce90f:::obj439:head, version: 111'439, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:424ce90f:::obj439:head@111'439, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5697040 con 0x4acb000 2026-02-11T23:31:27.586+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:424ce90f:::obj439:head, version: 111'439, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:424ce90f:::obj439:head@111'439, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.586+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:424ce90f:::obj439:head, version: 111'439, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:424ce90f:::obj439:head@111'439, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.586+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.586+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 360 ==== pg_backfill(progress 1.0 e 116/116 lb 1:41ea52ba:::obj210:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc3600 con 0x4acb000 2026-02-11T23:31:27.586+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:41ea52ba:::obj210:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.586+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:424ce90f:::obj439:head, version: 111'439, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:424ce90f:::obj439:head@111'439, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.586+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:41ea52ba:::obj210:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.586+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.586+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:424ce90f:::obj439:head, version: 111'439, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:424ce90f:::obj439:head@111'439, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.586+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.586+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:424ce90f:::obj439:head, version: 111'439, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:424ce90f:::obj439:head@111'439, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:424ce90f:::obj439:head, version: 111'439, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:424ce90f:::obj439:head@111'439, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:41892471:::obj524:head local-lis/les=113/114 n=161 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.586+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.586+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:41892471:::obj524:head local-lis/les=113/114 n=161 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:424ce90f:::obj439:head, version: 111'439, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:424ce90f:::obj439:head@111'439, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.586+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:41892471:::obj524:head local-lis/les=113/114 n=161 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:424ce90f:::obj439:head@111'439, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.586+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.586+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:41ea52ba:::obj210:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.586+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:41892471:::obj524:head local-lis/les=113/114 n=161 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:424ce90f:::obj439:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.586+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:41892471:::obj524:head local-lis/les=113/114 n=161 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:424ce90f:::obj439:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.586+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:41892471:::obj524:head local-lis/les=113/114 n=161 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:424ce90f:::obj439:head 2026-02-11T23:31:27.586+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:41892471:::obj524:head local-lis/les=113/114 n=161 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:424ce90f:::obj439:head v 111'439 2026-02-11T23:31:27.586+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:41892471:::obj524:head local-lis/les=113/114 n=161 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.586+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.586+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:424ce90f:::obj439:head, version: 111'439, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:424ce90f:::obj439:head@111'439, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000043 2026-02-11T23:31:27.586+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.586+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.586+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:41ea52ba:::obj210:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.586+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:41ea52ba:::obj210:head) v3 prio 15 cost 0 latency 0.000059 pg_backfill(progress 1.0 e 116/116 lb 1:41ea52ba:::obj210:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:41892471:::obj524:head local-lis/les=113/114 n=161 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.586+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.586+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:41892471:::obj524:head local-lis/les=113/114 n=161 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:41ea52ba:::obj210:head) v3 2026-02-11T23:31:27.586+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:41892471:::obj524:head local-lis/les=113/114 n=161 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:41ea52ba:::obj210:head) 2026-02-11T23:31:27.586+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.586+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:41ea52ba:::obj210:head) v3 finish latency 0.000012 2026-02-11T23:31:27.586+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:41ea52ba:::obj210:head local-lis/les=113/114 n=162 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.586+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.586+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.586+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:41ea52ba:::obj210:head local-lis/les=113/114 n=162 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.586+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.586+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.586+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.586+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e0a480 con 0x4acb000 2026-02-11T23:31:27.586+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:424ce90f:::obj439:head)]) -- 0x5daaf00 con 0x4acb000 2026-02-11T23:31:27.586+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.587+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 361 ==== MOSDPGPush(1.0 116/113 [PushOp(1:42d0e5ef:::obj468:head, version: 111'468, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:42d0e5ef:::obj468:head@111'468, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5697520 con 0x4acb000 2026-02-11T23:31:27.587+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:42d0e5ef:::obj468:head, version: 111'468, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:42d0e5ef:::obj468:head@111'468, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.587+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:42d0e5ef:::obj468:head, version: 111'468, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:42d0e5ef:::obj468:head@111'468, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.587+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.587+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 362 ==== pg_backfill(progress 1.0 e 116/116 lb 1:424ce90f:::obj439:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc4c00 con 0x4acb000 2026-02-11T23:31:27.587+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:424ce90f:::obj439:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.587+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:42d0e5ef:::obj468:head, version: 111'468, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:42d0e5ef:::obj468:head@111'468, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.587+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:424ce90f:::obj439:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.587+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.587+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:42d0e5ef:::obj468:head, version: 111'468, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:42d0e5ef:::obj468:head@111'468, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.587+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:42d0e5ef:::obj468:head, version: 111'468, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:42d0e5ef:::obj468:head@111'468, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:42d0e5ef:::obj468:head, version: 111'468, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:42d0e5ef:::obj468:head@111'468, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:41ea52ba:::obj210:head local-lis/les=113/114 n=162 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.587+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.587+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.587+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:41ea52ba:::obj210:head local-lis/les=113/114 n=162 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:42d0e5ef:::obj468:head, version: 111'468, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:42d0e5ef:::obj468:head@111'468, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.587+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:424ce90f:::obj439:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.587+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:41ea52ba:::obj210:head local-lis/les=113/114 n=162 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:42d0e5ef:::obj468:head@111'468, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.587+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:41ea52ba:::obj210:head local-lis/les=113/114 n=162 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:42d0e5ef:::obj468:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.587+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:41ea52ba:::obj210:head local-lis/les=113/114 n=162 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:42d0e5ef:::obj468:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.587+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:41ea52ba:::obj210:head local-lis/les=113/114 n=162 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:42d0e5ef:::obj468:head 2026-02-11T23:31:27.587+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:41ea52ba:::obj210:head local-lis/les=113/114 n=162 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:42d0e5ef:::obj468:head v 111'468 2026-02-11T23:31:27.587+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:41ea52ba:::obj210:head local-lis/les=113/114 n=162 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.587+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.587+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:42d0e5ef:::obj468:head, version: 111'468, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:42d0e5ef:::obj468:head@111'468, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:27.587+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.587+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.587+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:424ce90f:::obj439:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.587+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:424ce90f:::obj439:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:424ce90f:::obj439:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:41ea52ba:::obj210:head local-lis/les=113/114 n=162 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.587+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.587+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:41ea52ba:::obj210:head local-lis/les=113/114 n=162 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:424ce90f:::obj439:head) v3 2026-02-11T23:31:27.587+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:41ea52ba:::obj210:head local-lis/les=113/114 n=162 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:424ce90f:::obj439:head) 2026-02-11T23:31:27.587+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.587+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:424ce90f:::obj439:head) v3 finish latency 0.000008 2026-02-11T23:31:27.587+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:424ce90f:::obj439:head local-lis/les=113/114 n=163 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.587+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.587+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.587+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:424ce90f:::obj439:head local-lis/les=113/114 n=163 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.587+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.587+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.587+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.587+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e0a900 con 0x4acb000 2026-02-11T23:31:27.587+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:42d0e5ef:::obj468:head)]) -- 0x5dab200 con 0x4acb000 2026-02-11T23:31:27.587+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.587+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 363 ==== MOSDPGPush(1.0 116/113 [PushOp(1:436594ec:::obj300:head, version: 111'300, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:436594ec:::obj300:head@111'300, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d9d40 con 0x4acb000 2026-02-11T23:31:27.587+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:436594ec:::obj300:head, version: 111'300, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:436594ec:::obj300:head@111'300, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.587+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:436594ec:::obj300:head, version: 111'300, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:436594ec:::obj300:head@111'300, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.587+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.587+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:436594ec:::obj300:head, version: 111'300, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:436594ec:::obj300:head@111'300, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.587+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 364 ==== pg_backfill(progress 1.0 e 116/116 lb 1:42d0e5ef:::obj468:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc4680 con 0x4acb000 2026-02-11T23:31:27.587+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.587+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:42d0e5ef:::obj468:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 116 2026-02-11T23:31:27.587+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:436594ec:::obj300:head, version: 111'300, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:436594ec:::obj300:head@111'300, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.587+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:42d0e5ef:::obj468:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.587+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.587+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:42d0e5ef:::obj468:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.587+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:436594ec:::obj300:head, version: 111'300, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:436594ec:::obj300:head@111'300, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:436594ec:::obj300:head, version: 111'300, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:436594ec:::obj300:head@111'300, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:424ce90f:::obj439:head local-lis/les=113/114 n=163 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.587+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.587+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:424ce90f:::obj439:head local-lis/les=113/114 n=163 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:436594ec:::obj300:head, version: 111'300, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:436594ec:::obj300:head@111'300, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.587+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:424ce90f:::obj439:head local-lis/les=113/114 n=163 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:436594ec:::obj300:head@111'300, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.587+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:424ce90f:::obj439:head local-lis/les=113/114 n=163 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:436594ec:::obj300:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.587+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:424ce90f:::obj439:head local-lis/les=113/114 n=163 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:436594ec:::obj300:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.587+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:424ce90f:::obj439:head local-lis/les=113/114 n=163 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:436594ec:::obj300:head 2026-02-11T23:31:27.587+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:424ce90f:::obj439:head local-lis/les=113/114 n=163 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:436594ec:::obj300:head v 111'300 2026-02-11T23:31:27.587+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:424ce90f:::obj439:head local-lis/les=113/114 n=163 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.587+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.587+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:436594ec:::obj300:head, version: 111'300, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:436594ec:::obj300:head@111'300, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.587+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.587+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.587+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:42d0e5ef:::obj468:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.587+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:42d0e5ef:::obj468:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 1.0 e 116/116 lb 1:42d0e5ef:::obj468:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:424ce90f:::obj439:head local-lis/les=113/114 n=163 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.587+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.587+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:424ce90f:::obj439:head local-lis/les=113/114 n=163 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:42d0e5ef:::obj468:head) v3 2026-02-11T23:31:27.587+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:424ce90f:::obj439:head local-lis/les=113/114 n=163 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:42d0e5ef:::obj468:head) 2026-02-11T23:31:27.587+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.587+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:42d0e5ef:::obj468:head) v3 finish latency 0.000009 2026-02-11T23:31:27.587+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:42d0e5ef:::obj468:head local-lis/les=113/114 n=164 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.587+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.588+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.588+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:42d0e5ef:::obj468:head local-lis/les=113/114 n=164 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.588+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.588+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.588+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.588+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e0a600 con 0x4acb000 2026-02-11T23:31:27.588+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:436594ec:::obj300:head)]) -- 0x5dab500 con 0x4acb000 2026-02-11T23:31:27.588+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.588+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 365 ==== MOSDPGPush(1.0 116/113 [PushOp(1:454d8265:::obj256:head, version: 111'256, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:454d8265:::obj256:head@111'256, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5625ba0 con 0x4acb000 2026-02-11T23:31:27.588+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:454d8265:::obj256:head, version: 111'256, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:454d8265:::obj256:head@111'256, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.588+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:454d8265:::obj256:head, version: 111'256, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:454d8265:::obj256:head@111'256, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.588+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.588+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 366 ==== pg_backfill(progress 1.0 e 116/116 lb 1:436594ec:::obj300:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc3b80 con 0x4acb000 2026-02-11T23:31:27.588+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:436594ec:::obj300:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.588+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:454d8265:::obj256:head, version: 111'256, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:454d8265:::obj256:head@111'256, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.588+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:436594ec:::obj300:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.588+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.588+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:454d8265:::obj256:head, version: 111'256, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:454d8265:::obj256:head@111'256, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.588+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:454d8265:::obj256:head, version: 111'256, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:454d8265:::obj256:head@111'256, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 116/113 [PushOp(1:454d8265:::obj256:head, version: 111'256, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:454d8265:::obj256:head@111'256, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:42d0e5ef:::obj468:head local-lis/les=113/114 n=164 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.588+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.588+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.588+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:42d0e5ef:::obj468:head local-lis/les=113/114 n=164 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:454d8265:::obj256:head, version: 111'256, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:454d8265:::obj256:head@111'256, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.588+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:436594ec:::obj300:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.588+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:42d0e5ef:::obj468:head local-lis/les=113/114 n=164 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:454d8265:::obj256:head@111'256, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.588+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:42d0e5ef:::obj468:head local-lis/les=113/114 n=164 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:454d8265:::obj256:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.588+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:42d0e5ef:::obj468:head local-lis/les=113/114 n=164 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:454d8265:::obj256:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.588+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:42d0e5ef:::obj468:head local-lis/les=113/114 n=164 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:454d8265:::obj256:head 2026-02-11T23:31:27.588+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:42d0e5ef:::obj468:head local-lis/les=113/114 n=164 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:454d8265:::obj256:head v 111'256 2026-02-11T23:31:27.588+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:42d0e5ef:::obj468:head local-lis/les=113/114 n=164 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.588+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.588+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:454d8265:::obj256:head, version: 111'256, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:454d8265:::obj256:head@111'256, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:27.588+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.588+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.588+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:436594ec:::obj300:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.588+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:436594ec:::obj300:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 116/116 lb 1:436594ec:::obj300:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:42d0e5ef:::obj468:head local-lis/les=113/114 n=164 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.588+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.588+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:42d0e5ef:::obj468:head local-lis/les=113/114 n=164 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:436594ec:::obj300:head) v3 2026-02-11T23:31:27.588+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:42d0e5ef:::obj468:head local-lis/les=113/114 n=164 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:436594ec:::obj300:head) 2026-02-11T23:31:27.588+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.588+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:436594ec:::obj300:head) v3 finish latency 0.000009 2026-02-11T23:31:27.588+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:436594ec:::obj300:head local-lis/les=113/114 n=165 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.588+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.588+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.588+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:436594ec:::obj300:head local-lis/les=113/114 n=165 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.588+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.588+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.588+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.588+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e0aa80 con 0x4acb000 2026-02-11T23:31:27.588+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:454d8265:::obj256:head)]) -- 0x5dab800 con 0x4acb000 2026-02-11T23:31:27.588+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.589+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 367 ==== MOSDPGPush(1.0 116/113 [PushOp(1:461f8b5e:::obj16:head, version: 111'16, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:461f8b5e:::obj16:head@111'16, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x55f04e0 con 0x4acb000 2026-02-11T23:31:27.589+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:461f8b5e:::obj16:head, version: 111'16, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:461f8b5e:::obj16:head@111'16, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.589+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:461f8b5e:::obj16:head, version: 111'16, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:461f8b5e:::obj16:head@111'16, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.589+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.589+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 368 ==== pg_backfill(progress 1.0 e 116/116 lb 1:454d8265:::obj256:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc2000 con 0x4acb000 2026-02-11T23:31:27.589+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:461f8b5e:::obj16:head, version: 111'16, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:461f8b5e:::obj16:head@111'16, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.589+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:454d8265:::obj256:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.589+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.589+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:454d8265:::obj256:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.589+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:461f8b5e:::obj16:head, version: 111'16, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:461f8b5e:::obj16:head@111'16, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.589+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.589+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:461f8b5e:::obj16:head, version: 111'16, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:461f8b5e:::obj16:head@111'16, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:461f8b5e:::obj16:head, version: 111'16, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:461f8b5e:::obj16:head@111'16, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:436594ec:::obj300:head local-lis/les=113/114 n=165 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.589+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.589+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:436594ec:::obj300:head local-lis/les=113/114 n=165 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:461f8b5e:::obj16:head, version: 111'16, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:461f8b5e:::obj16:head@111'16, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.589+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:436594ec:::obj300:head local-lis/les=113/114 n=165 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:461f8b5e:::obj16:head@111'16, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.589+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.589+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:454d8265:::obj256:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.589+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:436594ec:::obj300:head local-lis/les=113/114 n=165 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:461f8b5e:::obj16:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.589+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:436594ec:::obj300:head local-lis/les=113/114 n=165 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:461f8b5e:::obj16:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.589+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:436594ec:::obj300:head local-lis/les=113/114 n=165 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:461f8b5e:::obj16:head 2026-02-11T23:31:27.589+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:436594ec:::obj300:head local-lis/les=113/114 n=165 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:461f8b5e:::obj16:head v 111'16 2026-02-11T23:31:27.589+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:436594ec:::obj300:head local-lis/les=113/114 n=165 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.589+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.589+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:461f8b5e:::obj16:head, version: 111'16, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:461f8b5e:::obj16:head@111'16, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.589+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.589+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.589+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:454d8265:::obj256:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.589+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:454d8265:::obj256:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 1.0 e 116/116 lb 1:454d8265:::obj256:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:436594ec:::obj300:head local-lis/les=113/114 n=165 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.589+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.589+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:436594ec:::obj300:head local-lis/les=113/114 n=165 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:454d8265:::obj256:head) v3 2026-02-11T23:31:27.589+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:436594ec:::obj300:head local-lis/les=113/114 n=165 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:454d8265:::obj256:head) 2026-02-11T23:31:27.589+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.589+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:454d8265:::obj256:head) v3 finish latency 0.000010 2026-02-11T23:31:27.589+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:454d8265:::obj256:head local-lis/les=113/114 n=166 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.589+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.589+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.589+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:454d8265:::obj256:head local-lis/les=113/114 n=166 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.589+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.589+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.589+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.589+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e0af00 con 0x4acb000 2026-02-11T23:31:27.589+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:461f8b5e:::obj16:head)]) -- 0x5dab980 con 0x4acb000 2026-02-11T23:31:27.589+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.589+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 369 ==== MOSDPGPush(1.0 116/113 [PushOp(1:4667f6bf:::obj584:head, version: 111'584, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4667f6bf:::obj584:head@111'584, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5616b60 con 0x4acb000 2026-02-11T23:31:27.589+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:4667f6bf:::obj584:head, version: 111'584, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4667f6bf:::obj584:head@111'584, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.589+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4667f6bf:::obj584:head, version: 111'584, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4667f6bf:::obj584:head@111'584, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.589+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.589+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 370 ==== pg_backfill(progress 1.0 e 116/116 lb 1:461f8b5e:::obj16:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d98000 con 0x4acb000 2026-02-11T23:31:27.589+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:461f8b5e:::obj16:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.589+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4667f6bf:::obj584:head, version: 111'584, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4667f6bf:::obj584:head@111'584, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.589+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:461f8b5e:::obj16:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.589+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.589+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4667f6bf:::obj584:head, version: 111'584, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4667f6bf:::obj584:head@111'584, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.589+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:4667f6bf:::obj584:head, version: 111'584, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4667f6bf:::obj584:head@111'584, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:4667f6bf:::obj584:head, version: 111'584, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4667f6bf:::obj584:head@111'584, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:454d8265:::obj256:head local-lis/les=113/114 n=166 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.589+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.589+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:454d8265:::obj256:head local-lis/les=113/114 n=166 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:4667f6bf:::obj584:head, version: 111'584, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4667f6bf:::obj584:head@111'584, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.589+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.589+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:461f8b5e:::obj16:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.589+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:454d8265:::obj256:head local-lis/les=113/114 n=166 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:4667f6bf:::obj584:head@111'584, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.589+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:454d8265:::obj256:head local-lis/les=113/114 n=166 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:4667f6bf:::obj584:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.589+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:454d8265:::obj256:head local-lis/les=113/114 n=166 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:4667f6bf:::obj584:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.589+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:454d8265:::obj256:head local-lis/les=113/114 n=166 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:4667f6bf:::obj584:head 2026-02-11T23:31:27.589+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:454d8265:::obj256:head local-lis/les=113/114 n=166 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:4667f6bf:::obj584:head v 111'584 2026-02-11T23:31:27.589+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:454d8265:::obj256:head local-lis/les=113/114 n=166 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.589+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.589+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:4667f6bf:::obj584:head, version: 111'584, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4667f6bf:::obj584:head@111'584, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.589+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.589+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.589+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:461f8b5e:::obj16:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.589+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:461f8b5e:::obj16:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:461f8b5e:::obj16:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:454d8265:::obj256:head local-lis/les=113/114 n=166 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.589+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.589+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:454d8265:::obj256:head local-lis/les=113/114 n=166 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:461f8b5e:::obj16:head) v3 2026-02-11T23:31:27.589+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:454d8265:::obj256:head local-lis/les=113/114 n=166 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:461f8b5e:::obj16:head) 2026-02-11T23:31:27.589+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.589+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:461f8b5e:::obj16:head) v3 finish latency 0.000010 2026-02-11T23:31:27.589+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:461f8b5e:::obj16:head local-lis/les=113/114 n=167 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.589+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.589+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.589+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:461f8b5e:::obj16:head local-lis/les=113/114 n=167 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.589+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.589+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.589+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.589+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e0ac00 con 0x4acb000 2026-02-11T23:31:27.589+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:4667f6bf:::obj584:head)]) -- 0x5dabb00 con 0x4acb000 2026-02-11T23:31:27.589+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.590+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 371 ==== MOSDPGPush(1.0 116/113 [PushOp(1:46b5a6d0:::obj580:head, version: 111'580, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:46b5a6d0:::obj580:head@111'580, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5617040 con 0x4acb000 2026-02-11T23:31:27.590+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:46b5a6d0:::obj580:head, version: 111'580, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:46b5a6d0:::obj580:head@111'580, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.590+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:46b5a6d0:::obj580:head, version: 111'580, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:46b5a6d0:::obj580:head@111'580, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.590+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.590+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 372 ==== pg_backfill(progress 1.0 e 116/116 lb 1:4667f6bf:::obj584:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98580 con 0x4acb000 2026-02-11T23:31:27.590+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:46b5a6d0:::obj580:head, version: 111'580, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:46b5a6d0:::obj580:head@111'580, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.590+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:4667f6bf:::obj584:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.590+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4667f6bf:::obj584:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.590+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.590+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:46b5a6d0:::obj580:head, version: 111'580, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:46b5a6d0:::obj580:head@111'580, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.590+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.590+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:46b5a6d0:::obj580:head, version: 111'580, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:46b5a6d0:::obj580:head@111'580, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 116/113 [PushOp(1:46b5a6d0:::obj580:head, version: 111'580, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:46b5a6d0:::obj580:head@111'580, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:461f8b5e:::obj16:head local-lis/les=113/114 n=167 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.590+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.590+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:461f8b5e:::obj16:head local-lis/les=113/114 n=167 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:46b5a6d0:::obj580:head, version: 111'580, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:46b5a6d0:::obj580:head@111'580, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.590+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:461f8b5e:::obj16:head local-lis/les=113/114 n=167 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:46b5a6d0:::obj580:head@111'580, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.590+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.590+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4667f6bf:::obj584:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.590+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:461f8b5e:::obj16:head local-lis/les=113/114 n=167 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:46b5a6d0:::obj580:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.590+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:461f8b5e:::obj16:head local-lis/les=113/114 n=167 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:46b5a6d0:::obj580:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.590+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:461f8b5e:::obj16:head local-lis/les=113/114 n=167 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:46b5a6d0:::obj580:head 2026-02-11T23:31:27.590+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:461f8b5e:::obj16:head local-lis/les=113/114 n=167 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:46b5a6d0:::obj580:head v 111'580 2026-02-11T23:31:27.590+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:461f8b5e:::obj16:head local-lis/les=113/114 n=167 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.590+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.590+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:46b5a6d0:::obj580:head, version: 111'580, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:46b5a6d0:::obj580:head@111'580, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000044 2026-02-11T23:31:27.590+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.590+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.590+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4667f6bf:::obj584:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.590+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:4667f6bf:::obj584:head) v3 prio 15 cost 0 latency 0.000060 pg_backfill(progress 1.0 e 116/116 lb 1:4667f6bf:::obj584:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:461f8b5e:::obj16:head local-lis/les=113/114 n=167 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.590+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.590+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:461f8b5e:::obj16:head local-lis/les=113/114 n=167 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:4667f6bf:::obj584:head) v3 2026-02-11T23:31:27.590+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:461f8b5e:::obj16:head local-lis/les=113/114 n=167 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:4667f6bf:::obj584:head) 2026-02-11T23:31:27.590+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.590+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:4667f6bf:::obj584:head) v3 finish latency 0.000010 2026-02-11T23:31:27.590+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4667f6bf:::obj584:head local-lis/les=113/114 n=168 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.590+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.590+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.590+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4667f6bf:::obj584:head local-lis/les=113/114 n=168 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.590+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.590+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.590+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.590+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e7ed80 con 0x4acb000 2026-02-11T23:31:27.590+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:46b5a6d0:::obj580:head)]) -- 0x5dabe00 con 0x4acb000 2026-02-11T23:31:27.590+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.590+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 373 ==== MOSDPGPush(1.0 116/113 [PushOp(1:4851a45a:::obj44:head, version: 111'44, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4851a45a:::obj44:head@111'44, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5646340 con 0x4acb000 2026-02-11T23:31:27.590+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:4851a45a:::obj44:head, version: 111'44, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4851a45a:::obj44:head@111'44, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.590+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4851a45a:::obj44:head, version: 111'44, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4851a45a:::obj44:head@111'44, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.590+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.590+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 374 ==== pg_backfill(progress 1.0 e 116/116 lb 1:46b5a6d0:::obj580:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2d080 con 0x4acb000 2026-02-11T23:31:27.590+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:46b5a6d0:::obj580:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.590+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4851a45a:::obj44:head, version: 111'44, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4851a45a:::obj44:head@111'44, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.590+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:46b5a6d0:::obj580:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.590+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.590+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4851a45a:::obj44:head, version: 111'44, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4851a45a:::obj44:head@111'44, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.590+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:4851a45a:::obj44:head, version: 111'44, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4851a45a:::obj44:head@111'44, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:4851a45a:::obj44:head, version: 111'44, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4851a45a:::obj44:head@111'44, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:4667f6bf:::obj584:head local-lis/les=113/114 n=168 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.590+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.590+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4667f6bf:::obj584:head local-lis/les=113/114 n=168 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:4851a45a:::obj44:head, version: 111'44, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4851a45a:::obj44:head@111'44, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.590+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.590+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:46b5a6d0:::obj580:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.590+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4667f6bf:::obj584:head local-lis/les=113/114 n=168 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:4851a45a:::obj44:head@111'44, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.590+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4667f6bf:::obj584:head local-lis/les=113/114 n=168 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:4851a45a:::obj44:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.590+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4667f6bf:::obj584:head local-lis/les=113/114 n=168 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:4851a45a:::obj44:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.590+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4667f6bf:::obj584:head local-lis/les=113/114 n=168 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:4851a45a:::obj44:head 2026-02-11T23:31:27.590+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4667f6bf:::obj584:head local-lis/les=113/114 n=168 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:4851a45a:::obj44:head v 111'44 2026-02-11T23:31:27.590+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4667f6bf:::obj584:head local-lis/les=113/114 n=168 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.590+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.590+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:4851a45a:::obj44:head, version: 111'44, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4851a45a:::obj44:head@111'44, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:27.590+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.590+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.590+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:46b5a6d0:::obj580:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.590+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:46b5a6d0:::obj580:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 1.0 e 116/116 lb 1:46b5a6d0:::obj580:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:4667f6bf:::obj584:head local-lis/les=113/114 n=168 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.590+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.590+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4667f6bf:::obj584:head local-lis/les=113/114 n=168 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:46b5a6d0:::obj580:head) v3 2026-02-11T23:31:27.590+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4667f6bf:::obj584:head local-lis/les=113/114 n=168 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:46b5a6d0:::obj580:head) 2026-02-11T23:31:27.590+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.590+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:46b5a6d0:::obj580:head) v3 finish latency 0.000010 2026-02-11T23:31:27.590+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:46b5a6d0:::obj580:head local-lis/les=113/114 n=169 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.590+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.591+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.591+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:46b5a6d0:::obj580:head local-lis/les=113/114 n=169 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.591+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.591+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.591+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.591+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e6b980 con 0x4acb000 2026-02-11T23:31:27.591+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:4851a45a:::obj44:head)]) -- 0x5fd2000 con 0x4acb000 2026-02-11T23:31:27.591+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.591+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 375 ==== MOSDPGPush(1.0 116/113 [PushOp(1:48c1458b:::obj274:head, version: 111'274, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:48c1458b:::obj274:head@111'274, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5646820 con 0x4acb000 2026-02-11T23:31:27.591+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:48c1458b:::obj274:head, version: 111'274, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:48c1458b:::obj274:head@111'274, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.591+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:48c1458b:::obj274:head, version: 111'274, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:48c1458b:::obj274:head@111'274, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.591+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.591+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 376 ==== pg_backfill(progress 1.0 e 116/116 lb 1:4851a45a:::obj44:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d2e680 con 0x4acb000 2026-02-11T23:31:27.591+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:48c1458b:::obj274:head, version: 111'274, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:48c1458b:::obj274:head@111'274, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.591+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:4851a45a:::obj44:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.591+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.591+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4851a45a:::obj44:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.591+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:48c1458b:::obj274:head, version: 111'274, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:48c1458b:::obj274:head@111'274, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.591+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.591+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:48c1458b:::obj274:head, version: 111'274, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:48c1458b:::obj274:head@111'274, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 116/113 [PushOp(1:48c1458b:::obj274:head, version: 111'274, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:48c1458b:::obj274:head@111'274, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:46b5a6d0:::obj580:head local-lis/les=113/114 n=169 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.591+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.591+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:46b5a6d0:::obj580:head local-lis/les=113/114 n=169 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:48c1458b:::obj274:head, version: 111'274, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:48c1458b:::obj274:head@111'274, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.591+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:46b5a6d0:::obj580:head local-lis/les=113/114 n=169 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:48c1458b:::obj274:head@111'274, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.591+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.591+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4851a45a:::obj44:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.591+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:46b5a6d0:::obj580:head local-lis/les=113/114 n=169 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:48c1458b:::obj274:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.591+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:46b5a6d0:::obj580:head local-lis/les=113/114 n=169 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:48c1458b:::obj274:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.591+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:46b5a6d0:::obj580:head local-lis/les=113/114 n=169 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:48c1458b:::obj274:head 2026-02-11T23:31:27.591+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:46b5a6d0:::obj580:head local-lis/les=113/114 n=169 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:48c1458b:::obj274:head v 111'274 2026-02-11T23:31:27.591+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:46b5a6d0:::obj580:head local-lis/les=113/114 n=169 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.591+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.591+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:48c1458b:::obj274:head, version: 111'274, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:48c1458b:::obj274:head@111'274, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.591+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.591+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.591+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4851a45a:::obj44:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.591+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:4851a45a:::obj44:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 1.0 e 116/116 lb 1:4851a45a:::obj44:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:46b5a6d0:::obj580:head local-lis/les=113/114 n=169 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.591+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.591+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:46b5a6d0:::obj580:head local-lis/les=113/114 n=169 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:4851a45a:::obj44:head) v3 2026-02-11T23:31:27.591+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:46b5a6d0:::obj580:head local-lis/les=113/114 n=169 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:4851a45a:::obj44:head) 2026-02-11T23:31:27.591+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.591+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:4851a45a:::obj44:head) v3 finish latency 0.000010 2026-02-11T23:31:27.591+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4851a45a:::obj44:head local-lis/les=113/114 n=170 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.591+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.591+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.591+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4851a45a:::obj44:head local-lis/les=113/114 n=170 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.591+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.591+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.591+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.591+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5df9980 con 0x4acb000 2026-02-11T23:31:27.591+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:48c1458b:::obj274:head)]) -- 0x5fd2180 con 0x4acb000 2026-02-11T23:31:27.591+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.591+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 377 ==== MOSDPGPush(1.0 116/113 [PushOp(1:4911cfa9:::obj108:head, version: 111'108, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4911cfa9:::obj108:head@111'108, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x555c1a0 con 0x4acb000 2026-02-11T23:31:27.591+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:4911cfa9:::obj108:head, version: 111'108, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4911cfa9:::obj108:head@111'108, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.591+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4911cfa9:::obj108:head, version: 111'108, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4911cfa9:::obj108:head@111'108, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.591+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.591+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 378 ==== pg_backfill(progress 1.0 e 116/116 lb 1:48c1458b:::obj274:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e100 con 0x4acb000 2026-02-11T23:31:27.591+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4911cfa9:::obj108:head, version: 111'108, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4911cfa9:::obj108:head@111'108, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.591+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:48c1458b:::obj274:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.591+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:48c1458b:::obj274:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.591+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.591+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4911cfa9:::obj108:head, version: 111'108, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4911cfa9:::obj108:head@111'108, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.591+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:4911cfa9:::obj108:head, version: 111'108, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4911cfa9:::obj108:head@111'108, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:4911cfa9:::obj108:head, version: 111'108, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4911cfa9:::obj108:head@111'108, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:4851a45a:::obj44:head local-lis/les=113/114 n=170 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.591+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.591+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4851a45a:::obj44:head local-lis/les=113/114 n=170 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:4911cfa9:::obj108:head, version: 111'108, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4911cfa9:::obj108:head@111'108, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.591+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.591+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4851a45a:::obj44:head local-lis/les=113/114 n=170 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:4911cfa9:::obj108:head@111'108, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.591+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:48c1458b:::obj274:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.591+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4851a45a:::obj44:head local-lis/les=113/114 n=170 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:4911cfa9:::obj108:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.591+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4851a45a:::obj44:head local-lis/les=113/114 n=170 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:4911cfa9:::obj108:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.591+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4851a45a:::obj44:head local-lis/les=113/114 n=170 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:4911cfa9:::obj108:head 2026-02-11T23:31:27.591+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4851a45a:::obj44:head local-lis/les=113/114 n=170 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:4911cfa9:::obj108:head v 111'108 2026-02-11T23:31:27.591+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4851a45a:::obj44:head local-lis/les=113/114 n=170 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.591+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.592+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:4911cfa9:::obj108:head, version: 111'108, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4911cfa9:::obj108:head@111'108, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:27.592+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.592+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.592+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:48c1458b:::obj274:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.592+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:48c1458b:::obj274:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 1.0 e 116/116 lb 1:48c1458b:::obj274:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:4851a45a:::obj44:head local-lis/les=113/114 n=170 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.592+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.592+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4851a45a:::obj44:head local-lis/les=113/114 n=170 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:48c1458b:::obj274:head) v3 2026-02-11T23:31:27.592+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4851a45a:::obj44:head local-lis/les=113/114 n=170 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:48c1458b:::obj274:head) 2026-02-11T23:31:27.592+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.592+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:48c1458b:::obj274:head) v3 finish latency 0.000010 2026-02-11T23:31:27.592+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:48c1458b:::obj274:head local-lis/les=113/114 n=171 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.592+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.592+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.592+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:48c1458b:::obj274:head local-lis/les=113/114 n=171 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.592+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.592+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.592+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.592+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5df8000 con 0x4acb000 2026-02-11T23:31:27.592+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:4911cfa9:::obj108:head)]) -- 0x5fd2480 con 0x4acb000 2026-02-11T23:31:27.592+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.592+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 379 ==== MOSDPGPush(1.0 116/113 [PushOp(1:4918aa91:::obj303:head, version: 111'303, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4918aa91:::obj303:head@111'303, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5617860 con 0x4acb000 2026-02-11T23:31:27.592+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:4918aa91:::obj303:head, version: 111'303, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4918aa91:::obj303:head@111'303, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.592+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4918aa91:::obj303:head, version: 111'303, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4918aa91:::obj303:head@111'303, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.592+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.592+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 380 ==== pg_backfill(progress 1.0 e 116/116 lb 1:4911cfa9:::obj108:head) ==== 1062+0+0 (unknown 0 0 0) 0x4a2cb00 con 0x4acb000 2026-02-11T23:31:27.592+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4918aa91:::obj303:head, version: 111'303, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4918aa91:::obj303:head@111'303, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.592+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:4911cfa9:::obj108:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.592+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4911cfa9:::obj108:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.592+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.592+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4918aa91:::obj303:head, version: 111'303, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4918aa91:::obj303:head@111'303, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.592+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.592+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:4918aa91:::obj303:head, version: 111'303, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4918aa91:::obj303:head@111'303, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:4918aa91:::obj303:head, version: 111'303, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4918aa91:::obj303:head@111'303, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:48c1458b:::obj274:head local-lis/les=113/114 n=171 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.592+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.592+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.592+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4911cfa9:::obj108:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.592+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:48c1458b:::obj274:head local-lis/les=113/114 n=171 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:4918aa91:::obj303:head, version: 111'303, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4918aa91:::obj303:head@111'303, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.592+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:48c1458b:::obj274:head local-lis/les=113/114 n=171 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:4918aa91:::obj303:head@111'303, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.592+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:48c1458b:::obj274:head local-lis/les=113/114 n=171 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:4918aa91:::obj303:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.592+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:48c1458b:::obj274:head local-lis/les=113/114 n=171 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:4918aa91:::obj303:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.592+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:48c1458b:::obj274:head local-lis/les=113/114 n=171 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:4918aa91:::obj303:head 2026-02-11T23:31:27.592+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:48c1458b:::obj274:head local-lis/les=113/114 n=171 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:4918aa91:::obj303:head v 111'303 2026-02-11T23:31:27.592+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:48c1458b:::obj274:head local-lis/les=113/114 n=171 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.592+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.592+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:4918aa91:::obj303:head, version: 111'303, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4918aa91:::obj303:head@111'303, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:27.592+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.592+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.592+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4911cfa9:::obj108:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.592+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:4911cfa9:::obj108:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 1.0 e 116/116 lb 1:4911cfa9:::obj108:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:48c1458b:::obj274:head local-lis/les=113/114 n=171 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.592+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.592+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:48c1458b:::obj274:head local-lis/les=113/114 n=171 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:4911cfa9:::obj108:head) v3 2026-02-11T23:31:27.592+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:48c1458b:::obj274:head local-lis/les=113/114 n=171 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:4911cfa9:::obj108:head) 2026-02-11T23:31:27.592+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.592+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:4911cfa9:::obj108:head) v3 finish latency 0.000010 2026-02-11T23:31:27.592+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4911cfa9:::obj108:head local-lis/les=113/114 n=172 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.592+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.592+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.592+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4911cfa9:::obj108:head local-lis/les=113/114 n=172 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.592+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.592+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.592+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.592+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5df8780 con 0x4acb000 2026-02-11T23:31:27.592+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:4918aa91:::obj303:head)]) -- 0x5fd2780 con 0x4acb000 2026-02-11T23:31:27.592+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.593+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 381 ==== MOSDPGPush(1.0 116/113 [PushOp(1:492a75ed:::obj72:head, version: 111'72, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:492a75ed:::obj72:head@111'72, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5617d40 con 0x4acb000 2026-02-11T23:31:27.593+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:492a75ed:::obj72:head, version: 111'72, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:492a75ed:::obj72:head@111'72, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.593+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:492a75ed:::obj72:head, version: 111'72, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:492a75ed:::obj72:head@111'72, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.593+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.593+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 382 ==== pg_backfill(progress 1.0 e 116/116 lb 1:4918aa91:::obj303:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc5180 con 0x4acb000 2026-02-11T23:31:27.593+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:492a75ed:::obj72:head, version: 111'72, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:492a75ed:::obj72:head@111'72, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.593+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:4918aa91:::obj303:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 116 2026-02-11T23:31:27.593+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.593+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4918aa91:::obj303:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.593+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:492a75ed:::obj72:head, version: 111'72, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:492a75ed:::obj72:head@111'72, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.593+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.593+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:492a75ed:::obj72:head, version: 111'72, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:492a75ed:::obj72:head@111'72, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:492a75ed:::obj72:head, version: 111'72, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:492a75ed:::obj72:head@111'72, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:4911cfa9:::obj108:head local-lis/les=113/114 n=172 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.593+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4918aa91:::obj303:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.593+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.593+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4911cfa9:::obj108:head local-lis/les=113/114 n=172 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:492a75ed:::obj72:head, version: 111'72, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:492a75ed:::obj72:head@111'72, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.593+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4911cfa9:::obj108:head local-lis/les=113/114 n=172 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:492a75ed:::obj72:head@111'72, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.593+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4911cfa9:::obj108:head local-lis/les=113/114 n=172 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:492a75ed:::obj72:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.593+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4911cfa9:::obj108:head local-lis/les=113/114 n=172 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:492a75ed:::obj72:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.593+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4911cfa9:::obj108:head local-lis/les=113/114 n=172 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:492a75ed:::obj72:head 2026-02-11T23:31:27.593+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4911cfa9:::obj108:head local-lis/les=113/114 n=172 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:492a75ed:::obj72:head v 111'72 2026-02-11T23:31:27.593+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4911cfa9:::obj108:head local-lis/les=113/114 n=172 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.593+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.593+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:492a75ed:::obj72:head, version: 111'72, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:492a75ed:::obj72:head@111'72, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.593+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.593+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.593+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4918aa91:::obj303:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.593+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:4918aa91:::obj303:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 1.0 e 116/116 lb 1:4918aa91:::obj303:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:4911cfa9:::obj108:head local-lis/les=113/114 n=172 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.593+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.593+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4911cfa9:::obj108:head local-lis/les=113/114 n=172 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:4918aa91:::obj303:head) v3 2026-02-11T23:31:27.593+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4911cfa9:::obj108:head local-lis/les=113/114 n=172 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:4918aa91:::obj303:head) 2026-02-11T23:31:27.593+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.593+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:4918aa91:::obj303:head) v3 finish latency 0.000009 2026-02-11T23:31:27.593+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4918aa91:::obj303:head local-lis/les=113/114 n=173 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.593+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.593+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.593+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4918aa91:::obj303:head local-lis/les=113/114 n=173 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.593+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.593+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.593+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.593+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5df8a80 con 0x4acb000 2026-02-11T23:31:27.593+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:492a75ed:::obj72:head)]) -- 0x5fd2a80 con 0x4acb000 2026-02-11T23:31:27.593+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.593+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 383 ==== MOSDPGPush(1.0 116/113 [PushOp(1:498092df:::obj260:head, version: 111'260, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:498092df:::obj260:head@111'260, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f0340 con 0x4acb000 2026-02-11T23:31:27.593+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:498092df:::obj260:head, version: 111'260, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:498092df:::obj260:head@111'260, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.593+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:498092df:::obj260:head, version: 111'260, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:498092df:::obj260:head@111'260, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.593+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.593+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 384 ==== pg_backfill(progress 1.0 e 116/116 lb 1:492a75ed:::obj72:head) ==== 1061+0+0 (unknown 0 0 0) 0x5dc4100 con 0x4acb000 2026-02-11T23:31:27.593+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:492a75ed:::obj72:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.593+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:498092df:::obj260:head, version: 111'260, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:498092df:::obj260:head@111'260, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.593+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:492a75ed:::obj72:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.593+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.593+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:498092df:::obj260:head, version: 111'260, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:498092df:::obj260:head@111'260, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.593+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:498092df:::obj260:head, version: 111'260, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:498092df:::obj260:head@111'260, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:498092df:::obj260:head, version: 111'260, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:498092df:::obj260:head@111'260, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:4918aa91:::obj303:head local-lis/les=113/114 n=173 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.593+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.593+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4918aa91:::obj303:head local-lis/les=113/114 n=173 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:498092df:::obj260:head, version: 111'260, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:498092df:::obj260:head@111'260, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.593+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.593+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4918aa91:::obj303:head local-lis/les=113/114 n=173 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:498092df:::obj260:head@111'260, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.593+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:492a75ed:::obj72:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.593+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4918aa91:::obj303:head local-lis/les=113/114 n=173 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:498092df:::obj260:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.593+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4918aa91:::obj303:head local-lis/les=113/114 n=173 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:498092df:::obj260:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.593+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4918aa91:::obj303:head local-lis/les=113/114 n=173 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:498092df:::obj260:head 2026-02-11T23:31:27.593+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4918aa91:::obj303:head local-lis/les=113/114 n=173 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:498092df:::obj260:head v 111'260 2026-02-11T23:31:27.593+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4918aa91:::obj303:head local-lis/les=113/114 n=173 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.593+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.593+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:498092df:::obj260:head, version: 111'260, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:498092df:::obj260:head@111'260, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.593+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.593+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.593+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:492a75ed:::obj72:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.593+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:492a75ed:::obj72:head) v3 prio 15 cost 0 latency 0.000060 pg_backfill(progress 1.0 e 116/116 lb 1:492a75ed:::obj72:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:4918aa91:::obj303:head local-lis/les=113/114 n=173 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.593+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.593+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4918aa91:::obj303:head local-lis/les=113/114 n=173 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:492a75ed:::obj72:head) v3 2026-02-11T23:31:27.593+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4918aa91:::obj303:head local-lis/les=113/114 n=173 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:492a75ed:::obj72:head) 2026-02-11T23:31:27.593+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.593+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:492a75ed:::obj72:head) v3 finish latency 0.000017 2026-02-11T23:31:27.593+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:492a75ed:::obj72:head local-lis/les=113/114 n=174 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.593+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.593+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.593+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:492a75ed:::obj72:head local-lis/les=113/114 n=174 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.593+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.593+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.593+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.593+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5df9500 con 0x4acb000 2026-02-11T23:31:27.593+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:498092df:::obj260:head)]) -- 0x5fd2c00 con 0x4acb000 2026-02-11T23:31:27.593+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.594+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 385 ==== MOSDPGPush(1.0 116/113 [PushOp(1:4987b93a:::obj56:head, version: 111'56, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4987b93a:::obj56:head@111'56, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x55f1860 con 0x4acb000 2026-02-11T23:31:27.594+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:4987b93a:::obj56:head, version: 111'56, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4987b93a:::obj56:head@111'56, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.594+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4987b93a:::obj56:head, version: 111'56, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4987b93a:::obj56:head@111'56, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.594+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.594+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 386 ==== pg_backfill(progress 1.0 e 116/116 lb 1:498092df:::obj260:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc2b00 con 0x4acb000 2026-02-11T23:31:27.594+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4987b93a:::obj56:head, version: 111'56, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4987b93a:::obj56:head@111'56, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.594+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:498092df:::obj260:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 116 2026-02-11T23:31:27.594+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.594+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:498092df:::obj260:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.594+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4987b93a:::obj56:head, version: 111'56, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4987b93a:::obj56:head@111'56, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.594+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.594+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:4987b93a:::obj56:head, version: 111'56, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4987b93a:::obj56:head@111'56, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 116/113 [PushOp(1:4987b93a:::obj56:head, version: 111'56, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4987b93a:::obj56:head@111'56, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:492a75ed:::obj72:head local-lis/les=113/114 n=174 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.594+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.594+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:492a75ed:::obj72:head local-lis/les=113/114 n=174 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:4987b93a:::obj56:head, version: 111'56, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4987b93a:::obj56:head@111'56, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.594+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:492a75ed:::obj72:head local-lis/les=113/114 n=174 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:4987b93a:::obj56:head@111'56, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.594+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.594+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:498092df:::obj260:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.594+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:492a75ed:::obj72:head local-lis/les=113/114 n=174 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:4987b93a:::obj56:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.594+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:492a75ed:::obj72:head local-lis/les=113/114 n=174 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:4987b93a:::obj56:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.594+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:492a75ed:::obj72:head local-lis/les=113/114 n=174 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:4987b93a:::obj56:head 2026-02-11T23:31:27.594+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:492a75ed:::obj72:head local-lis/les=113/114 n=174 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:4987b93a:::obj56:head v 111'56 2026-02-11T23:31:27.594+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:492a75ed:::obj72:head local-lis/les=113/114 n=174 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.594+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.594+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:4987b93a:::obj56:head, version: 111'56, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4987b93a:::obj56:head@111'56, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.594+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.594+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.594+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:498092df:::obj260:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.594+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:498092df:::obj260:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 1.0 e 116/116 lb 1:498092df:::obj260:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:492a75ed:::obj72:head local-lis/les=113/114 n=174 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.594+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.594+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:492a75ed:::obj72:head local-lis/les=113/114 n=174 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:498092df:::obj260:head) v3 2026-02-11T23:31:27.594+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:492a75ed:::obj72:head local-lis/les=113/114 n=174 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:498092df:::obj260:head) 2026-02-11T23:31:27.594+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.594+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:498092df:::obj260:head) v3 finish latency 0.000010 2026-02-11T23:31:27.594+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:498092df:::obj260:head local-lis/les=113/114 n=175 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.594+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.594+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.594+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:498092df:::obj260:head local-lis/les=113/114 n=175 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.594+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.594+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.594+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.594+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5df9380 con 0x4acb000 2026-02-11T23:31:27.594+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:4987b93a:::obj56:head)]) -- 0x5fd2f00 con 0x4acb000 2026-02-11T23:31:27.594+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.594+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 387 ==== MOSDPGPush(1.0 116/113 [PushOp(1:498e91f7:::obj463:head, version: 111'463, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:498e91f7:::obj463:head@111'463, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f0d00 con 0x4acb000 2026-02-11T23:31:27.594+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:498e91f7:::obj463:head, version: 111'463, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:498e91f7:::obj463:head@111'463, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.594+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:498e91f7:::obj463:head, version: 111'463, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:498e91f7:::obj463:head@111'463, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.594+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.594+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 388 ==== pg_backfill(progress 1.0 e 116/116 lb 1:4987b93a:::obj56:head) ==== 1061+0+0 (unknown 0 0 0) 0x5dc5700 con 0x4acb000 2026-02-11T23:31:27.594+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:4987b93a:::obj56:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.594+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:498e91f7:::obj463:head, version: 111'463, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:498e91f7:::obj463:head@111'463, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.594+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4987b93a:::obj56:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.594+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.594+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:498e91f7:::obj463:head, version: 111'463, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:498e91f7:::obj463:head@111'463, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.594+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:498e91f7:::obj463:head, version: 111'463, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:498e91f7:::obj463:head@111'463, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:498e91f7:::obj463:head, version: 111'463, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:498e91f7:::obj463:head@111'463, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:498092df:::obj260:head local-lis/les=113/114 n=175 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.594+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.594+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.594+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:498092df:::obj260:head local-lis/les=113/114 n=175 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:498e91f7:::obj463:head, version: 111'463, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:498e91f7:::obj463:head@111'463, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.594+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4987b93a:::obj56:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.594+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:498092df:::obj260:head local-lis/les=113/114 n=175 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:498e91f7:::obj463:head@111'463, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.594+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:498092df:::obj260:head local-lis/les=113/114 n=175 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:498e91f7:::obj463:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.594+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:498092df:::obj260:head local-lis/les=113/114 n=175 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:498e91f7:::obj463:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.594+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:498092df:::obj260:head local-lis/les=113/114 n=175 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:498e91f7:::obj463:head 2026-02-11T23:31:27.594+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:498092df:::obj260:head local-lis/les=113/114 n=175 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:498e91f7:::obj463:head v 111'463 2026-02-11T23:31:27.594+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:498092df:::obj260:head local-lis/les=113/114 n=175 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.594+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.594+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:498e91f7:::obj463:head, version: 111'463, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:498e91f7:::obj463:head@111'463, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000044 2026-02-11T23:31:27.594+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.594+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.594+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4987b93a:::obj56:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.594+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:4987b93a:::obj56:head) v3 prio 15 cost 0 latency 0.000060 pg_backfill(progress 1.0 e 116/116 lb 1:4987b93a:::obj56:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:498092df:::obj260:head local-lis/les=113/114 n=175 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.594+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.594+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:498092df:::obj260:head local-lis/les=113/114 n=175 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:4987b93a:::obj56:head) v3 2026-02-11T23:31:27.594+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:498092df:::obj260:head local-lis/les=113/114 n=175 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:4987b93a:::obj56:head) 2026-02-11T23:31:27.594+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.594+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:4987b93a:::obj56:head) v3 finish latency 0.000010 2026-02-11T23:31:27.594+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4987b93a:::obj56:head local-lis/les=113/114 n=176 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.594+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.595+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.595+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4987b93a:::obj56:head local-lis/les=113/114 n=176 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.595+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.595+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.595+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.595+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d79980 con 0x4acb000 2026-02-11T23:31:27.595+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:498e91f7:::obj463:head)]) -- 0x5fd3080 con 0x4acb000 2026-02-11T23:31:27.595+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.595+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 389 ==== MOSDPGPush(1.0 116/113 [PushOp(1:49f89eae:::obj182:head, version: 111'182, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:49f89eae:::obj182:head@111'182, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c000 con 0x4acb000 2026-02-11T23:31:27.595+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:49f89eae:::obj182:head, version: 111'182, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:49f89eae:::obj182:head@111'182, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.595+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:49f89eae:::obj182:head, version: 111'182, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:49f89eae:::obj182:head@111'182, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.595+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.595+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 390 ==== pg_backfill(progress 1.0 e 116/116 lb 1:498e91f7:::obj463:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc2580 con 0x4acb000 2026-02-11T23:31:27.595+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:498e91f7:::obj463:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.595+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:49f89eae:::obj182:head, version: 111'182, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:49f89eae:::obj182:head@111'182, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.595+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:498e91f7:::obj463:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.595+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.595+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:49f89eae:::obj182:head, version: 111'182, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:49f89eae:::obj182:head@111'182, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.595+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.595+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:49f89eae:::obj182:head, version: 111'182, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:49f89eae:::obj182:head@111'182, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:49f89eae:::obj182:head, version: 111'182, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:49f89eae:::obj182:head@111'182, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:4987b93a:::obj56:head local-lis/les=113/114 n=176 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.595+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.595+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4987b93a:::obj56:head local-lis/les=113/114 n=176 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:49f89eae:::obj182:head, version: 111'182, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:49f89eae:::obj182:head@111'182, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.595+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4987b93a:::obj56:head local-lis/les=113/114 n=176 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:49f89eae:::obj182:head@111'182, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.595+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.595+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:498e91f7:::obj463:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.595+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4987b93a:::obj56:head local-lis/les=113/114 n=176 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:49f89eae:::obj182:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.595+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4987b93a:::obj56:head local-lis/les=113/114 n=176 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:49f89eae:::obj182:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.595+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4987b93a:::obj56:head local-lis/les=113/114 n=176 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:49f89eae:::obj182:head 2026-02-11T23:31:27.595+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4987b93a:::obj56:head local-lis/les=113/114 n=176 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:49f89eae:::obj182:head v 111'182 2026-02-11T23:31:27.595+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4987b93a:::obj56:head local-lis/les=113/114 n=176 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.595+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.595+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:49f89eae:::obj182:head, version: 111'182, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:49f89eae:::obj182:head@111'182, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:27.595+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4987b93a:::obj56:head local-lis/les=113/114 n=176 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.595+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.595+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.595+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:498e91f7:::obj463:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.595+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:498e91f7:::obj463:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 1.0 e 116/116 lb 1:498e91f7:::obj463:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:4987b93a:::obj56:head local-lis/les=113/114 n=176 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.595+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.595+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4987b93a:::obj56:head local-lis/les=113/114 n=176 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:498e91f7:::obj463:head) v3 2026-02-11T23:31:27.595+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4987b93a:::obj56:head local-lis/les=113/114 n=176 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:498e91f7:::obj463:head) 2026-02-11T23:31:27.595+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.595+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:498e91f7:::obj463:head) v3 finish latency 0.000011 2026-02-11T23:31:27.595+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.595+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.595+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:498e91f7:::obj463:head local-lis/les=113/114 n=177 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.595+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.595+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.595+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.595+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d79e00 con 0x4acb000 2026-02-11T23:31:27.595+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:49f89eae:::obj182:head)]) -- 0x5d78d80 con 0x4acb000 2026-02-11T23:31:27.595+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.596+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 391 ==== MOSDPGPush(1.0 116/113 [PushOp(1:4b1d3252:::obj86:head, version: 111'86, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b1d3252:::obj86:head@111'86, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x55f1380 con 0x4acb000 2026-02-11T23:31:27.596+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:4b1d3252:::obj86:head, version: 111'86, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b1d3252:::obj86:head@111'86, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.596+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4b1d3252:::obj86:head, version: 111'86, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b1d3252:::obj86:head@111'86, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.596+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.596+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 392 ==== pg_backfill(progress 1.0 e 116/116 lb 1:49f89eae:::obj182:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc3080 con 0x4acb000 2026-02-11T23:31:27.596+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4b1d3252:::obj86:head, version: 111'86, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b1d3252:::obj86:head@111'86, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.596+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:49f89eae:::obj182:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.596+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.596+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:49f89eae:::obj182:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.596+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4b1d3252:::obj86:head, version: 111'86, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b1d3252:::obj86:head@111'86, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.596+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.596+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:4b1d3252:::obj86:head, version: 111'86, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b1d3252:::obj86:head@111'86, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 116/113 [PushOp(1:4b1d3252:::obj86:head, version: 111'86, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b1d3252:::obj86:head@111'86, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:498e91f7:::obj463:head local-lis/les=113/114 n=177 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.596+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.596+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:498e91f7:::obj463:head local-lis/les=113/114 n=177 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:4b1d3252:::obj86:head, version: 111'86, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b1d3252:::obj86:head@111'86, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.596+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:498e91f7:::obj463:head local-lis/les=113/114 n=177 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:4b1d3252:::obj86:head@111'86, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.596+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.596+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:49f89eae:::obj182:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.596+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:498e91f7:::obj463:head local-lis/les=113/114 n=177 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:4b1d3252:::obj86:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.596+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:498e91f7:::obj463:head local-lis/les=113/114 n=177 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:4b1d3252:::obj86:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.596+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:498e91f7:::obj463:head local-lis/les=113/114 n=177 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:4b1d3252:::obj86:head 2026-02-11T23:31:27.596+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:498e91f7:::obj463:head local-lis/les=113/114 n=177 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:4b1d3252:::obj86:head v 111'86 2026-02-11T23:31:27.596+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:498e91f7:::obj463:head local-lis/les=113/114 n=177 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.596+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.596+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:4b1d3252:::obj86:head, version: 111'86, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b1d3252:::obj86:head@111'86, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000044 2026-02-11T23:31:27.596+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.596+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.596+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:49f89eae:::obj182:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.596+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:49f89eae:::obj182:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 1.0 e 116/116 lb 1:49f89eae:::obj182:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:498e91f7:::obj463:head local-lis/les=113/114 n=177 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.596+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.596+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:498e91f7:::obj463:head local-lis/les=113/114 n=177 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:49f89eae:::obj182:head) v3 2026-02-11T23:31:27.596+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:498e91f7:::obj463:head local-lis/les=113/114 n=177 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:49f89eae:::obj182:head) 2026-02-11T23:31:27.596+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.596+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:49f89eae:::obj182:head) v3 finish latency 0.000013 2026-02-11T23:31:27.596+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:49f89eae:::obj182:head local-lis/les=113/114 n=178 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.596+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.596+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.596+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:49f89eae:::obj182:head local-lis/les=113/114 n=178 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.596+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.596+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.596+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.596+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d8e180 con 0x4acb000 2026-02-11T23:31:27.596+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:4b1d3252:::obj86:head)]) -- 0x5fd3380 con 0x4acb000 2026-02-11T23:31:27.596+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.596+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 393 ==== MOSDPGPush(1.0 116/113 [PushOp(1:4b57ab44:::obj335:head, version: 111'335, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b57ab44:::obj335:head@111'335, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559cd00 con 0x4acb000 2026-02-11T23:31:27.596+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:4b57ab44:::obj335:head, version: 111'335, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b57ab44:::obj335:head@111'335, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.596+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4b57ab44:::obj335:head, version: 111'335, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b57ab44:::obj335:head@111'335, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.596+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.596+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 394 ==== pg_backfill(progress 1.0 e 116/116 lb 1:4b1d3252:::obj86:head) ==== 1061+0+0 (unknown 0 0 0) 0x5dc3600 con 0x4acb000 2026-02-11T23:31:27.596+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:4b1d3252:::obj86:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.596+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4b57ab44:::obj335:head, version: 111'335, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b57ab44:::obj335:head@111'335, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.596+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4b1d3252:::obj86:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.596+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.596+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4b1d3252:::obj86:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.596+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.596+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4b57ab44:::obj335:head, version: 111'335, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b57ab44:::obj335:head@111'335, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.596+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:4b57ab44:::obj335:head, version: 111'335, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b57ab44:::obj335:head@111'335, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000041 MOSDPGPush(1.0 116/113 [PushOp(1:4b57ab44:::obj335:head, version: 111'335, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b57ab44:::obj335:head@111'335, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:49f89eae:::obj182:head local-lis/les=113/114 n=178 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.596+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.596+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:49f89eae:::obj182:head local-lis/les=113/114 n=178 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:4b57ab44:::obj335:head, version: 111'335, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b57ab44:::obj335:head@111'335, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.596+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:49f89eae:::obj182:head local-lis/les=113/114 n=178 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:4b57ab44:::obj335:head@111'335, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.596+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:49f89eae:::obj182:head local-lis/les=113/114 n=178 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:4b57ab44:::obj335:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.596+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:49f89eae:::obj182:head local-lis/les=113/114 n=178 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:4b57ab44:::obj335:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.596+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:49f89eae:::obj182:head local-lis/les=113/114 n=178 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:4b57ab44:::obj335:head 2026-02-11T23:31:27.596+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:49f89eae:::obj182:head local-lis/les=113/114 n=178 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:4b57ab44:::obj335:head v 111'335 2026-02-11T23:31:27.596+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:49f89eae:::obj182:head local-lis/les=113/114 n=178 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.596+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.596+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:4b57ab44:::obj335:head, version: 111'335, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b57ab44:::obj335:head@111'335, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.596+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:49f89eae:::obj182:head local-lis/les=113/114 n=178 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.596+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.596+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.596+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4b1d3252:::obj86:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.596+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:4b1d3252:::obj86:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 1.0 e 116/116 lb 1:4b1d3252:::obj86:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:49f89eae:::obj182:head local-lis/les=113/114 n=178 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.596+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.596+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:49f89eae:::obj182:head local-lis/les=113/114 n=178 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:4b1d3252:::obj86:head) v3 2026-02-11T23:31:27.596+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:49f89eae:::obj182:head local-lis/les=113/114 n=178 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:4b1d3252:::obj86:head) 2026-02-11T23:31:27.596+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.596+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:4b1d3252:::obj86:head) v3 finish latency 0.000012 2026-02-11T23:31:27.596+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.596+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.596+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4b1d3252:::obj86:head local-lis/les=113/114 n=179 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.596+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.596+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.596+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.596+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d8e480 con 0x4acb000 2026-02-11T23:31:27.596+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:4b57ab44:::obj335:head)]) -- 0x5d8e900 con 0x4acb000 2026-02-11T23:31:27.596+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.597+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 395 ==== MOSDPGPush(1.0 116/113 [PushOp(1:4b639beb:::obj280:head, version: 111'280, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b639beb:::obj280:head@111'280, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559d1e0 con 0x4acb000 2026-02-11T23:31:27.597+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:4b639beb:::obj280:head, version: 111'280, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b639beb:::obj280:head@111'280, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.597+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4b639beb:::obj280:head, version: 111'280, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b639beb:::obj280:head@111'280, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.597+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.597+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 396 ==== pg_backfill(progress 1.0 e 116/116 lb 1:4b57ab44:::obj335:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc4c00 con 0x4acb000 2026-02-11T23:31:27.597+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4b639beb:::obj280:head, version: 111'280, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b639beb:::obj280:head@111'280, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.597+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:4b57ab44:::obj335:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.597+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.597+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4b57ab44:::obj335:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.597+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4b639beb:::obj280:head, version: 111'280, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b639beb:::obj280:head@111'280, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.597+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.597+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:4b639beb:::obj280:head, version: 111'280, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b639beb:::obj280:head@111'280, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 116/113 [PushOp(1:4b639beb:::obj280:head, version: 111'280, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b639beb:::obj280:head@111'280, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:4b1d3252:::obj86:head local-lis/les=113/114 n=179 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.597+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.597+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4b1d3252:::obj86:head local-lis/les=113/114 n=179 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:4b639beb:::obj280:head, version: 111'280, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b639beb:::obj280:head@111'280, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.597+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4b1d3252:::obj86:head local-lis/les=113/114 n=179 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:4b639beb:::obj280:head@111'280, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.597+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.597+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4b57ab44:::obj335:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.597+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4b1d3252:::obj86:head local-lis/les=113/114 n=179 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:4b639beb:::obj280:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.597+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4b1d3252:::obj86:head local-lis/les=113/114 n=179 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:4b639beb:::obj280:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.597+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4b1d3252:::obj86:head local-lis/les=113/114 n=179 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:4b639beb:::obj280:head 2026-02-11T23:31:27.597+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4b1d3252:::obj86:head local-lis/les=113/114 n=179 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:4b639beb:::obj280:head v 111'280 2026-02-11T23:31:27.597+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4b1d3252:::obj86:head local-lis/les=113/114 n=179 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.597+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.597+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:4b639beb:::obj280:head, version: 111'280, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b639beb:::obj280:head@111'280, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.597+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.597+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.597+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4b57ab44:::obj335:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.597+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:4b57ab44:::obj335:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:4b57ab44:::obj335:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:4b1d3252:::obj86:head local-lis/les=113/114 n=179 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.597+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.597+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4b1d3252:::obj86:head local-lis/les=113/114 n=179 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:4b57ab44:::obj335:head) v3 2026-02-11T23:31:27.597+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4b1d3252:::obj86:head local-lis/les=113/114 n=179 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:4b57ab44:::obj335:head) 2026-02-11T23:31:27.597+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.597+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:4b57ab44:::obj335:head) v3 finish latency 0.000011 2026-02-11T23:31:27.597+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4b57ab44:::obj335:head local-lis/les=113/114 n=180 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.597+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.597+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.597+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4b57ab44:::obj335:head local-lis/les=113/114 n=180 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.597+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.597+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.597+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.597+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d8f200 con 0x4acb000 2026-02-11T23:31:27.597+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:4b639beb:::obj280:head)]) -- 0x5fd3500 con 0x4acb000 2026-02-11T23:31:27.597+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.597+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 397 ==== MOSDPGPush(1.0 116/113 [PushOp(1:4baa8bb7:::obj594:head, version: 111'594, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4baa8bb7:::obj594:head@111'594, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c340 con 0x4acb000 2026-02-11T23:31:27.597+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:4baa8bb7:::obj594:head, version: 111'594, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4baa8bb7:::obj594:head@111'594, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.597+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4baa8bb7:::obj594:head, version: 111'594, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4baa8bb7:::obj594:head@111'594, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.597+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.597+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 398 ==== pg_backfill(progress 1.0 e 116/116 lb 1:4b639beb:::obj280:head) ==== 1062+0+0 (unknown 0 0 0) 0x5dc4680 con 0x4acb000 2026-02-11T23:31:27.597+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4baa8bb7:::obj594:head, version: 111'594, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4baa8bb7:::obj594:head@111'594, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.597+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:4b639beb:::obj280:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.597+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.597+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4baa8bb7:::obj594:head, version: 111'594, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4baa8bb7:::obj594:head@111'594, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.597+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4b639beb:::obj280:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.597+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.597+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:4baa8bb7:::obj594:head, version: 111'594, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4baa8bb7:::obj594:head@111'594, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 116/113 [PushOp(1:4baa8bb7:::obj594:head, version: 111'594, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4baa8bb7:::obj594:head@111'594, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:4b57ab44:::obj335:head local-lis/les=113/114 n=180 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.597+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.597+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4b57ab44:::obj335:head local-lis/les=113/114 n=180 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:4baa8bb7:::obj594:head, version: 111'594, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4baa8bb7:::obj594:head@111'594, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.597+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4b57ab44:::obj335:head local-lis/les=113/114 n=180 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:4baa8bb7:::obj594:head@111'594, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.597+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.597+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4b639beb:::obj280:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.597+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4b57ab44:::obj335:head local-lis/les=113/114 n=180 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:4baa8bb7:::obj594:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.597+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4b57ab44:::obj335:head local-lis/les=113/114 n=180 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:4baa8bb7:::obj594:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.597+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4b57ab44:::obj335:head local-lis/les=113/114 n=180 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:4baa8bb7:::obj594:head 2026-02-11T23:31:27.597+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4b57ab44:::obj335:head local-lis/les=113/114 n=180 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:4baa8bb7:::obj594:head v 111'594 2026-02-11T23:31:27.597+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4b57ab44:::obj335:head local-lis/les=113/114 n=180 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.597+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.597+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:4baa8bb7:::obj594:head, version: 111'594, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4baa8bb7:::obj594:head@111'594, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000043 2026-02-11T23:31:27.597+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.597+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.597+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4b639beb:::obj280:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.597+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:4b639beb:::obj280:head) v3 prio 15 cost 0 latency 0.000058 pg_backfill(progress 1.0 e 116/116 lb 1:4b639beb:::obj280:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:4b57ab44:::obj335:head local-lis/les=113/114 n=180 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.597+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.597+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4b57ab44:::obj335:head local-lis/les=113/114 n=180 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:4b639beb:::obj280:head) v3 2026-02-11T23:31:27.597+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4b57ab44:::obj335:head local-lis/les=113/114 n=180 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:4b639beb:::obj280:head) 2026-02-11T23:31:27.597+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.597+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:4b639beb:::obj280:head) v3 finish latency 0.000018 2026-02-11T23:31:27.597+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4b639beb:::obj280:head local-lis/les=113/114 n=181 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.597+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.598+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.598+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4b639beb:::obj280:head local-lis/les=113/114 n=181 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.598+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.598+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.598+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.598+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d8f500 con 0x4acb000 2026-02-11T23:31:27.598+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:4baa8bb7:::obj594:head)]) -- 0x5fd3800 con 0x4acb000 2026-02-11T23:31:27.598+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.598+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 399 ==== MOSDPGPush(1.0 116/113 [PushOp(1:4c3f91c8:::obj470:head, version: 111'470, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c3f91c8:::obj470:head@111'470, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c820 con 0x4acb000 2026-02-11T23:31:27.598+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:4c3f91c8:::obj470:head, version: 111'470, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c3f91c8:::obj470:head@111'470, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.598+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4c3f91c8:::obj470:head, version: 111'470, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c3f91c8:::obj470:head@111'470, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.598+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.598+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 400 ==== pg_backfill(progress 1.0 e 116/116 lb 1:4baa8bb7:::obj594:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98b00 con 0x4acb000 2026-02-11T23:31:27.598+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:4baa8bb7:::obj594:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.598+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4c3f91c8:::obj470:head, version: 111'470, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c3f91c8:::obj470:head@111'470, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.598+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4baa8bb7:::obj594:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.598+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.598+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4c3f91c8:::obj470:head, version: 111'470, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c3f91c8:::obj470:head@111'470, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.598+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:4c3f91c8:::obj470:head, version: 111'470, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c3f91c8:::obj470:head@111'470, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 116/113 [PushOp(1:4c3f91c8:::obj470:head, version: 111'470, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c3f91c8:::obj470:head@111'470, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:4b639beb:::obj280:head local-lis/les=113/114 n=181 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.598+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.598+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4b639beb:::obj280:head local-lis/les=113/114 n=181 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:4c3f91c8:::obj470:head, version: 111'470, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c3f91c8:::obj470:head@111'470, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.598+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.598+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4baa8bb7:::obj594:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.598+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4b639beb:::obj280:head local-lis/les=113/114 n=181 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:4c3f91c8:::obj470:head@111'470, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.598+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4b639beb:::obj280:head local-lis/les=113/114 n=181 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:4c3f91c8:::obj470:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.598+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4b639beb:::obj280:head local-lis/les=113/114 n=181 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:4c3f91c8:::obj470:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.598+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4b639beb:::obj280:head local-lis/les=113/114 n=181 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:4c3f91c8:::obj470:head 2026-02-11T23:31:27.598+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4b639beb:::obj280:head local-lis/les=113/114 n=181 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:4c3f91c8:::obj470:head v 111'470 2026-02-11T23:31:27.598+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4b639beb:::obj280:head local-lis/les=113/114 n=181 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.598+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.598+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:4c3f91c8:::obj470:head, version: 111'470, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c3f91c8:::obj470:head@111'470, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:27.598+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.598+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.598+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4baa8bb7:::obj594:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.598+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:4baa8bb7:::obj594:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 1.0 e 116/116 lb 1:4baa8bb7:::obj594:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:4b639beb:::obj280:head local-lis/les=113/114 n=181 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.598+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.598+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4b639beb:::obj280:head local-lis/les=113/114 n=181 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:4baa8bb7:::obj594:head) v3 2026-02-11T23:31:27.598+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4b639beb:::obj280:head local-lis/les=113/114 n=181 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:4baa8bb7:::obj594:head) 2026-02-11T23:31:27.598+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.598+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:4baa8bb7:::obj594:head) v3 finish latency 0.000010 2026-02-11T23:31:27.598+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4baa8bb7:::obj594:head local-lis/les=113/114 n=182 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.598+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.598+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.598+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4baa8bb7:::obj594:head local-lis/les=113/114 n=182 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.598+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.598+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.598+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.598+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d8fb00 con 0x4acb000 2026-02-11T23:31:27.598+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:4c3f91c8:::obj470:head)]) -- 0x5fd3b00 con 0x4acb000 2026-02-11T23:31:27.598+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.599+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 401 ==== MOSDPGPush(1.0 116/113 [PushOp(1:4c59dc70:::obj344:head, version: 111'344, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c59dc70:::obj344:head@111'344, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559dd40 con 0x4acb000 2026-02-11T23:31:27.599+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:4c59dc70:::obj344:head, version: 111'344, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c59dc70:::obj344:head@111'344, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.599+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4c59dc70:::obj344:head, version: 111'344, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c59dc70:::obj344:head@111'344, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.599+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.599+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 402 ==== pg_backfill(progress 1.0 e 116/116 lb 1:4c3f91c8:::obj470:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99080 con 0x4acb000 2026-02-11T23:31:27.599+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4c59dc70:::obj344:head, version: 111'344, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c59dc70:::obj344:head@111'344, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.599+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:4c3f91c8:::obj470:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.599+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4c3f91c8:::obj470:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.599+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.599+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4c59dc70:::obj344:head, version: 111'344, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c59dc70:::obj344:head@111'344, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.599+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.599+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:4c59dc70:::obj344:head, version: 111'344, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c59dc70:::obj344:head@111'344, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 116/113 [PushOp(1:4c59dc70:::obj344:head, version: 111'344, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c59dc70:::obj344:head@111'344, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:4baa8bb7:::obj594:head local-lis/les=113/114 n=182 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.599+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.599+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4baa8bb7:::obj594:head local-lis/les=113/114 n=182 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:4c59dc70:::obj344:head, version: 111'344, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c59dc70:::obj344:head@111'344, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.599+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.599+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4baa8bb7:::obj594:head local-lis/les=113/114 n=182 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:4c59dc70:::obj344:head@111'344, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.599+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4c3f91c8:::obj470:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.599+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4baa8bb7:::obj594:head local-lis/les=113/114 n=182 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:4c59dc70:::obj344:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.599+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4baa8bb7:::obj594:head local-lis/les=113/114 n=182 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:4c59dc70:::obj344:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.599+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4baa8bb7:::obj594:head local-lis/les=113/114 n=182 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:4c59dc70:::obj344:head 2026-02-11T23:31:27.599+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4baa8bb7:::obj594:head local-lis/les=113/114 n=182 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:4c59dc70:::obj344:head v 111'344 2026-02-11T23:31:27.599+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4baa8bb7:::obj594:head local-lis/les=113/114 n=182 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.599+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.599+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:4c59dc70:::obj344:head, version: 111'344, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c59dc70:::obj344:head@111'344, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:27.599+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.599+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.599+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4c3f91c8:::obj470:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.599+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:4c3f91c8:::obj470:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 1.0 e 116/116 lb 1:4c3f91c8:::obj470:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:4baa8bb7:::obj594:head local-lis/les=113/114 n=182 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.599+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.599+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4baa8bb7:::obj594:head local-lis/les=113/114 n=182 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:4c3f91c8:::obj470:head) v3 2026-02-11T23:31:27.599+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4baa8bb7:::obj594:head local-lis/les=113/114 n=182 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:4c3f91c8:::obj470:head) 2026-02-11T23:31:27.599+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.599+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:4c3f91c8:::obj470:head) v3 finish latency 0.000011 2026-02-11T23:31:27.599+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c3f91c8:::obj470:head local-lis/les=113/114 n=183 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.599+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.599+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.599+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c3f91c8:::obj470:head local-lis/les=113/114 n=183 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.599+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.599+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.599+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.599+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x4ec2780 con 0x4acb000 2026-02-11T23:31:27.599+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:4c59dc70:::obj344:head)]) -- 0x5daa300 con 0x4acb000 2026-02-11T23:31:27.599+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.599+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 403 ==== MOSDPGPush(1.0 116/113 [PushOp(1:4c73ba95:::obj68:head, version: 111'68, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c73ba95:::obj68:head@111'68, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x55d8680 con 0x4acb000 2026-02-11T23:31:27.599+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:4c73ba95:::obj68:head, version: 111'68, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c73ba95:::obj68:head@111'68, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.599+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4c73ba95:::obj68:head, version: 111'68, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c73ba95:::obj68:head@111'68, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.599+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.599+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 404 ==== pg_backfill(progress 1.0 e 116/116 lb 1:4c59dc70:::obj344:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99600 con 0x4acb000 2026-02-11T23:31:27.599+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:4c59dc70:::obj344:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.599+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4c73ba95:::obj68:head, version: 111'68, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c73ba95:::obj68:head@111'68, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.599+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.599+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4c59dc70:::obj344:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.599+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4c73ba95:::obj68:head, version: 111'68, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c73ba95:::obj68:head@111'68, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.599+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.599+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:4c73ba95:::obj68:head, version: 111'68, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c73ba95:::obj68:head@111'68, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:4c73ba95:::obj68:head, version: 111'68, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c73ba95:::obj68:head@111'68, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:4c3f91c8:::obj470:head local-lis/les=113/114 n=183 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.599+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.599+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c3f91c8:::obj470:head local-lis/les=113/114 n=183 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:4c73ba95:::obj68:head, version: 111'68, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c73ba95:::obj68:head@111'68, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.599+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c3f91c8:::obj470:head local-lis/les=113/114 n=183 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:4c73ba95:::obj68:head@111'68, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.599+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.599+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4c59dc70:::obj344:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.599+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c3f91c8:::obj470:head local-lis/les=113/114 n=183 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:4c73ba95:::obj68:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.599+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c3f91c8:::obj470:head local-lis/les=113/114 n=183 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:4c73ba95:::obj68:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.599+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c3f91c8:::obj470:head local-lis/les=113/114 n=183 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:4c73ba95:::obj68:head 2026-02-11T23:31:27.599+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c3f91c8:::obj470:head local-lis/les=113/114 n=183 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:4c73ba95:::obj68:head v 111'68 2026-02-11T23:31:27.599+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c3f91c8:::obj470:head local-lis/les=113/114 n=183 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.599+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.599+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:4c73ba95:::obj68:head, version: 111'68, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c73ba95:::obj68:head@111'68, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.599+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.599+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.599+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4c59dc70:::obj344:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.599+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:4c59dc70:::obj344:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 1.0 e 116/116 lb 1:4c59dc70:::obj344:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:4c3f91c8:::obj470:head local-lis/les=113/114 n=183 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.599+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.599+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c3f91c8:::obj470:head local-lis/les=113/114 n=183 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:4c59dc70:::obj344:head) v3 2026-02-11T23:31:27.599+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c3f91c8:::obj470:head local-lis/les=113/114 n=183 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:4c59dc70:::obj344:head) 2026-02-11T23:31:27.599+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.599+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:4c59dc70:::obj344:head) v3 finish latency 0.000010 2026-02-11T23:31:27.599+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c59dc70:::obj344:head local-lis/les=113/114 n=184 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.599+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.599+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.599+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c59dc70:::obj344:head local-lis/les=113/114 n=184 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.599+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.599+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.599+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.599+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fd2000 con 0x4acb000 2026-02-11T23:31:27.599+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:4c73ba95:::obj68:head)]) -- 0x5daa900 con 0x4acb000 2026-02-11T23:31:27.599+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.600+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 405 ==== MOSDPGPush(1.0 116/113 [PushOp(1:4c7e2cfa:::obj454:head, version: 111'454, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c7e2cfa:::obj454:head@111'454, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559d6c0 con 0x4acb000 2026-02-11T23:31:27.600+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:4c7e2cfa:::obj454:head, version: 111'454, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c7e2cfa:::obj454:head@111'454, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.600+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4c7e2cfa:::obj454:head, version: 111'454, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c7e2cfa:::obj454:head@111'454, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.600+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.600+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4c7e2cfa:::obj454:head, version: 111'454, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c7e2cfa:::obj454:head@111'454, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.600+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.600+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 406 ==== pg_backfill(progress 1.0 e 116/116 lb 1:4c73ba95:::obj68:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d99b80 con 0x4acb000 2026-02-11T23:31:27.600+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4c7e2cfa:::obj454:head, version: 111'454, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c7e2cfa:::obj454:head@111'454, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.600+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:4c73ba95:::obj68:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.600+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4c73ba95:::obj68:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.600+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.600+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:4c7e2cfa:::obj454:head, version: 111'454, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c7e2cfa:::obj454:head@111'454, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 116/113 [PushOp(1:4c7e2cfa:::obj454:head, version: 111'454, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c7e2cfa:::obj454:head@111'454, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:4c59dc70:::obj344:head local-lis/les=113/114 n=184 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.600+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.600+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c59dc70:::obj344:head local-lis/les=113/114 n=184 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:4c7e2cfa:::obj454:head, version: 111'454, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c7e2cfa:::obj454:head@111'454, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.600+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.600+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c59dc70:::obj344:head local-lis/les=113/114 n=184 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:4c7e2cfa:::obj454:head@111'454, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.600+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4c73ba95:::obj68:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.600+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c59dc70:::obj344:head local-lis/les=113/114 n=184 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:4c7e2cfa:::obj454:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.600+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c59dc70:::obj344:head local-lis/les=113/114 n=184 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:4c7e2cfa:::obj454:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.600+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c59dc70:::obj344:head local-lis/les=113/114 n=184 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:4c7e2cfa:::obj454:head 2026-02-11T23:31:27.600+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c59dc70:::obj344:head local-lis/les=113/114 n=184 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:4c7e2cfa:::obj454:head v 111'454 2026-02-11T23:31:27.600+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c59dc70:::obj344:head local-lis/les=113/114 n=184 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.600+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.600+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:4c7e2cfa:::obj454:head, version: 111'454, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c7e2cfa:::obj454:head@111'454, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:27.600+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.600+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.600+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4c73ba95:::obj68:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.600+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:4c73ba95:::obj68:head) v3 prio 15 cost 0 latency 0.000059 pg_backfill(progress 1.0 e 116/116 lb 1:4c73ba95:::obj68:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:4c59dc70:::obj344:head local-lis/les=113/114 n=184 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.600+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.600+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c59dc70:::obj344:head local-lis/les=113/114 n=184 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:4c73ba95:::obj68:head) v3 2026-02-11T23:31:27.600+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c59dc70:::obj344:head local-lis/les=113/114 n=184 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:4c73ba95:::obj68:head) 2026-02-11T23:31:27.600+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.600+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:4c73ba95:::obj68:head) v3 finish latency 0.000009 2026-02-11T23:31:27.600+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c73ba95:::obj68:head local-lis/les=113/114 n=185 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.600+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.600+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.600+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c73ba95:::obj68:head local-lis/les=113/114 n=185 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.600+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.600+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.600+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.600+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fd3c80 con 0x4acb000 2026-02-11T23:31:27.600+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:4c7e2cfa:::obj454:head)]) -- 0x5e7ed80 con 0x4acb000 2026-02-11T23:31:27.600+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.601+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 407 ==== MOSDPGPush(1.0 116/113 [PushOp(1:4c9f7fa8:::obj477:head, version: 111'477, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c9f7fa8:::obj477:head@111'477, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d8000 con 0x4acb000 2026-02-11T23:31:27.601+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:4c9f7fa8:::obj477:head, version: 111'477, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c9f7fa8:::obj477:head@111'477, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.601+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4c9f7fa8:::obj477:head, version: 111'477, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c9f7fa8:::obj477:head@111'477, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.601+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.601+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 408 ==== pg_backfill(progress 1.0 e 116/116 lb 1:4c7e2cfa:::obj454:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9a100 con 0x4acb000 2026-02-11T23:31:27.601+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:4c7e2cfa:::obj454:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.601+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4c9f7fa8:::obj477:head, version: 111'477, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c9f7fa8:::obj477:head@111'477, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.601+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4c7e2cfa:::obj454:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.601+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.601+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4c9f7fa8:::obj477:head, version: 111'477, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c9f7fa8:::obj477:head@111'477, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.601+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:4c9f7fa8:::obj477:head, version: 111'477, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c9f7fa8:::obj477:head@111'477, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:4c9f7fa8:::obj477:head, version: 111'477, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c9f7fa8:::obj477:head@111'477, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:4c73ba95:::obj68:head local-lis/les=113/114 n=185 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.601+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.601+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.601+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c73ba95:::obj68:head local-lis/les=113/114 n=185 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:4c9f7fa8:::obj477:head, version: 111'477, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c9f7fa8:::obj477:head@111'477, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.601+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4c7e2cfa:::obj454:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.601+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c73ba95:::obj68:head local-lis/les=113/114 n=185 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:4c9f7fa8:::obj477:head@111'477, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.601+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c73ba95:::obj68:head local-lis/les=113/114 n=185 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:4c9f7fa8:::obj477:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.601+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c73ba95:::obj68:head local-lis/les=113/114 n=185 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:4c9f7fa8:::obj477:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.601+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c73ba95:::obj68:head local-lis/les=113/114 n=185 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:4c9f7fa8:::obj477:head 2026-02-11T23:31:27.601+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c73ba95:::obj68:head local-lis/les=113/114 n=185 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:4c9f7fa8:::obj477:head v 111'477 2026-02-11T23:31:27.601+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c73ba95:::obj68:head local-lis/les=113/114 n=185 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.601+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.601+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:4c9f7fa8:::obj477:head, version: 111'477, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c9f7fa8:::obj477:head@111'477, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:27.601+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.601+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.601+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4c7e2cfa:::obj454:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.601+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:4c7e2cfa:::obj454:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 116/116 lb 1:4c7e2cfa:::obj454:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:4c73ba95:::obj68:head local-lis/les=113/114 n=185 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.601+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.601+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c73ba95:::obj68:head local-lis/les=113/114 n=185 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:4c7e2cfa:::obj454:head) v3 2026-02-11T23:31:27.601+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c73ba95:::obj68:head local-lis/les=113/114 n=185 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:4c7e2cfa:::obj454:head) 2026-02-11T23:31:27.601+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.601+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:4c7e2cfa:::obj454:head) v3 finish latency 0.000010 2026-02-11T23:31:27.601+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c7e2cfa:::obj454:head local-lis/les=113/114 n=186 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.601+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.601+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.601+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c7e2cfa:::obj454:head local-lis/les=113/114 n=186 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.601+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.601+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.601+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.601+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fd3e00 con 0x4acb000 2026-02-11T23:31:27.601+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:4c9f7fa8:::obj477:head)]) -- 0x5e0b200 con 0x4acb000 2026-02-11T23:31:27.601+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.601+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 409 ==== MOSDPGPush(1.0 116/113 [PushOp(1:4dc3027d:::obj550:head, version: 111'550, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4dc3027d:::obj550:head@111'550, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d84e0 con 0x4acb000 2026-02-11T23:31:27.601+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:4dc3027d:::obj550:head, version: 111'550, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4dc3027d:::obj550:head@111'550, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.601+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4dc3027d:::obj550:head, version: 111'550, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4dc3027d:::obj550:head@111'550, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.601+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.601+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 410 ==== pg_backfill(progress 1.0 e 116/116 lb 1:4c9f7fa8:::obj477:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9a680 con 0x4acb000 2026-02-11T23:31:27.601+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4dc3027d:::obj550:head, version: 111'550, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4dc3027d:::obj550:head@111'550, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.601+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:4c9f7fa8:::obj477:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.601+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4c9f7fa8:::obj477:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.601+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.601+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4dc3027d:::obj550:head, version: 111'550, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4dc3027d:::obj550:head@111'550, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.601+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:4dc3027d:::obj550:head, version: 111'550, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4dc3027d:::obj550:head@111'550, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:4dc3027d:::obj550:head, version: 111'550, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4dc3027d:::obj550:head@111'550, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:4c7e2cfa:::obj454:head local-lis/les=113/114 n=186 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.601+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.601+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.601+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c7e2cfa:::obj454:head local-lis/les=113/114 n=186 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:4dc3027d:::obj550:head, version: 111'550, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4dc3027d:::obj550:head@111'550, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.601+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4c9f7fa8:::obj477:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.601+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c7e2cfa:::obj454:head local-lis/les=113/114 n=186 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:4dc3027d:::obj550:head@111'550, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.601+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c7e2cfa:::obj454:head local-lis/les=113/114 n=186 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:4dc3027d:::obj550:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.601+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c7e2cfa:::obj454:head local-lis/les=113/114 n=186 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:4dc3027d:::obj550:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.601+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c7e2cfa:::obj454:head local-lis/les=113/114 n=186 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:4dc3027d:::obj550:head 2026-02-11T23:31:27.601+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c7e2cfa:::obj454:head local-lis/les=113/114 n=186 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:4dc3027d:::obj550:head v 111'550 2026-02-11T23:31:27.601+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c7e2cfa:::obj454:head local-lis/les=113/114 n=186 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.601+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.601+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:4dc3027d:::obj550:head, version: 111'550, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4dc3027d:::obj550:head@111'550, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000044 2026-02-11T23:31:27.601+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.601+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.601+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4c9f7fa8:::obj477:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.601+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:4c9f7fa8:::obj477:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:4c9f7fa8:::obj477:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:4c7e2cfa:::obj454:head local-lis/les=113/114 n=186 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.601+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.601+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c7e2cfa:::obj454:head local-lis/les=113/114 n=186 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:4c9f7fa8:::obj477:head) v3 2026-02-11T23:31:27.601+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c7e2cfa:::obj454:head local-lis/les=113/114 n=186 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:4c9f7fa8:::obj477:head) 2026-02-11T23:31:27.601+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.601+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:4c9f7fa8:::obj477:head) v3 finish latency 0.000010 2026-02-11T23:31:27.601+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c9f7fa8:::obj477:head local-lis/les=113/114 n=187 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.601+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.601+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.601+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c9f7fa8:::obj477:head local-lis/les=113/114 n=187 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.601+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.601+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.601+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.601+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5df9980 con 0x4acb000 2026-02-11T23:31:27.601+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:4dc3027d:::obj550:head)]) -- 0x5e0b800 con 0x4acb000 2026-02-11T23:31:27.601+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.602+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 411 ==== MOSDPGPush(1.0 116/113 [PushOp(1:4dd77094:::obj38:head, version: 111'38, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4dd77094:::obj38:head@111'38, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5647380 con 0x4acb000 2026-02-11T23:31:27.602+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:4dd77094:::obj38:head, version: 111'38, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4dd77094:::obj38:head@111'38, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.602+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4dd77094:::obj38:head, version: 111'38, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4dd77094:::obj38:head@111'38, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.602+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.602+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 412 ==== pg_backfill(progress 1.0 e 116/116 lb 1:4dc3027d:::obj550:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9ac00 con 0x4acb000 2026-02-11T23:31:27.602+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:4dc3027d:::obj550:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.602+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4dd77094:::obj38:head, version: 111'38, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4dd77094:::obj38:head@111'38, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.602+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4dc3027d:::obj550:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.602+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.602+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4dc3027d:::obj550:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.602+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.602+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4dd77094:::obj38:head, version: 111'38, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4dd77094:::obj38:head@111'38, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.602+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:4dd77094:::obj38:head, version: 111'38, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4dd77094:::obj38:head@111'38, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(1.0 116/113 [PushOp(1:4dd77094:::obj38:head, version: 111'38, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4dd77094:::obj38:head@111'38, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:4c9f7fa8:::obj477:head local-lis/les=113/114 n=187 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.602+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.602+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c9f7fa8:::obj477:head local-lis/les=113/114 n=187 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:4dd77094:::obj38:head, version: 111'38, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4dd77094:::obj38:head@111'38, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.602+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c9f7fa8:::obj477:head local-lis/les=113/114 n=187 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:4dd77094:::obj38:head@111'38, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.602+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c9f7fa8:::obj477:head local-lis/les=113/114 n=187 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:4dd77094:::obj38:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.602+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c9f7fa8:::obj477:head local-lis/les=113/114 n=187 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:4dd77094:::obj38:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.602+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c9f7fa8:::obj477:head local-lis/les=113/114 n=187 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:4dd77094:::obj38:head 2026-02-11T23:31:27.602+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c9f7fa8:::obj477:head local-lis/les=113/114 n=187 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:4dd77094:::obj38:head v 111'38 2026-02-11T23:31:27.602+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c9f7fa8:::obj477:head local-lis/les=113/114 n=187 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.602+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.602+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:4dd77094:::obj38:head, version: 111'38, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4dd77094:::obj38:head@111'38, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:27.602+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c9f7fa8:::obj477:head local-lis/les=113/114 n=187 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.602+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.602+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.602+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4dc3027d:::obj550:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.602+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:4dc3027d:::obj550:head) v3 prio 15 cost 0 latency 0.000080 pg_backfill(progress 1.0 e 116/116 lb 1:4dc3027d:::obj550:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:4c9f7fa8:::obj477:head local-lis/les=113/114 n=187 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.602+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.602+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c9f7fa8:::obj477:head local-lis/les=113/114 n=187 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:4dc3027d:::obj550:head) v3 2026-02-11T23:31:27.602+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4c9f7fa8:::obj477:head local-lis/les=113/114 n=187 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:4dc3027d:::obj550:head) 2026-02-11T23:31:27.602+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.602+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:4dc3027d:::obj550:head) v3 finish latency 0.000011 2026-02-11T23:31:27.602+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.602+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.602+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4dc3027d:::obj550:head local-lis/les=113/114 n=188 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.602+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.602+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.602+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.602+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5daac00 con 0x4acb000 2026-02-11T23:31:27.602+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:4dd77094:::obj38:head)]) -- 0x5e6b980 con 0x4acb000 2026-02-11T23:31:27.602+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.602+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 413 ==== MOSDPGPush(1.0 116/113 [PushOp(1:4eb30133:::obj30:head, version: 111'30, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4eb30133:::obj30:head@111'30, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x55d8ea0 con 0x4acb000 2026-02-11T23:31:27.602+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:4eb30133:::obj30:head, version: 111'30, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4eb30133:::obj30:head@111'30, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.602+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4eb30133:::obj30:head, version: 111'30, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4eb30133:::obj30:head@111'30, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.602+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.602+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 414 ==== pg_backfill(progress 1.0 e 116/116 lb 1:4dd77094:::obj38:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d9b180 con 0x4acb000 2026-02-11T23:31:27.602+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:4dd77094:::obj38:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.602+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4eb30133:::obj30:head, version: 111'30, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4eb30133:::obj30:head@111'30, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.602+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4dd77094:::obj38:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.602+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.602+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4dd77094:::obj38:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.602+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.603+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4eb30133:::obj30:head, version: 111'30, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4eb30133:::obj30:head@111'30, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.603+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:4eb30133:::obj30:head, version: 111'30, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4eb30133:::obj30:head@111'30, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000044 MOSDPGPush(1.0 116/113 [PushOp(1:4eb30133:::obj30:head, version: 111'30, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4eb30133:::obj30:head@111'30, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:4dc3027d:::obj550:head local-lis/les=113/114 n=188 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.603+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.603+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4dc3027d:::obj550:head local-lis/les=113/114 n=188 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:4eb30133:::obj30:head, version: 111'30, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4eb30133:::obj30:head@111'30, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.603+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4dc3027d:::obj550:head local-lis/les=113/114 n=188 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:4eb30133:::obj30:head@111'30, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.603+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4dc3027d:::obj550:head local-lis/les=113/114 n=188 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:4eb30133:::obj30:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.603+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4dc3027d:::obj550:head local-lis/les=113/114 n=188 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:4eb30133:::obj30:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.603+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4dc3027d:::obj550:head local-lis/les=113/114 n=188 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:4eb30133:::obj30:head 2026-02-11T23:31:27.603+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4dc3027d:::obj550:head local-lis/les=113/114 n=188 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:4eb30133:::obj30:head v 111'30 2026-02-11T23:31:27.603+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4dc3027d:::obj550:head local-lis/les=113/114 n=188 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.603+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.603+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:4eb30133:::obj30:head, version: 111'30, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4eb30133:::obj30:head@111'30, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:27.603+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.603+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.603+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4dd77094:::obj38:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.603+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:4dd77094:::obj38:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 1.0 e 116/116 lb 1:4dd77094:::obj38:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:4dc3027d:::obj550:head local-lis/les=113/114 n=188 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.603+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.603+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4dc3027d:::obj550:head local-lis/les=113/114 n=188 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:4dd77094:::obj38:head) v3 2026-02-11T23:31:27.603+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4dc3027d:::obj550:head local-lis/les=113/114 n=188 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:4dd77094:::obj38:head) 2026-02-11T23:31:27.603+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.603+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:4dd77094:::obj38:head) v3 finish latency 0.000011 2026-02-11T23:31:27.603+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4dd77094:::obj38:head local-lis/les=113/114 n=189 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.603+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.603+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.603+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4dd77094:::obj38:head local-lis/les=113/114 n=189 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.603+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.603+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.603+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.603+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5daaf00 con 0x4acb000 2026-02-11T23:31:27.603+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:4eb30133:::obj30:head)]) -- 0x5e0bb00 con 0x4acb000 2026-02-11T23:31:27.603+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.603+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 415 ==== MOSDPGPush(1.0 116/113 [PushOp(1:4fbee9e6:::obj123:head, version: 111'123, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4fbee9e6:::obj123:head@111'123, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d9380 con 0x4acb000 2026-02-11T23:31:27.603+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:4fbee9e6:::obj123:head, version: 111'123, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4fbee9e6:::obj123:head@111'123, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.603+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4fbee9e6:::obj123:head, version: 111'123, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4fbee9e6:::obj123:head@111'123, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.603+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.603+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 416 ==== pg_backfill(progress 1.0 e 116/116 lb 1:4eb30133:::obj30:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d98000 con 0x4acb000 2026-02-11T23:31:27.603+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:4eb30133:::obj30:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 116 2026-02-11T23:31:27.603+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4fbee9e6:::obj123:head, version: 111'123, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4fbee9e6:::obj123:head@111'123, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.603+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4eb30133:::obj30:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.603+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.603+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4fbee9e6:::obj123:head, version: 111'123, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4fbee9e6:::obj123:head@111'123, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.603+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:4fbee9e6:::obj123:head, version: 111'123, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4fbee9e6:::obj123:head@111'123, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 116/113 [PushOp(1:4fbee9e6:::obj123:head, version: 111'123, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4fbee9e6:::obj123:head@111'123, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:4dd77094:::obj38:head local-lis/les=113/114 n=189 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.603+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.603+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.603+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4dd77094:::obj38:head local-lis/les=113/114 n=189 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:4fbee9e6:::obj123:head, version: 111'123, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4fbee9e6:::obj123:head@111'123, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.603+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4eb30133:::obj30:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.603+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4dd77094:::obj38:head local-lis/les=113/114 n=189 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:4fbee9e6:::obj123:head@111'123, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.603+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4dd77094:::obj38:head local-lis/les=113/114 n=189 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:4fbee9e6:::obj123:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.603+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4dd77094:::obj38:head local-lis/les=113/114 n=189 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:4fbee9e6:::obj123:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.603+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4dd77094:::obj38:head local-lis/les=113/114 n=189 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:4fbee9e6:::obj123:head 2026-02-11T23:31:27.603+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4dd77094:::obj38:head local-lis/les=113/114 n=189 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:4fbee9e6:::obj123:head v 111'123 2026-02-11T23:31:27.603+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4dd77094:::obj38:head local-lis/les=113/114 n=189 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.603+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.603+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:4fbee9e6:::obj123:head, version: 111'123, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4fbee9e6:::obj123:head@111'123, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.603+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.603+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.603+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4eb30133:::obj30:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.603+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:4eb30133:::obj30:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 1.0 e 116/116 lb 1:4eb30133:::obj30:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:4dd77094:::obj38:head local-lis/les=113/114 n=189 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.603+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.603+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4dd77094:::obj38:head local-lis/les=113/114 n=189 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:4eb30133:::obj30:head) v3 2026-02-11T23:31:27.603+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4dd77094:::obj38:head local-lis/les=113/114 n=189 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:4eb30133:::obj30:head) 2026-02-11T23:31:27.603+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.603+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:4eb30133:::obj30:head) v3 finish latency 0.000009 2026-02-11T23:31:27.603+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4eb30133:::obj30:head local-lis/les=113/114 n=190 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.603+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.603+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.603+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4eb30133:::obj30:head local-lis/les=113/114 n=190 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.603+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.603+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.603+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.603+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5dab200 con 0x4acb000 2026-02-11T23:31:27.603+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:4fbee9e6:::obj123:head)]) -- 0x5e0bc80 con 0x4acb000 2026-02-11T23:31:27.603+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.604+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 417 ==== MOSDPGPush(1.0 116/113 [PushOp(1:4feb0eaf:::obj556:head, version: 111'556, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4feb0eaf:::obj556:head@111'556, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56cb040 con 0x4acb000 2026-02-11T23:31:27.604+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:4feb0eaf:::obj556:head, version: 111'556, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4feb0eaf:::obj556:head@111'556, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.604+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4feb0eaf:::obj556:head, version: 111'556, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4feb0eaf:::obj556:head@111'556, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.604+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.604+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 418 ==== pg_backfill(progress 1.0 e 116/116 lb 1:4fbee9e6:::obj123:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98580 con 0x4acb000 2026-02-11T23:31:27.604+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:4fbee9e6:::obj123:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.604+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4feb0eaf:::obj556:head, version: 111'556, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4feb0eaf:::obj556:head@111'556, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.604+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4fbee9e6:::obj123:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.604+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.604+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:4feb0eaf:::obj556:head, version: 111'556, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4feb0eaf:::obj556:head@111'556, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.604+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:4feb0eaf:::obj556:head, version: 111'556, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4feb0eaf:::obj556:head@111'556, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:4feb0eaf:::obj556:head, version: 111'556, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4feb0eaf:::obj556:head@111'556, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:4eb30133:::obj30:head local-lis/les=113/114 n=190 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.604+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.604+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4eb30133:::obj30:head local-lis/les=113/114 n=190 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:4feb0eaf:::obj556:head, version: 111'556, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4feb0eaf:::obj556:head@111'556, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.604+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.604+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4eb30133:::obj30:head local-lis/les=113/114 n=190 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:4feb0eaf:::obj556:head@111'556, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.604+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4fbee9e6:::obj123:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.604+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4eb30133:::obj30:head local-lis/les=113/114 n=190 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:4feb0eaf:::obj556:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.604+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4eb30133:::obj30:head local-lis/les=113/114 n=190 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:4feb0eaf:::obj556:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.604+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4eb30133:::obj30:head local-lis/les=113/114 n=190 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:4feb0eaf:::obj556:head 2026-02-11T23:31:27.604+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4eb30133:::obj30:head local-lis/les=113/114 n=190 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:4feb0eaf:::obj556:head v 111'556 2026-02-11T23:31:27.604+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4eb30133:::obj30:head local-lis/les=113/114 n=190 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.604+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.604+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:4feb0eaf:::obj556:head, version: 111'556, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4feb0eaf:::obj556:head@111'556, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.604+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.604+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.604+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4fbee9e6:::obj123:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.604+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:4fbee9e6:::obj123:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:4fbee9e6:::obj123:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:4eb30133:::obj30:head local-lis/les=113/114 n=190 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.604+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.604+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4eb30133:::obj30:head local-lis/les=113/114 n=190 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:4fbee9e6:::obj123:head) v3 2026-02-11T23:31:27.604+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4eb30133:::obj30:head local-lis/les=113/114 n=190 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:4fbee9e6:::obj123:head) 2026-02-11T23:31:27.604+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.604+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:4fbee9e6:::obj123:head) v3 finish latency 0.000008 2026-02-11T23:31:27.604+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4fbee9e6:::obj123:head local-lis/les=113/114 n=191 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.604+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.604+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.604+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4fbee9e6:::obj123:head local-lis/les=113/114 n=191 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.604+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.604+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.604+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.604+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5dab500 con 0x4acb000 2026-02-11T23:31:27.604+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:4feb0eaf:::obj556:head)]) -- 0x5e0be00 con 0x4acb000 2026-02-11T23:31:27.604+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.604+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 419 ==== MOSDPGPush(1.0 116/113 [PushOp(1:50174016:::obj321:head, version: 111'321, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:50174016:::obj321:head@111'321, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56ca1a0 con 0x4acb000 2026-02-11T23:31:27.604+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:50174016:::obj321:head, version: 111'321, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:50174016:::obj321:head@111'321, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.604+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:50174016:::obj321:head, version: 111'321, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:50174016:::obj321:head@111'321, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.604+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.604+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 420 ==== pg_backfill(progress 1.0 e 116/116 lb 1:4feb0eaf:::obj556:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98b00 con 0x4acb000 2026-02-11T23:31:27.604+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:4feb0eaf:::obj556:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.604+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:50174016:::obj321:head, version: 111'321, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:50174016:::obj321:head@111'321, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.604+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4feb0eaf:::obj556:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.604+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.604+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:50174016:::obj321:head, version: 111'321, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:50174016:::obj321:head@111'321, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.604+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:50174016:::obj321:head, version: 111'321, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:50174016:::obj321:head@111'321, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 116/113 [PushOp(1:50174016:::obj321:head, version: 111'321, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:50174016:::obj321:head@111'321, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:4fbee9e6:::obj123:head local-lis/les=113/114 n=191 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.604+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.604+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.604+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4fbee9e6:::obj123:head local-lis/les=113/114 n=191 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:50174016:::obj321:head, version: 111'321, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:50174016:::obj321:head@111'321, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.604+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4feb0eaf:::obj556:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.604+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4fbee9e6:::obj123:head local-lis/les=113/114 n=191 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:50174016:::obj321:head@111'321, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.604+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4fbee9e6:::obj123:head local-lis/les=113/114 n=191 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:50174016:::obj321:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.604+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4fbee9e6:::obj123:head local-lis/les=113/114 n=191 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:50174016:::obj321:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.604+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4fbee9e6:::obj123:head local-lis/les=113/114 n=191 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:50174016:::obj321:head 2026-02-11T23:31:27.604+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4fbee9e6:::obj123:head local-lis/les=113/114 n=191 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:50174016:::obj321:head v 111'321 2026-02-11T23:31:27.604+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4fbee9e6:::obj123:head local-lis/les=113/114 n=191 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.604+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.604+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:50174016:::obj321:head, version: 111'321, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:50174016:::obj321:head@111'321, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.604+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.604+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.604+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:4feb0eaf:::obj556:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.604+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:4feb0eaf:::obj556:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 1.0 e 116/116 lb 1:4feb0eaf:::obj556:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:4fbee9e6:::obj123:head local-lis/les=113/114 n=191 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.604+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.604+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4fbee9e6:::obj123:head local-lis/les=113/114 n=191 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:4feb0eaf:::obj556:head) v3 2026-02-11T23:31:27.604+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4fbee9e6:::obj123:head local-lis/les=113/114 n=191 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:4feb0eaf:::obj556:head) 2026-02-11T23:31:27.604+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.604+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:4feb0eaf:::obj556:head) v3 finish latency 0.000008 2026-02-11T23:31:27.604+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4feb0eaf:::obj556:head local-lis/les=113/114 n=192 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.604+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.604+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.604+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4feb0eaf:::obj556:head local-lis/les=113/114 n=192 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.604+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.604+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.604+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.604+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5dab800 con 0x4acb000 2026-02-11T23:31:27.604+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:50174016:::obj321:head)]) -- 0x5e0a900 con 0x4acb000 2026-02-11T23:31:27.604+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.605+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 421 ==== MOSDPGPush(1.0 116/113 [PushOp(1:502bc4ef:::obj531:head, version: 111'531, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:502bc4ef:::obj531:head@111'531, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d8680 con 0x4acb000 2026-02-11T23:31:27.605+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:502bc4ef:::obj531:head, version: 111'531, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:502bc4ef:::obj531:head@111'531, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.605+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:502bc4ef:::obj531:head, version: 111'531, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:502bc4ef:::obj531:head@111'531, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.605+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.605+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 422 ==== pg_backfill(progress 1.0 e 116/116 lb 1:50174016:::obj321:head) ==== 1062+0+0 (unknown 0 0 0) 0x4a2cb00 con 0x4acb000 2026-02-11T23:31:27.605+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:50174016:::obj321:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.605+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:502bc4ef:::obj531:head, version: 111'531, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:502bc4ef:::obj531:head@111'531, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.605+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:50174016:::obj321:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.605+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.605+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:50174016:::obj321:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.605+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.605+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:502bc4ef:::obj531:head, version: 111'531, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:502bc4ef:::obj531:head@111'531, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.605+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:502bc4ef:::obj531:head, version: 111'531, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:502bc4ef:::obj531:head@111'531, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(1.0 116/113 [PushOp(1:502bc4ef:::obj531:head, version: 111'531, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:502bc4ef:::obj531:head@111'531, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:4feb0eaf:::obj556:head local-lis/les=113/114 n=192 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.605+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.605+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4feb0eaf:::obj556:head local-lis/les=113/114 n=192 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:502bc4ef:::obj531:head, version: 111'531, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:502bc4ef:::obj531:head@111'531, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.605+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4feb0eaf:::obj556:head local-lis/les=113/114 n=192 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:502bc4ef:::obj531:head@111'531, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.605+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4feb0eaf:::obj556:head local-lis/les=113/114 n=192 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:502bc4ef:::obj531:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.605+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4feb0eaf:::obj556:head local-lis/les=113/114 n=192 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:502bc4ef:::obj531:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.605+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4feb0eaf:::obj556:head local-lis/les=113/114 n=192 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:502bc4ef:::obj531:head 2026-02-11T23:31:27.605+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4feb0eaf:::obj556:head local-lis/les=113/114 n=192 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:502bc4ef:::obj531:head v 111'531 2026-02-11T23:31:27.605+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4feb0eaf:::obj556:head local-lis/les=113/114 n=192 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.605+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.605+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:502bc4ef:::obj531:head, version: 111'531, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:502bc4ef:::obj531:head@111'531, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.605+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.605+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.605+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:50174016:::obj321:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.605+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:50174016:::obj321:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 1.0 e 116/116 lb 1:50174016:::obj321:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:4feb0eaf:::obj556:head local-lis/les=113/114 n=192 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.605+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.605+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4feb0eaf:::obj556:head local-lis/les=113/114 n=192 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:50174016:::obj321:head) v3 2026-02-11T23:31:27.605+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:4feb0eaf:::obj556:head local-lis/les=113/114 n=192 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:50174016:::obj321:head) 2026-02-11T23:31:27.605+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.605+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:50174016:::obj321:head) v3 finish latency 0.000009 2026-02-11T23:31:27.605+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:50174016:::obj321:head local-lis/les=113/114 n=193 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.605+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.605+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.605+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:50174016:::obj321:head local-lis/les=113/114 n=193 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.605+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.605+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.605+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.605+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5dab980 con 0x4acb000 2026-02-11T23:31:27.605+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:502bc4ef:::obj531:head)]) -- 0x5e0aa80 con 0x4acb000 2026-02-11T23:31:27.605+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.605+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 423 ==== MOSDPGPush(1.0 116/113 [PushOp(1:506f5ba0:::obj253:head, version: 111'253, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:506f5ba0:::obj253:head@111'253, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559dd40 con 0x4acb000 2026-02-11T23:31:27.605+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:506f5ba0:::obj253:head, version: 111'253, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:506f5ba0:::obj253:head@111'253, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.605+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:506f5ba0:::obj253:head, version: 111'253, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:506f5ba0:::obj253:head@111'253, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.605+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.605+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 424 ==== pg_backfill(progress 1.0 e 116/116 lb 1:502bc4ef:::obj531:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2d080 con 0x4acb000 2026-02-11T23:31:27.605+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:506f5ba0:::obj253:head, version: 111'253, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:506f5ba0:::obj253:head@111'253, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.605+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:502bc4ef:::obj531:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.605+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:502bc4ef:::obj531:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.605+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.605+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:506f5ba0:::obj253:head, version: 111'253, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:506f5ba0:::obj253:head@111'253, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.605+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.605+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:506f5ba0:::obj253:head, version: 111'253, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:506f5ba0:::obj253:head@111'253, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 116/113 [PushOp(1:506f5ba0:::obj253:head, version: 111'253, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:506f5ba0:::obj253:head@111'253, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:50174016:::obj321:head local-lis/les=113/114 n=193 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.605+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.605+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:50174016:::obj321:head local-lis/les=113/114 n=193 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:506f5ba0:::obj253:head, version: 111'253, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:506f5ba0:::obj253:head@111'253, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.605+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:50174016:::obj321:head local-lis/les=113/114 n=193 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:506f5ba0:::obj253:head@111'253, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.605+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.605+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:50174016:::obj321:head local-lis/les=113/114 n=193 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:506f5ba0:::obj253:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.605+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:502bc4ef:::obj531:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.605+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:50174016:::obj321:head local-lis/les=113/114 n=193 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:506f5ba0:::obj253:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.605+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:50174016:::obj321:head local-lis/les=113/114 n=193 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:506f5ba0:::obj253:head 2026-02-11T23:31:27.605+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:50174016:::obj321:head local-lis/les=113/114 n=193 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:506f5ba0:::obj253:head v 111'253 2026-02-11T23:31:27.605+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:50174016:::obj321:head local-lis/les=113/114 n=193 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.605+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.605+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:506f5ba0:::obj253:head, version: 111'253, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:506f5ba0:::obj253:head@111'253, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.605+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.605+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.605+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:502bc4ef:::obj531:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.605+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:502bc4ef:::obj531:head) v3 prio 15 cost 0 latency 0.000060 pg_backfill(progress 1.0 e 116/116 lb 1:502bc4ef:::obj531:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:50174016:::obj321:head local-lis/les=113/114 n=193 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.605+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.605+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:50174016:::obj321:head local-lis/les=113/114 n=193 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:502bc4ef:::obj531:head) v3 2026-02-11T23:31:27.605+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:50174016:::obj321:head local-lis/les=113/114 n=193 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:502bc4ef:::obj531:head) 2026-02-11T23:31:27.605+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.605+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:502bc4ef:::obj531:head) v3 finish latency 0.000009 2026-02-11T23:31:27.605+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:502bc4ef:::obj531:head local-lis/les=113/114 n=194 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.605+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.606+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.606+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:502bc4ef:::obj531:head local-lis/les=113/114 n=194 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.606+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.606+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.606+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.606+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5dabb00 con 0x4acb000 2026-02-11T23:31:27.606+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:506f5ba0:::obj253:head)]) -- 0x5e0ac00 con 0x4acb000 2026-02-11T23:31:27.606+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.606+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 425 ==== MOSDPGPush(1.0 116/113 [PushOp(1:50fb58bc:::obj174:head, version: 111'174, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:50fb58bc:::obj174:head@111'174, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c820 con 0x4acb000 2026-02-11T23:31:27.606+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:50fb58bc:::obj174:head, version: 111'174, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:50fb58bc:::obj174:head@111'174, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.606+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:50fb58bc:::obj174:head, version: 111'174, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:50fb58bc:::obj174:head@111'174, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.606+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.606+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 426 ==== pg_backfill(progress 1.0 e 116/116 lb 1:506f5ba0:::obj253:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e680 con 0x4acb000 2026-02-11T23:31:27.606+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:506f5ba0:::obj253:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.606+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:50fb58bc:::obj174:head, version: 111'174, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:50fb58bc:::obj174:head@111'174, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.606+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:506f5ba0:::obj253:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.606+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.606+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:50fb58bc:::obj174:head, version: 111'174, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:50fb58bc:::obj174:head@111'174, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.606+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:50fb58bc:::obj174:head, version: 111'174, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:50fb58bc:::obj174:head@111'174, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 116/113 [PushOp(1:50fb58bc:::obj174:head, version: 111'174, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:50fb58bc:::obj174:head@111'174, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:502bc4ef:::obj531:head local-lis/les=113/114 n=194 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.606+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.606+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.606+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:502bc4ef:::obj531:head local-lis/les=113/114 n=194 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:50fb58bc:::obj174:head, version: 111'174, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:50fb58bc:::obj174:head@111'174, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.606+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:506f5ba0:::obj253:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.606+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:502bc4ef:::obj531:head local-lis/les=113/114 n=194 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:50fb58bc:::obj174:head@111'174, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.606+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:502bc4ef:::obj531:head local-lis/les=113/114 n=194 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:50fb58bc:::obj174:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.606+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:502bc4ef:::obj531:head local-lis/les=113/114 n=194 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:50fb58bc:::obj174:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.606+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:502bc4ef:::obj531:head local-lis/les=113/114 n=194 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:50fb58bc:::obj174:head 2026-02-11T23:31:27.606+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:502bc4ef:::obj531:head local-lis/les=113/114 n=194 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:50fb58bc:::obj174:head v 111'174 2026-02-11T23:31:27.606+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:502bc4ef:::obj531:head local-lis/les=113/114 n=194 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.606+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.606+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:50fb58bc:::obj174:head, version: 111'174, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:50fb58bc:::obj174:head@111'174, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.606+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.606+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.606+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:506f5ba0:::obj253:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.606+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:506f5ba0:::obj253:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 1.0 e 116/116 lb 1:506f5ba0:::obj253:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:502bc4ef:::obj531:head local-lis/les=113/114 n=194 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.606+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.606+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:502bc4ef:::obj531:head local-lis/les=113/114 n=194 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:506f5ba0:::obj253:head) v3 2026-02-11T23:31:27.606+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:502bc4ef:::obj531:head local-lis/les=113/114 n=194 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:506f5ba0:::obj253:head) 2026-02-11T23:31:27.606+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.606+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:506f5ba0:::obj253:head) v3 finish latency 0.000010 2026-02-11T23:31:27.606+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:506f5ba0:::obj253:head local-lis/les=113/114 n=195 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.606+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.606+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.606+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:506f5ba0:::obj253:head local-lis/les=113/114 n=195 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.606+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.606+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.606+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.606+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5dabe00 con 0x4acb000 2026-02-11T23:31:27.606+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:50fb58bc:::obj174:head)]) -- 0x5d1c180 con 0x4acb000 2026-02-11T23:31:27.606+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.606+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 427 ==== MOSDPGPush(1.0 116/113 [PushOp(1:51058f37:::obj394:head, version: 111'394, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:51058f37:::obj394:head@111'394, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c340 con 0x4acb000 2026-02-11T23:31:27.607+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:51058f37:::obj394:head, version: 111'394, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:51058f37:::obj394:head@111'394, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 116 2026-02-11T23:31:27.607+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:51058f37:::obj394:head, version: 111'394, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:51058f37:::obj394:head@111'394, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.607+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.607+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 428 ==== pg_backfill(progress 1.0 e 116/116 lb 1:50fb58bc:::obj174:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e100 con 0x4acb000 2026-02-11T23:31:27.607+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:50fb58bc:::obj174:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.607+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:51058f37:::obj394:head, version: 111'394, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:51058f37:::obj394:head@111'394, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.607+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:50fb58bc:::obj174:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.607+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.607+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:51058f37:::obj394:head, version: 111'394, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:51058f37:::obj394:head@111'394, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.607+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.607+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:51058f37:::obj394:head, version: 111'394, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:51058f37:::obj394:head@111'394, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 116/113 [PushOp(1:51058f37:::obj394:head, version: 111'394, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:51058f37:::obj394:head@111'394, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:506f5ba0:::obj253:head local-lis/les=113/114 n=195 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.607+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.607+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:506f5ba0:::obj253:head local-lis/les=113/114 n=195 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:51058f37:::obj394:head, version: 111'394, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:51058f37:::obj394:head@111'394, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.607+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:506f5ba0:::obj253:head local-lis/les=113/114 n=195 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:51058f37:::obj394:head@111'394, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.607+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.607+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:50fb58bc:::obj174:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.607+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:506f5ba0:::obj253:head local-lis/les=113/114 n=195 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:51058f37:::obj394:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.607+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:506f5ba0:::obj253:head local-lis/les=113/114 n=195 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:51058f37:::obj394:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.607+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:506f5ba0:::obj253:head local-lis/les=113/114 n=195 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:51058f37:::obj394:head 2026-02-11T23:31:27.607+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:506f5ba0:::obj253:head local-lis/les=113/114 n=195 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:51058f37:::obj394:head v 111'394 2026-02-11T23:31:27.607+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:506f5ba0:::obj253:head local-lis/les=113/114 n=195 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.607+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.607+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:51058f37:::obj394:head, version: 111'394, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:51058f37:::obj394:head@111'394, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.607+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.607+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.607+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:50fb58bc:::obj174:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.607+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:50fb58bc:::obj174:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 1.0 e 116/116 lb 1:50fb58bc:::obj174:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:506f5ba0:::obj253:head local-lis/les=113/114 n=195 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.607+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.607+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:506f5ba0:::obj253:head local-lis/les=113/114 n=195 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:50fb58bc:::obj174:head) v3 2026-02-11T23:31:27.607+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:506f5ba0:::obj253:head local-lis/les=113/114 n=195 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:50fb58bc:::obj174:head) 2026-02-11T23:31:27.607+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.607+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:50fb58bc:::obj174:head) v3 finish latency 0.000009 2026-02-11T23:31:27.607+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:50fb58bc:::obj174:head local-lis/les=113/114 n=196 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.607+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.607+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.607+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:50fb58bc:::obj174:head local-lis/les=113/114 n=196 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.607+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.607+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.607+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.607+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fec000 con 0x4acb000 2026-02-11T23:31:27.607+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:51058f37:::obj394:head)]) -- 0x5d1c480 con 0x4acb000 2026-02-11T23:31:27.607+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.607+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 429 ==== MOSDPGPush(1.0 116/113 [PushOp(1:5170da72:::obj410:head, version: 111'410, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5170da72:::obj410:head@111'410, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559d1e0 con 0x4acb000 2026-02-11T23:31:27.607+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:5170da72:::obj410:head, version: 111'410, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5170da72:::obj410:head@111'410, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.607+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:5170da72:::obj410:head, version: 111'410, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5170da72:::obj410:head@111'410, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.607+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.607+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:5170da72:::obj410:head, version: 111'410, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5170da72:::obj410:head@111'410, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.607+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.607+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:5170da72:::obj410:head, version: 111'410, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5170da72:::obj410:head@111'410, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.607+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 430 ==== pg_backfill(progress 1.0 e 116/116 lb 1:51058f37:::obj394:head) ==== 1062+0+0 (unknown 0 0 0) 0x5fee000 con 0x4acb000 2026-02-11T23:31:27.607+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:51058f37:::obj394:head) v3 prio 15 type 95 cost 0 latency 0.000011 epoch 116 2026-02-11T23:31:27.607+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.607+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:5170da72:::obj410:head, version: 111'410, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5170da72:::obj410:head@111'410, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 116/113 [PushOp(1:5170da72:::obj410:head, version: 111'410, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5170da72:::obj410:head@111'410, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:50fb58bc:::obj174:head local-lis/les=113/114 n=196 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.607+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:51058f37:::obj394:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.607+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.607+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:50fb58bc:::obj174:head local-lis/les=113/114 n=196 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:5170da72:::obj410:head, version: 111'410, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5170da72:::obj410:head@111'410, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.607+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.607+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:50fb58bc:::obj174:head local-lis/les=113/114 n=196 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:5170da72:::obj410:head@111'410, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.607+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:51058f37:::obj394:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.607+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:50fb58bc:::obj174:head local-lis/les=113/114 n=196 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:5170da72:::obj410:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.607+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:50fb58bc:::obj174:head local-lis/les=113/114 n=196 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:5170da72:::obj410:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.607+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:50fb58bc:::obj174:head local-lis/les=113/114 n=196 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:5170da72:::obj410:head 2026-02-11T23:31:27.607+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:50fb58bc:::obj174:head local-lis/les=113/114 n=196 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:5170da72:::obj410:head v 111'410 2026-02-11T23:31:27.607+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:50fb58bc:::obj174:head local-lis/les=113/114 n=196 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.607+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.607+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:5170da72:::obj410:head, version: 111'410, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5170da72:::obj410:head@111'410, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:27.607+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.607+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.607+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:51058f37:::obj394:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.607+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:51058f37:::obj394:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:51058f37:::obj394:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:50fb58bc:::obj174:head local-lis/les=113/114 n=196 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.607+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.607+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:50fb58bc:::obj174:head local-lis/les=113/114 n=196 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:51058f37:::obj394:head) v3 2026-02-11T23:31:27.607+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:50fb58bc:::obj174:head local-lis/les=113/114 n=196 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:51058f37:::obj394:head) 2026-02-11T23:31:27.607+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.607+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:51058f37:::obj394:head) v3 finish latency 0.000009 2026-02-11T23:31:27.607+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:51058f37:::obj394:head local-lis/les=113/114 n=197 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.607+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.607+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.607+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:51058f37:::obj394:head local-lis/les=113/114 n=197 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.607+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.607+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.607+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.607+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fec180 con 0x4acb000 2026-02-11T23:31:27.607+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:5170da72:::obj410:head)]) -- 0x5d1c780 con 0x4acb000 2026-02-11T23:31:27.607+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.608+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 431 ==== MOSDPGPush(1.0 116/113 [PushOp(1:5178a392:::obj281:head, version: 111'281, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5178a392:::obj281:head@111'281, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559cd00 con 0x4acb000 2026-02-11T23:31:27.608+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:5178a392:::obj281:head, version: 111'281, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5178a392:::obj281:head@111'281, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.608+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:5178a392:::obj281:head, version: 111'281, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5178a392:::obj281:head@111'281, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.608+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.608+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 432 ==== pg_backfill(progress 1.0 e 116/116 lb 1:5170da72:::obj410:head) ==== 1062+0+0 (unknown 0 0 0) 0x5fee580 con 0x4acb000 2026-02-11T23:31:27.608+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:5170da72:::obj410:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.608+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:5178a392:::obj281:head, version: 111'281, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5178a392:::obj281:head@111'281, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.608+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:5170da72:::obj410:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.608+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.608+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:5170da72:::obj410:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.608+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.608+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:5178a392:::obj281:head, version: 111'281, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5178a392:::obj281:head@111'281, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.608+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:5178a392:::obj281:head, version: 111'281, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5178a392:::obj281:head@111'281, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(1.0 116/113 [PushOp(1:5178a392:::obj281:head, version: 111'281, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5178a392:::obj281:head@111'281, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:51058f37:::obj394:head local-lis/les=113/114 n=197 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.608+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.608+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:51058f37:::obj394:head local-lis/les=113/114 n=197 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:5178a392:::obj281:head, version: 111'281, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5178a392:::obj281:head@111'281, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.608+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:51058f37:::obj394:head local-lis/les=113/114 n=197 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:5178a392:::obj281:head@111'281, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.608+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:51058f37:::obj394:head local-lis/les=113/114 n=197 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:5178a392:::obj281:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.608+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:51058f37:::obj394:head local-lis/les=113/114 n=197 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:5178a392:::obj281:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.608+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:51058f37:::obj394:head local-lis/les=113/114 n=197 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:5178a392:::obj281:head 2026-02-11T23:31:27.608+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:51058f37:::obj394:head local-lis/les=113/114 n=197 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:5178a392:::obj281:head v 111'281 2026-02-11T23:31:27.608+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:51058f37:::obj394:head local-lis/les=113/114 n=197 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.608+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.608+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:5178a392:::obj281:head, version: 111'281, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5178a392:::obj281:head@111'281, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:27.608+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:51058f37:::obj394:head local-lis/les=113/114 n=197 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.608+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.608+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.608+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:5170da72:::obj410:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.608+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:5170da72:::obj410:head) v3 prio 15 cost 0 latency 0.000081 pg_backfill(progress 1.0 e 116/116 lb 1:5170da72:::obj410:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:51058f37:::obj394:head local-lis/les=113/114 n=197 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.608+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.608+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:51058f37:::obj394:head local-lis/les=113/114 n=197 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:5170da72:::obj410:head) v3 2026-02-11T23:31:27.608+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:51058f37:::obj394:head local-lis/les=113/114 n=197 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:5170da72:::obj410:head) 2026-02-11T23:31:27.608+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.608+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:5170da72:::obj410:head) v3 finish latency 0.000010 2026-02-11T23:31:27.608+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.608+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.608+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5170da72:::obj410:head local-lis/les=113/114 n=198 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.608+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.608+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.608+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.608+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fec600 con 0x4acb000 2026-02-11T23:31:27.608+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:5178a392:::obj281:head)]) -- 0x5fec300 con 0x4acb000 2026-02-11T23:31:27.608+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.608+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 433 ==== MOSDPGPush(1.0 116/113 [PushOp(1:52d1d495:::obj578:head, version: 111'578, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:52d1d495:::obj578:head@111'578, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f1380 con 0x4acb000 2026-02-11T23:31:27.608+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:52d1d495:::obj578:head, version: 111'578, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:52d1d495:::obj578:head@111'578, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.608+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:52d1d495:::obj578:head, version: 111'578, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:52d1d495:::obj578:head@111'578, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.608+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.608+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 434 ==== pg_backfill(progress 1.0 e 116/116 lb 1:5178a392:::obj281:head) ==== 1062+0+0 (unknown 0 0 0) 0x5feeb00 con 0x4acb000 2026-02-11T23:31:27.608+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:52d1d495:::obj578:head, version: 111'578, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:52d1d495:::obj578:head@111'578, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.608+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:5178a392:::obj281:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.608+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:5178a392:::obj281:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.608+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.608+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:52d1d495:::obj578:head, version: 111'578, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:52d1d495:::obj578:head@111'578, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.608+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.608+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:52d1d495:::obj578:head, version: 111'578, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:52d1d495:::obj578:head@111'578, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 116/113 [PushOp(1:52d1d495:::obj578:head, version: 111'578, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:52d1d495:::obj578:head@111'578, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:5170da72:::obj410:head local-lis/les=113/114 n=198 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.608+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.608+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5170da72:::obj410:head local-lis/les=113/114 n=198 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:52d1d495:::obj578:head, version: 111'578, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:52d1d495:::obj578:head@111'578, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.608+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5170da72:::obj410:head local-lis/les=113/114 n=198 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:52d1d495:::obj578:head@111'578, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.608+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.608+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:5178a392:::obj281:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.608+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5170da72:::obj410:head local-lis/les=113/114 n=198 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:52d1d495:::obj578:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.608+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5170da72:::obj410:head local-lis/les=113/114 n=198 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:52d1d495:::obj578:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.608+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5170da72:::obj410:head local-lis/les=113/114 n=198 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:52d1d495:::obj578:head 2026-02-11T23:31:27.608+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5170da72:::obj410:head local-lis/les=113/114 n=198 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:52d1d495:::obj578:head v 111'578 2026-02-11T23:31:27.608+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5170da72:::obj410:head local-lis/les=113/114 n=198 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.608+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.608+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:52d1d495:::obj578:head, version: 111'578, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:52d1d495:::obj578:head@111'578, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:27.608+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.608+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.608+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:5178a392:::obj281:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.608+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:5178a392:::obj281:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 1.0 e 116/116 lb 1:5178a392:::obj281:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:5170da72:::obj410:head local-lis/les=113/114 n=198 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.608+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.608+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5170da72:::obj410:head local-lis/les=113/114 n=198 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:5178a392:::obj281:head) v3 2026-02-11T23:31:27.608+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5170da72:::obj410:head local-lis/les=113/114 n=198 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:5178a392:::obj281:head) 2026-02-11T23:31:27.608+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.608+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:5178a392:::obj281:head) v3 finish latency 0.000011 2026-02-11T23:31:27.608+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5178a392:::obj281:head local-lis/les=113/114 n=199 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.608+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.609+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.609+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5178a392:::obj281:head local-lis/les=113/114 n=199 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.609+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.609+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.609+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.609+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fec780 con 0x4acb000 2026-02-11T23:31:27.609+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:52d1d495:::obj578:head)]) -- 0x5d1ca80 con 0x4acb000 2026-02-11T23:31:27.609+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.609+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 435 ==== MOSDPGPush(1.0 116/113 [PushOp(1:52e063dd:::obj399:head, version: 111'399, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:52e063dd:::obj399:head@111'399, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c000 con 0x4acb000 2026-02-11T23:31:27.609+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:52e063dd:::obj399:head, version: 111'399, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:52e063dd:::obj399:head@111'399, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.609+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:52e063dd:::obj399:head, version: 111'399, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:52e063dd:::obj399:head@111'399, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.609+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.609+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 436 ==== pg_backfill(progress 1.0 e 116/116 lb 1:52d1d495:::obj578:head) ==== 1062+0+0 (unknown 0 0 0) 0x5fef080 con 0x4acb000 2026-02-11T23:31:27.609+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:52d1d495:::obj578:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.609+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:52e063dd:::obj399:head, version: 111'399, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:52e063dd:::obj399:head@111'399, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.609+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:52d1d495:::obj578:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.609+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.609+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:52e063dd:::obj399:head, version: 111'399, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:52e063dd:::obj399:head@111'399, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.609+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:52e063dd:::obj399:head, version: 111'399, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:52e063dd:::obj399:head@111'399, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:52e063dd:::obj399:head, version: 111'399, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:52e063dd:::obj399:head@111'399, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:5178a392:::obj281:head local-lis/les=113/114 n=199 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.609+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.609+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.609+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5178a392:::obj281:head local-lis/les=113/114 n=199 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:52e063dd:::obj399:head, version: 111'399, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:52e063dd:::obj399:head@111'399, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.609+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:52d1d495:::obj578:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.609+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5178a392:::obj281:head local-lis/les=113/114 n=199 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:52e063dd:::obj399:head@111'399, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.609+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5178a392:::obj281:head local-lis/les=113/114 n=199 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:52e063dd:::obj399:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.609+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5178a392:::obj281:head local-lis/les=113/114 n=199 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:52e063dd:::obj399:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.609+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5178a392:::obj281:head local-lis/les=113/114 n=199 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:52e063dd:::obj399:head 2026-02-11T23:31:27.609+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5178a392:::obj281:head local-lis/les=113/114 n=199 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:52e063dd:::obj399:head v 111'399 2026-02-11T23:31:27.609+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5178a392:::obj281:head local-lis/les=113/114 n=199 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.609+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.609+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:52e063dd:::obj399:head, version: 111'399, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:52e063dd:::obj399:head@111'399, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.609+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.609+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.609+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:52d1d495:::obj578:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.609+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:52d1d495:::obj578:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 1.0 e 116/116 lb 1:52d1d495:::obj578:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:5178a392:::obj281:head local-lis/les=113/114 n=199 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.609+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.609+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5178a392:::obj281:head local-lis/les=113/114 n=199 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:52d1d495:::obj578:head) v3 2026-02-11T23:31:27.609+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5178a392:::obj281:head local-lis/les=113/114 n=199 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:52d1d495:::obj578:head) 2026-02-11T23:31:27.609+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.609+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:52d1d495:::obj578:head) v3 finish latency 0.000010 2026-02-11T23:31:27.609+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:52d1d495:::obj578:head local-lis/les=113/114 n=200 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.609+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.609+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.609+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:52d1d495:::obj578:head local-lis/les=113/114 n=200 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.609+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.609+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.609+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.609+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fec900 con 0x4acb000 2026-02-11T23:31:27.609+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:52e063dd:::obj399:head)]) -- 0x5d1cd80 con 0x4acb000 2026-02-11T23:31:27.609+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.609+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 437 ==== MOSDPGPush(1.0 116/113 [PushOp(1:53438dcf:::obj478:head, version: 111'478, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:53438dcf:::obj478:head@111'478, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f0d00 con 0x4acb000 2026-02-11T23:31:27.609+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:53438dcf:::obj478:head, version: 111'478, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:53438dcf:::obj478:head@111'478, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.609+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:53438dcf:::obj478:head, version: 111'478, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:53438dcf:::obj478:head@111'478, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.609+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.609+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 438 ==== pg_backfill(progress 1.0 e 116/116 lb 1:52e063dd:::obj399:head) ==== 1062+0+0 (unknown 0 0 0) 0x5fef600 con 0x4acb000 2026-02-11T23:31:27.609+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:53438dcf:::obj478:head, version: 111'478, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:53438dcf:::obj478:head@111'478, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.609+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:52e063dd:::obj399:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.609+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.609+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:52e063dd:::obj399:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.609+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:53438dcf:::obj478:head, version: 111'478, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:53438dcf:::obj478:head@111'478, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.609+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.609+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:53438dcf:::obj478:head, version: 111'478, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:53438dcf:::obj478:head@111'478, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:53438dcf:::obj478:head, version: 111'478, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:53438dcf:::obj478:head@111'478, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:52d1d495:::obj578:head local-lis/les=113/114 n=200 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.609+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.609+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:52d1d495:::obj578:head local-lis/les=113/114 n=200 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:53438dcf:::obj478:head, version: 111'478, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:53438dcf:::obj478:head@111'478, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.609+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:52d1d495:::obj578:head local-lis/les=113/114 n=200 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:53438dcf:::obj478:head@111'478, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.609+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.609+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:52e063dd:::obj399:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.609+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:52d1d495:::obj578:head local-lis/les=113/114 n=200 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:53438dcf:::obj478:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.609+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:52d1d495:::obj578:head local-lis/les=113/114 n=200 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:53438dcf:::obj478:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.609+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:52d1d495:::obj578:head local-lis/les=113/114 n=200 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:53438dcf:::obj478:head 2026-02-11T23:31:27.609+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:52d1d495:::obj578:head local-lis/les=113/114 n=200 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:53438dcf:::obj478:head v 111'478 2026-02-11T23:31:27.609+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:52d1d495:::obj578:head local-lis/les=113/114 n=200 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.609+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.609+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:53438dcf:::obj478:head, version: 111'478, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:53438dcf:::obj478:head@111'478, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000043 2026-02-11T23:31:27.609+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.609+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.609+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:52e063dd:::obj399:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.609+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:52e063dd:::obj399:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 1.0 e 116/116 lb 1:52e063dd:::obj399:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:52d1d495:::obj578:head local-lis/les=113/114 n=200 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.609+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.609+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:52d1d495:::obj578:head local-lis/les=113/114 n=200 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:52e063dd:::obj399:head) v3 2026-02-11T23:31:27.609+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:52d1d495:::obj578:head local-lis/les=113/114 n=200 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:52e063dd:::obj399:head) 2026-02-11T23:31:27.609+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.609+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:52e063dd:::obj399:head) v3 finish latency 0.000011 2026-02-11T23:31:27.609+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:52e063dd:::obj399:head local-lis/les=113/114 n=201 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.609+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.610+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.610+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:52e063dd:::obj399:head local-lis/les=113/114 n=201 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.610+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.610+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.610+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.610+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5feca80 con 0x4acb000 2026-02-11T23:31:27.610+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:53438dcf:::obj478:head)]) -- 0x5d1d080 con 0x4acb000 2026-02-11T23:31:27.610+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.610+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 439 ==== MOSDPGPush(1.0 116/113 [PushOp(1:53c971fd:::obj132:head, version: 111'132, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:53c971fd:::obj132:head@111'132, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f1860 con 0x4acb000 2026-02-11T23:31:27.610+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:53c971fd:::obj132:head, version: 111'132, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:53c971fd:::obj132:head@111'132, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.610+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:53c971fd:::obj132:head, version: 111'132, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:53c971fd:::obj132:head@111'132, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.610+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.610+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 440 ==== pg_backfill(progress 1.0 e 116/116 lb 1:53438dcf:::obj478:head) ==== 1062+0+0 (unknown 0 0 0) 0x5fefb80 con 0x4acb000 2026-02-11T23:31:27.610+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:53c971fd:::obj132:head, version: 111'132, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:53c971fd:::obj132:head@111'132, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.610+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:53438dcf:::obj478:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.610+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:53438dcf:::obj478:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.610+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.610+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:53c971fd:::obj132:head, version: 111'132, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:53c971fd:::obj132:head@111'132, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.610+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.610+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:53c971fd:::obj132:head, version: 111'132, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:53c971fd:::obj132:head@111'132, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 116/113 [PushOp(1:53c971fd:::obj132:head, version: 111'132, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:53c971fd:::obj132:head@111'132, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:52e063dd:::obj399:head local-lis/les=113/114 n=201 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.610+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.610+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:52e063dd:::obj399:head local-lis/les=113/114 n=201 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:53c971fd:::obj132:head, version: 111'132, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:53c971fd:::obj132:head@111'132, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.610+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:52e063dd:::obj399:head local-lis/les=113/114 n=201 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:53c971fd:::obj132:head@111'132, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.610+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.610+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:53438dcf:::obj478:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.610+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:52e063dd:::obj399:head local-lis/les=113/114 n=201 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:53c971fd:::obj132:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.610+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:52e063dd:::obj399:head local-lis/les=113/114 n=201 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:53c971fd:::obj132:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.610+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:52e063dd:::obj399:head local-lis/les=113/114 n=201 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:53c971fd:::obj132:head 2026-02-11T23:31:27.610+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:52e063dd:::obj399:head local-lis/les=113/114 n=201 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:53c971fd:::obj132:head v 111'132 2026-02-11T23:31:27.610+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:52e063dd:::obj399:head local-lis/les=113/114 n=201 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.610+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.610+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:53c971fd:::obj132:head, version: 111'132, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:53c971fd:::obj132:head@111'132, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.610+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.610+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.610+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:53438dcf:::obj478:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.610+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:53438dcf:::obj478:head) v3 prio 15 cost 0 latency 0.000059 pg_backfill(progress 1.0 e 116/116 lb 1:53438dcf:::obj478:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:52e063dd:::obj399:head local-lis/les=113/114 n=201 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.610+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.610+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:52e063dd:::obj399:head local-lis/les=113/114 n=201 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:53438dcf:::obj478:head) v3 2026-02-11T23:31:27.610+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:52e063dd:::obj399:head local-lis/les=113/114 n=201 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:53438dcf:::obj478:head) 2026-02-11T23:31:27.610+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.610+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:53438dcf:::obj478:head) v3 finish latency 0.000009 2026-02-11T23:31:27.610+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:53438dcf:::obj478:head local-lis/les=113/114 n=202 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.610+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.610+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.610+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:53438dcf:::obj478:head local-lis/les=113/114 n=202 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.610+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.610+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.610+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.610+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fecc00 con 0x4acb000 2026-02-11T23:31:27.610+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:53c971fd:::obj132:head)]) -- 0x5d1d380 con 0x4acb000 2026-02-11T23:31:27.610+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.611+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 441 ==== MOSDPGPush(1.0 116/113 [PushOp(1:541e28f4:::obj33:head, version: 111'33, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:541e28f4:::obj33:head@111'33, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x559c4e0 con 0x4acb000 2026-02-11T23:31:27.611+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:541e28f4:::obj33:head, version: 111'33, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:541e28f4:::obj33:head@111'33, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.611+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:541e28f4:::obj33:head, version: 111'33, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:541e28f4:::obj33:head@111'33, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.611+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.611+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 442 ==== pg_backfill(progress 1.0 e 116/116 lb 1:53c971fd:::obj132:head) ==== 1062+0+0 (unknown 0 0 0) 0x5ff0100 con 0x4acb000 2026-02-11T23:31:27.611+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:53c971fd:::obj132:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.611+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:541e28f4:::obj33:head, version: 111'33, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:541e28f4:::obj33:head@111'33, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.611+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:53c971fd:::obj132:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.611+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.611+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:541e28f4:::obj33:head, version: 111'33, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:541e28f4:::obj33:head@111'33, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.611+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.611+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:541e28f4:::obj33:head, version: 111'33, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:541e28f4:::obj33:head@111'33, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 116/113 [PushOp(1:541e28f4:::obj33:head, version: 111'33, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:541e28f4:::obj33:head@111'33, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:53438dcf:::obj478:head local-lis/les=113/114 n=202 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.611+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.611+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:53c971fd:::obj132:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.611+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:53438dcf:::obj478:head local-lis/les=113/114 n=202 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:541e28f4:::obj33:head, version: 111'33, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:541e28f4:::obj33:head@111'33, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.611+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:53438dcf:::obj478:head local-lis/les=113/114 n=202 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:541e28f4:::obj33:head@111'33, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.611+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:53438dcf:::obj478:head local-lis/les=113/114 n=202 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:541e28f4:::obj33:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.611+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:53438dcf:::obj478:head local-lis/les=113/114 n=202 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:541e28f4:::obj33:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.611+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:53438dcf:::obj478:head local-lis/les=113/114 n=202 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:541e28f4:::obj33:head 2026-02-11T23:31:27.611+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:53438dcf:::obj478:head local-lis/les=113/114 n=202 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:541e28f4:::obj33:head v 111'33 2026-02-11T23:31:27.611+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:53438dcf:::obj478:head local-lis/les=113/114 n=202 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.611+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.611+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:541e28f4:::obj33:head, version: 111'33, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:541e28f4:::obj33:head@111'33, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.611+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.611+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.611+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:53c971fd:::obj132:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.611+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:53c971fd:::obj132:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:53c971fd:::obj132:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:53438dcf:::obj478:head local-lis/les=113/114 n=202 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.611+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.611+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:53438dcf:::obj478:head local-lis/les=113/114 n=202 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:53c971fd:::obj132:head) v3 2026-02-11T23:31:27.611+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:53438dcf:::obj478:head local-lis/les=113/114 n=202 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:53c971fd:::obj132:head) 2026-02-11T23:31:27.611+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.611+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:53c971fd:::obj132:head) v3 finish latency 0.000009 2026-02-11T23:31:27.611+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:53c971fd:::obj132:head local-lis/les=113/114 n=203 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.611+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.611+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.611+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:53c971fd:::obj132:head local-lis/les=113/114 n=203 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.611+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.611+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.611+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.611+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fecd80 con 0x4acb000 2026-02-11T23:31:27.611+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:541e28f4:::obj33:head)]) -- 0x5d1d680 con 0x4acb000 2026-02-11T23:31:27.611+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.611+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 443 ==== MOSDPGPush(1.0 116/113 [PushOp(1:545db4bf:::obj139:head, version: 111'139, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:545db4bf:::obj139:head@111'139, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559da00 con 0x4acb000 2026-02-11T23:31:27.611+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:545db4bf:::obj139:head, version: 111'139, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:545db4bf:::obj139:head@111'139, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.611+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:545db4bf:::obj139:head, version: 111'139, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:545db4bf:::obj139:head@111'139, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.611+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.611+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 444 ==== pg_backfill(progress 1.0 e 116/116 lb 1:541e28f4:::obj33:head) ==== 1061+0+0 (unknown 0 0 0) 0x5ff0680 con 0x4acb000 2026-02-11T23:31:27.611+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:541e28f4:::obj33:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.611+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:545db4bf:::obj139:head, version: 111'139, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:545db4bf:::obj139:head@111'139, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.611+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:541e28f4:::obj33:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.611+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.611+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:545db4bf:::obj139:head, version: 111'139, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:545db4bf:::obj139:head@111'139, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.611+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:545db4bf:::obj139:head, version: 111'139, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:545db4bf:::obj139:head@111'139, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000040 MOSDPGPush(1.0 116/113 [PushOp(1:545db4bf:::obj139:head, version: 111'139, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:545db4bf:::obj139:head@111'139, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:53c971fd:::obj132:head local-lis/les=113/114 n=203 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.611+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.611+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.611+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:53c971fd:::obj132:head local-lis/les=113/114 n=203 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:545db4bf:::obj139:head, version: 111'139, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:545db4bf:::obj139:head@111'139, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.611+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:541e28f4:::obj33:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.611+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:53c971fd:::obj132:head local-lis/les=113/114 n=203 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:545db4bf:::obj139:head@111'139, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.611+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:53c971fd:::obj132:head local-lis/les=113/114 n=203 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:545db4bf:::obj139:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.611+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:53c971fd:::obj132:head local-lis/les=113/114 n=203 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:545db4bf:::obj139:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.611+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:53c971fd:::obj132:head local-lis/les=113/114 n=203 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:545db4bf:::obj139:head 2026-02-11T23:31:27.611+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:53c971fd:::obj132:head local-lis/les=113/114 n=203 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:545db4bf:::obj139:head v 111'139 2026-02-11T23:31:27.611+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:53c971fd:::obj132:head local-lis/les=113/114 n=203 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.611+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.611+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:545db4bf:::obj139:head, version: 111'139, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:545db4bf:::obj139:head@111'139, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.611+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.611+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.611+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:541e28f4:::obj33:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.611+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:541e28f4:::obj33:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 116/116 lb 1:541e28f4:::obj33:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:53c971fd:::obj132:head local-lis/les=113/114 n=203 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.611+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.611+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:53c971fd:::obj132:head local-lis/les=113/114 n=203 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:541e28f4:::obj33:head) v3 2026-02-11T23:31:27.611+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:53c971fd:::obj132:head local-lis/les=113/114 n=203 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:541e28f4:::obj33:head) 2026-02-11T23:31:27.611+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.611+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:541e28f4:::obj33:head) v3 finish latency 0.000010 2026-02-11T23:31:27.611+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:541e28f4:::obj33:head local-lis/les=113/114 n=204 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.611+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.611+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.611+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:541e28f4:::obj33:head local-lis/les=113/114 n=204 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.611+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.611+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.611+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.611+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fecf00 con 0x4acb000 2026-02-11T23:31:27.611+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:545db4bf:::obj139:head)]) -- 0x5d1d800 con 0x4acb000 2026-02-11T23:31:27.611+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.612+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 445 ==== MOSDPGPush(1.0 116/113 [PushOp(1:54a9e115:::obj404:head, version: 111'404, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:54a9e115:::obj404:head@111'404, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559cea0 con 0x4acb000 2026-02-11T23:31:27.612+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:54a9e115:::obj404:head, version: 111'404, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:54a9e115:::obj404:head@111'404, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.612+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:54a9e115:::obj404:head, version: 111'404, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:54a9e115:::obj404:head@111'404, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.612+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.612+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 446 ==== pg_backfill(progress 1.0 e 116/116 lb 1:545db4bf:::obj139:head) ==== 1062+0+0 (unknown 0 0 0) 0x5ff0c00 con 0x4acb000 2026-02-11T23:31:27.612+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:545db4bf:::obj139:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.612+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:54a9e115:::obj404:head, version: 111'404, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:54a9e115:::obj404:head@111'404, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.612+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:545db4bf:::obj139:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.612+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.612+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:54a9e115:::obj404:head, version: 111'404, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:54a9e115:::obj404:head@111'404, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.612+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:54a9e115:::obj404:head, version: 111'404, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:54a9e115:::obj404:head@111'404, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 116/113 [PushOp(1:54a9e115:::obj404:head, version: 111'404, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:54a9e115:::obj404:head@111'404, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:541e28f4:::obj33:head local-lis/les=113/114 n=204 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.612+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.612+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.612+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:541e28f4:::obj33:head local-lis/les=113/114 n=204 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:54a9e115:::obj404:head, version: 111'404, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:54a9e115:::obj404:head@111'404, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.612+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:545db4bf:::obj139:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.612+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:541e28f4:::obj33:head local-lis/les=113/114 n=204 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:54a9e115:::obj404:head@111'404, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.612+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:541e28f4:::obj33:head local-lis/les=113/114 n=204 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:54a9e115:::obj404:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.612+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:541e28f4:::obj33:head local-lis/les=113/114 n=204 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:54a9e115:::obj404:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.612+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:541e28f4:::obj33:head local-lis/les=113/114 n=204 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:54a9e115:::obj404:head 2026-02-11T23:31:27.612+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:541e28f4:::obj33:head local-lis/les=113/114 n=204 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:54a9e115:::obj404:head v 111'404 2026-02-11T23:31:27.612+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:541e28f4:::obj33:head local-lis/les=113/114 n=204 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.612+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.612+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:54a9e115:::obj404:head, version: 111'404, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:54a9e115:::obj404:head@111'404, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.612+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.612+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.612+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:545db4bf:::obj139:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.612+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:545db4bf:::obj139:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:545db4bf:::obj139:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:541e28f4:::obj33:head local-lis/les=113/114 n=204 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.612+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.612+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:541e28f4:::obj33:head local-lis/les=113/114 n=204 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:545db4bf:::obj139:head) v3 2026-02-11T23:31:27.612+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:541e28f4:::obj33:head local-lis/les=113/114 n=204 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:545db4bf:::obj139:head) 2026-02-11T23:31:27.612+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.612+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:545db4bf:::obj139:head) v3 finish latency 0.000009 2026-02-11T23:31:27.612+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:545db4bf:::obj139:head local-lis/les=113/114 n=205 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.612+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.612+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.612+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:545db4bf:::obj139:head local-lis/les=113/114 n=205 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.612+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.612+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.612+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.612+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fed080 con 0x4acb000 2026-02-11T23:31:27.612+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:54a9e115:::obj404:head)]) -- 0x5d1db00 con 0x4acb000 2026-02-11T23:31:27.612+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.612+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 447 ==== MOSDPGPush(1.0 116/113 [PushOp(1:54cd4daa:::obj313:head, version: 111'313, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:54cd4daa:::obj313:head@111'313, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d8820 con 0x4acb000 2026-02-11T23:31:27.612+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:54cd4daa:::obj313:head, version: 111'313, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:54cd4daa:::obj313:head@111'313, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.612+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:54cd4daa:::obj313:head, version: 111'313, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:54cd4daa:::obj313:head@111'313, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.612+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.612+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 448 ==== pg_backfill(progress 1.0 e 116/116 lb 1:54a9e115:::obj404:head) ==== 1062+0+0 (unknown 0 0 0) 0x5ff1180 con 0x4acb000 2026-02-11T23:31:27.612+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:54a9e115:::obj404:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.612+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:54cd4daa:::obj313:head, version: 111'313, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:54cd4daa:::obj313:head@111'313, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.612+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:54a9e115:::obj404:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.612+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.612+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:54cd4daa:::obj313:head, version: 111'313, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:54cd4daa:::obj313:head@111'313, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.612+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:54cd4daa:::obj313:head, version: 111'313, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:54cd4daa:::obj313:head@111'313, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:54cd4daa:::obj313:head, version: 111'313, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:54cd4daa:::obj313:head@111'313, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:545db4bf:::obj139:head local-lis/les=113/114 n=205 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.612+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.612+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.612+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:545db4bf:::obj139:head local-lis/les=113/114 n=205 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:54cd4daa:::obj313:head, version: 111'313, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:54cd4daa:::obj313:head@111'313, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.612+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:54a9e115:::obj404:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.612+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:545db4bf:::obj139:head local-lis/les=113/114 n=205 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:54cd4daa:::obj313:head@111'313, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.612+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:545db4bf:::obj139:head local-lis/les=113/114 n=205 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:54cd4daa:::obj313:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.612+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:545db4bf:::obj139:head local-lis/les=113/114 n=205 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:54cd4daa:::obj313:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.612+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:545db4bf:::obj139:head local-lis/les=113/114 n=205 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:54cd4daa:::obj313:head 2026-02-11T23:31:27.612+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:545db4bf:::obj139:head local-lis/les=113/114 n=205 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:54cd4daa:::obj313:head v 111'313 2026-02-11T23:31:27.612+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:545db4bf:::obj139:head local-lis/les=113/114 n=205 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.612+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.612+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:54cd4daa:::obj313:head, version: 111'313, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:54cd4daa:::obj313:head@111'313, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.612+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.612+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.612+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:54a9e115:::obj404:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.612+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:54a9e115:::obj404:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:54a9e115:::obj404:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:545db4bf:::obj139:head local-lis/les=113/114 n=205 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.612+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.612+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:545db4bf:::obj139:head local-lis/les=113/114 n=205 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:54a9e115:::obj404:head) v3 2026-02-11T23:31:27.612+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:545db4bf:::obj139:head local-lis/les=113/114 n=205 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:54a9e115:::obj404:head) 2026-02-11T23:31:27.612+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.612+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:54a9e115:::obj404:head) v3 finish latency 0.000009 2026-02-11T23:31:27.612+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:54a9e115:::obj404:head local-lis/les=113/114 n=206 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.612+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.613+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.613+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:54a9e115:::obj404:head local-lis/les=113/114 n=206 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.613+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.613+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.613+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.613+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fed200 con 0x4acb000 2026-02-11T23:31:27.613+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:54cd4daa:::obj313:head)]) -- 0x5d1de00 con 0x4acb000 2026-02-11T23:31:27.613+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.613+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 449 ==== MOSDPGPush(1.0 116/113 [PushOp(1:554d3990:::obj276:head, version: 111'276, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:554d3990:::obj276:head@111'276, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56ca000 con 0x4acb000 2026-02-11T23:31:27.613+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:554d3990:::obj276:head, version: 111'276, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:554d3990:::obj276:head@111'276, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.613+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:554d3990:::obj276:head, version: 111'276, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:554d3990:::obj276:head@111'276, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.613+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.613+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 450 ==== pg_backfill(progress 1.0 e 116/116 lb 1:54cd4daa:::obj313:head) ==== 1062+0+0 (unknown 0 0 0) 0x5ff1700 con 0x4acb000 2026-02-11T23:31:27.613+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:554d3990:::obj276:head, version: 111'276, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:554d3990:::obj276:head@111'276, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.613+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:54cd4daa:::obj313:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.613+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.613+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:54cd4daa:::obj313:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.613+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:554d3990:::obj276:head, version: 111'276, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:554d3990:::obj276:head@111'276, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.613+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.613+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:554d3990:::obj276:head, version: 111'276, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:554d3990:::obj276:head@111'276, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:554d3990:::obj276:head, version: 111'276, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:554d3990:::obj276:head@111'276, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:54a9e115:::obj404:head local-lis/les=113/114 n=206 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.613+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.613+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:54a9e115:::obj404:head local-lis/les=113/114 n=206 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:554d3990:::obj276:head, version: 111'276, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:554d3990:::obj276:head@111'276, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.613+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:54a9e115:::obj404:head local-lis/les=113/114 n=206 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:554d3990:::obj276:head@111'276, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.613+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.613+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:54cd4daa:::obj313:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.613+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:54a9e115:::obj404:head local-lis/les=113/114 n=206 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:554d3990:::obj276:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.613+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:54a9e115:::obj404:head local-lis/les=113/114 n=206 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:554d3990:::obj276:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.613+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:54a9e115:::obj404:head local-lis/les=113/114 n=206 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:554d3990:::obj276:head 2026-02-11T23:31:27.613+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:54a9e115:::obj404:head local-lis/les=113/114 n=206 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:554d3990:::obj276:head v 111'276 2026-02-11T23:31:27.613+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:54a9e115:::obj404:head local-lis/les=113/114 n=206 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.613+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.613+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:554d3990:::obj276:head, version: 111'276, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:554d3990:::obj276:head@111'276, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:27.613+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:54a9e115:::obj404:head local-lis/les=113/114 n=206 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.613+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.613+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.613+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:54cd4daa:::obj313:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.613+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:54cd4daa:::obj313:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 1.0 e 116/116 lb 1:54cd4daa:::obj313:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:54a9e115:::obj404:head local-lis/les=113/114 n=206 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.613+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.613+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:54a9e115:::obj404:head local-lis/les=113/114 n=206 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:54cd4daa:::obj313:head) v3 2026-02-11T23:31:27.613+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:54a9e115:::obj404:head local-lis/les=113/114 n=206 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:54cd4daa:::obj313:head) 2026-02-11T23:31:27.613+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.613+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:54cd4daa:::obj313:head) v3 finish latency 0.000011 2026-02-11T23:31:27.613+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.613+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.613+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:54cd4daa:::obj313:head local-lis/les=113/114 n=207 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.613+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.613+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.613+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.613+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fed680 con 0x4acb000 2026-02-11T23:31:27.613+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:554d3990:::obj276:head)]) -- 0x5fed380 con 0x4acb000 2026-02-11T23:31:27.613+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.614+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 451 ==== MOSDPGPush(1.0 116/113 [PushOp(1:55c8626c:::obj516:head, version: 111'516, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:55c8626c:::obj516:head@111'516, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d96c0 con 0x4acb000 2026-02-11T23:31:27.614+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:55c8626c:::obj516:head, version: 111'516, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:55c8626c:::obj516:head@111'516, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 116 2026-02-11T23:31:27.614+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:55c8626c:::obj516:head, version: 111'516, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:55c8626c:::obj516:head@111'516, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.614+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.614+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 452 ==== pg_backfill(progress 1.0 e 116/116 lb 1:554d3990:::obj276:head) ==== 1062+0+0 (unknown 0 0 0) 0x5fee000 con 0x4acb000 2026-02-11T23:31:27.614+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:55c8626c:::obj516:head, version: 111'516, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:55c8626c:::obj516:head@111'516, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.614+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:554d3990:::obj276:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.614+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.614+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:554d3990:::obj276:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.614+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:55c8626c:::obj516:head, version: 111'516, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:55c8626c:::obj516:head@111'516, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.614+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.614+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:55c8626c:::obj516:head, version: 111'516, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:55c8626c:::obj516:head@111'516, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 116/113 [PushOp(1:55c8626c:::obj516:head, version: 111'516, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:55c8626c:::obj516:head@111'516, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:54cd4daa:::obj313:head local-lis/les=113/114 n=207 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.614+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.614+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:54cd4daa:::obj313:head local-lis/les=113/114 n=207 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:55c8626c:::obj516:head, version: 111'516, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:55c8626c:::obj516:head@111'516, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.614+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:54cd4daa:::obj313:head local-lis/les=113/114 n=207 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:55c8626c:::obj516:head@111'516, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.614+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.614+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:554d3990:::obj276:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.614+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:54cd4daa:::obj313:head local-lis/les=113/114 n=207 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:55c8626c:::obj516:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.614+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:54cd4daa:::obj313:head local-lis/les=113/114 n=207 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:55c8626c:::obj516:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.614+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:54cd4daa:::obj313:head local-lis/les=113/114 n=207 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:55c8626c:::obj516:head 2026-02-11T23:31:27.614+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:54cd4daa:::obj313:head local-lis/les=113/114 n=207 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:55c8626c:::obj516:head v 111'516 2026-02-11T23:31:27.614+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:54cd4daa:::obj313:head local-lis/les=113/114 n=207 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.614+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.614+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:55c8626c:::obj516:head, version: 111'516, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:55c8626c:::obj516:head@111'516, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:27.614+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.614+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.614+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:554d3990:::obj276:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.614+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:554d3990:::obj276:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 1.0 e 116/116 lb 1:554d3990:::obj276:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:54cd4daa:::obj313:head local-lis/les=113/114 n=207 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.614+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.614+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:54cd4daa:::obj313:head local-lis/les=113/114 n=207 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:554d3990:::obj276:head) v3 2026-02-11T23:31:27.614+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:54cd4daa:::obj313:head local-lis/les=113/114 n=207 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:554d3990:::obj276:head) 2026-02-11T23:31:27.614+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.614+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:554d3990:::obj276:head) v3 finish latency 0.000010 2026-02-11T23:31:27.614+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:554d3990:::obj276:head local-lis/les=113/114 n=208 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.614+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.614+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.614+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:554d3990:::obj276:head local-lis/les=113/114 n=208 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.614+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.614+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.614+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.614+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fed800 con 0x4acb000 2026-02-11T23:31:27.614+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:55c8626c:::obj516:head)]) -- 0x5fce180 con 0x4acb000 2026-02-11T23:31:27.614+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.614+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 453 ==== MOSDPGPush(1.0 116/113 [PushOp(1:55d2244d:::obj498:head, version: 111'498, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:55d2244d:::obj498:head@111'498, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d91e0 con 0x4acb000 2026-02-11T23:31:27.614+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:55d2244d:::obj498:head, version: 111'498, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:55d2244d:::obj498:head@111'498, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.614+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:55d2244d:::obj498:head, version: 111'498, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:55d2244d:::obj498:head@111'498, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.614+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.614+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 454 ==== pg_backfill(progress 1.0 e 116/116 lb 1:55c8626c:::obj516:head) ==== 1062+0+0 (unknown 0 0 0) 0x5fee580 con 0x4acb000 2026-02-11T23:31:27.614+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:55c8626c:::obj516:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.614+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:55d2244d:::obj498:head, version: 111'498, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:55d2244d:::obj498:head@111'498, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.614+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:55c8626c:::obj516:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.614+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.614+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:55d2244d:::obj498:head, version: 111'498, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:55d2244d:::obj498:head@111'498, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.614+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:55d2244d:::obj498:head, version: 111'498, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:55d2244d:::obj498:head@111'498, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:55d2244d:::obj498:head, version: 111'498, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:55d2244d:::obj498:head@111'498, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:554d3990:::obj276:head local-lis/les=113/114 n=208 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.614+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.614+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:554d3990:::obj276:head local-lis/les=113/114 n=208 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:55d2244d:::obj498:head, version: 111'498, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:55d2244d:::obj498:head@111'498, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.614+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.614+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:55c8626c:::obj516:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.614+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:554d3990:::obj276:head local-lis/les=113/114 n=208 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:55d2244d:::obj498:head@111'498, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.614+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:554d3990:::obj276:head local-lis/les=113/114 n=208 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:55d2244d:::obj498:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.614+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:554d3990:::obj276:head local-lis/les=113/114 n=208 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:55d2244d:::obj498:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.614+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:554d3990:::obj276:head local-lis/les=113/114 n=208 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:55d2244d:::obj498:head 2026-02-11T23:31:27.614+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:554d3990:::obj276:head local-lis/les=113/114 n=208 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:55d2244d:::obj498:head v 111'498 2026-02-11T23:31:27.614+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:554d3990:::obj276:head local-lis/les=113/114 n=208 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.614+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.614+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:55d2244d:::obj498:head, version: 111'498, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:55d2244d:::obj498:head@111'498, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.614+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.614+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.614+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:55c8626c:::obj516:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.614+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:55c8626c:::obj516:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 1.0 e 116/116 lb 1:55c8626c:::obj516:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:554d3990:::obj276:head local-lis/les=113/114 n=208 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.614+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.614+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:554d3990:::obj276:head local-lis/les=113/114 n=208 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:55c8626c:::obj516:head) v3 2026-02-11T23:31:27.614+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:554d3990:::obj276:head local-lis/les=113/114 n=208 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:55c8626c:::obj516:head) 2026-02-11T23:31:27.614+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.614+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:55c8626c:::obj516:head) v3 finish latency 0.000009 2026-02-11T23:31:27.614+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:55c8626c:::obj516:head local-lis/les=113/114 n=209 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.614+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.614+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.614+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:55c8626c:::obj516:head local-lis/les=113/114 n=209 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.614+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.614+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.614+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.614+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fed980 con 0x4acb000 2026-02-11T23:31:27.614+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:55d2244d:::obj498:head)]) -- 0x5fce480 con 0x4acb000 2026-02-11T23:31:27.614+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.615+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 455 ==== MOSDPGPush(1.0 116/113 [PushOp(1:565a3fbf:::obj474:head, version: 111'474, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:565a3fbf:::obj474:head@111'474, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d8d00 con 0x4acb000 2026-02-11T23:31:27.615+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:565a3fbf:::obj474:head, version: 111'474, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:565a3fbf:::obj474:head@111'474, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000018 epoch 116 2026-02-11T23:31:27.615+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:565a3fbf:::obj474:head, version: 111'474, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:565a3fbf:::obj474:head@111'474, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.615+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.615+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:565a3fbf:::obj474:head, version: 111'474, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:565a3fbf:::obj474:head@111'474, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.615+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 456 ==== pg_backfill(progress 1.0 e 116/116 lb 1:55d2244d:::obj498:head) ==== 1062+0+0 (unknown 0 0 0) 0x5feeb00 con 0x4acb000 2026-02-11T23:31:27.615+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.615+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:55d2244d:::obj498:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 116 2026-02-11T23:31:27.615+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:565a3fbf:::obj474:head, version: 111'474, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:565a3fbf:::obj474:head@111'474, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.615+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:55d2244d:::obj498:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.615+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:565a3fbf:::obj474:head, version: 111'474, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:565a3fbf:::obj474:head@111'474, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000046 MOSDPGPush(1.0 116/113 [PushOp(1:565a3fbf:::obj474:head, version: 111'474, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:565a3fbf:::obj474:head@111'474, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:55c8626c:::obj516:head local-lis/les=113/114 n=209 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.615+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.615+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.615+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:55c8626c:::obj516:head local-lis/les=113/114 n=209 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:565a3fbf:::obj474:head, version: 111'474, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:565a3fbf:::obj474:head@111'474, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.615+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:55d2244d:::obj498:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.615+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:55c8626c:::obj516:head local-lis/les=113/114 n=209 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:565a3fbf:::obj474:head@111'474, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.615+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:55c8626c:::obj516:head local-lis/les=113/114 n=209 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:565a3fbf:::obj474:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.615+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:55c8626c:::obj516:head local-lis/les=113/114 n=209 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:565a3fbf:::obj474:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.615+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:55c8626c:::obj516:head local-lis/les=113/114 n=209 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:565a3fbf:::obj474:head 2026-02-11T23:31:27.615+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:55c8626c:::obj516:head local-lis/les=113/114 n=209 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:565a3fbf:::obj474:head v 111'474 2026-02-11T23:31:27.615+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:55c8626c:::obj516:head local-lis/les=113/114 n=209 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.615+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.615+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:565a3fbf:::obj474:head, version: 111'474, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:565a3fbf:::obj474:head@111'474, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.615+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.615+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.615+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:55d2244d:::obj498:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.615+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:55d2244d:::obj498:head) v3 prio 15 cost 0 latency 0.000060 pg_backfill(progress 1.0 e 116/116 lb 1:55d2244d:::obj498:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:55c8626c:::obj516:head local-lis/les=113/114 n=209 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.615+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.615+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:55c8626c:::obj516:head local-lis/les=113/114 n=209 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:55d2244d:::obj498:head) v3 2026-02-11T23:31:27.615+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:55c8626c:::obj516:head local-lis/les=113/114 n=209 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:55d2244d:::obj498:head) 2026-02-11T23:31:27.615+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.615+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:55d2244d:::obj498:head) v3 finish latency 0.000009 2026-02-11T23:31:27.615+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:55d2244d:::obj498:head local-lis/les=113/114 n=210 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.615+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.615+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.615+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:55d2244d:::obj498:head local-lis/les=113/114 n=210 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.615+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.615+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.615+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.615+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fedb00 con 0x4acb000 2026-02-11T23:31:27.615+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:565a3fbf:::obj474:head)]) -- 0x5fce780 con 0x4acb000 2026-02-11T23:31:27.615+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.615+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 457 ==== MOSDPGPush(1.0 116/113 [PushOp(1:5695f913:::obj241:head, version: 111'241, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5695f913:::obj241:head@111'241, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56471e0 con 0x4acb000 2026-02-11T23:31:27.615+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:5695f913:::obj241:head, version: 111'241, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5695f913:::obj241:head@111'241, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 116 2026-02-11T23:31:27.615+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:5695f913:::obj241:head, version: 111'241, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5695f913:::obj241:head@111'241, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.615+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.615+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 458 ==== pg_backfill(progress 1.0 e 116/116 lb 1:565a3fbf:::obj474:head) ==== 1062+0+0 (unknown 0 0 0) 0x5fef080 con 0x4acb000 2026-02-11T23:31:27.615+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:5695f913:::obj241:head, version: 111'241, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5695f913:::obj241:head@111'241, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.615+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:565a3fbf:::obj474:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.615+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.615+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:565a3fbf:::obj474:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.615+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:5695f913:::obj241:head, version: 111'241, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5695f913:::obj241:head@111'241, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.615+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.615+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:5695f913:::obj241:head, version: 111'241, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5695f913:::obj241:head@111'241, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 116/113 [PushOp(1:5695f913:::obj241:head, version: 111'241, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5695f913:::obj241:head@111'241, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:55d2244d:::obj498:head local-lis/les=113/114 n=210 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.615+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.615+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:55d2244d:::obj498:head local-lis/les=113/114 n=210 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:5695f913:::obj241:head, version: 111'241, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5695f913:::obj241:head@111'241, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.615+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:55d2244d:::obj498:head local-lis/les=113/114 n=210 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:5695f913:::obj241:head@111'241, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.615+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.615+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:565a3fbf:::obj474:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.615+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:55d2244d:::obj498:head local-lis/les=113/114 n=210 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:5695f913:::obj241:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.615+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:55d2244d:::obj498:head local-lis/les=113/114 n=210 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:5695f913:::obj241:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.615+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:55d2244d:::obj498:head local-lis/les=113/114 n=210 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:5695f913:::obj241:head 2026-02-11T23:31:27.615+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:55d2244d:::obj498:head local-lis/les=113/114 n=210 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:5695f913:::obj241:head v 111'241 2026-02-11T23:31:27.615+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:55d2244d:::obj498:head local-lis/les=113/114 n=210 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.615+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.615+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:5695f913:::obj241:head, version: 111'241, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5695f913:::obj241:head@111'241, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.615+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.615+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.615+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:565a3fbf:::obj474:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.615+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:565a3fbf:::obj474:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:565a3fbf:::obj474:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:55d2244d:::obj498:head local-lis/les=113/114 n=210 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.615+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.615+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:55d2244d:::obj498:head local-lis/les=113/114 n=210 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:565a3fbf:::obj474:head) v3 2026-02-11T23:31:27.615+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:55d2244d:::obj498:head local-lis/les=113/114 n=210 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:565a3fbf:::obj474:head) 2026-02-11T23:31:27.615+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.615+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:565a3fbf:::obj474:head) v3 finish latency 0.000008 2026-02-11T23:31:27.615+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:565a3fbf:::obj474:head local-lis/les=113/114 n=211 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.615+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.615+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.615+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:565a3fbf:::obj474:head local-lis/les=113/114 n=211 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.615+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.615+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.616+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.616+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fedc80 con 0x4acb000 2026-02-11T23:31:27.616+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:5695f913:::obj241:head)]) -- 0x5fcea80 con 0x4acb000 2026-02-11T23:31:27.616+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.616+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 459 ==== MOSDPGPush(1.0 116/113 [PushOp(1:56fc6216:::obj287:head, version: 111'287, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:56fc6216:::obj287:head@111'287, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d8340 con 0x4acb000 2026-02-11T23:31:27.616+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:56fc6216:::obj287:head, version: 111'287, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:56fc6216:::obj287:head@111'287, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.616+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:56fc6216:::obj287:head, version: 111'287, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:56fc6216:::obj287:head@111'287, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.616+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.616+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 460 ==== pg_backfill(progress 1.0 e 116/116 lb 1:5695f913:::obj241:head) ==== 1062+0+0 (unknown 0 0 0) 0x5fef600 con 0x4acb000 2026-02-11T23:31:27.616+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:56fc6216:::obj287:head, version: 111'287, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:56fc6216:::obj287:head@111'287, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.616+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.616+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:56fc6216:::obj287:head, version: 111'287, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:56fc6216:::obj287:head@111'287, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.616+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:5695f913:::obj241:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 116 2026-02-11T23:31:27.616+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.616+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:5695f913:::obj241:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.616+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:56fc6216:::obj287:head, version: 111'287, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:56fc6216:::obj287:head@111'287, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:56fc6216:::obj287:head, version: 111'287, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:56fc6216:::obj287:head@111'287, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:565a3fbf:::obj474:head local-lis/les=113/114 n=211 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.616+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.616+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:565a3fbf:::obj474:head local-lis/les=113/114 n=211 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:56fc6216:::obj287:head, version: 111'287, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:56fc6216:::obj287:head@111'287, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.616+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:565a3fbf:::obj474:head local-lis/les=113/114 n=211 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:56fc6216:::obj287:head@111'287, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.616+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.616+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:5695f913:::obj241:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.616+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:565a3fbf:::obj474:head local-lis/les=113/114 n=211 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:56fc6216:::obj287:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.616+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:565a3fbf:::obj474:head local-lis/les=113/114 n=211 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:56fc6216:::obj287:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.616+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:565a3fbf:::obj474:head local-lis/les=113/114 n=211 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:56fc6216:::obj287:head 2026-02-11T23:31:27.616+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:565a3fbf:::obj474:head local-lis/les=113/114 n=211 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:56fc6216:::obj287:head v 111'287 2026-02-11T23:31:27.616+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:565a3fbf:::obj474:head local-lis/les=113/114 n=211 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.616+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.616+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:56fc6216:::obj287:head, version: 111'287, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:56fc6216:::obj287:head@111'287, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.616+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.616+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.616+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:5695f913:::obj241:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.616+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:5695f913:::obj241:head) v3 prio 15 cost 0 latency 0.000060 pg_backfill(progress 1.0 e 116/116 lb 1:5695f913:::obj241:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:565a3fbf:::obj474:head local-lis/les=113/114 n=211 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.616+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.616+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:565a3fbf:::obj474:head local-lis/les=113/114 n=211 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:5695f913:::obj241:head) v3 2026-02-11T23:31:27.616+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:565a3fbf:::obj474:head local-lis/les=113/114 n=211 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:5695f913:::obj241:head) 2026-02-11T23:31:27.616+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.616+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:5695f913:::obj241:head) v3 finish latency 0.000009 2026-02-11T23:31:27.616+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5695f913:::obj241:head local-lis/les=113/114 n=212 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.616+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.616+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.616+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5695f913:::obj241:head local-lis/les=113/114 n=212 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.616+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.616+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.616+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.616+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fede00 con 0x4acb000 2026-02-11T23:31:27.616+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:56fc6216:::obj287:head)]) -- 0x5fced80 con 0x4acb000 2026-02-11T23:31:27.616+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.616+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 461 ==== MOSDPGPush(1.0 116/113 [PushOp(1:578304e3:::obj249:head, version: 111'249, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:578304e3:::obj249:head@111'249, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d9860 con 0x4acb000 2026-02-11T23:31:27.616+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:578304e3:::obj249:head, version: 111'249, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:578304e3:::obj249:head@111'249, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.616+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:578304e3:::obj249:head, version: 111'249, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:578304e3:::obj249:head@111'249, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.616+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.616+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 462 ==== pg_backfill(progress 1.0 e 116/116 lb 1:56fc6216:::obj287:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2d080 con 0x4acb000 2026-02-11T23:31:27.616+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:578304e3:::obj249:head, version: 111'249, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:578304e3:::obj249:head@111'249, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.616+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:56fc6216:::obj287:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.616+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.616+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:56fc6216:::obj287:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.616+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:578304e3:::obj249:head, version: 111'249, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:578304e3:::obj249:head@111'249, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.616+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.616+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:578304e3:::obj249:head, version: 111'249, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:578304e3:::obj249:head@111'249, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 116/113 [PushOp(1:578304e3:::obj249:head, version: 111'249, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:578304e3:::obj249:head@111'249, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:5695f913:::obj241:head local-lis/les=113/114 n=212 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.617+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.617+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.617+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5695f913:::obj241:head local-lis/les=113/114 n=212 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:578304e3:::obj249:head, version: 111'249, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:578304e3:::obj249:head@111'249, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.617+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:56fc6216:::obj287:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.617+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5695f913:::obj241:head local-lis/les=113/114 n=212 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:578304e3:::obj249:head@111'249, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.617+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5695f913:::obj241:head local-lis/les=113/114 n=212 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:578304e3:::obj249:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.617+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5695f913:::obj241:head local-lis/les=113/114 n=212 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:578304e3:::obj249:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.617+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5695f913:::obj241:head local-lis/les=113/114 n=212 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:578304e3:::obj249:head 2026-02-11T23:31:27.617+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5695f913:::obj241:head local-lis/les=113/114 n=212 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:578304e3:::obj249:head v 111'249 2026-02-11T23:31:27.617+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5695f913:::obj241:head local-lis/les=113/114 n=212 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.617+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.617+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:578304e3:::obj249:head, version: 111'249, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:578304e3:::obj249:head@111'249, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000065 2026-02-11T23:31:27.617+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.617+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.617+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:56fc6216:::obj287:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.617+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:56fc6216:::obj287:head) v3 prio 15 cost 0 latency 0.000081 pg_backfill(progress 1.0 e 116/116 lb 1:56fc6216:::obj287:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:5695f913:::obj241:head local-lis/les=113/114 n=212 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.617+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.617+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5695f913:::obj241:head local-lis/les=113/114 n=212 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:56fc6216:::obj287:head) v3 2026-02-11T23:31:27.617+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5695f913:::obj241:head local-lis/les=113/114 n=212 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:56fc6216:::obj287:head) 2026-02-11T23:31:27.617+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.617+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:56fc6216:::obj287:head) v3 finish latency 0.000009 2026-02-11T23:31:27.617+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:56fc6216:::obj287:head local-lis/les=113/114 n=213 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.617+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.617+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.617+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:56fc6216:::obj287:head local-lis/les=113/114 n=213 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.617+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.617+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.617+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.617+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x600a000 con 0x4acb000 2026-02-11T23:31:27.617+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:578304e3:::obj249:head)]) -- 0x5fcf080 con 0x4acb000 2026-02-11T23:31:27.617+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.617+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 463 ==== MOSDPGPush(1.0 116/113 [PushOp(1:57ba9973:::obj296:head, version: 111'296, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:57ba9973:::obj296:head@111'296, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d8820 con 0x4acb000 2026-02-11T23:31:27.617+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:57ba9973:::obj296:head, version: 111'296, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:57ba9973:::obj296:head@111'296, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.617+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:57ba9973:::obj296:head, version: 111'296, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:57ba9973:::obj296:head@111'296, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.617+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.617+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 464 ==== pg_backfill(progress 1.0 e 116/116 lb 1:578304e3:::obj249:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e680 con 0x4acb000 2026-02-11T23:31:27.617+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:578304e3:::obj249:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.617+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:57ba9973:::obj296:head, version: 111'296, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:57ba9973:::obj296:head@111'296, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.617+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:578304e3:::obj249:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.617+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.617+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:57ba9973:::obj296:head, version: 111'296, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:57ba9973:::obj296:head@111'296, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.617+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.617+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:57ba9973:::obj296:head, version: 111'296, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:57ba9973:::obj296:head@111'296, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:57ba9973:::obj296:head, version: 111'296, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:57ba9973:::obj296:head@111'296, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:56fc6216:::obj287:head local-lis/les=113/114 n=213 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.617+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.617+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:56fc6216:::obj287:head local-lis/les=113/114 n=213 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:57ba9973:::obj296:head, version: 111'296, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:57ba9973:::obj296:head@111'296, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.617+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:56fc6216:::obj287:head local-lis/les=113/114 n=213 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:57ba9973:::obj296:head@111'296, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.617+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.617+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:578304e3:::obj249:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.617+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:56fc6216:::obj287:head local-lis/les=113/114 n=213 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:57ba9973:::obj296:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.617+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:56fc6216:::obj287:head local-lis/les=113/114 n=213 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:57ba9973:::obj296:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.617+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:56fc6216:::obj287:head local-lis/les=113/114 n=213 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:57ba9973:::obj296:head 2026-02-11T23:31:27.617+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:56fc6216:::obj287:head local-lis/les=113/114 n=213 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:57ba9973:::obj296:head v 111'296 2026-02-11T23:31:27.617+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:56fc6216:::obj287:head local-lis/les=113/114 n=213 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.617+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.617+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:57ba9973:::obj296:head, version: 111'296, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:57ba9973:::obj296:head@111'296, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.617+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.617+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.617+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:578304e3:::obj249:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.617+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:578304e3:::obj249:head) v3 prio 15 cost 0 latency 0.000060 pg_backfill(progress 1.0 e 116/116 lb 1:578304e3:::obj249:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:56fc6216:::obj287:head local-lis/les=113/114 n=213 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.617+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.617+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:56fc6216:::obj287:head local-lis/les=113/114 n=213 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:578304e3:::obj249:head) v3 2026-02-11T23:31:27.617+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:56fc6216:::obj287:head local-lis/les=113/114 n=213 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:578304e3:::obj249:head) 2026-02-11T23:31:27.617+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.617+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:578304e3:::obj249:head) v3 finish latency 0.000009 2026-02-11T23:31:27.617+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:578304e3:::obj249:head local-lis/les=113/114 n=214 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.617+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.617+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.617+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:578304e3:::obj249:head local-lis/les=113/114 n=214 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.617+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.617+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.617+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.617+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x600a180 con 0x4acb000 2026-02-11T23:31:27.617+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:57ba9973:::obj296:head)]) -- 0x5fcf380 con 0x4acb000 2026-02-11T23:31:27.617+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.618+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 465 ==== MOSDPGPush(1.0 116/113 [PushOp(1:5880e7a7:::obj64:head, version: 111'64, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5880e7a7:::obj64:head@111'64, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x559cea0 con 0x4acb000 2026-02-11T23:31:27.618+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:5880e7a7:::obj64:head, version: 111'64, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5880e7a7:::obj64:head@111'64, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.618+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:5880e7a7:::obj64:head, version: 111'64, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5880e7a7:::obj64:head@111'64, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.618+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.618+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 466 ==== pg_backfill(progress 1.0 e 116/116 lb 1:57ba9973:::obj296:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e100 con 0x4acb000 2026-02-11T23:31:27.618+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:57ba9973:::obj296:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.618+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:5880e7a7:::obj64:head, version: 111'64, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5880e7a7:::obj64:head@111'64, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.618+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:57ba9973:::obj296:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.618+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.618+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:5880e7a7:::obj64:head, version: 111'64, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5880e7a7:::obj64:head@111'64, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.618+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:5880e7a7:::obj64:head, version: 111'64, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5880e7a7:::obj64:head@111'64, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:5880e7a7:::obj64:head, version: 111'64, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5880e7a7:::obj64:head@111'64, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:578304e3:::obj249:head local-lis/les=113/114 n=214 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.618+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.618+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.618+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:578304e3:::obj249:head local-lis/les=113/114 n=214 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:5880e7a7:::obj64:head, version: 111'64, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5880e7a7:::obj64:head@111'64, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.618+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:57ba9973:::obj296:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.618+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:578304e3:::obj249:head local-lis/les=113/114 n=214 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:5880e7a7:::obj64:head@111'64, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.618+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:578304e3:::obj249:head local-lis/les=113/114 n=214 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:5880e7a7:::obj64:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.618+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:578304e3:::obj249:head local-lis/les=113/114 n=214 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:5880e7a7:::obj64:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.618+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:578304e3:::obj249:head local-lis/les=113/114 n=214 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:5880e7a7:::obj64:head 2026-02-11T23:31:27.618+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:578304e3:::obj249:head local-lis/les=113/114 n=214 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:5880e7a7:::obj64:head v 111'64 2026-02-11T23:31:27.618+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:578304e3:::obj249:head local-lis/les=113/114 n=214 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.618+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.618+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:5880e7a7:::obj64:head, version: 111'64, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5880e7a7:::obj64:head@111'64, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.618+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.618+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.618+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:57ba9973:::obj296:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.618+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:57ba9973:::obj296:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:57ba9973:::obj296:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:578304e3:::obj249:head local-lis/les=113/114 n=214 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.618+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.618+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:578304e3:::obj249:head local-lis/les=113/114 n=214 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:57ba9973:::obj296:head) v3 2026-02-11T23:31:27.618+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:578304e3:::obj249:head local-lis/les=113/114 n=214 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:57ba9973:::obj296:head) 2026-02-11T23:31:27.618+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.618+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:57ba9973:::obj296:head) v3 finish latency 0.000009 2026-02-11T23:31:27.618+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:57ba9973:::obj296:head local-lis/les=113/114 n=215 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.618+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.618+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.618+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:57ba9973:::obj296:head local-lis/les=113/114 n=215 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.618+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.618+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.618+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.618+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x600a300 con 0x4acb000 2026-02-11T23:31:27.618+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:5880e7a7:::obj64:head)]) -- 0x5fcf680 con 0x4acb000 2026-02-11T23:31:27.618+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.618+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 467 ==== MOSDPGPush(1.0 116/113 [PushOp(1:58980d3a:::obj339:head, version: 111'339, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:58980d3a:::obj339:head@111'339, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559da00 con 0x4acb000 2026-02-11T23:31:27.618+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:58980d3a:::obj339:head, version: 111'339, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:58980d3a:::obj339:head@111'339, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.618+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:58980d3a:::obj339:head, version: 111'339, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:58980d3a:::obj339:head@111'339, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.618+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.618+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 468 ==== pg_backfill(progress 1.0 e 116/116 lb 1:5880e7a7:::obj64:head) ==== 1061+0+0 (unknown 0 0 0) 0x4a2cb00 con 0x4acb000 2026-02-11T23:31:27.618+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:5880e7a7:::obj64:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.618+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:58980d3a:::obj339:head, version: 111'339, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:58980d3a:::obj339:head@111'339, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.618+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:5880e7a7:::obj64:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.618+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.618+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:5880e7a7:::obj64:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.618+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.618+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:58980d3a:::obj339:head, version: 111'339, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:58980d3a:::obj339:head@111'339, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.618+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:58980d3a:::obj339:head, version: 111'339, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:58980d3a:::obj339:head@111'339, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000041 MOSDPGPush(1.0 116/113 [PushOp(1:58980d3a:::obj339:head, version: 111'339, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:58980d3a:::obj339:head@111'339, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:57ba9973:::obj296:head local-lis/les=113/114 n=215 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.618+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.618+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:57ba9973:::obj296:head local-lis/les=113/114 n=215 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:58980d3a:::obj339:head, version: 111'339, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:58980d3a:::obj339:head@111'339, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.618+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:57ba9973:::obj296:head local-lis/les=113/114 n=215 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:58980d3a:::obj339:head@111'339, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.618+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:57ba9973:::obj296:head local-lis/les=113/114 n=215 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:58980d3a:::obj339:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.618+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:57ba9973:::obj296:head local-lis/les=113/114 n=215 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:58980d3a:::obj339:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.618+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:57ba9973:::obj296:head local-lis/les=113/114 n=215 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:58980d3a:::obj339:head 2026-02-11T23:31:27.618+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:57ba9973:::obj296:head local-lis/les=113/114 n=215 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:58980d3a:::obj339:head v 111'339 2026-02-11T23:31:27.618+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:57ba9973:::obj296:head local-lis/les=113/114 n=215 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.618+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.618+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:58980d3a:::obj339:head, version: 111'339, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:58980d3a:::obj339:head@111'339, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.618+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.618+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.618+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:5880e7a7:::obj64:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.618+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:5880e7a7:::obj64:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 1.0 e 116/116 lb 1:5880e7a7:::obj64:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:57ba9973:::obj296:head local-lis/les=113/114 n=215 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.618+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.618+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:57ba9973:::obj296:head local-lis/les=113/114 n=215 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:5880e7a7:::obj64:head) v3 2026-02-11T23:31:27.618+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:57ba9973:::obj296:head local-lis/les=113/114 n=215 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:5880e7a7:::obj64:head) 2026-02-11T23:31:27.618+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.618+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:5880e7a7:::obj64:head) v3 finish latency 0.000010 2026-02-11T23:31:27.618+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5880e7a7:::obj64:head local-lis/les=113/114 n=216 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.618+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.618+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.618+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5880e7a7:::obj64:head local-lis/les=113/114 n=216 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.618+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.618+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.618+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.618+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x600a480 con 0x4acb000 2026-02-11T23:31:27.618+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:58980d3a:::obj339:head)]) -- 0x5fcf800 con 0x4acb000 2026-02-11T23:31:27.618+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.619+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 469 ==== MOSDPGPush(1.0 116/113 [PushOp(1:5946d48f:::obj257:head, version: 111'257, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5946d48f:::obj257:head@111'257, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c4e0 con 0x4acb000 2026-02-11T23:31:27.619+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:5946d48f:::obj257:head, version: 111'257, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5946d48f:::obj257:head@111'257, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.619+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:5946d48f:::obj257:head, version: 111'257, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5946d48f:::obj257:head@111'257, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.619+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.619+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 470 ==== pg_backfill(progress 1.0 e 116/116 lb 1:58980d3a:::obj339:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99080 con 0x4acb000 2026-02-11T23:31:27.619+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:5946d48f:::obj257:head, version: 111'257, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5946d48f:::obj257:head@111'257, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.619+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:58980d3a:::obj339:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.619+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.619+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:58980d3a:::obj339:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.619+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:5946d48f:::obj257:head, version: 111'257, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5946d48f:::obj257:head@111'257, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.619+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.619+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:5946d48f:::obj257:head, version: 111'257, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5946d48f:::obj257:head@111'257, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:5946d48f:::obj257:head, version: 111'257, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5946d48f:::obj257:head@111'257, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:5880e7a7:::obj64:head local-lis/les=113/114 n=216 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.619+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.619+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5880e7a7:::obj64:head local-lis/les=113/114 n=216 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:5946d48f:::obj257:head, version: 111'257, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5946d48f:::obj257:head@111'257, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.619+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5880e7a7:::obj64:head local-lis/les=113/114 n=216 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:5946d48f:::obj257:head@111'257, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.619+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.619+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:58980d3a:::obj339:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.619+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5880e7a7:::obj64:head local-lis/les=113/114 n=216 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:5946d48f:::obj257:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.619+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5880e7a7:::obj64:head local-lis/les=113/114 n=216 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:5946d48f:::obj257:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.619+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5880e7a7:::obj64:head local-lis/les=113/114 n=216 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:5946d48f:::obj257:head 2026-02-11T23:31:27.619+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5880e7a7:::obj64:head local-lis/les=113/114 n=216 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:5946d48f:::obj257:head v 111'257 2026-02-11T23:31:27.619+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5880e7a7:::obj64:head local-lis/les=113/114 n=216 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.619+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.619+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:5946d48f:::obj257:head, version: 111'257, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5946d48f:::obj257:head@111'257, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:27.619+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5880e7a7:::obj64:head local-lis/les=113/114 n=216 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.619+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.619+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.619+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:58980d3a:::obj339:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.619+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:58980d3a:::obj339:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 1.0 e 116/116 lb 1:58980d3a:::obj339:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:5880e7a7:::obj64:head local-lis/les=113/114 n=216 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.619+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.619+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5880e7a7:::obj64:head local-lis/les=113/114 n=216 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:58980d3a:::obj339:head) v3 2026-02-11T23:31:27.619+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5880e7a7:::obj64:head local-lis/les=113/114 n=216 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:58980d3a:::obj339:head) 2026-02-11T23:31:27.619+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.619+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:58980d3a:::obj339:head) v3 finish latency 0.000012 2026-02-11T23:31:27.619+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.619+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.619+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:58980d3a:::obj339:head local-lis/les=113/114 n=217 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.619+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.619+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.619+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.619+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x600a900 con 0x4acb000 2026-02-11T23:31:27.619+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:5946d48f:::obj257:head)]) -- 0x600a600 con 0x4acb000 2026-02-11T23:31:27.619+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.619+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 471 ==== MOSDPGPush(1.0 116/113 [PushOp(1:5a01aba7:::obj211:head, version: 111'211, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a01aba7:::obj211:head@111'211, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f1860 con 0x4acb000 2026-02-11T23:31:27.619+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:5a01aba7:::obj211:head, version: 111'211, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a01aba7:::obj211:head@111'211, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.619+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:5a01aba7:::obj211:head, version: 111'211, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a01aba7:::obj211:head@111'211, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.619+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.619+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 472 ==== pg_backfill(progress 1.0 e 116/116 lb 1:5946d48f:::obj257:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99600 con 0x4acb000 2026-02-11T23:31:27.619+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:5a01aba7:::obj211:head, version: 111'211, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a01aba7:::obj211:head@111'211, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.619+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:5946d48f:::obj257:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.619+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:5946d48f:::obj257:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.619+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.619+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:5a01aba7:::obj211:head, version: 111'211, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a01aba7:::obj211:head@111'211, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.619+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.619+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:5a01aba7:::obj211:head, version: 111'211, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a01aba7:::obj211:head@111'211, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 116/113 [PushOp(1:5a01aba7:::obj211:head, version: 111'211, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a01aba7:::obj211:head@111'211, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:58980d3a:::obj339:head local-lis/les=113/114 n=217 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.619+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.619+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:58980d3a:::obj339:head local-lis/les=113/114 n=217 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:5a01aba7:::obj211:head, version: 111'211, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a01aba7:::obj211:head@111'211, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.619+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.619+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:58980d3a:::obj339:head local-lis/les=113/114 n=217 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:5a01aba7:::obj211:head@111'211, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.619+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:5946d48f:::obj257:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.619+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:58980d3a:::obj339:head local-lis/les=113/114 n=217 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:5a01aba7:::obj211:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.619+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:58980d3a:::obj339:head local-lis/les=113/114 n=217 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:5a01aba7:::obj211:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.619+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:58980d3a:::obj339:head local-lis/les=113/114 n=217 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:5a01aba7:::obj211:head 2026-02-11T23:31:27.619+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:58980d3a:::obj339:head local-lis/les=113/114 n=217 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:5a01aba7:::obj211:head v 111'211 2026-02-11T23:31:27.619+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:58980d3a:::obj339:head local-lis/les=113/114 n=217 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.619+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.619+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:5a01aba7:::obj211:head, version: 111'211, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a01aba7:::obj211:head@111'211, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.619+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.619+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.619+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:5946d48f:::obj257:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.619+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:5946d48f:::obj257:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 116/116 lb 1:5946d48f:::obj257:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:58980d3a:::obj339:head local-lis/les=113/114 n=217 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.619+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.619+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:58980d3a:::obj339:head local-lis/les=113/114 n=217 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:5946d48f:::obj257:head) v3 2026-02-11T23:31:27.619+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:58980d3a:::obj339:head local-lis/les=113/114 n=217 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:5946d48f:::obj257:head) 2026-02-11T23:31:27.619+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.619+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:5946d48f:::obj257:head) v3 finish latency 0.000011 2026-02-11T23:31:27.619+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5946d48f:::obj257:head local-lis/les=113/114 n=218 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.619+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.620+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.620+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5946d48f:::obj257:head local-lis/les=113/114 n=218 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.620+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.620+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.620+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.620+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x600aa80 con 0x4acb000 2026-02-11T23:31:27.620+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:5a01aba7:::obj211:head)]) -- 0x5fcfb00 con 0x4acb000 2026-02-11T23:31:27.620+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.620+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 473 ==== MOSDPGPush(1.0 116/113 [PushOp(1:5a322cd1:::obj402:head, version: 111'402, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a322cd1:::obj402:head@111'402, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f0d00 con 0x4acb000 2026-02-11T23:31:27.620+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:5a322cd1:::obj402:head, version: 111'402, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a322cd1:::obj402:head@111'402, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.620+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:5a322cd1:::obj402:head, version: 111'402, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a322cd1:::obj402:head@111'402, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.620+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.620+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 474 ==== pg_backfill(progress 1.0 e 116/116 lb 1:5a01aba7:::obj211:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99b80 con 0x4acb000 2026-02-11T23:31:27.620+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:5a01aba7:::obj211:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.620+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:5a322cd1:::obj402:head, version: 111'402, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a322cd1:::obj402:head@111'402, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.620+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:5a01aba7:::obj211:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.620+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.620+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:5a322cd1:::obj402:head, version: 111'402, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a322cd1:::obj402:head@111'402, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.620+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:5a322cd1:::obj402:head, version: 111'402, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a322cd1:::obj402:head@111'402, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 116/113 [PushOp(1:5a322cd1:::obj402:head, version: 111'402, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a322cd1:::obj402:head@111'402, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:5946d48f:::obj257:head local-lis/les=113/114 n=218 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.620+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.620+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.620+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5946d48f:::obj257:head local-lis/les=113/114 n=218 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:5a322cd1:::obj402:head, version: 111'402, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a322cd1:::obj402:head@111'402, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.620+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:5a01aba7:::obj211:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.620+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5946d48f:::obj257:head local-lis/les=113/114 n=218 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:5a322cd1:::obj402:head@111'402, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.620+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5946d48f:::obj257:head local-lis/les=113/114 n=218 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:5a322cd1:::obj402:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.620+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5946d48f:::obj257:head local-lis/les=113/114 n=218 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:5a322cd1:::obj402:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.620+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5946d48f:::obj257:head local-lis/les=113/114 n=218 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:5a322cd1:::obj402:head 2026-02-11T23:31:27.620+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5946d48f:::obj257:head local-lis/les=113/114 n=218 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:5a322cd1:::obj402:head v 111'402 2026-02-11T23:31:27.620+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5946d48f:::obj257:head local-lis/les=113/114 n=218 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.620+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.620+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:5a322cd1:::obj402:head, version: 111'402, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a322cd1:::obj402:head@111'402, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.620+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.620+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.620+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:5a01aba7:::obj211:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.620+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:5a01aba7:::obj211:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 116/116 lb 1:5a01aba7:::obj211:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:5946d48f:::obj257:head local-lis/les=113/114 n=218 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.620+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.620+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5946d48f:::obj257:head local-lis/les=113/114 n=218 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:5a01aba7:::obj211:head) v3 2026-02-11T23:31:27.620+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5946d48f:::obj257:head local-lis/les=113/114 n=218 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:5a01aba7:::obj211:head) 2026-02-11T23:31:27.620+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.620+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:5a01aba7:::obj211:head) v3 finish latency 0.000009 2026-02-11T23:31:27.620+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5a01aba7:::obj211:head local-lis/les=113/114 n=219 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.620+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.620+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.620+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5a01aba7:::obj211:head local-lis/les=113/114 n=219 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.620+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.620+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.620+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.620+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x600ac00 con 0x4acb000 2026-02-11T23:31:27.620+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:5a322cd1:::obj402:head)]) -- 0x5fcfe00 con 0x4acb000 2026-02-11T23:31:27.620+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.620+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 475 ==== MOSDPGPush(1.0 116/113 [PushOp(1:5a3c6224:::obj352:head, version: 111'352, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a3c6224:::obj352:head@111'352, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c000 con 0x4acb000 2026-02-11T23:31:27.620+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:5a3c6224:::obj352:head, version: 111'352, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a3c6224:::obj352:head@111'352, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.620+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:5a3c6224:::obj352:head, version: 111'352, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a3c6224:::obj352:head@111'352, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.620+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.620+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 476 ==== pg_backfill(progress 1.0 e 116/116 lb 1:5a322cd1:::obj402:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9a100 con 0x4acb000 2026-02-11T23:31:27.620+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:5a322cd1:::obj402:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.620+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:5a3c6224:::obj352:head, version: 111'352, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a3c6224:::obj352:head@111'352, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.620+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:5a322cd1:::obj402:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.620+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.620+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:5a3c6224:::obj352:head, version: 111'352, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a3c6224:::obj352:head@111'352, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.621+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:5a3c6224:::obj352:head, version: 111'352, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a3c6224:::obj352:head@111'352, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 116/113 [PushOp(1:5a3c6224:::obj352:head, version: 111'352, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a3c6224:::obj352:head@111'352, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:5a01aba7:::obj211:head local-lis/les=113/114 n=219 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.621+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.621+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.621+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5a01aba7:::obj211:head local-lis/les=113/114 n=219 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:5a3c6224:::obj352:head, version: 111'352, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a3c6224:::obj352:head@111'352, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.621+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:5a322cd1:::obj402:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.621+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5a01aba7:::obj211:head local-lis/les=113/114 n=219 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:5a3c6224:::obj352:head@111'352, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.621+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5a01aba7:::obj211:head local-lis/les=113/114 n=219 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:5a3c6224:::obj352:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.621+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5a01aba7:::obj211:head local-lis/les=113/114 n=219 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:5a3c6224:::obj352:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.621+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5a01aba7:::obj211:head local-lis/les=113/114 n=219 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:5a3c6224:::obj352:head 2026-02-11T23:31:27.621+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5a01aba7:::obj211:head local-lis/les=113/114 n=219 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:5a3c6224:::obj352:head v 111'352 2026-02-11T23:31:27.621+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5a01aba7:::obj211:head local-lis/les=113/114 n=219 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.621+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.621+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:5a3c6224:::obj352:head, version: 111'352, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a3c6224:::obj352:head@111'352, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.621+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5a01aba7:::obj211:head local-lis/les=113/114 n=219 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.621+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.621+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.621+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:5a322cd1:::obj402:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.621+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:5a322cd1:::obj402:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 1.0 e 116/116 lb 1:5a322cd1:::obj402:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:5a01aba7:::obj211:head local-lis/les=113/114 n=219 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.621+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.621+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5a01aba7:::obj211:head local-lis/les=113/114 n=219 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:5a322cd1:::obj402:head) v3 2026-02-11T23:31:27.621+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5a01aba7:::obj211:head local-lis/les=113/114 n=219 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:5a322cd1:::obj402:head) 2026-02-11T23:31:27.621+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.621+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:5a322cd1:::obj402:head) v3 finish latency 0.000011 2026-02-11T23:31:27.621+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.621+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.621+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5a322cd1:::obj402:head local-lis/les=113/114 n=220 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.621+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.621+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.621+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.621+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x600b080 con 0x4acb000 2026-02-11T23:31:27.621+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:5a3c6224:::obj352:head)]) -- 0x600ad80 con 0x4acb000 2026-02-11T23:31:27.621+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.621+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 477 ==== MOSDPGPush(1.0 116/113 [PushOp(1:5d17b8a1:::obj341:head, version: 111'341, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5d17b8a1:::obj341:head@111'341, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f1380 con 0x4acb000 2026-02-11T23:31:27.621+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:5d17b8a1:::obj341:head, version: 111'341, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5d17b8a1:::obj341:head@111'341, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.621+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:5d17b8a1:::obj341:head, version: 111'341, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5d17b8a1:::obj341:head@111'341, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.621+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.621+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 478 ==== pg_backfill(progress 1.0 e 116/116 lb 1:5a3c6224:::obj352:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9a680 con 0x4acb000 2026-02-11T23:31:27.621+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:5a3c6224:::obj352:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.621+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:5d17b8a1:::obj341:head, version: 111'341, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5d17b8a1:::obj341:head@111'341, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.621+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:5a3c6224:::obj352:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.621+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.621+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:5d17b8a1:::obj341:head, version: 111'341, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5d17b8a1:::obj341:head@111'341, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.621+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.621+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:5d17b8a1:::obj341:head, version: 111'341, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5d17b8a1:::obj341:head@111'341, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 116/113 [PushOp(1:5d17b8a1:::obj341:head, version: 111'341, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5d17b8a1:::obj341:head@111'341, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:5a322cd1:::obj402:head local-lis/les=113/114 n=220 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.621+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.621+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:5a3c6224:::obj352:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.621+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5a322cd1:::obj402:head local-lis/les=113/114 n=220 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:5d17b8a1:::obj341:head, version: 111'341, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5d17b8a1:::obj341:head@111'341, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.621+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5a322cd1:::obj402:head local-lis/les=113/114 n=220 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:5d17b8a1:::obj341:head@111'341, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.621+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5a322cd1:::obj402:head local-lis/les=113/114 n=220 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:5d17b8a1:::obj341:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.621+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5a322cd1:::obj402:head local-lis/les=113/114 n=220 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:5d17b8a1:::obj341:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.621+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5a322cd1:::obj402:head local-lis/les=113/114 n=220 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:5d17b8a1:::obj341:head 2026-02-11T23:31:27.621+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5a322cd1:::obj402:head local-lis/les=113/114 n=220 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:5d17b8a1:::obj341:head v 111'341 2026-02-11T23:31:27.621+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5a322cd1:::obj402:head local-lis/les=113/114 n=220 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.621+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.621+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:5d17b8a1:::obj341:head, version: 111'341, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5d17b8a1:::obj341:head@111'341, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.621+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.621+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.621+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:5a3c6224:::obj352:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.621+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:5a3c6224:::obj352:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 1.0 e 116/116 lb 1:5a3c6224:::obj352:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:5a322cd1:::obj402:head local-lis/les=113/114 n=220 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.621+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.621+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5a322cd1:::obj402:head local-lis/les=113/114 n=220 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:5a3c6224:::obj352:head) v3 2026-02-11T23:31:27.621+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5a322cd1:::obj402:head local-lis/les=113/114 n=220 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:5a3c6224:::obj352:head) 2026-02-11T23:31:27.621+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.621+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:5a3c6224:::obj352:head) v3 finish latency 0.000010 2026-02-11T23:31:27.621+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5a3c6224:::obj352:head local-lis/les=113/114 n=221 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.621+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.621+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.621+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5a3c6224:::obj352:head local-lis/les=113/114 n=221 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.621+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.621+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.621+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.621+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x600b200 con 0x4acb000 2026-02-11T23:31:27.621+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:5d17b8a1:::obj341:head)]) -- 0x6000180 con 0x4acb000 2026-02-11T23:31:27.621+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.622+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 479 ==== MOSDPGPush(1.0 116/113 [PushOp(1:5e27c68c:::obj582:head, version: 111'582, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5e27c68c:::obj582:head@111'582, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559cd00 con 0x4acb000 2026-02-11T23:31:27.622+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:5e27c68c:::obj582:head, version: 111'582, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5e27c68c:::obj582:head@111'582, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.622+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:5e27c68c:::obj582:head, version: 111'582, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5e27c68c:::obj582:head@111'582, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.622+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.622+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 480 ==== pg_backfill(progress 1.0 e 116/116 lb 1:5d17b8a1:::obj341:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9ac00 con 0x4acb000 2026-02-11T23:31:27.622+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:5e27c68c:::obj582:head, version: 111'582, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5e27c68c:::obj582:head@111'582, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.622+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:5d17b8a1:::obj341:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.622+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:5d17b8a1:::obj341:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.622+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.622+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:5e27c68c:::obj582:head, version: 111'582, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5e27c68c:::obj582:head@111'582, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.622+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.622+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:5e27c68c:::obj582:head, version: 111'582, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5e27c68c:::obj582:head@111'582, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:5e27c68c:::obj582:head, version: 111'582, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5e27c68c:::obj582:head@111'582, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:5a3c6224:::obj352:head local-lis/les=113/114 n=221 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.622+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.622+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5a3c6224:::obj352:head local-lis/les=113/114 n=221 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:5e27c68c:::obj582:head, version: 111'582, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5e27c68c:::obj582:head@111'582, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.622+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5a3c6224:::obj352:head local-lis/les=113/114 n=221 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:5e27c68c:::obj582:head@111'582, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.622+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.622+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:5d17b8a1:::obj341:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.622+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5a3c6224:::obj352:head local-lis/les=113/114 n=221 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:5e27c68c:::obj582:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.622+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5a3c6224:::obj352:head local-lis/les=113/114 n=221 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:5e27c68c:::obj582:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.622+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5a3c6224:::obj352:head local-lis/les=113/114 n=221 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:5e27c68c:::obj582:head 2026-02-11T23:31:27.622+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5a3c6224:::obj352:head local-lis/les=113/114 n=221 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:5e27c68c:::obj582:head v 111'582 2026-02-11T23:31:27.622+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5a3c6224:::obj352:head local-lis/les=113/114 n=221 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.622+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.622+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:5e27c68c:::obj582:head, version: 111'582, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5e27c68c:::obj582:head@111'582, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.622+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.622+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.622+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:5d17b8a1:::obj341:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.622+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:5d17b8a1:::obj341:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 1.0 e 116/116 lb 1:5d17b8a1:::obj341:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:5a3c6224:::obj352:head local-lis/les=113/114 n=221 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.622+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.622+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5a3c6224:::obj352:head local-lis/les=113/114 n=221 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:5d17b8a1:::obj341:head) v3 2026-02-11T23:31:27.622+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5a3c6224:::obj352:head local-lis/les=113/114 n=221 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:5d17b8a1:::obj341:head) 2026-02-11T23:31:27.622+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.622+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:5d17b8a1:::obj341:head) v3 finish latency 0.000009 2026-02-11T23:31:27.622+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5d17b8a1:::obj341:head local-lis/les=113/114 n=222 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.622+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.622+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.622+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5d17b8a1:::obj341:head local-lis/les=113/114 n=222 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.622+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.622+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.622+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.622+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x600b380 con 0x4acb000 2026-02-11T23:31:27.622+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:5e27c68c:::obj582:head)]) -- 0x6000480 con 0x4acb000 2026-02-11T23:31:27.622+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.622+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 481 ==== MOSDPGPush(1.0 116/113 [PushOp(1:6131e0b1:::obj290:head, version: 111'290, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6131e0b1:::obj290:head@111'290, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559d1e0 con 0x4acb000 2026-02-11T23:31:27.622+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:6131e0b1:::obj290:head, version: 111'290, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6131e0b1:::obj290:head@111'290, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.622+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:6131e0b1:::obj290:head, version: 111'290, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6131e0b1:::obj290:head@111'290, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.622+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.622+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 482 ==== pg_backfill(progress 1.0 e 116/116 lb 1:5e27c68c:::obj582:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9b180 con 0x4acb000 2026-02-11T23:31:27.622+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:5e27c68c:::obj582:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.622+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:6131e0b1:::obj290:head, version: 111'290, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6131e0b1:::obj290:head@111'290, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.622+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:5e27c68c:::obj582:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.622+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.622+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:6131e0b1:::obj290:head, version: 111'290, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6131e0b1:::obj290:head@111'290, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.622+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:6131e0b1:::obj290:head, version: 111'290, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6131e0b1:::obj290:head@111'290, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 116/113 [PushOp(1:6131e0b1:::obj290:head, version: 111'290, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6131e0b1:::obj290:head@111'290, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:5d17b8a1:::obj341:head local-lis/les=113/114 n=222 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.622+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.622+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.622+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5d17b8a1:::obj341:head local-lis/les=113/114 n=222 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:6131e0b1:::obj290:head, version: 111'290, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6131e0b1:::obj290:head@111'290, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.622+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:5e27c68c:::obj582:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.622+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5d17b8a1:::obj341:head local-lis/les=113/114 n=222 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:6131e0b1:::obj290:head@111'290, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.622+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5d17b8a1:::obj341:head local-lis/les=113/114 n=222 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:6131e0b1:::obj290:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.622+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5d17b8a1:::obj341:head local-lis/les=113/114 n=222 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:6131e0b1:::obj290:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.622+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5d17b8a1:::obj341:head local-lis/les=113/114 n=222 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:6131e0b1:::obj290:head 2026-02-11T23:31:27.622+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5d17b8a1:::obj341:head local-lis/les=113/114 n=222 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:6131e0b1:::obj290:head v 111'290 2026-02-11T23:31:27.622+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5d17b8a1:::obj341:head local-lis/les=113/114 n=222 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.622+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.622+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:6131e0b1:::obj290:head, version: 111'290, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6131e0b1:::obj290:head@111'290, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000044 2026-02-11T23:31:27.622+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.622+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.622+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:5e27c68c:::obj582:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.622+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:5e27c68c:::obj582:head) v3 prio 15 cost 0 latency 0.000060 pg_backfill(progress 1.0 e 116/116 lb 1:5e27c68c:::obj582:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:5d17b8a1:::obj341:head local-lis/les=113/114 n=222 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.622+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.622+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5d17b8a1:::obj341:head local-lis/les=113/114 n=222 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:5e27c68c:::obj582:head) v3 2026-02-11T23:31:27.622+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5d17b8a1:::obj341:head local-lis/les=113/114 n=222 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:5e27c68c:::obj582:head) 2026-02-11T23:31:27.622+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.622+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:5e27c68c:::obj582:head) v3 finish latency 0.000009 2026-02-11T23:31:27.622+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5e27c68c:::obj582:head local-lis/les=113/114 n=223 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.622+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.623+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.623+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5e27c68c:::obj582:head local-lis/les=113/114 n=223 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.623+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.623+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.623+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.623+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x600b500 con 0x4acb000 2026-02-11T23:31:27.623+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:6131e0b1:::obj290:head)]) -- 0x6000780 con 0x4acb000 2026-02-11T23:31:27.623+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.623+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 483 ==== MOSDPGPush(1.0 116/113 [PushOp(1:614752af:::obj331:head, version: 111'331, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:614752af:::obj331:head@111'331, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c340 con 0x4acb000 2026-02-11T23:31:27.623+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:614752af:::obj331:head, version: 111'331, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:614752af:::obj331:head@111'331, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.623+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:614752af:::obj331:head, version: 111'331, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:614752af:::obj331:head@111'331, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.623+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.623+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 484 ==== pg_backfill(progress 1.0 e 116/116 lb 1:6131e0b1:::obj290:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98000 con 0x4acb000 2026-02-11T23:31:27.623+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:6131e0b1:::obj290:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.623+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:614752af:::obj331:head, version: 111'331, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:614752af:::obj331:head@111'331, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.623+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:6131e0b1:::obj290:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.623+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.623+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:614752af:::obj331:head, version: 111'331, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:614752af:::obj331:head@111'331, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.623+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:614752af:::obj331:head, version: 111'331, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:614752af:::obj331:head@111'331, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:614752af:::obj331:head, version: 111'331, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:614752af:::obj331:head@111'331, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:5e27c68c:::obj582:head local-lis/les=113/114 n=223 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.623+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.623+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.623+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5e27c68c:::obj582:head local-lis/les=113/114 n=223 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:614752af:::obj331:head, version: 111'331, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:614752af:::obj331:head@111'331, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.623+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:6131e0b1:::obj290:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.623+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5e27c68c:::obj582:head local-lis/les=113/114 n=223 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:614752af:::obj331:head@111'331, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.623+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5e27c68c:::obj582:head local-lis/les=113/114 n=223 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:614752af:::obj331:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.623+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5e27c68c:::obj582:head local-lis/les=113/114 n=223 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:614752af:::obj331:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.623+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5e27c68c:::obj582:head local-lis/les=113/114 n=223 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:614752af:::obj331:head 2026-02-11T23:31:27.623+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5e27c68c:::obj582:head local-lis/les=113/114 n=223 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:614752af:::obj331:head v 111'331 2026-02-11T23:31:27.623+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5e27c68c:::obj582:head local-lis/les=113/114 n=223 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.623+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.623+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:614752af:::obj331:head, version: 111'331, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:614752af:::obj331:head@111'331, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:27.623+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.623+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.623+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:6131e0b1:::obj290:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.623+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:6131e0b1:::obj290:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 116/116 lb 1:6131e0b1:::obj290:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:5e27c68c:::obj582:head local-lis/les=113/114 n=223 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.623+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.623+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5e27c68c:::obj582:head local-lis/les=113/114 n=223 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:6131e0b1:::obj290:head) v3 2026-02-11T23:31:27.623+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:5e27c68c:::obj582:head local-lis/les=113/114 n=223 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:6131e0b1:::obj290:head) 2026-02-11T23:31:27.623+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.623+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:6131e0b1:::obj290:head) v3 finish latency 0.000009 2026-02-11T23:31:27.623+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6131e0b1:::obj290:head local-lis/les=113/114 n=224 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.623+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.623+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.623+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6131e0b1:::obj290:head local-lis/les=113/114 n=224 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.623+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.623+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.623+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.623+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x600b680 con 0x4acb000 2026-02-11T23:31:27.623+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:614752af:::obj331:head)]) -- 0x6000a80 con 0x4acb000 2026-02-11T23:31:27.623+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.623+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 485 ==== MOSDPGPush(1.0 116/113 [PushOp(1:61741462:::obj327:head, version: 111'327, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:61741462:::obj327:head@111'327, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c820 con 0x4acb000 2026-02-11T23:31:27.623+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:61741462:::obj327:head, version: 111'327, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:61741462:::obj327:head@111'327, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.623+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:61741462:::obj327:head, version: 111'327, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:61741462:::obj327:head@111'327, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.623+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.623+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 486 ==== pg_backfill(progress 1.0 e 116/116 lb 1:614752af:::obj331:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98580 con 0x4acb000 2026-02-11T23:31:27.623+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:614752af:::obj331:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.623+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:61741462:::obj327:head, version: 111'327, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:61741462:::obj327:head@111'327, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.623+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:614752af:::obj331:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.623+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.623+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:61741462:::obj327:head, version: 111'327, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:61741462:::obj327:head@111'327, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.623+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:61741462:::obj327:head, version: 111'327, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:61741462:::obj327:head@111'327, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 116/113 [PushOp(1:61741462:::obj327:head, version: 111'327, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:61741462:::obj327:head@111'327, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:6131e0b1:::obj290:head local-lis/les=113/114 n=224 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.623+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.623+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.623+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6131e0b1:::obj290:head local-lis/les=113/114 n=224 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:61741462:::obj327:head, version: 111'327, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:61741462:::obj327:head@111'327, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.623+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:614752af:::obj331:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.623+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6131e0b1:::obj290:head local-lis/les=113/114 n=224 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:61741462:::obj327:head@111'327, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.623+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6131e0b1:::obj290:head local-lis/les=113/114 n=224 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:61741462:::obj327:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.623+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6131e0b1:::obj290:head local-lis/les=113/114 n=224 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:61741462:::obj327:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.623+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6131e0b1:::obj290:head local-lis/les=113/114 n=224 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:61741462:::obj327:head 2026-02-11T23:31:27.623+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6131e0b1:::obj290:head local-lis/les=113/114 n=224 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:61741462:::obj327:head v 111'327 2026-02-11T23:31:27.623+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6131e0b1:::obj290:head local-lis/les=113/114 n=224 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.623+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.623+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:61741462:::obj327:head, version: 111'327, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:61741462:::obj327:head@111'327, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.623+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.623+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.623+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:614752af:::obj331:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.623+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:614752af:::obj331:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 1.0 e 116/116 lb 1:614752af:::obj331:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:6131e0b1:::obj290:head local-lis/les=113/114 n=224 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.623+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.623+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6131e0b1:::obj290:head local-lis/les=113/114 n=224 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:614752af:::obj331:head) v3 2026-02-11T23:31:27.623+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6131e0b1:::obj290:head local-lis/les=113/114 n=224 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:614752af:::obj331:head) 2026-02-11T23:31:27.623+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.623+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:614752af:::obj331:head) v3 finish latency 0.000009 2026-02-11T23:31:27.623+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:614752af:::obj331:head local-lis/les=113/114 n=225 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.623+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.624+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.624+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:614752af:::obj331:head local-lis/les=113/114 n=225 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.624+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.624+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.624+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.624+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x600b800 con 0x4acb000 2026-02-11T23:31:27.624+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:61741462:::obj327:head)]) -- 0x6000d80 con 0x4acb000 2026-02-11T23:31:27.624+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.624+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 487 ==== MOSDPGPush(1.0 116/113 [PushOp(1:61dc80df:::obj452:head, version: 111'452, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:61dc80df:::obj452:head@111'452, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559dd40 con 0x4acb000 2026-02-11T23:31:27.624+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:61dc80df:::obj452:head, version: 111'452, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:61dc80df:::obj452:head@111'452, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.624+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:61dc80df:::obj452:head, version: 111'452, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:61dc80df:::obj452:head@111'452, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.624+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.624+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 488 ==== pg_backfill(progress 1.0 e 116/116 lb 1:61741462:::obj327:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98b00 con 0x4acb000 2026-02-11T23:31:27.624+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:61741462:::obj327:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.624+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:61dc80df:::obj452:head, version: 111'452, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:61dc80df:::obj452:head@111'452, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.624+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:61741462:::obj327:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.624+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.624+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:61dc80df:::obj452:head, version: 111'452, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:61dc80df:::obj452:head@111'452, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.624+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:61dc80df:::obj452:head, version: 111'452, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:61dc80df:::obj452:head@111'452, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 116/113 [PushOp(1:61dc80df:::obj452:head, version: 111'452, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:61dc80df:::obj452:head@111'452, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:614752af:::obj331:head local-lis/les=113/114 n=225 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.624+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.624+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:614752af:::obj331:head local-lis/les=113/114 n=225 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:61dc80df:::obj452:head, version: 111'452, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:61dc80df:::obj452:head@111'452, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.624+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.624+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:61741462:::obj327:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.624+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:614752af:::obj331:head local-lis/les=113/114 n=225 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:61dc80df:::obj452:head@111'452, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.624+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:614752af:::obj331:head local-lis/les=113/114 n=225 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:61dc80df:::obj452:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.624+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:614752af:::obj331:head local-lis/les=113/114 n=225 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:61dc80df:::obj452:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.624+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:614752af:::obj331:head local-lis/les=113/114 n=225 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:61dc80df:::obj452:head 2026-02-11T23:31:27.624+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:614752af:::obj331:head local-lis/les=113/114 n=225 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:61dc80df:::obj452:head v 111'452 2026-02-11T23:31:27.624+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:614752af:::obj331:head local-lis/les=113/114 n=225 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.624+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.624+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:61dc80df:::obj452:head, version: 111'452, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:61dc80df:::obj452:head@111'452, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.624+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:614752af:::obj331:head local-lis/les=113/114 n=225 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.624+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.624+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.624+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:61741462:::obj327:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.624+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:61741462:::obj327:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 1.0 e 116/116 lb 1:61741462:::obj327:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:614752af:::obj331:head local-lis/les=113/114 n=225 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.624+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.624+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:614752af:::obj331:head local-lis/les=113/114 n=225 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:61741462:::obj327:head) v3 2026-02-11T23:31:27.624+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:614752af:::obj331:head local-lis/les=113/114 n=225 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:61741462:::obj327:head) 2026-02-11T23:31:27.624+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.624+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:61741462:::obj327:head) v3 finish latency 0.000010 2026-02-11T23:31:27.624+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.624+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.624+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:61741462:::obj327:head local-lis/les=113/114 n=226 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.624+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.624+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.624+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.624+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x600bc80 con 0x4acb000 2026-02-11T23:31:27.624+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:61dc80df:::obj452:head)]) -- 0x600b980 con 0x4acb000 2026-02-11T23:31:27.624+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.625+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 489 ==== MOSDPGPush(1.0 116/113 [PushOp(1:622f1330:::obj230:head, version: 111'230, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:622f1330:::obj230:head@111'230, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d8680 con 0x4acb000 2026-02-11T23:31:27.625+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:622f1330:::obj230:head, version: 111'230, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:622f1330:::obj230:head@111'230, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.625+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:622f1330:::obj230:head, version: 111'230, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:622f1330:::obj230:head@111'230, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.625+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.625+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 490 ==== pg_backfill(progress 1.0 e 116/116 lb 1:61dc80df:::obj452:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99080 con 0x4acb000 2026-02-11T23:31:27.625+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:622f1330:::obj230:head, version: 111'230, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:622f1330:::obj230:head@111'230, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.625+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:61dc80df:::obj452:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.625+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:61dc80df:::obj452:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.625+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.625+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:622f1330:::obj230:head, version: 111'230, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:622f1330:::obj230:head@111'230, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.625+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.625+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:622f1330:::obj230:head, version: 111'230, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:622f1330:::obj230:head@111'230, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 116/113 [PushOp(1:622f1330:::obj230:head, version: 111'230, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:622f1330:::obj230:head@111'230, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:61741462:::obj327:head local-lis/les=113/114 n=226 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.625+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.625+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:61741462:::obj327:head local-lis/les=113/114 n=226 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:622f1330:::obj230:head, version: 111'230, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:622f1330:::obj230:head@111'230, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.625+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:61741462:::obj327:head local-lis/les=113/114 n=226 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:622f1330:::obj230:head@111'230, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.625+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.625+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:61dc80df:::obj452:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.625+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:61741462:::obj327:head local-lis/les=113/114 n=226 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:622f1330:::obj230:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.625+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:61741462:::obj327:head local-lis/les=113/114 n=226 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:622f1330:::obj230:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.625+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:61741462:::obj327:head local-lis/les=113/114 n=226 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:622f1330:::obj230:head 2026-02-11T23:31:27.625+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:61741462:::obj327:head local-lis/les=113/114 n=226 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:622f1330:::obj230:head v 111'230 2026-02-11T23:31:27.625+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:61741462:::obj327:head local-lis/les=113/114 n=226 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.625+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.625+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:622f1330:::obj230:head, version: 111'230, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:622f1330:::obj230:head@111'230, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.625+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.625+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.625+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:61dc80df:::obj452:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.625+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:61dc80df:::obj452:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 1.0 e 116/116 lb 1:61dc80df:::obj452:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:61741462:::obj327:head local-lis/les=113/114 n=226 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.625+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.625+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:61741462:::obj327:head local-lis/les=113/114 n=226 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:61dc80df:::obj452:head) v3 2026-02-11T23:31:27.625+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:61741462:::obj327:head local-lis/les=113/114 n=226 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:61dc80df:::obj452:head) 2026-02-11T23:31:27.625+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.625+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:61dc80df:::obj452:head) v3 finish latency 0.000010 2026-02-11T23:31:27.625+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:61dc80df:::obj452:head local-lis/les=113/114 n=227 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.625+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.625+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.625+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:61dc80df:::obj452:head local-lis/les=113/114 n=227 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.625+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.625+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.625+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.625+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x600be00 con 0x4acb000 2026-02-11T23:31:27.625+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:622f1330:::obj230:head)]) -- 0x6001080 con 0x4acb000 2026-02-11T23:31:27.625+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.625+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 491 ==== MOSDPGPush(1.0 116/113 [PushOp(1:624e19c5:::obj464:head, version: 111'464, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:624e19c5:::obj464:head@111'464, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559d860 con 0x4acb000 2026-02-11T23:31:27.625+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:624e19c5:::obj464:head, version: 111'464, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:624e19c5:::obj464:head@111'464, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.625+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:624e19c5:::obj464:head, version: 111'464, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:624e19c5:::obj464:head@111'464, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.625+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.625+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 492 ==== pg_backfill(progress 1.0 e 116/116 lb 1:622f1330:::obj230:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99600 con 0x4acb000 2026-02-11T23:31:27.625+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:622f1330:::obj230:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.625+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:624e19c5:::obj464:head, version: 111'464, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:624e19c5:::obj464:head@111'464, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.625+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:622f1330:::obj230:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.625+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.625+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:624e19c5:::obj464:head, version: 111'464, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:624e19c5:::obj464:head@111'464, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.625+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:624e19c5:::obj464:head, version: 111'464, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:624e19c5:::obj464:head@111'464, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 116/113 [PushOp(1:624e19c5:::obj464:head, version: 111'464, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:624e19c5:::obj464:head@111'464, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:61dc80df:::obj452:head local-lis/les=113/114 n=227 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.625+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.625+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.625+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:61dc80df:::obj452:head local-lis/les=113/114 n=227 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:624e19c5:::obj464:head, version: 111'464, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:624e19c5:::obj464:head@111'464, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.625+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:622f1330:::obj230:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.625+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:61dc80df:::obj452:head local-lis/les=113/114 n=227 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:624e19c5:::obj464:head@111'464, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.625+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:61dc80df:::obj452:head local-lis/les=113/114 n=227 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:624e19c5:::obj464:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.625+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:61dc80df:::obj452:head local-lis/les=113/114 n=227 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:624e19c5:::obj464:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.625+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:61dc80df:::obj452:head local-lis/les=113/114 n=227 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:624e19c5:::obj464:head 2026-02-11T23:31:27.625+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:61dc80df:::obj452:head local-lis/les=113/114 n=227 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:624e19c5:::obj464:head v 111'464 2026-02-11T23:31:27.625+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:61dc80df:::obj452:head local-lis/les=113/114 n=227 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.625+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.625+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:624e19c5:::obj464:head, version: 111'464, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:624e19c5:::obj464:head@111'464, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.625+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.625+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.625+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:622f1330:::obj230:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.625+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:622f1330:::obj230:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 1.0 e 116/116 lb 1:622f1330:::obj230:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:61dc80df:::obj452:head local-lis/les=113/114 n=227 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.625+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.625+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:61dc80df:::obj452:head local-lis/les=113/114 n=227 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:622f1330:::obj230:head) v3 2026-02-11T23:31:27.625+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:61dc80df:::obj452:head local-lis/les=113/114 n=227 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:622f1330:::obj230:head) 2026-02-11T23:31:27.625+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.625+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:622f1330:::obj230:head) v3 finish latency 0.000009 2026-02-11T23:31:27.625+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:622f1330:::obj230:head local-lis/les=113/114 n=228 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.625+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.625+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.625+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:622f1330:::obj230:head local-lis/les=113/114 n=228 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.625+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.625+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.625+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.625+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fcfb00 con 0x4acb000 2026-02-11T23:31:27.625+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:624e19c5:::obj464:head)]) -- 0x6001380 con 0x4acb000 2026-02-11T23:31:27.625+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.626+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 493 ==== MOSDPGPush(1.0 116/113 [PushOp(1:62eb9ce0:::obj205:head, version: 111'205, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:62eb9ce0:::obj205:head@111'205, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d81a0 con 0x4acb000 2026-02-11T23:31:27.626+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:62eb9ce0:::obj205:head, version: 111'205, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:62eb9ce0:::obj205:head@111'205, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.626+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:62eb9ce0:::obj205:head, version: 111'205, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:62eb9ce0:::obj205:head@111'205, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.626+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.626+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 494 ==== pg_backfill(progress 1.0 e 116/116 lb 1:624e19c5:::obj464:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99b80 con 0x4acb000 2026-02-11T23:31:27.626+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:624e19c5:::obj464:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.626+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:62eb9ce0:::obj205:head, version: 111'205, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:62eb9ce0:::obj205:head@111'205, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.626+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.626+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:624e19c5:::obj464:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.626+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:62eb9ce0:::obj205:head, version: 111'205, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:62eb9ce0:::obj205:head@111'205, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.626+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.626+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:62eb9ce0:::obj205:head, version: 111'205, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:62eb9ce0:::obj205:head@111'205, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:62eb9ce0:::obj205:head, version: 111'205, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:62eb9ce0:::obj205:head@111'205, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:622f1330:::obj230:head local-lis/les=113/114 n=228 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.626+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.626+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:622f1330:::obj230:head local-lis/les=113/114 n=228 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:62eb9ce0:::obj205:head, version: 111'205, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:62eb9ce0:::obj205:head@111'205, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.626+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:622f1330:::obj230:head local-lis/les=113/114 n=228 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:62eb9ce0:::obj205:head@111'205, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.626+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.626+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:624e19c5:::obj464:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.626+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:622f1330:::obj230:head local-lis/les=113/114 n=228 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:62eb9ce0:::obj205:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.626+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:622f1330:::obj230:head local-lis/les=113/114 n=228 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:62eb9ce0:::obj205:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.626+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:622f1330:::obj230:head local-lis/les=113/114 n=228 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:62eb9ce0:::obj205:head 2026-02-11T23:31:27.626+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:622f1330:::obj230:head local-lis/les=113/114 n=228 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:62eb9ce0:::obj205:head v 111'205 2026-02-11T23:31:27.626+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:622f1330:::obj230:head local-lis/les=113/114 n=228 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.626+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.626+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:62eb9ce0:::obj205:head, version: 111'205, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:62eb9ce0:::obj205:head@111'205, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:27.626+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:622f1330:::obj230:head local-lis/les=113/114 n=228 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.626+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.626+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.626+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:624e19c5:::obj464:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.626+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:624e19c5:::obj464:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 1.0 e 116/116 lb 1:624e19c5:::obj464:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:622f1330:::obj230:head local-lis/les=113/114 n=228 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.626+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.626+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:622f1330:::obj230:head local-lis/les=113/114 n=228 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:624e19c5:::obj464:head) v3 2026-02-11T23:31:27.626+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:622f1330:::obj230:head local-lis/les=113/114 n=228 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:624e19c5:::obj464:head) 2026-02-11T23:31:27.626+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.626+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:624e19c5:::obj464:head) v3 finish latency 0.000010 2026-02-11T23:31:27.626+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.626+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.626+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:624e19c5:::obj464:head local-lis/les=113/114 n=229 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.626+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.626+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.626+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.626+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x600a900 con 0x4acb000 2026-02-11T23:31:27.626+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:62eb9ce0:::obj205:head)]) -- 0x600aa80 con 0x4acb000 2026-02-11T23:31:27.626+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.627+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 495 ==== MOSDPGPush(1.0 116/113 [PushOp(1:63191178:::obj308:head, version: 111'308, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:63191178:::obj308:head@111'308, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5647040 con 0x4acb000 2026-02-11T23:31:27.627+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:63191178:::obj308:head, version: 111'308, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:63191178:::obj308:head@111'308, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 116 2026-02-11T23:31:27.627+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:63191178:::obj308:head, version: 111'308, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:63191178:::obj308:head@111'308, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.627+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.627+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:63191178:::obj308:head, version: 111'308, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:63191178:::obj308:head@111'308, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.627+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 496 ==== pg_backfill(progress 1.0 e 116/116 lb 1:62eb9ce0:::obj205:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9a100 con 0x4acb000 2026-02-11T23:31:27.627+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.627+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:62eb9ce0:::obj205:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 116 2026-02-11T23:31:27.627+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:63191178:::obj308:head, version: 111'308, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:63191178:::obj308:head@111'308, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.627+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:62eb9ce0:::obj205:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.627+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.627+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:63191178:::obj308:head, version: 111'308, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:63191178:::obj308:head@111'308, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 116/113 [PushOp(1:63191178:::obj308:head, version: 111'308, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:63191178:::obj308:head@111'308, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:624e19c5:::obj464:head local-lis/les=113/114 n=229 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.627+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:62eb9ce0:::obj205:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.627+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.627+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:624e19c5:::obj464:head local-lis/les=113/114 n=229 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:63191178:::obj308:head, version: 111'308, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:63191178:::obj308:head@111'308, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.627+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:624e19c5:::obj464:head local-lis/les=113/114 n=229 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:63191178:::obj308:head@111'308, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.627+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:624e19c5:::obj464:head local-lis/les=113/114 n=229 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:63191178:::obj308:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.627+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:624e19c5:::obj464:head local-lis/les=113/114 n=229 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:63191178:::obj308:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.627+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:624e19c5:::obj464:head local-lis/les=113/114 n=229 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:63191178:::obj308:head 2026-02-11T23:31:27.627+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:624e19c5:::obj464:head local-lis/les=113/114 n=229 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:63191178:::obj308:head v 111'308 2026-02-11T23:31:27.627+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:624e19c5:::obj464:head local-lis/les=113/114 n=229 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.627+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.627+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:63191178:::obj308:head, version: 111'308, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:63191178:::obj308:head@111'308, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.627+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.627+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.627+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:62eb9ce0:::obj205:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.627+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:62eb9ce0:::obj205:head) v3 prio 15 cost 0 latency 0.000059 pg_backfill(progress 1.0 e 116/116 lb 1:62eb9ce0:::obj205:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:624e19c5:::obj464:head local-lis/les=113/114 n=229 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.627+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.627+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:624e19c5:::obj464:head local-lis/les=113/114 n=229 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:62eb9ce0:::obj205:head) v3 2026-02-11T23:31:27.627+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:624e19c5:::obj464:head local-lis/les=113/114 n=229 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:62eb9ce0:::obj205:head) 2026-02-11T23:31:27.627+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.627+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:62eb9ce0:::obj205:head) v3 finish latency 0.000010 2026-02-11T23:31:27.627+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:62eb9ce0:::obj205:head local-lis/les=113/114 n=230 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.627+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.627+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.627+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:62eb9ce0:::obj205:head local-lis/les=113/114 n=230 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.627+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.627+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.627+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.627+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fcf800 con 0x4acb000 2026-02-11T23:31:27.627+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:63191178:::obj308:head)]) -- 0x6001680 con 0x4acb000 2026-02-11T23:31:27.627+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.627+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 497 ==== MOSDPGPush(1.0 116/113 [PushOp(1:640d3668:::obj286:head, version: 111'286, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:640d3668:::obj286:head@111'286, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d8b60 con 0x4acb000 2026-02-11T23:31:27.627+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:640d3668:::obj286:head, version: 111'286, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:640d3668:::obj286:head@111'286, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000015 epoch 116 2026-02-11T23:31:27.627+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:640d3668:::obj286:head, version: 111'286, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:640d3668:::obj286:head@111'286, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.627+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.627+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 498 ==== pg_backfill(progress 1.0 e 116/116 lb 1:63191178:::obj308:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9a680 con 0x4acb000 2026-02-11T23:31:27.627+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:640d3668:::obj286:head, version: 111'286, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:640d3668:::obj286:head@111'286, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.627+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:63191178:::obj308:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.627+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:63191178:::obj308:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.627+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.627+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:640d3668:::obj286:head, version: 111'286, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:640d3668:::obj286:head@111'286, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.627+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.627+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:640d3668:::obj286:head, version: 111'286, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:640d3668:::obj286:head@111'286, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(1.0 116/113 [PushOp(1:640d3668:::obj286:head, version: 111'286, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:640d3668:::obj286:head@111'286, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:62eb9ce0:::obj205:head local-lis/les=113/114 n=230 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.627+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.627+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:62eb9ce0:::obj205:head local-lis/les=113/114 n=230 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:640d3668:::obj286:head, version: 111'286, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:640d3668:::obj286:head@111'286, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.627+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:62eb9ce0:::obj205:head local-lis/les=113/114 n=230 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:640d3668:::obj286:head@111'286, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.627+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.627+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:63191178:::obj308:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.627+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:62eb9ce0:::obj205:head local-lis/les=113/114 n=230 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:640d3668:::obj286:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.627+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:62eb9ce0:::obj205:head local-lis/les=113/114 n=230 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:640d3668:::obj286:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.627+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:62eb9ce0:::obj205:head local-lis/les=113/114 n=230 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:640d3668:::obj286:head 2026-02-11T23:31:27.627+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:62eb9ce0:::obj205:head local-lis/les=113/114 n=230 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:640d3668:::obj286:head v 111'286 2026-02-11T23:31:27.627+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:62eb9ce0:::obj205:head local-lis/les=113/114 n=230 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.627+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.627+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:640d3668:::obj286:head, version: 111'286, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:640d3668:::obj286:head@111'286, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000043 2026-02-11T23:31:27.627+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.627+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.627+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:63191178:::obj308:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.627+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:63191178:::obj308:head) v3 prio 15 cost 0 latency 0.000059 pg_backfill(progress 1.0 e 116/116 lb 1:63191178:::obj308:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:62eb9ce0:::obj205:head local-lis/les=113/114 n=230 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.627+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.627+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:62eb9ce0:::obj205:head local-lis/les=113/114 n=230 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:63191178:::obj308:head) v3 2026-02-11T23:31:27.627+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:62eb9ce0:::obj205:head local-lis/les=113/114 n=230 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:63191178:::obj308:head) 2026-02-11T23:31:27.627+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.627+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:63191178:::obj308:head) v3 finish latency 0.000010 2026-02-11T23:31:27.627+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:63191178:::obj308:head local-lis/les=113/114 n=231 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.627+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.627+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.627+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:63191178:::obj308:head local-lis/les=113/114 n=231 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.627+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.627+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.627+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.627+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x600a480 con 0x4acb000 2026-02-11T23:31:27.627+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:640d3668:::obj286:head)]) -- 0x6001980 con 0x4acb000 2026-02-11T23:31:27.627+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.628+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 499 ==== MOSDPGPush(1.0 116/113 [PushOp(1:649aec78:::obj275:head, version: 111'275, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:649aec78:::obj275:head@111'275, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d9040 con 0x4acb000 2026-02-11T23:31:27.628+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:649aec78:::obj275:head, version: 111'275, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:649aec78:::obj275:head@111'275, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 116 2026-02-11T23:31:27.628+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:649aec78:::obj275:head, version: 111'275, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:649aec78:::obj275:head@111'275, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.628+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.628+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 500 ==== pg_backfill(progress 1.0 e 116/116 lb 1:640d3668:::obj286:head) ==== 1062+0+0 (unknown 0 0 0) 0x4a2cb00 con 0x4acb000 2026-02-11T23:31:27.628+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:640d3668:::obj286:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.628+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:649aec78:::obj275:head, version: 111'275, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:649aec78:::obj275:head@111'275, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.628+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:640d3668:::obj286:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.628+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.628+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:640d3668:::obj286:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.628+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.628+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:649aec78:::obj275:head, version: 111'275, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:649aec78:::obj275:head@111'275, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.628+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:649aec78:::obj275:head, version: 111'275, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:649aec78:::obj275:head@111'275, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000045 MOSDPGPush(1.0 116/113 [PushOp(1:649aec78:::obj275:head, version: 111'275, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:649aec78:::obj275:head@111'275, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:63191178:::obj308:head local-lis/les=113/114 n=231 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.628+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.628+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:63191178:::obj308:head local-lis/les=113/114 n=231 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:649aec78:::obj275:head, version: 111'275, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:649aec78:::obj275:head@111'275, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.628+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:63191178:::obj308:head local-lis/les=113/114 n=231 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:649aec78:::obj275:head@111'275, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.628+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:63191178:::obj308:head local-lis/les=113/114 n=231 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:649aec78:::obj275:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.628+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:63191178:::obj308:head local-lis/les=113/114 n=231 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:649aec78:::obj275:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.628+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:63191178:::obj308:head local-lis/les=113/114 n=231 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:649aec78:::obj275:head 2026-02-11T23:31:27.628+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:63191178:::obj308:head local-lis/les=113/114 n=231 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:649aec78:::obj275:head v 111'275 2026-02-11T23:31:27.628+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:63191178:::obj308:head local-lis/les=113/114 n=231 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.628+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.628+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:649aec78:::obj275:head, version: 111'275, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:649aec78:::obj275:head@111'275, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.628+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.628+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.628+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:640d3668:::obj286:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.628+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:640d3668:::obj286:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 1.0 e 116/116 lb 1:640d3668:::obj286:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:63191178:::obj308:head local-lis/les=113/114 n=231 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.628+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.628+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:63191178:::obj308:head local-lis/les=113/114 n=231 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:640d3668:::obj286:head) v3 2026-02-11T23:31:27.628+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:63191178:::obj308:head local-lis/les=113/114 n=231 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:640d3668:::obj286:head) 2026-02-11T23:31:27.628+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.628+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:640d3668:::obj286:head) v3 finish latency 0.000010 2026-02-11T23:31:27.628+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:640d3668:::obj286:head local-lis/les=113/114 n=232 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.628+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.628+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.628+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:640d3668:::obj286:head local-lis/les=113/114 n=232 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.628+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.628+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.628+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.628+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fcf680 con 0x4acb000 2026-02-11T23:31:27.628+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:649aec78:::obj275:head)]) -- 0x6001c80 con 0x4acb000 2026-02-11T23:31:27.628+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.628+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 501 ==== MOSDPGPush(1.0 116/113 [PushOp(1:64b337fe:::obj71:head, version: 111'71, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64b337fe:::obj71:head@111'71, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x55d9520 con 0x4acb000 2026-02-11T23:31:27.628+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:64b337fe:::obj71:head, version: 111'71, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64b337fe:::obj71:head@111'71, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 116 2026-02-11T23:31:27.628+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:64b337fe:::obj71:head, version: 111'71, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64b337fe:::obj71:head@111'71, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.628+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.628+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 502 ==== pg_backfill(progress 1.0 e 116/116 lb 1:649aec78:::obj275:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2d080 con 0x4acb000 2026-02-11T23:31:27.628+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:649aec78:::obj275:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.628+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:64b337fe:::obj71:head, version: 111'71, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64b337fe:::obj71:head@111'71, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.628+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:649aec78:::obj275:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.628+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.628+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:649aec78:::obj275:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.628+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.628+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:64b337fe:::obj71:head, version: 111'71, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64b337fe:::obj71:head@111'71, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.628+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:64b337fe:::obj71:head, version: 111'71, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64b337fe:::obj71:head@111'71, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000042 MOSDPGPush(1.0 116/113 [PushOp(1:64b337fe:::obj71:head, version: 111'71, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64b337fe:::obj71:head@111'71, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:640d3668:::obj286:head local-lis/les=113/114 n=232 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.628+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.628+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:640d3668:::obj286:head local-lis/les=113/114 n=232 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:64b337fe:::obj71:head, version: 111'71, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64b337fe:::obj71:head@111'71, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.628+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:640d3668:::obj286:head local-lis/les=113/114 n=232 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:64b337fe:::obj71:head@111'71, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.628+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:640d3668:::obj286:head local-lis/les=113/114 n=232 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:64b337fe:::obj71:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.628+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:640d3668:::obj286:head local-lis/les=113/114 n=232 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:64b337fe:::obj71:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.628+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:640d3668:::obj286:head local-lis/les=113/114 n=232 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:64b337fe:::obj71:head 2026-02-11T23:31:27.628+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:640d3668:::obj286:head local-lis/les=113/114 n=232 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:64b337fe:::obj71:head v 111'71 2026-02-11T23:31:27.628+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:640d3668:::obj286:head local-lis/les=113/114 n=232 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.628+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.628+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:64b337fe:::obj71:head, version: 111'71, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64b337fe:::obj71:head@111'71, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000044 2026-02-11T23:31:27.628+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.628+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.628+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:649aec78:::obj275:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.628+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:649aec78:::obj275:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 1.0 e 116/116 lb 1:649aec78:::obj275:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:640d3668:::obj286:head local-lis/les=113/114 n=232 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.628+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.628+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:640d3668:::obj286:head local-lis/les=113/114 n=232 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:649aec78:::obj275:head) v3 2026-02-11T23:31:27.628+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:640d3668:::obj286:head local-lis/les=113/114 n=232 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:649aec78:::obj275:head) 2026-02-11T23:31:27.628+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.628+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:649aec78:::obj275:head) v3 finish latency 0.000009 2026-02-11T23:31:27.628+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:649aec78:::obj275:head local-lis/les=113/114 n=233 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.628+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.628+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.628+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:649aec78:::obj275:head local-lis/les=113/114 n=233 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.628+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.628+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.628+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.628+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x600a300 con 0x4acb000 2026-02-11T23:31:27.628+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:64b337fe:::obj71:head)]) -- 0x6004000 con 0x4acb000 2026-02-11T23:31:27.628+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.629+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 503 ==== MOSDPGPush(1.0 116/113 [PushOp(1:64bc0383:::obj214:head, version: 111'214, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64bc0383:::obj214:head@111'214, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56cb1e0 con 0x4acb000 2026-02-11T23:31:27.629+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:64bc0383:::obj214:head, version: 111'214, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64bc0383:::obj214:head@111'214, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 116 2026-02-11T23:31:27.629+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:64bc0383:::obj214:head, version: 111'214, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64bc0383:::obj214:head@111'214, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.629+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.629+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 504 ==== pg_backfill(progress 1.0 e 116/116 lb 1:64b337fe:::obj71:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d2e680 con 0x4acb000 2026-02-11T23:31:27.629+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:64b337fe:::obj71:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.629+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:64bc0383:::obj214:head, version: 111'214, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64bc0383:::obj214:head@111'214, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.629+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:64b337fe:::obj71:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.629+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.629+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:64bc0383:::obj214:head, version: 111'214, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64bc0383:::obj214:head@111'214, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.629+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:64bc0383:::obj214:head, version: 111'214, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64bc0383:::obj214:head@111'214, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(1.0 116/113 [PushOp(1:64bc0383:::obj214:head, version: 111'214, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64bc0383:::obj214:head@111'214, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:649aec78:::obj275:head local-lis/les=113/114 n=233 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.629+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.629+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.629+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:649aec78:::obj275:head local-lis/les=113/114 n=233 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:64bc0383:::obj214:head, version: 111'214, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64bc0383:::obj214:head@111'214, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.629+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:64b337fe:::obj71:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.629+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:649aec78:::obj275:head local-lis/les=113/114 n=233 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:64bc0383:::obj214:head@111'214, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.629+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:649aec78:::obj275:head local-lis/les=113/114 n=233 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:64bc0383:::obj214:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.629+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:649aec78:::obj275:head local-lis/les=113/114 n=233 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:64bc0383:::obj214:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.629+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:649aec78:::obj275:head local-lis/les=113/114 n=233 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:64bc0383:::obj214:head 2026-02-11T23:31:27.629+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:649aec78:::obj275:head local-lis/les=113/114 n=233 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:64bc0383:::obj214:head v 111'214 2026-02-11T23:31:27.629+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:649aec78:::obj275:head local-lis/les=113/114 n=233 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.629+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.629+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:64bc0383:::obj214:head, version: 111'214, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64bc0383:::obj214:head@111'214, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.629+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.629+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.629+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:64b337fe:::obj71:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.629+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:64b337fe:::obj71:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 1.0 e 116/116 lb 1:64b337fe:::obj71:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:649aec78:::obj275:head local-lis/les=113/114 n=233 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.629+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.629+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:649aec78:::obj275:head local-lis/les=113/114 n=233 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:64b337fe:::obj71:head) v3 2026-02-11T23:31:27.629+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:649aec78:::obj275:head local-lis/les=113/114 n=233 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:64b337fe:::obj71:head) 2026-02-11T23:31:27.629+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.629+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:64b337fe:::obj71:head) v3 finish latency 0.000009 2026-02-11T23:31:27.629+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:64b337fe:::obj71:head local-lis/les=113/114 n=234 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.629+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.629+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.629+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:64b337fe:::obj71:head local-lis/les=113/114 n=234 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.629+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.629+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.629+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.629+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fcf380 con 0x4acb000 2026-02-11T23:31:27.629+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:64bc0383:::obj214:head)]) -- 0x6004180 con 0x4acb000 2026-02-11T23:31:27.629+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.629+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 505 ==== MOSDPGPush(1.0 116/113 [PushOp(1:64c1dddc:::obj83:head, version: 111'83, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64c1dddc:::obj83:head@111'83, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x55d89c0 con 0x4acb000 2026-02-11T23:31:27.629+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:64c1dddc:::obj83:head, version: 111'83, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64c1dddc:::obj83:head@111'83, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000021 epoch 116 2026-02-11T23:31:27.629+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:64c1dddc:::obj83:head, version: 111'83, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64c1dddc:::obj83:head@111'83, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.629+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.629+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 506 ==== pg_backfill(progress 1.0 e 116/116 lb 1:64bc0383:::obj214:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e100 con 0x4acb000 2026-02-11T23:31:27.629+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:64c1dddc:::obj83:head, version: 111'83, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64c1dddc:::obj83:head@111'83, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.629+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:64bc0383:::obj214:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.629+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.629+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:64bc0383:::obj214:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.629+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:64c1dddc:::obj83:head, version: 111'83, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64c1dddc:::obj83:head@111'83, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.629+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.629+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:64bc0383:::obj214:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.629+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:64c1dddc:::obj83:head, version: 111'83, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64c1dddc:::obj83:head@111'83, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000042 MOSDPGPush(1.0 116/113 [PushOp(1:64c1dddc:::obj83:head, version: 111'83, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64c1dddc:::obj83:head@111'83, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:64b337fe:::obj71:head local-lis/les=113/114 n=234 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.629+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.629+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:64b337fe:::obj71:head local-lis/les=113/114 n=234 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:64c1dddc:::obj83:head, version: 111'83, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64c1dddc:::obj83:head@111'83, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.629+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:64b337fe:::obj71:head local-lis/les=113/114 n=234 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:64c1dddc:::obj83:head@111'83, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.629+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:64b337fe:::obj71:head local-lis/les=113/114 n=234 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:64c1dddc:::obj83:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.629+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:64b337fe:::obj71:head local-lis/les=113/114 n=234 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:64c1dddc:::obj83:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.629+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:64b337fe:::obj71:head local-lis/les=113/114 n=234 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:64c1dddc:::obj83:head 2026-02-11T23:31:27.629+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:64b337fe:::obj71:head local-lis/les=113/114 n=234 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:64c1dddc:::obj83:head v 111'83 2026-02-11T23:31:27.629+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:64b337fe:::obj71:head local-lis/les=113/114 n=234 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.629+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.630+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:64c1dddc:::obj83:head, version: 111'83, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64c1dddc:::obj83:head@111'83, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:27.630+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.630+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.630+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:64bc0383:::obj214:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.630+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:64bc0383:::obj214:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 1.0 e 116/116 lb 1:64bc0383:::obj214:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:64b337fe:::obj71:head local-lis/les=113/114 n=234 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.630+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.630+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:64b337fe:::obj71:head local-lis/les=113/114 n=234 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:64bc0383:::obj214:head) v3 2026-02-11T23:31:27.630+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:64b337fe:::obj71:head local-lis/les=113/114 n=234 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:64bc0383:::obj214:head) 2026-02-11T23:31:27.630+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.630+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:64bc0383:::obj214:head) v3 finish latency 0.000010 2026-02-11T23:31:27.630+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:64bc0383:::obj214:head local-lis/les=113/114 n=235 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.630+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.630+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.630+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:64bc0383:::obj214:head local-lis/les=113/114 n=235 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.630+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.630+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.630+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.630+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x600a180 con 0x4acb000 2026-02-11T23:31:27.630+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:64c1dddc:::obj83:head)]) -- 0x6004480 con 0x4acb000 2026-02-11T23:31:27.630+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.630+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 507 ==== MOSDPGPush(1.0 116/113 [PushOp(1:657806de:::obj544:head, version: 111'544, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:657806de:::obj544:head@111'544, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559d040 con 0x4acb000 2026-02-11T23:31:27.630+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:657806de:::obj544:head, version: 111'544, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:657806de:::obj544:head@111'544, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 116 2026-02-11T23:31:27.630+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:657806de:::obj544:head, version: 111'544, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:657806de:::obj544:head@111'544, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.630+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.630+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:657806de:::obj544:head, version: 111'544, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:657806de:::obj544:head@111'544, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.630+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.630+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:657806de:::obj544:head, version: 111'544, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:657806de:::obj544:head@111'544, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.630+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 508 ==== pg_backfill(progress 1.0 e 116/116 lb 1:64c1dddc:::obj83:head) ==== 1061+0+0 (unknown 0 0 0) 0x6014000 con 0x4acb000 2026-02-11T23:31:27.630+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.630+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:64c1dddc:::obj83:head) v3 prio 15 type 95 cost 0 latency 0.000014 epoch 116 2026-02-11T23:31:27.630+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:657806de:::obj544:head, version: 111'544, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:657806de:::obj544:head@111'544, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(1.0 116/113 [PushOp(1:657806de:::obj544:head, version: 111'544, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:657806de:::obj544:head@111'544, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:64bc0383:::obj214:head local-lis/les=113/114 n=235 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.630+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.630+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:64c1dddc:::obj83:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.630+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:64bc0383:::obj214:head local-lis/les=113/114 n=235 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:657806de:::obj544:head, version: 111'544, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:657806de:::obj544:head@111'544, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.630+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:64bc0383:::obj214:head local-lis/les=113/114 n=235 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:657806de:::obj544:head@111'544, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.630+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.630+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:64c1dddc:::obj83:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.630+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:64bc0383:::obj214:head local-lis/les=113/114 n=235 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:657806de:::obj544:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.630+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:64bc0383:::obj214:head local-lis/les=113/114 n=235 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:657806de:::obj544:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.630+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:64bc0383:::obj214:head local-lis/les=113/114 n=235 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:657806de:::obj544:head 2026-02-11T23:31:27.630+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:64bc0383:::obj214:head local-lis/les=113/114 n=235 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:657806de:::obj544:head v 111'544 2026-02-11T23:31:27.630+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:64bc0383:::obj214:head local-lis/les=113/114 n=235 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.630+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.630+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:657806de:::obj544:head, version: 111'544, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:657806de:::obj544:head@111'544, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.630+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.630+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.630+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:64c1dddc:::obj83:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.630+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:64c1dddc:::obj83:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 1.0 e 116/116 lb 1:64c1dddc:::obj83:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:64bc0383:::obj214:head local-lis/les=113/114 n=235 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.630+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.630+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:64bc0383:::obj214:head local-lis/les=113/114 n=235 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:64c1dddc:::obj83:head) v3 2026-02-11T23:31:27.630+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:64bc0383:::obj214:head local-lis/les=113/114 n=235 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:64c1dddc:::obj83:head) 2026-02-11T23:31:27.630+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.630+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:64c1dddc:::obj83:head) v3 finish latency 0.000009 2026-02-11T23:31:27.630+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:64c1dddc:::obj83:head local-lis/les=113/114 n=236 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.630+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.630+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.630+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:64c1dddc:::obj83:head local-lis/les=113/114 n=236 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.630+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.630+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.630+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.630+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fcf080 con 0x4acb000 2026-02-11T23:31:27.630+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:657806de:::obj544:head)]) -- 0x6004600 con 0x4acb000 2026-02-11T23:31:27.630+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.631+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 509 ==== MOSDPGPush(1.0 116/113 [PushOp(1:65a1355c:::obj536:head, version: 111'536, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:65a1355c:::obj536:head@111'536, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559dba0 con 0x4acb000 2026-02-11T23:31:27.631+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:65a1355c:::obj536:head, version: 111'536, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:65a1355c:::obj536:head@111'536, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000015 epoch 116 2026-02-11T23:31:27.631+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:65a1355c:::obj536:head, version: 111'536, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:65a1355c:::obj536:head@111'536, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.631+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.631+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 510 ==== pg_backfill(progress 1.0 e 116/116 lb 1:657806de:::obj544:head) ==== 1062+0+0 (unknown 0 0 0) 0x6014580 con 0x4acb000 2026-02-11T23:31:27.631+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:657806de:::obj544:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.631+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:65a1355c:::obj536:head, version: 111'536, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:65a1355c:::obj536:head@111'536, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.631+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:657806de:::obj544:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.631+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.631+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:65a1355c:::obj536:head, version: 111'536, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:65a1355c:::obj536:head@111'536, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.631+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:65a1355c:::obj536:head, version: 111'536, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:65a1355c:::obj536:head@111'536, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(1.0 116/113 [PushOp(1:65a1355c:::obj536:head, version: 111'536, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:65a1355c:::obj536:head@111'536, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:64c1dddc:::obj83:head local-lis/les=113/114 n=236 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.631+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.631+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.631+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:64c1dddc:::obj83:head local-lis/les=113/114 n=236 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:65a1355c:::obj536:head, version: 111'536, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:65a1355c:::obj536:head@111'536, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.631+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:657806de:::obj544:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.631+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:64c1dddc:::obj83:head local-lis/les=113/114 n=236 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:65a1355c:::obj536:head@111'536, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.631+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:64c1dddc:::obj83:head local-lis/les=113/114 n=236 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:65a1355c:::obj536:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.631+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:64c1dddc:::obj83:head local-lis/les=113/114 n=236 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:65a1355c:::obj536:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.631+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:64c1dddc:::obj83:head local-lis/les=113/114 n=236 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:65a1355c:::obj536:head 2026-02-11T23:31:27.631+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:64c1dddc:::obj83:head local-lis/les=113/114 n=236 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:65a1355c:::obj536:head v 111'536 2026-02-11T23:31:27.631+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:64c1dddc:::obj83:head local-lis/les=113/114 n=236 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.631+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.631+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:65a1355c:::obj536:head, version: 111'536, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:65a1355c:::obj536:head@111'536, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.631+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.631+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.631+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:657806de:::obj544:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.631+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:657806de:::obj544:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 1.0 e 116/116 lb 1:657806de:::obj544:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:64c1dddc:::obj83:head local-lis/les=113/114 n=236 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.631+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.631+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:64c1dddc:::obj83:head local-lis/les=113/114 n=236 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:657806de:::obj544:head) v3 2026-02-11T23:31:27.631+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:64c1dddc:::obj83:head local-lis/les=113/114 n=236 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:657806de:::obj544:head) 2026-02-11T23:31:27.631+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.631+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:657806de:::obj544:head) v3 finish latency 0.000009 2026-02-11T23:31:27.631+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:657806de:::obj544:head local-lis/les=113/114 n=237 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.631+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.631+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.631+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:657806de:::obj544:head local-lis/les=113/114 n=237 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.631+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.631+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.631+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.631+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x600a000 con 0x4acb000 2026-02-11T23:31:27.631+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:65a1355c:::obj536:head)]) -- 0x6004900 con 0x4acb000 2026-02-11T23:31:27.631+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.631+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 511 ==== MOSDPGPush(1.0 116/113 [PushOp(1:6623e3eb:::obj517:head, version: 111'517, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6623e3eb:::obj517:head@111'517, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c680 con 0x4acb000 2026-02-11T23:31:27.631+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:6623e3eb:::obj517:head, version: 111'517, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6623e3eb:::obj517:head@111'517, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 116 2026-02-11T23:31:27.631+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:6623e3eb:::obj517:head, version: 111'517, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6623e3eb:::obj517:head@111'517, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.631+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.631+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 512 ==== pg_backfill(progress 1.0 e 116/116 lb 1:65a1355c:::obj536:head) ==== 1062+0+0 (unknown 0 0 0) 0x6014b00 con 0x4acb000 2026-02-11T23:31:27.631+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:6623e3eb:::obj517:head, version: 111'517, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6623e3eb:::obj517:head@111'517, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.631+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:65a1355c:::obj536:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.631+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:65a1355c:::obj536:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.631+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.631+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:6623e3eb:::obj517:head, version: 111'517, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6623e3eb:::obj517:head@111'517, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.631+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.631+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:6623e3eb:::obj517:head, version: 111'517, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6623e3eb:::obj517:head@111'517, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(1.0 116/113 [PushOp(1:6623e3eb:::obj517:head, version: 111'517, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6623e3eb:::obj517:head@111'517, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:657806de:::obj544:head local-lis/les=113/114 n=237 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.631+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.631+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:657806de:::obj544:head local-lis/les=113/114 n=237 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:6623e3eb:::obj517:head, version: 111'517, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6623e3eb:::obj517:head@111'517, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.631+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:657806de:::obj544:head local-lis/les=113/114 n=237 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:6623e3eb:::obj517:head@111'517, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.631+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.631+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:65a1355c:::obj536:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.631+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:657806de:::obj544:head local-lis/les=113/114 n=237 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:6623e3eb:::obj517:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.631+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:657806de:::obj544:head local-lis/les=113/114 n=237 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:6623e3eb:::obj517:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.631+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:657806de:::obj544:head local-lis/les=113/114 n=237 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:6623e3eb:::obj517:head 2026-02-11T23:31:27.631+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:657806de:::obj544:head local-lis/les=113/114 n=237 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:6623e3eb:::obj517:head v 111'517 2026-02-11T23:31:27.631+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:657806de:::obj544:head local-lis/les=113/114 n=237 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.631+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.631+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:6623e3eb:::obj517:head, version: 111'517, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6623e3eb:::obj517:head@111'517, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.631+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.631+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.631+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:65a1355c:::obj536:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.631+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:65a1355c:::obj536:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 1.0 e 116/116 lb 1:65a1355c:::obj536:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:657806de:::obj544:head local-lis/les=113/114 n=237 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.631+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.631+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:657806de:::obj544:head local-lis/les=113/114 n=237 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:65a1355c:::obj536:head) v3 2026-02-11T23:31:27.631+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:657806de:::obj544:head local-lis/les=113/114 n=237 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:65a1355c:::obj536:head) 2026-02-11T23:31:27.631+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.631+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:65a1355c:::obj536:head) v3 finish latency 0.000009 2026-02-11T23:31:27.631+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:65a1355c:::obj536:head local-lis/les=113/114 n=238 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.631+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.631+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.631+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:65a1355c:::obj536:head local-lis/les=113/114 n=238 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.631+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.631+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.631+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.631+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fced80 con 0x4acb000 2026-02-11T23:31:27.631+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:6623e3eb:::obj517:head)]) -- 0x6004c00 con 0x4acb000 2026-02-11T23:31:27.631+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.632+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 513 ==== MOSDPGPush(1.0 116/113 [PushOp(1:6681cfe7:::obj102:head, version: 111'102, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6681cfe7:::obj102:head@111'102, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559d520 con 0x4acb000 2026-02-11T23:31:27.632+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:6681cfe7:::obj102:head, version: 111'102, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6681cfe7:::obj102:head@111'102, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 116 2026-02-11T23:31:27.632+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:6681cfe7:::obj102:head, version: 111'102, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6681cfe7:::obj102:head@111'102, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.632+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.632+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 514 ==== pg_backfill(progress 1.0 e 116/116 lb 1:6623e3eb:::obj517:head) ==== 1062+0+0 (unknown 0 0 0) 0x6015080 con 0x4acb000 2026-02-11T23:31:27.632+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:6623e3eb:::obj517:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.632+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:6681cfe7:::obj102:head, version: 111'102, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6681cfe7:::obj102:head@111'102, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.632+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:6623e3eb:::obj517:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.632+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.632+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:6681cfe7:::obj102:head, version: 111'102, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6681cfe7:::obj102:head@111'102, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.632+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:6681cfe7:::obj102:head, version: 111'102, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6681cfe7:::obj102:head@111'102, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(1.0 116/113 [PushOp(1:6681cfe7:::obj102:head, version: 111'102, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6681cfe7:::obj102:head@111'102, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:65a1355c:::obj536:head local-lis/les=113/114 n=238 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.632+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.632+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.632+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:65a1355c:::obj536:head local-lis/les=113/114 n=238 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:6681cfe7:::obj102:head, version: 111'102, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6681cfe7:::obj102:head@111'102, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.632+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:6623e3eb:::obj517:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.632+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:65a1355c:::obj536:head local-lis/les=113/114 n=238 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:6681cfe7:::obj102:head@111'102, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.632+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:65a1355c:::obj536:head local-lis/les=113/114 n=238 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:6681cfe7:::obj102:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.632+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:65a1355c:::obj536:head local-lis/les=113/114 n=238 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:6681cfe7:::obj102:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.632+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:65a1355c:::obj536:head local-lis/les=113/114 n=238 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:6681cfe7:::obj102:head 2026-02-11T23:31:27.632+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:65a1355c:::obj536:head local-lis/les=113/114 n=238 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:6681cfe7:::obj102:head v 111'102 2026-02-11T23:31:27.632+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:65a1355c:::obj536:head local-lis/les=113/114 n=238 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.632+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.632+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:6681cfe7:::obj102:head, version: 111'102, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6681cfe7:::obj102:head@111'102, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:27.632+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.632+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.632+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:6623e3eb:::obj517:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.632+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:6623e3eb:::obj517:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 1.0 e 116/116 lb 1:6623e3eb:::obj517:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:65a1355c:::obj536:head local-lis/les=113/114 n=238 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.632+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.632+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:65a1355c:::obj536:head local-lis/les=113/114 n=238 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:6623e3eb:::obj517:head) v3 2026-02-11T23:31:27.632+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:65a1355c:::obj536:head local-lis/les=113/114 n=238 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:6623e3eb:::obj517:head) 2026-02-11T23:31:27.632+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.632+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:6623e3eb:::obj517:head) v3 finish latency 0.000009 2026-02-11T23:31:27.632+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6623e3eb:::obj517:head local-lis/les=113/114 n=239 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.632+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.632+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.632+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6623e3eb:::obj517:head local-lis/les=113/114 n=239 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.632+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.632+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.632+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.632+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fede00 con 0x4acb000 2026-02-11T23:31:27.632+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:6681cfe7:::obj102:head)]) -- 0x6004f00 con 0x4acb000 2026-02-11T23:31:27.632+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.632+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 515 ==== MOSDPGPush(1.0 116/113 [PushOp(1:6727c4f6:::obj39:head, version: 111'39, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6727c4f6:::obj39:head@111'39, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x55f1ba0 con 0x4acb000 2026-02-11T23:31:27.632+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:6727c4f6:::obj39:head, version: 111'39, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6727c4f6:::obj39:head@111'39, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 116 2026-02-11T23:31:27.632+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:6727c4f6:::obj39:head, version: 111'39, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6727c4f6:::obj39:head@111'39, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.632+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.632+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 516 ==== pg_backfill(progress 1.0 e 116/116 lb 1:6681cfe7:::obj102:head) ==== 1062+0+0 (unknown 0 0 0) 0x6015600 con 0x4acb000 2026-02-11T23:31:27.632+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:6727c4f6:::obj39:head, version: 111'39, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6727c4f6:::obj39:head@111'39, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.632+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:6681cfe7:::obj102:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 116 2026-02-11T23:31:27.632+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.632+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:6681cfe7:::obj102:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.632+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:6727c4f6:::obj39:head, version: 111'39, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6727c4f6:::obj39:head@111'39, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.632+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.632+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:6727c4f6:::obj39:head, version: 111'39, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6727c4f6:::obj39:head@111'39, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 116/113 [PushOp(1:6727c4f6:::obj39:head, version: 111'39, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6727c4f6:::obj39:head@111'39, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:6623e3eb:::obj517:head local-lis/les=113/114 n=239 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.632+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.632+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6623e3eb:::obj517:head local-lis/les=113/114 n=239 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:6727c4f6:::obj39:head, version: 111'39, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6727c4f6:::obj39:head@111'39, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.632+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.632+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6623e3eb:::obj517:head local-lis/les=113/114 n=239 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:6727c4f6:::obj39:head@111'39, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.632+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:6681cfe7:::obj102:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.632+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6623e3eb:::obj517:head local-lis/les=113/114 n=239 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:6727c4f6:::obj39:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.632+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6623e3eb:::obj517:head local-lis/les=113/114 n=239 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:6727c4f6:::obj39:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.632+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6623e3eb:::obj517:head local-lis/les=113/114 n=239 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:6727c4f6:::obj39:head 2026-02-11T23:31:27.632+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6623e3eb:::obj517:head local-lis/les=113/114 n=239 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:6727c4f6:::obj39:head v 111'39 2026-02-11T23:31:27.632+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6623e3eb:::obj517:head local-lis/les=113/114 n=239 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.632+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.632+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:6727c4f6:::obj39:head, version: 111'39, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6727c4f6:::obj39:head@111'39, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:27.632+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.632+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.632+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:6681cfe7:::obj102:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.632+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:6681cfe7:::obj102:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 1.0 e 116/116 lb 1:6681cfe7:::obj102:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:6623e3eb:::obj517:head local-lis/les=113/114 n=239 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.632+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.632+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6623e3eb:::obj517:head local-lis/les=113/114 n=239 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:6681cfe7:::obj102:head) v3 2026-02-11T23:31:27.632+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6623e3eb:::obj517:head local-lis/les=113/114 n=239 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:6681cfe7:::obj102:head) 2026-02-11T23:31:27.632+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.632+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:6681cfe7:::obj102:head) v3 finish latency 0.000009 2026-02-11T23:31:27.632+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6681cfe7:::obj102:head local-lis/les=113/114 n=240 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.632+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.633+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.633+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6681cfe7:::obj102:head local-lis/les=113/114 n=240 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.633+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.633+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.633+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.633+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fcea80 con 0x4acb000 2026-02-11T23:31:27.633+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:6727c4f6:::obj39:head)]) -- 0x6005200 con 0x4acb000 2026-02-11T23:31:27.633+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.633+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 517 ==== MOSDPGPush(1.0 116/113 [PushOp(1:67a26c0a:::obj53:head, version: 111'53, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:67a26c0a:::obj53:head@111'53, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x559cb60 con 0x4acb000 2026-02-11T23:31:27.633+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:67a26c0a:::obj53:head, version: 111'53, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:67a26c0a:::obj53:head@111'53, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000016 epoch 116 2026-02-11T23:31:27.633+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:67a26c0a:::obj53:head, version: 111'53, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:67a26c0a:::obj53:head@111'53, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.633+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.633+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 518 ==== pg_backfill(progress 1.0 e 116/116 lb 1:6727c4f6:::obj39:head) ==== 1061+0+0 (unknown 0 0 0) 0x6015b80 con 0x4acb000 2026-02-11T23:31:27.633+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:6727c4f6:::obj39:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.633+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:67a26c0a:::obj53:head, version: 111'53, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:67a26c0a:::obj53:head@111'53, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.633+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:6727c4f6:::obj39:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.633+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.633+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:67a26c0a:::obj53:head, version: 111'53, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:67a26c0a:::obj53:head@111'53, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.633+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.633+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:6727c4f6:::obj39:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.633+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:67a26c0a:::obj53:head, version: 111'53, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:67a26c0a:::obj53:head@111'53, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(1.0 116/113 [PushOp(1:67a26c0a:::obj53:head, version: 111'53, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:67a26c0a:::obj53:head@111'53, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:6681cfe7:::obj102:head local-lis/les=113/114 n=240 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.633+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.633+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6681cfe7:::obj102:head local-lis/les=113/114 n=240 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:67a26c0a:::obj53:head, version: 111'53, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:67a26c0a:::obj53:head@111'53, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.633+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6681cfe7:::obj102:head local-lis/les=113/114 n=240 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:67a26c0a:::obj53:head@111'53, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.633+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6681cfe7:::obj102:head local-lis/les=113/114 n=240 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:67a26c0a:::obj53:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.633+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6681cfe7:::obj102:head local-lis/les=113/114 n=240 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:67a26c0a:::obj53:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.633+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6681cfe7:::obj102:head local-lis/les=113/114 n=240 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:67a26c0a:::obj53:head 2026-02-11T23:31:27.633+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6681cfe7:::obj102:head local-lis/les=113/114 n=240 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:67a26c0a:::obj53:head v 111'53 2026-02-11T23:31:27.633+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6681cfe7:::obj102:head local-lis/les=113/114 n=240 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.633+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.633+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:67a26c0a:::obj53:head, version: 111'53, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:67a26c0a:::obj53:head@111'53, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.633+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.633+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.633+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:6727c4f6:::obj39:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.633+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:6727c4f6:::obj39:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 116/116 lb 1:6727c4f6:::obj39:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:6681cfe7:::obj102:head local-lis/les=113/114 n=240 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.633+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.633+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6681cfe7:::obj102:head local-lis/les=113/114 n=240 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:6727c4f6:::obj39:head) v3 2026-02-11T23:31:27.633+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6681cfe7:::obj102:head local-lis/les=113/114 n=240 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:6727c4f6:::obj39:head) 2026-02-11T23:31:27.633+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.633+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:6727c4f6:::obj39:head) v3 finish latency 0.000009 2026-02-11T23:31:27.633+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6727c4f6:::obj39:head local-lis/les=113/114 n=241 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.633+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.633+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.633+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6727c4f6:::obj39:head local-lis/les=113/114 n=241 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.633+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.633+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.633+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.633+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fedc80 con 0x4acb000 2026-02-11T23:31:27.633+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:67a26c0a:::obj53:head)]) -- 0x6005380 con 0x4acb000 2026-02-11T23:31:27.633+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.633+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 519 ==== MOSDPGPush(1.0 116/113 [PushOp(1:685e1da9:::obj264:head, version: 111'264, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:685e1da9:::obj264:head@111'264, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f11e0 con 0x4acb000 2026-02-11T23:31:27.633+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:685e1da9:::obj264:head, version: 111'264, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:685e1da9:::obj264:head@111'264, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 116 2026-02-11T23:31:27.633+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:685e1da9:::obj264:head, version: 111'264, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:685e1da9:::obj264:head@111'264, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.633+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.633+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 520 ==== pg_backfill(progress 1.0 e 116/116 lb 1:67a26c0a:::obj53:head) ==== 1061+0+0 (unknown 0 0 0) 0x6016100 con 0x4acb000 2026-02-11T23:31:27.633+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:67a26c0a:::obj53:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.633+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:685e1da9:::obj264:head, version: 111'264, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:685e1da9:::obj264:head@111'264, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.633+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:67a26c0a:::obj53:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.633+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.634+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:685e1da9:::obj264:head, version: 111'264, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:685e1da9:::obj264:head@111'264, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.634+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:685e1da9:::obj264:head, version: 111'264, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:685e1da9:::obj264:head@111'264, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 116/113 [PushOp(1:685e1da9:::obj264:head, version: 111'264, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:685e1da9:::obj264:head@111'264, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:6727c4f6:::obj39:head local-lis/les=113/114 n=241 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.634+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.634+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.634+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:67a26c0a:::obj53:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.634+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6727c4f6:::obj39:head local-lis/les=113/114 n=241 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:685e1da9:::obj264:head, version: 111'264, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:685e1da9:::obj264:head@111'264, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.634+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6727c4f6:::obj39:head local-lis/les=113/114 n=241 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:685e1da9:::obj264:head@111'264, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.634+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6727c4f6:::obj39:head local-lis/les=113/114 n=241 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:685e1da9:::obj264:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.634+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6727c4f6:::obj39:head local-lis/les=113/114 n=241 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:685e1da9:::obj264:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.634+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6727c4f6:::obj39:head local-lis/les=113/114 n=241 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:685e1da9:::obj264:head 2026-02-11T23:31:27.634+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6727c4f6:::obj39:head local-lis/les=113/114 n=241 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:685e1da9:::obj264:head v 111'264 2026-02-11T23:31:27.634+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6727c4f6:::obj39:head local-lis/les=113/114 n=241 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.634+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.634+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:685e1da9:::obj264:head, version: 111'264, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:685e1da9:::obj264:head@111'264, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:27.634+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.634+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.634+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:67a26c0a:::obj53:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.634+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:67a26c0a:::obj53:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 1.0 e 116/116 lb 1:67a26c0a:::obj53:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:6727c4f6:::obj39:head local-lis/les=113/114 n=241 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.634+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.634+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6727c4f6:::obj39:head local-lis/les=113/114 n=241 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:67a26c0a:::obj53:head) v3 2026-02-11T23:31:27.634+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6727c4f6:::obj39:head local-lis/les=113/114 n=241 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:67a26c0a:::obj53:head) 2026-02-11T23:31:27.634+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.634+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:67a26c0a:::obj53:head) v3 finish latency 0.000010 2026-02-11T23:31:27.634+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:67a26c0a:::obj53:head local-lis/les=113/114 n=242 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.634+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.634+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.634+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:67a26c0a:::obj53:head local-lis/les=113/114 n=242 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.634+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.634+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.634+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.634+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fce780 con 0x4acb000 2026-02-11T23:31:27.634+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:685e1da9:::obj264:head)]) -- 0x6005500 con 0x4acb000 2026-02-11T23:31:27.634+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.634+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 521 ==== MOSDPGPush(1.0 116/113 [PushOp(1:685e6e29:::obj176:head, version: 111'176, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:685e6e29:::obj176:head@111'176, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f1d40 con 0x4acb000 2026-02-11T23:31:27.634+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:685e6e29:::obj176:head, version: 111'176, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:685e6e29:::obj176:head@111'176, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 116 2026-02-11T23:31:27.634+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:685e6e29:::obj176:head, version: 111'176, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:685e6e29:::obj176:head@111'176, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.634+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.634+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 522 ==== pg_backfill(progress 1.0 e 116/116 lb 1:685e1da9:::obj264:head) ==== 1062+0+0 (unknown 0 0 0) 0x6016680 con 0x4acb000 2026-02-11T23:31:27.634+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:685e1da9:::obj264:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.634+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:685e6e29:::obj176:head, version: 111'176, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:685e6e29:::obj176:head@111'176, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.634+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:685e1da9:::obj264:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.634+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.634+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:685e6e29:::obj176:head, version: 111'176, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:685e6e29:::obj176:head@111'176, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.634+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:685e6e29:::obj176:head, version: 111'176, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:685e6e29:::obj176:head@111'176, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 116/113 [PushOp(1:685e6e29:::obj176:head, version: 111'176, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:685e6e29:::obj176:head@111'176, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:67a26c0a:::obj53:head local-lis/les=113/114 n=242 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.634+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.634+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.634+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:685e1da9:::obj264:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.634+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:67a26c0a:::obj53:head local-lis/les=113/114 n=242 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:685e6e29:::obj176:head, version: 111'176, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:685e6e29:::obj176:head@111'176, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.634+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:67a26c0a:::obj53:head local-lis/les=113/114 n=242 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:685e6e29:::obj176:head@111'176, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.634+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:67a26c0a:::obj53:head local-lis/les=113/114 n=242 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:685e6e29:::obj176:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.634+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:67a26c0a:::obj53:head local-lis/les=113/114 n=242 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:685e6e29:::obj176:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.634+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:67a26c0a:::obj53:head local-lis/les=113/114 n=242 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:685e6e29:::obj176:head 2026-02-11T23:31:27.634+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:67a26c0a:::obj53:head local-lis/les=113/114 n=242 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:685e6e29:::obj176:head v 111'176 2026-02-11T23:31:27.634+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:67a26c0a:::obj53:head local-lis/les=113/114 n=242 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.634+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.634+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:685e6e29:::obj176:head, version: 111'176, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:685e6e29:::obj176:head@111'176, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.634+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.634+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.634+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:685e1da9:::obj264:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.634+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:685e1da9:::obj264:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 1.0 e 116/116 lb 1:685e1da9:::obj264:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:67a26c0a:::obj53:head local-lis/les=113/114 n=242 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.634+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.634+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:67a26c0a:::obj53:head local-lis/les=113/114 n=242 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:685e1da9:::obj264:head) v3 2026-02-11T23:31:27.634+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:67a26c0a:::obj53:head local-lis/les=113/114 n=242 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:685e1da9:::obj264:head) 2026-02-11T23:31:27.634+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.634+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:685e1da9:::obj264:head) v3 finish latency 0.000009 2026-02-11T23:31:27.634+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:685e1da9:::obj264:head local-lis/les=113/114 n=243 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.634+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.634+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.634+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:685e1da9:::obj264:head local-lis/les=113/114 n=243 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.634+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.634+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.634+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.634+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fedb00 con 0x4acb000 2026-02-11T23:31:27.634+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:685e6e29:::obj176:head)]) -- 0x6005800 con 0x4acb000 2026-02-11T23:31:27.634+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.635+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 523 ==== MOSDPGPush(1.0 116/113 [PushOp(1:68743261:::obj370:head, version: 111'370, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:68743261:::obj370:head@111'370, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f0b60 con 0x4acb000 2026-02-11T23:31:27.635+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:68743261:::obj370:head, version: 111'370, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:68743261:::obj370:head@111'370, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 116 2026-02-11T23:31:27.635+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:68743261:::obj370:head, version: 111'370, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:68743261:::obj370:head@111'370, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.635+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.635+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 524 ==== pg_backfill(progress 1.0 e 116/116 lb 1:685e6e29:::obj176:head) ==== 1062+0+0 (unknown 0 0 0) 0x6016c00 con 0x4acb000 2026-02-11T23:31:27.635+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:685e6e29:::obj176:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.635+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:685e6e29:::obj176:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.635+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:68743261:::obj370:head, version: 111'370, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:68743261:::obj370:head@111'370, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.635+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.635+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:68743261:::obj370:head, version: 111'370, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:68743261:::obj370:head@111'370, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.635+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:68743261:::obj370:head, version: 111'370, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:68743261:::obj370:head@111'370, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 116/113 [PushOp(1:68743261:::obj370:head, version: 111'370, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:68743261:::obj370:head@111'370, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:685e1da9:::obj264:head local-lis/les=113/114 n=243 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.635+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.635+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.635+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:685e6e29:::obj176:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.635+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:685e1da9:::obj264:head local-lis/les=113/114 n=243 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:68743261:::obj370:head, version: 111'370, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:68743261:::obj370:head@111'370, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.635+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:685e1da9:::obj264:head local-lis/les=113/114 n=243 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:68743261:::obj370:head@111'370, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.635+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:685e1da9:::obj264:head local-lis/les=113/114 n=243 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:68743261:::obj370:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.635+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:685e1da9:::obj264:head local-lis/les=113/114 n=243 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:68743261:::obj370:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.635+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:685e1da9:::obj264:head local-lis/les=113/114 n=243 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:68743261:::obj370:head 2026-02-11T23:31:27.635+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:685e1da9:::obj264:head local-lis/les=113/114 n=243 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:68743261:::obj370:head v 111'370 2026-02-11T23:31:27.635+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:685e1da9:::obj264:head local-lis/les=113/114 n=243 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.635+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.635+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:68743261:::obj370:head, version: 111'370, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:68743261:::obj370:head@111'370, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:27.635+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.635+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.635+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:685e6e29:::obj176:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.635+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:685e6e29:::obj176:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 1.0 e 116/116 lb 1:685e6e29:::obj176:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:685e1da9:::obj264:head local-lis/les=113/114 n=243 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.635+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.635+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:685e1da9:::obj264:head local-lis/les=113/114 n=243 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:685e6e29:::obj176:head) v3 2026-02-11T23:31:27.635+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:685e1da9:::obj264:head local-lis/les=113/114 n=243 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:685e6e29:::obj176:head) 2026-02-11T23:31:27.635+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.635+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:685e6e29:::obj176:head) v3 finish latency 0.000009 2026-02-11T23:31:27.635+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:685e6e29:::obj176:head local-lis/les=113/114 n=244 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.635+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.635+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.635+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:685e6e29:::obj176:head local-lis/les=113/114 n=244 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.635+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.635+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.635+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.635+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fce480 con 0x4acb000 2026-02-11T23:31:27.635+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:68743261:::obj370:head)]) -- 0x6005b00 con 0x4acb000 2026-02-11T23:31:27.635+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.635+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 525 ==== MOSDPGPush(1.0 116/113 [PushOp(1:68869f42:::obj433:head, version: 111'433, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:68869f42:::obj433:head@111'433, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f16c0 con 0x4acb000 2026-02-11T23:31:27.635+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:68869f42:::obj433:head, version: 111'433, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:68869f42:::obj433:head@111'433, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 116 2026-02-11T23:31:27.635+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:68869f42:::obj433:head, version: 111'433, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:68869f42:::obj433:head@111'433, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.635+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.635+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 526 ==== pg_backfill(progress 1.0 e 116/116 lb 1:68743261:::obj370:head) ==== 1062+0+0 (unknown 0 0 0) 0x6017180 con 0x4acb000 2026-02-11T23:31:27.635+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:68869f42:::obj433:head, version: 111'433, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:68869f42:::obj433:head@111'433, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.635+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:68743261:::obj370:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 116 2026-02-11T23:31:27.635+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.635+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:68869f42:::obj433:head, version: 111'433, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:68869f42:::obj433:head@111'433, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.635+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:68743261:::obj370:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.635+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.635+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:68743261:::obj370:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.635+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:68869f42:::obj433:head, version: 111'433, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:68869f42:::obj433:head@111'433, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 116/113 [PushOp(1:68869f42:::obj433:head, version: 111'433, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:68869f42:::obj433:head@111'433, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:685e6e29:::obj176:head local-lis/les=113/114 n=244 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.635+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.635+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:685e6e29:::obj176:head local-lis/les=113/114 n=244 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:68869f42:::obj433:head, version: 111'433, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:68869f42:::obj433:head@111'433, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.635+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:685e6e29:::obj176:head local-lis/les=113/114 n=244 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:68869f42:::obj433:head@111'433, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.635+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:685e6e29:::obj176:head local-lis/les=113/114 n=244 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:68869f42:::obj433:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.635+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:685e6e29:::obj176:head local-lis/les=113/114 n=244 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:68869f42:::obj433:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.635+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:685e6e29:::obj176:head local-lis/les=113/114 n=244 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:68869f42:::obj433:head 2026-02-11T23:31:27.635+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:685e6e29:::obj176:head local-lis/les=113/114 n=244 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:68869f42:::obj433:head v 111'433 2026-02-11T23:31:27.635+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:685e6e29:::obj176:head local-lis/les=113/114 n=244 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.635+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.635+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:68869f42:::obj433:head, version: 111'433, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:68869f42:::obj433:head@111'433, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.635+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.635+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.635+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:68743261:::obj370:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.635+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:68743261:::obj370:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 1.0 e 116/116 lb 1:68743261:::obj370:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:685e6e29:::obj176:head local-lis/les=113/114 n=244 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.635+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.635+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:685e6e29:::obj176:head local-lis/les=113/114 n=244 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:68743261:::obj370:head) v3 2026-02-11T23:31:27.635+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:685e6e29:::obj176:head local-lis/les=113/114 n=244 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:68743261:::obj370:head) 2026-02-11T23:31:27.635+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.635+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:68743261:::obj370:head) v3 finish latency 0.000010 2026-02-11T23:31:27.635+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:68743261:::obj370:head local-lis/les=113/114 n=245 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.635+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.636+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.636+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:68743261:::obj370:head local-lis/les=113/114 n=245 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.636+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.636+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.636+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.636+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fed980 con 0x4acb000 2026-02-11T23:31:27.636+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:68869f42:::obj433:head)]) -- 0x6005e00 con 0x4acb000 2026-02-11T23:31:27.636+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.636+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 527 ==== MOSDPGPush(1.0 116/113 [PushOp(1:69182b79:::obj301:head, version: 111'301, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:69182b79:::obj301:head@111'301, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c680 con 0x4acb000 2026-02-11T23:31:27.636+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:69182b79:::obj301:head, version: 111'301, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:69182b79:::obj301:head@111'301, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 116 2026-02-11T23:31:27.636+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:69182b79:::obj301:head, version: 111'301, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:69182b79:::obj301:head@111'301, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.636+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.636+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 528 ==== pg_backfill(progress 1.0 e 116/116 lb 1:68869f42:::obj433:head) ==== 1062+0+0 (unknown 0 0 0) 0x6017700 con 0x4acb000 2026-02-11T23:31:27.636+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:68869f42:::obj433:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.636+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:69182b79:::obj301:head, version: 111'301, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:69182b79:::obj301:head@111'301, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.636+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:68869f42:::obj433:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.636+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.636+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:69182b79:::obj301:head, version: 111'301, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:69182b79:::obj301:head@111'301, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.636+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:69182b79:::obj301:head, version: 111'301, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:69182b79:::obj301:head@111'301, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000040 MOSDPGPush(1.0 116/113 [PushOp(1:69182b79:::obj301:head, version: 111'301, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:69182b79:::obj301:head@111'301, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:68743261:::obj370:head local-lis/les=113/114 n=245 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.636+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.636+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.636+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:68743261:::obj370:head local-lis/les=113/114 n=245 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:69182b79:::obj301:head, version: 111'301, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:69182b79:::obj301:head@111'301, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.636+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:68869f42:::obj433:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.636+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:68743261:::obj370:head local-lis/les=113/114 n=245 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:69182b79:::obj301:head@111'301, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.636+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:68743261:::obj370:head local-lis/les=113/114 n=245 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:69182b79:::obj301:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.636+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:68743261:::obj370:head local-lis/les=113/114 n=245 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:69182b79:::obj301:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.636+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:68743261:::obj370:head local-lis/les=113/114 n=245 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:69182b79:::obj301:head 2026-02-11T23:31:27.636+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:68743261:::obj370:head local-lis/les=113/114 n=245 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:69182b79:::obj301:head v 111'301 2026-02-11T23:31:27.636+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:68743261:::obj370:head local-lis/les=113/114 n=245 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.636+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.636+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:69182b79:::obj301:head, version: 111'301, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:69182b79:::obj301:head@111'301, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.636+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.636+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.636+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:68869f42:::obj433:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.636+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:68869f42:::obj433:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:68869f42:::obj433:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:68743261:::obj370:head local-lis/les=113/114 n=245 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.636+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.636+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:68743261:::obj370:head local-lis/les=113/114 n=245 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:68869f42:::obj433:head) v3 2026-02-11T23:31:27.636+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:68743261:::obj370:head local-lis/les=113/114 n=245 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:68869f42:::obj433:head) 2026-02-11T23:31:27.636+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.636+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:68869f42:::obj433:head) v3 finish latency 0.000010 2026-02-11T23:31:27.636+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:68869f42:::obj433:head local-lis/les=113/114 n=246 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.636+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.636+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.636+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:68869f42:::obj433:head local-lis/les=113/114 n=246 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.636+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.636+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.636+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.636+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fce180 con 0x4acb000 2026-02-11T23:31:27.636+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:69182b79:::obj301:head)]) -- 0x5ca0180 con 0x4acb000 2026-02-11T23:31:27.636+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.636+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 529 ==== MOSDPGPush(1.0 116/113 [PushOp(1:6934f76e:::obj318:head, version: 111'318, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6934f76e:::obj318:head@111'318, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559dba0 con 0x4acb000 2026-02-11T23:31:27.636+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:6934f76e:::obj318:head, version: 111'318, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6934f76e:::obj318:head@111'318, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000015 epoch 116 2026-02-11T23:31:27.636+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:6934f76e:::obj318:head, version: 111'318, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6934f76e:::obj318:head@111'318, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.636+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.636+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 530 ==== pg_backfill(progress 1.0 e 116/116 lb 1:69182b79:::obj301:head) ==== 1062+0+0 (unknown 0 0 0) 0x6014000 con 0x4acb000 2026-02-11T23:31:27.636+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:6934f76e:::obj318:head, version: 111'318, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6934f76e:::obj318:head@111'318, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.636+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:69182b79:::obj301:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 116 2026-02-11T23:31:27.636+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.636+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:6934f76e:::obj318:head, version: 111'318, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6934f76e:::obj318:head@111'318, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.636+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:69182b79:::obj301:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.636+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:6934f76e:::obj318:head, version: 111'318, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6934f76e:::obj318:head@111'318, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(1.0 116/113 [PushOp(1:6934f76e:::obj318:head, version: 111'318, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6934f76e:::obj318:head@111'318, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:68869f42:::obj433:head local-lis/les=113/114 n=246 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.636+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.636+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.636+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:69182b79:::obj301:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.636+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:68869f42:::obj433:head local-lis/les=113/114 n=246 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:6934f76e:::obj318:head, version: 111'318, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6934f76e:::obj318:head@111'318, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.636+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:68869f42:::obj433:head local-lis/les=113/114 n=246 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:6934f76e:::obj318:head@111'318, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.636+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:68869f42:::obj433:head local-lis/les=113/114 n=246 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:6934f76e:::obj318:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.636+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:68869f42:::obj433:head local-lis/les=113/114 n=246 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:6934f76e:::obj318:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.636+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:68869f42:::obj433:head local-lis/les=113/114 n=246 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:6934f76e:::obj318:head 2026-02-11T23:31:27.636+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:68869f42:::obj433:head local-lis/les=113/114 n=246 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:6934f76e:::obj318:head v 111'318 2026-02-11T23:31:27.636+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:68869f42:::obj433:head local-lis/les=113/114 n=246 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.636+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.636+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:6934f76e:::obj318:head, version: 111'318, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6934f76e:::obj318:head@111'318, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.637+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.637+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.637+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:69182b79:::obj301:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.637+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:69182b79:::obj301:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 1.0 e 116/116 lb 1:69182b79:::obj301:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:68869f42:::obj433:head local-lis/les=113/114 n=246 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.637+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.637+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:68869f42:::obj433:head local-lis/les=113/114 n=246 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:69182b79:::obj301:head) v3 2026-02-11T23:31:27.637+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:68869f42:::obj433:head local-lis/les=113/114 n=246 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:69182b79:::obj301:head) 2026-02-11T23:31:27.637+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.637+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:69182b79:::obj301:head) v3 finish latency 0.000009 2026-02-11T23:31:27.637+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:69182b79:::obj301:head local-lis/les=113/114 n=247 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.637+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.637+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.637+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:69182b79:::obj301:head local-lis/les=113/114 n=247 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.637+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.637+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.637+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.637+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fed800 con 0x4acb000 2026-02-11T23:31:27.637+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:6934f76e:::obj318:head)]) -- 0x5ca0480 con 0x4acb000 2026-02-11T23:31:27.637+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.637+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 531 ==== MOSDPGPush(1.0 116/113 [PushOp(1:6971da6a:::obj293:head, version: 111'293, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6971da6a:::obj293:head@111'293, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559d040 con 0x4acb000 2026-02-11T23:31:27.637+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:6971da6a:::obj293:head, version: 111'293, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6971da6a:::obj293:head@111'293, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 116 2026-02-11T23:31:27.637+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:6971da6a:::obj293:head, version: 111'293, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6971da6a:::obj293:head@111'293, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.637+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.637+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 532 ==== pg_backfill(progress 1.0 e 116/116 lb 1:6934f76e:::obj318:head) ==== 1062+0+0 (unknown 0 0 0) 0x6014580 con 0x4acb000 2026-02-11T23:31:27.637+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:6971da6a:::obj293:head, version: 111'293, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6971da6a:::obj293:head@111'293, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.637+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:6934f76e:::obj318:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.637+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.637+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:6934f76e:::obj318:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.637+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:6971da6a:::obj293:head, version: 111'293, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6971da6a:::obj293:head@111'293, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.637+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.637+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:6971da6a:::obj293:head, version: 111'293, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6971da6a:::obj293:head@111'293, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 116/113 [PushOp(1:6971da6a:::obj293:head, version: 111'293, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6971da6a:::obj293:head@111'293, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:69182b79:::obj301:head local-lis/les=113/114 n=247 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.637+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.637+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:69182b79:::obj301:head local-lis/les=113/114 n=247 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:6971da6a:::obj293:head, version: 111'293, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6971da6a:::obj293:head@111'293, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.637+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:69182b79:::obj301:head local-lis/les=113/114 n=247 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:6971da6a:::obj293:head@111'293, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.637+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.637+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:6934f76e:::obj318:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.637+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:69182b79:::obj301:head local-lis/les=113/114 n=247 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:6971da6a:::obj293:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.637+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:69182b79:::obj301:head local-lis/les=113/114 n=247 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:6971da6a:::obj293:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.637+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:69182b79:::obj301:head local-lis/les=113/114 n=247 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:6971da6a:::obj293:head 2026-02-11T23:31:27.637+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:69182b79:::obj301:head local-lis/les=113/114 n=247 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:6971da6a:::obj293:head v 111'293 2026-02-11T23:31:27.637+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:69182b79:::obj301:head local-lis/les=113/114 n=247 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.637+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.637+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:6971da6a:::obj293:head, version: 111'293, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6971da6a:::obj293:head@111'293, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.637+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.637+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.637+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:6934f76e:::obj318:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.637+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:6934f76e:::obj318:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:6934f76e:::obj318:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:69182b79:::obj301:head local-lis/les=113/114 n=247 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.637+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.637+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:69182b79:::obj301:head local-lis/les=113/114 n=247 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:6934f76e:::obj318:head) v3 2026-02-11T23:31:27.637+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:69182b79:::obj301:head local-lis/les=113/114 n=247 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:6934f76e:::obj318:head) 2026-02-11T23:31:27.637+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.637+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:6934f76e:::obj318:head) v3 finish latency 0.000009 2026-02-11T23:31:27.637+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6934f76e:::obj318:head local-lis/les=113/114 n=248 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.637+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.637+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.637+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6934f76e:::obj318:head local-lis/les=113/114 n=248 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.637+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.637+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.637+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.637+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fed380 con 0x4acb000 2026-02-11T23:31:27.637+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:6971da6a:::obj293:head)]) -- 0x5ca0780 con 0x4acb000 2026-02-11T23:31:27.637+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.638+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 533 ==== MOSDPGPush(1.0 116/113 [PushOp(1:698e6c1e:::obj207:head, version: 111'207, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:698e6c1e:::obj207:head@111'207, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d89c0 con 0x4acb000 2026-02-11T23:31:27.638+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:698e6c1e:::obj207:head, version: 111'207, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:698e6c1e:::obj207:head@111'207, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 116 2026-02-11T23:31:27.638+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:698e6c1e:::obj207:head, version: 111'207, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:698e6c1e:::obj207:head@111'207, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.638+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.638+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 534 ==== pg_backfill(progress 1.0 e 116/116 lb 1:6971da6a:::obj293:head) ==== 1062+0+0 (unknown 0 0 0) 0x6014b00 con 0x4acb000 2026-02-11T23:31:27.638+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:698e6c1e:::obj207:head, version: 111'207, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:698e6c1e:::obj207:head@111'207, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.638+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:6971da6a:::obj293:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.638+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.638+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:6971da6a:::obj293:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.638+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:698e6c1e:::obj207:head, version: 111'207, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:698e6c1e:::obj207:head@111'207, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.638+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.638+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:698e6c1e:::obj207:head, version: 111'207, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:698e6c1e:::obj207:head@111'207, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 116/113 [PushOp(1:698e6c1e:::obj207:head, version: 111'207, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:698e6c1e:::obj207:head@111'207, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:6934f76e:::obj318:head local-lis/les=113/114 n=248 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.638+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.638+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6934f76e:::obj318:head local-lis/les=113/114 n=248 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:698e6c1e:::obj207:head, version: 111'207, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:698e6c1e:::obj207:head@111'207, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.638+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6934f76e:::obj318:head local-lis/les=113/114 n=248 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:698e6c1e:::obj207:head@111'207, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.638+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.638+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:6971da6a:::obj293:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.638+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6934f76e:::obj318:head local-lis/les=113/114 n=248 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:698e6c1e:::obj207:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.638+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6934f76e:::obj318:head local-lis/les=113/114 n=248 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:698e6c1e:::obj207:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.638+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6934f76e:::obj318:head local-lis/les=113/114 n=248 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:698e6c1e:::obj207:head 2026-02-11T23:31:27.638+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6934f76e:::obj318:head local-lis/les=113/114 n=248 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:698e6c1e:::obj207:head v 111'207 2026-02-11T23:31:27.638+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6934f76e:::obj318:head local-lis/les=113/114 n=248 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.638+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.638+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:698e6c1e:::obj207:head, version: 111'207, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:698e6c1e:::obj207:head@111'207, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:27.638+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.638+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.638+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:6971da6a:::obj293:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.638+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:6971da6a:::obj293:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 1.0 e 116/116 lb 1:6971da6a:::obj293:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:6934f76e:::obj318:head local-lis/les=113/114 n=248 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.638+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.638+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6934f76e:::obj318:head local-lis/les=113/114 n=248 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:6971da6a:::obj293:head) v3 2026-02-11T23:31:27.638+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6934f76e:::obj318:head local-lis/les=113/114 n=248 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:6971da6a:::obj293:head) 2026-02-11T23:31:27.638+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.638+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:6971da6a:::obj293:head) v3 finish latency 0.000010 2026-02-11T23:31:27.638+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6971da6a:::obj293:head local-lis/les=113/114 n=249 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.638+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.638+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.638+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6971da6a:::obj293:head local-lis/les=113/114 n=249 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.638+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.638+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.638+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.638+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fed680 con 0x4acb000 2026-02-11T23:31:27.638+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:698e6c1e:::obj207:head)]) -- 0x5ca0a80 con 0x4acb000 2026-02-11T23:31:27.638+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.638+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 535 ==== MOSDPGPush(1.0 116/113 [PushOp(1:699d4764:::obj50:head, version: 111'50, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:699d4764:::obj50:head@111'50, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x56cb1e0 con 0x4acb000 2026-02-11T23:31:27.638+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:699d4764:::obj50:head, version: 111'50, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:699d4764:::obj50:head@111'50, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.638+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:699d4764:::obj50:head, version: 111'50, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:699d4764:::obj50:head@111'50, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.638+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.638+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 536 ==== pg_backfill(progress 1.0 e 116/116 lb 1:698e6c1e:::obj207:head) ==== 1062+0+0 (unknown 0 0 0) 0x6015080 con 0x4acb000 2026-02-11T23:31:27.638+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:699d4764:::obj50:head, version: 111'50, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:699d4764:::obj50:head@111'50, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.638+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:698e6c1e:::obj207:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.638+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.638+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:698e6c1e:::obj207:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.638+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:699d4764:::obj50:head, version: 111'50, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:699d4764:::obj50:head@111'50, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.638+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.638+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:699d4764:::obj50:head, version: 111'50, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:699d4764:::obj50:head@111'50, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:699d4764:::obj50:head, version: 111'50, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:699d4764:::obj50:head@111'50, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:6971da6a:::obj293:head local-lis/les=113/114 n=249 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.638+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.638+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6971da6a:::obj293:head local-lis/les=113/114 n=249 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:699d4764:::obj50:head, version: 111'50, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:699d4764:::obj50:head@111'50, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.638+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6971da6a:::obj293:head local-lis/les=113/114 n=249 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:699d4764:::obj50:head@111'50, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.638+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.638+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:698e6c1e:::obj207:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.638+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6971da6a:::obj293:head local-lis/les=113/114 n=249 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:699d4764:::obj50:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.638+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6971da6a:::obj293:head local-lis/les=113/114 n=249 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:699d4764:::obj50:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.638+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6971da6a:::obj293:head local-lis/les=113/114 n=249 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:699d4764:::obj50:head 2026-02-11T23:31:27.638+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6971da6a:::obj293:head local-lis/les=113/114 n=249 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:699d4764:::obj50:head v 111'50 2026-02-11T23:31:27.638+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6971da6a:::obj293:head local-lis/les=113/114 n=249 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.638+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.638+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:699d4764:::obj50:head, version: 111'50, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:699d4764:::obj50:head@111'50, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.638+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.638+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.638+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:698e6c1e:::obj207:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.638+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:698e6c1e:::obj207:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 1.0 e 116/116 lb 1:698e6c1e:::obj207:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:6971da6a:::obj293:head local-lis/les=113/114 n=249 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.638+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.638+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6971da6a:::obj293:head local-lis/les=113/114 n=249 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:698e6c1e:::obj207:head) v3 2026-02-11T23:31:27.638+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6971da6a:::obj293:head local-lis/les=113/114 n=249 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:698e6c1e:::obj207:head) 2026-02-11T23:31:27.638+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.638+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:698e6c1e:::obj207:head) v3 finish latency 0.000009 2026-02-11T23:31:27.638+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:698e6c1e:::obj207:head local-lis/les=113/114 n=250 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.638+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.638+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.638+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:698e6c1e:::obj207:head local-lis/les=113/114 n=250 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.638+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.638+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.638+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.638+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d1de00 con 0x4acb000 2026-02-11T23:31:27.638+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:699d4764:::obj50:head)]) -- 0x5ca0d80 con 0x4acb000 2026-02-11T23:31:27.638+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.639+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 537 ==== MOSDPGPush(1.0 116/113 [PushOp(1:69f21cc4:::obj383:head, version: 111'383, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:69f21cc4:::obj383:head@111'383, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d9520 con 0x4acb000 2026-02-11T23:31:27.639+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:69f21cc4:::obj383:head, version: 111'383, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:69f21cc4:::obj383:head@111'383, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 116 2026-02-11T23:31:27.639+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:69f21cc4:::obj383:head, version: 111'383, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:69f21cc4:::obj383:head@111'383, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.639+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.639+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 538 ==== pg_backfill(progress 1.0 e 116/116 lb 1:699d4764:::obj50:head) ==== 1061+0+0 (unknown 0 0 0) 0x6015600 con 0x4acb000 2026-02-11T23:31:27.639+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:69f21cc4:::obj383:head, version: 111'383, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:69f21cc4:::obj383:head@111'383, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.639+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.639+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:69f21cc4:::obj383:head, version: 111'383, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:69f21cc4:::obj383:head@111'383, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.639+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:699d4764:::obj50:head) v3 prio 15 type 95 cost 0 latency 0.000012 epoch 116 2026-02-11T23:31:27.639+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:69f21cc4:::obj383:head, version: 111'383, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:69f21cc4:::obj383:head@111'383, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(1.0 116/113 [PushOp(1:69f21cc4:::obj383:head, version: 111'383, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:69f21cc4:::obj383:head@111'383, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:698e6c1e:::obj207:head local-lis/les=113/114 n=250 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.639+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.639+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:699d4764:::obj50:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.639+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:698e6c1e:::obj207:head local-lis/les=113/114 n=250 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:69f21cc4:::obj383:head, version: 111'383, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:69f21cc4:::obj383:head@111'383, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.639+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.639+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:698e6c1e:::obj207:head local-lis/les=113/114 n=250 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:69f21cc4:::obj383:head@111'383, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.639+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:698e6c1e:::obj207:head local-lis/les=113/114 n=250 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:69f21cc4:::obj383:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.639+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:698e6c1e:::obj207:head local-lis/les=113/114 n=250 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:69f21cc4:::obj383:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.639+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:698e6c1e:::obj207:head local-lis/les=113/114 n=250 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:69f21cc4:::obj383:head 2026-02-11T23:31:27.639+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.639+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:699d4764:::obj50:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.639+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:698e6c1e:::obj207:head local-lis/les=113/114 n=250 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:69f21cc4:::obj383:head v 111'383 2026-02-11T23:31:27.639+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:698e6c1e:::obj207:head local-lis/les=113/114 n=250 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.639+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.639+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:69f21cc4:::obj383:head, version: 111'383, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:69f21cc4:::obj383:head@111'383, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.639+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.639+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.639+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:699d4764:::obj50:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.639+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:699d4764:::obj50:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:699d4764:::obj50:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:698e6c1e:::obj207:head local-lis/les=113/114 n=250 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.639+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.639+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:698e6c1e:::obj207:head local-lis/les=113/114 n=250 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:699d4764:::obj50:head) v3 2026-02-11T23:31:27.639+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:698e6c1e:::obj207:head local-lis/les=113/114 n=250 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:699d4764:::obj50:head) 2026-02-11T23:31:27.639+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.639+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:699d4764:::obj50:head) v3 finish latency 0.000009 2026-02-11T23:31:27.639+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:699d4764:::obj50:head local-lis/les=113/114 n=251 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.639+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.639+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.639+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:699d4764:::obj50:head local-lis/les=113/114 n=251 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.639+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.639+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.639+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.639+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fed200 con 0x4acb000 2026-02-11T23:31:27.639+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:69f21cc4:::obj383:head)]) -- 0x5ca0f00 con 0x4acb000 2026-02-11T23:31:27.639+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.639+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 539 ==== MOSDPGPush(1.0 116/113 [PushOp(1:6c7994f7:::obj432:head, version: 111'432, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6c7994f7:::obj432:head@111'432, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d9040 con 0x4acb000 2026-02-11T23:31:27.639+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:6c7994f7:::obj432:head, version: 111'432, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6c7994f7:::obj432:head@111'432, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.639+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:6c7994f7:::obj432:head, version: 111'432, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6c7994f7:::obj432:head@111'432, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.639+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.639+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 540 ==== pg_backfill(progress 1.0 e 116/116 lb 1:69f21cc4:::obj383:head) ==== 1062+0+0 (unknown 0 0 0) 0x6015b80 con 0x4acb000 2026-02-11T23:31:27.639+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:6c7994f7:::obj432:head, version: 111'432, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6c7994f7:::obj432:head@111'432, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.639+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:69f21cc4:::obj383:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.639+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.639+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:69f21cc4:::obj383:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.639+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:6c7994f7:::obj432:head, version: 111'432, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6c7994f7:::obj432:head@111'432, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.639+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.639+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:6c7994f7:::obj432:head, version: 111'432, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6c7994f7:::obj432:head@111'432, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:6c7994f7:::obj432:head, version: 111'432, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6c7994f7:::obj432:head@111'432, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:699d4764:::obj50:head local-lis/les=113/114 n=251 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.639+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.639+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:699d4764:::obj50:head local-lis/les=113/114 n=251 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:6c7994f7:::obj432:head, version: 111'432, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6c7994f7:::obj432:head@111'432, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.639+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:699d4764:::obj50:head local-lis/les=113/114 n=251 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:6c7994f7:::obj432:head@111'432, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.639+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.639+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:69f21cc4:::obj383:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.639+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:699d4764:::obj50:head local-lis/les=113/114 n=251 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:6c7994f7:::obj432:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.639+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:699d4764:::obj50:head local-lis/les=113/114 n=251 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:6c7994f7:::obj432:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.639+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:699d4764:::obj50:head local-lis/les=113/114 n=251 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:6c7994f7:::obj432:head 2026-02-11T23:31:27.639+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:699d4764:::obj50:head local-lis/les=113/114 n=251 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:6c7994f7:::obj432:head v 111'432 2026-02-11T23:31:27.639+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:699d4764:::obj50:head local-lis/les=113/114 n=251 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.639+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.639+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:6c7994f7:::obj432:head, version: 111'432, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6c7994f7:::obj432:head@111'432, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000043 2026-02-11T23:31:27.639+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.639+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.639+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:69f21cc4:::obj383:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.639+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:69f21cc4:::obj383:head) v3 prio 15 cost 0 latency 0.000059 pg_backfill(progress 1.0 e 116/116 lb 1:69f21cc4:::obj383:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:699d4764:::obj50:head local-lis/les=113/114 n=251 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.639+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.639+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:699d4764:::obj50:head local-lis/les=113/114 n=251 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:69f21cc4:::obj383:head) v3 2026-02-11T23:31:27.639+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:699d4764:::obj50:head local-lis/les=113/114 n=251 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:69f21cc4:::obj383:head) 2026-02-11T23:31:27.639+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.639+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:69f21cc4:::obj383:head) v3 finish latency 0.000008 2026-02-11T23:31:27.639+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:69f21cc4:::obj383:head local-lis/les=113/114 n=252 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.639+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.640+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.640+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:69f21cc4:::obj383:head local-lis/les=113/114 n=252 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.640+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.640+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.640+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.640+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d1db00 con 0x4acb000 2026-02-11T23:31:27.640+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:6c7994f7:::obj432:head)]) -- 0x5ca1200 con 0x4acb000 2026-02-11T23:31:27.640+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.640+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 541 ==== MOSDPGPush(1.0 116/113 [PushOp(1:6dc7b1fd:::obj49:head, version: 111'49, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6dc7b1fd:::obj49:head@111'49, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x55d8b60 con 0x4acb000 2026-02-11T23:31:27.640+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:6dc7b1fd:::obj49:head, version: 111'49, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6dc7b1fd:::obj49:head@111'49, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.640+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:6dc7b1fd:::obj49:head, version: 111'49, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6dc7b1fd:::obj49:head@111'49, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.640+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.640+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 542 ==== pg_backfill(progress 1.0 e 116/116 lb 1:6c7994f7:::obj432:head) ==== 1062+0+0 (unknown 0 0 0) 0x6016100 con 0x4acb000 2026-02-11T23:31:27.640+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:6dc7b1fd:::obj49:head, version: 111'49, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6dc7b1fd:::obj49:head@111'49, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.640+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:6c7994f7:::obj432:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.640+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:6c7994f7:::obj432:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.640+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.640+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:6dc7b1fd:::obj49:head, version: 111'49, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6dc7b1fd:::obj49:head@111'49, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.640+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.640+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:6dc7b1fd:::obj49:head, version: 111'49, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6dc7b1fd:::obj49:head@111'49, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 116/113 [PushOp(1:6dc7b1fd:::obj49:head, version: 111'49, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6dc7b1fd:::obj49:head@111'49, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:69f21cc4:::obj383:head local-lis/les=113/114 n=252 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.640+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.640+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:69f21cc4:::obj383:head local-lis/les=113/114 n=252 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:6dc7b1fd:::obj49:head, version: 111'49, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6dc7b1fd:::obj49:head@111'49, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.640+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.640+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:6c7994f7:::obj432:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.640+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:69f21cc4:::obj383:head local-lis/les=113/114 n=252 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:6dc7b1fd:::obj49:head@111'49, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.640+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:69f21cc4:::obj383:head local-lis/les=113/114 n=252 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:6dc7b1fd:::obj49:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.640+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:69f21cc4:::obj383:head local-lis/les=113/114 n=252 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:6dc7b1fd:::obj49:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.640+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:69f21cc4:::obj383:head local-lis/les=113/114 n=252 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:6dc7b1fd:::obj49:head 2026-02-11T23:31:27.640+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:69f21cc4:::obj383:head local-lis/les=113/114 n=252 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:6dc7b1fd:::obj49:head v 111'49 2026-02-11T23:31:27.640+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:69f21cc4:::obj383:head local-lis/les=113/114 n=252 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.640+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.640+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:6dc7b1fd:::obj49:head, version: 111'49, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6dc7b1fd:::obj49:head@111'49, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:27.640+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.640+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.640+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:6c7994f7:::obj432:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.640+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:6c7994f7:::obj432:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 1.0 e 116/116 lb 1:6c7994f7:::obj432:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:69f21cc4:::obj383:head local-lis/les=113/114 n=252 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.640+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.640+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:69f21cc4:::obj383:head local-lis/les=113/114 n=252 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:6c7994f7:::obj432:head) v3 2026-02-11T23:31:27.640+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:69f21cc4:::obj383:head local-lis/les=113/114 n=252 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:6c7994f7:::obj432:head) 2026-02-11T23:31:27.640+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.640+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:6c7994f7:::obj432:head) v3 finish latency 0.000011 2026-02-11T23:31:27.640+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6c7994f7:::obj432:head local-lis/les=113/114 n=253 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.640+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.640+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.640+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6c7994f7:::obj432:head local-lis/les=113/114 n=253 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.640+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.640+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.640+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.640+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fed080 con 0x4acb000 2026-02-11T23:31:27.640+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:6dc7b1fd:::obj49:head)]) -- 0x5ca1500 con 0x4acb000 2026-02-11T23:31:27.640+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.641+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 543 ==== MOSDPGPush(1.0 116/113 [PushOp(1:6e0bfbf2:::obj372:head, version: 111'372, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6e0bfbf2:::obj372:head@111'372, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5647040 con 0x4acb000 2026-02-11T23:31:27.641+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:6e0bfbf2:::obj372:head, version: 111'372, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6e0bfbf2:::obj372:head@111'372, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.641+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:6e0bfbf2:::obj372:head, version: 111'372, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6e0bfbf2:::obj372:head@111'372, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.641+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.641+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:6e0bfbf2:::obj372:head, version: 111'372, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6e0bfbf2:::obj372:head@111'372, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.641+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.641+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:6e0bfbf2:::obj372:head, version: 111'372, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6e0bfbf2:::obj372:head@111'372, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.641+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 544 ==== pg_backfill(progress 1.0 e 116/116 lb 1:6dc7b1fd:::obj49:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d2d080 con 0x4acb000 2026-02-11T23:31:27.641+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:6dc7b1fd:::obj49:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.641+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.641+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:6dc7b1fd:::obj49:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.641+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:6e0bfbf2:::obj372:head, version: 111'372, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6e0bfbf2:::obj372:head@111'372, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 116/113 [PushOp(1:6e0bfbf2:::obj372:head, version: 111'372, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6e0bfbf2:::obj372:head@111'372, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:6c7994f7:::obj432:head local-lis/les=113/114 n=253 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.641+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.641+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6c7994f7:::obj432:head local-lis/les=113/114 n=253 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:6e0bfbf2:::obj372:head, version: 111'372, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6e0bfbf2:::obj372:head@111'372, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.641+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.641+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6c7994f7:::obj432:head local-lis/les=113/114 n=253 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:6e0bfbf2:::obj372:head@111'372, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.641+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:6dc7b1fd:::obj49:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.641+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6c7994f7:::obj432:head local-lis/les=113/114 n=253 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:6e0bfbf2:::obj372:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.641+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6c7994f7:::obj432:head local-lis/les=113/114 n=253 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:6e0bfbf2:::obj372:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.641+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6c7994f7:::obj432:head local-lis/les=113/114 n=253 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:6e0bfbf2:::obj372:head 2026-02-11T23:31:27.641+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6c7994f7:::obj432:head local-lis/les=113/114 n=253 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:6e0bfbf2:::obj372:head v 111'372 2026-02-11T23:31:27.641+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6c7994f7:::obj432:head local-lis/les=113/114 n=253 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.641+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.641+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:6e0bfbf2:::obj372:head, version: 111'372, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6e0bfbf2:::obj372:head@111'372, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.641+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.641+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.641+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:6dc7b1fd:::obj49:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.641+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:6dc7b1fd:::obj49:head) v3 prio 15 cost 0 latency 0.000058 pg_backfill(progress 1.0 e 116/116 lb 1:6dc7b1fd:::obj49:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:6c7994f7:::obj432:head local-lis/les=113/114 n=253 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.641+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.641+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6c7994f7:::obj432:head local-lis/les=113/114 n=253 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:6dc7b1fd:::obj49:head) v3 2026-02-11T23:31:27.641+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6c7994f7:::obj432:head local-lis/les=113/114 n=253 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:6dc7b1fd:::obj49:head) 2026-02-11T23:31:27.641+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.641+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:6dc7b1fd:::obj49:head) v3 finish latency 0.000010 2026-02-11T23:31:27.641+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6dc7b1fd:::obj49:head local-lis/les=113/114 n=254 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.641+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.641+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.641+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6dc7b1fd:::obj49:head local-lis/les=113/114 n=254 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.641+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.641+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.641+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.641+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d1d800 con 0x4acb000 2026-02-11T23:31:27.641+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:6e0bfbf2:::obj372:head)]) -- 0x5ca1680 con 0x4acb000 2026-02-11T23:31:27.641+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.641+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 545 ==== MOSDPGPush(1.0 116/113 [PushOp(1:6ec34796:::obj408:head, version: 111'408, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec34796:::obj408:head@111'408, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d81a0 con 0x4acb000 2026-02-11T23:31:27.641+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:6ec34796:::obj408:head, version: 111'408, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec34796:::obj408:head@111'408, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.641+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:6ec34796:::obj408:head, version: 111'408, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec34796:::obj408:head@111'408, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.641+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.641+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 546 ==== pg_backfill(progress 1.0 e 116/116 lb 1:6e0bfbf2:::obj372:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e680 con 0x4acb000 2026-02-11T23:31:27.641+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:6e0bfbf2:::obj372:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.641+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:6ec34796:::obj408:head, version: 111'408, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec34796:::obj408:head@111'408, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.641+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:6e0bfbf2:::obj372:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.641+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.641+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:6ec34796:::obj408:head, version: 111'408, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec34796:::obj408:head@111'408, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.641+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:6ec34796:::obj408:head, version: 111'408, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec34796:::obj408:head@111'408, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:6ec34796:::obj408:head, version: 111'408, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec34796:::obj408:head@111'408, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:6dc7b1fd:::obj49:head local-lis/les=113/114 n=254 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.641+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.641+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.641+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:6e0bfbf2:::obj372:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.641+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6dc7b1fd:::obj49:head local-lis/les=113/114 n=254 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:6ec34796:::obj408:head, version: 111'408, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec34796:::obj408:head@111'408, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.641+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6dc7b1fd:::obj49:head local-lis/les=113/114 n=254 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:6ec34796:::obj408:head@111'408, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.641+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6dc7b1fd:::obj49:head local-lis/les=113/114 n=254 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:6ec34796:::obj408:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.641+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6dc7b1fd:::obj49:head local-lis/les=113/114 n=254 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:6ec34796:::obj408:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.641+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6dc7b1fd:::obj49:head local-lis/les=113/114 n=254 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:6ec34796:::obj408:head 2026-02-11T23:31:27.641+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6dc7b1fd:::obj49:head local-lis/les=113/114 n=254 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:6ec34796:::obj408:head v 111'408 2026-02-11T23:31:27.641+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6dc7b1fd:::obj49:head local-lis/les=113/114 n=254 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.641+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.641+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:6ec34796:::obj408:head, version: 111'408, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec34796:::obj408:head@111'408, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.641+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.641+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.641+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:6e0bfbf2:::obj372:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.641+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:6e0bfbf2:::obj372:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 1.0 e 116/116 lb 1:6e0bfbf2:::obj372:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:6dc7b1fd:::obj49:head local-lis/les=113/114 n=254 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.641+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.641+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6dc7b1fd:::obj49:head local-lis/les=113/114 n=254 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:6e0bfbf2:::obj372:head) v3 2026-02-11T23:31:27.641+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6dc7b1fd:::obj49:head local-lis/les=113/114 n=254 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:6e0bfbf2:::obj372:head) 2026-02-11T23:31:27.641+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.641+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:6e0bfbf2:::obj372:head) v3 finish latency 0.000009 2026-02-11T23:31:27.641+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6e0bfbf2:::obj372:head local-lis/les=113/114 n=255 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.641+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.642+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.642+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6e0bfbf2:::obj372:head local-lis/les=113/114 n=255 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.642+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.642+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.642+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.642+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fecf00 con 0x4acb000 2026-02-11T23:31:27.642+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:6ec34796:::obj408:head)]) -- 0x5ca1980 con 0x4acb000 2026-02-11T23:31:27.642+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.642+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 547 ==== MOSDPGPush(1.0 116/113 [PushOp(1:6ec93c40:::obj366:head, version: 111'366, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec93c40:::obj366:head@111'366, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559d860 con 0x4acb000 2026-02-11T23:31:27.642+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:6ec93c40:::obj366:head, version: 111'366, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec93c40:::obj366:head@111'366, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 116 2026-02-11T23:31:27.642+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:6ec93c40:::obj366:head, version: 111'366, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec93c40:::obj366:head@111'366, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.642+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.642+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 548 ==== pg_backfill(progress 1.0 e 116/116 lb 1:6ec34796:::obj408:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e100 con 0x4acb000 2026-02-11T23:31:27.642+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:6ec34796:::obj408:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.642+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:6ec93c40:::obj366:head, version: 111'366, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec93c40:::obj366:head@111'366, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.642+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:6ec34796:::obj408:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.642+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.642+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:6ec93c40:::obj366:head, version: 111'366, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec93c40:::obj366:head@111'366, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.642+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:6ec93c40:::obj366:head, version: 111'366, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec93c40:::obj366:head@111'366, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(1.0 116/113 [PushOp(1:6ec93c40:::obj366:head, version: 111'366, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec93c40:::obj366:head@111'366, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:6e0bfbf2:::obj372:head local-lis/les=113/114 n=255 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.642+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.642+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.642+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6e0bfbf2:::obj372:head local-lis/les=113/114 n=255 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:6ec93c40:::obj366:head, version: 111'366, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec93c40:::obj366:head@111'366, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.642+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:6ec34796:::obj408:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.642+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6e0bfbf2:::obj372:head local-lis/les=113/114 n=255 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:6ec93c40:::obj366:head@111'366, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.642+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6e0bfbf2:::obj372:head local-lis/les=113/114 n=255 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:6ec93c40:::obj366:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.642+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6e0bfbf2:::obj372:head local-lis/les=113/114 n=255 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:6ec93c40:::obj366:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.642+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6e0bfbf2:::obj372:head local-lis/les=113/114 n=255 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:6ec93c40:::obj366:head 2026-02-11T23:31:27.642+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6e0bfbf2:::obj372:head local-lis/les=113/114 n=255 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:6ec93c40:::obj366:head v 111'366 2026-02-11T23:31:27.642+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6e0bfbf2:::obj372:head local-lis/les=113/114 n=255 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.642+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.642+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:6ec93c40:::obj366:head, version: 111'366, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec93c40:::obj366:head@111'366, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:27.642+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.642+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.642+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:6ec34796:::obj408:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.642+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:6ec34796:::obj408:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 1.0 e 116/116 lb 1:6ec34796:::obj408:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:6e0bfbf2:::obj372:head local-lis/les=113/114 n=255 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.642+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.642+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6e0bfbf2:::obj372:head local-lis/les=113/114 n=255 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:6ec34796:::obj408:head) v3 2026-02-11T23:31:27.642+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6e0bfbf2:::obj372:head local-lis/les=113/114 n=255 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:6ec34796:::obj408:head) 2026-02-11T23:31:27.642+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.642+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:6ec34796:::obj408:head) v3 finish latency 0.000009 2026-02-11T23:31:27.642+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6ec34796:::obj408:head local-lis/les=113/114 n=256 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.642+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.642+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.642+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6ec34796:::obj408:head local-lis/les=113/114 n=256 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.642+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.642+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.642+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.642+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d1d680 con 0x4acb000 2026-02-11T23:31:27.642+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:6ec93c40:::obj366:head)]) -- 0x5ca1c80 con 0x4acb000 2026-02-11T23:31:27.642+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.643+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 549 ==== MOSDPGPush(1.0 116/113 [PushOp(1:6ec9e2eb:::obj591:head, version: 111'591, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec9e2eb:::obj591:head@111'591, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d8680 con 0x4acb000 2026-02-11T23:31:27.643+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:6ec9e2eb:::obj591:head, version: 111'591, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec9e2eb:::obj591:head@111'591, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.643+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:6ec9e2eb:::obj591:head, version: 111'591, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec9e2eb:::obj591:head@111'591, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.643+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.643+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 550 ==== pg_backfill(progress 1.0 e 116/116 lb 1:6ec93c40:::obj366:head) ==== 1062+0+0 (unknown 0 0 0) 0x4a2cb00 con 0x4acb000 2026-02-11T23:31:27.643+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:6ec93c40:::obj366:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.643+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:6ec9e2eb:::obj591:head, version: 111'591, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec9e2eb:::obj591:head@111'591, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.643+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:6ec93c40:::obj366:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.643+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.643+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:6ec9e2eb:::obj591:head, version: 111'591, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec9e2eb:::obj591:head@111'591, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.643+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:6ec9e2eb:::obj591:head, version: 111'591, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec9e2eb:::obj591:head@111'591, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 116/113 [PushOp(1:6ec9e2eb:::obj591:head, version: 111'591, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec9e2eb:::obj591:head@111'591, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:6ec34796:::obj408:head local-lis/les=113/114 n=256 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.643+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.643+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6ec34796:::obj408:head local-lis/les=113/114 n=256 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:6ec9e2eb:::obj591:head, version: 111'591, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec9e2eb:::obj591:head@111'591, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.643+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.643+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:6ec93c40:::obj366:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.643+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6ec34796:::obj408:head local-lis/les=113/114 n=256 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:6ec9e2eb:::obj591:head@111'591, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.643+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6ec34796:::obj408:head local-lis/les=113/114 n=256 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:6ec9e2eb:::obj591:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.643+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6ec34796:::obj408:head local-lis/les=113/114 n=256 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:6ec9e2eb:::obj591:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.643+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6ec34796:::obj408:head local-lis/les=113/114 n=256 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:6ec9e2eb:::obj591:head 2026-02-11T23:31:27.643+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6ec34796:::obj408:head local-lis/les=113/114 n=256 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:6ec9e2eb:::obj591:head v 111'591 2026-02-11T23:31:27.643+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6ec34796:::obj408:head local-lis/les=113/114 n=256 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.643+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.643+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:6ec9e2eb:::obj591:head, version: 111'591, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec9e2eb:::obj591:head@111'591, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:27.643+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.643+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.643+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:6ec93c40:::obj366:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.643+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:6ec93c40:::obj366:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 1.0 e 116/116 lb 1:6ec93c40:::obj366:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:6ec34796:::obj408:head local-lis/les=113/114 n=256 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.643+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.643+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6ec34796:::obj408:head local-lis/les=113/114 n=256 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:6ec93c40:::obj366:head) v3 2026-02-11T23:31:27.643+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6ec34796:::obj408:head local-lis/les=113/114 n=256 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:6ec93c40:::obj366:head) 2026-02-11T23:31:27.643+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.643+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:6ec93c40:::obj366:head) v3 finish latency 0.000008 2026-02-11T23:31:27.643+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6ec93c40:::obj366:head local-lis/les=113/114 n=257 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.643+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.643+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.643+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6ec93c40:::obj366:head local-lis/les=113/114 n=257 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.643+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.643+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.643+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.643+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fecd80 con 0x4acb000 2026-02-11T23:31:27.643+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:6ec9e2eb:::obj591:head)]) -- 0x612e000 con 0x4acb000 2026-02-11T23:31:27.643+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.643+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 551 ==== MOSDPGPush(1.0 116/113 [PushOp(1:6f4176ae:::obj571:head, version: 111'571, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6f4176ae:::obj571:head@111'571, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559dd40 con 0x4acb000 2026-02-11T23:31:27.643+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:6f4176ae:::obj571:head, version: 111'571, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6f4176ae:::obj571:head@111'571, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.643+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:6f4176ae:::obj571:head, version: 111'571, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6f4176ae:::obj571:head@111'571, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.643+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.643+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 552 ==== pg_backfill(progress 1.0 e 116/116 lb 1:6ec9e2eb:::obj591:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9ac00 con 0x4acb000 2026-02-11T23:31:27.643+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:6ec9e2eb:::obj591:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.643+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:6f4176ae:::obj571:head, version: 111'571, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6f4176ae:::obj571:head@111'571, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.643+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.643+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:6ec9e2eb:::obj591:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.643+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:6f4176ae:::obj571:head, version: 111'571, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6f4176ae:::obj571:head@111'571, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.643+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.643+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:6f4176ae:::obj571:head, version: 111'571, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6f4176ae:::obj571:head@111'571, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 116/113 [PushOp(1:6f4176ae:::obj571:head, version: 111'571, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6f4176ae:::obj571:head@111'571, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:6ec93c40:::obj366:head local-lis/les=113/114 n=257 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.643+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.643+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6ec93c40:::obj366:head local-lis/les=113/114 n=257 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:6f4176ae:::obj571:head, version: 111'571, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6f4176ae:::obj571:head@111'571, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.643+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6ec93c40:::obj366:head local-lis/les=113/114 n=257 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:6f4176ae:::obj571:head@111'571, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.643+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.643+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:6ec9e2eb:::obj591:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.643+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6ec93c40:::obj366:head local-lis/les=113/114 n=257 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:6f4176ae:::obj571:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.643+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6ec93c40:::obj366:head local-lis/les=113/114 n=257 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:6f4176ae:::obj571:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.643+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6ec93c40:::obj366:head local-lis/les=113/114 n=257 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:6f4176ae:::obj571:head 2026-02-11T23:31:27.643+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6ec93c40:::obj366:head local-lis/les=113/114 n=257 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:6f4176ae:::obj571:head v 111'571 2026-02-11T23:31:27.643+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6ec93c40:::obj366:head local-lis/les=113/114 n=257 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.643+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.643+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:6f4176ae:::obj571:head, version: 111'571, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6f4176ae:::obj571:head@111'571, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.643+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.643+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.643+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:6ec9e2eb:::obj591:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.643+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:6ec9e2eb:::obj591:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 1.0 e 116/116 lb 1:6ec9e2eb:::obj591:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:6ec93c40:::obj366:head local-lis/les=113/114 n=257 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.643+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.643+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6ec93c40:::obj366:head local-lis/les=113/114 n=257 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:6ec9e2eb:::obj591:head) v3 2026-02-11T23:31:27.643+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6ec93c40:::obj366:head local-lis/les=113/114 n=257 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:6ec9e2eb:::obj591:head) 2026-02-11T23:31:27.643+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.643+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:6ec9e2eb:::obj591:head) v3 finish latency 0.000010 2026-02-11T23:31:27.643+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6ec9e2eb:::obj591:head local-lis/les=113/114 n=258 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.643+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.643+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.643+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6ec9e2eb:::obj591:head local-lis/les=113/114 n=258 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.643+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.643+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.643+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.643+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5ca0d80 con 0x4acb000 2026-02-11T23:31:27.643+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:6f4176ae:::obj571:head)]) -- 0x612e300 con 0x4acb000 2026-02-11T23:31:27.643+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.644+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 553 ==== MOSDPGPush(1.0 116/113 [PushOp(1:6f5862a4:::obj430:head, version: 111'430, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6f5862a4:::obj430:head@111'430, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c820 con 0x4acb000 2026-02-11T23:31:27.644+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:6f5862a4:::obj430:head, version: 111'430, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6f5862a4:::obj430:head@111'430, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.644+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:6f5862a4:::obj430:head, version: 111'430, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6f5862a4:::obj430:head@111'430, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.644+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.644+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 554 ==== pg_backfill(progress 1.0 e 116/116 lb 1:6f4176ae:::obj571:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9b180 con 0x4acb000 2026-02-11T23:31:27.644+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:6f4176ae:::obj571:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.644+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:6f5862a4:::obj430:head, version: 111'430, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6f5862a4:::obj430:head@111'430, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.644+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:6f4176ae:::obj571:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.644+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.644+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:6f5862a4:::obj430:head, version: 111'430, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6f5862a4:::obj430:head@111'430, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.644+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.644+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:6f5862a4:::obj430:head, version: 111'430, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6f5862a4:::obj430:head@111'430, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:6f5862a4:::obj430:head, version: 111'430, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6f5862a4:::obj430:head@111'430, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:6ec9e2eb:::obj591:head local-lis/les=113/114 n=258 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.644+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:6f4176ae:::obj571:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.644+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.644+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6ec9e2eb:::obj591:head local-lis/les=113/114 n=258 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:6f5862a4:::obj430:head, version: 111'430, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6f5862a4:::obj430:head@111'430, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.644+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6ec9e2eb:::obj591:head local-lis/les=113/114 n=258 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:6f5862a4:::obj430:head@111'430, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.644+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6ec9e2eb:::obj591:head local-lis/les=113/114 n=258 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:6f5862a4:::obj430:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.644+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6ec9e2eb:::obj591:head local-lis/les=113/114 n=258 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:6f5862a4:::obj430:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.644+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6ec9e2eb:::obj591:head local-lis/les=113/114 n=258 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:6f5862a4:::obj430:head 2026-02-11T23:31:27.644+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6ec9e2eb:::obj591:head local-lis/les=113/114 n=258 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:6f5862a4:::obj430:head v 111'430 2026-02-11T23:31:27.644+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6ec9e2eb:::obj591:head local-lis/les=113/114 n=258 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.644+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.644+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:6f5862a4:::obj430:head, version: 111'430, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6f5862a4:::obj430:head@111'430, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.644+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.644+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.644+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:6f4176ae:::obj571:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.644+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:6f4176ae:::obj571:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 1.0 e 116/116 lb 1:6f4176ae:::obj571:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:6ec9e2eb:::obj591:head local-lis/les=113/114 n=258 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.644+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.644+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6ec9e2eb:::obj591:head local-lis/les=113/114 n=258 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:6f4176ae:::obj571:head) v3 2026-02-11T23:31:27.644+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6ec9e2eb:::obj591:head local-lis/les=113/114 n=258 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:6f4176ae:::obj571:head) 2026-02-11T23:31:27.644+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.644+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:6f4176ae:::obj571:head) v3 finish latency 0.000009 2026-02-11T23:31:27.644+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6f4176ae:::obj571:head local-lis/les=113/114 n=259 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.644+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.644+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.644+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6f4176ae:::obj571:head local-lis/les=113/114 n=259 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.644+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.644+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.644+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.644+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d1de00 con 0x4acb000 2026-02-11T23:31:27.644+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:6f5862a4:::obj430:head)]) -- 0x612e600 con 0x4acb000 2026-02-11T23:31:27.644+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.644+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 555 ==== MOSDPGPush(1.0 116/113 [PushOp(1:70360d67:::obj288:head, version: 111'288, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:70360d67:::obj288:head@111'288, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f1520 con 0x4acb000 2026-02-11T23:31:27.644+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:70360d67:::obj288:head, version: 111'288, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:70360d67:::obj288:head@111'288, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.644+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:70360d67:::obj288:head, version: 111'288, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:70360d67:::obj288:head@111'288, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.644+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.644+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 556 ==== pg_backfill(progress 1.0 e 116/116 lb 1:6f5862a4:::obj430:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98000 con 0x4acb000 2026-02-11T23:31:27.644+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:70360d67:::obj288:head, version: 111'288, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:70360d67:::obj288:head@111'288, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.644+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:6f5862a4:::obj430:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.644+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:6f5862a4:::obj430:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.644+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.644+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:70360d67:::obj288:head, version: 111'288, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:70360d67:::obj288:head@111'288, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.644+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.644+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:70360d67:::obj288:head, version: 111'288, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:70360d67:::obj288:head@111'288, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 116/113 [PushOp(1:70360d67:::obj288:head, version: 111'288, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:70360d67:::obj288:head@111'288, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:6f4176ae:::obj571:head local-lis/les=113/114 n=259 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.644+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.644+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6f4176ae:::obj571:head local-lis/les=113/114 n=259 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:70360d67:::obj288:head, version: 111'288, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:70360d67:::obj288:head@111'288, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.644+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6f4176ae:::obj571:head local-lis/les=113/114 n=259 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:70360d67:::obj288:head@111'288, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.644+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.644+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:6f5862a4:::obj430:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.644+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6f4176ae:::obj571:head local-lis/les=113/114 n=259 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:70360d67:::obj288:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.644+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6f4176ae:::obj571:head local-lis/les=113/114 n=259 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:70360d67:::obj288:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.644+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6f4176ae:::obj571:head local-lis/les=113/114 n=259 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:70360d67:::obj288:head 2026-02-11T23:31:27.644+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6f4176ae:::obj571:head local-lis/les=113/114 n=259 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:70360d67:::obj288:head v 111'288 2026-02-11T23:31:27.644+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6f4176ae:::obj571:head local-lis/les=113/114 n=259 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.644+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.644+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:70360d67:::obj288:head, version: 111'288, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:70360d67:::obj288:head@111'288, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.644+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.644+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.644+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:6f5862a4:::obj430:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.644+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:6f5862a4:::obj430:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 1.0 e 116/116 lb 1:6f5862a4:::obj430:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:6f4176ae:::obj571:head local-lis/les=113/114 n=259 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.644+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.644+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6f4176ae:::obj571:head local-lis/les=113/114 n=259 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:6f5862a4:::obj430:head) v3 2026-02-11T23:31:27.644+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6f4176ae:::obj571:head local-lis/les=113/114 n=259 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:6f5862a4:::obj430:head) 2026-02-11T23:31:27.644+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.645+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:6f5862a4:::obj430:head) v3 finish latency 0.000013 2026-02-11T23:31:27.645+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6f5862a4:::obj430:head local-lis/les=113/114 n=260 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.645+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.645+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.645+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6f5862a4:::obj430:head local-lis/les=113/114 n=260 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.645+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.645+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.645+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.645+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5ca0a80 con 0x4acb000 2026-02-11T23:31:27.645+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:70360d67:::obj288:head)]) -- 0x612e900 con 0x4acb000 2026-02-11T23:31:27.645+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.645+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 557 ==== MOSDPGPush(1.0 116/113 [PushOp(1:707a62d6:::obj465:head, version: 111'465, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:707a62d6:::obj465:head@111'465, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f09c0 con 0x4acb000 2026-02-11T23:31:27.645+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:707a62d6:::obj465:head, version: 111'465, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:707a62d6:::obj465:head@111'465, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.645+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:707a62d6:::obj465:head, version: 111'465, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:707a62d6:::obj465:head@111'465, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.645+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.645+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 558 ==== pg_backfill(progress 1.0 e 116/116 lb 1:70360d67:::obj288:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98580 con 0x4acb000 2026-02-11T23:31:27.645+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:707a62d6:::obj465:head, version: 111'465, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:707a62d6:::obj465:head@111'465, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.645+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:70360d67:::obj288:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.645+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.645+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:707a62d6:::obj465:head, version: 111'465, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:707a62d6:::obj465:head@111'465, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.645+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:70360d67:::obj288:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.645+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.645+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:707a62d6:::obj465:head, version: 111'465, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:707a62d6:::obj465:head@111'465, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:707a62d6:::obj465:head, version: 111'465, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:707a62d6:::obj465:head@111'465, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:6f5862a4:::obj430:head local-lis/les=113/114 n=260 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.645+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:70360d67:::obj288:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.645+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.645+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6f5862a4:::obj430:head local-lis/les=113/114 n=260 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:707a62d6:::obj465:head, version: 111'465, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:707a62d6:::obj465:head@111'465, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.645+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6f5862a4:::obj430:head local-lis/les=113/114 n=260 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:707a62d6:::obj465:head@111'465, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.645+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6f5862a4:::obj430:head local-lis/les=113/114 n=260 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:707a62d6:::obj465:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.645+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6f5862a4:::obj430:head local-lis/les=113/114 n=260 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:707a62d6:::obj465:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.645+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6f5862a4:::obj430:head local-lis/les=113/114 n=260 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:707a62d6:::obj465:head 2026-02-11T23:31:27.645+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6f5862a4:::obj430:head local-lis/les=113/114 n=260 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:707a62d6:::obj465:head v 111'465 2026-02-11T23:31:27.645+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6f5862a4:::obj430:head local-lis/les=113/114 n=260 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.645+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.645+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:707a62d6:::obj465:head, version: 111'465, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:707a62d6:::obj465:head@111'465, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.645+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6f5862a4:::obj430:head local-lis/les=113/114 n=260 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.645+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.645+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.645+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:70360d67:::obj288:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.645+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:70360d67:::obj288:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 116/116 lb 1:70360d67:::obj288:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:6f5862a4:::obj430:head local-lis/les=113/114 n=260 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.645+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.645+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6f5862a4:::obj430:head local-lis/les=113/114 n=260 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:70360d67:::obj288:head) v3 2026-02-11T23:31:27.645+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:6f5862a4:::obj430:head local-lis/les=113/114 n=260 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:70360d67:::obj288:head) 2026-02-11T23:31:27.645+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.645+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:70360d67:::obj288:head) v3 finish latency 0.000010 2026-02-11T23:31:27.645+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.645+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.645+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:70360d67:::obj288:head local-lis/les=113/114 n=261 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.645+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.645+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.645+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.645+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fed380 con 0x4acb000 2026-02-11T23:31:27.645+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:707a62d6:::obj465:head)]) -- 0x5fed680 con 0x4acb000 2026-02-11T23:31:27.645+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.646+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 559 ==== MOSDPGPush(1.0 116/113 [PushOp(1:7109478f:::obj25:head, version: 111'25, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7109478f:::obj25:head@111'25, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x55f0ea0 con 0x4acb000 2026-02-11T23:31:27.646+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:7109478f:::obj25:head, version: 111'25, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7109478f:::obj25:head@111'25, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.646+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7109478f:::obj25:head, version: 111'25, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7109478f:::obj25:head@111'25, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.646+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.646+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 560 ==== pg_backfill(progress 1.0 e 116/116 lb 1:707a62d6:::obj465:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98b00 con 0x4acb000 2026-02-11T23:31:27.646+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7109478f:::obj25:head, version: 111'25, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7109478f:::obj25:head@111'25, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.646+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:707a62d6:::obj465:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.646+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:707a62d6:::obj465:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.646+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.646+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7109478f:::obj25:head, version: 111'25, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7109478f:::obj25:head@111'25, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.646+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.646+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:7109478f:::obj25:head, version: 111'25, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7109478f:::obj25:head@111'25, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 116/113 [PushOp(1:7109478f:::obj25:head, version: 111'25, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7109478f:::obj25:head@111'25, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:70360d67:::obj288:head local-lis/les=113/114 n=261 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.646+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.646+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:70360d67:::obj288:head local-lis/les=113/114 n=261 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:7109478f:::obj25:head, version: 111'25, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7109478f:::obj25:head@111'25, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.646+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:70360d67:::obj288:head local-lis/les=113/114 n=261 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:7109478f:::obj25:head@111'25, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.646+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.646+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:707a62d6:::obj465:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.646+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:70360d67:::obj288:head local-lis/les=113/114 n=261 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:7109478f:::obj25:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.646+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:70360d67:::obj288:head local-lis/les=113/114 n=261 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:7109478f:::obj25:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.646+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:70360d67:::obj288:head local-lis/les=113/114 n=261 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:7109478f:::obj25:head 2026-02-11T23:31:27.646+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:70360d67:::obj288:head local-lis/les=113/114 n=261 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:7109478f:::obj25:head v 111'25 2026-02-11T23:31:27.646+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:70360d67:::obj288:head local-lis/les=113/114 n=261 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.646+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.646+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:7109478f:::obj25:head, version: 111'25, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7109478f:::obj25:head@111'25, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.646+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.646+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.646+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:707a62d6:::obj465:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.646+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:707a62d6:::obj465:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:707a62d6:::obj465:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:70360d67:::obj288:head local-lis/les=113/114 n=261 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.646+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.646+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:70360d67:::obj288:head local-lis/les=113/114 n=261 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:707a62d6:::obj465:head) v3 2026-02-11T23:31:27.646+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:70360d67:::obj288:head local-lis/les=113/114 n=261 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:707a62d6:::obj465:head) 2026-02-11T23:31:27.646+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.646+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:707a62d6:::obj465:head) v3 finish latency 0.000010 2026-02-11T23:31:27.646+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:707a62d6:::obj465:head local-lis/les=113/114 n=262 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.646+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.646+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.646+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:707a62d6:::obj465:head local-lis/les=113/114 n=262 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.646+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.646+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.646+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.646+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5ca0480 con 0x4acb000 2026-02-11T23:31:27.646+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:7109478f:::obj25:head)]) -- 0x612ec00 con 0x4acb000 2026-02-11T23:31:27.646+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.646+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 561 ==== MOSDPGPush(1.0 116/113 [PushOp(1:7135fd38:::obj57:head, version: 111'57, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7135fd38:::obj57:head@111'57, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x559c1a0 con 0x4acb000 2026-02-11T23:31:27.646+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:7135fd38:::obj57:head, version: 111'57, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7135fd38:::obj57:head@111'57, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.646+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7135fd38:::obj57:head, version: 111'57, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7135fd38:::obj57:head@111'57, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.646+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.646+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 562 ==== pg_backfill(progress 1.0 e 116/116 lb 1:7109478f:::obj25:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d99080 con 0x4acb000 2026-02-11T23:31:27.646+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:7109478f:::obj25:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.646+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7135fd38:::obj57:head, version: 111'57, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7135fd38:::obj57:head@111'57, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.646+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7109478f:::obj25:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.646+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.646+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7135fd38:::obj57:head, version: 111'57, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7135fd38:::obj57:head@111'57, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.646+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.646+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7109478f:::obj25:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.646+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:7135fd38:::obj57:head, version: 111'57, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7135fd38:::obj57:head@111'57, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 116/113 [PushOp(1:7135fd38:::obj57:head, version: 111'57, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7135fd38:::obj57:head@111'57, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:707a62d6:::obj465:head local-lis/les=113/114 n=262 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.646+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.646+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:707a62d6:::obj465:head local-lis/les=113/114 n=262 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:7135fd38:::obj57:head, version: 111'57, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7135fd38:::obj57:head@111'57, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.646+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:707a62d6:::obj465:head local-lis/les=113/114 n=262 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:7135fd38:::obj57:head@111'57, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.646+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:707a62d6:::obj465:head local-lis/les=113/114 n=262 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:7135fd38:::obj57:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.646+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:707a62d6:::obj465:head local-lis/les=113/114 n=262 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:7135fd38:::obj57:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.646+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:707a62d6:::obj465:head local-lis/les=113/114 n=262 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:7135fd38:::obj57:head 2026-02-11T23:31:27.646+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:707a62d6:::obj465:head local-lis/les=113/114 n=262 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:7135fd38:::obj57:head v 111'57 2026-02-11T23:31:27.646+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:707a62d6:::obj465:head local-lis/les=113/114 n=262 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.646+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.646+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:7135fd38:::obj57:head, version: 111'57, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7135fd38:::obj57:head@111'57, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:27.646+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.646+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.646+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7109478f:::obj25:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.646+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:7109478f:::obj25:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 1.0 e 116/116 lb 1:7109478f:::obj25:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:707a62d6:::obj465:head local-lis/les=113/114 n=262 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.646+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.646+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:707a62d6:::obj465:head local-lis/les=113/114 n=262 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:7109478f:::obj25:head) v3 2026-02-11T23:31:27.646+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:707a62d6:::obj465:head local-lis/les=113/114 n=262 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:7109478f:::obj25:head) 2026-02-11T23:31:27.646+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.646+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:7109478f:::obj25:head) v3 finish latency 0.000009 2026-02-11T23:31:27.646+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7109478f:::obj25:head local-lis/les=113/114 n=263 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.646+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.646+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.646+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7109478f:::obj25:head local-lis/les=113/114 n=263 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.646+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.646+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.646+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.646+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fed800 con 0x4acb000 2026-02-11T23:31:27.646+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:7135fd38:::obj57:head)]) -- 0x612ed80 con 0x4acb000 2026-02-11T23:31:27.646+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.647+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 563 ==== MOSDPGPush(1.0 116/113 [PushOp(1:72da0e1a:::obj147:head, version: 111'147, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:72da0e1a:::obj147:head@111'147, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c9c0 con 0x4acb000 2026-02-11T23:31:27.647+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:72da0e1a:::obj147:head, version: 111'147, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:72da0e1a:::obj147:head@111'147, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.647+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:72da0e1a:::obj147:head, version: 111'147, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:72da0e1a:::obj147:head@111'147, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.647+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.647+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 564 ==== pg_backfill(progress 1.0 e 116/116 lb 1:7135fd38:::obj57:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d99600 con 0x4acb000 2026-02-11T23:31:27.647+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:7135fd38:::obj57:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.647+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:72da0e1a:::obj147:head, version: 111'147, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:72da0e1a:::obj147:head@111'147, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.647+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7135fd38:::obj57:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.647+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.647+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:72da0e1a:::obj147:head, version: 111'147, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:72da0e1a:::obj147:head@111'147, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.647+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.647+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:72da0e1a:::obj147:head, version: 111'147, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:72da0e1a:::obj147:head@111'147, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 116/113 [PushOp(1:72da0e1a:::obj147:head, version: 111'147, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:72da0e1a:::obj147:head@111'147, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:7109478f:::obj25:head local-lis/les=113/114 n=263 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.647+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.647+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7109478f:::obj25:head local-lis/les=113/114 n=263 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:72da0e1a:::obj147:head, version: 111'147, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:72da0e1a:::obj147:head@111'147, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.647+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7109478f:::obj25:head local-lis/les=113/114 n=263 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:72da0e1a:::obj147:head@111'147, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.647+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.647+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7135fd38:::obj57:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.647+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7109478f:::obj25:head local-lis/les=113/114 n=263 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:72da0e1a:::obj147:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.647+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7109478f:::obj25:head local-lis/les=113/114 n=263 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:72da0e1a:::obj147:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.647+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7109478f:::obj25:head local-lis/les=113/114 n=263 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:72da0e1a:::obj147:head 2026-02-11T23:31:27.647+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7109478f:::obj25:head local-lis/les=113/114 n=263 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:72da0e1a:::obj147:head v 111'147 2026-02-11T23:31:27.647+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7109478f:::obj25:head local-lis/les=113/114 n=263 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.647+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.647+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:72da0e1a:::obj147:head, version: 111'147, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:72da0e1a:::obj147:head@111'147, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.647+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.647+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.647+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7135fd38:::obj57:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.647+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:7135fd38:::obj57:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 1.0 e 116/116 lb 1:7135fd38:::obj57:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:7109478f:::obj25:head local-lis/les=113/114 n=263 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.647+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.647+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7109478f:::obj25:head local-lis/les=113/114 n=263 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:7135fd38:::obj57:head) v3 2026-02-11T23:31:27.647+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7109478f:::obj25:head local-lis/les=113/114 n=263 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:7135fd38:::obj57:head) 2026-02-11T23:31:27.647+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.647+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:7135fd38:::obj57:head) v3 finish latency 0.000009 2026-02-11T23:31:27.647+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7135fd38:::obj57:head local-lis/les=113/114 n=264 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.647+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.647+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.647+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7135fd38:::obj57:head local-lis/les=113/114 n=264 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.647+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.647+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.647+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.647+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5ca0180 con 0x4acb000 2026-02-11T23:31:27.647+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:72da0e1a:::obj147:head)]) -- 0x612ef00 con 0x4acb000 2026-02-11T23:31:27.647+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.647+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 565 ==== MOSDPGPush(1.0 116/113 [PushOp(1:73627f05:::obj80:head, version: 111'80, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:73627f05:::obj80:head@111'80, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x55f1a00 con 0x4acb000 2026-02-11T23:31:27.647+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:73627f05:::obj80:head, version: 111'80, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:73627f05:::obj80:head@111'80, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.647+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:73627f05:::obj80:head, version: 111'80, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:73627f05:::obj80:head@111'80, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.647+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.647+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 566 ==== pg_backfill(progress 1.0 e 116/116 lb 1:72da0e1a:::obj147:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99b80 con 0x4acb000 2026-02-11T23:31:27.647+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:72da0e1a:::obj147:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.647+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:72da0e1a:::obj147:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.647+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:73627f05:::obj80:head, version: 111'80, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:73627f05:::obj80:head@111'80, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.647+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.647+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:73627f05:::obj80:head, version: 111'80, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:73627f05:::obj80:head@111'80, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.647+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:73627f05:::obj80:head, version: 111'80, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:73627f05:::obj80:head@111'80, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 116/113 [PushOp(1:73627f05:::obj80:head, version: 111'80, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:73627f05:::obj80:head@111'80, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:7135fd38:::obj57:head local-lis/les=113/114 n=264 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.647+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.648+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7135fd38:::obj57:head local-lis/les=113/114 n=264 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:73627f05:::obj80:head, version: 111'80, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:73627f05:::obj80:head@111'80, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.648+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.648+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:72da0e1a:::obj147:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.648+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7135fd38:::obj57:head local-lis/les=113/114 n=264 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:73627f05:::obj80:head@111'80, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.648+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7135fd38:::obj57:head local-lis/les=113/114 n=264 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:73627f05:::obj80:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.648+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7135fd38:::obj57:head local-lis/les=113/114 n=264 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:73627f05:::obj80:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.648+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7135fd38:::obj57:head local-lis/les=113/114 n=264 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:73627f05:::obj80:head 2026-02-11T23:31:27.648+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7135fd38:::obj57:head local-lis/les=113/114 n=264 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:73627f05:::obj80:head v 111'80 2026-02-11T23:31:27.648+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7135fd38:::obj57:head local-lis/les=113/114 n=264 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.648+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.648+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:73627f05:::obj80:head, version: 111'80, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:73627f05:::obj80:head@111'80, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:27.648+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.648+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.648+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:72da0e1a:::obj147:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.648+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:72da0e1a:::obj147:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 1.0 e 116/116 lb 1:72da0e1a:::obj147:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:7135fd38:::obj57:head local-lis/les=113/114 n=264 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.648+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.648+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7135fd38:::obj57:head local-lis/les=113/114 n=264 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:72da0e1a:::obj147:head) v3 2026-02-11T23:31:27.648+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7135fd38:::obj57:head local-lis/les=113/114 n=264 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:72da0e1a:::obj147:head) 2026-02-11T23:31:27.648+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.648+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:72da0e1a:::obj147:head) v3 finish latency 0.000009 2026-02-11T23:31:27.648+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:72da0e1a:::obj147:head local-lis/les=113/114 n=265 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.648+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.648+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.648+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:72da0e1a:::obj147:head local-lis/les=113/114 n=265 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.648+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.648+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.648+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.648+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fce180 con 0x4acb000 2026-02-11T23:31:27.648+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:73627f05:::obj80:head)]) -- 0x612f200 con 0x4acb000 2026-02-11T23:31:27.648+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.648+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 567 ==== MOSDPGPush(1.0 116/113 [PushOp(1:737833f6:::obj391:head, version: 111'391, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:737833f6:::obj391:head@111'391, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559d380 con 0x4acb000 2026-02-11T23:31:27.648+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:737833f6:::obj391:head, version: 111'391, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:737833f6:::obj391:head@111'391, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.648+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:737833f6:::obj391:head, version: 111'391, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:737833f6:::obj391:head@111'391, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.648+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.648+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 568 ==== pg_backfill(progress 1.0 e 116/116 lb 1:73627f05:::obj80:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d9a100 con 0x4acb000 2026-02-11T23:31:27.648+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:737833f6:::obj391:head, version: 111'391, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:737833f6:::obj391:head@111'391, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.648+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:73627f05:::obj80:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.648+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:73627f05:::obj80:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.648+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.648+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:737833f6:::obj391:head, version: 111'391, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:737833f6:::obj391:head@111'391, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.648+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.648+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:737833f6:::obj391:head, version: 111'391, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:737833f6:::obj391:head@111'391, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 116/113 [PushOp(1:737833f6:::obj391:head, version: 111'391, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:737833f6:::obj391:head@111'391, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:72da0e1a:::obj147:head local-lis/les=113/114 n=265 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.648+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.648+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:72da0e1a:::obj147:head local-lis/les=113/114 n=265 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:737833f6:::obj391:head, version: 111'391, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:737833f6:::obj391:head@111'391, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.648+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:72da0e1a:::obj147:head local-lis/les=113/114 n=265 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:737833f6:::obj391:head@111'391, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.648+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.648+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:72da0e1a:::obj147:head local-lis/les=113/114 n=265 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:737833f6:::obj391:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.648+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:73627f05:::obj80:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.648+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:72da0e1a:::obj147:head local-lis/les=113/114 n=265 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:737833f6:::obj391:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.648+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:72da0e1a:::obj147:head local-lis/les=113/114 n=265 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:737833f6:::obj391:head 2026-02-11T23:31:27.648+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:72da0e1a:::obj147:head local-lis/les=113/114 n=265 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:737833f6:::obj391:head v 111'391 2026-02-11T23:31:27.648+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:72da0e1a:::obj147:head local-lis/les=113/114 n=265 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.648+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.648+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:737833f6:::obj391:head, version: 111'391, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:737833f6:::obj391:head@111'391, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000043 2026-02-11T23:31:27.648+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.648+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.648+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:73627f05:::obj80:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.648+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:73627f05:::obj80:head) v3 prio 15 cost 0 latency 0.000059 pg_backfill(progress 1.0 e 116/116 lb 1:73627f05:::obj80:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:72da0e1a:::obj147:head local-lis/les=113/114 n=265 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.648+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.648+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:72da0e1a:::obj147:head local-lis/les=113/114 n=265 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:73627f05:::obj80:head) v3 2026-02-11T23:31:27.648+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:72da0e1a:::obj147:head local-lis/les=113/114 n=265 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:73627f05:::obj80:head) 2026-02-11T23:31:27.648+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.648+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:73627f05:::obj80:head) v3 finish latency 0.000009 2026-02-11T23:31:27.648+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:73627f05:::obj80:head local-lis/les=113/114 n=266 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.648+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.648+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.648+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:73627f05:::obj80:head local-lis/les=113/114 n=266 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.648+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.648+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.648+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.648+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6005e00 con 0x4acb000 2026-02-11T23:31:27.648+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:737833f6:::obj391:head)]) -- 0x612f380 con 0x4acb000 2026-02-11T23:31:27.648+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.649+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 569 ==== MOSDPGPush(1.0 116/113 [PushOp(1:744a8d59:::obj243:head, version: 111'243, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:744a8d59:::obj243:head@111'243, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c4e0 con 0x4acb000 2026-02-11T23:31:27.649+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:744a8d59:::obj243:head, version: 111'243, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:744a8d59:::obj243:head@111'243, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.649+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:744a8d59:::obj243:head, version: 111'243, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:744a8d59:::obj243:head@111'243, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.649+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.649+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 570 ==== pg_backfill(progress 1.0 e 116/116 lb 1:737833f6:::obj391:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9a680 con 0x4acb000 2026-02-11T23:31:27.649+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:737833f6:::obj391:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.649+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:744a8d59:::obj243:head, version: 111'243, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:744a8d59:::obj243:head@111'243, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.649+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:737833f6:::obj391:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.649+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.649+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:744a8d59:::obj243:head, version: 111'243, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:744a8d59:::obj243:head@111'243, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.649+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.649+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:744a8d59:::obj243:head, version: 111'243, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:744a8d59:::obj243:head@111'243, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 116/113 [PushOp(1:744a8d59:::obj243:head, version: 111'243, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:744a8d59:::obj243:head@111'243, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:73627f05:::obj80:head local-lis/les=113/114 n=266 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.649+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:737833f6:::obj391:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.649+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.649+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:73627f05:::obj80:head local-lis/les=113/114 n=266 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:744a8d59:::obj243:head, version: 111'243, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:744a8d59:::obj243:head@111'243, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.649+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:73627f05:::obj80:head local-lis/les=113/114 n=266 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:744a8d59:::obj243:head@111'243, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.649+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:73627f05:::obj80:head local-lis/les=113/114 n=266 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:744a8d59:::obj243:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.649+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:73627f05:::obj80:head local-lis/les=113/114 n=266 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:744a8d59:::obj243:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.649+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:73627f05:::obj80:head local-lis/les=113/114 n=266 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:744a8d59:::obj243:head 2026-02-11T23:31:27.649+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:73627f05:::obj80:head local-lis/les=113/114 n=266 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:744a8d59:::obj243:head v 111'243 2026-02-11T23:31:27.649+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:73627f05:::obj80:head local-lis/les=113/114 n=266 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.649+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.649+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:744a8d59:::obj243:head, version: 111'243, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:744a8d59:::obj243:head@111'243, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.649+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.649+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.649+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:737833f6:::obj391:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.649+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:737833f6:::obj391:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 116/116 lb 1:737833f6:::obj391:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:73627f05:::obj80:head local-lis/les=113/114 n=266 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.649+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.649+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:73627f05:::obj80:head local-lis/les=113/114 n=266 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:737833f6:::obj391:head) v3 2026-02-11T23:31:27.649+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:73627f05:::obj80:head local-lis/les=113/114 n=266 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:737833f6:::obj391:head) 2026-02-11T23:31:27.649+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.649+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:737833f6:::obj391:head) v3 finish latency 0.000009 2026-02-11T23:31:27.649+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:737833f6:::obj391:head local-lis/les=113/114 n=267 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.649+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.649+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.649+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:737833f6:::obj391:head local-lis/les=113/114 n=267 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.649+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.649+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.649+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.649+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fed980 con 0x4acb000 2026-02-11T23:31:27.649+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:744a8d59:::obj243:head)]) -- 0x612f680 con 0x4acb000 2026-02-11T23:31:27.649+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.649+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 571 ==== MOSDPGPush(1.0 116/113 [PushOp(1:748dbcc6:::obj428:head, version: 111'428, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:748dbcc6:::obj428:head@111'428, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559da00 con 0x4acb000 2026-02-11T23:31:27.649+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:748dbcc6:::obj428:head, version: 111'428, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:748dbcc6:::obj428:head@111'428, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.649+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:748dbcc6:::obj428:head, version: 111'428, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:748dbcc6:::obj428:head@111'428, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.649+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.649+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 572 ==== pg_backfill(progress 1.0 e 116/116 lb 1:744a8d59:::obj243:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9ac00 con 0x4acb000 2026-02-11T23:31:27.649+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:748dbcc6:::obj428:head, version: 111'428, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:748dbcc6:::obj428:head@111'428, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.649+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:744a8d59:::obj243:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.649+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:744a8d59:::obj243:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.649+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.649+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:748dbcc6:::obj428:head, version: 111'428, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:748dbcc6:::obj428:head@111'428, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.649+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.649+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:748dbcc6:::obj428:head, version: 111'428, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:748dbcc6:::obj428:head@111'428, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 116/113 [PushOp(1:748dbcc6:::obj428:head, version: 111'428, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:748dbcc6:::obj428:head@111'428, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:737833f6:::obj391:head local-lis/les=113/114 n=267 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.649+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.649+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:737833f6:::obj391:head local-lis/les=113/114 n=267 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:748dbcc6:::obj428:head, version: 111'428, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:748dbcc6:::obj428:head@111'428, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.649+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:737833f6:::obj391:head local-lis/les=113/114 n=267 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:748dbcc6:::obj428:head@111'428, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.649+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.649+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:744a8d59:::obj243:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.649+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:737833f6:::obj391:head local-lis/les=113/114 n=267 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:748dbcc6:::obj428:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.649+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:737833f6:::obj391:head local-lis/les=113/114 n=267 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:748dbcc6:::obj428:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.649+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:737833f6:::obj391:head local-lis/les=113/114 n=267 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:748dbcc6:::obj428:head 2026-02-11T23:31:27.649+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:737833f6:::obj391:head local-lis/les=113/114 n=267 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:748dbcc6:::obj428:head v 111'428 2026-02-11T23:31:27.649+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:737833f6:::obj391:head local-lis/les=113/114 n=267 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.649+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.649+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:748dbcc6:::obj428:head, version: 111'428, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:748dbcc6:::obj428:head@111'428, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000044 2026-02-11T23:31:27.649+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.649+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.649+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:744a8d59:::obj243:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.649+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:744a8d59:::obj243:head) v3 prio 15 cost 0 latency 0.000060 pg_backfill(progress 1.0 e 116/116 lb 1:744a8d59:::obj243:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:737833f6:::obj391:head local-lis/les=113/114 n=267 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.649+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.649+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:737833f6:::obj391:head local-lis/les=113/114 n=267 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:744a8d59:::obj243:head) v3 2026-02-11T23:31:27.649+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:737833f6:::obj391:head local-lis/les=113/114 n=267 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:744a8d59:::obj243:head) 2026-02-11T23:31:27.649+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.649+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:744a8d59:::obj243:head) v3 finish latency 0.000009 2026-02-11T23:31:27.649+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:744a8d59:::obj243:head local-lis/les=113/114 n=268 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.649+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.649+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.649+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:744a8d59:::obj243:head local-lis/les=113/114 n=268 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.649+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.649+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.649+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.649+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6005b00 con 0x4acb000 2026-02-11T23:31:27.649+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:748dbcc6:::obj428:head)]) -- 0x612f980 con 0x4acb000 2026-02-11T23:31:27.649+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.650+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 573 ==== MOSDPGPush(1.0 116/113 [PushOp(1:7492d19a:::obj168:head, version: 111'168, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7492d19a:::obj168:head@111'168, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559cea0 con 0x4acb000 2026-02-11T23:31:27.650+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:7492d19a:::obj168:head, version: 111'168, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7492d19a:::obj168:head@111'168, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.650+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7492d19a:::obj168:head, version: 111'168, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7492d19a:::obj168:head@111'168, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.650+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.650+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 574 ==== pg_backfill(progress 1.0 e 116/116 lb 1:748dbcc6:::obj428:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9b180 con 0x4acb000 2026-02-11T23:31:27.650+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7492d19a:::obj168:head, version: 111'168, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7492d19a:::obj168:head@111'168, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.650+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:748dbcc6:::obj428:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.650+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.650+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:748dbcc6:::obj428:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.650+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7492d19a:::obj168:head, version: 111'168, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7492d19a:::obj168:head@111'168, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.650+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.650+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:7492d19a:::obj168:head, version: 111'168, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7492d19a:::obj168:head@111'168, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:7492d19a:::obj168:head, version: 111'168, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7492d19a:::obj168:head@111'168, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:744a8d59:::obj243:head local-lis/les=113/114 n=268 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.650+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.650+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:744a8d59:::obj243:head local-lis/les=113/114 n=268 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:7492d19a:::obj168:head, version: 111'168, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7492d19a:::obj168:head@111'168, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.650+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:744a8d59:::obj243:head local-lis/les=113/114 n=268 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:7492d19a:::obj168:head@111'168, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.650+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.650+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:748dbcc6:::obj428:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.650+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:744a8d59:::obj243:head local-lis/les=113/114 n=268 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:7492d19a:::obj168:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.650+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:744a8d59:::obj243:head local-lis/les=113/114 n=268 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:7492d19a:::obj168:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.650+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:744a8d59:::obj243:head local-lis/les=113/114 n=268 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:7492d19a:::obj168:head 2026-02-11T23:31:27.650+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:744a8d59:::obj243:head local-lis/les=113/114 n=268 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:7492d19a:::obj168:head v 111'168 2026-02-11T23:31:27.650+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:744a8d59:::obj243:head local-lis/les=113/114 n=268 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.650+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.650+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:7492d19a:::obj168:head, version: 111'168, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7492d19a:::obj168:head@111'168, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.650+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.650+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.650+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:748dbcc6:::obj428:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.650+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:748dbcc6:::obj428:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 1.0 e 116/116 lb 1:748dbcc6:::obj428:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:744a8d59:::obj243:head local-lis/les=113/114 n=268 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.650+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.650+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:744a8d59:::obj243:head local-lis/les=113/114 n=268 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:748dbcc6:::obj428:head) v3 2026-02-11T23:31:27.650+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:744a8d59:::obj243:head local-lis/les=113/114 n=268 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:748dbcc6:::obj428:head) 2026-02-11T23:31:27.650+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.650+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:748dbcc6:::obj428:head) v3 finish latency 0.000010 2026-02-11T23:31:27.650+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:748dbcc6:::obj428:head local-lis/les=113/114 n=269 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.650+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.650+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.650+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:748dbcc6:::obj428:head local-lis/les=113/114 n=269 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.650+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.650+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.650+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.650+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fce480 con 0x4acb000 2026-02-11T23:31:27.650+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:7492d19a:::obj168:head)]) -- 0x612fc80 con 0x4acb000 2026-02-11T23:31:27.650+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.651+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 575 ==== MOSDPGPush(1.0 116/113 [PushOp(1:750d67aa:::obj572:head, version: 111'572, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:750d67aa:::obj572:head@111'572, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d8820 con 0x4acb000 2026-02-11T23:31:27.651+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:750d67aa:::obj572:head, version: 111'572, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:750d67aa:::obj572:head@111'572, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000020 epoch 116 2026-02-11T23:31:27.651+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:750d67aa:::obj572:head, version: 111'572, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:750d67aa:::obj572:head@111'572, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.651+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.651+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:750d67aa:::obj572:head, version: 111'572, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:750d67aa:::obj572:head@111'572, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.651+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.651+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 576 ==== pg_backfill(progress 1.0 e 116/116 lb 1:7492d19a:::obj168:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98000 con 0x4acb000 2026-02-11T23:31:27.651+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:750d67aa:::obj572:head, version: 111'572, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:750d67aa:::obj572:head@111'572, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.651+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.651+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:7492d19a:::obj168:head) v3 prio 15 type 95 cost 0 latency 0.000010 epoch 116 2026-02-11T23:31:27.651+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:750d67aa:::obj572:head, version: 111'572, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:750d67aa:::obj572:head@111'572, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000049 MOSDPGPush(1.0 116/113 [PushOp(1:750d67aa:::obj572:head, version: 111'572, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:750d67aa:::obj572:head@111'572, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:748dbcc6:::obj428:head local-lis/les=113/114 n=269 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.651+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.651+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7492d19a:::obj168:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.651+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:748dbcc6:::obj428:head local-lis/les=113/114 n=269 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:750d67aa:::obj572:head, version: 111'572, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:750d67aa:::obj572:head@111'572, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.651+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:748dbcc6:::obj428:head local-lis/les=113/114 n=269 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:750d67aa:::obj572:head@111'572, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.651+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.651+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7492d19a:::obj168:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.651+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:748dbcc6:::obj428:head local-lis/les=113/114 n=269 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:750d67aa:::obj572:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.651+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:748dbcc6:::obj428:head local-lis/les=113/114 n=269 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:750d67aa:::obj572:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.651+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:748dbcc6:::obj428:head local-lis/les=113/114 n=269 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:750d67aa:::obj572:head 2026-02-11T23:31:27.651+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:748dbcc6:::obj428:head local-lis/les=113/114 n=269 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:750d67aa:::obj572:head v 111'572 2026-02-11T23:31:27.651+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:748dbcc6:::obj428:head local-lis/les=113/114 n=269 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.651+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.651+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:750d67aa:::obj572:head, version: 111'572, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:750d67aa:::obj572:head@111'572, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:27.651+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.651+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.651+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7492d19a:::obj168:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.651+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:7492d19a:::obj168:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 1.0 e 116/116 lb 1:7492d19a:::obj168:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:748dbcc6:::obj428:head local-lis/les=113/114 n=269 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.651+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.651+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:748dbcc6:::obj428:head local-lis/les=113/114 n=269 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:7492d19a:::obj168:head) v3 2026-02-11T23:31:27.651+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:748dbcc6:::obj428:head local-lis/les=113/114 n=269 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:7492d19a:::obj168:head) 2026-02-11T23:31:27.651+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.651+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:7492d19a:::obj168:head) v3 finish latency 0.000008 2026-02-11T23:31:27.651+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7492d19a:::obj168:head local-lis/les=113/114 n=270 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.651+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.651+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.651+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7492d19a:::obj168:head local-lis/les=113/114 n=270 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.651+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.651+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.651+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.651+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6005800 con 0x4acb000 2026-02-11T23:31:27.651+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:750d67aa:::obj572:head)]) -- 0x6014000 con 0x4acb000 2026-02-11T23:31:27.651+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.651+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 577 ==== MOSDPGPush(1.0 116/113 [PushOp(1:76c71d4e:::obj92:head, version: 111'92, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:76c71d4e:::obj92:head@111'92, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x55d9860 con 0x4acb000 2026-02-11T23:31:27.651+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:76c71d4e:::obj92:head, version: 111'92, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:76c71d4e:::obj92:head@111'92, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.651+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:76c71d4e:::obj92:head, version: 111'92, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:76c71d4e:::obj92:head@111'92, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.651+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.651+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 578 ==== pg_backfill(progress 1.0 e 116/116 lb 1:750d67aa:::obj572:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98580 con 0x4acb000 2026-02-11T23:31:27.651+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:76c71d4e:::obj92:head, version: 111'92, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:76c71d4e:::obj92:head@111'92, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.651+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:750d67aa:::obj572:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.651+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:750d67aa:::obj572:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.651+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.651+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:76c71d4e:::obj92:head, version: 111'92, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:76c71d4e:::obj92:head@111'92, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.651+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.651+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:76c71d4e:::obj92:head, version: 111'92, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:76c71d4e:::obj92:head@111'92, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:76c71d4e:::obj92:head, version: 111'92, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:76c71d4e:::obj92:head@111'92, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:7492d19a:::obj168:head local-lis/les=113/114 n=270 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.651+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.651+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7492d19a:::obj168:head local-lis/les=113/114 n=270 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:76c71d4e:::obj92:head, version: 111'92, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:76c71d4e:::obj92:head@111'92, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.651+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7492d19a:::obj168:head local-lis/les=113/114 n=270 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:76c71d4e:::obj92:head@111'92, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.651+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.651+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:750d67aa:::obj572:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.651+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7492d19a:::obj168:head local-lis/les=113/114 n=270 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:76c71d4e:::obj92:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.651+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7492d19a:::obj168:head local-lis/les=113/114 n=270 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:76c71d4e:::obj92:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.651+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7492d19a:::obj168:head local-lis/les=113/114 n=270 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:76c71d4e:::obj92:head 2026-02-11T23:31:27.651+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7492d19a:::obj168:head local-lis/les=113/114 n=270 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:76c71d4e:::obj92:head v 111'92 2026-02-11T23:31:27.651+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7492d19a:::obj168:head local-lis/les=113/114 n=270 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.651+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.651+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:76c71d4e:::obj92:head, version: 111'92, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:76c71d4e:::obj92:head@111'92, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.651+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.651+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.651+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:750d67aa:::obj572:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.651+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:750d67aa:::obj572:head) v3 prio 15 cost 0 latency 0.000060 pg_backfill(progress 1.0 e 116/116 lb 1:750d67aa:::obj572:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:7492d19a:::obj168:head local-lis/les=113/114 n=270 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.651+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.651+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7492d19a:::obj168:head local-lis/les=113/114 n=270 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:750d67aa:::obj572:head) v3 2026-02-11T23:31:27.651+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7492d19a:::obj168:head local-lis/les=113/114 n=270 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:750d67aa:::obj572:head) 2026-02-11T23:31:27.651+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.651+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:750d67aa:::obj572:head) v3 finish latency 0.000009 2026-02-11T23:31:27.651+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:750d67aa:::obj572:head local-lis/les=113/114 n=271 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.651+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.651+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.651+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:750d67aa:::obj572:head local-lis/les=113/114 n=271 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.651+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.651+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.651+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.651+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fedb00 con 0x4acb000 2026-02-11T23:31:27.651+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:76c71d4e:::obj92:head)]) -- 0x6014300 con 0x4acb000 2026-02-11T23:31:27.651+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.652+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 579 ==== MOSDPGPush(1.0 116/113 [PushOp(1:76f9746b:::obj110:head, version: 111'110, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:76f9746b:::obj110:head@111'110, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d8340 con 0x4acb000 2026-02-11T23:31:27.652+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:76f9746b:::obj110:head, version: 111'110, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:76f9746b:::obj110:head@111'110, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.652+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:76f9746b:::obj110:head, version: 111'110, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:76f9746b:::obj110:head@111'110, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.652+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.652+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 580 ==== pg_backfill(progress 1.0 e 116/116 lb 1:76c71d4e:::obj92:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d98b00 con 0x4acb000 2026-02-11T23:31:27.652+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:76c71d4e:::obj92:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.652+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:76f9746b:::obj110:head, version: 111'110, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:76f9746b:::obj110:head@111'110, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.652+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:76c71d4e:::obj92:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.652+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.652+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:76f9746b:::obj110:head, version: 111'110, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:76f9746b:::obj110:head@111'110, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.652+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:76f9746b:::obj110:head, version: 111'110, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:76f9746b:::obj110:head@111'110, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:76f9746b:::obj110:head, version: 111'110, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:76f9746b:::obj110:head@111'110, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:750d67aa:::obj572:head local-lis/les=113/114 n=271 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.652+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.652+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:750d67aa:::obj572:head local-lis/les=113/114 n=271 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:76f9746b:::obj110:head, version: 111'110, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:76f9746b:::obj110:head@111'110, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.652+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.652+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:750d67aa:::obj572:head local-lis/les=113/114 n=271 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:76f9746b:::obj110:head@111'110, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.652+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:76c71d4e:::obj92:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.652+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:750d67aa:::obj572:head local-lis/les=113/114 n=271 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:76f9746b:::obj110:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.652+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:750d67aa:::obj572:head local-lis/les=113/114 n=271 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:76f9746b:::obj110:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.652+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:750d67aa:::obj572:head local-lis/les=113/114 n=271 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:76f9746b:::obj110:head 2026-02-11T23:31:27.652+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:750d67aa:::obj572:head local-lis/les=113/114 n=271 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:76f9746b:::obj110:head v 111'110 2026-02-11T23:31:27.652+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:750d67aa:::obj572:head local-lis/les=113/114 n=271 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.652+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.652+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:76f9746b:::obj110:head, version: 111'110, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:76f9746b:::obj110:head@111'110, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:27.652+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.652+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.652+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:76c71d4e:::obj92:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.652+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:76c71d4e:::obj92:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 1.0 e 116/116 lb 1:76c71d4e:::obj92:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:750d67aa:::obj572:head local-lis/les=113/114 n=271 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.652+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.652+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:750d67aa:::obj572:head local-lis/les=113/114 n=271 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:76c71d4e:::obj92:head) v3 2026-02-11T23:31:27.652+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:750d67aa:::obj572:head local-lis/les=113/114 n=271 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:76c71d4e:::obj92:head) 2026-02-11T23:31:27.652+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.652+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:76c71d4e:::obj92:head) v3 finish latency 0.000009 2026-02-11T23:31:27.652+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:76c71d4e:::obj92:head local-lis/les=113/114 n=272 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.652+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.652+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.652+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:76c71d4e:::obj92:head local-lis/les=113/114 n=272 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.652+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.652+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.652+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.652+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6005500 con 0x4acb000 2026-02-11T23:31:27.652+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:76f9746b:::obj110:head)]) -- 0x6014480 con 0x4acb000 2026-02-11T23:31:27.652+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.652+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 581 ==== MOSDPGPush(1.0 116/113 [PushOp(1:775ec905:::obj98:head, version: 111'98, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:775ec905:::obj98:head@111'98, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x56471e0 con 0x4acb000 2026-02-11T23:31:27.652+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:775ec905:::obj98:head, version: 111'98, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:775ec905:::obj98:head@111'98, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.652+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:775ec905:::obj98:head, version: 111'98, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:775ec905:::obj98:head@111'98, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.652+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.652+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 582 ==== pg_backfill(progress 1.0 e 116/116 lb 1:76f9746b:::obj110:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99080 con 0x4acb000 2026-02-11T23:31:27.652+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:775ec905:::obj98:head, version: 111'98, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:775ec905:::obj98:head@111'98, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.652+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:76f9746b:::obj110:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.652+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.652+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:76f9746b:::obj110:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.652+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:775ec905:::obj98:head, version: 111'98, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:775ec905:::obj98:head@111'98, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.652+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.652+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:775ec905:::obj98:head, version: 111'98, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:775ec905:::obj98:head@111'98, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 116/113 [PushOp(1:775ec905:::obj98:head, version: 111'98, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:775ec905:::obj98:head@111'98, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:76c71d4e:::obj92:head local-lis/les=113/114 n=272 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.652+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.652+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:76c71d4e:::obj92:head local-lis/les=113/114 n=272 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:775ec905:::obj98:head, version: 111'98, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:775ec905:::obj98:head@111'98, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.652+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:76c71d4e:::obj92:head local-lis/les=113/114 n=272 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:775ec905:::obj98:head@111'98, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.652+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.652+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:76f9746b:::obj110:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.652+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:76c71d4e:::obj92:head local-lis/les=113/114 n=272 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:775ec905:::obj98:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.652+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:76c71d4e:::obj92:head local-lis/les=113/114 n=272 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:775ec905:::obj98:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.652+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:76c71d4e:::obj92:head local-lis/les=113/114 n=272 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:775ec905:::obj98:head 2026-02-11T23:31:27.652+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:76c71d4e:::obj92:head local-lis/les=113/114 n=272 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:775ec905:::obj98:head v 111'98 2026-02-11T23:31:27.652+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:76c71d4e:::obj92:head local-lis/les=113/114 n=272 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.652+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.652+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:775ec905:::obj98:head, version: 111'98, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:775ec905:::obj98:head@111'98, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:27.652+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.652+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.652+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:76f9746b:::obj110:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.652+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:76f9746b:::obj110:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:76f9746b:::obj110:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:76c71d4e:::obj92:head local-lis/les=113/114 n=272 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.652+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.652+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:76c71d4e:::obj92:head local-lis/les=113/114 n=272 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:76f9746b:::obj110:head) v3 2026-02-11T23:31:27.652+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:76c71d4e:::obj92:head local-lis/les=113/114 n=272 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:76f9746b:::obj110:head) 2026-02-11T23:31:27.652+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.652+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:76f9746b:::obj110:head) v3 finish latency 0.000009 2026-02-11T23:31:27.652+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:76f9746b:::obj110:head local-lis/les=113/114 n=273 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.652+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.653+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.653+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:76f9746b:::obj110:head local-lis/les=113/114 n=273 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.653+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.653+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.653+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.653+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fce780 con 0x4acb000 2026-02-11T23:31:27.653+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:775ec905:::obj98:head)]) -- 0x6014780 con 0x4acb000 2026-02-11T23:31:27.653+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.653+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 583 ==== MOSDPGPush(1.0 116/113 [PushOp(1:776b5b83:::obj306:head, version: 111'306, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:776b5b83:::obj306:head@111'306, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d8d00 con 0x4acb000 2026-02-11T23:31:27.653+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:776b5b83:::obj306:head, version: 111'306, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:776b5b83:::obj306:head@111'306, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.653+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:776b5b83:::obj306:head, version: 111'306, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:776b5b83:::obj306:head@111'306, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.653+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.653+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 584 ==== pg_backfill(progress 1.0 e 116/116 lb 1:775ec905:::obj98:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d99600 con 0x4acb000 2026-02-11T23:31:27.653+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:776b5b83:::obj306:head, version: 111'306, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:776b5b83:::obj306:head@111'306, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.653+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:775ec905:::obj98:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.653+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:775ec905:::obj98:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.653+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.653+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:776b5b83:::obj306:head, version: 111'306, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:776b5b83:::obj306:head@111'306, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.653+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.653+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:776b5b83:::obj306:head, version: 111'306, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:776b5b83:::obj306:head@111'306, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(1.0 116/113 [PushOp(1:776b5b83:::obj306:head, version: 111'306, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:776b5b83:::obj306:head@111'306, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:76f9746b:::obj110:head local-lis/les=113/114 n=273 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.653+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.653+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:76f9746b:::obj110:head local-lis/les=113/114 n=273 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:776b5b83:::obj306:head, version: 111'306, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:776b5b83:::obj306:head@111'306, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.653+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:76f9746b:::obj110:head local-lis/les=113/114 n=273 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:776b5b83:::obj306:head@111'306, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.653+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.653+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:775ec905:::obj98:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.653+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:76f9746b:::obj110:head local-lis/les=113/114 n=273 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:776b5b83:::obj306:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.653+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:76f9746b:::obj110:head local-lis/les=113/114 n=273 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:776b5b83:::obj306:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.653+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:76f9746b:::obj110:head local-lis/les=113/114 n=273 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:776b5b83:::obj306:head 2026-02-11T23:31:27.653+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:76f9746b:::obj110:head local-lis/les=113/114 n=273 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:776b5b83:::obj306:head v 111'306 2026-02-11T23:31:27.653+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:76f9746b:::obj110:head local-lis/les=113/114 n=273 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.653+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.653+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:776b5b83:::obj306:head, version: 111'306, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:776b5b83:::obj306:head@111'306, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.653+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.653+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.653+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:775ec905:::obj98:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.653+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:775ec905:::obj98:head) v3 prio 15 cost 0 latency 0.000060 pg_backfill(progress 1.0 e 116/116 lb 1:775ec905:::obj98:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:76f9746b:::obj110:head local-lis/les=113/114 n=273 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.653+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.653+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:76f9746b:::obj110:head local-lis/les=113/114 n=273 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:775ec905:::obj98:head) v3 2026-02-11T23:31:27.653+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:76f9746b:::obj110:head local-lis/les=113/114 n=273 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:775ec905:::obj98:head) 2026-02-11T23:31:27.653+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.653+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:775ec905:::obj98:head) v3 finish latency 0.000009 2026-02-11T23:31:27.653+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:775ec905:::obj98:head local-lis/les=113/114 n=274 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.653+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.653+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.653+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:775ec905:::obj98:head local-lis/les=113/114 n=274 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.653+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.653+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.653+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.653+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6005380 con 0x4acb000 2026-02-11T23:31:27.653+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:776b5b83:::obj306:head)]) -- 0x6014900 con 0x4acb000 2026-02-11T23:31:27.653+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.654+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 585 ==== MOSDPGPush(1.0 116/113 [PushOp(1:779a6cde:::obj377:head, version: 111'377, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:779a6cde:::obj377:head@111'377, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d91e0 con 0x4acb000 2026-02-11T23:31:27.654+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:779a6cde:::obj377:head, version: 111'377, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:779a6cde:::obj377:head@111'377, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.654+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:779a6cde:::obj377:head, version: 111'377, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:779a6cde:::obj377:head@111'377, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.654+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.654+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 586 ==== pg_backfill(progress 1.0 e 116/116 lb 1:776b5b83:::obj306:head) ==== 1062+0+0 (unknown 0 0 0) 0x4a2cb00 con 0x4acb000 2026-02-11T23:31:27.654+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:779a6cde:::obj377:head, version: 111'377, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:779a6cde:::obj377:head@111'377, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.654+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:776b5b83:::obj306:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.654+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:776b5b83:::obj306:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.654+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.654+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:779a6cde:::obj377:head, version: 111'377, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:779a6cde:::obj377:head@111'377, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.654+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.654+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:779a6cde:::obj377:head, version: 111'377, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:779a6cde:::obj377:head@111'377, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:779a6cde:::obj377:head, version: 111'377, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:779a6cde:::obj377:head@111'377, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:775ec905:::obj98:head local-lis/les=113/114 n=274 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.654+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.654+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:775ec905:::obj98:head local-lis/les=113/114 n=274 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:779a6cde:::obj377:head, version: 111'377, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:779a6cde:::obj377:head@111'377, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.654+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:775ec905:::obj98:head local-lis/les=113/114 n=274 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:779a6cde:::obj377:head@111'377, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.654+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.654+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:776b5b83:::obj306:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.654+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:775ec905:::obj98:head local-lis/les=113/114 n=274 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:779a6cde:::obj377:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.654+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:775ec905:::obj98:head local-lis/les=113/114 n=274 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:779a6cde:::obj377:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.654+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:775ec905:::obj98:head local-lis/les=113/114 n=274 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:779a6cde:::obj377:head 2026-02-11T23:31:27.654+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:775ec905:::obj98:head local-lis/les=113/114 n=274 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:779a6cde:::obj377:head v 111'377 2026-02-11T23:31:27.654+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:775ec905:::obj98:head local-lis/les=113/114 n=274 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.654+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.654+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:779a6cde:::obj377:head, version: 111'377, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:779a6cde:::obj377:head@111'377, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.654+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.654+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.654+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:776b5b83:::obj306:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.654+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:776b5b83:::obj306:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:776b5b83:::obj306:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:775ec905:::obj98:head local-lis/les=113/114 n=274 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.654+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.654+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:775ec905:::obj98:head local-lis/les=113/114 n=274 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:776b5b83:::obj306:head) v3 2026-02-11T23:31:27.654+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:775ec905:::obj98:head local-lis/les=113/114 n=274 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:776b5b83:::obj306:head) 2026-02-11T23:31:27.654+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.654+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:776b5b83:::obj306:head) v3 finish latency 0.000009 2026-02-11T23:31:27.654+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:776b5b83:::obj306:head local-lis/les=113/114 n=275 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.654+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.654+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.654+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:776b5b83:::obj306:head local-lis/les=113/114 n=275 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.654+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.654+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.654+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.654+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fedc80 con 0x4acb000 2026-02-11T23:31:27.654+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:779a6cde:::obj377:head)]) -- 0x6014c00 con 0x4acb000 2026-02-11T23:31:27.654+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.654+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 587 ==== MOSDPGPush(1.0 116/113 [PushOp(1:786d07d6:::obj325:head, version: 111'325, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:786d07d6:::obj325:head@111'325, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d96c0 con 0x4acb000 2026-02-11T23:31:27.654+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:786d07d6:::obj325:head, version: 111'325, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:786d07d6:::obj325:head@111'325, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.654+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:786d07d6:::obj325:head, version: 111'325, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:786d07d6:::obj325:head@111'325, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.654+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.654+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 588 ==== pg_backfill(progress 1.0 e 116/116 lb 1:779a6cde:::obj377:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2d080 con 0x4acb000 2026-02-11T23:31:27.654+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:779a6cde:::obj377:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.654+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:786d07d6:::obj325:head, version: 111'325, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:786d07d6:::obj325:head@111'325, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.654+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:779a6cde:::obj377:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.654+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.654+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:786d07d6:::obj325:head, version: 111'325, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:786d07d6:::obj325:head@111'325, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.654+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:786d07d6:::obj325:head, version: 111'325, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:786d07d6:::obj325:head@111'325, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:786d07d6:::obj325:head, version: 111'325, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:786d07d6:::obj325:head@111'325, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:776b5b83:::obj306:head local-lis/les=113/114 n=275 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.654+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.654+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.654+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:776b5b83:::obj306:head local-lis/les=113/114 n=275 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:786d07d6:::obj325:head, version: 111'325, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:786d07d6:::obj325:head@111'325, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.654+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:779a6cde:::obj377:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.654+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:776b5b83:::obj306:head local-lis/les=113/114 n=275 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:786d07d6:::obj325:head@111'325, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.654+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:776b5b83:::obj306:head local-lis/les=113/114 n=275 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:786d07d6:::obj325:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.654+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:776b5b83:::obj306:head local-lis/les=113/114 n=275 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:786d07d6:::obj325:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.654+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:776b5b83:::obj306:head local-lis/les=113/114 n=275 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:786d07d6:::obj325:head 2026-02-11T23:31:27.654+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:776b5b83:::obj306:head local-lis/les=113/114 n=275 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:786d07d6:::obj325:head v 111'325 2026-02-11T23:31:27.654+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:776b5b83:::obj306:head local-lis/les=113/114 n=275 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.654+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.654+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:786d07d6:::obj325:head, version: 111'325, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:786d07d6:::obj325:head@111'325, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.654+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.654+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.654+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:779a6cde:::obj377:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.654+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:779a6cde:::obj377:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:779a6cde:::obj377:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:776b5b83:::obj306:head local-lis/les=113/114 n=275 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.654+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.654+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:776b5b83:::obj306:head local-lis/les=113/114 n=275 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:779a6cde:::obj377:head) v3 2026-02-11T23:31:27.654+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:776b5b83:::obj306:head local-lis/les=113/114 n=275 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:779a6cde:::obj377:head) 2026-02-11T23:31:27.654+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.654+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:779a6cde:::obj377:head) v3 finish latency 0.000009 2026-02-11T23:31:27.654+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:779a6cde:::obj377:head local-lis/les=113/114 n=276 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.654+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.654+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.654+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:779a6cde:::obj377:head local-lis/les=113/114 n=276 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.654+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.654+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.654+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.654+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6005200 con 0x4acb000 2026-02-11T23:31:27.654+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:786d07d6:::obj325:head)]) -- 0x6014f00 con 0x4acb000 2026-02-11T23:31:27.654+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.655+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 589 ==== MOSDPGPush(1.0 116/113 [PushOp(1:78a5b78c:::obj388:head, version: 111'388, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:78a5b78c:::obj388:head@111'388, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56ca000 con 0x4acb000 2026-02-11T23:31:27.655+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:78a5b78c:::obj388:head, version: 111'388, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:78a5b78c:::obj388:head@111'388, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.655+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:78a5b78c:::obj388:head, version: 111'388, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:78a5b78c:::obj388:head@111'388, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.655+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.655+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 590 ==== pg_backfill(progress 1.0 e 116/116 lb 1:786d07d6:::obj325:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e680 con 0x4acb000 2026-02-11T23:31:27.655+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:78a5b78c:::obj388:head, version: 111'388, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:78a5b78c:::obj388:head@111'388, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.655+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.655+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:786d07d6:::obj325:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.655+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:78a5b78c:::obj388:head, version: 111'388, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:78a5b78c:::obj388:head@111'388, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.655+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:786d07d6:::obj325:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.655+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.655+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:786d07d6:::obj325:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.655+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:78a5b78c:::obj388:head, version: 111'388, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:78a5b78c:::obj388:head@111'388, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:78a5b78c:::obj388:head, version: 111'388, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:78a5b78c:::obj388:head@111'388, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:779a6cde:::obj377:head local-lis/les=113/114 n=276 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.655+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.655+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:779a6cde:::obj377:head local-lis/les=113/114 n=276 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:78a5b78c:::obj388:head, version: 111'388, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:78a5b78c:::obj388:head@111'388, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.655+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:779a6cde:::obj377:head local-lis/les=113/114 n=276 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:78a5b78c:::obj388:head@111'388, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.655+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:779a6cde:::obj377:head local-lis/les=113/114 n=276 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:78a5b78c:::obj388:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.655+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:779a6cde:::obj377:head local-lis/les=113/114 n=276 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:78a5b78c:::obj388:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.655+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:779a6cde:::obj377:head local-lis/les=113/114 n=276 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:78a5b78c:::obj388:head 2026-02-11T23:31:27.655+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:779a6cde:::obj377:head local-lis/les=113/114 n=276 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:78a5b78c:::obj388:head v 111'388 2026-02-11T23:31:27.655+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:779a6cde:::obj377:head local-lis/les=113/114 n=276 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.655+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.655+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:78a5b78c:::obj388:head, version: 111'388, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:78a5b78c:::obj388:head@111'388, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.655+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.655+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.655+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:786d07d6:::obj325:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.655+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:786d07d6:::obj325:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 1.0 e 116/116 lb 1:786d07d6:::obj325:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:779a6cde:::obj377:head local-lis/les=113/114 n=276 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.655+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.655+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:779a6cde:::obj377:head local-lis/les=113/114 n=276 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:786d07d6:::obj325:head) v3 2026-02-11T23:31:27.655+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:779a6cde:::obj377:head local-lis/les=113/114 n=276 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:786d07d6:::obj325:head) 2026-02-11T23:31:27.655+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.655+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:786d07d6:::obj325:head) v3 finish latency 0.000010 2026-02-11T23:31:27.655+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:786d07d6:::obj325:head local-lis/les=113/114 n=277 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.655+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.655+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.655+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:786d07d6:::obj325:head local-lis/les=113/114 n=277 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.655+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.655+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.655+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.655+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fcea80 con 0x4acb000 2026-02-11T23:31:27.655+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:78a5b78c:::obj388:head)]) -- 0x6015200 con 0x4acb000 2026-02-11T23:31:27.655+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.655+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 591 ==== MOSDPGPush(1.0 116/113 [PushOp(1:7903d8f8:::obj457:head, version: 111'457, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7903d8f8:::obj457:head@111'457, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d8820 con 0x4acb000 2026-02-11T23:31:27.655+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:7903d8f8:::obj457:head, version: 111'457, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7903d8f8:::obj457:head@111'457, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 116 2026-02-11T23:31:27.655+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7903d8f8:::obj457:head, version: 111'457, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7903d8f8:::obj457:head@111'457, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.655+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.655+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 592 ==== pg_backfill(progress 1.0 e 116/116 lb 1:78a5b78c:::obj388:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e100 con 0x4acb000 2026-02-11T23:31:27.655+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:78a5b78c:::obj388:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.655+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7903d8f8:::obj457:head, version: 111'457, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7903d8f8:::obj457:head@111'457, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.655+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:78a5b78c:::obj388:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.655+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.655+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7903d8f8:::obj457:head, version: 111'457, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7903d8f8:::obj457:head@111'457, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.655+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:7903d8f8:::obj457:head, version: 111'457, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7903d8f8:::obj457:head@111'457, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(1.0 116/113 [PushOp(1:7903d8f8:::obj457:head, version: 111'457, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7903d8f8:::obj457:head@111'457, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:786d07d6:::obj325:head local-lis/les=113/114 n=277 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.655+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.655+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:786d07d6:::obj325:head local-lis/les=113/114 n=277 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:7903d8f8:::obj457:head, version: 111'457, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7903d8f8:::obj457:head@111'457, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.655+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.655+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:78a5b78c:::obj388:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.655+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:786d07d6:::obj325:head local-lis/les=113/114 n=277 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:7903d8f8:::obj457:head@111'457, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.655+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:786d07d6:::obj325:head local-lis/les=113/114 n=277 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:7903d8f8:::obj457:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.655+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:786d07d6:::obj325:head local-lis/les=113/114 n=277 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:7903d8f8:::obj457:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.655+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:786d07d6:::obj325:head local-lis/les=113/114 n=277 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:7903d8f8:::obj457:head 2026-02-11T23:31:27.655+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:786d07d6:::obj325:head local-lis/les=113/114 n=277 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:7903d8f8:::obj457:head v 111'457 2026-02-11T23:31:27.655+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:786d07d6:::obj325:head local-lis/les=113/114 n=277 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.655+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.655+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:7903d8f8:::obj457:head, version: 111'457, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7903d8f8:::obj457:head@111'457, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000041 2026-02-11T23:31:27.655+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:786d07d6:::obj325:head local-lis/les=113/114 n=277 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.655+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.655+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.655+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:78a5b78c:::obj388:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.655+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:78a5b78c:::obj388:head) v3 prio 15 cost 0 latency 0.000060 pg_backfill(progress 1.0 e 116/116 lb 1:78a5b78c:::obj388:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:786d07d6:::obj325:head local-lis/les=113/114 n=277 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.655+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.655+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:786d07d6:::obj325:head local-lis/les=113/114 n=277 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:78a5b78c:::obj388:head) v3 2026-02-11T23:31:27.655+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:786d07d6:::obj325:head local-lis/les=113/114 n=277 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:78a5b78c:::obj388:head) 2026-02-11T23:31:27.655+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.655+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:78a5b78c:::obj388:head) v3 finish latency 0.000011 2026-02-11T23:31:27.655+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.656+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.656+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:78a5b78c:::obj388:head local-lis/les=113/114 n=278 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.656+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.656+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.656+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.656+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6004c00 con 0x4acb000 2026-02-11T23:31:27.656+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:7903d8f8:::obj457:head)]) -- 0x6004f00 con 0x4acb000 2026-02-11T23:31:27.656+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.656+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 593 ==== MOSDPGPush(1.0 116/113 [PushOp(1:79ec8b1f:::obj508:head, version: 111'508, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:79ec8b1f:::obj508:head@111'508, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559cea0 con 0x4acb000 2026-02-11T23:31:27.656+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:79ec8b1f:::obj508:head, version: 111'508, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:79ec8b1f:::obj508:head@111'508, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.656+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:79ec8b1f:::obj508:head, version: 111'508, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:79ec8b1f:::obj508:head@111'508, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.656+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.656+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:79ec8b1f:::obj508:head, version: 111'508, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:79ec8b1f:::obj508:head@111'508, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.656+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.656+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:79ec8b1f:::obj508:head, version: 111'508, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:79ec8b1f:::obj508:head@111'508, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.656+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 594 ==== pg_backfill(progress 1.0 e 116/116 lb 1:7903d8f8:::obj457:head) ==== 1062+0+0 (unknown 0 0 0) 0x6142000 con 0x4acb000 2026-02-11T23:31:27.656+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:7903d8f8:::obj457:head) v3 prio 15 type 95 cost 0 latency 0.000012 epoch 116 2026-02-11T23:31:27.656+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.656+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:79ec8b1f:::obj508:head, version: 111'508, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:79ec8b1f:::obj508:head@111'508, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:79ec8b1f:::obj508:head, version: 111'508, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:79ec8b1f:::obj508:head@111'508, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:78a5b78c:::obj388:head local-lis/les=113/114 n=278 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.656+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.656+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7903d8f8:::obj457:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.656+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:78a5b78c:::obj388:head local-lis/les=113/114 n=278 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:79ec8b1f:::obj508:head, version: 111'508, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:79ec8b1f:::obj508:head@111'508, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.656+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:78a5b78c:::obj388:head local-lis/les=113/114 n=278 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:79ec8b1f:::obj508:head@111'508, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.656+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.656+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7903d8f8:::obj457:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.656+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:78a5b78c:::obj388:head local-lis/les=113/114 n=278 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:79ec8b1f:::obj508:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.656+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:78a5b78c:::obj388:head local-lis/les=113/114 n=278 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:79ec8b1f:::obj508:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.656+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:78a5b78c:::obj388:head local-lis/les=113/114 n=278 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:79ec8b1f:::obj508:head 2026-02-11T23:31:27.656+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:78a5b78c:::obj388:head local-lis/les=113/114 n=278 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:79ec8b1f:::obj508:head v 111'508 2026-02-11T23:31:27.656+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:78a5b78c:::obj388:head local-lis/les=113/114 n=278 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.656+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.656+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:79ec8b1f:::obj508:head, version: 111'508, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:79ec8b1f:::obj508:head@111'508, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000043 2026-02-11T23:31:27.656+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.656+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.656+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7903d8f8:::obj457:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.656+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:7903d8f8:::obj457:head) v3 prio 15 cost 0 latency 0.000060 pg_backfill(progress 1.0 e 116/116 lb 1:7903d8f8:::obj457:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:78a5b78c:::obj388:head local-lis/les=113/114 n=278 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.656+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.656+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:78a5b78c:::obj388:head local-lis/les=113/114 n=278 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:7903d8f8:::obj457:head) v3 2026-02-11T23:31:27.656+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:78a5b78c:::obj388:head local-lis/les=113/114 n=278 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:7903d8f8:::obj457:head) 2026-02-11T23:31:27.656+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.656+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:7903d8f8:::obj457:head) v3 finish latency 0.000011 2026-02-11T23:31:27.656+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7903d8f8:::obj457:head local-lis/les=113/114 n=279 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.656+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.656+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.656+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7903d8f8:::obj457:head local-lis/les=113/114 n=279 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.656+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.656+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.656+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.656+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fced80 con 0x4acb000 2026-02-11T23:31:27.656+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:79ec8b1f:::obj508:head)]) -- 0x6015500 con 0x4acb000 2026-02-11T23:31:27.656+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.657+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 595 ==== MOSDPGPush(1.0 116/113 [PushOp(1:7a057f92:::obj247:head, version: 111'247, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a057f92:::obj247:head@111'247, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559da00 con 0x4acb000 2026-02-11T23:31:27.657+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:7a057f92:::obj247:head, version: 111'247, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a057f92:::obj247:head@111'247, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.657+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7a057f92:::obj247:head, version: 111'247, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a057f92:::obj247:head@111'247, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.657+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.657+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 596 ==== pg_backfill(progress 1.0 e 116/116 lb 1:79ec8b1f:::obj508:head) ==== 1062+0+0 (unknown 0 0 0) 0x6142580 con 0x4acb000 2026-02-11T23:31:27.657+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:79ec8b1f:::obj508:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.657+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7a057f92:::obj247:head, version: 111'247, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a057f92:::obj247:head@111'247, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.657+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:79ec8b1f:::obj508:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.657+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.657+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7a057f92:::obj247:head, version: 111'247, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a057f92:::obj247:head@111'247, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.657+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:7a057f92:::obj247:head, version: 111'247, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a057f92:::obj247:head@111'247, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:7a057f92:::obj247:head, version: 111'247, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a057f92:::obj247:head@111'247, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:7903d8f8:::obj457:head local-lis/les=113/114 n=279 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.657+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.657+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.657+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7903d8f8:::obj457:head local-lis/les=113/114 n=279 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:7a057f92:::obj247:head, version: 111'247, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a057f92:::obj247:head@111'247, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.657+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:79ec8b1f:::obj508:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.657+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7903d8f8:::obj457:head local-lis/les=113/114 n=279 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:7a057f92:::obj247:head@111'247, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.657+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7903d8f8:::obj457:head local-lis/les=113/114 n=279 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:7a057f92:::obj247:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.657+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7903d8f8:::obj457:head local-lis/les=113/114 n=279 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:7a057f92:::obj247:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.657+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7903d8f8:::obj457:head local-lis/les=113/114 n=279 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:7a057f92:::obj247:head 2026-02-11T23:31:27.657+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7903d8f8:::obj457:head local-lis/les=113/114 n=279 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:7a057f92:::obj247:head v 111'247 2026-02-11T23:31:27.657+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7903d8f8:::obj457:head local-lis/les=113/114 n=279 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.657+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.657+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:7a057f92:::obj247:head, version: 111'247, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a057f92:::obj247:head@111'247, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.657+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.657+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.657+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:79ec8b1f:::obj508:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.657+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:79ec8b1f:::obj508:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 116/116 lb 1:79ec8b1f:::obj508:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:7903d8f8:::obj457:head local-lis/les=113/114 n=279 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.657+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.657+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7903d8f8:::obj457:head local-lis/les=113/114 n=279 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:79ec8b1f:::obj508:head) v3 2026-02-11T23:31:27.657+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7903d8f8:::obj457:head local-lis/les=113/114 n=279 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:79ec8b1f:::obj508:head) 2026-02-11T23:31:27.657+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.657+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:79ec8b1f:::obj508:head) v3 finish latency 0.000009 2026-02-11T23:31:27.657+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:79ec8b1f:::obj508:head local-lis/les=113/114 n=280 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.657+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.657+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.657+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:79ec8b1f:::obj508:head local-lis/les=113/114 n=280 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.657+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.657+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.657+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.657+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6004900 con 0x4acb000 2026-02-11T23:31:27.657+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:7a057f92:::obj247:head)]) -- 0x6015800 con 0x4acb000 2026-02-11T23:31:27.657+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.657+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 597 ==== MOSDPGPush(1.0 116/113 [PushOp(1:7a539578:::obj175:head, version: 111'175, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a539578:::obj175:head@111'175, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c4e0 con 0x4acb000 2026-02-11T23:31:27.657+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:7a539578:::obj175:head, version: 111'175, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a539578:::obj175:head@111'175, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.657+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7a539578:::obj175:head, version: 111'175, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a539578:::obj175:head@111'175, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.657+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.657+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 598 ==== pg_backfill(progress 1.0 e 116/116 lb 1:7a057f92:::obj247:head) ==== 1062+0+0 (unknown 0 0 0) 0x6142b00 con 0x4acb000 2026-02-11T23:31:27.657+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7a539578:::obj175:head, version: 111'175, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a539578:::obj175:head@111'175, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.657+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:7a057f92:::obj247:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.657+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.657+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7a539578:::obj175:head, version: 111'175, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a539578:::obj175:head@111'175, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.657+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7a057f92:::obj247:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.657+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.657+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7a057f92:::obj247:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.657+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:7a539578:::obj175:head, version: 111'175, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a539578:::obj175:head@111'175, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 116/113 [PushOp(1:7a539578:::obj175:head, version: 111'175, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a539578:::obj175:head@111'175, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:79ec8b1f:::obj508:head local-lis/les=113/114 n=280 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.657+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.657+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:79ec8b1f:::obj508:head local-lis/les=113/114 n=280 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:7a539578:::obj175:head, version: 111'175, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a539578:::obj175:head@111'175, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.657+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:79ec8b1f:::obj508:head local-lis/les=113/114 n=280 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:7a539578:::obj175:head@111'175, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.657+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:79ec8b1f:::obj508:head local-lis/les=113/114 n=280 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:7a539578:::obj175:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.657+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:79ec8b1f:::obj508:head local-lis/les=113/114 n=280 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:7a539578:::obj175:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.657+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:79ec8b1f:::obj508:head local-lis/les=113/114 n=280 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:7a539578:::obj175:head 2026-02-11T23:31:27.657+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:79ec8b1f:::obj508:head local-lis/les=113/114 n=280 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:7a539578:::obj175:head v 111'175 2026-02-11T23:31:27.657+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:79ec8b1f:::obj508:head local-lis/les=113/114 n=280 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.657+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.657+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:7a539578:::obj175:head, version: 111'175, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a539578:::obj175:head@111'175, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:27.657+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.657+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.657+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7a057f92:::obj247:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.657+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:7a057f92:::obj247:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 1.0 e 116/116 lb 1:7a057f92:::obj247:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:79ec8b1f:::obj508:head local-lis/les=113/114 n=280 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.657+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.657+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:79ec8b1f:::obj508:head local-lis/les=113/114 n=280 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:7a057f92:::obj247:head) v3 2026-02-11T23:31:27.657+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:79ec8b1f:::obj508:head local-lis/les=113/114 n=280 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:7a057f92:::obj247:head) 2026-02-11T23:31:27.657+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.657+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:7a057f92:::obj247:head) v3 finish latency 0.000010 2026-02-11T23:31:27.657+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a057f92:::obj247:head local-lis/les=113/114 n=281 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.657+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.657+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.657+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a057f92:::obj247:head local-lis/les=113/114 n=281 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.657+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.657+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.657+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.657+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x600a000 con 0x4acb000 2026-02-11T23:31:27.657+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:7a539578:::obj175:head)]) -- 0x6015b00 con 0x4acb000 2026-02-11T23:31:27.657+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.658+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 599 ==== MOSDPGPush(1.0 116/113 [PushOp(1:7a71687f:::obj592:head, version: 111'592, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a71687f:::obj592:head@111'592, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559d380 con 0x4acb000 2026-02-11T23:31:27.658+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:7a71687f:::obj592:head, version: 111'592, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a71687f:::obj592:head@111'592, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.658+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7a71687f:::obj592:head, version: 111'592, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a71687f:::obj592:head@111'592, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.658+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.658+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 600 ==== pg_backfill(progress 1.0 e 116/116 lb 1:7a539578:::obj175:head) ==== 1062+0+0 (unknown 0 0 0) 0x6143080 con 0x4acb000 2026-02-11T23:31:27.658+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:7a539578:::obj175:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.658+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7a71687f:::obj592:head, version: 111'592, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a71687f:::obj592:head@111'592, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.658+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7a539578:::obj175:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.658+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.658+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7a71687f:::obj592:head, version: 111'592, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a71687f:::obj592:head@111'592, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.658+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:7a71687f:::obj592:head, version: 111'592, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a71687f:::obj592:head@111'592, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:7a71687f:::obj592:head, version: 111'592, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a71687f:::obj592:head@111'592, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:7a057f92:::obj247:head local-lis/les=113/114 n=281 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.658+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.658+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.658+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7a539578:::obj175:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.658+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a057f92:::obj247:head local-lis/les=113/114 n=281 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:7a71687f:::obj592:head, version: 111'592, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a71687f:::obj592:head@111'592, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.658+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a057f92:::obj247:head local-lis/les=113/114 n=281 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:7a71687f:::obj592:head@111'592, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.658+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a057f92:::obj247:head local-lis/les=113/114 n=281 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:7a71687f:::obj592:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.658+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a057f92:::obj247:head local-lis/les=113/114 n=281 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:7a71687f:::obj592:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.658+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a057f92:::obj247:head local-lis/les=113/114 n=281 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:7a71687f:::obj592:head 2026-02-11T23:31:27.658+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a057f92:::obj247:head local-lis/les=113/114 n=281 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:7a71687f:::obj592:head v 111'592 2026-02-11T23:31:27.658+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a057f92:::obj247:head local-lis/les=113/114 n=281 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.658+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.658+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:7a71687f:::obj592:head, version: 111'592, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a71687f:::obj592:head@111'592, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:27.658+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.658+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.658+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7a539578:::obj175:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.658+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:7a539578:::obj175:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 1.0 e 116/116 lb 1:7a539578:::obj175:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:7a057f92:::obj247:head local-lis/les=113/114 n=281 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.658+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.658+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a057f92:::obj247:head local-lis/les=113/114 n=281 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:7a539578:::obj175:head) v3 2026-02-11T23:31:27.658+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a057f92:::obj247:head local-lis/les=113/114 n=281 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:7a539578:::obj175:head) 2026-02-11T23:31:27.658+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.658+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:7a539578:::obj175:head) v3 finish latency 0.000009 2026-02-11T23:31:27.658+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a539578:::obj175:head local-lis/les=113/114 n=282 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.658+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.658+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.658+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a539578:::obj175:head local-lis/les=113/114 n=282 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.658+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.658+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.658+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.658+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6004600 con 0x4acb000 2026-02-11T23:31:27.658+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:7a71687f:::obj592:head)]) -- 0x6015e00 con 0x4acb000 2026-02-11T23:31:27.658+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.658+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 601 ==== MOSDPGPush(1.0 116/113 [PushOp(1:7a94cb37:::obj345:head, version: 111'345, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a94cb37:::obj345:head@111'345, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f1a00 con 0x4acb000 2026-02-11T23:31:27.658+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:7a94cb37:::obj345:head, version: 111'345, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a94cb37:::obj345:head@111'345, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.658+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7a94cb37:::obj345:head, version: 111'345, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a94cb37:::obj345:head@111'345, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.658+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.658+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 602 ==== pg_backfill(progress 1.0 e 116/116 lb 1:7a71687f:::obj592:head) ==== 1062+0+0 (unknown 0 0 0) 0x6143600 con 0x4acb000 2026-02-11T23:31:27.658+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:7a71687f:::obj592:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.658+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7a94cb37:::obj345:head, version: 111'345, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a94cb37:::obj345:head@111'345, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.658+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7a71687f:::obj592:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.658+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.658+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7a94cb37:::obj345:head, version: 111'345, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a94cb37:::obj345:head@111'345, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.658+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.658+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:7a94cb37:::obj345:head, version: 111'345, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a94cb37:::obj345:head@111'345, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:7a94cb37:::obj345:head, version: 111'345, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a94cb37:::obj345:head@111'345, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:7a539578:::obj175:head local-lis/les=113/114 n=282 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.658+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.658+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a539578:::obj175:head local-lis/les=113/114 n=282 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:7a94cb37:::obj345:head, version: 111'345, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a94cb37:::obj345:head@111'345, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.658+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a539578:::obj175:head local-lis/les=113/114 n=282 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:7a94cb37:::obj345:head@111'345, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.658+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.658+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7a71687f:::obj592:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.658+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a539578:::obj175:head local-lis/les=113/114 n=282 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:7a94cb37:::obj345:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.658+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a539578:::obj175:head local-lis/les=113/114 n=282 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:7a94cb37:::obj345:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.658+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a539578:::obj175:head local-lis/les=113/114 n=282 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:7a94cb37:::obj345:head 2026-02-11T23:31:27.658+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a539578:::obj175:head local-lis/les=113/114 n=282 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:7a94cb37:::obj345:head v 111'345 2026-02-11T23:31:27.658+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a539578:::obj175:head local-lis/les=113/114 n=282 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.658+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.659+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:7a94cb37:::obj345:head, version: 111'345, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a94cb37:::obj345:head@111'345, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:27.659+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.659+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.659+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7a71687f:::obj592:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.659+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:7a71687f:::obj592:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 1.0 e 116/116 lb 1:7a71687f:::obj592:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:7a539578:::obj175:head local-lis/les=113/114 n=282 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.659+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.659+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a539578:::obj175:head local-lis/les=113/114 n=282 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:7a71687f:::obj592:head) v3 2026-02-11T23:31:27.659+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a539578:::obj175:head local-lis/les=113/114 n=282 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:7a71687f:::obj592:head) 2026-02-11T23:31:27.659+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.659+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:7a71687f:::obj592:head) v3 finish latency 0.000009 2026-02-11T23:31:27.659+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a71687f:::obj592:head local-lis/les=113/114 n=283 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.659+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.659+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.659+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a71687f:::obj592:head local-lis/les=113/114 n=283 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.659+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.659+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.659+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.659+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fcf080 con 0x4acb000 2026-02-11T23:31:27.659+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:7a94cb37:::obj345:head)]) -- 0x614c180 con 0x4acb000 2026-02-11T23:31:27.659+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.659+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 603 ==== MOSDPGPush(1.0 116/113 [PushOp(1:7a95c0e7:::obj245:head, version: 111'245, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a95c0e7:::obj245:head@111'245, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c9c0 con 0x4acb000 2026-02-11T23:31:27.659+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:7a95c0e7:::obj245:head, version: 111'245, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a95c0e7:::obj245:head@111'245, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.659+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7a95c0e7:::obj245:head, version: 111'245, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a95c0e7:::obj245:head@111'245, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.659+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.659+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 604 ==== pg_backfill(progress 1.0 e 116/116 lb 1:7a94cb37:::obj345:head) ==== 1062+0+0 (unknown 0 0 0) 0x6143b80 con 0x4acb000 2026-02-11T23:31:27.659+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:7a94cb37:::obj345:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.659+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7a95c0e7:::obj245:head, version: 111'245, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a95c0e7:::obj245:head@111'245, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.659+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7a94cb37:::obj345:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.659+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.659+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7a95c0e7:::obj245:head, version: 111'245, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a95c0e7:::obj245:head@111'245, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.659+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:7a95c0e7:::obj245:head, version: 111'245, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a95c0e7:::obj245:head@111'245, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 116/113 [PushOp(1:7a95c0e7:::obj245:head, version: 111'245, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a95c0e7:::obj245:head@111'245, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:7a71687f:::obj592:head local-lis/les=113/114 n=283 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.659+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.659+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.659+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7a94cb37:::obj345:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.659+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a71687f:::obj592:head local-lis/les=113/114 n=283 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:7a95c0e7:::obj245:head, version: 111'245, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a95c0e7:::obj245:head@111'245, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.659+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a71687f:::obj592:head local-lis/les=113/114 n=283 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:7a95c0e7:::obj245:head@111'245, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.659+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a71687f:::obj592:head local-lis/les=113/114 n=283 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:7a95c0e7:::obj245:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.659+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a71687f:::obj592:head local-lis/les=113/114 n=283 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:7a95c0e7:::obj245:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.659+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a71687f:::obj592:head local-lis/les=113/114 n=283 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:7a95c0e7:::obj245:head 2026-02-11T23:31:27.659+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a71687f:::obj592:head local-lis/les=113/114 n=283 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:7a95c0e7:::obj245:head v 111'245 2026-02-11T23:31:27.659+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a71687f:::obj592:head local-lis/les=113/114 n=283 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.659+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.659+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:7a95c0e7:::obj245:head, version: 111'245, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a95c0e7:::obj245:head@111'245, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.659+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.659+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.659+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7a94cb37:::obj345:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.659+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:7a94cb37:::obj345:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 1.0 e 116/116 lb 1:7a94cb37:::obj345:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:7a71687f:::obj592:head local-lis/les=113/114 n=283 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.659+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.659+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a71687f:::obj592:head local-lis/les=113/114 n=283 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:7a94cb37:::obj345:head) v3 2026-02-11T23:31:27.659+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a71687f:::obj592:head local-lis/les=113/114 n=283 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:7a94cb37:::obj345:head) 2026-02-11T23:31:27.659+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.659+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:7a94cb37:::obj345:head) v3 finish latency 0.000009 2026-02-11T23:31:27.659+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a94cb37:::obj345:head local-lis/les=113/114 n=284 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.659+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.659+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.659+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a94cb37:::obj345:head local-lis/les=113/114 n=284 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.659+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.659+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.659+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.659+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6004480 con 0x4acb000 2026-02-11T23:31:27.659+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:7a95c0e7:::obj245:head)]) -- 0x614c480 con 0x4acb000 2026-02-11T23:31:27.659+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.660+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 605 ==== MOSDPGPush(1.0 116/113 [PushOp(1:7ac8ba2d:::obj506:head, version: 111'506, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ac8ba2d:::obj506:head@111'506, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c1a0 con 0x4acb000 2026-02-11T23:31:27.660+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:7ac8ba2d:::obj506:head, version: 111'506, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ac8ba2d:::obj506:head@111'506, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.660+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7ac8ba2d:::obj506:head, version: 111'506, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ac8ba2d:::obj506:head@111'506, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.660+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.660+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 606 ==== pg_backfill(progress 1.0 e 116/116 lb 1:7a95c0e7:::obj245:head) ==== 1062+0+0 (unknown 0 0 0) 0x6144100 con 0x4acb000 2026-02-11T23:31:27.660+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:7a95c0e7:::obj245:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.660+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7ac8ba2d:::obj506:head, version: 111'506, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ac8ba2d:::obj506:head@111'506, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.660+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7a95c0e7:::obj245:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.660+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.660+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7ac8ba2d:::obj506:head, version: 111'506, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ac8ba2d:::obj506:head@111'506, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.660+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:7ac8ba2d:::obj506:head, version: 111'506, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ac8ba2d:::obj506:head@111'506, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:7ac8ba2d:::obj506:head, version: 111'506, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ac8ba2d:::obj506:head@111'506, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:7a94cb37:::obj345:head local-lis/les=113/114 n=284 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.660+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.660+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a94cb37:::obj345:head local-lis/les=113/114 n=284 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:7ac8ba2d:::obj506:head, version: 111'506, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ac8ba2d:::obj506:head@111'506, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.660+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.660+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7a95c0e7:::obj245:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.660+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a94cb37:::obj345:head local-lis/les=113/114 n=284 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:7ac8ba2d:::obj506:head@111'506, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.660+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a94cb37:::obj345:head local-lis/les=113/114 n=284 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:7ac8ba2d:::obj506:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.660+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a94cb37:::obj345:head local-lis/les=113/114 n=284 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:7ac8ba2d:::obj506:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.660+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a94cb37:::obj345:head local-lis/les=113/114 n=284 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:7ac8ba2d:::obj506:head 2026-02-11T23:31:27.660+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a94cb37:::obj345:head local-lis/les=113/114 n=284 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:7ac8ba2d:::obj506:head v 111'506 2026-02-11T23:31:27.660+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a94cb37:::obj345:head local-lis/les=113/114 n=284 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.660+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.660+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:7ac8ba2d:::obj506:head, version: 111'506, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ac8ba2d:::obj506:head@111'506, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.660+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a94cb37:::obj345:head local-lis/les=113/114 n=284 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.660+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.660+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.660+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7a95c0e7:::obj245:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.660+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:7a95c0e7:::obj245:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 1.0 e 116/116 lb 1:7a95c0e7:::obj245:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:7a94cb37:::obj345:head local-lis/les=113/114 n=284 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.660+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.660+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a94cb37:::obj345:head local-lis/les=113/114 n=284 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:7a95c0e7:::obj245:head) v3 2026-02-11T23:31:27.660+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a94cb37:::obj345:head local-lis/les=113/114 n=284 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:7a95c0e7:::obj245:head) 2026-02-11T23:31:27.660+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.660+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:7a95c0e7:::obj245:head) v3 finish latency 0.000011 2026-02-11T23:31:27.660+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.660+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.660+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a95c0e7:::obj245:head local-lis/les=113/114 n=285 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.660+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.660+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.660+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.660+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6004600 con 0x4acb000 2026-02-11T23:31:27.660+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:7ac8ba2d:::obj506:head)]) -- 0x600a180 con 0x4acb000 2026-02-11T23:31:27.660+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.660+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 607 ==== MOSDPGPush(1.0 116/113 [PushOp(1:7af66aa8:::obj163:head, version: 111'163, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7af66aa8:::obj163:head@111'163, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f0ea0 con 0x4acb000 2026-02-11T23:31:27.660+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:7af66aa8:::obj163:head, version: 111'163, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7af66aa8:::obj163:head@111'163, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.660+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7af66aa8:::obj163:head, version: 111'163, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7af66aa8:::obj163:head@111'163, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.660+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.660+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 608 ==== pg_backfill(progress 1.0 e 116/116 lb 1:7ac8ba2d:::obj506:head) ==== 1062+0+0 (unknown 0 0 0) 0x6144680 con 0x4acb000 2026-02-11T23:31:27.660+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7af66aa8:::obj163:head, version: 111'163, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7af66aa8:::obj163:head@111'163, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.660+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:7ac8ba2d:::obj506:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.660+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.660+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7ac8ba2d:::obj506:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.660+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7af66aa8:::obj163:head, version: 111'163, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7af66aa8:::obj163:head@111'163, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.660+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.660+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:7af66aa8:::obj163:head, version: 111'163, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7af66aa8:::obj163:head@111'163, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:7af66aa8:::obj163:head, version: 111'163, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7af66aa8:::obj163:head@111'163, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:7a95c0e7:::obj245:head local-lis/les=113/114 n=285 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.660+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.660+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a95c0e7:::obj245:head local-lis/les=113/114 n=285 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:7af66aa8:::obj163:head, version: 111'163, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7af66aa8:::obj163:head@111'163, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.660+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.660+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a95c0e7:::obj245:head local-lis/les=113/114 n=285 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:7af66aa8:::obj163:head@111'163, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.660+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7ac8ba2d:::obj506:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.660+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a95c0e7:::obj245:head local-lis/les=113/114 n=285 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:7af66aa8:::obj163:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.660+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a95c0e7:::obj245:head local-lis/les=113/114 n=285 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:7af66aa8:::obj163:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.660+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a95c0e7:::obj245:head local-lis/les=113/114 n=285 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:7af66aa8:::obj163:head 2026-02-11T23:31:27.660+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a95c0e7:::obj245:head local-lis/les=113/114 n=285 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:7af66aa8:::obj163:head v 111'163 2026-02-11T23:31:27.660+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a95c0e7:::obj245:head local-lis/les=113/114 n=285 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.660+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.660+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:7af66aa8:::obj163:head, version: 111'163, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7af66aa8:::obj163:head@111'163, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:27.660+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a95c0e7:::obj245:head local-lis/les=113/114 n=285 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.660+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.660+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.660+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7ac8ba2d:::obj506:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.660+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:7ac8ba2d:::obj506:head) v3 prio 15 cost 0 latency 0.000080 pg_backfill(progress 1.0 e 116/116 lb 1:7ac8ba2d:::obj506:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:7a95c0e7:::obj245:head local-lis/les=113/114 n=285 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.660+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.660+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a95c0e7:::obj245:head local-lis/les=113/114 n=285 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:7ac8ba2d:::obj506:head) v3 2026-02-11T23:31:27.660+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7a95c0e7:::obj245:head local-lis/les=113/114 n=285 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:7ac8ba2d:::obj506:head) 2026-02-11T23:31:27.660+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.660+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:7ac8ba2d:::obj506:head) v3 finish latency 0.000009 2026-02-11T23:31:27.660+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.661+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.661+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ac8ba2d:::obj506:head local-lis/les=113/114 n=286 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.661+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.661+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.661+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.661+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6015800 con 0x4acb000 2026-02-11T23:31:27.661+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:7af66aa8:::obj163:head)]) -- 0x6015b00 con 0x4acb000 2026-02-11T23:31:27.661+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.661+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 609 ==== MOSDPGPush(1.0 116/113 [PushOp(1:7b326036:::obj411:head, version: 111'411, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7b326036:::obj411:head@111'411, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f09c0 con 0x4acb000 2026-02-11T23:31:27.661+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:7b326036:::obj411:head, version: 111'411, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7b326036:::obj411:head@111'411, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.661+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7b326036:::obj411:head, version: 111'411, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7b326036:::obj411:head@111'411, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.661+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.661+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 610 ==== pg_backfill(progress 1.0 e 116/116 lb 1:7af66aa8:::obj163:head) ==== 1062+0+0 (unknown 0 0 0) 0x6144c00 con 0x4acb000 2026-02-11T23:31:27.661+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7b326036:::obj411:head, version: 111'411, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7b326036:::obj411:head@111'411, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.661+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:7af66aa8:::obj163:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.661+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7af66aa8:::obj163:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.661+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.661+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7b326036:::obj411:head, version: 111'411, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7b326036:::obj411:head@111'411, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.661+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.661+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:7b326036:::obj411:head, version: 111'411, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7b326036:::obj411:head@111'411, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 116/113 [PushOp(1:7b326036:::obj411:head, version: 111'411, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7b326036:::obj411:head@111'411, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:7ac8ba2d:::obj506:head local-lis/les=113/114 n=286 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.661+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.661+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ac8ba2d:::obj506:head local-lis/les=113/114 n=286 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:7b326036:::obj411:head, version: 111'411, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7b326036:::obj411:head@111'411, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.661+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ac8ba2d:::obj506:head local-lis/les=113/114 n=286 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:7b326036:::obj411:head@111'411, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.661+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.661+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7af66aa8:::obj163:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.661+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ac8ba2d:::obj506:head local-lis/les=113/114 n=286 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:7b326036:::obj411:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.661+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ac8ba2d:::obj506:head local-lis/les=113/114 n=286 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:7b326036:::obj411:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.661+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ac8ba2d:::obj506:head local-lis/les=113/114 n=286 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:7b326036:::obj411:head 2026-02-11T23:31:27.661+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ac8ba2d:::obj506:head local-lis/les=113/114 n=286 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:7b326036:::obj411:head v 111'411 2026-02-11T23:31:27.661+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ac8ba2d:::obj506:head local-lis/les=113/114 n=286 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.661+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.661+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:7b326036:::obj411:head, version: 111'411, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7b326036:::obj411:head@111'411, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.661+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.661+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.661+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7af66aa8:::obj163:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.661+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:7af66aa8:::obj163:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 1.0 e 116/116 lb 1:7af66aa8:::obj163:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:7ac8ba2d:::obj506:head local-lis/les=113/114 n=286 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.661+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.661+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ac8ba2d:::obj506:head local-lis/les=113/114 n=286 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:7af66aa8:::obj163:head) v3 2026-02-11T23:31:27.661+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ac8ba2d:::obj506:head local-lis/les=113/114 n=286 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:7af66aa8:::obj163:head) 2026-02-11T23:31:27.661+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.661+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:7af66aa8:::obj163:head) v3 finish latency 0.000011 2026-02-11T23:31:27.661+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7af66aa8:::obj163:head local-lis/les=113/114 n=287 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.661+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.661+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.661+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7af66aa8:::obj163:head local-lis/les=113/114 n=287 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.661+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.661+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.661+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.661+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6004900 con 0x4acb000 2026-02-11T23:31:27.661+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:7b326036:::obj411:head)]) -- 0x614c780 con 0x4acb000 2026-02-11T23:31:27.661+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.662+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 611 ==== MOSDPGPush(1.0 116/113 [PushOp(1:7c121b42:::obj116:head, version: 111'116, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c121b42:::obj116:head@111'116, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f1520 con 0x4acb000 2026-02-11T23:31:27.662+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:7c121b42:::obj116:head, version: 111'116, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c121b42:::obj116:head@111'116, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 116 2026-02-11T23:31:27.662+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7c121b42:::obj116:head, version: 111'116, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c121b42:::obj116:head@111'116, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.662+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.662+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 612 ==== pg_backfill(progress 1.0 e 116/116 lb 1:7b326036:::obj411:head) ==== 1062+0+0 (unknown 0 0 0) 0x6145180 con 0x4acb000 2026-02-11T23:31:27.662+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:7b326036:::obj411:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.662+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7c121b42:::obj116:head, version: 111'116, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c121b42:::obj116:head@111'116, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.662+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7b326036:::obj411:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.662+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.662+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7c121b42:::obj116:head, version: 111'116, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c121b42:::obj116:head@111'116, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.662+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:7c121b42:::obj116:head, version: 111'116, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c121b42:::obj116:head@111'116, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(1.0 116/113 [PushOp(1:7c121b42:::obj116:head, version: 111'116, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c121b42:::obj116:head@111'116, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:7af66aa8:::obj163:head local-lis/les=113/114 n=287 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.662+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.662+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.662+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7af66aa8:::obj163:head local-lis/les=113/114 n=287 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:7c121b42:::obj116:head, version: 111'116, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c121b42:::obj116:head@111'116, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.662+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7b326036:::obj411:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.662+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7af66aa8:::obj163:head local-lis/les=113/114 n=287 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:7c121b42:::obj116:head@111'116, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.662+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7af66aa8:::obj163:head local-lis/les=113/114 n=287 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:7c121b42:::obj116:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.662+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7af66aa8:::obj163:head local-lis/les=113/114 n=287 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:7c121b42:::obj116:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.662+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7af66aa8:::obj163:head local-lis/les=113/114 n=287 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:7c121b42:::obj116:head 2026-02-11T23:31:27.662+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7af66aa8:::obj163:head local-lis/les=113/114 n=287 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:7c121b42:::obj116:head v 111'116 2026-02-11T23:31:27.662+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7af66aa8:::obj163:head local-lis/les=113/114 n=287 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.662+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.662+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:7c121b42:::obj116:head, version: 111'116, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c121b42:::obj116:head@111'116, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.662+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.662+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.662+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7b326036:::obj411:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.662+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:7b326036:::obj411:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:7b326036:::obj411:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:7af66aa8:::obj163:head local-lis/les=113/114 n=287 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.662+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.662+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7af66aa8:::obj163:head local-lis/les=113/114 n=287 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:7b326036:::obj411:head) v3 2026-02-11T23:31:27.662+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7af66aa8:::obj163:head local-lis/les=113/114 n=287 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:7b326036:::obj411:head) 2026-02-11T23:31:27.662+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.662+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:7b326036:::obj411:head) v3 finish latency 0.000010 2026-02-11T23:31:27.662+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7b326036:::obj411:head local-lis/les=113/114 n=288 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.662+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.662+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.662+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7b326036:::obj411:head local-lis/les=113/114 n=288 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.662+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.662+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.662+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.662+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6015500 con 0x4acb000 2026-02-11T23:31:27.662+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:7c121b42:::obj116:head)]) -- 0x614ca80 con 0x4acb000 2026-02-11T23:31:27.662+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.662+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 613 ==== MOSDPGPush(1.0 116/113 [PushOp(1:7c3a2962:::obj277:head, version: 111'277, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c3a2962:::obj277:head@111'277, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c820 con 0x4acb000 2026-02-11T23:31:27.662+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:7c3a2962:::obj277:head, version: 111'277, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c3a2962:::obj277:head@111'277, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.662+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7c3a2962:::obj277:head, version: 111'277, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c3a2962:::obj277:head@111'277, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.662+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.662+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 614 ==== pg_backfill(progress 1.0 e 116/116 lb 1:7c121b42:::obj116:head) ==== 1062+0+0 (unknown 0 0 0) 0x6145700 con 0x4acb000 2026-02-11T23:31:27.662+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7c3a2962:::obj277:head, version: 111'277, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c3a2962:::obj277:head@111'277, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.662+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:7c121b42:::obj116:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.662+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7c121b42:::obj116:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.662+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.662+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7c3a2962:::obj277:head, version: 111'277, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c3a2962:::obj277:head@111'277, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.662+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.662+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:7c3a2962:::obj277:head, version: 111'277, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c3a2962:::obj277:head@111'277, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:7c3a2962:::obj277:head, version: 111'277, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c3a2962:::obj277:head@111'277, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:7b326036:::obj411:head local-lis/les=113/114 n=288 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.662+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.662+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7b326036:::obj411:head local-lis/les=113/114 n=288 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:7c3a2962:::obj277:head, version: 111'277, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c3a2962:::obj277:head@111'277, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.662+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7b326036:::obj411:head local-lis/les=113/114 n=288 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:7c3a2962:::obj277:head@111'277, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.662+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.662+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7c121b42:::obj116:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.662+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7b326036:::obj411:head local-lis/les=113/114 n=288 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:7c3a2962:::obj277:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.662+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7b326036:::obj411:head local-lis/les=113/114 n=288 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:7c3a2962:::obj277:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.662+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7b326036:::obj411:head local-lis/les=113/114 n=288 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:7c3a2962:::obj277:head 2026-02-11T23:31:27.662+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7b326036:::obj411:head local-lis/les=113/114 n=288 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:7c3a2962:::obj277:head v 111'277 2026-02-11T23:31:27.662+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7b326036:::obj411:head local-lis/les=113/114 n=288 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.662+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.662+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:7c3a2962:::obj277:head, version: 111'277, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c3a2962:::obj277:head@111'277, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000043 2026-02-11T23:31:27.662+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.662+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.662+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7c121b42:::obj116:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.662+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:7c121b42:::obj116:head) v3 prio 15 cost 0 latency 0.000057 pg_backfill(progress 1.0 e 116/116 lb 1:7c121b42:::obj116:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:7b326036:::obj411:head local-lis/les=113/114 n=288 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.662+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.662+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7b326036:::obj411:head local-lis/les=113/114 n=288 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:7c121b42:::obj116:head) v3 2026-02-11T23:31:27.662+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7b326036:::obj411:head local-lis/les=113/114 n=288 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:7c121b42:::obj116:head) 2026-02-11T23:31:27.662+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.662+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:7c121b42:::obj116:head) v3 finish latency 0.000010 2026-02-11T23:31:27.662+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7c121b42:::obj116:head local-lis/les=113/114 n=289 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.662+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.662+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.662+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7c121b42:::obj116:head local-lis/les=113/114 n=289 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.662+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.662+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.662+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.662+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fced80 con 0x4acb000 2026-02-11T23:31:27.662+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:7c3a2962:::obj277:head)]) -- 0x614cd80 con 0x4acb000 2026-02-11T23:31:27.662+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.663+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 615 ==== MOSDPGPush(1.0 116/113 [PushOp(1:7c406887:::obj566:head, version: 111'566, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c406887:::obj566:head@111'566, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559dd40 con 0x4acb000 2026-02-11T23:31:27.663+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:7c406887:::obj566:head, version: 111'566, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c406887:::obj566:head@111'566, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.663+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7c406887:::obj566:head, version: 111'566, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c406887:::obj566:head@111'566, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.663+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.663+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7c406887:::obj566:head, version: 111'566, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c406887:::obj566:head@111'566, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.663+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.663+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 616 ==== pg_backfill(progress 1.0 e 116/116 lb 1:7c3a2962:::obj277:head) ==== 1062+0+0 (unknown 0 0 0) 0x6142000 con 0x4acb000 2026-02-11T23:31:27.663+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7c406887:::obj566:head, version: 111'566, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c406887:::obj566:head@111'566, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.663+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:7c3a2962:::obj277:head) v3 prio 15 type 95 cost 0 latency 0.000010 epoch 116 2026-02-11T23:31:27.663+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7c3a2962:::obj277:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.663+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:7c406887:::obj566:head, version: 111'566, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c406887:::obj566:head@111'566, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:7c406887:::obj566:head, version: 111'566, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c406887:::obj566:head@111'566, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:7c121b42:::obj116:head local-lis/les=113/114 n=289 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.663+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.663+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7c121b42:::obj116:head local-lis/les=113/114 n=289 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:7c406887:::obj566:head, version: 111'566, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c406887:::obj566:head@111'566, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.663+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.663+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7c3a2962:::obj277:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.663+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7c121b42:::obj116:head local-lis/les=113/114 n=289 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:7c406887:::obj566:head@111'566, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.663+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7c121b42:::obj116:head local-lis/les=113/114 n=289 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:7c406887:::obj566:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.663+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7c121b42:::obj116:head local-lis/les=113/114 n=289 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:7c406887:::obj566:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.663+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7c121b42:::obj116:head local-lis/les=113/114 n=289 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:7c406887:::obj566:head 2026-02-11T23:31:27.663+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7c121b42:::obj116:head local-lis/les=113/114 n=289 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:7c406887:::obj566:head v 111'566 2026-02-11T23:31:27.663+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7c121b42:::obj116:head local-lis/les=113/114 n=289 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.663+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.663+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:7c406887:::obj566:head, version: 111'566, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c406887:::obj566:head@111'566, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.663+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.663+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.663+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7c3a2962:::obj277:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.663+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:7c3a2962:::obj277:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 1.0 e 116/116 lb 1:7c3a2962:::obj277:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:7c121b42:::obj116:head local-lis/les=113/114 n=289 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.663+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.663+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7c121b42:::obj116:head local-lis/les=113/114 n=289 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:7c3a2962:::obj277:head) v3 2026-02-11T23:31:27.663+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7c121b42:::obj116:head local-lis/les=113/114 n=289 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:7c3a2962:::obj277:head) 2026-02-11T23:31:27.663+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.663+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:7c3a2962:::obj277:head) v3 finish latency 0.000010 2026-02-11T23:31:27.663+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7c3a2962:::obj277:head local-lis/les=113/114 n=290 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.663+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.663+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.663+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7c3a2962:::obj277:head local-lis/les=113/114 n=290 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.663+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.663+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.663+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.663+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6004f00 con 0x4acb000 2026-02-11T23:31:27.663+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:7c406887:::obj566:head)]) -- 0x614d080 con 0x4acb000 2026-02-11T23:31:27.663+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.663+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 617 ==== MOSDPGPush(1.0 116/113 [PushOp(1:7c955dfc:::obj322:head, version: 111'322, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c955dfc:::obj322:head@111'322, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d8680 con 0x4acb000 2026-02-11T23:31:27.663+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:7c955dfc:::obj322:head, version: 111'322, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c955dfc:::obj322:head@111'322, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.663+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7c955dfc:::obj322:head, version: 111'322, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c955dfc:::obj322:head@111'322, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.663+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.663+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 618 ==== pg_backfill(progress 1.0 e 116/116 lb 1:7c406887:::obj566:head) ==== 1062+0+0 (unknown 0 0 0) 0x6142580 con 0x4acb000 2026-02-11T23:31:27.663+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7c955dfc:::obj322:head, version: 111'322, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c955dfc:::obj322:head@111'322, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.663+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:7c406887:::obj566:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.663+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.663+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7c406887:::obj566:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.663+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7c955dfc:::obj322:head, version: 111'322, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c955dfc:::obj322:head@111'322, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.663+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.663+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:7c955dfc:::obj322:head, version: 111'322, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c955dfc:::obj322:head@111'322, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:7c955dfc:::obj322:head, version: 111'322, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c955dfc:::obj322:head@111'322, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:7c3a2962:::obj277:head local-lis/les=113/114 n=290 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.663+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.663+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7c3a2962:::obj277:head local-lis/les=113/114 n=290 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:7c955dfc:::obj322:head, version: 111'322, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c955dfc:::obj322:head@111'322, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.663+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7c3a2962:::obj277:head local-lis/les=113/114 n=290 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:7c955dfc:::obj322:head@111'322, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.663+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.663+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7c406887:::obj566:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.663+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7c3a2962:::obj277:head local-lis/les=113/114 n=290 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:7c955dfc:::obj322:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.663+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7c3a2962:::obj277:head local-lis/les=113/114 n=290 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:7c955dfc:::obj322:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.663+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7c3a2962:::obj277:head local-lis/les=113/114 n=290 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:7c955dfc:::obj322:head 2026-02-11T23:31:27.663+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7c3a2962:::obj277:head local-lis/les=113/114 n=290 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:7c955dfc:::obj322:head v 111'322 2026-02-11T23:31:27.663+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7c3a2962:::obj277:head local-lis/les=113/114 n=290 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.663+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.663+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:7c955dfc:::obj322:head, version: 111'322, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c955dfc:::obj322:head@111'322, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.663+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7c3a2962:::obj277:head local-lis/les=113/114 n=290 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.663+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.663+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.663+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7c406887:::obj566:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.663+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:7c406887:::obj566:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 1.0 e 116/116 lb 1:7c406887:::obj566:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:7c3a2962:::obj277:head local-lis/les=113/114 n=290 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.663+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.663+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7c3a2962:::obj277:head local-lis/les=113/114 n=290 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:7c406887:::obj566:head) v3 2026-02-11T23:31:27.663+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7c3a2962:::obj277:head local-lis/les=113/114 n=290 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:7c406887:::obj566:head) 2026-02-11T23:31:27.663+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.663+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:7c406887:::obj566:head) v3 finish latency 0.000011 2026-02-11T23:31:27.663+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.664+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.664+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7c406887:::obj566:head local-lis/les=113/114 n=291 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.664+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.664+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.664+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.664+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fcea80 con 0x4acb000 2026-02-11T23:31:27.664+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:7c955dfc:::obj322:head)]) -- 0x6004c00 con 0x4acb000 2026-02-11T23:31:27.664+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.664+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 619 ==== MOSDPGPush(1.0 116/113 [PushOp(1:7ccaea2b:::obj486:head, version: 111'486, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ccaea2b:::obj486:head@111'486, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56ca1a0 con 0x4acb000 2026-02-11T23:31:27.664+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:7ccaea2b:::obj486:head, version: 111'486, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ccaea2b:::obj486:head@111'486, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.664+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7ccaea2b:::obj486:head, version: 111'486, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ccaea2b:::obj486:head@111'486, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.664+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.664+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 620 ==== pg_backfill(progress 1.0 e 116/116 lb 1:7c955dfc:::obj322:head) ==== 1062+0+0 (unknown 0 0 0) 0x6142b00 con 0x4acb000 2026-02-11T23:31:27.664+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:7c955dfc:::obj322:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.664+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7ccaea2b:::obj486:head, version: 111'486, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ccaea2b:::obj486:head@111'486, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.664+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7c955dfc:::obj322:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.664+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.664+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7ccaea2b:::obj486:head, version: 111'486, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ccaea2b:::obj486:head@111'486, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.664+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:7ccaea2b:::obj486:head, version: 111'486, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ccaea2b:::obj486:head@111'486, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(1.0 116/113 [PushOp(1:7ccaea2b:::obj486:head, version: 111'486, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ccaea2b:::obj486:head@111'486, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:7c406887:::obj566:head local-lis/les=113/114 n=291 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.664+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.664+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7c406887:::obj566:head local-lis/les=113/114 n=291 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:7ccaea2b:::obj486:head, version: 111'486, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ccaea2b:::obj486:head@111'486, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.664+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.664+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7c955dfc:::obj322:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.664+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7c406887:::obj566:head local-lis/les=113/114 n=291 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:7ccaea2b:::obj486:head@111'486, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.664+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7c406887:::obj566:head local-lis/les=113/114 n=291 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:7ccaea2b:::obj486:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.664+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7c406887:::obj566:head local-lis/les=113/114 n=291 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:7ccaea2b:::obj486:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.664+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7c406887:::obj566:head local-lis/les=113/114 n=291 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:7ccaea2b:::obj486:head 2026-02-11T23:31:27.664+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7c406887:::obj566:head local-lis/les=113/114 n=291 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:7ccaea2b:::obj486:head v 111'486 2026-02-11T23:31:27.664+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7c406887:::obj566:head local-lis/les=113/114 n=291 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.664+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.664+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:7ccaea2b:::obj486:head, version: 111'486, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ccaea2b:::obj486:head@111'486, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.664+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7c406887:::obj566:head local-lis/les=113/114 n=291 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.664+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.664+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.664+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7c955dfc:::obj322:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.664+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:7c955dfc:::obj322:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 1.0 e 116/116 lb 1:7c955dfc:::obj322:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:7c406887:::obj566:head local-lis/les=113/114 n=291 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.664+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.664+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7c406887:::obj566:head local-lis/les=113/114 n=291 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:7c955dfc:::obj322:head) v3 2026-02-11T23:31:27.664+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7c406887:::obj566:head local-lis/les=113/114 n=291 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:7c955dfc:::obj322:head) 2026-02-11T23:31:27.664+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.664+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:7c955dfc:::obj322:head) v3 finish latency 0.000009 2026-02-11T23:31:27.664+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.664+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.664+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7c955dfc:::obj322:head local-lis/les=113/114 n=292 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.664+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.664+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.664+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.664+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6014c00 con 0x4acb000 2026-02-11T23:31:27.664+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:7ccaea2b:::obj486:head)]) -- 0x6014f00 con 0x4acb000 2026-02-11T23:31:27.664+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.665+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 621 ==== MOSDPGPush(1.0 116/113 [PushOp(1:7e5f2af9:::obj239:head, version: 111'239, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7e5f2af9:::obj239:head@111'239, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56cb040 con 0x4acb000 2026-02-11T23:31:27.665+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:7e5f2af9:::obj239:head, version: 111'239, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7e5f2af9:::obj239:head@111'239, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.665+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7e5f2af9:::obj239:head, version: 111'239, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7e5f2af9:::obj239:head@111'239, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.665+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.665+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 622 ==== pg_backfill(progress 1.0 e 116/116 lb 1:7ccaea2b:::obj486:head) ==== 1062+0+0 (unknown 0 0 0) 0x6143080 con 0x4acb000 2026-02-11T23:31:27.665+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7e5f2af9:::obj239:head, version: 111'239, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7e5f2af9:::obj239:head@111'239, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.665+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:7ccaea2b:::obj486:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.665+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.665+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7ccaea2b:::obj486:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.665+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7e5f2af9:::obj239:head, version: 111'239, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7e5f2af9:::obj239:head@111'239, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.665+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.665+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:7e5f2af9:::obj239:head, version: 111'239, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7e5f2af9:::obj239:head@111'239, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 116/113 [PushOp(1:7e5f2af9:::obj239:head, version: 111'239, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7e5f2af9:::obj239:head@111'239, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:7c955dfc:::obj322:head local-lis/les=113/114 n=292 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.665+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.665+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7c955dfc:::obj322:head local-lis/les=113/114 n=292 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:7e5f2af9:::obj239:head, version: 111'239, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7e5f2af9:::obj239:head@111'239, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.665+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7c955dfc:::obj322:head local-lis/les=113/114 n=292 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:7e5f2af9:::obj239:head@111'239, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.665+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.665+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7ccaea2b:::obj486:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.665+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7c955dfc:::obj322:head local-lis/les=113/114 n=292 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:7e5f2af9:::obj239:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.665+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7c955dfc:::obj322:head local-lis/les=113/114 n=292 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:7e5f2af9:::obj239:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.665+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7c955dfc:::obj322:head local-lis/les=113/114 n=292 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:7e5f2af9:::obj239:head 2026-02-11T23:31:27.665+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7c955dfc:::obj322:head local-lis/les=113/114 n=292 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:7e5f2af9:::obj239:head v 111'239 2026-02-11T23:31:27.665+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7c955dfc:::obj322:head local-lis/les=113/114 n=292 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.665+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.665+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:7e5f2af9:::obj239:head, version: 111'239, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7e5f2af9:::obj239:head@111'239, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.665+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.665+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.665+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7ccaea2b:::obj486:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.665+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:7ccaea2b:::obj486:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 116/116 lb 1:7ccaea2b:::obj486:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:7c955dfc:::obj322:head local-lis/les=113/114 n=292 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.665+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.665+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7c955dfc:::obj322:head local-lis/les=113/114 n=292 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:7ccaea2b:::obj486:head) v3 2026-02-11T23:31:27.665+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7c955dfc:::obj322:head local-lis/les=113/114 n=292 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:7ccaea2b:::obj486:head) 2026-02-11T23:31:27.665+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.665+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:7ccaea2b:::obj486:head) v3 finish latency 0.000011 2026-02-11T23:31:27.665+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ccaea2b:::obj486:head local-lis/les=113/114 n=293 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.665+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.665+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.665+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ccaea2b:::obj486:head local-lis/les=113/114 n=293 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.665+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.665+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.665+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.665+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fedc80 con 0x4acb000 2026-02-11T23:31:27.665+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:7e5f2af9:::obj239:head)]) -- 0x614d380 con 0x4acb000 2026-02-11T23:31:27.665+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.665+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 623 ==== MOSDPGPush(1.0 116/113 [PushOp(1:7eacede5:::obj24:head, version: 111'24, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7eacede5:::obj24:head@111'24, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x55d9380 con 0x4acb000 2026-02-11T23:31:27.665+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:7eacede5:::obj24:head, version: 111'24, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7eacede5:::obj24:head@111'24, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.665+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7eacede5:::obj24:head, version: 111'24, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7eacede5:::obj24:head@111'24, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.665+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.665+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 624 ==== pg_backfill(progress 1.0 e 116/116 lb 1:7e5f2af9:::obj239:head) ==== 1062+0+0 (unknown 0 0 0) 0x6143600 con 0x4acb000 2026-02-11T23:31:27.665+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:7e5f2af9:::obj239:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.665+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7e5f2af9:::obj239:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.665+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7eacede5:::obj24:head, version: 111'24, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7eacede5:::obj24:head@111'24, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.665+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.665+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7eacede5:::obj24:head, version: 111'24, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7eacede5:::obj24:head@111'24, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.665+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:7eacede5:::obj24:head, version: 111'24, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7eacede5:::obj24:head@111'24, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000040 MOSDPGPush(1.0 116/113 [PushOp(1:7eacede5:::obj24:head, version: 111'24, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7eacede5:::obj24:head@111'24, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:7ccaea2b:::obj486:head local-lis/les=113/114 n=293 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.665+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.665+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.665+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ccaea2b:::obj486:head local-lis/les=113/114 n=293 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:7eacede5:::obj24:head, version: 111'24, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7eacede5:::obj24:head@111'24, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.665+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7e5f2af9:::obj239:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.665+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ccaea2b:::obj486:head local-lis/les=113/114 n=293 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:7eacede5:::obj24:head@111'24, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.665+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ccaea2b:::obj486:head local-lis/les=113/114 n=293 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:7eacede5:::obj24:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.665+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ccaea2b:::obj486:head local-lis/les=113/114 n=293 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:7eacede5:::obj24:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.665+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ccaea2b:::obj486:head local-lis/les=113/114 n=293 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:7eacede5:::obj24:head 2026-02-11T23:31:27.665+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ccaea2b:::obj486:head local-lis/les=113/114 n=293 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:7eacede5:::obj24:head v 111'24 2026-02-11T23:31:27.665+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ccaea2b:::obj486:head local-lis/les=113/114 n=293 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.665+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.665+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:7eacede5:::obj24:head, version: 111'24, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7eacede5:::obj24:head@111'24, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.665+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.665+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.665+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7e5f2af9:::obj239:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.665+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:7e5f2af9:::obj239:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 1.0 e 116/116 lb 1:7e5f2af9:::obj239:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:7ccaea2b:::obj486:head local-lis/les=113/114 n=293 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.665+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.665+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ccaea2b:::obj486:head local-lis/les=113/114 n=293 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:7e5f2af9:::obj239:head) v3 2026-02-11T23:31:27.665+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ccaea2b:::obj486:head local-lis/les=113/114 n=293 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:7e5f2af9:::obj239:head) 2026-02-11T23:31:27.665+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.665+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:7e5f2af9:::obj239:head) v3 finish latency 0.000009 2026-02-11T23:31:27.665+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7e5f2af9:::obj239:head local-lis/les=113/114 n=294 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.665+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.666+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.666+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7e5f2af9:::obj239:head local-lis/les=113/114 n=294 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.666+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.666+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.666+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.666+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6014900 con 0x4acb000 2026-02-11T23:31:27.666+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:7eacede5:::obj24:head)]) -- 0x614d680 con 0x4acb000 2026-02-11T23:31:27.666+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.666+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 625 ==== MOSDPGPush(1.0 116/113 [PushOp(1:7ec5cfba:::obj597:head, version: 111'597, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ec5cfba:::obj597:head@111'597, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d8ea0 con 0x4acb000 2026-02-11T23:31:27.666+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:7ec5cfba:::obj597:head, version: 111'597, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ec5cfba:::obj597:head@111'597, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.666+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7ec5cfba:::obj597:head, version: 111'597, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ec5cfba:::obj597:head@111'597, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.666+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.666+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 626 ==== pg_backfill(progress 1.0 e 116/116 lb 1:7eacede5:::obj24:head) ==== 1061+0+0 (unknown 0 0 0) 0x6143b80 con 0x4acb000 2026-02-11T23:31:27.666+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7ec5cfba:::obj597:head, version: 111'597, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ec5cfba:::obj597:head@111'597, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.666+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:7eacede5:::obj24:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.666+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.666+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7eacede5:::obj24:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.666+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7ec5cfba:::obj597:head, version: 111'597, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ec5cfba:::obj597:head@111'597, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.666+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.666+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:7ec5cfba:::obj597:head, version: 111'597, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ec5cfba:::obj597:head@111'597, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:7ec5cfba:::obj597:head, version: 111'597, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ec5cfba:::obj597:head@111'597, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:7e5f2af9:::obj239:head local-lis/les=113/114 n=294 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.666+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.666+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.666+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7eacede5:::obj24:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.666+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7e5f2af9:::obj239:head local-lis/les=113/114 n=294 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:7ec5cfba:::obj597:head, version: 111'597, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ec5cfba:::obj597:head@111'597, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.666+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7e5f2af9:::obj239:head local-lis/les=113/114 n=294 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:7ec5cfba:::obj597:head@111'597, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.666+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7e5f2af9:::obj239:head local-lis/les=113/114 n=294 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:7ec5cfba:::obj597:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.666+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7e5f2af9:::obj239:head local-lis/les=113/114 n=294 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:7ec5cfba:::obj597:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.666+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7e5f2af9:::obj239:head local-lis/les=113/114 n=294 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:7ec5cfba:::obj597:head 2026-02-11T23:31:27.666+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7e5f2af9:::obj239:head local-lis/les=113/114 n=294 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:7ec5cfba:::obj597:head v 111'597 2026-02-11T23:31:27.666+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7e5f2af9:::obj239:head local-lis/les=113/114 n=294 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.666+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.666+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:7ec5cfba:::obj597:head, version: 111'597, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ec5cfba:::obj597:head@111'597, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:27.666+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.666+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.666+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7eacede5:::obj24:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.666+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:7eacede5:::obj24:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 1.0 e 116/116 lb 1:7eacede5:::obj24:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:7e5f2af9:::obj239:head local-lis/les=113/114 n=294 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.666+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.666+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7e5f2af9:::obj239:head local-lis/les=113/114 n=294 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:7eacede5:::obj24:head) v3 2026-02-11T23:31:27.666+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7e5f2af9:::obj239:head local-lis/les=113/114 n=294 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:7eacede5:::obj24:head) 2026-02-11T23:31:27.666+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.666+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:7eacede5:::obj24:head) v3 finish latency 0.000010 2026-02-11T23:31:27.666+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7eacede5:::obj24:head local-lis/les=113/114 n=295 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.666+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.666+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.666+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7eacede5:::obj24:head local-lis/les=113/114 n=295 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.666+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.666+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.666+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.666+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6005380 con 0x4acb000 2026-02-11T23:31:27.666+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:7ec5cfba:::obj597:head)]) -- 0x614d800 con 0x4acb000 2026-02-11T23:31:27.666+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.666+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 627 ==== MOSDPGPush(1.0 116/113 [PushOp(1:7ee98ed8:::obj216:head, version: 111'216, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ee98ed8:::obj216:head@111'216, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5647380 con 0x4acb000 2026-02-11T23:31:27.666+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:7ee98ed8:::obj216:head, version: 111'216, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ee98ed8:::obj216:head@111'216, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.666+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7ee98ed8:::obj216:head, version: 111'216, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ee98ed8:::obj216:head@111'216, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.666+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.666+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 628 ==== pg_backfill(progress 1.0 e 116/116 lb 1:7ec5cfba:::obj597:head) ==== 1062+0+0 (unknown 0 0 0) 0x6144100 con 0x4acb000 2026-02-11T23:31:27.666+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7ee98ed8:::obj216:head, version: 111'216, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ee98ed8:::obj216:head@111'216, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.666+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:7ec5cfba:::obj597:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.666+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.666+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7ec5cfba:::obj597:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.666+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7ee98ed8:::obj216:head, version: 111'216, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ee98ed8:::obj216:head@111'216, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.666+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.666+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:7ee98ed8:::obj216:head, version: 111'216, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ee98ed8:::obj216:head@111'216, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 116/113 [PushOp(1:7ee98ed8:::obj216:head, version: 111'216, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ee98ed8:::obj216:head@111'216, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:7eacede5:::obj24:head local-lis/les=113/114 n=295 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.666+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.666+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7eacede5:::obj24:head local-lis/les=113/114 n=295 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:7ee98ed8:::obj216:head, version: 111'216, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ee98ed8:::obj216:head@111'216, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.667+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.667+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7ec5cfba:::obj597:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.667+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7eacede5:::obj24:head local-lis/les=113/114 n=295 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:7ee98ed8:::obj216:head@111'216, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.667+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7eacede5:::obj24:head local-lis/les=113/114 n=295 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:7ee98ed8:::obj216:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.667+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7eacede5:::obj24:head local-lis/les=113/114 n=295 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:7ee98ed8:::obj216:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.667+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7eacede5:::obj24:head local-lis/les=113/114 n=295 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:7ee98ed8:::obj216:head 2026-02-11T23:31:27.667+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7eacede5:::obj24:head local-lis/les=113/114 n=295 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:7ee98ed8:::obj216:head v 111'216 2026-02-11T23:31:27.667+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7eacede5:::obj24:head local-lis/les=113/114 n=295 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.667+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.667+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:7ee98ed8:::obj216:head, version: 111'216, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ee98ed8:::obj216:head@111'216, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.667+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.667+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.667+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7ec5cfba:::obj597:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.667+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:7ec5cfba:::obj597:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 1.0 e 116/116 lb 1:7ec5cfba:::obj597:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:7eacede5:::obj24:head local-lis/les=113/114 n=295 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.667+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.667+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7eacede5:::obj24:head local-lis/les=113/114 n=295 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:7ec5cfba:::obj597:head) v3 2026-02-11T23:31:27.667+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7eacede5:::obj24:head local-lis/les=113/114 n=295 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:7ec5cfba:::obj597:head) 2026-02-11T23:31:27.667+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.667+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:7ec5cfba:::obj597:head) v3 finish latency 0.000010 2026-02-11T23:31:27.667+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ec5cfba:::obj597:head local-lis/les=113/114 n=296 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.667+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.667+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.667+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ec5cfba:::obj597:head local-lis/les=113/114 n=296 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.667+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.667+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.667+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.667+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6014780 con 0x4acb000 2026-02-11T23:31:27.667+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:7ee98ed8:::obj216:head)]) -- 0x614db00 con 0x4acb000 2026-02-11T23:31:27.667+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.667+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 629 ==== MOSDPGPush(1.0 116/113 [PushOp(1:7ee9cafa:::obj105:head, version: 111'105, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ee9cafa:::obj105:head@111'105, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d84e0 con 0x4acb000 2026-02-11T23:31:27.667+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:7ee9cafa:::obj105:head, version: 111'105, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ee9cafa:::obj105:head@111'105, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.667+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7ee9cafa:::obj105:head, version: 111'105, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ee9cafa:::obj105:head@111'105, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.667+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.667+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 630 ==== pg_backfill(progress 1.0 e 116/116 lb 1:7ee98ed8:::obj216:head) ==== 1062+0+0 (unknown 0 0 0) 0x6144680 con 0x4acb000 2026-02-11T23:31:27.667+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:7ee98ed8:::obj216:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.667+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7ee9cafa:::obj105:head, version: 111'105, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ee9cafa:::obj105:head@111'105, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.667+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7ee98ed8:::obj216:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.667+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.667+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7ee9cafa:::obj105:head, version: 111'105, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ee9cafa:::obj105:head@111'105, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.667+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:7ee9cafa:::obj105:head, version: 111'105, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ee9cafa:::obj105:head@111'105, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:7ee9cafa:::obj105:head, version: 111'105, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ee9cafa:::obj105:head@111'105, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:7ec5cfba:::obj597:head local-lis/les=113/114 n=296 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.667+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.667+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.667+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ec5cfba:::obj597:head local-lis/les=113/114 n=296 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:7ee9cafa:::obj105:head, version: 111'105, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ee9cafa:::obj105:head@111'105, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.667+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7ee98ed8:::obj216:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.667+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ec5cfba:::obj597:head local-lis/les=113/114 n=296 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:7ee9cafa:::obj105:head@111'105, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.667+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ec5cfba:::obj597:head local-lis/les=113/114 n=296 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:7ee9cafa:::obj105:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.667+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ec5cfba:::obj597:head local-lis/les=113/114 n=296 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:7ee9cafa:::obj105:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.667+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ec5cfba:::obj597:head local-lis/les=113/114 n=296 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:7ee9cafa:::obj105:head 2026-02-11T23:31:27.667+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ec5cfba:::obj597:head local-lis/les=113/114 n=296 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:7ee9cafa:::obj105:head v 111'105 2026-02-11T23:31:27.667+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ec5cfba:::obj597:head local-lis/les=113/114 n=296 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.667+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.667+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:7ee9cafa:::obj105:head, version: 111'105, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ee9cafa:::obj105:head@111'105, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:27.667+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.667+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.667+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7ee98ed8:::obj216:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.667+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:7ee98ed8:::obj216:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 116/116 lb 1:7ee98ed8:::obj216:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:7ec5cfba:::obj597:head local-lis/les=113/114 n=296 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.667+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.667+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ec5cfba:::obj597:head local-lis/les=113/114 n=296 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:7ee98ed8:::obj216:head) v3 2026-02-11T23:31:27.667+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ec5cfba:::obj597:head local-lis/les=113/114 n=296 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:7ee98ed8:::obj216:head) 2026-02-11T23:31:27.667+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.667+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:7ee98ed8:::obj216:head) v3 finish latency 0.000010 2026-02-11T23:31:27.667+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ee98ed8:::obj216:head local-lis/les=113/114 n=297 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.667+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.667+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.667+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ee98ed8:::obj216:head local-lis/les=113/114 n=297 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.667+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.667+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.667+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.667+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fce780 con 0x4acb000 2026-02-11T23:31:27.667+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:7ee9cafa:::obj105:head)]) -- 0x614de00 con 0x4acb000 2026-02-11T23:31:27.667+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.668+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 631 ==== MOSDPGPush(1.0 116/113 [PushOp(1:7f12fc77:::obj195:head, version: 111'195, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f12fc77:::obj195:head@111'195, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d8000 con 0x4acb000 2026-02-11T23:31:27.668+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:7f12fc77:::obj195:head, version: 111'195, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f12fc77:::obj195:head@111'195, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.668+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7f12fc77:::obj195:head, version: 111'195, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f12fc77:::obj195:head@111'195, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.668+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.668+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 632 ==== pg_backfill(progress 1.0 e 116/116 lb 1:7ee9cafa:::obj105:head) ==== 1062+0+0 (unknown 0 0 0) 0x6144c00 con 0x4acb000 2026-02-11T23:31:27.668+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7f12fc77:::obj195:head, version: 111'195, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f12fc77:::obj195:head@111'195, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.668+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:7ee9cafa:::obj105:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.668+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.668+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7ee9cafa:::obj105:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.668+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7f12fc77:::obj195:head, version: 111'195, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f12fc77:::obj195:head@111'195, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.668+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.668+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:7f12fc77:::obj195:head, version: 111'195, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f12fc77:::obj195:head@111'195, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:7f12fc77:::obj195:head, version: 111'195, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f12fc77:::obj195:head@111'195, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:7ee98ed8:::obj216:head local-lis/les=113/114 n=297 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.668+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.668+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ee98ed8:::obj216:head local-lis/les=113/114 n=297 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:7f12fc77:::obj195:head, version: 111'195, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f12fc77:::obj195:head@111'195, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.668+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ee98ed8:::obj216:head local-lis/les=113/114 n=297 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:7f12fc77:::obj195:head@111'195, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.668+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.668+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7ee9cafa:::obj105:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.668+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ee98ed8:::obj216:head local-lis/les=113/114 n=297 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:7f12fc77:::obj195:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.668+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ee98ed8:::obj216:head local-lis/les=113/114 n=297 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:7f12fc77:::obj195:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.668+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ee98ed8:::obj216:head local-lis/les=113/114 n=297 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:7f12fc77:::obj195:head 2026-02-11T23:31:27.668+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ee98ed8:::obj216:head local-lis/les=113/114 n=297 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:7f12fc77:::obj195:head v 111'195 2026-02-11T23:31:27.668+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ee98ed8:::obj216:head local-lis/les=113/114 n=297 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.668+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.668+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:7f12fc77:::obj195:head, version: 111'195, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f12fc77:::obj195:head@111'195, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:27.668+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.668+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.668+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7ee9cafa:::obj105:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.668+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:7ee9cafa:::obj105:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 1.0 e 116/116 lb 1:7ee9cafa:::obj105:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:7ee98ed8:::obj216:head local-lis/les=113/114 n=297 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.668+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.668+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ee98ed8:::obj216:head local-lis/les=113/114 n=297 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:7ee9cafa:::obj105:head) v3 2026-02-11T23:31:27.668+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ee98ed8:::obj216:head local-lis/les=113/114 n=297 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:7ee9cafa:::obj105:head) 2026-02-11T23:31:27.668+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.668+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:7ee9cafa:::obj105:head) v3 finish latency 0.000009 2026-02-11T23:31:27.668+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ee9cafa:::obj105:head local-lis/les=113/114 n=298 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.668+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.668+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.668+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ee9cafa:::obj105:head local-lis/les=113/114 n=298 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.668+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.668+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.668+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.668+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6014480 con 0x4acb000 2026-02-11T23:31:27.668+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:7f12fc77:::obj195:head)]) -- 0x615c180 con 0x4acb000 2026-02-11T23:31:27.668+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.668+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 633 ==== MOSDPGPush(1.0 116/113 [PushOp(1:7f1dde7e:::obj491:head, version: 111'491, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f1dde7e:::obj491:head@111'491, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d89c0 con 0x4acb000 2026-02-11T23:31:27.668+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:7f1dde7e:::obj491:head, version: 111'491, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f1dde7e:::obj491:head@111'491, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.668+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7f1dde7e:::obj491:head, version: 111'491, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f1dde7e:::obj491:head@111'491, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.668+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.668+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 634 ==== pg_backfill(progress 1.0 e 116/116 lb 1:7f12fc77:::obj195:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2d080 con 0x4acb000 2026-02-11T23:31:27.668+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:7f12fc77:::obj195:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.668+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7f1dde7e:::obj491:head, version: 111'491, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f1dde7e:::obj491:head@111'491, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.668+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7f12fc77:::obj195:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.668+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.668+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7f12fc77:::obj195:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.668+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.668+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7f1dde7e:::obj491:head, version: 111'491, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f1dde7e:::obj491:head@111'491, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.668+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:7f1dde7e:::obj491:head, version: 111'491, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f1dde7e:::obj491:head@111'491, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(1.0 116/113 [PushOp(1:7f1dde7e:::obj491:head, version: 111'491, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f1dde7e:::obj491:head@111'491, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:7ee9cafa:::obj105:head local-lis/les=113/114 n=298 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.668+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.668+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ee9cafa:::obj105:head local-lis/les=113/114 n=298 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:7f1dde7e:::obj491:head, version: 111'491, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f1dde7e:::obj491:head@111'491, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.668+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ee9cafa:::obj105:head local-lis/les=113/114 n=298 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:7f1dde7e:::obj491:head@111'491, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.668+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ee9cafa:::obj105:head local-lis/les=113/114 n=298 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:7f1dde7e:::obj491:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.668+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ee9cafa:::obj105:head local-lis/les=113/114 n=298 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:7f1dde7e:::obj491:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.668+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ee9cafa:::obj105:head local-lis/les=113/114 n=298 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:7f1dde7e:::obj491:head 2026-02-11T23:31:27.668+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ee9cafa:::obj105:head local-lis/les=113/114 n=298 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:7f1dde7e:::obj491:head v 111'491 2026-02-11T23:31:27.668+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ee9cafa:::obj105:head local-lis/les=113/114 n=298 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.668+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.668+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:7f1dde7e:::obj491:head, version: 111'491, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f1dde7e:::obj491:head@111'491, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000044 2026-02-11T23:31:27.668+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.668+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.668+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7f12fc77:::obj195:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.668+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:7f12fc77:::obj195:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 1.0 e 116/116 lb 1:7f12fc77:::obj195:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:7ee9cafa:::obj105:head local-lis/les=113/114 n=298 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.668+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.668+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ee9cafa:::obj105:head local-lis/les=113/114 n=298 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:7f12fc77:::obj195:head) v3 2026-02-11T23:31:27.668+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7ee9cafa:::obj105:head local-lis/les=113/114 n=298 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:7f12fc77:::obj195:head) 2026-02-11T23:31:27.668+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.668+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:7f12fc77:::obj195:head) v3 finish latency 0.000009 2026-02-11T23:31:27.668+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7f12fc77:::obj195:head local-lis/les=113/114 n=299 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.668+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.669+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.669+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7f12fc77:::obj195:head local-lis/les=113/114 n=299 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.669+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.669+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.669+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.669+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6005500 con 0x4acb000 2026-02-11T23:31:27.669+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:7f1dde7e:::obj491:head)]) -- 0x615c480 con 0x4acb000 2026-02-11T23:31:27.669+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.669+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 635 ==== MOSDPGPush(1.0 116/113 [PushOp(1:7f2a7765:::obj252:head, version: 111'252, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f2a7765:::obj252:head@111'252, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559d040 con 0x4acb000 2026-02-11T23:31:27.669+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:7f2a7765:::obj252:head, version: 111'252, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f2a7765:::obj252:head@111'252, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.669+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7f2a7765:::obj252:head, version: 111'252, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f2a7765:::obj252:head@111'252, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.669+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.669+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 636 ==== pg_backfill(progress 1.0 e 116/116 lb 1:7f1dde7e:::obj491:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e680 con 0x4acb000 2026-02-11T23:31:27.669+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:7f1dde7e:::obj491:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.669+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7f2a7765:::obj252:head, version: 111'252, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f2a7765:::obj252:head@111'252, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.669+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7f1dde7e:::obj491:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.669+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.669+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7f2a7765:::obj252:head, version: 111'252, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f2a7765:::obj252:head@111'252, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.669+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:7f2a7765:::obj252:head, version: 111'252, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f2a7765:::obj252:head@111'252, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 116/113 [PushOp(1:7f2a7765:::obj252:head, version: 111'252, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f2a7765:::obj252:head@111'252, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:7f12fc77:::obj195:head local-lis/les=113/114 n=299 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.669+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.669+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.669+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7f12fc77:::obj195:head local-lis/les=113/114 n=299 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:7f2a7765:::obj252:head, version: 111'252, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f2a7765:::obj252:head@111'252, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.669+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7f1dde7e:::obj491:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.669+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7f12fc77:::obj195:head local-lis/les=113/114 n=299 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:7f2a7765:::obj252:head@111'252, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.669+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7f12fc77:::obj195:head local-lis/les=113/114 n=299 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:7f2a7765:::obj252:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.669+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7f12fc77:::obj195:head local-lis/les=113/114 n=299 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:7f2a7765:::obj252:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.669+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7f12fc77:::obj195:head local-lis/les=113/114 n=299 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:7f2a7765:::obj252:head 2026-02-11T23:31:27.669+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7f12fc77:::obj195:head local-lis/les=113/114 n=299 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:7f2a7765:::obj252:head v 111'252 2026-02-11T23:31:27.669+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7f12fc77:::obj195:head local-lis/les=113/114 n=299 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.669+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.669+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:7f2a7765:::obj252:head, version: 111'252, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f2a7765:::obj252:head@111'252, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.669+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.669+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.669+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7f1dde7e:::obj491:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.669+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:7f1dde7e:::obj491:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:7f1dde7e:::obj491:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:7f12fc77:::obj195:head local-lis/les=113/114 n=299 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.669+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.669+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7f12fc77:::obj195:head local-lis/les=113/114 n=299 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:7f1dde7e:::obj491:head) v3 2026-02-11T23:31:27.669+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7f12fc77:::obj195:head local-lis/les=113/114 n=299 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:7f1dde7e:::obj491:head) 2026-02-11T23:31:27.669+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.669+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:7f1dde7e:::obj491:head) v3 finish latency 0.000008 2026-02-11T23:31:27.669+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7f1dde7e:::obj491:head local-lis/les=113/114 n=300 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.669+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.669+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.669+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7f1dde7e:::obj491:head local-lis/les=113/114 n=300 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.669+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.669+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.669+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.669+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6014300 con 0x4acb000 2026-02-11T23:31:27.669+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:7f2a7765:::obj252:head)]) -- 0x615c780 con 0x4acb000 2026-02-11T23:31:27.669+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.669+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 637 ==== MOSDPGPush(1.0 116/113 [PushOp(1:7fffd9c5:::obj553:head, version: 111'553, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7fffd9c5:::obj553:head@111'553, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559dba0 con 0x4acb000 2026-02-11T23:31:27.669+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:7fffd9c5:::obj553:head, version: 111'553, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7fffd9c5:::obj553:head@111'553, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.669+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7fffd9c5:::obj553:head, version: 111'553, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7fffd9c5:::obj553:head@111'553, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.669+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.669+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 638 ==== pg_backfill(progress 1.0 e 116/116 lb 1:7f2a7765:::obj252:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e100 con 0x4acb000 2026-02-11T23:31:27.669+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7fffd9c5:::obj553:head, version: 111'553, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7fffd9c5:::obj553:head@111'553, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.669+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:7f2a7765:::obj252:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.669+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.669+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7f2a7765:::obj252:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.669+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:7fffd9c5:::obj553:head, version: 111'553, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7fffd9c5:::obj553:head@111'553, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.669+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.669+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:7fffd9c5:::obj553:head, version: 111'553, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7fffd9c5:::obj553:head@111'553, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:7fffd9c5:::obj553:head, version: 111'553, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7fffd9c5:::obj553:head@111'553, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:7f1dde7e:::obj491:head local-lis/les=113/114 n=300 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.669+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.669+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7f1dde7e:::obj491:head local-lis/les=113/114 n=300 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:7fffd9c5:::obj553:head, version: 111'553, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7fffd9c5:::obj553:head@111'553, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.669+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7f1dde7e:::obj491:head local-lis/les=113/114 n=300 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:7fffd9c5:::obj553:head@111'553, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.669+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.669+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7f2a7765:::obj252:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.669+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7f1dde7e:::obj491:head local-lis/les=113/114 n=300 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:7fffd9c5:::obj553:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.670+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7f1dde7e:::obj491:head local-lis/les=113/114 n=300 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:7fffd9c5:::obj553:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.670+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7f1dde7e:::obj491:head local-lis/les=113/114 n=300 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:7fffd9c5:::obj553:head 2026-02-11T23:31:27.670+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7f1dde7e:::obj491:head local-lis/les=113/114 n=300 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:7fffd9c5:::obj553:head v 111'553 2026-02-11T23:31:27.670+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7f1dde7e:::obj491:head local-lis/les=113/114 n=300 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.670+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.670+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:7fffd9c5:::obj553:head, version: 111'553, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7fffd9c5:::obj553:head@111'553, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.670+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.670+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.670+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7f2a7765:::obj252:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.670+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:7f2a7765:::obj252:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 1.0 e 116/116 lb 1:7f2a7765:::obj252:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:7f1dde7e:::obj491:head local-lis/les=113/114 n=300 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.670+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.670+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7f1dde7e:::obj491:head local-lis/les=113/114 n=300 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:7f2a7765:::obj252:head) v3 2026-02-11T23:31:27.670+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7f1dde7e:::obj491:head local-lis/les=113/114 n=300 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:7f2a7765:::obj252:head) 2026-02-11T23:31:27.670+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.670+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:7f2a7765:::obj252:head) v3 finish latency 0.000008 2026-02-11T23:31:27.670+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7f2a7765:::obj252:head local-lis/les=113/114 n=301 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.670+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.670+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.670+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7f2a7765:::obj252:head local-lis/les=113/114 n=301 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.670+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.670+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.670+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.670+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fedb00 con 0x4acb000 2026-02-11T23:31:27.670+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:7fffd9c5:::obj553:head)]) -- 0x615ca80 con 0x4acb000 2026-02-11T23:31:27.670+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.670+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 639 ==== MOSDPGPush(1.0 116/113 [PushOp(1:81617860:::obj28:head, version: 111'28, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:81617860:::obj28:head@111'28, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x559c680 con 0x4acb000 2026-02-11T23:31:27.670+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:81617860:::obj28:head, version: 111'28, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:81617860:::obj28:head@111'28, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.670+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:81617860:::obj28:head, version: 111'28, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:81617860:::obj28:head@111'28, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.670+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.670+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 640 ==== pg_backfill(progress 1.0 e 116/116 lb 1:7fffd9c5:::obj553:head) ==== 1062+0+0 (unknown 0 0 0) 0x4a2cb00 con 0x4acb000 2026-02-11T23:31:27.670+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:7fffd9c5:::obj553:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.670+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:81617860:::obj28:head, version: 111'28, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:81617860:::obj28:head@111'28, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.670+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7fffd9c5:::obj553:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.670+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.670+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:81617860:::obj28:head, version: 111'28, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:81617860:::obj28:head@111'28, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.670+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:81617860:::obj28:head, version: 111'28, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:81617860:::obj28:head@111'28, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(1.0 116/113 [PushOp(1:81617860:::obj28:head, version: 111'28, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:81617860:::obj28:head@111'28, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:7f2a7765:::obj252:head local-lis/les=113/114 n=301 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.670+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.670+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.670+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7f2a7765:::obj252:head local-lis/les=113/114 n=301 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:81617860:::obj28:head, version: 111'28, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:81617860:::obj28:head@111'28, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.670+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7fffd9c5:::obj553:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.670+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7f2a7765:::obj252:head local-lis/les=113/114 n=301 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:81617860:::obj28:head@111'28, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.670+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7f2a7765:::obj252:head local-lis/les=113/114 n=301 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:81617860:::obj28:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.670+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7f2a7765:::obj252:head local-lis/les=113/114 n=301 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:81617860:::obj28:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.670+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7f2a7765:::obj252:head local-lis/les=113/114 n=301 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:81617860:::obj28:head 2026-02-11T23:31:27.670+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7f2a7765:::obj252:head local-lis/les=113/114 n=301 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:81617860:::obj28:head v 111'28 2026-02-11T23:31:27.670+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7f2a7765:::obj252:head local-lis/les=113/114 n=301 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.670+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.670+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:81617860:::obj28:head, version: 111'28, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:81617860:::obj28:head@111'28, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.670+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.670+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.670+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:7fffd9c5:::obj553:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.670+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:7fffd9c5:::obj553:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:7fffd9c5:::obj553:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:7f2a7765:::obj252:head local-lis/les=113/114 n=301 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.670+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.670+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7f2a7765:::obj252:head local-lis/les=113/114 n=301 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:7fffd9c5:::obj553:head) v3 2026-02-11T23:31:27.670+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7f2a7765:::obj252:head local-lis/les=113/114 n=301 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:7fffd9c5:::obj553:head) 2026-02-11T23:31:27.670+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.670+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:7fffd9c5:::obj553:head) v3 finish latency 0.000008 2026-02-11T23:31:27.670+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7fffd9c5:::obj553:head local-lis/les=113/114 n=302 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.670+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.670+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.670+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7fffd9c5:::obj553:head local-lis/les=113/114 n=302 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.670+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.670+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.670+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.670+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6014000 con 0x4acb000 2026-02-11T23:31:27.670+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:81617860:::obj28:head)]) -- 0x615cd80 con 0x4acb000 2026-02-11T23:31:27.670+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.671+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 641 ==== MOSDPGPush(1.0 116/113 [PushOp(1:81fa42e1:::obj447:head, version: 111'447, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:81fa42e1:::obj447:head@111'447, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f16c0 con 0x4acb000 2026-02-11T23:31:27.671+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:81fa42e1:::obj447:head, version: 111'447, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:81fa42e1:::obj447:head@111'447, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.671+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:81fa42e1:::obj447:head, version: 111'447, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:81fa42e1:::obj447:head@111'447, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.671+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.671+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 642 ==== pg_backfill(progress 1.0 e 116/116 lb 1:81617860:::obj28:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d99b80 con 0x4acb000 2026-02-11T23:31:27.671+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:81617860:::obj28:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.671+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:81fa42e1:::obj447:head, version: 111'447, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:81fa42e1:::obj447:head@111'447, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.671+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.671+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:81617860:::obj28:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.671+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:81fa42e1:::obj447:head, version: 111'447, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:81fa42e1:::obj447:head@111'447, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.671+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.671+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:81617860:::obj28:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.671+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:81fa42e1:::obj447:head, version: 111'447, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:81fa42e1:::obj447:head@111'447, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 116/113 [PushOp(1:81fa42e1:::obj447:head, version: 111'447, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:81fa42e1:::obj447:head@111'447, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:7fffd9c5:::obj553:head local-lis/les=113/114 n=302 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.671+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.671+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7fffd9c5:::obj553:head local-lis/les=113/114 n=302 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:81fa42e1:::obj447:head, version: 111'447, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:81fa42e1:::obj447:head@111'447, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.671+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7fffd9c5:::obj553:head local-lis/les=113/114 n=302 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:81fa42e1:::obj447:head@111'447, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.671+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7fffd9c5:::obj553:head local-lis/les=113/114 n=302 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:81fa42e1:::obj447:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.671+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7fffd9c5:::obj553:head local-lis/les=113/114 n=302 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:81fa42e1:::obj447:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.671+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7fffd9c5:::obj553:head local-lis/les=113/114 n=302 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:81fa42e1:::obj447:head 2026-02-11T23:31:27.671+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7fffd9c5:::obj553:head local-lis/les=113/114 n=302 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:81fa42e1:::obj447:head v 111'447 2026-02-11T23:31:27.671+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7fffd9c5:::obj553:head local-lis/les=113/114 n=302 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.671+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.671+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:81fa42e1:::obj447:head, version: 111'447, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:81fa42e1:::obj447:head@111'447, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.671+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.671+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.671+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:81617860:::obj28:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.671+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:81617860:::obj28:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:81617860:::obj28:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:7fffd9c5:::obj553:head local-lis/les=113/114 n=302 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.671+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.671+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7fffd9c5:::obj553:head local-lis/les=113/114 n=302 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:81617860:::obj28:head) v3 2026-02-11T23:31:27.671+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:7fffd9c5:::obj553:head local-lis/les=113/114 n=302 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:81617860:::obj28:head) 2026-02-11T23:31:27.671+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.671+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:81617860:::obj28:head) v3 finish latency 0.000009 2026-02-11T23:31:27.671+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:81617860:::obj28:head local-lis/les=113/114 n=303 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.671+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.671+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.671+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:81617860:::obj28:head local-lis/les=113/114 n=303 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.671+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.671+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.671+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.671+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6005800 con 0x4acb000 2026-02-11T23:31:27.671+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:81fa42e1:::obj447:head)]) -- 0x615cf00 con 0x4acb000 2026-02-11T23:31:27.671+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.671+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 643 ==== MOSDPGPush(1.0 116/113 [PushOp(1:8205b5d8:::obj400:head, version: 111'400, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8205b5d8:::obj400:head@111'400, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f0b60 con 0x4acb000 2026-02-11T23:31:27.671+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:8205b5d8:::obj400:head, version: 111'400, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8205b5d8:::obj400:head@111'400, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.671+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8205b5d8:::obj400:head, version: 111'400, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8205b5d8:::obj400:head@111'400, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.671+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.671+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 644 ==== pg_backfill(progress 1.0 e 116/116 lb 1:81fa42e1:::obj447:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9a100 con 0x4acb000 2026-02-11T23:31:27.671+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8205b5d8:::obj400:head, version: 111'400, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8205b5d8:::obj400:head@111'400, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.671+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:81fa42e1:::obj447:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.671+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:81fa42e1:::obj447:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.671+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.671+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8205b5d8:::obj400:head, version: 111'400, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8205b5d8:::obj400:head@111'400, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.671+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8205b5d8:::obj400:head, version: 111'400, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8205b5d8:::obj400:head@111'400, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:8205b5d8:::obj400:head, version: 111'400, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8205b5d8:::obj400:head@111'400, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:81617860:::obj28:head local-lis/les=113/114 n=303 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.671+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.671+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.671+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:81fa42e1:::obj447:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.671+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:81617860:::obj28:head local-lis/les=113/114 n=303 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:8205b5d8:::obj400:head, version: 111'400, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8205b5d8:::obj400:head@111'400, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.671+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:81617860:::obj28:head local-lis/les=113/114 n=303 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8205b5d8:::obj400:head@111'400, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.671+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:81617860:::obj28:head local-lis/les=113/114 n=303 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:8205b5d8:::obj400:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.671+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:81617860:::obj28:head local-lis/les=113/114 n=303 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:8205b5d8:::obj400:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.671+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:81617860:::obj28:head local-lis/les=113/114 n=303 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:8205b5d8:::obj400:head 2026-02-11T23:31:27.671+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:81617860:::obj28:head local-lis/les=113/114 n=303 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:8205b5d8:::obj400:head v 111'400 2026-02-11T23:31:27.671+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:81617860:::obj28:head local-lis/les=113/114 n=303 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.671+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.671+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8205b5d8:::obj400:head, version: 111'400, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8205b5d8:::obj400:head@111'400, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:27.671+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.671+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.671+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:81fa42e1:::obj447:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.671+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:81fa42e1:::obj447:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 1.0 e 116/116 lb 1:81fa42e1:::obj447:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:81617860:::obj28:head local-lis/les=113/114 n=303 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.671+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.671+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:81617860:::obj28:head local-lis/les=113/114 n=303 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:81fa42e1:::obj447:head) v3 2026-02-11T23:31:27.671+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:81617860:::obj28:head local-lis/les=113/114 n=303 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:81fa42e1:::obj447:head) 2026-02-11T23:31:27.671+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.671+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:81fa42e1:::obj447:head) v3 finish latency 0.000009 2026-02-11T23:31:27.671+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:81fa42e1:::obj447:head local-lis/les=113/114 n=304 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.671+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.672+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.672+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:81fa42e1:::obj447:head local-lis/les=113/114 n=304 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.672+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.672+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.672+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.672+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x612fc80 con 0x4acb000 2026-02-11T23:31:27.672+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:8205b5d8:::obj400:head)]) -- 0x615d200 con 0x4acb000 2026-02-11T23:31:27.672+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.672+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 645 ==== MOSDPGPush(1.0 116/113 [PushOp(1:8250b831:::obj124:head, version: 111'124, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8250b831:::obj124:head@111'124, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f1d40 con 0x4acb000 2026-02-11T23:31:27.672+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:8250b831:::obj124:head, version: 111'124, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8250b831:::obj124:head@111'124, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.672+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8250b831:::obj124:head, version: 111'124, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8250b831:::obj124:head@111'124, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.672+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.672+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 646 ==== pg_backfill(progress 1.0 e 116/116 lb 1:8205b5d8:::obj400:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9a680 con 0x4acb000 2026-02-11T23:31:27.672+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:8205b5d8:::obj400:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.672+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8250b831:::obj124:head, version: 111'124, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8250b831:::obj124:head@111'124, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.672+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8205b5d8:::obj400:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.672+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.672+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8250b831:::obj124:head, version: 111'124, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8250b831:::obj124:head@111'124, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.672+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8250b831:::obj124:head, version: 111'124, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8250b831:::obj124:head@111'124, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:8250b831:::obj124:head, version: 111'124, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8250b831:::obj124:head@111'124, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:81fa42e1:::obj447:head local-lis/les=113/114 n=304 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.672+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.672+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.672+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:81fa42e1:::obj447:head local-lis/les=113/114 n=304 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:8250b831:::obj124:head, version: 111'124, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8250b831:::obj124:head@111'124, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.672+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8205b5d8:::obj400:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.672+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:81fa42e1:::obj447:head local-lis/les=113/114 n=304 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8250b831:::obj124:head@111'124, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.672+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:81fa42e1:::obj447:head local-lis/les=113/114 n=304 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:8250b831:::obj124:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.672+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:81fa42e1:::obj447:head local-lis/les=113/114 n=304 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:8250b831:::obj124:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.672+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:81fa42e1:::obj447:head local-lis/les=113/114 n=304 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:8250b831:::obj124:head 2026-02-11T23:31:27.672+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:81fa42e1:::obj447:head local-lis/les=113/114 n=304 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:8250b831:::obj124:head v 111'124 2026-02-11T23:31:27.672+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:81fa42e1:::obj447:head local-lis/les=113/114 n=304 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.672+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.672+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8250b831:::obj124:head, version: 111'124, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8250b831:::obj124:head@111'124, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.672+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.672+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.672+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8205b5d8:::obj400:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.672+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8205b5d8:::obj400:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 116/116 lb 1:8205b5d8:::obj400:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:81fa42e1:::obj447:head local-lis/les=113/114 n=304 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.672+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.672+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:81fa42e1:::obj447:head local-lis/les=113/114 n=304 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:8205b5d8:::obj400:head) v3 2026-02-11T23:31:27.672+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:81fa42e1:::obj447:head local-lis/les=113/114 n=304 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:8205b5d8:::obj400:head) 2026-02-11T23:31:27.672+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.672+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8205b5d8:::obj400:head) v3 finish latency 0.000008 2026-02-11T23:31:27.672+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8205b5d8:::obj400:head local-lis/les=113/114 n=305 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.672+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.672+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.672+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8205b5d8:::obj400:head local-lis/les=113/114 n=305 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.672+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.672+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.672+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.672+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fce480 con 0x4acb000 2026-02-11T23:31:27.672+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:8250b831:::obj124:head)]) -- 0x615d500 con 0x4acb000 2026-02-11T23:31:27.672+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.673+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 647 ==== MOSDPGPush(1.0 116/113 [PushOp(1:8260b93f:::obj338:head, version: 111'338, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8260b93f:::obj338:head@111'338, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f11e0 con 0x4acb000 2026-02-11T23:31:27.673+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:8260b93f:::obj338:head, version: 111'338, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8260b93f:::obj338:head@111'338, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.673+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8260b93f:::obj338:head, version: 111'338, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8260b93f:::obj338:head@111'338, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.673+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.673+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 648 ==== pg_backfill(progress 1.0 e 116/116 lb 1:8250b831:::obj124:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9ac00 con 0x4acb000 2026-02-11T23:31:27.673+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8260b93f:::obj338:head, version: 111'338, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8260b93f:::obj338:head@111'338, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.673+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:8250b831:::obj124:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.673+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8250b831:::obj124:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.673+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.673+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8260b93f:::obj338:head, version: 111'338, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8260b93f:::obj338:head@111'338, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.673+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8260b93f:::obj338:head, version: 111'338, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8260b93f:::obj338:head@111'338, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 116/113 [PushOp(1:8260b93f:::obj338:head, version: 111'338, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8260b93f:::obj338:head@111'338, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8205b5d8:::obj400:head local-lis/les=113/114 n=305 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.673+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.673+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.673+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8205b5d8:::obj400:head local-lis/les=113/114 n=305 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:8260b93f:::obj338:head, version: 111'338, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8260b93f:::obj338:head@111'338, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.673+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8250b831:::obj124:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.673+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8205b5d8:::obj400:head local-lis/les=113/114 n=305 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8260b93f:::obj338:head@111'338, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.673+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8205b5d8:::obj400:head local-lis/les=113/114 n=305 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:8260b93f:::obj338:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.673+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8205b5d8:::obj400:head local-lis/les=113/114 n=305 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:8260b93f:::obj338:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.673+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8205b5d8:::obj400:head local-lis/les=113/114 n=305 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:8260b93f:::obj338:head 2026-02-11T23:31:27.673+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8205b5d8:::obj400:head local-lis/les=113/114 n=305 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:8260b93f:::obj338:head v 111'338 2026-02-11T23:31:27.673+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8205b5d8:::obj400:head local-lis/les=113/114 n=305 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.673+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.673+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8260b93f:::obj338:head, version: 111'338, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8260b93f:::obj338:head@111'338, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.673+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.673+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.673+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8250b831:::obj124:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.673+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8250b831:::obj124:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 116/116 lb 1:8250b831:::obj124:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8205b5d8:::obj400:head local-lis/les=113/114 n=305 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.673+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.673+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8205b5d8:::obj400:head local-lis/les=113/114 n=305 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:8250b831:::obj124:head) v3 2026-02-11T23:31:27.673+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8205b5d8:::obj400:head local-lis/les=113/114 n=305 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:8250b831:::obj124:head) 2026-02-11T23:31:27.673+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.673+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8250b831:::obj124:head) v3 finish latency 0.000010 2026-02-11T23:31:27.673+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8250b831:::obj124:head local-lis/les=113/114 n=306 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.673+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.673+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.673+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8250b831:::obj124:head local-lis/les=113/114 n=306 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.673+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.673+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.673+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.673+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x612f980 con 0x4acb000 2026-02-11T23:31:27.673+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:8260b93f:::obj338:head)]) -- 0x615d800 con 0x4acb000 2026-02-11T23:31:27.673+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.673+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 649 ==== MOSDPGPush(1.0 116/113 [PushOp(1:8280535c:::obj236:head, version: 111'236, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8280535c:::obj236:head@111'236, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559cb60 con 0x4acb000 2026-02-11T23:31:27.673+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:8280535c:::obj236:head, version: 111'236, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8280535c:::obj236:head@111'236, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.673+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8280535c:::obj236:head, version: 111'236, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8280535c:::obj236:head@111'236, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.673+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.673+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 650 ==== pg_backfill(progress 1.0 e 116/116 lb 1:8260b93f:::obj338:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9b180 con 0x4acb000 2026-02-11T23:31:27.673+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:8260b93f:::obj338:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.673+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8280535c:::obj236:head, version: 111'236, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8280535c:::obj236:head@111'236, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.673+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8260b93f:::obj338:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.673+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.673+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8280535c:::obj236:head, version: 111'236, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8280535c:::obj236:head@111'236, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.673+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8280535c:::obj236:head, version: 111'236, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8280535c:::obj236:head@111'236, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 116/113 [PushOp(1:8280535c:::obj236:head, version: 111'236, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8280535c:::obj236:head@111'236, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8250b831:::obj124:head local-lis/les=113/114 n=306 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.673+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.673+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.673+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8250b831:::obj124:head local-lis/les=113/114 n=306 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:8280535c:::obj236:head, version: 111'236, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8280535c:::obj236:head@111'236, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.673+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8260b93f:::obj338:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.673+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8250b831:::obj124:head local-lis/les=113/114 n=306 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8280535c:::obj236:head@111'236, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.673+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8250b831:::obj124:head local-lis/les=113/114 n=306 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:8280535c:::obj236:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.673+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8250b831:::obj124:head local-lis/les=113/114 n=306 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:8280535c:::obj236:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.673+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8250b831:::obj124:head local-lis/les=113/114 n=306 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:8280535c:::obj236:head 2026-02-11T23:31:27.673+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8250b831:::obj124:head local-lis/les=113/114 n=306 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:8280535c:::obj236:head v 111'236 2026-02-11T23:31:27.673+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8250b831:::obj124:head local-lis/les=113/114 n=306 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.673+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.673+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8280535c:::obj236:head, version: 111'236, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8280535c:::obj236:head@111'236, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.673+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.673+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.673+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8260b93f:::obj338:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.673+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8260b93f:::obj338:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 1.0 e 116/116 lb 1:8260b93f:::obj338:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8250b831:::obj124:head local-lis/les=113/114 n=306 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.673+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.673+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8250b831:::obj124:head local-lis/les=113/114 n=306 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:8260b93f:::obj338:head) v3 2026-02-11T23:31:27.673+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8250b831:::obj124:head local-lis/les=113/114 n=306 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:8260b93f:::obj338:head) 2026-02-11T23:31:27.673+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.673+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8260b93f:::obj338:head) v3 finish latency 0.000009 2026-02-11T23:31:27.673+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8260b93f:::obj338:head local-lis/les=113/114 n=307 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.673+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.673+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.673+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8260b93f:::obj338:head local-lis/les=113/114 n=307 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.673+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.673+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.673+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.673+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6005b00 con 0x4acb000 2026-02-11T23:31:27.673+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:8280535c:::obj236:head)]) -- 0x615db00 con 0x4acb000 2026-02-11T23:31:27.673+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.674+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 651 ==== MOSDPGPush(1.0 116/113 [PushOp(1:8421ae49:::obj196:head, version: 111'196, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8421ae49:::obj196:head@111'196, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f1ba0 con 0x4acb000 2026-02-11T23:31:27.674+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:8421ae49:::obj196:head, version: 111'196, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8421ae49:::obj196:head@111'196, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.674+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8421ae49:::obj196:head, version: 111'196, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8421ae49:::obj196:head@111'196, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.674+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.674+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8421ae49:::obj196:head, version: 111'196, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8421ae49:::obj196:head@111'196, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.674+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.674+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8421ae49:::obj196:head, version: 111'196, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8421ae49:::obj196:head@111'196, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.674+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 652 ==== pg_backfill(progress 1.0 e 116/116 lb 1:8280535c:::obj236:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98000 con 0x4acb000 2026-02-11T23:31:27.674+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:8280535c:::obj236:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.674+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.674+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8280535c:::obj236:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.674+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8421ae49:::obj196:head, version: 111'196, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8421ae49:::obj196:head@111'196, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:8421ae49:::obj196:head, version: 111'196, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8421ae49:::obj196:head@111'196, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8260b93f:::obj338:head local-lis/les=113/114 n=307 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.674+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.674+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8260b93f:::obj338:head local-lis/les=113/114 n=307 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:8421ae49:::obj196:head, version: 111'196, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8421ae49:::obj196:head@111'196, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.674+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.674+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8260b93f:::obj338:head local-lis/les=113/114 n=307 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8421ae49:::obj196:head@111'196, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.674+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8280535c:::obj236:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.674+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8260b93f:::obj338:head local-lis/les=113/114 n=307 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:8421ae49:::obj196:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.674+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8260b93f:::obj338:head local-lis/les=113/114 n=307 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:8421ae49:::obj196:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.674+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8260b93f:::obj338:head local-lis/les=113/114 n=307 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:8421ae49:::obj196:head 2026-02-11T23:31:27.674+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8260b93f:::obj338:head local-lis/les=113/114 n=307 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:8421ae49:::obj196:head v 111'196 2026-02-11T23:31:27.674+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8260b93f:::obj338:head local-lis/les=113/114 n=307 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.674+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.674+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8421ae49:::obj196:head, version: 111'196, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8421ae49:::obj196:head@111'196, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.674+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.674+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.674+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8280535c:::obj236:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.674+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8280535c:::obj236:head) v3 prio 15 cost 0 latency 0.000058 pg_backfill(progress 1.0 e 116/116 lb 1:8280535c:::obj236:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8260b93f:::obj338:head local-lis/les=113/114 n=307 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.674+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.674+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8260b93f:::obj338:head local-lis/les=113/114 n=307 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:8280535c:::obj236:head) v3 2026-02-11T23:31:27.674+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8260b93f:::obj338:head local-lis/les=113/114 n=307 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:8280535c:::obj236:head) 2026-02-11T23:31:27.674+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.674+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8280535c:::obj236:head) v3 finish latency 0.000008 2026-02-11T23:31:27.674+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8280535c:::obj236:head local-lis/les=113/114 n=308 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.674+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.674+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.674+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8280535c:::obj236:head local-lis/les=113/114 n=308 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.674+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.674+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.674+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.674+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x612f680 con 0x4acb000 2026-02-11T23:31:27.674+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:8421ae49:::obj196:head)]) -- 0x615de00 con 0x4acb000 2026-02-11T23:31:27.674+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.674+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 653 ==== MOSDPGPush(1.0 116/113 [PushOp(1:84855744:::obj451:head, version: 111'451, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:84855744:::obj451:head@111'451, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559d520 con 0x4acb000 2026-02-11T23:31:27.674+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:84855744:::obj451:head, version: 111'451, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:84855744:::obj451:head@111'451, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000016 epoch 116 2026-02-11T23:31:27.674+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:84855744:::obj451:head, version: 111'451, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:84855744:::obj451:head@111'451, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.674+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.674+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 654 ==== pg_backfill(progress 1.0 e 116/116 lb 1:8421ae49:::obj196:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98580 con 0x4acb000 2026-02-11T23:31:27.674+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:8421ae49:::obj196:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.674+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:84855744:::obj451:head, version: 111'451, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:84855744:::obj451:head@111'451, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.674+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8421ae49:::obj196:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.674+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.674+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8421ae49:::obj196:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.674+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.674+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:84855744:::obj451:head, version: 111'451, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:84855744:::obj451:head@111'451, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.674+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:84855744:::obj451:head, version: 111'451, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:84855744:::obj451:head@111'451, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000049 MOSDPGPush(1.0 116/113 [PushOp(1:84855744:::obj451:head, version: 111'451, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:84855744:::obj451:head@111'451, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8280535c:::obj236:head local-lis/les=113/114 n=308 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.675+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.675+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8280535c:::obj236:head local-lis/les=113/114 n=308 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:84855744:::obj451:head, version: 111'451, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:84855744:::obj451:head@111'451, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.675+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8280535c:::obj236:head local-lis/les=113/114 n=308 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:84855744:::obj451:head@111'451, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.675+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8280535c:::obj236:head local-lis/les=113/114 n=308 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:84855744:::obj451:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.675+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8280535c:::obj236:head local-lis/les=113/114 n=308 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:84855744:::obj451:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.675+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8280535c:::obj236:head local-lis/les=113/114 n=308 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:84855744:::obj451:head 2026-02-11T23:31:27.675+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8280535c:::obj236:head local-lis/les=113/114 n=308 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:84855744:::obj451:head v 111'451 2026-02-11T23:31:27.675+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8280535c:::obj236:head local-lis/les=113/114 n=308 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.675+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.675+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:84855744:::obj451:head, version: 111'451, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:84855744:::obj451:head@111'451, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:27.675+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.675+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.675+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8421ae49:::obj196:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.675+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8421ae49:::obj196:head) v3 prio 15 cost 0 latency 0.000081 pg_backfill(progress 1.0 e 116/116 lb 1:8421ae49:::obj196:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8280535c:::obj236:head local-lis/les=113/114 n=308 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.675+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.675+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8280535c:::obj236:head local-lis/les=113/114 n=308 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:8421ae49:::obj196:head) v3 2026-02-11T23:31:27.675+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8280535c:::obj236:head local-lis/les=113/114 n=308 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:8421ae49:::obj196:head) 2026-02-11T23:31:27.675+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.675+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8421ae49:::obj196:head) v3 finish latency 0.000009 2026-02-11T23:31:27.675+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8421ae49:::obj196:head local-lis/les=113/114 n=309 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.675+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.675+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.675+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8421ae49:::obj196:head local-lis/les=113/114 n=309 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.675+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.675+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.675+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.675+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fed980 con 0x4acb000 2026-02-11T23:31:27.675+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:84855744:::obj451:head)]) -- 0x613e180 con 0x4acb000 2026-02-11T23:31:27.675+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.675+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 655 ==== MOSDPGPush(1.0 116/113 [PushOp(1:84c1344d:::obj97:head, version: 111'97, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:84c1344d:::obj97:head@111'97, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x559c680 con 0x4acb000 2026-02-11T23:31:27.675+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:84c1344d:::obj97:head, version: 111'97, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:84c1344d:::obj97:head@111'97, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.675+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:84c1344d:::obj97:head, version: 111'97, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:84c1344d:::obj97:head@111'97, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.675+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.675+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 656 ==== pg_backfill(progress 1.0 e 116/116 lb 1:84855744:::obj451:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98b00 con 0x4acb000 2026-02-11T23:31:27.675+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:84855744:::obj451:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.675+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:84c1344d:::obj97:head, version: 111'97, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:84c1344d:::obj97:head@111'97, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.675+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:84855744:::obj451:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.675+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.675+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:84c1344d:::obj97:head, version: 111'97, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:84c1344d:::obj97:head@111'97, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.675+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:84c1344d:::obj97:head, version: 111'97, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:84c1344d:::obj97:head@111'97, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:84c1344d:::obj97:head, version: 111'97, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:84c1344d:::obj97:head@111'97, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8421ae49:::obj196:head local-lis/les=113/114 n=309 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.675+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.675+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.675+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:84855744:::obj451:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.675+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8421ae49:::obj196:head local-lis/les=113/114 n=309 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:84c1344d:::obj97:head, version: 111'97, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:84c1344d:::obj97:head@111'97, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.675+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8421ae49:::obj196:head local-lis/les=113/114 n=309 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:84c1344d:::obj97:head@111'97, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.675+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8421ae49:::obj196:head local-lis/les=113/114 n=309 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:84c1344d:::obj97:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.675+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8421ae49:::obj196:head local-lis/les=113/114 n=309 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:84c1344d:::obj97:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.675+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8421ae49:::obj196:head local-lis/les=113/114 n=309 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:84c1344d:::obj97:head 2026-02-11T23:31:27.675+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8421ae49:::obj196:head local-lis/les=113/114 n=309 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:84c1344d:::obj97:head v 111'97 2026-02-11T23:31:27.675+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8421ae49:::obj196:head local-lis/les=113/114 n=309 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.675+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.675+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:84c1344d:::obj97:head, version: 111'97, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:84c1344d:::obj97:head@111'97, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:27.675+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.675+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.675+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:84855744:::obj451:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.675+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:84855744:::obj451:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 116/116 lb 1:84855744:::obj451:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8421ae49:::obj196:head local-lis/les=113/114 n=309 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.675+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.675+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8421ae49:::obj196:head local-lis/les=113/114 n=309 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:84855744:::obj451:head) v3 2026-02-11T23:31:27.675+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8421ae49:::obj196:head local-lis/les=113/114 n=309 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:84855744:::obj451:head) 2026-02-11T23:31:27.675+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.675+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:84855744:::obj451:head) v3 finish latency 0.000009 2026-02-11T23:31:27.675+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:84855744:::obj451:head local-lis/les=113/114 n=310 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.675+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.675+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.675+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:84855744:::obj451:head local-lis/les=113/114 n=310 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.675+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.675+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.675+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.675+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x615c180 con 0x4acb000 2026-02-11T23:31:27.675+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:84c1344d:::obj97:head)]) -- 0x613e480 con 0x4acb000 2026-02-11T23:31:27.675+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.676+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 657 ==== MOSDPGPush(1.0 116/113 [PushOp(1:86bb8a56:::obj416:head, version: 111'416, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:86bb8a56:::obj416:head@111'416, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559dba0 con 0x4acb000 2026-02-11T23:31:27.676+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:86bb8a56:::obj416:head, version: 111'416, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:86bb8a56:::obj416:head@111'416, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 116 2026-02-11T23:31:27.676+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:86bb8a56:::obj416:head, version: 111'416, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:86bb8a56:::obj416:head@111'416, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.676+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.676+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 658 ==== pg_backfill(progress 1.0 e 116/116 lb 1:84c1344d:::obj97:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d99080 con 0x4acb000 2026-02-11T23:31:27.676+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:84c1344d:::obj97:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.676+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:86bb8a56:::obj416:head, version: 111'416, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:86bb8a56:::obj416:head@111'416, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.676+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:84c1344d:::obj97:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.676+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.676+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:86bb8a56:::obj416:head, version: 111'416, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:86bb8a56:::obj416:head@111'416, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.676+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:86bb8a56:::obj416:head, version: 111'416, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:86bb8a56:::obj416:head@111'416, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 116/113 [PushOp(1:86bb8a56:::obj416:head, version: 111'416, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:86bb8a56:::obj416:head@111'416, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:84855744:::obj451:head local-lis/les=113/114 n=310 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.676+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.676+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:84855744:::obj451:head local-lis/les=113/114 n=310 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:86bb8a56:::obj416:head, version: 111'416, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:86bb8a56:::obj416:head@111'416, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.676+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.676+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:84855744:::obj451:head local-lis/les=113/114 n=310 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:86bb8a56:::obj416:head@111'416, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.676+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:84c1344d:::obj97:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.676+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:84855744:::obj451:head local-lis/les=113/114 n=310 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:86bb8a56:::obj416:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.676+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:84855744:::obj451:head local-lis/les=113/114 n=310 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:86bb8a56:::obj416:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.676+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:84855744:::obj451:head local-lis/les=113/114 n=310 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:86bb8a56:::obj416:head 2026-02-11T23:31:27.676+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:84855744:::obj451:head local-lis/les=113/114 n=310 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:86bb8a56:::obj416:head v 111'416 2026-02-11T23:31:27.676+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:84855744:::obj451:head local-lis/les=113/114 n=310 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.676+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.676+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:86bb8a56:::obj416:head, version: 111'416, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:86bb8a56:::obj416:head@111'416, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.676+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.676+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.676+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:84c1344d:::obj97:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.676+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:84c1344d:::obj97:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 116/116 lb 1:84c1344d:::obj97:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:84855744:::obj451:head local-lis/les=113/114 n=310 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.676+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.676+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:84855744:::obj451:head local-lis/les=113/114 n=310 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:84c1344d:::obj97:head) v3 2026-02-11T23:31:27.676+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:84855744:::obj451:head local-lis/les=113/114 n=310 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:84c1344d:::obj97:head) 2026-02-11T23:31:27.676+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.676+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:84c1344d:::obj97:head) v3 finish latency 0.000009 2026-02-11T23:31:27.676+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:84c1344d:::obj97:head local-lis/les=113/114 n=311 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.676+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.676+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.676+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:84c1344d:::obj97:head local-lis/les=113/114 n=311 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.676+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.676+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.676+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.676+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6014480 con 0x4acb000 2026-02-11T23:31:27.676+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:86bb8a56:::obj416:head)]) -- 0x613e600 con 0x4acb000 2026-02-11T23:31:27.676+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.676+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 659 ==== MOSDPGPush(1.0 116/113 [PushOp(1:8745f65b:::obj476:head, version: 111'476, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8745f65b:::obj476:head@111'476, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559d040 con 0x4acb000 2026-02-11T23:31:27.676+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:8745f65b:::obj476:head, version: 111'476, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8745f65b:::obj476:head@111'476, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.676+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8745f65b:::obj476:head, version: 111'476, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8745f65b:::obj476:head@111'476, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.676+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.676+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 660 ==== pg_backfill(progress 1.0 e 116/116 lb 1:86bb8a56:::obj416:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99600 con 0x4acb000 2026-02-11T23:31:27.676+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8745f65b:::obj476:head, version: 111'476, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8745f65b:::obj476:head@111'476, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.676+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:86bb8a56:::obj416:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.676+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:86bb8a56:::obj416:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.676+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.676+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8745f65b:::obj476:head, version: 111'476, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8745f65b:::obj476:head@111'476, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.676+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.676+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8745f65b:::obj476:head, version: 111'476, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8745f65b:::obj476:head@111'476, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:8745f65b:::obj476:head, version: 111'476, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8745f65b:::obj476:head@111'476, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:84c1344d:::obj97:head local-lis/les=113/114 n=311 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.676+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.676+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:84c1344d:::obj97:head local-lis/les=113/114 n=311 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:8745f65b:::obj476:head, version: 111'476, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8745f65b:::obj476:head@111'476, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.676+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:84c1344d:::obj97:head local-lis/les=113/114 n=311 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8745f65b:::obj476:head@111'476, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.676+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.676+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:86bb8a56:::obj416:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.676+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:84c1344d:::obj97:head local-lis/les=113/114 n=311 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:8745f65b:::obj476:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.676+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:84c1344d:::obj97:head local-lis/les=113/114 n=311 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:8745f65b:::obj476:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.676+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:84c1344d:::obj97:head local-lis/les=113/114 n=311 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:8745f65b:::obj476:head 2026-02-11T23:31:27.676+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:84c1344d:::obj97:head local-lis/les=113/114 n=311 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:8745f65b:::obj476:head v 111'476 2026-02-11T23:31:27.676+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:84c1344d:::obj97:head local-lis/les=113/114 n=311 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.676+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.676+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8745f65b:::obj476:head, version: 111'476, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8745f65b:::obj476:head@111'476, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000044 2026-02-11T23:31:27.676+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.676+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.676+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:86bb8a56:::obj416:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.676+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:86bb8a56:::obj416:head) v3 prio 15 cost 0 latency 0.000060 pg_backfill(progress 1.0 e 116/116 lb 1:86bb8a56:::obj416:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:84c1344d:::obj97:head local-lis/les=113/114 n=311 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.676+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.676+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:84c1344d:::obj97:head local-lis/les=113/114 n=311 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:86bb8a56:::obj416:head) v3 2026-02-11T23:31:27.676+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:84c1344d:::obj97:head local-lis/les=113/114 n=311 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:86bb8a56:::obj416:head) 2026-02-11T23:31:27.676+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.676+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:86bb8a56:::obj416:head) v3 finish latency 0.000009 2026-02-11T23:31:27.676+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:86bb8a56:::obj416:head local-lis/les=113/114 n=312 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.676+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.676+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.676+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:86bb8a56:::obj416:head local-lis/les=113/114 n=312 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.676+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.676+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.676+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.676+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x614de00 con 0x4acb000 2026-02-11T23:31:27.676+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:8745f65b:::obj476:head)]) -- 0x613e900 con 0x4acb000 2026-02-11T23:31:27.676+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.677+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 661 ==== MOSDPGPush(1.0 116/113 [PushOp(1:87eb7818:::obj91:head, version: 111'91, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:87eb7818:::obj91:head@111'91, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x55d89c0 con 0x4acb000 2026-02-11T23:31:27.677+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:87eb7818:::obj91:head, version: 111'91, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:87eb7818:::obj91:head@111'91, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.677+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:87eb7818:::obj91:head, version: 111'91, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:87eb7818:::obj91:head@111'91, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.677+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.677+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 662 ==== pg_backfill(progress 1.0 e 116/116 lb 1:8745f65b:::obj476:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99b80 con 0x4acb000 2026-02-11T23:31:27.677+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:87eb7818:::obj91:head, version: 111'91, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:87eb7818:::obj91:head@111'91, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.677+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:8745f65b:::obj476:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 116 2026-02-11T23:31:27.677+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.677+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8745f65b:::obj476:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.677+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:87eb7818:::obj91:head, version: 111'91, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:87eb7818:::obj91:head@111'91, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.677+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.677+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:87eb7818:::obj91:head, version: 111'91, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:87eb7818:::obj91:head@111'91, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:87eb7818:::obj91:head, version: 111'91, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:87eb7818:::obj91:head@111'91, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:86bb8a56:::obj416:head local-lis/les=113/114 n=312 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.677+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.677+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:86bb8a56:::obj416:head local-lis/les=113/114 n=312 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:87eb7818:::obj91:head, version: 111'91, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:87eb7818:::obj91:head@111'91, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.677+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:86bb8a56:::obj416:head local-lis/les=113/114 n=312 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:87eb7818:::obj91:head@111'91, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.677+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.677+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8745f65b:::obj476:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.677+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:86bb8a56:::obj416:head local-lis/les=113/114 n=312 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:87eb7818:::obj91:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.677+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:86bb8a56:::obj416:head local-lis/les=113/114 n=312 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:87eb7818:::obj91:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.677+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:86bb8a56:::obj416:head local-lis/les=113/114 n=312 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:87eb7818:::obj91:head 2026-02-11T23:31:27.677+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:86bb8a56:::obj416:head local-lis/les=113/114 n=312 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:87eb7818:::obj91:head v 111'91 2026-02-11T23:31:27.677+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:86bb8a56:::obj416:head local-lis/les=113/114 n=312 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.677+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.677+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:87eb7818:::obj91:head, version: 111'91, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:87eb7818:::obj91:head@111'91, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.677+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.677+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.677+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8745f65b:::obj476:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.677+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8745f65b:::obj476:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:8745f65b:::obj476:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:86bb8a56:::obj416:head local-lis/les=113/114 n=312 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.677+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.677+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:86bb8a56:::obj416:head local-lis/les=113/114 n=312 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:8745f65b:::obj476:head) v3 2026-02-11T23:31:27.677+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:86bb8a56:::obj416:head local-lis/les=113/114 n=312 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:8745f65b:::obj476:head) 2026-02-11T23:31:27.677+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.677+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8745f65b:::obj476:head) v3 finish latency 0.000009 2026-02-11T23:31:27.677+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8745f65b:::obj476:head local-lis/les=113/114 n=313 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.677+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.677+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.677+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8745f65b:::obj476:head local-lis/les=113/114 n=313 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.677+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.677+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.677+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.677+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fce780 con 0x4acb000 2026-02-11T23:31:27.677+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:87eb7818:::obj91:head)]) -- 0x613ec00 con 0x4acb000 2026-02-11T23:31:27.677+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.677+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 663 ==== MOSDPGPush(1.0 116/113 [PushOp(1:881499a5:::obj460:head, version: 111'460, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:881499a5:::obj460:head@111'460, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d8000 con 0x4acb000 2026-02-11T23:31:27.677+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:881499a5:::obj460:head, version: 111'460, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:881499a5:::obj460:head@111'460, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.677+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:881499a5:::obj460:head, version: 111'460, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:881499a5:::obj460:head@111'460, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.677+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.677+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 664 ==== pg_backfill(progress 1.0 e 116/116 lb 1:87eb7818:::obj91:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d9a100 con 0x4acb000 2026-02-11T23:31:27.677+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:881499a5:::obj460:head, version: 111'460, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:881499a5:::obj460:head@111'460, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.677+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:87eb7818:::obj91:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.677+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:87eb7818:::obj91:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.677+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.677+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:881499a5:::obj460:head, version: 111'460, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:881499a5:::obj460:head@111'460, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.678+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.678+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:881499a5:::obj460:head, version: 111'460, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:881499a5:::obj460:head@111'460, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(1.0 116/113 [PushOp(1:881499a5:::obj460:head, version: 111'460, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:881499a5:::obj460:head@111'460, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8745f65b:::obj476:head local-lis/les=113/114 n=313 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.678+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.678+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8745f65b:::obj476:head local-lis/les=113/114 n=313 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:881499a5:::obj460:head, version: 111'460, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:881499a5:::obj460:head@111'460, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.678+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8745f65b:::obj476:head local-lis/les=113/114 n=313 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:881499a5:::obj460:head@111'460, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.678+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.678+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:87eb7818:::obj91:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.678+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8745f65b:::obj476:head local-lis/les=113/114 n=313 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:881499a5:::obj460:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.678+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8745f65b:::obj476:head local-lis/les=113/114 n=313 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:881499a5:::obj460:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.678+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8745f65b:::obj476:head local-lis/les=113/114 n=313 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:881499a5:::obj460:head 2026-02-11T23:31:27.678+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8745f65b:::obj476:head local-lis/les=113/114 n=313 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:881499a5:::obj460:head v 111'460 2026-02-11T23:31:27.678+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8745f65b:::obj476:head local-lis/les=113/114 n=313 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.678+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.678+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:881499a5:::obj460:head, version: 111'460, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:881499a5:::obj460:head@111'460, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.678+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.678+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.678+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:87eb7818:::obj91:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.678+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:87eb7818:::obj91:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 1.0 e 116/116 lb 1:87eb7818:::obj91:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8745f65b:::obj476:head local-lis/les=113/114 n=313 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.678+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.678+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8745f65b:::obj476:head local-lis/les=113/114 n=313 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:87eb7818:::obj91:head) v3 2026-02-11T23:31:27.678+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8745f65b:::obj476:head local-lis/les=113/114 n=313 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:87eb7818:::obj91:head) 2026-02-11T23:31:27.678+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.678+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:87eb7818:::obj91:head) v3 finish latency 0.000010 2026-02-11T23:31:27.678+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:87eb7818:::obj91:head local-lis/les=113/114 n=314 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.678+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.678+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.678+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:87eb7818:::obj91:head local-lis/les=113/114 n=314 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.678+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.678+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.678+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.678+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x614db00 con 0x4acb000 2026-02-11T23:31:27.678+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:881499a5:::obj460:head)]) -- 0x613ed80 con 0x4acb000 2026-02-11T23:31:27.678+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.678+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 665 ==== MOSDPGPush(1.0 116/113 [PushOp(1:8835249a:::obj346:head, version: 111'346, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8835249a:::obj346:head@111'346, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d84e0 con 0x4acb000 2026-02-11T23:31:27.678+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:8835249a:::obj346:head, version: 111'346, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8835249a:::obj346:head@111'346, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.678+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8835249a:::obj346:head, version: 111'346, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8835249a:::obj346:head@111'346, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.678+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.678+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 666 ==== pg_backfill(progress 1.0 e 116/116 lb 1:881499a5:::obj460:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9a680 con 0x4acb000 2026-02-11T23:31:27.678+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8835249a:::obj346:head, version: 111'346, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8835249a:::obj346:head@111'346, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.678+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:881499a5:::obj460:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.678+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:881499a5:::obj460:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.678+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.678+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8835249a:::obj346:head, version: 111'346, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8835249a:::obj346:head@111'346, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.678+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.678+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8835249a:::obj346:head, version: 111'346, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8835249a:::obj346:head@111'346, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 116/113 [PushOp(1:8835249a:::obj346:head, version: 111'346, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8835249a:::obj346:head@111'346, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:87eb7818:::obj91:head local-lis/les=113/114 n=314 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.678+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.678+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:87eb7818:::obj91:head local-lis/les=113/114 n=314 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:8835249a:::obj346:head, version: 111'346, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8835249a:::obj346:head@111'346, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.678+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:87eb7818:::obj91:head local-lis/les=113/114 n=314 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8835249a:::obj346:head@111'346, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.678+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.678+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:881499a5:::obj460:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.678+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:87eb7818:::obj91:head local-lis/les=113/114 n=314 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:8835249a:::obj346:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.678+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:87eb7818:::obj91:head local-lis/les=113/114 n=314 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:8835249a:::obj346:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.678+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:87eb7818:::obj91:head local-lis/les=113/114 n=314 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:8835249a:::obj346:head 2026-02-11T23:31:27.678+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:87eb7818:::obj91:head local-lis/les=113/114 n=314 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:8835249a:::obj346:head v 111'346 2026-02-11T23:31:27.678+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:87eb7818:::obj91:head local-lis/les=113/114 n=314 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.678+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.678+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8835249a:::obj346:head, version: 111'346, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8835249a:::obj346:head@111'346, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000043 2026-02-11T23:31:27.678+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.678+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.678+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:881499a5:::obj460:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.678+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:881499a5:::obj460:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:881499a5:::obj460:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:87eb7818:::obj91:head local-lis/les=113/114 n=314 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.678+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.678+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:87eb7818:::obj91:head local-lis/les=113/114 n=314 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:881499a5:::obj460:head) v3 2026-02-11T23:31:27.678+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:87eb7818:::obj91:head local-lis/les=113/114 n=314 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:881499a5:::obj460:head) 2026-02-11T23:31:27.678+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.678+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:881499a5:::obj460:head) v3 finish latency 0.000009 2026-02-11T23:31:27.678+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:881499a5:::obj460:head local-lis/les=113/114 n=315 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.678+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.678+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.678+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:881499a5:::obj460:head local-lis/les=113/114 n=315 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.678+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.678+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.678+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.678+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6014780 con 0x4acb000 2026-02-11T23:31:27.678+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:8835249a:::obj346:head)]) -- 0x613f080 con 0x4acb000 2026-02-11T23:31:27.678+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.679+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 667 ==== MOSDPGPush(1.0 116/113 [PushOp(1:88c90a82:::obj574:head, version: 111'574, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:88c90a82:::obj574:head@111'574, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5647380 con 0x4acb000 2026-02-11T23:31:27.679+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:88c90a82:::obj574:head, version: 111'574, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:88c90a82:::obj574:head@111'574, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.679+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:88c90a82:::obj574:head, version: 111'574, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:88c90a82:::obj574:head@111'574, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.679+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.679+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 668 ==== pg_backfill(progress 1.0 e 116/116 lb 1:8835249a:::obj346:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9ac00 con 0x4acb000 2026-02-11T23:31:27.679+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:88c90a82:::obj574:head, version: 111'574, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:88c90a82:::obj574:head@111'574, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.679+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.679+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:8835249a:::obj346:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.679+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:88c90a82:::obj574:head, version: 111'574, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:88c90a82:::obj574:head@111'574, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.679+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8835249a:::obj346:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.679+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.679+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:88c90a82:::obj574:head, version: 111'574, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:88c90a82:::obj574:head@111'574, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:88c90a82:::obj574:head, version: 111'574, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:88c90a82:::obj574:head@111'574, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:881499a5:::obj460:head local-lis/les=113/114 n=315 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.679+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8835249a:::obj346:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.679+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.679+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:881499a5:::obj460:head local-lis/les=113/114 n=315 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:88c90a82:::obj574:head, version: 111'574, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:88c90a82:::obj574:head@111'574, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.679+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:881499a5:::obj460:head local-lis/les=113/114 n=315 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:88c90a82:::obj574:head@111'574, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.679+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:881499a5:::obj460:head local-lis/les=113/114 n=315 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:88c90a82:::obj574:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.679+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:881499a5:::obj460:head local-lis/les=113/114 n=315 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:88c90a82:::obj574:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.679+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:881499a5:::obj460:head local-lis/les=113/114 n=315 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:88c90a82:::obj574:head 2026-02-11T23:31:27.679+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:881499a5:::obj460:head local-lis/les=113/114 n=315 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:88c90a82:::obj574:head v 111'574 2026-02-11T23:31:27.679+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:881499a5:::obj460:head local-lis/les=113/114 n=315 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.679+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.679+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:88c90a82:::obj574:head, version: 111'574, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:88c90a82:::obj574:head@111'574, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.679+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.679+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.679+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8835249a:::obj346:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.679+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8835249a:::obj346:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:8835249a:::obj346:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:881499a5:::obj460:head local-lis/les=113/114 n=315 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.679+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.679+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:881499a5:::obj460:head local-lis/les=113/114 n=315 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:8835249a:::obj346:head) v3 2026-02-11T23:31:27.679+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:881499a5:::obj460:head local-lis/les=113/114 n=315 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:8835249a:::obj346:head) 2026-02-11T23:31:27.679+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.679+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8835249a:::obj346:head) v3 finish latency 0.000009 2026-02-11T23:31:27.679+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8835249a:::obj346:head local-lis/les=113/114 n=316 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.679+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.679+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.679+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8835249a:::obj346:head local-lis/les=113/114 n=316 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.679+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.679+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.679+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.679+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x614d800 con 0x4acb000 2026-02-11T23:31:27.679+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:88c90a82:::obj574:head)]) -- 0x613f380 con 0x4acb000 2026-02-11T23:31:27.679+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.679+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 669 ==== MOSDPGPush(1.0 116/113 [PushOp(1:88e6b0fd:::obj99:head, version: 111'99, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:88e6b0fd:::obj99:head@111'99, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x55d8ea0 con 0x4acb000 2026-02-11T23:31:27.679+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:88e6b0fd:::obj99:head, version: 111'99, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:88e6b0fd:::obj99:head@111'99, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.679+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:88e6b0fd:::obj99:head, version: 111'99, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:88e6b0fd:::obj99:head@111'99, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.679+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.679+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 670 ==== pg_backfill(progress 1.0 e 116/116 lb 1:88c90a82:::obj574:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9b180 con 0x4acb000 2026-02-11T23:31:27.679+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:88c90a82:::obj574:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.679+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:88e6b0fd:::obj99:head, version: 111'99, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:88e6b0fd:::obj99:head@111'99, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.679+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:88c90a82:::obj574:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.679+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.679+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:88e6b0fd:::obj99:head, version: 111'99, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:88e6b0fd:::obj99:head@111'99, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.679+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:88e6b0fd:::obj99:head, version: 111'99, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:88e6b0fd:::obj99:head@111'99, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:88e6b0fd:::obj99:head, version: 111'99, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:88e6b0fd:::obj99:head@111'99, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8835249a:::obj346:head local-lis/les=113/114 n=316 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.679+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.679+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8835249a:::obj346:head local-lis/les=113/114 n=316 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:88e6b0fd:::obj99:head, version: 111'99, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:88e6b0fd:::obj99:head@111'99, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.679+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.679+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:88c90a82:::obj574:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.679+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8835249a:::obj346:head local-lis/les=113/114 n=316 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:88e6b0fd:::obj99:head@111'99, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.679+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8835249a:::obj346:head local-lis/les=113/114 n=316 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:88e6b0fd:::obj99:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.679+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8835249a:::obj346:head local-lis/les=113/114 n=316 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:88e6b0fd:::obj99:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.679+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8835249a:::obj346:head local-lis/les=113/114 n=316 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:88e6b0fd:::obj99:head 2026-02-11T23:31:27.679+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8835249a:::obj346:head local-lis/les=113/114 n=316 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:88e6b0fd:::obj99:head v 111'99 2026-02-11T23:31:27.679+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8835249a:::obj346:head local-lis/les=113/114 n=316 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.679+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.679+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:88e6b0fd:::obj99:head, version: 111'99, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:88e6b0fd:::obj99:head@111'99, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:27.679+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8835249a:::obj346:head local-lis/les=113/114 n=316 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.679+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.679+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.679+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:88c90a82:::obj574:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.679+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:88c90a82:::obj574:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 1.0 e 116/116 lb 1:88c90a82:::obj574:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8835249a:::obj346:head local-lis/les=113/114 n=316 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.679+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.679+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8835249a:::obj346:head local-lis/les=113/114 n=316 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:88c90a82:::obj574:head) v3 2026-02-11T23:31:27.679+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8835249a:::obj346:head local-lis/les=113/114 n=316 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:88c90a82:::obj574:head) 2026-02-11T23:31:27.679+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.679+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:88c90a82:::obj574:head) v3 finish latency 0.000011 2026-02-11T23:31:27.679+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.679+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.679+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:88c90a82:::obj574:head local-lis/les=113/114 n=317 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.679+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.679+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.679+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.679+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x614d680 con 0x4acb000 2026-02-11T23:31:27.679+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:88e6b0fd:::obj99:head)]) -- 0x6005380 con 0x4acb000 2026-02-11T23:31:27.679+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.680+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 671 ==== MOSDPGPush(1.0 116/113 [PushOp(1:89565a01:::obj330:head, version: 111'330, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:89565a01:::obj330:head@111'330, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d9380 con 0x4acb000 2026-02-11T23:31:27.680+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:89565a01:::obj330:head, version: 111'330, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:89565a01:::obj330:head@111'330, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.680+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:89565a01:::obj330:head, version: 111'330, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:89565a01:::obj330:head@111'330, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.680+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.680+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 672 ==== pg_backfill(progress 1.0 e 116/116 lb 1:88e6b0fd:::obj99:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d98000 con 0x4acb000 2026-02-11T23:31:27.680+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:89565a01:::obj330:head, version: 111'330, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:89565a01:::obj330:head@111'330, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.680+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:88e6b0fd:::obj99:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 116 2026-02-11T23:31:27.680+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.680+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:88e6b0fd:::obj99:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.680+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:89565a01:::obj330:head, version: 111'330, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:89565a01:::obj330:head@111'330, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.680+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.680+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:89565a01:::obj330:head, version: 111'330, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:89565a01:::obj330:head@111'330, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:89565a01:::obj330:head, version: 111'330, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:89565a01:::obj330:head@111'330, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:88c90a82:::obj574:head local-lis/les=113/114 n=317 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.680+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.680+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:88c90a82:::obj574:head local-lis/les=113/114 n=317 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:89565a01:::obj330:head, version: 111'330, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:89565a01:::obj330:head@111'330, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.680+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:88c90a82:::obj574:head local-lis/les=113/114 n=317 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:89565a01:::obj330:head@111'330, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.680+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.680+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:88e6b0fd:::obj99:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.680+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:88c90a82:::obj574:head local-lis/les=113/114 n=317 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:89565a01:::obj330:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.680+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:88c90a82:::obj574:head local-lis/les=113/114 n=317 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:89565a01:::obj330:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.680+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:88c90a82:::obj574:head local-lis/les=113/114 n=317 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:89565a01:::obj330:head 2026-02-11T23:31:27.680+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:88c90a82:::obj574:head local-lis/les=113/114 n=317 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:89565a01:::obj330:head v 111'330 2026-02-11T23:31:27.680+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:88c90a82:::obj574:head local-lis/les=113/114 n=317 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.680+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.680+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:89565a01:::obj330:head, version: 111'330, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:89565a01:::obj330:head@111'330, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:27.680+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.680+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.680+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:88e6b0fd:::obj99:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.680+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:88e6b0fd:::obj99:head) v3 prio 15 cost 0 latency 0.000080 pg_backfill(progress 1.0 e 116/116 lb 1:88e6b0fd:::obj99:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:88c90a82:::obj574:head local-lis/les=113/114 n=317 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.680+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.680+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:88c90a82:::obj574:head local-lis/les=113/114 n=317 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:88e6b0fd:::obj99:head) v3 2026-02-11T23:31:27.680+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:88c90a82:::obj574:head local-lis/les=113/114 n=317 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:88e6b0fd:::obj99:head) 2026-02-11T23:31:27.680+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.680+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:88e6b0fd:::obj99:head) v3 finish latency 0.000010 2026-02-11T23:31:27.680+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:88e6b0fd:::obj99:head local-lis/les=113/114 n=318 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.680+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.680+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.680+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:88e6b0fd:::obj99:head local-lis/les=113/114 n=318 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.680+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.680+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.680+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.680+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6014900 con 0x4acb000 2026-02-11T23:31:27.680+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:89565a01:::obj330:head)]) -- 0x613f680 con 0x4acb000 2026-02-11T23:31:27.680+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.680+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 673 ==== MOSDPGPush(1.0 116/113 [PushOp(1:8a5b4a06:::obj393:head, version: 111'393, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8a5b4a06:::obj393:head@111'393, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56cb040 con 0x4acb000 2026-02-11T23:31:27.680+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:8a5b4a06:::obj393:head, version: 111'393, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8a5b4a06:::obj393:head@111'393, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.681+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8a5b4a06:::obj393:head, version: 111'393, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8a5b4a06:::obj393:head@111'393, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.681+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.681+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8a5b4a06:::obj393:head, version: 111'393, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8a5b4a06:::obj393:head@111'393, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.681+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.681+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 674 ==== pg_backfill(progress 1.0 e 116/116 lb 1:89565a01:::obj330:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98580 con 0x4acb000 2026-02-11T23:31:27.681+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8a5b4a06:::obj393:head, version: 111'393, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8a5b4a06:::obj393:head@111'393, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.681+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:89565a01:::obj330:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 116 2026-02-11T23:31:27.681+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.681+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:89565a01:::obj330:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.681+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8a5b4a06:::obj393:head, version: 111'393, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8a5b4a06:::obj393:head@111'393, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000044 MOSDPGPush(1.0 116/113 [PushOp(1:8a5b4a06:::obj393:head, version: 111'393, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8a5b4a06:::obj393:head@111'393, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:88e6b0fd:::obj99:head local-lis/les=113/114 n=318 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.681+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.681+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:88e6b0fd:::obj99:head local-lis/les=113/114 n=318 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:8a5b4a06:::obj393:head, version: 111'393, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8a5b4a06:::obj393:head@111'393, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.681+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.681+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:88e6b0fd:::obj99:head local-lis/les=113/114 n=318 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8a5b4a06:::obj393:head@111'393, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.681+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:89565a01:::obj330:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.681+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:88e6b0fd:::obj99:head local-lis/les=113/114 n=318 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:8a5b4a06:::obj393:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.681+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:88e6b0fd:::obj99:head local-lis/les=113/114 n=318 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:8a5b4a06:::obj393:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.681+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:88e6b0fd:::obj99:head local-lis/les=113/114 n=318 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:8a5b4a06:::obj393:head 2026-02-11T23:31:27.681+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:88e6b0fd:::obj99:head local-lis/les=113/114 n=318 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:8a5b4a06:::obj393:head v 111'393 2026-02-11T23:31:27.681+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:88e6b0fd:::obj99:head local-lis/les=113/114 n=318 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.681+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.681+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8a5b4a06:::obj393:head, version: 111'393, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8a5b4a06:::obj393:head@111'393, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:27.681+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:88e6b0fd:::obj99:head local-lis/les=113/114 n=318 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.681+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.681+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.681+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:89565a01:::obj330:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.681+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:89565a01:::obj330:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 1.0 e 116/116 lb 1:89565a01:::obj330:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:88e6b0fd:::obj99:head local-lis/les=113/114 n=318 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.681+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.681+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:88e6b0fd:::obj99:head local-lis/les=113/114 n=318 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:89565a01:::obj330:head) v3 2026-02-11T23:31:27.681+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:88e6b0fd:::obj99:head local-lis/les=113/114 n=318 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:89565a01:::obj330:head) 2026-02-11T23:31:27.681+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.681+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:89565a01:::obj330:head) v3 finish latency 0.000016 2026-02-11T23:31:27.681+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.681+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.681+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:89565a01:::obj330:head local-lis/les=113/114 n=319 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.681+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.681+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.681+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.681+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6014f00 con 0x4acb000 2026-02-11T23:31:27.681+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:8a5b4a06:::obj393:head)]) -- 0x614d380 con 0x4acb000 2026-02-11T23:31:27.681+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.681+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 675 ==== MOSDPGPush(1.0 116/113 [PushOp(1:8a7d5b8a:::obj217:head, version: 111'217, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8a7d5b8a:::obj217:head@111'217, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56ca1a0 con 0x4acb000 2026-02-11T23:31:27.681+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:8a7d5b8a:::obj217:head, version: 111'217, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8a7d5b8a:::obj217:head@111'217, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 116 2026-02-11T23:31:27.681+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8a7d5b8a:::obj217:head, version: 111'217, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8a7d5b8a:::obj217:head@111'217, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.681+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.681+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 676 ==== pg_backfill(progress 1.0 e 116/116 lb 1:8a5b4a06:::obj393:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98b00 con 0x4acb000 2026-02-11T23:31:27.681+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:8a5b4a06:::obj393:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.681+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8a7d5b8a:::obj217:head, version: 111'217, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8a7d5b8a:::obj217:head@111'217, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.681+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8a5b4a06:::obj393:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.681+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.681+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8a7d5b8a:::obj217:head, version: 111'217, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8a7d5b8a:::obj217:head@111'217, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.681+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8a7d5b8a:::obj217:head, version: 111'217, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8a7d5b8a:::obj217:head@111'217, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 116/113 [PushOp(1:8a7d5b8a:::obj217:head, version: 111'217, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8a7d5b8a:::obj217:head@111'217, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:89565a01:::obj330:head local-lis/les=113/114 n=319 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.681+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.681+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.681+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:89565a01:::obj330:head local-lis/les=113/114 n=319 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:8a7d5b8a:::obj217:head, version: 111'217, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8a7d5b8a:::obj217:head@111'217, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.681+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8a5b4a06:::obj393:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.681+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:89565a01:::obj330:head local-lis/les=113/114 n=319 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8a7d5b8a:::obj217:head@111'217, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.681+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:89565a01:::obj330:head local-lis/les=113/114 n=319 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:8a7d5b8a:::obj217:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.681+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:89565a01:::obj330:head local-lis/les=113/114 n=319 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:8a7d5b8a:::obj217:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.681+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:89565a01:::obj330:head local-lis/les=113/114 n=319 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:8a7d5b8a:::obj217:head 2026-02-11T23:31:27.681+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:89565a01:::obj330:head local-lis/les=113/114 n=319 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:8a7d5b8a:::obj217:head v 111'217 2026-02-11T23:31:27.681+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:89565a01:::obj330:head local-lis/les=113/114 n=319 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.681+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.681+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8a7d5b8a:::obj217:head, version: 111'217, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8a7d5b8a:::obj217:head@111'217, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:27.681+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.681+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.681+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8a5b4a06:::obj393:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.681+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8a5b4a06:::obj393:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 1.0 e 116/116 lb 1:8a5b4a06:::obj393:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:89565a01:::obj330:head local-lis/les=113/114 n=319 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.681+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.681+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:89565a01:::obj330:head local-lis/les=113/114 n=319 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:8a5b4a06:::obj393:head) v3 2026-02-11T23:31:27.681+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:89565a01:::obj330:head local-lis/les=113/114 n=319 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:8a5b4a06:::obj393:head) 2026-02-11T23:31:27.681+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.681+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8a5b4a06:::obj393:head) v3 finish latency 0.000012 2026-02-11T23:31:27.681+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8a5b4a06:::obj393:head local-lis/les=113/114 n=320 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.681+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.681+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.681+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8a5b4a06:::obj393:head local-lis/les=113/114 n=320 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.681+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.681+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.681+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.681+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6014c00 con 0x4acb000 2026-02-11T23:31:27.681+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:8a7d5b8a:::obj217:head)]) -- 0x613f980 con 0x4acb000 2026-02-11T23:31:27.681+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.682+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 677 ==== MOSDPGPush(1.0 116/113 [PushOp(1:8aa18807:::obj353:head, version: 111'353, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8aa18807:::obj353:head@111'353, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d8680 con 0x4acb000 2026-02-11T23:31:27.682+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:8aa18807:::obj353:head, version: 111'353, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8aa18807:::obj353:head@111'353, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 116 2026-02-11T23:31:27.682+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8aa18807:::obj353:head, version: 111'353, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8aa18807:::obj353:head@111'353, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.682+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.682+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 678 ==== pg_backfill(progress 1.0 e 116/116 lb 1:8a7d5b8a:::obj217:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99080 con 0x4acb000 2026-02-11T23:31:27.682+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8aa18807:::obj353:head, version: 111'353, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8aa18807:::obj353:head@111'353, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.682+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:8a7d5b8a:::obj217:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.682+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.682+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8a7d5b8a:::obj217:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.682+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8aa18807:::obj353:head, version: 111'353, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8aa18807:::obj353:head@111'353, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.682+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.682+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8aa18807:::obj353:head, version: 111'353, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8aa18807:::obj353:head@111'353, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000040 MOSDPGPush(1.0 116/113 [PushOp(1:8aa18807:::obj353:head, version: 111'353, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8aa18807:::obj353:head@111'353, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8a5b4a06:::obj393:head local-lis/les=113/114 n=320 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.682+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.682+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8a5b4a06:::obj393:head local-lis/les=113/114 n=320 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:8aa18807:::obj353:head, version: 111'353, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8aa18807:::obj353:head@111'353, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.682+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8a5b4a06:::obj393:head local-lis/les=113/114 n=320 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8aa18807:::obj353:head@111'353, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.682+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.682+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8a7d5b8a:::obj217:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.682+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8a5b4a06:::obj393:head local-lis/les=113/114 n=320 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:8aa18807:::obj353:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.682+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8a5b4a06:::obj393:head local-lis/les=113/114 n=320 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:8aa18807:::obj353:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.682+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8a5b4a06:::obj393:head local-lis/les=113/114 n=320 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:8aa18807:::obj353:head 2026-02-11T23:31:27.682+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8a5b4a06:::obj393:head local-lis/les=113/114 n=320 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:8aa18807:::obj353:head v 111'353 2026-02-11T23:31:27.682+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8a5b4a06:::obj393:head local-lis/les=113/114 n=320 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.682+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.682+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8aa18807:::obj353:head, version: 111'353, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8aa18807:::obj353:head@111'353, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.682+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8a5b4a06:::obj393:head local-lis/les=113/114 n=320 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.682+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.682+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.682+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8a7d5b8a:::obj217:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.682+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8a7d5b8a:::obj217:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 1.0 e 116/116 lb 1:8a7d5b8a:::obj217:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8a5b4a06:::obj393:head local-lis/les=113/114 n=320 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.682+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.682+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8a5b4a06:::obj393:head local-lis/les=113/114 n=320 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:8a7d5b8a:::obj217:head) v3 2026-02-11T23:31:27.682+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8a5b4a06:::obj393:head local-lis/les=113/114 n=320 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:8a7d5b8a:::obj217:head) 2026-02-11T23:31:27.682+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.682+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8a7d5b8a:::obj217:head) v3 finish latency 0.000014 2026-02-11T23:31:27.682+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.682+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.682+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8a7d5b8a:::obj217:head local-lis/les=113/114 n=321 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.682+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.682+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.682+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.682+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x614d080 con 0x4acb000 2026-02-11T23:31:27.682+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:8aa18807:::obj353:head)]) -- 0x6004c00 con 0x4acb000 2026-02-11T23:31:27.682+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.682+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 679 ==== MOSDPGPush(1.0 116/113 [PushOp(1:8b0915dc:::obj278:head, version: 111'278, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b0915dc:::obj278:head@111'278, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559dd40 con 0x4acb000 2026-02-11T23:31:27.682+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:8b0915dc:::obj278:head, version: 111'278, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b0915dc:::obj278:head@111'278, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 116 2026-02-11T23:31:27.682+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8b0915dc:::obj278:head, version: 111'278, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b0915dc:::obj278:head@111'278, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.682+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.682+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 680 ==== pg_backfill(progress 1.0 e 116/116 lb 1:8aa18807:::obj353:head) ==== 1062+0+0 (unknown 0 0 0) 0x4a2cb00 con 0x4acb000 2026-02-11T23:31:27.682+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8b0915dc:::obj278:head, version: 111'278, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b0915dc:::obj278:head@111'278, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.682+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:8aa18807:::obj353:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.682+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.682+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8aa18807:::obj353:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.682+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8b0915dc:::obj278:head, version: 111'278, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b0915dc:::obj278:head@111'278, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.682+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.682+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8b0915dc:::obj278:head, version: 111'278, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b0915dc:::obj278:head@111'278, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(1.0 116/113 [PushOp(1:8b0915dc:::obj278:head, version: 111'278, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b0915dc:::obj278:head@111'278, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8a7d5b8a:::obj217:head local-lis/les=113/114 n=321 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.682+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.682+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8a7d5b8a:::obj217:head local-lis/les=113/114 n=321 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:8b0915dc:::obj278:head, version: 111'278, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b0915dc:::obj278:head@111'278, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.682+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8a7d5b8a:::obj217:head local-lis/les=113/114 n=321 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8b0915dc:::obj278:head@111'278, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.682+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.682+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8aa18807:::obj353:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.682+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8a7d5b8a:::obj217:head local-lis/les=113/114 n=321 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:8b0915dc:::obj278:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.682+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8a7d5b8a:::obj217:head local-lis/les=113/114 n=321 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:8b0915dc:::obj278:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.682+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8a7d5b8a:::obj217:head local-lis/les=113/114 n=321 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:8b0915dc:::obj278:head 2026-02-11T23:31:27.682+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8a7d5b8a:::obj217:head local-lis/les=113/114 n=321 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:8b0915dc:::obj278:head v 111'278 2026-02-11T23:31:27.682+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8a7d5b8a:::obj217:head local-lis/les=113/114 n=321 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.682+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.682+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8b0915dc:::obj278:head, version: 111'278, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b0915dc:::obj278:head@111'278, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:27.682+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8a7d5b8a:::obj217:head local-lis/les=113/114 n=321 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.683+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.683+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.683+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8aa18807:::obj353:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.683+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8aa18807:::obj353:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 1.0 e 116/116 lb 1:8aa18807:::obj353:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8a7d5b8a:::obj217:head local-lis/les=113/114 n=321 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.683+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.683+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8a7d5b8a:::obj217:head local-lis/les=113/114 n=321 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:8aa18807:::obj353:head) v3 2026-02-11T23:31:27.683+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8a7d5b8a:::obj217:head local-lis/les=113/114 n=321 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:8aa18807:::obj353:head) 2026-02-11T23:31:27.683+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.683+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8aa18807:::obj353:head) v3 finish latency 0.000011 2026-02-11T23:31:27.683+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.683+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.683+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8aa18807:::obj353:head local-lis/les=113/114 n=322 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.683+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.683+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.683+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.683+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fced80 con 0x4acb000 2026-02-11T23:31:27.683+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:8b0915dc:::obj278:head)]) -- 0x6004f00 con 0x4acb000 2026-02-11T23:31:27.683+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.683+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 681 ==== MOSDPGPush(1.0 116/113 [PushOp(1:8b38e8a9:::obj385:head, version: 111'385, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b38e8a9:::obj385:head@111'385, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c820 con 0x4acb000 2026-02-11T23:31:27.683+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:8b38e8a9:::obj385:head, version: 111'385, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b38e8a9:::obj385:head@111'385, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 116 2026-02-11T23:31:27.683+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8b38e8a9:::obj385:head, version: 111'385, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b38e8a9:::obj385:head@111'385, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.683+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.683+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 682 ==== pg_backfill(progress 1.0 e 116/116 lb 1:8b0915dc:::obj278:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2d080 con 0x4acb000 2026-02-11T23:31:27.683+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:8b0915dc:::obj278:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.683+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8b38e8a9:::obj385:head, version: 111'385, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b38e8a9:::obj385:head@111'385, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.683+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8b0915dc:::obj278:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.683+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.683+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8b0915dc:::obj278:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.683+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.683+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8b38e8a9:::obj385:head, version: 111'385, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b38e8a9:::obj385:head@111'385, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.683+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8b38e8a9:::obj385:head, version: 111'385, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b38e8a9:::obj385:head@111'385, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000050 MOSDPGPush(1.0 116/113 [PushOp(1:8b38e8a9:::obj385:head, version: 111'385, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b38e8a9:::obj385:head@111'385, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8aa18807:::obj353:head local-lis/les=113/114 n=322 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.683+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.683+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8aa18807:::obj353:head local-lis/les=113/114 n=322 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:8b38e8a9:::obj385:head, version: 111'385, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b38e8a9:::obj385:head@111'385, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.683+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8aa18807:::obj353:head local-lis/les=113/114 n=322 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8b38e8a9:::obj385:head@111'385, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.683+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8aa18807:::obj353:head local-lis/les=113/114 n=322 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:8b38e8a9:::obj385:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.683+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8aa18807:::obj353:head local-lis/les=113/114 n=322 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:8b38e8a9:::obj385:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.683+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8aa18807:::obj353:head local-lis/les=113/114 n=322 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:8b38e8a9:::obj385:head 2026-02-11T23:31:27.683+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8aa18807:::obj353:head local-lis/les=113/114 n=322 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:8b38e8a9:::obj385:head v 111'385 2026-02-11T23:31:27.683+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8aa18807:::obj353:head local-lis/les=113/114 n=322 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.683+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.683+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8b38e8a9:::obj385:head, version: 111'385, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b38e8a9:::obj385:head@111'385, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:27.683+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.683+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.683+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8b0915dc:::obj278:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.683+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8b0915dc:::obj278:head) v3 prio 15 cost 0 latency 0.000079 pg_backfill(progress 1.0 e 116/116 lb 1:8b0915dc:::obj278:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8aa18807:::obj353:head local-lis/les=113/114 n=322 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.683+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.683+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8aa18807:::obj353:head local-lis/les=113/114 n=322 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:8b0915dc:::obj278:head) v3 2026-02-11T23:31:27.683+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8aa18807:::obj353:head local-lis/les=113/114 n=322 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:8b0915dc:::obj278:head) 2026-02-11T23:31:27.683+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.683+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8b0915dc:::obj278:head) v3 finish latency 0.000011 2026-02-11T23:31:27.683+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8b0915dc:::obj278:head local-lis/les=113/114 n=323 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.683+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.683+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.683+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8b0915dc:::obj278:head local-lis/les=113/114 n=323 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.683+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.683+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.683+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.683+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x614ca80 con 0x4acb000 2026-02-11T23:31:27.683+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:8b38e8a9:::obj385:head)]) -- 0x613fc80 con 0x4acb000 2026-02-11T23:31:27.683+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.684+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 683 ==== MOSDPGPush(1.0 116/113 [PushOp(1:8b716ab6:::obj324:head, version: 111'324, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b716ab6:::obj324:head@111'324, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c340 con 0x4acb000 2026-02-11T23:31:27.684+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:8b716ab6:::obj324:head, version: 111'324, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b716ab6:::obj324:head@111'324, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 116 2026-02-11T23:31:27.684+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8b716ab6:::obj324:head, version: 111'324, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b716ab6:::obj324:head@111'324, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.684+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.684+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 684 ==== pg_backfill(progress 1.0 e 116/116 lb 1:8b38e8a9:::obj385:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e680 con 0x4acb000 2026-02-11T23:31:27.684+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:8b38e8a9:::obj385:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.684+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8b716ab6:::obj324:head, version: 111'324, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b716ab6:::obj324:head@111'324, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.684+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8b38e8a9:::obj385:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.684+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.684+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8b716ab6:::obj324:head, version: 111'324, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b716ab6:::obj324:head@111'324, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.684+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8b716ab6:::obj324:head, version: 111'324, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b716ab6:::obj324:head@111'324, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000040 MOSDPGPush(1.0 116/113 [PushOp(1:8b716ab6:::obj324:head, version: 111'324, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b716ab6:::obj324:head@111'324, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8b0915dc:::obj278:head local-lis/les=113/114 n=323 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.684+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.684+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.684+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8b0915dc:::obj278:head local-lis/les=113/114 n=323 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:8b716ab6:::obj324:head, version: 111'324, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b716ab6:::obj324:head@111'324, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.684+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8b38e8a9:::obj385:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.684+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8b0915dc:::obj278:head local-lis/les=113/114 n=323 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8b716ab6:::obj324:head@111'324, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.684+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8b0915dc:::obj278:head local-lis/les=113/114 n=323 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:8b716ab6:::obj324:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.684+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8b0915dc:::obj278:head local-lis/les=113/114 n=323 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:8b716ab6:::obj324:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.684+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8b0915dc:::obj278:head local-lis/les=113/114 n=323 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:8b716ab6:::obj324:head 2026-02-11T23:31:27.684+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8b0915dc:::obj278:head local-lis/les=113/114 n=323 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:8b716ab6:::obj324:head v 111'324 2026-02-11T23:31:27.684+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8b0915dc:::obj278:head local-lis/les=113/114 n=323 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.684+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.684+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8b716ab6:::obj324:head, version: 111'324, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b716ab6:::obj324:head@111'324, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:27.684+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.684+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.684+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8b38e8a9:::obj385:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.684+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8b38e8a9:::obj385:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 1.0 e 116/116 lb 1:8b38e8a9:::obj385:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8b0915dc:::obj278:head local-lis/les=113/114 n=323 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.684+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.684+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8b0915dc:::obj278:head local-lis/les=113/114 n=323 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:8b38e8a9:::obj385:head) v3 2026-02-11T23:31:27.684+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8b0915dc:::obj278:head local-lis/les=113/114 n=323 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:8b38e8a9:::obj385:head) 2026-02-11T23:31:27.684+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.684+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8b38e8a9:::obj385:head) v3 finish latency 0.000011 2026-02-11T23:31:27.684+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8b38e8a9:::obj385:head local-lis/les=113/114 n=324 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.684+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.684+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.684+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8b38e8a9:::obj385:head local-lis/les=113/114 n=324 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.684+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.684+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.684+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.684+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6015500 con 0x4acb000 2026-02-11T23:31:27.684+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:8b716ab6:::obj324:head)]) -- 0x6170000 con 0x4acb000 2026-02-11T23:31:27.684+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.684+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 685 ==== MOSDPGPush(1.0 116/113 [PushOp(1:8b7a7bb6:::obj294:head, version: 111'294, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b7a7bb6:::obj294:head@111'294, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559d1e0 con 0x4acb000 2026-02-11T23:31:27.684+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:8b7a7bb6:::obj294:head, version: 111'294, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b7a7bb6:::obj294:head@111'294, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000021 epoch 116 2026-02-11T23:31:27.684+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8b7a7bb6:::obj294:head, version: 111'294, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b7a7bb6:::obj294:head@111'294, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.684+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.684+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 686 ==== pg_backfill(progress 1.0 e 116/116 lb 1:8b716ab6:::obj324:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e100 con 0x4acb000 2026-02-11T23:31:27.684+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:8b716ab6:::obj324:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.684+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8b7a7bb6:::obj294:head, version: 111'294, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b7a7bb6:::obj294:head@111'294, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.684+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8b716ab6:::obj324:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.684+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.684+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8b7a7bb6:::obj294:head, version: 111'294, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b7a7bb6:::obj294:head@111'294, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.684+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8b7a7bb6:::obj294:head, version: 111'294, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b7a7bb6:::obj294:head@111'294, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000045 MOSDPGPush(1.0 116/113 [PushOp(1:8b7a7bb6:::obj294:head, version: 111'294, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b7a7bb6:::obj294:head@111'294, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8b38e8a9:::obj385:head local-lis/les=113/114 n=324 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.684+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.684+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.684+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8b38e8a9:::obj385:head local-lis/les=113/114 n=324 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:8b7a7bb6:::obj294:head, version: 111'294, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b7a7bb6:::obj294:head@111'294, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.684+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8b716ab6:::obj324:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.684+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8b38e8a9:::obj385:head local-lis/les=113/114 n=324 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8b7a7bb6:::obj294:head@111'294, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.684+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8b38e8a9:::obj385:head local-lis/les=113/114 n=324 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:8b7a7bb6:::obj294:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.684+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8b38e8a9:::obj385:head local-lis/les=113/114 n=324 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:8b7a7bb6:::obj294:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.684+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8b38e8a9:::obj385:head local-lis/les=113/114 n=324 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:8b7a7bb6:::obj294:head 2026-02-11T23:31:27.684+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8b38e8a9:::obj385:head local-lis/les=113/114 n=324 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:8b7a7bb6:::obj294:head v 111'294 2026-02-11T23:31:27.684+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8b38e8a9:::obj385:head local-lis/les=113/114 n=324 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.684+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.684+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8b7a7bb6:::obj294:head, version: 111'294, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b7a7bb6:::obj294:head@111'294, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:27.684+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.684+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.684+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8b716ab6:::obj324:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.684+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8b716ab6:::obj324:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 1.0 e 116/116 lb 1:8b716ab6:::obj324:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8b38e8a9:::obj385:head local-lis/les=113/114 n=324 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.684+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.684+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8b38e8a9:::obj385:head local-lis/les=113/114 n=324 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:8b716ab6:::obj324:head) v3 2026-02-11T23:31:27.684+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8b38e8a9:::obj385:head local-lis/les=113/114 n=324 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:8b716ab6:::obj324:head) 2026-02-11T23:31:27.684+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.684+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8b716ab6:::obj324:head) v3 finish latency 0.000011 2026-02-11T23:31:27.684+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8b716ab6:::obj324:head local-lis/les=113/114 n=325 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.684+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.684+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.684+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8b716ab6:::obj324:head local-lis/les=113/114 n=325 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.684+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.684+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.684+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.684+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x614c780 con 0x4acb000 2026-02-11T23:31:27.685+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:8b7a7bb6:::obj294:head)]) -- 0x6170300 con 0x4acb000 2026-02-11T23:31:27.685+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.685+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 687 ==== MOSDPGPush(1.0 116/113 [PushOp(1:8c123081:::obj473:head, version: 111'473, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8c123081:::obj473:head@111'473, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559cd00 con 0x4acb000 2026-02-11T23:31:27.685+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:8c123081:::obj473:head, version: 111'473, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8c123081:::obj473:head@111'473, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.685+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8c123081:::obj473:head, version: 111'473, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8c123081:::obj473:head@111'473, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.685+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.685+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8c123081:::obj473:head, version: 111'473, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8c123081:::obj473:head@111'473, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.685+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.685+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8c123081:::obj473:head, version: 111'473, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8c123081:::obj473:head@111'473, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.685+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 688 ==== pg_backfill(progress 1.0 e 116/116 lb 1:8b7a7bb6:::obj294:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176000 con 0x4acb000 2026-02-11T23:31:27.685+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:8b7a7bb6:::obj294:head) v3 prio 15 type 95 cost 0 latency 0.000012 epoch 116 2026-02-11T23:31:27.685+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.685+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8b7a7bb6:::obj294:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.685+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8c123081:::obj473:head, version: 111'473, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8c123081:::obj473:head@111'473, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 116/113 [PushOp(1:8c123081:::obj473:head, version: 111'473, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8c123081:::obj473:head@111'473, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8b716ab6:::obj324:head local-lis/les=113/114 n=325 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.685+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.685+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8b716ab6:::obj324:head local-lis/les=113/114 n=325 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:8c123081:::obj473:head, version: 111'473, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8c123081:::obj473:head@111'473, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.685+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.685+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8b716ab6:::obj324:head local-lis/les=113/114 n=325 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8c123081:::obj473:head@111'473, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.685+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8b7a7bb6:::obj294:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.685+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8b716ab6:::obj324:head local-lis/les=113/114 n=325 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:8c123081:::obj473:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.685+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8b716ab6:::obj324:head local-lis/les=113/114 n=325 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:8c123081:::obj473:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.685+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8b716ab6:::obj324:head local-lis/les=113/114 n=325 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:8c123081:::obj473:head 2026-02-11T23:31:27.685+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8b716ab6:::obj324:head local-lis/les=113/114 n=325 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:8c123081:::obj473:head v 111'473 2026-02-11T23:31:27.685+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8b716ab6:::obj324:head local-lis/les=113/114 n=325 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.685+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.685+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8c123081:::obj473:head, version: 111'473, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8c123081:::obj473:head@111'473, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:27.685+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.685+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.685+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8b7a7bb6:::obj294:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.685+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8b7a7bb6:::obj294:head) v3 prio 15 cost 0 latency 0.000080 pg_backfill(progress 1.0 e 116/116 lb 1:8b7a7bb6:::obj294:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8b716ab6:::obj324:head local-lis/les=113/114 n=325 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.685+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.685+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8b716ab6:::obj324:head local-lis/les=113/114 n=325 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:8b7a7bb6:::obj294:head) v3 2026-02-11T23:31:27.685+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8b716ab6:::obj324:head local-lis/les=113/114 n=325 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:8b7a7bb6:::obj294:head) 2026-02-11T23:31:27.685+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.685+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8b7a7bb6:::obj294:head) v3 finish latency 0.000010 2026-02-11T23:31:27.685+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8b7a7bb6:::obj294:head local-lis/les=113/114 n=326 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.685+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.685+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.685+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8b7a7bb6:::obj294:head local-lis/les=113/114 n=326 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.685+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.685+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.685+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.685+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6004900 con 0x4acb000 2026-02-11T23:31:27.685+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:8c123081:::obj473:head)]) -- 0x6170600 con 0x4acb000 2026-02-11T23:31:27.685+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.686+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 689 ==== MOSDPGPush(1.0 116/113 [PushOp(1:8c741e19:::obj599:head, version: 111'599, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8c741e19:::obj599:head@111'599, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f1380 con 0x4acb000 2026-02-11T23:31:27.686+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:8c741e19:::obj599:head, version: 111'599, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8c741e19:::obj599:head@111'599, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 116 2026-02-11T23:31:27.686+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8c741e19:::obj599:head, version: 111'599, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8c741e19:::obj599:head@111'599, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.686+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 690 ==== pg_backfill(progress 1.0 e 116/116 lb 1:8c123081:::obj473:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176580 con 0x4acb000 2026-02-11T23:31:27.686+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:8c123081:::obj473:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 116 2026-02-11T23:31:27.686+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.686+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8c123081:::obj473:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.686+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8c741e19:::obj599:head, version: 111'599, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8c741e19:::obj599:head@111'599, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.686+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.686+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8c741e19:::obj599:head, version: 111'599, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8c741e19:::obj599:head@111'599, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.686+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8c741e19:::obj599:head, version: 111'599, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8c741e19:::obj599:head@111'599, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000056 MOSDPGPush(1.0 116/113 [PushOp(1:8c741e19:::obj599:head, version: 111'599, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8c741e19:::obj599:head@111'599, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8b7a7bb6:::obj294:head local-lis/les=113/114 n=326 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.686+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.686+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.686+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8c123081:::obj473:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.686+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8b7a7bb6:::obj294:head local-lis/les=113/114 n=326 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:8c741e19:::obj599:head, version: 111'599, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8c741e19:::obj599:head@111'599, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.686+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8b7a7bb6:::obj294:head local-lis/les=113/114 n=326 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8c741e19:::obj599:head@111'599, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.686+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8b7a7bb6:::obj294:head local-lis/les=113/114 n=326 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:8c741e19:::obj599:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.686+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8b7a7bb6:::obj294:head local-lis/les=113/114 n=326 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:8c741e19:::obj599:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.686+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8b7a7bb6:::obj294:head local-lis/les=113/114 n=326 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:8c741e19:::obj599:head 2026-02-11T23:31:27.686+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8b7a7bb6:::obj294:head local-lis/les=113/114 n=326 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:8c741e19:::obj599:head v 111'599 2026-02-11T23:31:27.686+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8b7a7bb6:::obj294:head local-lis/les=113/114 n=326 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.686+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.686+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8c741e19:::obj599:head, version: 111'599, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8c741e19:::obj599:head@111'599, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000080 2026-02-11T23:31:27.686+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.686+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.686+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8c123081:::obj473:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.686+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8c123081:::obj473:head) v3 prio 15 cost 0 latency 0.000112 pg_backfill(progress 1.0 e 116/116 lb 1:8c123081:::obj473:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8b7a7bb6:::obj294:head local-lis/les=113/114 n=326 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.686+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.686+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8b7a7bb6:::obj294:head local-lis/les=113/114 n=326 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:8c123081:::obj473:head) v3 2026-02-11T23:31:27.686+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8b7a7bb6:::obj294:head local-lis/les=113/114 n=326 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:8c123081:::obj473:head) 2026-02-11T23:31:27.686+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.686+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8c123081:::obj473:head) v3 finish latency 0.000010 2026-02-11T23:31:27.686+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8c123081:::obj473:head local-lis/les=113/114 n=327 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.686+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.686+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.686+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8c123081:::obj473:head local-lis/les=113/114 n=327 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.686+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.686+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.686+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.686+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6015b00 con 0x4acb000 2026-02-11T23:31:27.686+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:8c741e19:::obj599:head)]) -- 0x6170900 con 0x4acb000 2026-02-11T23:31:27.686+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.686+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 691 ==== MOSDPGPush(1.0 116/113 [PushOp(1:8ca4bb13:::obj342:head, version: 111'342, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8ca4bb13:::obj342:head@111'342, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c000 con 0x4acb000 2026-02-11T23:31:27.686+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:8ca4bb13:::obj342:head, version: 111'342, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8ca4bb13:::obj342:head@111'342, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 116 2026-02-11T23:31:27.686+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8ca4bb13:::obj342:head, version: 111'342, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8ca4bb13:::obj342:head@111'342, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.686+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.686+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 692 ==== pg_backfill(progress 1.0 e 116/116 lb 1:8c741e19:::obj599:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176b00 con 0x4acb000 2026-02-11T23:31:27.686+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:8c741e19:::obj599:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.686+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8ca4bb13:::obj342:head, version: 111'342, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8ca4bb13:::obj342:head@111'342, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.686+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8c741e19:::obj599:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.686+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.686+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8ca4bb13:::obj342:head, version: 111'342, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8ca4bb13:::obj342:head@111'342, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.686+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.686+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8c741e19:::obj599:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.686+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8ca4bb13:::obj342:head, version: 111'342, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8ca4bb13:::obj342:head@111'342, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(1.0 116/113 [PushOp(1:8ca4bb13:::obj342:head, version: 111'342, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8ca4bb13:::obj342:head@111'342, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8c123081:::obj473:head local-lis/les=113/114 n=327 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.686+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.686+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8c123081:::obj473:head local-lis/les=113/114 n=327 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:8ca4bb13:::obj342:head, version: 111'342, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8ca4bb13:::obj342:head@111'342, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.686+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8c123081:::obj473:head local-lis/les=113/114 n=327 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8ca4bb13:::obj342:head@111'342, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.686+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8c123081:::obj473:head local-lis/les=113/114 n=327 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:8ca4bb13:::obj342:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.686+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8c123081:::obj473:head local-lis/les=113/114 n=327 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:8ca4bb13:::obj342:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.686+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8c123081:::obj473:head local-lis/les=113/114 n=327 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:8ca4bb13:::obj342:head 2026-02-11T23:31:27.686+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8c123081:::obj473:head local-lis/les=113/114 n=327 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:8ca4bb13:::obj342:head v 111'342 2026-02-11T23:31:27.686+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8c123081:::obj473:head local-lis/les=113/114 n=327 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.686+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.686+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8ca4bb13:::obj342:head, version: 111'342, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8ca4bb13:::obj342:head@111'342, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:27.686+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.686+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.686+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8c741e19:::obj599:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.686+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8c741e19:::obj599:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 1.0 e 116/116 lb 1:8c741e19:::obj599:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8c123081:::obj473:head local-lis/les=113/114 n=327 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.686+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.686+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8c123081:::obj473:head local-lis/les=113/114 n=327 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:8c741e19:::obj599:head) v3 2026-02-11T23:31:27.686+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8c123081:::obj473:head local-lis/les=113/114 n=327 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:8c741e19:::obj599:head) 2026-02-11T23:31:27.686+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.686+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8c741e19:::obj599:head) v3 finish latency 0.000010 2026-02-11T23:31:27.686+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8c741e19:::obj599:head local-lis/les=113/114 n=328 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.686+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.686+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.686+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8c741e19:::obj599:head local-lis/les=113/114 n=328 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.686+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.686+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.686+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.686+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6015800 con 0x4acb000 2026-02-11T23:31:27.686+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:8ca4bb13:::obj342:head)]) -- 0x6170c00 con 0x4acb000 2026-02-11T23:31:27.686+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.687+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 693 ==== MOSDPGPush(1.0 116/113 [PushOp(1:8cdc434f:::obj77:head, version: 111'77, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8cdc434f:::obj77:head@111'77, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x55f0d00 con 0x4acb000 2026-02-11T23:31:27.687+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:8cdc434f:::obj77:head, version: 111'77, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8cdc434f:::obj77:head@111'77, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 116 2026-02-11T23:31:27.687+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8cdc434f:::obj77:head, version: 111'77, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8cdc434f:::obj77:head@111'77, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.687+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.687+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 694 ==== pg_backfill(progress 1.0 e 116/116 lb 1:8ca4bb13:::obj342:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177080 con 0x4acb000 2026-02-11T23:31:27.687+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:8ca4bb13:::obj342:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.687+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8cdc434f:::obj77:head, version: 111'77, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8cdc434f:::obj77:head@111'77, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.687+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8ca4bb13:::obj342:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.687+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.687+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8cdc434f:::obj77:head, version: 111'77, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8cdc434f:::obj77:head@111'77, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.687+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8cdc434f:::obj77:head, version: 111'77, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8cdc434f:::obj77:head@111'77, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(1.0 116/113 [PushOp(1:8cdc434f:::obj77:head, version: 111'77, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8cdc434f:::obj77:head@111'77, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8c741e19:::obj599:head local-lis/les=113/114 n=328 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.687+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.687+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8c741e19:::obj599:head local-lis/les=113/114 n=328 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:8cdc434f:::obj77:head, version: 111'77, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8cdc434f:::obj77:head@111'77, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.687+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.687+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8c741e19:::obj599:head local-lis/les=113/114 n=328 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8cdc434f:::obj77:head@111'77, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.687+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8ca4bb13:::obj342:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.687+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8c741e19:::obj599:head local-lis/les=113/114 n=328 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:8cdc434f:::obj77:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.687+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8c741e19:::obj599:head local-lis/les=113/114 n=328 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:8cdc434f:::obj77:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.687+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8c741e19:::obj599:head local-lis/les=113/114 n=328 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:8cdc434f:::obj77:head 2026-02-11T23:31:27.687+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8c741e19:::obj599:head local-lis/les=113/114 n=328 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:8cdc434f:::obj77:head v 111'77 2026-02-11T23:31:27.687+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8c741e19:::obj599:head local-lis/les=113/114 n=328 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.687+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.687+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8cdc434f:::obj77:head, version: 111'77, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8cdc434f:::obj77:head@111'77, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:27.687+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.687+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.687+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8ca4bb13:::obj342:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.687+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8ca4bb13:::obj342:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 1.0 e 116/116 lb 1:8ca4bb13:::obj342:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8c741e19:::obj599:head local-lis/les=113/114 n=328 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.687+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.687+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8c741e19:::obj599:head local-lis/les=113/114 n=328 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:8ca4bb13:::obj342:head) v3 2026-02-11T23:31:27.687+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8c741e19:::obj599:head local-lis/les=113/114 n=328 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:8ca4bb13:::obj342:head) 2026-02-11T23:31:27.687+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.687+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8ca4bb13:::obj342:head) v3 finish latency 0.000010 2026-02-11T23:31:27.687+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8ca4bb13:::obj342:head local-lis/les=113/114 n=329 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.687+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.687+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.687+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8ca4bb13:::obj342:head local-lis/les=113/114 n=329 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.687+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.687+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.687+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.687+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x600a180 con 0x4acb000 2026-02-11T23:31:27.687+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:8cdc434f:::obj77:head)]) -- 0x6170f00 con 0x4acb000 2026-02-11T23:31:27.687+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.688+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 695 ==== MOSDPGPush(1.0 116/113 [PushOp(1:8d29890f:::obj305:head, version: 111'305, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d29890f:::obj305:head@111'305, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f1860 con 0x4acb000 2026-02-11T23:31:27.688+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:8d29890f:::obj305:head, version: 111'305, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d29890f:::obj305:head@111'305, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.688+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8d29890f:::obj305:head, version: 111'305, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d29890f:::obj305:head@111'305, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.688+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 696 ==== pg_backfill(progress 1.0 e 116/116 lb 1:8cdc434f:::obj77:head) ==== 1061+0+0 (unknown 0 0 0) 0x6177600 con 0x4acb000 2026-02-11T23:31:27.688+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:8cdc434f:::obj77:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 116 2026-02-11T23:31:27.688+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8cdc434f:::obj77:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.688+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.688+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8d29890f:::obj305:head, version: 111'305, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d29890f:::obj305:head@111'305, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.688+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.688+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8d29890f:::obj305:head, version: 111'305, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d29890f:::obj305:head@111'305, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.688+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8d29890f:::obj305:head, version: 111'305, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d29890f:::obj305:head@111'305, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000041 MOSDPGPush(1.0 116/113 [PushOp(1:8d29890f:::obj305:head, version: 111'305, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d29890f:::obj305:head@111'305, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8ca4bb13:::obj342:head local-lis/les=113/114 n=329 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.688+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.688+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.688+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8ca4bb13:::obj342:head local-lis/les=113/114 n=329 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:8d29890f:::obj305:head, version: 111'305, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d29890f:::obj305:head@111'305, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.688+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8cdc434f:::obj77:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.688+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8ca4bb13:::obj342:head local-lis/les=113/114 n=329 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8d29890f:::obj305:head@111'305, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.688+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8ca4bb13:::obj342:head local-lis/les=113/114 n=329 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:8d29890f:::obj305:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.688+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8ca4bb13:::obj342:head local-lis/les=113/114 n=329 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:8d29890f:::obj305:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.688+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8ca4bb13:::obj342:head local-lis/les=113/114 n=329 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:8d29890f:::obj305:head 2026-02-11T23:31:27.688+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8ca4bb13:::obj342:head local-lis/les=113/114 n=329 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:8d29890f:::obj305:head v 111'305 2026-02-11T23:31:27.688+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8ca4bb13:::obj342:head local-lis/les=113/114 n=329 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.688+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.688+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8d29890f:::obj305:head, version: 111'305, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d29890f:::obj305:head@111'305, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:27.688+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8ca4bb13:::obj342:head local-lis/les=113/114 n=329 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.688+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.688+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.688+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8cdc434f:::obj77:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.688+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8cdc434f:::obj77:head) v3 prio 15 cost 0 latency 0.000085 pg_backfill(progress 1.0 e 116/116 lb 1:8cdc434f:::obj77:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8ca4bb13:::obj342:head local-lis/les=113/114 n=329 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.688+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.688+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8ca4bb13:::obj342:head local-lis/les=113/114 n=329 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:8cdc434f:::obj77:head) v3 2026-02-11T23:31:27.688+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8ca4bb13:::obj342:head local-lis/les=113/114 n=329 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:8cdc434f:::obj77:head) 2026-02-11T23:31:27.688+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.688+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8cdc434f:::obj77:head) v3 finish latency 0.000015 2026-02-11T23:31:27.688+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.688+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.688+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8cdc434f:::obj77:head local-lis/les=113/114 n=330 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.688+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.688+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.688+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.688+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6004480 con 0x4acb000 2026-02-11T23:31:27.688+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:8d29890f:::obj305:head)]) -- 0x6004600 con 0x4acb000 2026-02-11T23:31:27.688+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.688+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 697 ==== MOSDPGPush(1.0 116/113 [PushOp(1:8d384ce8:::obj297:head, version: 111'297, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d384ce8:::obj297:head@111'297, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c4e0 con 0x4acb000 2026-02-11T23:31:27.688+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:8d384ce8:::obj297:head, version: 111'297, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d384ce8:::obj297:head@111'297, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.688+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8d384ce8:::obj297:head, version: 111'297, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d384ce8:::obj297:head@111'297, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.688+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 698 ==== pg_backfill(progress 1.0 e 116/116 lb 1:8d29890f:::obj305:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177b80 con 0x4acb000 2026-02-11T23:31:27.688+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:8d29890f:::obj305:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.688+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.688+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8d29890f:::obj305:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.688+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8d384ce8:::obj297:head, version: 111'297, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d384ce8:::obj297:head@111'297, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.688+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.688+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8d384ce8:::obj297:head, version: 111'297, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d384ce8:::obj297:head@111'297, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.688+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8d384ce8:::obj297:head, version: 111'297, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d384ce8:::obj297:head@111'297, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000040 MOSDPGPush(1.0 116/113 [PushOp(1:8d384ce8:::obj297:head, version: 111'297, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d384ce8:::obj297:head@111'297, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8cdc434f:::obj77:head local-lis/les=113/114 n=330 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.688+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.688+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.688+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8d29890f:::obj305:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.688+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8cdc434f:::obj77:head local-lis/les=113/114 n=330 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:8d384ce8:::obj297:head, version: 111'297, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d384ce8:::obj297:head@111'297, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.688+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8cdc434f:::obj77:head local-lis/les=113/114 n=330 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8d384ce8:::obj297:head@111'297, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.688+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8cdc434f:::obj77:head local-lis/les=113/114 n=330 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:8d384ce8:::obj297:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.688+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8cdc434f:::obj77:head local-lis/les=113/114 n=330 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:8d384ce8:::obj297:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.688+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8cdc434f:::obj77:head local-lis/les=113/114 n=330 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:8d384ce8:::obj297:head 2026-02-11T23:31:27.688+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8cdc434f:::obj77:head local-lis/les=113/114 n=330 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:8d384ce8:::obj297:head v 111'297 2026-02-11T23:31:27.688+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8cdc434f:::obj77:head local-lis/les=113/114 n=330 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.688+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.688+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8d384ce8:::obj297:head, version: 111'297, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d384ce8:::obj297:head@111'297, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:27.688+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.688+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.688+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8d29890f:::obj305:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.688+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8d29890f:::obj305:head) v3 prio 15 cost 0 latency 0.000085 pg_backfill(progress 1.0 e 116/116 lb 1:8d29890f:::obj305:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8cdc434f:::obj77:head local-lis/les=113/114 n=330 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.688+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.688+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8cdc434f:::obj77:head local-lis/les=113/114 n=330 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:8d29890f:::obj305:head) v3 2026-02-11T23:31:27.688+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8cdc434f:::obj77:head local-lis/les=113/114 n=330 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:8d29890f:::obj305:head) 2026-02-11T23:31:27.688+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.688+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8d29890f:::obj305:head) v3 finish latency 0.000012 2026-02-11T23:31:27.688+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8d29890f:::obj305:head local-lis/les=113/114 n=331 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.688+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.688+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.688+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8d29890f:::obj305:head local-lis/les=113/114 n=331 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.688+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.688+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.688+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.688+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x614c180 con 0x4acb000 2026-02-11T23:31:27.688+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:8d384ce8:::obj297:head)]) -- 0x6171080 con 0x4acb000 2026-02-11T23:31:27.688+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.689+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 699 ==== MOSDPGPush(1.0 116/113 [PushOp(1:8d6bd12e:::obj368:head, version: 111'368, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d6bd12e:::obj368:head@111'368, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559da00 con 0x4acb000 2026-02-11T23:31:27.689+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:8d6bd12e:::obj368:head, version: 111'368, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d6bd12e:::obj368:head@111'368, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.689+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8d6bd12e:::obj368:head, version: 111'368, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d6bd12e:::obj368:head@111'368, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.689+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 700 ==== pg_backfill(progress 1.0 e 116/116 lb 1:8d384ce8:::obj297:head) ==== 1062+0+0 (unknown 0 0 0) 0x6178100 con 0x4acb000 2026-02-11T23:31:27.689+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.689+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:8d384ce8:::obj297:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 116 2026-02-11T23:31:27.689+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8d6bd12e:::obj368:head, version: 111'368, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d6bd12e:::obj368:head@111'368, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.689+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8d384ce8:::obj297:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.689+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.689+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8d6bd12e:::obj368:head, version: 111'368, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d6bd12e:::obj368:head@111'368, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.689+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8d6bd12e:::obj368:head, version: 111'368, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d6bd12e:::obj368:head@111'368, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(1.0 116/113 [PushOp(1:8d6bd12e:::obj368:head, version: 111'368, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d6bd12e:::obj368:head@111'368, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8d29890f:::obj305:head local-lis/les=113/114 n=331 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.689+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.689+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.689+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8d384ce8:::obj297:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.689+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8d29890f:::obj305:head local-lis/les=113/114 n=331 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:8d6bd12e:::obj368:head, version: 111'368, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d6bd12e:::obj368:head@111'368, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.689+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8d29890f:::obj305:head local-lis/les=113/114 n=331 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8d6bd12e:::obj368:head@111'368, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.689+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8d29890f:::obj305:head local-lis/les=113/114 n=331 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:8d6bd12e:::obj368:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.689+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8d29890f:::obj305:head local-lis/les=113/114 n=331 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:8d6bd12e:::obj368:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.689+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8d29890f:::obj305:head local-lis/les=113/114 n=331 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:8d6bd12e:::obj368:head 2026-02-11T23:31:27.689+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8d29890f:::obj305:head local-lis/les=113/114 n=331 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:8d6bd12e:::obj368:head v 111'368 2026-02-11T23:31:27.689+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8d29890f:::obj305:head local-lis/les=113/114 n=331 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.689+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.689+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8d6bd12e:::obj368:head, version: 111'368, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d6bd12e:::obj368:head@111'368, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:27.689+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.689+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.689+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8d384ce8:::obj297:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.689+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8d384ce8:::obj297:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 1.0 e 116/116 lb 1:8d384ce8:::obj297:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8d29890f:::obj305:head local-lis/les=113/114 n=331 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.689+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.689+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8d29890f:::obj305:head local-lis/les=113/114 n=331 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:8d384ce8:::obj297:head) v3 2026-02-11T23:31:27.689+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8d29890f:::obj305:head local-lis/les=113/114 n=331 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:8d384ce8:::obj297:head) 2026-02-11T23:31:27.689+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.689+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8d384ce8:::obj297:head) v3 finish latency 0.000010 2026-02-11T23:31:27.689+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8d384ce8:::obj297:head local-lis/les=113/114 n=332 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.689+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.689+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.689+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8d384ce8:::obj297:head local-lis/les=113/114 n=332 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.689+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.689+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.689+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.689+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fcf080 con 0x4acb000 2026-02-11T23:31:27.689+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:8d6bd12e:::obj368:head)]) -- 0x6171380 con 0x4acb000 2026-02-11T23:31:27.689+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.689+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 701 ==== MOSDPGPush(1.0 116/113 [PushOp(1:8dd16f86:::obj3:head, version: 111'3, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8dd16f86:::obj3:head@111'3, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 4999+0+0 (unknown 0 0 0) 0x559cea0 con 0x4acb000 2026-02-11T23:31:27.689+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:8dd16f86:::obj3:head, version: 111'3, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8dd16f86:::obj3:head@111'3, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 116 2026-02-11T23:31:27.689+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8dd16f86:::obj3:head, version: 111'3, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8dd16f86:::obj3:head@111'3, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.689+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.689+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 702 ==== pg_backfill(progress 1.0 e 116/116 lb 1:8d6bd12e:::obj368:head) ==== 1062+0+0 (unknown 0 0 0) 0x6178680 con 0x4acb000 2026-02-11T23:31:27.689+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8dd16f86:::obj3:head, version: 111'3, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8dd16f86:::obj3:head@111'3, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.689+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:8d6bd12e:::obj368:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.689+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.689+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8d6bd12e:::obj368:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.689+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8dd16f86:::obj3:head, version: 111'3, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8dd16f86:::obj3:head@111'3, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.690+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.690+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8dd16f86:::obj3:head, version: 111'3, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8dd16f86:::obj3:head@111'3, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(1.0 116/113 [PushOp(1:8dd16f86:::obj3:head, version: 111'3, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8dd16f86:::obj3:head@111'3, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8d384ce8:::obj297:head local-lis/les=113/114 n=332 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.690+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8d6bd12e:::obj368:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.690+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.690+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8d384ce8:::obj297:head local-lis/les=113/114 n=332 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:8dd16f86:::obj3:head, version: 111'3, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8dd16f86:::obj3:head@111'3, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.690+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8d384ce8:::obj297:head local-lis/les=113/114 n=332 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8dd16f86:::obj3:head@111'3, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.690+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8d384ce8:::obj297:head local-lis/les=113/114 n=332 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:8dd16f86:::obj3:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.690+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8d384ce8:::obj297:head local-lis/les=113/114 n=332 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:8dd16f86:::obj3:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.690+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8d384ce8:::obj297:head local-lis/les=113/114 n=332 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:8dd16f86:::obj3:head 2026-02-11T23:31:27.690+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8d384ce8:::obj297:head local-lis/les=113/114 n=332 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:8dd16f86:::obj3:head v 111'3 2026-02-11T23:31:27.690+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8d384ce8:::obj297:head local-lis/les=113/114 n=332 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.690+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.690+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8dd16f86:::obj3:head, version: 111'3, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8dd16f86:::obj3:head@111'3, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.690+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.690+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.690+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8d6bd12e:::obj368:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.690+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8d6bd12e:::obj368:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 1.0 e 116/116 lb 1:8d6bd12e:::obj368:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8d384ce8:::obj297:head local-lis/les=113/114 n=332 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.690+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.690+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8d384ce8:::obj297:head local-lis/les=113/114 n=332 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:8d6bd12e:::obj368:head) v3 2026-02-11T23:31:27.690+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8d384ce8:::obj297:head local-lis/les=113/114 n=332 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:8d6bd12e:::obj368:head) 2026-02-11T23:31:27.690+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.690+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8d6bd12e:::obj368:head) v3 finish latency 0.000011 2026-02-11T23:31:27.690+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8d6bd12e:::obj368:head local-lis/les=113/114 n=333 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.690+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.690+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.690+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8d6bd12e:::obj368:head local-lis/les=113/114 n=333 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.690+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.690+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.690+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.690+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6004600 con 0x4acb000 2026-02-11T23:31:27.690+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:8dd16f86:::obj3:head)]) -- 0x6171680 con 0x4acb000 2026-02-11T23:31:27.690+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.690+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 703 ==== MOSDPGPush(1.0 116/113 [PushOp(1:8e988874:::obj200:head, version: 111'200, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8e988874:::obj200:head@111'200, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d8820 con 0x4acb000 2026-02-11T23:31:27.690+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:8e988874:::obj200:head, version: 111'200, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8e988874:::obj200:head@111'200, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 116 2026-02-11T23:31:27.690+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8e988874:::obj200:head, version: 111'200, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8e988874:::obj200:head@111'200, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.690+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.690+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 704 ==== pg_backfill(progress 1.0 e 116/116 lb 1:8dd16f86:::obj3:head) ==== 1060+0+0 (unknown 0 0 0) 0x6178c00 con 0x4acb000 2026-02-11T23:31:27.690+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8e988874:::obj200:head, version: 111'200, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8e988874:::obj200:head@111'200, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.690+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:8dd16f86:::obj3:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.690+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8dd16f86:::obj3:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.690+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.690+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8e988874:::obj200:head, version: 111'200, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8e988874:::obj200:head@111'200, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.690+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8e988874:::obj200:head, version: 111'200, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8e988874:::obj200:head@111'200, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 116/113 [PushOp(1:8e988874:::obj200:head, version: 111'200, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8e988874:::obj200:head@111'200, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8d6bd12e:::obj368:head local-lis/les=113/114 n=333 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.690+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.690+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.690+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8d6bd12e:::obj368:head local-lis/les=113/114 n=333 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:8e988874:::obj200:head, version: 111'200, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8e988874:::obj200:head@111'200, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.690+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8dd16f86:::obj3:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.690+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8d6bd12e:::obj368:head local-lis/les=113/114 n=333 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8e988874:::obj200:head@111'200, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.690+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8d6bd12e:::obj368:head local-lis/les=113/114 n=333 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:8e988874:::obj200:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.690+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8d6bd12e:::obj368:head local-lis/les=113/114 n=333 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:8e988874:::obj200:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.690+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8d6bd12e:::obj368:head local-lis/les=113/114 n=333 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:8e988874:::obj200:head 2026-02-11T23:31:27.690+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8d6bd12e:::obj368:head local-lis/les=113/114 n=333 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:8e988874:::obj200:head v 111'200 2026-02-11T23:31:27.690+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8d6bd12e:::obj368:head local-lis/les=113/114 n=333 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.690+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.690+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8e988874:::obj200:head, version: 111'200, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8e988874:::obj200:head@111'200, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.690+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.690+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.690+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8dd16f86:::obj3:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.690+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8dd16f86:::obj3:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:8dd16f86:::obj3:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8d6bd12e:::obj368:head local-lis/les=113/114 n=333 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.690+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.690+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8d6bd12e:::obj368:head local-lis/les=113/114 n=333 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:8dd16f86:::obj3:head) v3 2026-02-11T23:31:27.690+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8d6bd12e:::obj368:head local-lis/les=113/114 n=333 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:8dd16f86:::obj3:head) 2026-02-11T23:31:27.690+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.690+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8dd16f86:::obj3:head) v3 finish latency 0.000010 2026-02-11T23:31:27.690+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8dd16f86:::obj3:head local-lis/les=113/114 n=334 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.690+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.690+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.690+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8dd16f86:::obj3:head local-lis/les=113/114 n=334 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.690+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.690+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.690+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.690+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6004480 con 0x4acb000 2026-02-11T23:31:27.690+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:8e988874:::obj200:head)]) -- 0x6171800 con 0x4acb000 2026-02-11T23:31:27.690+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.691+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 705 ==== MOSDPGPush(1.0 116/113 [PushOp(1:8eb14ea4:::obj384:head, version: 111'384, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8eb14ea4:::obj384:head@111'384, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56ca000 con 0x4acb000 2026-02-11T23:31:27.691+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:8eb14ea4:::obj384:head, version: 111'384, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8eb14ea4:::obj384:head@111'384, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000018 epoch 116 2026-02-11T23:31:27.691+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8eb14ea4:::obj384:head, version: 111'384, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8eb14ea4:::obj384:head@111'384, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.691+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.691+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 706 ==== pg_backfill(progress 1.0 e 116/116 lb 1:8e988874:::obj200:head) ==== 1062+0+0 (unknown 0 0 0) 0x6179180 con 0x4acb000 2026-02-11T23:31:27.691+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:8e988874:::obj200:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.691+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8e988874:::obj200:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.691+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8eb14ea4:::obj384:head, version: 111'384, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8eb14ea4:::obj384:head@111'384, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.691+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.691+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8eb14ea4:::obj384:head, version: 111'384, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8eb14ea4:::obj384:head@111'384, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.691+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8eb14ea4:::obj384:head, version: 111'384, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8eb14ea4:::obj384:head@111'384, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000052 MOSDPGPush(1.0 116/113 [PushOp(1:8eb14ea4:::obj384:head, version: 111'384, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8eb14ea4:::obj384:head@111'384, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8dd16f86:::obj3:head local-lis/les=113/114 n=334 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.691+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.691+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8dd16f86:::obj3:head local-lis/les=113/114 n=334 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:8eb14ea4:::obj384:head, version: 111'384, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8eb14ea4:::obj384:head@111'384, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.691+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.691+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8dd16f86:::obj3:head local-lis/les=113/114 n=334 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8eb14ea4:::obj384:head@111'384, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.691+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8e988874:::obj200:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.691+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8dd16f86:::obj3:head local-lis/les=113/114 n=334 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:8eb14ea4:::obj384:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.691+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8dd16f86:::obj3:head local-lis/les=113/114 n=334 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:8eb14ea4:::obj384:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.691+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8dd16f86:::obj3:head local-lis/les=113/114 n=334 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:8eb14ea4:::obj384:head 2026-02-11T23:31:27.691+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8dd16f86:::obj3:head local-lis/les=113/114 n=334 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:8eb14ea4:::obj384:head v 111'384 2026-02-11T23:31:27.691+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8dd16f86:::obj3:head local-lis/les=113/114 n=334 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.691+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.691+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8eb14ea4:::obj384:head, version: 111'384, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8eb14ea4:::obj384:head@111'384, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:27.691+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8dd16f86:::obj3:head local-lis/les=113/114 n=334 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.691+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.691+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.691+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8e988874:::obj200:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.691+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8e988874:::obj200:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 1.0 e 116/116 lb 1:8e988874:::obj200:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8dd16f86:::obj3:head local-lis/les=113/114 n=334 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.691+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.691+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8dd16f86:::obj3:head local-lis/les=113/114 n=334 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:8e988874:::obj200:head) v3 2026-02-11T23:31:27.691+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8dd16f86:::obj3:head local-lis/les=113/114 n=334 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:8e988874:::obj200:head) 2026-02-11T23:31:27.691+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.691+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8e988874:::obj200:head) v3 finish latency 0.000010 2026-02-11T23:31:27.691+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.691+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.691+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8e988874:::obj200:head local-lis/les=113/114 n=335 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.691+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.691+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.691+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.691+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6170c00 con 0x4acb000 2026-02-11T23:31:27.691+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:8eb14ea4:::obj384:head)]) -- 0x6170f00 con 0x4acb000 2026-02-11T23:31:27.691+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.691+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 707 ==== MOSDPGPush(1.0 116/113 [PushOp(1:8f4c4fa9:::obj590:head, version: 111'590, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8f4c4fa9:::obj590:head@111'590, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d96c0 con 0x4acb000 2026-02-11T23:31:27.691+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:8f4c4fa9:::obj590:head, version: 111'590, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8f4c4fa9:::obj590:head@111'590, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000018 epoch 116 2026-02-11T23:31:27.691+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8f4c4fa9:::obj590:head, version: 111'590, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8f4c4fa9:::obj590:head@111'590, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.691+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.691+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8f4c4fa9:::obj590:head, version: 111'590, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8f4c4fa9:::obj590:head@111'590, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.691+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 708 ==== pg_backfill(progress 1.0 e 116/116 lb 1:8eb14ea4:::obj384:head) ==== 1062+0+0 (unknown 0 0 0) 0x6179700 con 0x4acb000 2026-02-11T23:31:27.691+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:8eb14ea4:::obj384:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.691+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.691+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8eb14ea4:::obj384:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.691+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8f4c4fa9:::obj590:head, version: 111'590, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8f4c4fa9:::obj590:head@111'590, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.691+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.691+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8f4c4fa9:::obj590:head, version: 111'590, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8f4c4fa9:::obj590:head@111'590, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000046 MOSDPGPush(1.0 116/113 [PushOp(1:8f4c4fa9:::obj590:head, version: 111'590, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8f4c4fa9:::obj590:head@111'590, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8e988874:::obj200:head local-lis/les=113/114 n=335 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.691+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.691+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8e988874:::obj200:head local-lis/les=113/114 n=335 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:8f4c4fa9:::obj590:head, version: 111'590, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8f4c4fa9:::obj590:head@111'590, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.691+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8e988874:::obj200:head local-lis/les=113/114 n=335 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8f4c4fa9:::obj590:head@111'590, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.691+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8e988874:::obj200:head local-lis/les=113/114 n=335 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:8f4c4fa9:::obj590:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.691+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.691+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8e988874:::obj200:head local-lis/les=113/114 n=335 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:8f4c4fa9:::obj590:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.691+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8eb14ea4:::obj384:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.691+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8e988874:::obj200:head local-lis/les=113/114 n=335 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:8f4c4fa9:::obj590:head 2026-02-11T23:31:27.691+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8e988874:::obj200:head local-lis/les=113/114 n=335 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:8f4c4fa9:::obj590:head v 111'590 2026-02-11T23:31:27.691+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8e988874:::obj200:head local-lis/les=113/114 n=335 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.691+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.691+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8f4c4fa9:::obj590:head, version: 111'590, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8f4c4fa9:::obj590:head@111'590, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.691+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.691+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.691+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8eb14ea4:::obj384:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.691+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8eb14ea4:::obj384:head) v3 prio 15 cost 0 latency 0.000060 pg_backfill(progress 1.0 e 116/116 lb 1:8eb14ea4:::obj384:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8e988874:::obj200:head local-lis/les=113/114 n=335 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.691+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.691+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8e988874:::obj200:head local-lis/les=113/114 n=335 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:8eb14ea4:::obj384:head) v3 2026-02-11T23:31:27.691+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8e988874:::obj200:head local-lis/les=113/114 n=335 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:8eb14ea4:::obj384:head) 2026-02-11T23:31:27.691+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.691+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8eb14ea4:::obj384:head) v3 finish latency 0.000011 2026-02-11T23:31:27.691+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8eb14ea4:::obj384:head local-lis/les=113/114 n=336 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.691+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.692+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.692+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8eb14ea4:::obj384:head local-lis/les=113/114 n=336 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.692+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.692+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.692+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.692+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6015800 con 0x4acb000 2026-02-11T23:31:27.692+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:8f4c4fa9:::obj590:head)]) -- 0x6171b00 con 0x4acb000 2026-02-11T23:31:27.692+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.692+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 709 ==== MOSDPGPush(1.0 116/113 [PushOp(1:8f7cdb2a:::obj538:head, version: 111'538, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8f7cdb2a:::obj538:head@111'538, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d91e0 con 0x4acb000 2026-02-11T23:31:27.692+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:8f7cdb2a:::obj538:head, version: 111'538, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8f7cdb2a:::obj538:head@111'538, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 116 2026-02-11T23:31:27.692+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8f7cdb2a:::obj538:head, version: 111'538, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8f7cdb2a:::obj538:head@111'538, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.692+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.692+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8f7cdb2a:::obj538:head, version: 111'538, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8f7cdb2a:::obj538:head@111'538, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.692+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 710 ==== pg_backfill(progress 1.0 e 116/116 lb 1:8f4c4fa9:::obj590:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176000 con 0x4acb000 2026-02-11T23:31:27.692+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:8f4c4fa9:::obj590:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.692+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.692+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8f4c4fa9:::obj590:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.692+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8f7cdb2a:::obj538:head, version: 111'538, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8f7cdb2a:::obj538:head@111'538, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.692+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.692+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8f7cdb2a:::obj538:head, version: 111'538, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8f7cdb2a:::obj538:head@111'538, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(1.0 116/113 [PushOp(1:8f7cdb2a:::obj538:head, version: 111'538, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8f7cdb2a:::obj538:head@111'538, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8eb14ea4:::obj384:head local-lis/les=113/114 n=336 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.692+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.692+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8eb14ea4:::obj384:head local-lis/les=113/114 n=336 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:8f7cdb2a:::obj538:head, version: 111'538, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8f7cdb2a:::obj538:head@111'538, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.692+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8eb14ea4:::obj384:head local-lis/les=113/114 n=336 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8f7cdb2a:::obj538:head@111'538, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.692+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.692+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8f4c4fa9:::obj590:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.692+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8eb14ea4:::obj384:head local-lis/les=113/114 n=336 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:8f7cdb2a:::obj538:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.692+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8eb14ea4:::obj384:head local-lis/les=113/114 n=336 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:8f7cdb2a:::obj538:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.692+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8eb14ea4:::obj384:head local-lis/les=113/114 n=336 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:8f7cdb2a:::obj538:head 2026-02-11T23:31:27.692+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8eb14ea4:::obj384:head local-lis/les=113/114 n=336 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:8f7cdb2a:::obj538:head v 111'538 2026-02-11T23:31:27.692+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8eb14ea4:::obj384:head local-lis/les=113/114 n=336 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.692+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.692+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8f7cdb2a:::obj538:head, version: 111'538, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8f7cdb2a:::obj538:head@111'538, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:27.692+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.692+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.692+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8f4c4fa9:::obj590:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.692+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8f4c4fa9:::obj590:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:8f4c4fa9:::obj590:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8eb14ea4:::obj384:head local-lis/les=113/114 n=336 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.692+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.692+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8eb14ea4:::obj384:head local-lis/les=113/114 n=336 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:8f4c4fa9:::obj590:head) v3 2026-02-11T23:31:27.692+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8eb14ea4:::obj384:head local-lis/les=113/114 n=336 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:8f4c4fa9:::obj590:head) 2026-02-11T23:31:27.692+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.692+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8f4c4fa9:::obj590:head) v3 finish latency 0.000010 2026-02-11T23:31:27.692+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8f4c4fa9:::obj590:head local-lis/les=113/114 n=337 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.692+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.692+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.692+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8f4c4fa9:::obj590:head local-lis/les=113/114 n=337 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.692+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.692+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.692+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.692+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6170900 con 0x4acb000 2026-02-11T23:31:27.692+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:8f7cdb2a:::obj538:head)]) -- 0x6171e00 con 0x4acb000 2026-02-11T23:31:27.692+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.693+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 711 ==== MOSDPGPush(1.0 116/113 [PushOp(1:8fcfcc34:::obj221:head, version: 111'221, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8fcfcc34:::obj221:head@111'221, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d8d00 con 0x4acb000 2026-02-11T23:31:27.693+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:8fcfcc34:::obj221:head, version: 111'221, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8fcfcc34:::obj221:head@111'221, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.693+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8fcfcc34:::obj221:head, version: 111'221, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8fcfcc34:::obj221:head@111'221, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.693+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.693+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 712 ==== pg_backfill(progress 1.0 e 116/116 lb 1:8f7cdb2a:::obj538:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176b00 con 0x4acb000 2026-02-11T23:31:27.693+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8fcfcc34:::obj221:head, version: 111'221, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8fcfcc34:::obj221:head@111'221, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.693+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:8f7cdb2a:::obj538:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 116 2026-02-11T23:31:27.693+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.693+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8f7cdb2a:::obj538:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.693+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:8fcfcc34:::obj221:head, version: 111'221, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8fcfcc34:::obj221:head@111'221, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.693+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.693+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8fcfcc34:::obj221:head, version: 111'221, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8fcfcc34:::obj221:head@111'221, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(1.0 116/113 [PushOp(1:8fcfcc34:::obj221:head, version: 111'221, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8fcfcc34:::obj221:head@111'221, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8f4c4fa9:::obj590:head local-lis/les=113/114 n=337 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.693+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.693+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8f4c4fa9:::obj590:head local-lis/les=113/114 n=337 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:8fcfcc34:::obj221:head, version: 111'221, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8fcfcc34:::obj221:head@111'221, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.693+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.693+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8f7cdb2a:::obj538:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.693+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8f4c4fa9:::obj590:head local-lis/les=113/114 n=337 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8fcfcc34:::obj221:head@111'221, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.693+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8f4c4fa9:::obj590:head local-lis/les=113/114 n=337 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:8fcfcc34:::obj221:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.693+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8f4c4fa9:::obj590:head local-lis/les=113/114 n=337 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:8fcfcc34:::obj221:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.693+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8f4c4fa9:::obj590:head local-lis/les=113/114 n=337 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:8fcfcc34:::obj221:head 2026-02-11T23:31:27.693+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8f4c4fa9:::obj590:head local-lis/les=113/114 n=337 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:8fcfcc34:::obj221:head v 111'221 2026-02-11T23:31:27.693+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8f4c4fa9:::obj590:head local-lis/les=113/114 n=337 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.693+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.693+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:8fcfcc34:::obj221:head, version: 111'221, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8fcfcc34:::obj221:head@111'221, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000064 2026-02-11T23:31:27.693+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.693+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.693+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8f7cdb2a:::obj538:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.693+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8f7cdb2a:::obj538:head) v3 prio 15 cost 0 latency 0.000084 pg_backfill(progress 1.0 e 116/116 lb 1:8f7cdb2a:::obj538:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8f4c4fa9:::obj590:head local-lis/les=113/114 n=337 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.693+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.693+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8f4c4fa9:::obj590:head local-lis/les=113/114 n=337 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:8f7cdb2a:::obj538:head) v3 2026-02-11T23:31:27.693+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8f4c4fa9:::obj590:head local-lis/les=113/114 n=337 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:8f7cdb2a:::obj538:head) 2026-02-11T23:31:27.693+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.693+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8f7cdb2a:::obj538:head) v3 finish latency 0.000011 2026-02-11T23:31:27.693+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8f7cdb2a:::obj538:head local-lis/les=113/114 n=338 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.693+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.693+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.693+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8f7cdb2a:::obj538:head local-lis/les=113/114 n=338 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.693+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.693+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.693+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.693+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6015b00 con 0x4acb000 2026-02-11T23:31:27.693+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:8fcfcc34:::obj221:head)]) -- 0x614db00 con 0x4acb000 2026-02-11T23:31:27.693+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.693+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 713 ==== MOSDPGPush(1.0 116/113 [PushOp(1:902252c4:::obj397:head, version: 111'397, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:902252c4:::obj397:head@111'397, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56471e0 con 0x4acb000 2026-02-11T23:31:27.693+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:902252c4:::obj397:head, version: 111'397, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:902252c4:::obj397:head@111'397, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 116 2026-02-11T23:31:27.693+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:902252c4:::obj397:head, version: 111'397, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:902252c4:::obj397:head@111'397, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.693+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.693+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 714 ==== pg_backfill(progress 1.0 e 116/116 lb 1:8fcfcc34:::obj221:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177080 con 0x4acb000 2026-02-11T23:31:27.693+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:8fcfcc34:::obj221:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.693+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:902252c4:::obj397:head, version: 111'397, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:902252c4:::obj397:head@111'397, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.693+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8fcfcc34:::obj221:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.693+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.693+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:902252c4:::obj397:head, version: 111'397, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:902252c4:::obj397:head@111'397, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.693+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.693+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:902252c4:::obj397:head, version: 111'397, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:902252c4:::obj397:head@111'397, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000042 MOSDPGPush(1.0 116/113 [PushOp(1:902252c4:::obj397:head, version: 111'397, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:902252c4:::obj397:head@111'397, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8f7cdb2a:::obj538:head local-lis/les=113/114 n=338 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.693+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8fcfcc34:::obj221:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.693+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.693+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8f7cdb2a:::obj538:head local-lis/les=113/114 n=338 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:902252c4:::obj397:head, version: 111'397, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:902252c4:::obj397:head@111'397, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.693+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8f7cdb2a:::obj538:head local-lis/les=113/114 n=338 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:902252c4:::obj397:head@111'397, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.693+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8f7cdb2a:::obj538:head local-lis/les=113/114 n=338 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:902252c4:::obj397:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.693+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8f7cdb2a:::obj538:head local-lis/les=113/114 n=338 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:902252c4:::obj397:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.693+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8f7cdb2a:::obj538:head local-lis/les=113/114 n=338 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:902252c4:::obj397:head 2026-02-11T23:31:27.693+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8f7cdb2a:::obj538:head local-lis/les=113/114 n=338 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:902252c4:::obj397:head v 111'397 2026-02-11T23:31:27.693+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8f7cdb2a:::obj538:head local-lis/les=113/114 n=338 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.693+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.693+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:902252c4:::obj397:head, version: 111'397, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:902252c4:::obj397:head@111'397, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:27.693+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.693+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.693+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:8fcfcc34:::obj221:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.693+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8fcfcc34:::obj221:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 1.0 e 116/116 lb 1:8fcfcc34:::obj221:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8f7cdb2a:::obj538:head local-lis/les=113/114 n=338 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.693+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.693+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8f7cdb2a:::obj538:head local-lis/les=113/114 n=338 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:8fcfcc34:::obj221:head) v3 2026-02-11T23:31:27.693+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8f7cdb2a:::obj538:head local-lis/les=113/114 n=338 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:8fcfcc34:::obj221:head) 2026-02-11T23:31:27.693+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.693+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:8fcfcc34:::obj221:head) v3 finish latency 0.000011 2026-02-11T23:31:27.693+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8fcfcc34:::obj221:head local-lis/les=113/114 n=339 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.693+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.693+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.693+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8fcfcc34:::obj221:head local-lis/les=113/114 n=339 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.693+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.693+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.693+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.693+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6170600 con 0x4acb000 2026-02-11T23:31:27.693+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:902252c4:::obj397:head)]) -- 0x5fce780 con 0x4acb000 2026-02-11T23:31:27.693+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.694+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 715 ==== MOSDPGPush(1.0 116/113 [PushOp(1:90d5bfbc:::obj593:head, version: 111'593, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:90d5bfbc:::obj593:head@111'593, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d8340 con 0x4acb000 2026-02-11T23:31:27.694+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:90d5bfbc:::obj593:head, version: 111'593, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:90d5bfbc:::obj593:head@111'593, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 116 2026-02-11T23:31:27.694+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:90d5bfbc:::obj593:head, version: 111'593, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:90d5bfbc:::obj593:head@111'593, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.694+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.694+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 716 ==== pg_backfill(progress 1.0 e 116/116 lb 1:902252c4:::obj397:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177600 con 0x4acb000 2026-02-11T23:31:27.694+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:902252c4:::obj397:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.694+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:90d5bfbc:::obj593:head, version: 111'593, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:90d5bfbc:::obj593:head@111'593, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.694+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:902252c4:::obj397:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.694+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.694+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:902252c4:::obj397:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.694+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.694+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:90d5bfbc:::obj593:head, version: 111'593, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:90d5bfbc:::obj593:head@111'593, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.694+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:90d5bfbc:::obj593:head, version: 111'593, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:90d5bfbc:::obj593:head@111'593, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000047 MOSDPGPush(1.0 116/113 [PushOp(1:90d5bfbc:::obj593:head, version: 111'593, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:90d5bfbc:::obj593:head@111'593, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8fcfcc34:::obj221:head local-lis/les=113/114 n=339 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.694+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.694+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8fcfcc34:::obj221:head local-lis/les=113/114 n=339 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:90d5bfbc:::obj593:head, version: 111'593, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:90d5bfbc:::obj593:head@111'593, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.694+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8fcfcc34:::obj221:head local-lis/les=113/114 n=339 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:90d5bfbc:::obj593:head@111'593, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.694+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8fcfcc34:::obj221:head local-lis/les=113/114 n=339 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:90d5bfbc:::obj593:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.694+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8fcfcc34:::obj221:head local-lis/les=113/114 n=339 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:90d5bfbc:::obj593:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.694+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8fcfcc34:::obj221:head local-lis/les=113/114 n=339 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:90d5bfbc:::obj593:head 2026-02-11T23:31:27.694+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8fcfcc34:::obj221:head local-lis/les=113/114 n=339 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:90d5bfbc:::obj593:head v 111'593 2026-02-11T23:31:27.694+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8fcfcc34:::obj221:head local-lis/les=113/114 n=339 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.694+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.694+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:90d5bfbc:::obj593:head, version: 111'593, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:90d5bfbc:::obj593:head@111'593, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:27.694+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.694+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.694+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:902252c4:::obj397:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.694+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:902252c4:::obj397:head) v3 prio 15 cost 0 latency 0.000085 pg_backfill(progress 1.0 e 116/116 lb 1:902252c4:::obj397:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:8fcfcc34:::obj221:head local-lis/les=113/114 n=339 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.694+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.694+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8fcfcc34:::obj221:head local-lis/les=113/114 n=339 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:902252c4:::obj397:head) v3 2026-02-11T23:31:27.694+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:8fcfcc34:::obj221:head local-lis/les=113/114 n=339 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:902252c4:::obj397:head) 2026-02-11T23:31:27.694+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.694+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:902252c4:::obj397:head) v3 finish latency 0.000011 2026-02-11T23:31:27.694+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:902252c4:::obj397:head local-lis/les=113/114 n=340 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.694+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.694+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.694+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:902252c4:::obj397:head local-lis/les=113/114 n=340 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.694+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.694+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.694+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.694+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6004900 con 0x4acb000 2026-02-11T23:31:27.694+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:90d5bfbc:::obj593:head)]) -- 0x614de00 con 0x4acb000 2026-02-11T23:31:27.694+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.694+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 717 ==== MOSDPGPush(1.0 116/113 [PushOp(1:917e3523:::obj419:head, version: 111'419, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:917e3523:::obj419:head@111'419, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d9860 con 0x4acb000 2026-02-11T23:31:27.694+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:917e3523:::obj419:head, version: 111'419, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:917e3523:::obj419:head@111'419, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 116 2026-02-11T23:31:27.694+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:917e3523:::obj419:head, version: 111'419, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:917e3523:::obj419:head@111'419, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.694+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.694+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 718 ==== pg_backfill(progress 1.0 e 116/116 lb 1:90d5bfbc:::obj593:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177b80 con 0x4acb000 2026-02-11T23:31:27.694+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:90d5bfbc:::obj593:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.694+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:917e3523:::obj419:head, version: 111'419, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:917e3523:::obj419:head@111'419, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.694+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:90d5bfbc:::obj593:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.694+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.694+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:917e3523:::obj419:head, version: 111'419, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:917e3523:::obj419:head@111'419, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.694+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:917e3523:::obj419:head, version: 111'419, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:917e3523:::obj419:head@111'419, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000044 MOSDPGPush(1.0 116/113 [PushOp(1:917e3523:::obj419:head, version: 111'419, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:917e3523:::obj419:head@111'419, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:902252c4:::obj397:head local-lis/les=113/114 n=340 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.694+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.694+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.694+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:90d5bfbc:::obj593:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.694+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:902252c4:::obj397:head local-lis/les=113/114 n=340 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:917e3523:::obj419:head, version: 111'419, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:917e3523:::obj419:head@111'419, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.694+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:902252c4:::obj397:head local-lis/les=113/114 n=340 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:917e3523:::obj419:head@111'419, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.694+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:902252c4:::obj397:head local-lis/les=113/114 n=340 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:917e3523:::obj419:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.694+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:902252c4:::obj397:head local-lis/les=113/114 n=340 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:917e3523:::obj419:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.694+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:902252c4:::obj397:head local-lis/les=113/114 n=340 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:917e3523:::obj419:head 2026-02-11T23:31:27.694+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:902252c4:::obj397:head local-lis/les=113/114 n=340 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:917e3523:::obj419:head v 111'419 2026-02-11T23:31:27.694+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:902252c4:::obj397:head local-lis/les=113/114 n=340 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.694+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.694+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:917e3523:::obj419:head, version: 111'419, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:917e3523:::obj419:head@111'419, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.694+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.694+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.694+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:90d5bfbc:::obj593:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.694+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:90d5bfbc:::obj593:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 1.0 e 116/116 lb 1:90d5bfbc:::obj593:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:902252c4:::obj397:head local-lis/les=113/114 n=340 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.694+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.694+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:902252c4:::obj397:head local-lis/les=113/114 n=340 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:90d5bfbc:::obj593:head) v3 2026-02-11T23:31:27.694+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:902252c4:::obj397:head local-lis/les=113/114 n=340 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:90d5bfbc:::obj593:head) 2026-02-11T23:31:27.694+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.695+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:90d5bfbc:::obj593:head) v3 finish latency 0.000011 2026-02-11T23:31:27.695+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:90d5bfbc:::obj593:head local-lis/les=113/114 n=341 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.695+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.695+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.695+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:90d5bfbc:::obj593:head local-lis/les=113/114 n=341 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.695+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.695+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.695+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.695+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6170300 con 0x4acb000 2026-02-11T23:31:27.695+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:917e3523:::obj419:head)]) -- 0x6014480 con 0x4acb000 2026-02-11T23:31:27.695+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.695+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 719 ==== MOSDPGPush(1.0 116/113 [PushOp(1:91fc975b:::obj401:head, version: 111'401, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:91fc975b:::obj401:head@111'401, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d89c0 con 0x4acb000 2026-02-11T23:31:27.695+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:91fc975b:::obj401:head, version: 111'401, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:91fc975b:::obj401:head@111'401, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.695+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:91fc975b:::obj401:head, version: 111'401, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:91fc975b:::obj401:head@111'401, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.695+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.695+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 720 ==== pg_backfill(progress 1.0 e 116/116 lb 1:917e3523:::obj419:head) ==== 1062+0+0 (unknown 0 0 0) 0x6178100 con 0x4acb000 2026-02-11T23:31:27.695+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:917e3523:::obj419:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.695+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:91fc975b:::obj401:head, version: 111'401, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:91fc975b:::obj401:head@111'401, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.695+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:917e3523:::obj419:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.695+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.695+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:91fc975b:::obj401:head, version: 111'401, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:91fc975b:::obj401:head@111'401, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.695+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:91fc975b:::obj401:head, version: 111'401, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:91fc975b:::obj401:head@111'401, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 116/113 [PushOp(1:91fc975b:::obj401:head, version: 111'401, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:91fc975b:::obj401:head@111'401, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:90d5bfbc:::obj593:head local-lis/les=113/114 n=341 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.695+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.695+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.695+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:90d5bfbc:::obj593:head local-lis/les=113/114 n=341 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:91fc975b:::obj401:head, version: 111'401, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:91fc975b:::obj401:head@111'401, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.695+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:917e3523:::obj419:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.695+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:90d5bfbc:::obj593:head local-lis/les=113/114 n=341 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:91fc975b:::obj401:head@111'401, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.695+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:90d5bfbc:::obj593:head local-lis/les=113/114 n=341 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:91fc975b:::obj401:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.695+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:90d5bfbc:::obj593:head local-lis/les=113/114 n=341 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:91fc975b:::obj401:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.695+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:90d5bfbc:::obj593:head local-lis/les=113/114 n=341 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:91fc975b:::obj401:head 2026-02-11T23:31:27.695+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:90d5bfbc:::obj593:head local-lis/les=113/114 n=341 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:91fc975b:::obj401:head v 111'401 2026-02-11T23:31:27.695+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:90d5bfbc:::obj593:head local-lis/les=113/114 n=341 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.695+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.695+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:91fc975b:::obj401:head, version: 111'401, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:91fc975b:::obj401:head@111'401, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.695+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.695+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.695+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:917e3523:::obj419:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.695+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:917e3523:::obj419:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 1.0 e 116/116 lb 1:917e3523:::obj419:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:90d5bfbc:::obj593:head local-lis/les=113/114 n=341 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.695+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.695+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:90d5bfbc:::obj593:head local-lis/les=113/114 n=341 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:917e3523:::obj419:head) v3 2026-02-11T23:31:27.695+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:90d5bfbc:::obj593:head local-lis/les=113/114 n=341 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:917e3523:::obj419:head) 2026-02-11T23:31:27.695+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.695+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:917e3523:::obj419:head) v3 finish latency 0.000010 2026-02-11T23:31:27.695+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:917e3523:::obj419:head local-lis/les=113/114 n=342 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.695+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.695+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.695+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:917e3523:::obj419:head local-lis/les=113/114 n=342 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.695+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.695+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.695+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.695+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x614c780 con 0x4acb000 2026-02-11T23:31:27.695+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:91fc975b:::obj401:head)]) -- 0x615c180 con 0x4acb000 2026-02-11T23:31:27.695+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.696+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 721 ==== MOSDPGPush(1.0 116/113 [PushOp(1:92b89151:::obj431:head, version: 111'431, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:92b89151:::obj431:head@111'431, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559d040 con 0x4acb000 2026-02-11T23:31:27.696+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:92b89151:::obj431:head, version: 111'431, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:92b89151:::obj431:head@111'431, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 116 2026-02-11T23:31:27.696+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:92b89151:::obj431:head, version: 111'431, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:92b89151:::obj431:head@111'431, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.696+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.696+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 722 ==== pg_backfill(progress 1.0 e 116/116 lb 1:91fc975b:::obj401:head) ==== 1062+0+0 (unknown 0 0 0) 0x6178680 con 0x4acb000 2026-02-11T23:31:27.696+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:92b89151:::obj431:head, version: 111'431, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:92b89151:::obj431:head@111'431, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.696+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:91fc975b:::obj401:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.696+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:91fc975b:::obj401:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.696+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.696+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:92b89151:::obj431:head, version: 111'431, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:92b89151:::obj431:head@111'431, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.696+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.696+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:92b89151:::obj431:head, version: 111'431, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:92b89151:::obj431:head@111'431, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000040 MOSDPGPush(1.0 116/113 [PushOp(1:92b89151:::obj431:head, version: 111'431, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:92b89151:::obj431:head@111'431, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:917e3523:::obj419:head local-lis/les=113/114 n=342 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.696+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.696+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:917e3523:::obj419:head local-lis/les=113/114 n=342 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:92b89151:::obj431:head, version: 111'431, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:92b89151:::obj431:head@111'431, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.696+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:917e3523:::obj419:head local-lis/les=113/114 n=342 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:92b89151:::obj431:head@111'431, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.696+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.696+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:91fc975b:::obj401:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.696+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:917e3523:::obj419:head local-lis/les=113/114 n=342 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:92b89151:::obj431:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.696+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:917e3523:::obj419:head local-lis/les=113/114 n=342 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:92b89151:::obj431:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.696+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:917e3523:::obj419:head local-lis/les=113/114 n=342 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:92b89151:::obj431:head 2026-02-11T23:31:27.696+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:917e3523:::obj419:head local-lis/les=113/114 n=342 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:92b89151:::obj431:head v 111'431 2026-02-11T23:31:27.696+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:917e3523:::obj419:head local-lis/les=113/114 n=342 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.696+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.696+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:92b89151:::obj431:head, version: 111'431, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:92b89151:::obj431:head@111'431, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:27.696+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:917e3523:::obj419:head local-lis/les=113/114 n=342 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.696+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.696+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.696+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:91fc975b:::obj401:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.696+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:91fc975b:::obj401:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 1.0 e 116/116 lb 1:91fc975b:::obj401:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:917e3523:::obj419:head local-lis/les=113/114 n=342 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.696+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.696+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:917e3523:::obj419:head local-lis/les=113/114 n=342 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:91fc975b:::obj401:head) v3 2026-02-11T23:31:27.696+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:917e3523:::obj419:head local-lis/les=113/114 n=342 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:91fc975b:::obj401:head) 2026-02-11T23:31:27.696+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.696+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:91fc975b:::obj401:head) v3 finish latency 0.000010 2026-02-11T23:31:27.696+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.696+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.696+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:91fc975b:::obj401:head local-lis/les=113/114 n=343 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.696+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.696+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.696+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.696+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x613fc80 con 0x4acb000 2026-02-11T23:31:27.696+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:92b89151:::obj431:head)]) -- 0x6170000 con 0x4acb000 2026-02-11T23:31:27.696+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.696+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 723 ==== MOSDPGPush(1.0 116/113 [PushOp(1:9315b0b3:::obj555:head, version: 111'555, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9315b0b3:::obj555:head@111'555, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559dba0 con 0x4acb000 2026-02-11T23:31:27.696+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:9315b0b3:::obj555:head, version: 111'555, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9315b0b3:::obj555:head@111'555, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 116 2026-02-11T23:31:27.696+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:9315b0b3:::obj555:head, version: 111'555, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9315b0b3:::obj555:head@111'555, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.696+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.696+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 724 ==== pg_backfill(progress 1.0 e 116/116 lb 1:92b89151:::obj431:head) ==== 1062+0+0 (unknown 0 0 0) 0x6178c00 con 0x4acb000 2026-02-11T23:31:27.696+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:92b89151:::obj431:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.696+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:9315b0b3:::obj555:head, version: 111'555, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9315b0b3:::obj555:head@111'555, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.696+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:92b89151:::obj431:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.696+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.696+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:9315b0b3:::obj555:head, version: 111'555, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9315b0b3:::obj555:head@111'555, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.696+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:9315b0b3:::obj555:head, version: 111'555, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9315b0b3:::obj555:head@111'555, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 116/113 [PushOp(1:9315b0b3:::obj555:head, version: 111'555, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9315b0b3:::obj555:head@111'555, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:91fc975b:::obj401:head local-lis/les=113/114 n=343 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.696+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.696+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.696+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:91fc975b:::obj401:head local-lis/les=113/114 n=343 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:9315b0b3:::obj555:head, version: 111'555, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9315b0b3:::obj555:head@111'555, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.696+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:92b89151:::obj431:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.696+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:91fc975b:::obj401:head local-lis/les=113/114 n=343 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:9315b0b3:::obj555:head@111'555, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.696+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:91fc975b:::obj401:head local-lis/les=113/114 n=343 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:9315b0b3:::obj555:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.696+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:91fc975b:::obj401:head local-lis/les=113/114 n=343 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:9315b0b3:::obj555:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.696+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:91fc975b:::obj401:head local-lis/les=113/114 n=343 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:9315b0b3:::obj555:head 2026-02-11T23:31:27.696+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:91fc975b:::obj401:head local-lis/les=113/114 n=343 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:9315b0b3:::obj555:head v 111'555 2026-02-11T23:31:27.696+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:91fc975b:::obj401:head local-lis/les=113/114 n=343 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.696+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.696+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:9315b0b3:::obj555:head, version: 111'555, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9315b0b3:::obj555:head@111'555, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.696+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.696+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.696+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:92b89151:::obj431:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.696+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:92b89151:::obj431:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 1.0 e 116/116 lb 1:92b89151:::obj431:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:91fc975b:::obj401:head local-lis/les=113/114 n=343 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.696+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.696+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:91fc975b:::obj401:head local-lis/les=113/114 n=343 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:92b89151:::obj431:head) v3 2026-02-11T23:31:27.696+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:91fc975b:::obj401:head local-lis/les=113/114 n=343 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:92b89151:::obj431:head) 2026-02-11T23:31:27.696+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.696+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:92b89151:::obj431:head) v3 finish latency 0.000011 2026-02-11T23:31:27.696+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:92b89151:::obj431:head local-lis/les=113/114 n=344 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.696+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.696+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.696+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:92b89151:::obj431:head local-lis/les=113/114 n=344 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.696+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.696+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.696+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.696+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x614ca80 con 0x4acb000 2026-02-11T23:31:27.696+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:9315b0b3:::obj555:head)]) -- 0x5fed980 con 0x4acb000 2026-02-11T23:31:27.696+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.697+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 725 ==== MOSDPGPush(1.0 116/113 [PushOp(1:9397791a:::obj547:head, version: 111'547, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9397791a:::obj547:head@111'547, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c680 con 0x4acb000 2026-02-11T23:31:27.697+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:9397791a:::obj547:head, version: 111'547, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9397791a:::obj547:head@111'547, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 116 2026-02-11T23:31:27.697+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:9397791a:::obj547:head, version: 111'547, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9397791a:::obj547:head@111'547, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.697+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.697+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 726 ==== pg_backfill(progress 1.0 e 116/116 lb 1:9315b0b3:::obj555:head) ==== 1062+0+0 (unknown 0 0 0) 0x6179180 con 0x4acb000 2026-02-11T23:31:27.697+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:9315b0b3:::obj555:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.697+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:9397791a:::obj547:head, version: 111'547, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9397791a:::obj547:head@111'547, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.697+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:9315b0b3:::obj555:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.697+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.697+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:9397791a:::obj547:head, version: 111'547, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9397791a:::obj547:head@111'547, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.697+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:9397791a:::obj547:head, version: 111'547, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9397791a:::obj547:head@111'547, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 116/113 [PushOp(1:9397791a:::obj547:head, version: 111'547, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9397791a:::obj547:head@111'547, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:92b89151:::obj431:head local-lis/les=113/114 n=344 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.697+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.697+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:92b89151:::obj431:head local-lis/les=113/114 n=344 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:9397791a:::obj547:head, version: 111'547, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9397791a:::obj547:head@111'547, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.697+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.697+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:9315b0b3:::obj555:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.697+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:92b89151:::obj431:head local-lis/les=113/114 n=344 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:9397791a:::obj547:head@111'547, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.697+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:92b89151:::obj431:head local-lis/les=113/114 n=344 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:9397791a:::obj547:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.697+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:92b89151:::obj431:head local-lis/les=113/114 n=344 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:9397791a:::obj547:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.697+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:92b89151:::obj431:head local-lis/les=113/114 n=344 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:9397791a:::obj547:head 2026-02-11T23:31:27.697+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:92b89151:::obj431:head local-lis/les=113/114 n=344 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:9397791a:::obj547:head v 111'547 2026-02-11T23:31:27.697+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:92b89151:::obj431:head local-lis/les=113/114 n=344 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.697+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.697+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:9397791a:::obj547:head, version: 111'547, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9397791a:::obj547:head@111'547, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:27.697+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:92b89151:::obj431:head local-lis/les=113/114 n=344 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.697+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.697+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.697+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:9315b0b3:::obj555:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.697+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:9315b0b3:::obj555:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 1.0 e 116/116 lb 1:9315b0b3:::obj555:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:92b89151:::obj431:head local-lis/les=113/114 n=344 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.697+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.697+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:92b89151:::obj431:head local-lis/les=113/114 n=344 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:9315b0b3:::obj555:head) v3 2026-02-11T23:31:27.697+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:92b89151:::obj431:head local-lis/les=113/114 n=344 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:9315b0b3:::obj555:head) 2026-02-11T23:31:27.697+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.697+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:9315b0b3:::obj555:head) v3 finish latency 0.000010 2026-02-11T23:31:27.697+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.697+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.697+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9315b0b3:::obj555:head local-lis/les=113/114 n=345 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.697+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.697+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.697+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.697+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6004c00 con 0x4acb000 2026-02-11T23:31:27.697+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:9397791a:::obj547:head)]) -- 0x6004f00 con 0x4acb000 2026-02-11T23:31:27.697+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.697+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 727 ==== MOSDPGPush(1.0 116/113 [PushOp(1:9455f78c:::obj228:head, version: 111'228, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9455f78c:::obj228:head@111'228, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f16c0 con 0x4acb000 2026-02-11T23:31:27.697+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:9455f78c:::obj228:head, version: 111'228, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9455f78c:::obj228:head@111'228, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 116 2026-02-11T23:31:27.697+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:9455f78c:::obj228:head, version: 111'228, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9455f78c:::obj228:head@111'228, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.697+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.697+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 728 ==== pg_backfill(progress 1.0 e 116/116 lb 1:9397791a:::obj547:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2d080 con 0x4acb000 2026-02-11T23:31:27.697+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:9455f78c:::obj228:head, version: 111'228, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9455f78c:::obj228:head@111'228, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.697+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:9397791a:::obj547:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.697+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:9397791a:::obj547:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.697+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.697+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:9455f78c:::obj228:head, version: 111'228, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9455f78c:::obj228:head@111'228, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.697+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:9455f78c:::obj228:head, version: 111'228, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9455f78c:::obj228:head@111'228, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(1.0 116/113 [PushOp(1:9455f78c:::obj228:head, version: 111'228, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9455f78c:::obj228:head@111'228, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:9315b0b3:::obj555:head local-lis/les=113/114 n=345 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.697+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.697+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.697+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9315b0b3:::obj555:head local-lis/les=113/114 n=345 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:9455f78c:::obj228:head, version: 111'228, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9455f78c:::obj228:head@111'228, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.697+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:9397791a:::obj547:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.697+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9315b0b3:::obj555:head local-lis/les=113/114 n=345 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:9455f78c:::obj228:head@111'228, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.697+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9315b0b3:::obj555:head local-lis/les=113/114 n=345 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:9455f78c:::obj228:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.697+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9315b0b3:::obj555:head local-lis/les=113/114 n=345 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:9455f78c:::obj228:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.697+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9315b0b3:::obj555:head local-lis/les=113/114 n=345 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:9455f78c:::obj228:head 2026-02-11T23:31:27.697+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9315b0b3:::obj555:head local-lis/les=113/114 n=345 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:9455f78c:::obj228:head v 111'228 2026-02-11T23:31:27.697+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9315b0b3:::obj555:head local-lis/les=113/114 n=345 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.697+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.697+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:9455f78c:::obj228:head, version: 111'228, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9455f78c:::obj228:head@111'228, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.697+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.697+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.697+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:9397791a:::obj547:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.697+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:9397791a:::obj547:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 1.0 e 116/116 lb 1:9397791a:::obj547:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:9315b0b3:::obj555:head local-lis/les=113/114 n=345 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.697+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.697+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9315b0b3:::obj555:head local-lis/les=113/114 n=345 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:9397791a:::obj547:head) v3 2026-02-11T23:31:27.697+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9315b0b3:::obj555:head local-lis/les=113/114 n=345 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:9397791a:::obj547:head) 2026-02-11T23:31:27.697+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.697+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:9397791a:::obj547:head) v3 finish latency 0.000011 2026-02-11T23:31:27.697+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9397791a:::obj547:head local-lis/les=113/114 n=346 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.697+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.697+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.697+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9397791a:::obj547:head local-lis/les=113/114 n=346 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.697+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.697+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.697+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.697+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x614d080 con 0x4acb000 2026-02-11T23:31:27.697+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:9455f78c:::obj228:head)]) -- 0x612f680 con 0x4acb000 2026-02-11T23:31:27.697+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.698+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 729 ==== MOSDPGPush(1.0 116/113 [PushOp(1:945b6e2e:::obj114:head, version: 111'114, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:945b6e2e:::obj114:head@111'114, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f0b60 con 0x4acb000 2026-02-11T23:31:27.698+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:945b6e2e:::obj114:head, version: 111'114, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:945b6e2e:::obj114:head@111'114, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 116 2026-02-11T23:31:27.698+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:945b6e2e:::obj114:head, version: 111'114, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:945b6e2e:::obj114:head@111'114, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.698+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.698+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 730 ==== pg_backfill(progress 1.0 e 116/116 lb 1:9455f78c:::obj228:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e680 con 0x4acb000 2026-02-11T23:31:27.698+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:9455f78c:::obj228:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.698+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:945b6e2e:::obj114:head, version: 111'114, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:945b6e2e:::obj114:head@111'114, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.698+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:9455f78c:::obj228:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.698+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.698+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:9455f78c:::obj228:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.698+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.698+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:945b6e2e:::obj114:head, version: 111'114, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:945b6e2e:::obj114:head@111'114, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.698+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:945b6e2e:::obj114:head, version: 111'114, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:945b6e2e:::obj114:head@111'114, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000046 MOSDPGPush(1.0 116/113 [PushOp(1:945b6e2e:::obj114:head, version: 111'114, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:945b6e2e:::obj114:head@111'114, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:9397791a:::obj547:head local-lis/les=113/114 n=346 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.698+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.698+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9397791a:::obj547:head local-lis/les=113/114 n=346 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:945b6e2e:::obj114:head, version: 111'114, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:945b6e2e:::obj114:head@111'114, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.698+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9397791a:::obj547:head local-lis/les=113/114 n=346 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:945b6e2e:::obj114:head@111'114, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.698+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9397791a:::obj547:head local-lis/les=113/114 n=346 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:945b6e2e:::obj114:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.698+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9397791a:::obj547:head local-lis/les=113/114 n=346 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:945b6e2e:::obj114:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.698+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9397791a:::obj547:head local-lis/les=113/114 n=346 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:945b6e2e:::obj114:head 2026-02-11T23:31:27.698+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9397791a:::obj547:head local-lis/les=113/114 n=346 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:945b6e2e:::obj114:head v 111'114 2026-02-11T23:31:27.698+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9397791a:::obj547:head local-lis/les=113/114 n=346 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.698+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.698+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:945b6e2e:::obj114:head, version: 111'114, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:945b6e2e:::obj114:head@111'114, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.698+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.698+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.698+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:9455f78c:::obj228:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.698+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:9455f78c:::obj228:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 1.0 e 116/116 lb 1:9455f78c:::obj228:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:9397791a:::obj547:head local-lis/les=113/114 n=346 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.698+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.698+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9397791a:::obj547:head local-lis/les=113/114 n=346 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:9455f78c:::obj228:head) v3 2026-02-11T23:31:27.698+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9397791a:::obj547:head local-lis/les=113/114 n=346 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:9455f78c:::obj228:head) 2026-02-11T23:31:27.698+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.698+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:9455f78c:::obj228:head) v3 finish latency 0.000010 2026-02-11T23:31:27.698+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9455f78c:::obj228:head local-lis/les=113/114 n=347 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.698+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.698+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.698+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9455f78c:::obj228:head local-lis/les=113/114 n=347 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.698+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.698+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.698+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.698+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x613f980 con 0x4acb000 2026-02-11T23:31:27.698+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:945b6e2e:::obj114:head)]) -- 0x6005b00 con 0x4acb000 2026-02-11T23:31:27.698+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.698+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 731 ==== MOSDPGPush(1.0 116/113 [PushOp(1:9477fafa:::obj499:head, version: 111'499, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9477fafa:::obj499:head@111'499, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f1d40 con 0x4acb000 2026-02-11T23:31:27.698+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:9477fafa:::obj499:head, version: 111'499, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9477fafa:::obj499:head@111'499, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 116 2026-02-11T23:31:27.698+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:9477fafa:::obj499:head, version: 111'499, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9477fafa:::obj499:head@111'499, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.698+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.698+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 732 ==== pg_backfill(progress 1.0 e 116/116 lb 1:945b6e2e:::obj114:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e100 con 0x4acb000 2026-02-11T23:31:27.698+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:945b6e2e:::obj114:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.698+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:945b6e2e:::obj114:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.698+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:9477fafa:::obj499:head, version: 111'499, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9477fafa:::obj499:head@111'499, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.698+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.698+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:9477fafa:::obj499:head, version: 111'499, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9477fafa:::obj499:head@111'499, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.698+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:9477fafa:::obj499:head, version: 111'499, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9477fafa:::obj499:head@111'499, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(1.0 116/113 [PushOp(1:9477fafa:::obj499:head, version: 111'499, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9477fafa:::obj499:head@111'499, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:9455f78c:::obj228:head local-lis/les=113/114 n=347 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.698+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.698+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9455f78c:::obj228:head local-lis/les=113/114 n=347 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:9477fafa:::obj499:head, version: 111'499, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9477fafa:::obj499:head@111'499, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.698+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.698+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:945b6e2e:::obj114:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.698+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9455f78c:::obj228:head local-lis/les=113/114 n=347 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:9477fafa:::obj499:head@111'499, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.698+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9455f78c:::obj228:head local-lis/les=113/114 n=347 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:9477fafa:::obj499:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.698+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9455f78c:::obj228:head local-lis/les=113/114 n=347 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:9477fafa:::obj499:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.698+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9455f78c:::obj228:head local-lis/les=113/114 n=347 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:9477fafa:::obj499:head 2026-02-11T23:31:27.698+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9455f78c:::obj228:head local-lis/les=113/114 n=347 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:9477fafa:::obj499:head v 111'499 2026-02-11T23:31:27.698+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9455f78c:::obj228:head local-lis/les=113/114 n=347 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.698+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.698+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:9477fafa:::obj499:head, version: 111'499, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9477fafa:::obj499:head@111'499, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.698+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9455f78c:::obj228:head local-lis/les=113/114 n=347 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.698+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.698+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.698+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:945b6e2e:::obj114:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.698+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:945b6e2e:::obj114:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 1.0 e 116/116 lb 1:945b6e2e:::obj114:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:9455f78c:::obj228:head local-lis/les=113/114 n=347 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.698+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.698+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9455f78c:::obj228:head local-lis/les=113/114 n=347 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:945b6e2e:::obj114:head) v3 2026-02-11T23:31:27.698+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9455f78c:::obj228:head local-lis/les=113/114 n=347 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:945b6e2e:::obj114:head) 2026-02-11T23:31:27.698+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.698+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:945b6e2e:::obj114:head) v3 finish latency 0.000011 2026-02-11T23:31:27.698+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.699+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.699+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:945b6e2e:::obj114:head local-lis/les=113/114 n=348 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.699+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.699+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.699+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.699+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6014f00 con 0x4acb000 2026-02-11T23:31:27.699+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:9477fafa:::obj499:head)]) -- 0x6014c00 con 0x4acb000 2026-02-11T23:31:27.699+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.699+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 733 ==== MOSDPGPush(1.0 116/113 [PushOp(1:95a9a074:::obj509:head, version: 111'509, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:95a9a074:::obj509:head@111'509, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e261a0 con 0x4acb000 2026-02-11T23:31:27.699+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:95a9a074:::obj509:head, version: 111'509, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:95a9a074:::obj509:head@111'509, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 116 2026-02-11T23:31:27.699+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:95a9a074:::obj509:head, version: 111'509, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:95a9a074:::obj509:head@111'509, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.699+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.699+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 734 ==== pg_backfill(progress 1.0 e 116/116 lb 1:9477fafa:::obj499:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2c000 con 0x4acb000 2026-02-11T23:31:27.699+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:95a9a074:::obj509:head, version: 111'509, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:95a9a074:::obj509:head@111'509, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.699+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:9477fafa:::obj499:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.699+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:9477fafa:::obj499:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.699+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.699+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:95a9a074:::obj509:head, version: 111'509, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:95a9a074:::obj509:head@111'509, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.699+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:95a9a074:::obj509:head, version: 111'509, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:95a9a074:::obj509:head@111'509, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 116/113 [PushOp(1:95a9a074:::obj509:head, version: 111'509, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:95a9a074:::obj509:head@111'509, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:945b6e2e:::obj114:head local-lis/les=113/114 n=348 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.699+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.699+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:945b6e2e:::obj114:head local-lis/les=113/114 n=348 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:95a9a074:::obj509:head, version: 111'509, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:95a9a074:::obj509:head@111'509, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.699+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.699+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:9477fafa:::obj499:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.699+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:945b6e2e:::obj114:head local-lis/les=113/114 n=348 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:95a9a074:::obj509:head@111'509, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.699+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:945b6e2e:::obj114:head local-lis/les=113/114 n=348 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:95a9a074:::obj509:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.699+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:945b6e2e:::obj114:head local-lis/les=113/114 n=348 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:95a9a074:::obj509:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.699+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:945b6e2e:::obj114:head local-lis/les=113/114 n=348 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:95a9a074:::obj509:head 2026-02-11T23:31:27.699+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:945b6e2e:::obj114:head local-lis/les=113/114 n=348 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:95a9a074:::obj509:head v 111'509 2026-02-11T23:31:27.699+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:945b6e2e:::obj114:head local-lis/les=113/114 n=348 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.699+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.699+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:95a9a074:::obj509:head, version: 111'509, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:95a9a074:::obj509:head@111'509, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.699+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.699+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.699+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:9477fafa:::obj499:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.699+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:9477fafa:::obj499:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:9477fafa:::obj499:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:945b6e2e:::obj114:head local-lis/les=113/114 n=348 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.699+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.699+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:945b6e2e:::obj114:head local-lis/les=113/114 n=348 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:9477fafa:::obj499:head) v3 2026-02-11T23:31:27.699+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:945b6e2e:::obj114:head local-lis/les=113/114 n=348 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:9477fafa:::obj499:head) 2026-02-11T23:31:27.699+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.699+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:9477fafa:::obj499:head) v3 finish latency 0.000010 2026-02-11T23:31:27.699+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9477fafa:::obj499:head local-lis/les=113/114 n=349 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.699+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.699+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.699+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9477fafa:::obj499:head local-lis/les=113/114 n=349 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.699+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.699+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.699+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.699+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x613f680 con 0x4acb000 2026-02-11T23:31:27.699+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:95a9a074:::obj509:head)]) -- 0x612f980 con 0x4acb000 2026-02-11T23:31:27.699+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.699+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 735 ==== MOSDPGPush(1.0 116/113 [PushOp(1:9604ab91:::obj494:head, version: 111'494, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9604ab91:::obj494:head@111'494, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559cd00 con 0x4acb000 2026-02-11T23:31:27.699+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:9604ab91:::obj494:head, version: 111'494, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9604ab91:::obj494:head@111'494, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.699+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:9604ab91:::obj494:head, version: 111'494, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9604ab91:::obj494:head@111'494, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.699+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.699+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 736 ==== pg_backfill(progress 1.0 e 116/116 lb 1:95a9a074:::obj509:head) ==== 1062+0+0 (unknown 0 0 0) 0x5982b00 con 0x4acb000 2026-02-11T23:31:27.699+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:9604ab91:::obj494:head, version: 111'494, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9604ab91:::obj494:head@111'494, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.699+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:95a9a074:::obj509:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.699+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:95a9a074:::obj509:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.699+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.699+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:9604ab91:::obj494:head, version: 111'494, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9604ab91:::obj494:head@111'494, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.699+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.699+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:9604ab91:::obj494:head, version: 111'494, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9604ab91:::obj494:head@111'494, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(1.0 116/113 [PushOp(1:9604ab91:::obj494:head, version: 111'494, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9604ab91:::obj494:head@111'494, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:9477fafa:::obj499:head local-lis/les=113/114 n=349 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.699+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.699+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9477fafa:::obj499:head local-lis/les=113/114 n=349 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:9604ab91:::obj494:head, version: 111'494, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9604ab91:::obj494:head@111'494, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.699+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9477fafa:::obj499:head local-lis/les=113/114 n=349 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:9604ab91:::obj494:head@111'494, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.699+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.699+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:95a9a074:::obj509:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.699+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9477fafa:::obj499:head local-lis/les=113/114 n=349 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:9604ab91:::obj494:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.699+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9477fafa:::obj499:head local-lis/les=113/114 n=349 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:9604ab91:::obj494:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.699+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9477fafa:::obj499:head local-lis/les=113/114 n=349 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:9604ab91:::obj494:head 2026-02-11T23:31:27.699+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9477fafa:::obj499:head local-lis/les=113/114 n=349 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:9604ab91:::obj494:head v 111'494 2026-02-11T23:31:27.699+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9477fafa:::obj499:head local-lis/les=113/114 n=349 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.699+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.699+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:9604ab91:::obj494:head, version: 111'494, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9604ab91:::obj494:head@111'494, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.699+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.699+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.699+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:95a9a074:::obj509:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.699+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:95a9a074:::obj509:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 1.0 e 116/116 lb 1:95a9a074:::obj509:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:9477fafa:::obj499:head local-lis/les=113/114 n=349 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.699+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.699+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9477fafa:::obj499:head local-lis/les=113/114 n=349 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:95a9a074:::obj509:head) v3 2026-02-11T23:31:27.699+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9477fafa:::obj499:head local-lis/les=113/114 n=349 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:95a9a074:::obj509:head) 2026-02-11T23:31:27.699+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.699+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:95a9a074:::obj509:head) v3 finish latency 0.000010 2026-02-11T23:31:27.699+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:95a9a074:::obj509:head local-lis/les=113/114 n=350 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.699+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.700+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.700+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:95a9a074:::obj509:head local-lis/les=113/114 n=350 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.700+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.700+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.700+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.700+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6014900 con 0x4acb000 2026-02-11T23:31:27.700+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:9604ab91:::obj494:head)]) -- 0x5fce480 con 0x4acb000 2026-02-11T23:31:27.700+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.700+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 737 ==== MOSDPGPush(1.0 116/113 [PushOp(1:96183f21:::obj585:head, version: 111'585, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:96183f21:::obj585:head@111'585, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559d1e0 con 0x4acb000 2026-02-11T23:31:27.700+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:96183f21:::obj585:head, version: 111'585, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:96183f21:::obj585:head@111'585, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 116 2026-02-11T23:31:27.700+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:96183f21:::obj585:head, version: 111'585, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:96183f21:::obj585:head@111'585, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.700+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.700+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 738 ==== pg_backfill(progress 1.0 e 116/116 lb 1:9604ab91:::obj494:head) ==== 1062+0+0 (unknown 0 0 0) 0x4a2cb00 con 0x4acb000 2026-02-11T23:31:27.700+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:96183f21:::obj585:head, version: 111'585, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:96183f21:::obj585:head@111'585, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.700+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:9604ab91:::obj494:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.700+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:9604ab91:::obj494:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.700+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.700+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:96183f21:::obj585:head, version: 111'585, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:96183f21:::obj585:head@111'585, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.700+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.700+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:96183f21:::obj585:head, version: 111'585, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:96183f21:::obj585:head@111'585, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(1.0 116/113 [PushOp(1:96183f21:::obj585:head, version: 111'585, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:96183f21:::obj585:head@111'585, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:95a9a074:::obj509:head local-lis/les=113/114 n=350 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.700+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.700+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:95a9a074:::obj509:head local-lis/les=113/114 n=350 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:96183f21:::obj585:head, version: 111'585, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:96183f21:::obj585:head@111'585, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.700+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:95a9a074:::obj509:head local-lis/les=113/114 n=350 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:96183f21:::obj585:head@111'585, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.700+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.700+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:9604ab91:::obj494:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.700+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:95a9a074:::obj509:head local-lis/les=113/114 n=350 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:96183f21:::obj585:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.700+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:95a9a074:::obj509:head local-lis/les=113/114 n=350 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:96183f21:::obj585:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.700+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:95a9a074:::obj509:head local-lis/les=113/114 n=350 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:96183f21:::obj585:head 2026-02-11T23:31:27.700+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:95a9a074:::obj509:head local-lis/les=113/114 n=350 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:96183f21:::obj585:head v 111'585 2026-02-11T23:31:27.700+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:95a9a074:::obj509:head local-lis/les=113/114 n=350 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.700+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.700+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:96183f21:::obj585:head, version: 111'585, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:96183f21:::obj585:head@111'585, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000044 2026-02-11T23:31:27.700+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.700+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.700+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:9604ab91:::obj494:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.700+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:9604ab91:::obj494:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 1.0 e 116/116 lb 1:9604ab91:::obj494:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:95a9a074:::obj509:head local-lis/les=113/114 n=350 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.700+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.700+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:95a9a074:::obj509:head local-lis/les=113/114 n=350 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:9604ab91:::obj494:head) v3 2026-02-11T23:31:27.700+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:95a9a074:::obj509:head local-lis/les=113/114 n=350 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:9604ab91:::obj494:head) 2026-02-11T23:31:27.700+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.700+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:9604ab91:::obj494:head) v3 finish latency 0.000009 2026-02-11T23:31:27.700+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9604ab91:::obj494:head local-lis/les=113/114 n=351 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.700+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.700+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.700+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9604ab91:::obj494:head local-lis/les=113/114 n=351 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.700+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.700+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.700+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.700+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6005380 con 0x4acb000 2026-02-11T23:31:27.700+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:96183f21:::obj585:head)]) -- 0x612fc80 con 0x4acb000 2026-02-11T23:31:27.700+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.701+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 739 ==== MOSDPGPush(1.0 116/113 [PushOp(1:96d0cad0:::obj307:head, version: 111'307, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:96d0cad0:::obj307:head@111'307, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c340 con 0x4acb000 2026-02-11T23:31:27.701+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:96d0cad0:::obj307:head, version: 111'307, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:96d0cad0:::obj307:head@111'307, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 116 2026-02-11T23:31:27.701+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:96d0cad0:::obj307:head, version: 111'307, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:96d0cad0:::obj307:head@111'307, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.701+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.701+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 740 ==== pg_backfill(progress 1.0 e 116/116 lb 1:96183f21:::obj585:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99600 con 0x4acb000 2026-02-11T23:31:27.701+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:96183f21:::obj585:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.701+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:96d0cad0:::obj307:head, version: 111'307, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:96d0cad0:::obj307:head@111'307, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.701+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:96183f21:::obj585:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.701+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.701+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:96d0cad0:::obj307:head, version: 111'307, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:96d0cad0:::obj307:head@111'307, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.701+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.701+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:96d0cad0:::obj307:head, version: 111'307, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:96d0cad0:::obj307:head@111'307, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 116/113 [PushOp(1:96d0cad0:::obj307:head, version: 111'307, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:96d0cad0:::obj307:head@111'307, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:9604ab91:::obj494:head local-lis/les=113/114 n=351 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.701+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:96183f21:::obj585:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.701+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.701+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9604ab91:::obj494:head local-lis/les=113/114 n=351 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:96d0cad0:::obj307:head, version: 111'307, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:96d0cad0:::obj307:head@111'307, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.701+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9604ab91:::obj494:head local-lis/les=113/114 n=351 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:96d0cad0:::obj307:head@111'307, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.701+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9604ab91:::obj494:head local-lis/les=113/114 n=351 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:96d0cad0:::obj307:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.701+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9604ab91:::obj494:head local-lis/les=113/114 n=351 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:96d0cad0:::obj307:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.701+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9604ab91:::obj494:head local-lis/les=113/114 n=351 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:96d0cad0:::obj307:head 2026-02-11T23:31:27.701+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9604ab91:::obj494:head local-lis/les=113/114 n=351 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:96d0cad0:::obj307:head v 111'307 2026-02-11T23:31:27.701+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9604ab91:::obj494:head local-lis/les=113/114 n=351 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.701+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.701+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:96d0cad0:::obj307:head, version: 111'307, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:96d0cad0:::obj307:head@111'307, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.701+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.701+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.701+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:96183f21:::obj585:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.701+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:96183f21:::obj585:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:96183f21:::obj585:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:9604ab91:::obj494:head local-lis/les=113/114 n=351 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.701+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.701+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9604ab91:::obj494:head local-lis/les=113/114 n=351 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:96183f21:::obj585:head) v3 2026-02-11T23:31:27.701+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9604ab91:::obj494:head local-lis/les=113/114 n=351 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:96183f21:::obj585:head) 2026-02-11T23:31:27.701+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.701+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:96183f21:::obj585:head) v3 finish latency 0.000009 2026-02-11T23:31:27.701+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:96183f21:::obj585:head local-lis/les=113/114 n=352 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.701+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.701+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.701+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:96183f21:::obj585:head local-lis/les=113/114 n=352 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.701+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.701+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.701+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.701+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x614d680 con 0x4acb000 2026-02-11T23:31:27.701+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:96d0cad0:::obj307:head)]) -- 0x6005800 con 0x4acb000 2026-02-11T23:31:27.701+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.701+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 741 ==== MOSDPGPush(1.0 116/113 [PushOp(1:9720f72d:::obj90:head, version: 111'90, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9720f72d:::obj90:head@111'90, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x559c820 con 0x4acb000 2026-02-11T23:31:27.701+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:9720f72d:::obj90:head, version: 111'90, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9720f72d:::obj90:head@111'90, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.701+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:9720f72d:::obj90:head, version: 111'90, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9720f72d:::obj90:head@111'90, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.701+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.701+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 742 ==== pg_backfill(progress 1.0 e 116/116 lb 1:96d0cad0:::obj307:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99b80 con 0x4acb000 2026-02-11T23:31:27.701+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:9720f72d:::obj90:head, version: 111'90, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9720f72d:::obj90:head@111'90, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.701+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:96d0cad0:::obj307:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.701+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:96d0cad0:::obj307:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.701+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.701+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:9720f72d:::obj90:head, version: 111'90, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9720f72d:::obj90:head@111'90, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.701+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.701+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:9720f72d:::obj90:head, version: 111'90, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9720f72d:::obj90:head@111'90, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 116/113 [PushOp(1:9720f72d:::obj90:head, version: 111'90, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9720f72d:::obj90:head@111'90, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:96183f21:::obj585:head local-lis/les=113/114 n=352 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.701+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.701+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:96183f21:::obj585:head local-lis/les=113/114 n=352 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:9720f72d:::obj90:head, version: 111'90, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9720f72d:::obj90:head@111'90, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.701+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:96183f21:::obj585:head local-lis/les=113/114 n=352 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:9720f72d:::obj90:head@111'90, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.701+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.701+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:96d0cad0:::obj307:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.701+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:96183f21:::obj585:head local-lis/les=113/114 n=352 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:9720f72d:::obj90:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.701+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:96183f21:::obj585:head local-lis/les=113/114 n=352 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:9720f72d:::obj90:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.701+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:96183f21:::obj585:head local-lis/les=113/114 n=352 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:9720f72d:::obj90:head 2026-02-11T23:31:27.701+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:96183f21:::obj585:head local-lis/les=113/114 n=352 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:9720f72d:::obj90:head v 111'90 2026-02-11T23:31:27.701+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:96183f21:::obj585:head local-lis/les=113/114 n=352 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.701+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.701+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:9720f72d:::obj90:head, version: 111'90, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9720f72d:::obj90:head@111'90, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.701+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.701+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.701+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:96d0cad0:::obj307:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.701+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:96d0cad0:::obj307:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 1.0 e 116/116 lb 1:96d0cad0:::obj307:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:96183f21:::obj585:head local-lis/les=113/114 n=352 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.701+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.701+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:96183f21:::obj585:head local-lis/les=113/114 n=352 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:96d0cad0:::obj307:head) v3 2026-02-11T23:31:27.701+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:96183f21:::obj585:head local-lis/les=113/114 n=352 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:96d0cad0:::obj307:head) 2026-02-11T23:31:27.701+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.701+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:96d0cad0:::obj307:head) v3 finish latency 0.000009 2026-02-11T23:31:27.701+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:96d0cad0:::obj307:head local-lis/les=113/114 n=353 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.701+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.701+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.701+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:96d0cad0:::obj307:head local-lis/les=113/114 n=353 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.701+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.701+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.701+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.701+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x613f380 con 0x4acb000 2026-02-11T23:31:27.701+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:9720f72d:::obj90:head)]) -- 0x6014000 con 0x4acb000 2026-02-11T23:31:27.701+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.702+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 743 ==== MOSDPGPush(1.0 116/113 [PushOp(1:98019cfb:::obj381:head, version: 111'381, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:98019cfb:::obj381:head@111'381, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559dd40 con 0x4acb000 2026-02-11T23:31:27.702+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:98019cfb:::obj381:head, version: 111'381, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:98019cfb:::obj381:head@111'381, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 116 2026-02-11T23:31:27.702+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:98019cfb:::obj381:head, version: 111'381, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:98019cfb:::obj381:head@111'381, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.702+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.702+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 744 ==== pg_backfill(progress 1.0 e 116/116 lb 1:9720f72d:::obj90:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d9a100 con 0x4acb000 2026-02-11T23:31:27.702+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:98019cfb:::obj381:head, version: 111'381, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:98019cfb:::obj381:head@111'381, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.702+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:9720f72d:::obj90:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.702+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:9720f72d:::obj90:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.702+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.702+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:98019cfb:::obj381:head, version: 111'381, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:98019cfb:::obj381:head@111'381, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.702+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.702+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:98019cfb:::obj381:head, version: 111'381, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:98019cfb:::obj381:head@111'381, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 116/113 [PushOp(1:98019cfb:::obj381:head, version: 111'381, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:98019cfb:::obj381:head@111'381, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:96d0cad0:::obj307:head local-lis/les=113/114 n=353 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.702+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.702+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:96d0cad0:::obj307:head local-lis/les=113/114 n=353 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:98019cfb:::obj381:head, version: 111'381, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:98019cfb:::obj381:head@111'381, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.702+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:96d0cad0:::obj307:head local-lis/les=113/114 n=353 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:98019cfb:::obj381:head@111'381, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.702+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.702+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:9720f72d:::obj90:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.702+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:96d0cad0:::obj307:head local-lis/les=113/114 n=353 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:98019cfb:::obj381:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.702+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:96d0cad0:::obj307:head local-lis/les=113/114 n=353 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:98019cfb:::obj381:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.702+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:96d0cad0:::obj307:head local-lis/les=113/114 n=353 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:98019cfb:::obj381:head 2026-02-11T23:31:27.702+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:96d0cad0:::obj307:head local-lis/les=113/114 n=353 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:98019cfb:::obj381:head v 111'381 2026-02-11T23:31:27.702+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:96d0cad0:::obj307:head local-lis/les=113/114 n=353 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.702+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.702+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:98019cfb:::obj381:head, version: 111'381, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:98019cfb:::obj381:head@111'381, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.702+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.702+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.702+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:9720f72d:::obj90:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.702+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:9720f72d:::obj90:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:9720f72d:::obj90:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:96d0cad0:::obj307:head local-lis/les=113/114 n=353 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.702+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.702+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:96d0cad0:::obj307:head local-lis/les=113/114 n=353 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:9720f72d:::obj90:head) v3 2026-02-11T23:31:27.702+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:96d0cad0:::obj307:head local-lis/les=113/114 n=353 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:9720f72d:::obj90:head) 2026-02-11T23:31:27.702+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.702+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:9720f72d:::obj90:head) v3 finish latency 0.000009 2026-02-11T23:31:27.702+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9720f72d:::obj90:head local-lis/les=113/114 n=354 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.702+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.702+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.702+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9720f72d:::obj90:head local-lis/les=113/114 n=354 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.702+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.702+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.702+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.702+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x614d800 con 0x4acb000 2026-02-11T23:31:27.702+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:98019cfb:::obj381:head)]) -- 0x615ca80 con 0x4acb000 2026-02-11T23:31:27.702+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.702+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 745 ==== MOSDPGPush(1.0 116/113 [PushOp(1:98b68518:::obj150:head, version: 111'150, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:98b68518:::obj150:head@111'150, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d8680 con 0x4acb000 2026-02-11T23:31:27.702+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:98b68518:::obj150:head, version: 111'150, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:98b68518:::obj150:head@111'150, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.702+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:98b68518:::obj150:head, version: 111'150, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:98b68518:::obj150:head@111'150, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.702+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.702+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 746 ==== pg_backfill(progress 1.0 e 116/116 lb 1:98019cfb:::obj381:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9a680 con 0x4acb000 2026-02-11T23:31:27.702+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:98019cfb:::obj381:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.702+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:98b68518:::obj150:head, version: 111'150, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:98b68518:::obj150:head@111'150, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.702+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:98019cfb:::obj381:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.702+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.702+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:98b68518:::obj150:head, version: 111'150, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:98b68518:::obj150:head@111'150, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.702+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:98b68518:::obj150:head, version: 111'150, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:98b68518:::obj150:head@111'150, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 116/113 [PushOp(1:98b68518:::obj150:head, version: 111'150, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:98b68518:::obj150:head@111'150, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:9720f72d:::obj90:head local-lis/les=113/114 n=354 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.702+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.702+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.702+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:98019cfb:::obj381:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.702+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9720f72d:::obj90:head local-lis/les=113/114 n=354 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:98b68518:::obj150:head, version: 111'150, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:98b68518:::obj150:head@111'150, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.702+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9720f72d:::obj90:head local-lis/les=113/114 n=354 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:98b68518:::obj150:head@111'150, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.702+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9720f72d:::obj90:head local-lis/les=113/114 n=354 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:98b68518:::obj150:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.702+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9720f72d:::obj90:head local-lis/les=113/114 n=354 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:98b68518:::obj150:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.702+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9720f72d:::obj90:head local-lis/les=113/114 n=354 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:98b68518:::obj150:head 2026-02-11T23:31:27.702+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9720f72d:::obj90:head local-lis/les=113/114 n=354 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:98b68518:::obj150:head v 111'150 2026-02-11T23:31:27.702+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9720f72d:::obj90:head local-lis/les=113/114 n=354 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.702+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.702+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:98b68518:::obj150:head, version: 111'150, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:98b68518:::obj150:head@111'150, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.702+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.702+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.702+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:98019cfb:::obj381:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.702+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:98019cfb:::obj381:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:98019cfb:::obj381:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:9720f72d:::obj90:head local-lis/les=113/114 n=354 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.702+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.702+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9720f72d:::obj90:head local-lis/les=113/114 n=354 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:98019cfb:::obj381:head) v3 2026-02-11T23:31:27.702+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9720f72d:::obj90:head local-lis/les=113/114 n=354 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:98019cfb:::obj381:head) 2026-02-11T23:31:27.702+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.702+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:98019cfb:::obj381:head) v3 finish latency 0.000010 2026-02-11T23:31:27.702+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:98019cfb:::obj381:head local-lis/les=113/114 n=355 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.702+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.703+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.703+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:98019cfb:::obj381:head local-lis/les=113/114 n=355 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.703+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.703+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.703+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.703+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x613f080 con 0x4acb000 2026-02-11T23:31:27.703+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:98b68518:::obj150:head)]) -- 0x615c780 con 0x4acb000 2026-02-11T23:31:27.703+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.703+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 747 ==== MOSDPGPush(1.0 116/113 [PushOp(1:9956259d:::obj240:head, version: 111'240, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9956259d:::obj240:head@111'240, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56ca1a0 con 0x4acb000 2026-02-11T23:31:27.703+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:9956259d:::obj240:head, version: 111'240, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9956259d:::obj240:head@111'240, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 116 2026-02-11T23:31:27.703+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:9956259d:::obj240:head, version: 111'240, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9956259d:::obj240:head@111'240, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.703+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.703+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 748 ==== pg_backfill(progress 1.0 e 116/116 lb 1:98b68518:::obj150:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9ac00 con 0x4acb000 2026-02-11T23:31:27.703+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:9956259d:::obj240:head, version: 111'240, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9956259d:::obj240:head@111'240, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.703+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:98b68518:::obj150:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.703+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:98b68518:::obj150:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.703+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.703+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:9956259d:::obj240:head, version: 111'240, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9956259d:::obj240:head@111'240, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.703+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:9956259d:::obj240:head, version: 111'240, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9956259d:::obj240:head@111'240, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(1.0 116/113 [PushOp(1:9956259d:::obj240:head, version: 111'240, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9956259d:::obj240:head@111'240, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:98019cfb:::obj381:head local-lis/les=113/114 n=355 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.703+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.703+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.703+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:98019cfb:::obj381:head local-lis/les=113/114 n=355 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:9956259d:::obj240:head, version: 111'240, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9956259d:::obj240:head@111'240, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.703+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:98b68518:::obj150:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.703+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:98019cfb:::obj381:head local-lis/les=113/114 n=355 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:9956259d:::obj240:head@111'240, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.703+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:98019cfb:::obj381:head local-lis/les=113/114 n=355 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:9956259d:::obj240:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.703+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:98019cfb:::obj381:head local-lis/les=113/114 n=355 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:9956259d:::obj240:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.703+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:98019cfb:::obj381:head local-lis/les=113/114 n=355 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:9956259d:::obj240:head 2026-02-11T23:31:27.703+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:98019cfb:::obj381:head local-lis/les=113/114 n=355 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:9956259d:::obj240:head v 111'240 2026-02-11T23:31:27.703+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:98019cfb:::obj381:head local-lis/les=113/114 n=355 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.703+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.703+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:9956259d:::obj240:head, version: 111'240, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9956259d:::obj240:head@111'240, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000044 2026-02-11T23:31:27.703+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.703+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.703+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:98b68518:::obj150:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.703+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:98b68518:::obj150:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 1.0 e 116/116 lb 1:98b68518:::obj150:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:98019cfb:::obj381:head local-lis/les=113/114 n=355 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.703+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.703+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:98019cfb:::obj381:head local-lis/les=113/114 n=355 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:98b68518:::obj150:head) v3 2026-02-11T23:31:27.703+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:98019cfb:::obj381:head local-lis/les=113/114 n=355 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:98b68518:::obj150:head) 2026-02-11T23:31:27.703+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.703+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:98b68518:::obj150:head) v3 finish latency 0.000010 2026-02-11T23:31:27.703+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:98b68518:::obj150:head local-lis/les=113/114 n=356 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.703+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.703+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.703+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:98b68518:::obj150:head local-lis/les=113/114 n=356 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.703+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.703+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.703+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.703+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6014780 con 0x4acb000 2026-02-11T23:31:27.703+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:9956259d:::obj240:head)]) -- 0x615c480 con 0x4acb000 2026-02-11T23:31:27.703+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.704+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 749 ==== MOSDPGPush(1.0 116/113 [PushOp(1:996db15f:::obj326:head, version: 111'326, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:996db15f:::obj326:head@111'326, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d8000 con 0x4acb000 2026-02-11T23:31:27.704+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:996db15f:::obj326:head, version: 111'326, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:996db15f:::obj326:head@111'326, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000016 epoch 116 2026-02-11T23:31:27.704+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:996db15f:::obj326:head, version: 111'326, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:996db15f:::obj326:head@111'326, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.704+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.704+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 750 ==== pg_backfill(progress 1.0 e 116/116 lb 1:9956259d:::obj240:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9b180 con 0x4acb000 2026-02-11T23:31:27.704+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:996db15f:::obj326:head, version: 111'326, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:996db15f:::obj326:head@111'326, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.704+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:9956259d:::obj240:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.704+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:9956259d:::obj240:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.704+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.704+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:996db15f:::obj326:head, version: 111'326, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:996db15f:::obj326:head@111'326, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.704+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.704+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:996db15f:::obj326:head, version: 111'326, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:996db15f:::obj326:head@111'326, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000040 MOSDPGPush(1.0 116/113 [PushOp(1:996db15f:::obj326:head, version: 111'326, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:996db15f:::obj326:head@111'326, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:98b68518:::obj150:head local-lis/les=113/114 n=356 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.704+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.704+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:98b68518:::obj150:head local-lis/les=113/114 n=356 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:996db15f:::obj326:head, version: 111'326, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:996db15f:::obj326:head@111'326, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.704+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:98b68518:::obj150:head local-lis/les=113/114 n=356 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:996db15f:::obj326:head@111'326, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.704+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.704+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:9956259d:::obj240:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.704+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:98b68518:::obj150:head local-lis/les=113/114 n=356 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:996db15f:::obj326:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.704+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:98b68518:::obj150:head local-lis/les=113/114 n=356 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:996db15f:::obj326:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.704+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:98b68518:::obj150:head local-lis/les=113/114 n=356 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:996db15f:::obj326:head 2026-02-11T23:31:27.704+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:98b68518:::obj150:head local-lis/les=113/114 n=356 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:996db15f:::obj326:head v 111'326 2026-02-11T23:31:27.704+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:98b68518:::obj150:head local-lis/les=113/114 n=356 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.704+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.704+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:996db15f:::obj326:head, version: 111'326, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:996db15f:::obj326:head@111'326, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.704+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.704+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.704+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:9956259d:::obj240:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.704+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:9956259d:::obj240:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 1.0 e 116/116 lb 1:9956259d:::obj240:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:98b68518:::obj150:head local-lis/les=113/114 n=356 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.704+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.704+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:98b68518:::obj150:head local-lis/les=113/114 n=356 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:9956259d:::obj240:head) v3 2026-02-11T23:31:27.704+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:98b68518:::obj150:head local-lis/les=113/114 n=356 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:9956259d:::obj240:head) 2026-02-11T23:31:27.704+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.704+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:9956259d:::obj240:head) v3 finish latency 0.000009 2026-02-11T23:31:27.704+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9956259d:::obj240:head local-lis/les=113/114 n=357 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.704+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.704+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.704+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9956259d:::obj240:head local-lis/les=113/114 n=357 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.704+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.704+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.704+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.704+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x612f380 con 0x4acb000 2026-02-11T23:31:27.704+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:996db15f:::obj326:head)]) -- 0x612f680 con 0x4acb000 2026-02-11T23:31:27.704+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.704+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 751 ==== MOSDPGPush(1.0 116/113 [PushOp(1:99944c24:::obj170:head, version: 111'170, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:99944c24:::obj170:head@111'170, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d84e0 con 0x4acb000 2026-02-11T23:31:27.704+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:99944c24:::obj170:head, version: 111'170, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:99944c24:::obj170:head@111'170, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000016 epoch 116 2026-02-11T23:31:27.704+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:99944c24:::obj170:head, version: 111'170, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:99944c24:::obj170:head@111'170, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.704+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.704+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:99944c24:::obj170:head, version: 111'170, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:99944c24:::obj170:head@111'170, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.704+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.704+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 752 ==== pg_backfill(progress 1.0 e 116/116 lb 1:996db15f:::obj326:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98000 con 0x4acb000 2026-02-11T23:31:27.704+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:99944c24:::obj170:head, version: 111'170, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:99944c24:::obj170:head@111'170, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.704+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:996db15f:::obj326:head) v3 prio 15 type 95 cost 0 latency 0.000010 epoch 116 2026-02-11T23:31:27.704+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:996db15f:::obj326:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.704+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:99944c24:::obj170:head, version: 111'170, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:99944c24:::obj170:head@111'170, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000043 MOSDPGPush(1.0 116/113 [PushOp(1:99944c24:::obj170:head, version: 111'170, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:99944c24:::obj170:head@111'170, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:9956259d:::obj240:head local-lis/les=113/114 n=357 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.704+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.704+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9956259d:::obj240:head local-lis/les=113/114 n=357 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:99944c24:::obj170:head, version: 111'170, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:99944c24:::obj170:head@111'170, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.704+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.704+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:996db15f:::obj326:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.704+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9956259d:::obj240:head local-lis/les=113/114 n=357 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:99944c24:::obj170:head@111'170, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.704+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9956259d:::obj240:head local-lis/les=113/114 n=357 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:99944c24:::obj170:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.704+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9956259d:::obj240:head local-lis/les=113/114 n=357 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:99944c24:::obj170:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.704+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9956259d:::obj240:head local-lis/les=113/114 n=357 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:99944c24:::obj170:head 2026-02-11T23:31:27.704+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9956259d:::obj240:head local-lis/les=113/114 n=357 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:99944c24:::obj170:head v 111'170 2026-02-11T23:31:27.704+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9956259d:::obj240:head local-lis/les=113/114 n=357 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.704+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.704+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:99944c24:::obj170:head, version: 111'170, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:99944c24:::obj170:head@111'170, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.704+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.704+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.704+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:996db15f:::obj326:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.704+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:996db15f:::obj326:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 1.0 e 116/116 lb 1:996db15f:::obj326:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:9956259d:::obj240:head local-lis/les=113/114 n=357 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.704+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.704+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9956259d:::obj240:head local-lis/les=113/114 n=357 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:996db15f:::obj326:head) v3 2026-02-11T23:31:27.704+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9956259d:::obj240:head local-lis/les=113/114 n=357 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:996db15f:::obj326:head) 2026-02-11T23:31:27.704+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.704+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:996db15f:::obj326:head) v3 finish latency 0.000009 2026-02-11T23:31:27.704+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:996db15f:::obj326:head local-lis/les=113/114 n=358 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.704+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.704+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.704+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:996db15f:::obj326:head local-lis/les=113/114 n=358 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.704+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.704+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.704+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.704+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6005e00 con 0x4acb000 2026-02-11T23:31:27.704+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:99944c24:::obj170:head)]) -- 0x6004f00 con 0x4acb000 2026-02-11T23:31:27.704+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.705+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 753 ==== MOSDPGPush(1.0 116/113 [PushOp(1:9a09113a:::obj7:head, version: 111'7, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9a09113a:::obj7:head@111'7, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 4999+0+0 (unknown 0 0 0) 0x5647380 con 0x4acb000 2026-02-11T23:31:27.705+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:9a09113a:::obj7:head, version: 111'7, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9a09113a:::obj7:head@111'7, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 116 2026-02-11T23:31:27.705+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:9a09113a:::obj7:head, version: 111'7, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9a09113a:::obj7:head@111'7, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.705+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.705+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 754 ==== pg_backfill(progress 1.0 e 116/116 lb 1:99944c24:::obj170:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98580 con 0x4acb000 2026-02-11T23:31:27.705+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:9a09113a:::obj7:head, version: 111'7, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9a09113a:::obj7:head@111'7, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.705+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:99944c24:::obj170:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.705+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.705+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:99944c24:::obj170:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.705+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:9a09113a:::obj7:head, version: 111'7, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9a09113a:::obj7:head@111'7, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.705+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.705+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:9a09113a:::obj7:head, version: 111'7, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9a09113a:::obj7:head@111'7, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 116/113 [PushOp(1:9a09113a:::obj7:head, version: 111'7, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9a09113a:::obj7:head@111'7, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:996db15f:::obj326:head local-lis/les=113/114 n=358 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.705+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.705+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:996db15f:::obj326:head local-lis/les=113/114 n=358 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:9a09113a:::obj7:head, version: 111'7, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9a09113a:::obj7:head@111'7, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.705+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:996db15f:::obj326:head local-lis/les=113/114 n=358 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:9a09113a:::obj7:head@111'7, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.705+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.705+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:99944c24:::obj170:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.705+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:996db15f:::obj326:head local-lis/les=113/114 n=358 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:9a09113a:::obj7:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.705+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:996db15f:::obj326:head local-lis/les=113/114 n=358 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:9a09113a:::obj7:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.705+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:996db15f:::obj326:head local-lis/les=113/114 n=358 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:9a09113a:::obj7:head 2026-02-11T23:31:27.705+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:996db15f:::obj326:head local-lis/les=113/114 n=358 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:9a09113a:::obj7:head v 111'7 2026-02-11T23:31:27.705+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:996db15f:::obj326:head local-lis/les=113/114 n=358 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.705+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.705+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:9a09113a:::obj7:head, version: 111'7, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9a09113a:::obj7:head@111'7, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.705+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.705+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.705+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:99944c24:::obj170:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.705+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:99944c24:::obj170:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 1.0 e 116/116 lb 1:99944c24:::obj170:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:996db15f:::obj326:head local-lis/les=113/114 n=358 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.705+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.705+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:996db15f:::obj326:head local-lis/les=113/114 n=358 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:99944c24:::obj170:head) v3 2026-02-11T23:31:27.705+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:996db15f:::obj326:head local-lis/les=113/114 n=358 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:99944c24:::obj170:head) 2026-02-11T23:31:27.705+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.705+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:99944c24:::obj170:head) v3 finish latency 0.000009 2026-02-11T23:31:27.705+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:99944c24:::obj170:head local-lis/les=113/114 n=359 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.705+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.705+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.705+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:99944c24:::obj170:head local-lis/les=113/114 n=359 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.705+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.705+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.705+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.705+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x612f200 con 0x4acb000 2026-02-11T23:31:27.705+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:9a09113a:::obj7:head)]) -- 0x5fed980 con 0x4acb000 2026-02-11T23:31:27.705+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.705+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 755 ==== MOSDPGPush(1.0 116/113 [PushOp(1:9bbc8a19:::obj348:head, version: 111'348, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9bbc8a19:::obj348:head@111'348, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d8ea0 con 0x4acb000 2026-02-11T23:31:27.705+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:9bbc8a19:::obj348:head, version: 111'348, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9bbc8a19:::obj348:head@111'348, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000019 epoch 116 2026-02-11T23:31:27.705+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:9bbc8a19:::obj348:head, version: 111'348, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9bbc8a19:::obj348:head@111'348, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.705+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.705+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 756 ==== pg_backfill(progress 1.0 e 116/116 lb 1:9a09113a:::obj7:head) ==== 1060+0+0 (unknown 0 0 0) 0x5d98b00 con 0x4acb000 2026-02-11T23:31:27.705+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:9bbc8a19:::obj348:head, version: 111'348, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9bbc8a19:::obj348:head@111'348, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.705+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:9a09113a:::obj7:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.705+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.705+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:9a09113a:::obj7:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.705+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:9bbc8a19:::obj348:head, version: 111'348, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9bbc8a19:::obj348:head@111'348, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.705+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.705+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:9bbc8a19:::obj348:head, version: 111'348, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9bbc8a19:::obj348:head@111'348, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000047 MOSDPGPush(1.0 116/113 [PushOp(1:9bbc8a19:::obj348:head, version: 111'348, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9bbc8a19:::obj348:head@111'348, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:99944c24:::obj170:head local-lis/les=113/114 n=359 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.705+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:9a09113a:::obj7:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.705+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.705+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:99944c24:::obj170:head local-lis/les=113/114 n=359 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:9bbc8a19:::obj348:head, version: 111'348, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9bbc8a19:::obj348:head@111'348, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.705+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:99944c24:::obj170:head local-lis/les=113/114 n=359 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:9bbc8a19:::obj348:head@111'348, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.705+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:99944c24:::obj170:head local-lis/les=113/114 n=359 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:9bbc8a19:::obj348:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.705+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:99944c24:::obj170:head local-lis/les=113/114 n=359 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:9bbc8a19:::obj348:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.705+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:99944c24:::obj170:head local-lis/les=113/114 n=359 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:9bbc8a19:::obj348:head 2026-02-11T23:31:27.705+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:99944c24:::obj170:head local-lis/les=113/114 n=359 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:9bbc8a19:::obj348:head v 111'348 2026-02-11T23:31:27.705+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:99944c24:::obj170:head local-lis/les=113/114 n=359 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.705+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.705+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:9bbc8a19:::obj348:head, version: 111'348, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9bbc8a19:::obj348:head@111'348, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.705+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.705+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.705+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:9a09113a:::obj7:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.705+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:9a09113a:::obj7:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 1.0 e 116/116 lb 1:9a09113a:::obj7:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:99944c24:::obj170:head local-lis/les=113/114 n=359 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.705+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.705+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:99944c24:::obj170:head local-lis/les=113/114 n=359 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:9a09113a:::obj7:head) v3 2026-02-11T23:31:27.705+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:99944c24:::obj170:head local-lis/les=113/114 n=359 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:9a09113a:::obj7:head) 2026-02-11T23:31:27.705+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.705+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:9a09113a:::obj7:head) v3 finish latency 0.000010 2026-02-11T23:31:27.705+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9a09113a:::obj7:head local-lis/les=113/114 n=360 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.705+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.706+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.706+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9a09113a:::obj7:head local-lis/les=113/114 n=360 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.706+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.706+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.706+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.706+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fce180 con 0x4acb000 2026-02-11T23:31:27.706+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:9bbc8a19:::obj348:head)]) -- 0x614ca80 con 0x4acb000 2026-02-11T23:31:27.706+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.706+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 757 ==== MOSDPGPush(1.0 116/113 [PushOp(1:9c3090f1:::obj522:head, version: 111'522, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c3090f1:::obj522:head@111'522, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d9380 con 0x4acb000 2026-02-11T23:31:27.706+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:9c3090f1:::obj522:head, version: 111'522, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c3090f1:::obj522:head@111'522, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 116 2026-02-11T23:31:27.706+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:9c3090f1:::obj522:head, version: 111'522, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c3090f1:::obj522:head@111'522, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.706+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.706+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 758 ==== pg_backfill(progress 1.0 e 116/116 lb 1:9bbc8a19:::obj348:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99080 con 0x4acb000 2026-02-11T23:31:27.706+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:9bbc8a19:::obj348:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 116 2026-02-11T23:31:27.706+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:9c3090f1:::obj522:head, version: 111'522, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c3090f1:::obj522:head@111'522, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.706+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:9bbc8a19:::obj348:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.706+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.706+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:9c3090f1:::obj522:head, version: 111'522, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c3090f1:::obj522:head@111'522, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.706+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:9c3090f1:::obj522:head, version: 111'522, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c3090f1:::obj522:head@111'522, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000040 MOSDPGPush(1.0 116/113 [PushOp(1:9c3090f1:::obj522:head, version: 111'522, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c3090f1:::obj522:head@111'522, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:9a09113a:::obj7:head local-lis/les=113/114 n=360 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.706+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.706+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.706+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9a09113a:::obj7:head local-lis/les=113/114 n=360 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:9c3090f1:::obj522:head, version: 111'522, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c3090f1:::obj522:head@111'522, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.706+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:9bbc8a19:::obj348:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.706+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9a09113a:::obj7:head local-lis/les=113/114 n=360 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:9c3090f1:::obj522:head@111'522, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.706+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9a09113a:::obj7:head local-lis/les=113/114 n=360 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:9c3090f1:::obj522:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.706+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9a09113a:::obj7:head local-lis/les=113/114 n=360 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:9c3090f1:::obj522:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.706+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9a09113a:::obj7:head local-lis/les=113/114 n=360 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:9c3090f1:::obj522:head 2026-02-11T23:31:27.706+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9a09113a:::obj7:head local-lis/les=113/114 n=360 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:9c3090f1:::obj522:head v 111'522 2026-02-11T23:31:27.706+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9a09113a:::obj7:head local-lis/les=113/114 n=360 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.706+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.706+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:9c3090f1:::obj522:head, version: 111'522, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c3090f1:::obj522:head@111'522, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.706+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.706+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.706+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:9bbc8a19:::obj348:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.706+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:9bbc8a19:::obj348:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 1.0 e 116/116 lb 1:9bbc8a19:::obj348:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:9a09113a:::obj7:head local-lis/les=113/114 n=360 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.706+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.706+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9a09113a:::obj7:head local-lis/les=113/114 n=360 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:9bbc8a19:::obj348:head) v3 2026-02-11T23:31:27.706+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9a09113a:::obj7:head local-lis/les=113/114 n=360 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:9bbc8a19:::obj348:head) 2026-02-11T23:31:27.706+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.706+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:9bbc8a19:::obj348:head) v3 finish latency 0.000021 2026-02-11T23:31:27.706+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9bbc8a19:::obj348:head local-lis/les=113/114 n=361 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.706+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.706+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.706+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9bbc8a19:::obj348:head local-lis/les=113/114 n=361 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.706+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.706+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.706+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.706+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x612ef00 con 0x4acb000 2026-02-11T23:31:27.706+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:9c3090f1:::obj522:head)]) -- 0x613fc80 con 0x4acb000 2026-02-11T23:31:27.706+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.707+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 759 ==== MOSDPGPush(1.0 116/113 [PushOp(1:9c626db4:::obj119:head, version: 111'119, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c626db4:::obj119:head@111'119, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56cb040 con 0x4acb000 2026-02-11T23:31:27.707+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:9c626db4:::obj119:head, version: 111'119, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c626db4:::obj119:head@111'119, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 116 2026-02-11T23:31:27.707+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:9c626db4:::obj119:head, version: 111'119, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c626db4:::obj119:head@111'119, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.707+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.707+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:9c626db4:::obj119:head, version: 111'119, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c626db4:::obj119:head@111'119, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.707+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 760 ==== pg_backfill(progress 1.0 e 116/116 lb 1:9c3090f1:::obj522:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99600 con 0x4acb000 2026-02-11T23:31:27.707+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:9c3090f1:::obj522:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 116 2026-02-11T23:31:27.707+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.707+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:9c3090f1:::obj522:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.707+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:9c626db4:::obj119:head, version: 111'119, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c626db4:::obj119:head@111'119, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.707+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.707+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:9c626db4:::obj119:head, version: 111'119, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c626db4:::obj119:head@111'119, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 116/113 [PushOp(1:9c626db4:::obj119:head, version: 111'119, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c626db4:::obj119:head@111'119, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:9bbc8a19:::obj348:head local-lis/les=113/114 n=361 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.707+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.707+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9bbc8a19:::obj348:head local-lis/les=113/114 n=361 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:9c626db4:::obj119:head, version: 111'119, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c626db4:::obj119:head@111'119, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.707+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9bbc8a19:::obj348:head local-lis/les=113/114 n=361 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:9c626db4:::obj119:head@111'119, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.707+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.707+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:9c3090f1:::obj522:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.707+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9bbc8a19:::obj348:head local-lis/les=113/114 n=361 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:9c626db4:::obj119:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.707+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9bbc8a19:::obj348:head local-lis/les=113/114 n=361 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:9c626db4:::obj119:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.707+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9bbc8a19:::obj348:head local-lis/les=113/114 n=361 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:9c626db4:::obj119:head 2026-02-11T23:31:27.707+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9bbc8a19:::obj348:head local-lis/les=113/114 n=361 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:9c626db4:::obj119:head v 111'119 2026-02-11T23:31:27.707+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9bbc8a19:::obj348:head local-lis/les=113/114 n=361 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.707+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.707+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:9c626db4:::obj119:head, version: 111'119, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c626db4:::obj119:head@111'119, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.707+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.707+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.707+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:9c3090f1:::obj522:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.707+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:9c3090f1:::obj522:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:9c3090f1:::obj522:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:9bbc8a19:::obj348:head local-lis/les=113/114 n=361 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.707+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.707+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9bbc8a19:::obj348:head local-lis/les=113/114 n=361 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:9c3090f1:::obj522:head) v3 2026-02-11T23:31:27.707+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9bbc8a19:::obj348:head local-lis/les=113/114 n=361 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:9c3090f1:::obj522:head) 2026-02-11T23:31:27.707+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.707+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:9c3090f1:::obj522:head) v3 finish latency 0.000012 2026-02-11T23:31:27.707+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9c3090f1:::obj522:head local-lis/les=113/114 n=362 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.707+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.707+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.707+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9c3090f1:::obj522:head local-lis/les=113/114 n=362 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.707+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.707+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.707+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.707+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5ca0180 con 0x4acb000 2026-02-11T23:31:27.707+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:9c626db4:::obj119:head)]) -- 0x614c780 con 0x4acb000 2026-02-11T23:31:27.707+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.707+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 761 ==== MOSDPGPush(1.0 116/113 [PushOp(1:9c9438f5:::obj513:head, version: 111'513, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c9438f5:::obj513:head@111'513, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56ca1a0 con 0x4acb000 2026-02-11T23:31:27.707+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:9c9438f5:::obj513:head, version: 111'513, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c9438f5:::obj513:head@111'513, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.707+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:9c9438f5:::obj513:head, version: 111'513, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c9438f5:::obj513:head@111'513, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.707+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.707+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 762 ==== pg_backfill(progress 1.0 e 116/116 lb 1:9c626db4:::obj119:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99b80 con 0x4acb000 2026-02-11T23:31:27.707+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:9c626db4:::obj119:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.707+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:9c9438f5:::obj513:head, version: 111'513, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c9438f5:::obj513:head@111'513, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.707+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:9c626db4:::obj119:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.707+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.707+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:9c9438f5:::obj513:head, version: 111'513, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c9438f5:::obj513:head@111'513, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.707+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:9c9438f5:::obj513:head, version: 111'513, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c9438f5:::obj513:head@111'513, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 116/113 [PushOp(1:9c9438f5:::obj513:head, version: 111'513, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c9438f5:::obj513:head@111'513, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:9c3090f1:::obj522:head local-lis/les=113/114 n=362 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.707+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.707+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.707+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:9c626db4:::obj119:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.707+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9c3090f1:::obj522:head local-lis/les=113/114 n=362 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:9c9438f5:::obj513:head, version: 111'513, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c9438f5:::obj513:head@111'513, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.707+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9c3090f1:::obj522:head local-lis/les=113/114 n=362 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:9c9438f5:::obj513:head@111'513, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.707+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9c3090f1:::obj522:head local-lis/les=113/114 n=362 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:9c9438f5:::obj513:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.707+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9c3090f1:::obj522:head local-lis/les=113/114 n=362 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:9c9438f5:::obj513:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.707+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9c3090f1:::obj522:head local-lis/les=113/114 n=362 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:9c9438f5:::obj513:head 2026-02-11T23:31:27.707+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9c3090f1:::obj522:head local-lis/les=113/114 n=362 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:9c9438f5:::obj513:head v 111'513 2026-02-11T23:31:27.707+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9c3090f1:::obj522:head local-lis/les=113/114 n=362 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.707+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.707+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:9c9438f5:::obj513:head, version: 111'513, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c9438f5:::obj513:head@111'513, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.707+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.707+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.707+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:9c626db4:::obj119:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.707+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:9c626db4:::obj119:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 1.0 e 116/116 lb 1:9c626db4:::obj119:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:9c3090f1:::obj522:head local-lis/les=113/114 n=362 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.707+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.707+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9c3090f1:::obj522:head local-lis/les=113/114 n=362 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:9c626db4:::obj119:head) v3 2026-02-11T23:31:27.707+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9c3090f1:::obj522:head local-lis/les=113/114 n=362 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:9c626db4:::obj119:head) 2026-02-11T23:31:27.707+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.707+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:9c626db4:::obj119:head) v3 finish latency 0.000010 2026-02-11T23:31:27.707+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9c626db4:::obj119:head local-lis/les=113/114 n=363 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.707+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.708+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.708+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9c626db4:::obj119:head local-lis/les=113/114 n=363 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.708+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.708+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.708+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.708+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x612ed80 con 0x4acb000 2026-02-11T23:31:27.708+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:9c9438f5:::obj513:head)]) -- 0x6170300 con 0x4acb000 2026-02-11T23:31:27.708+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.708+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 763 ==== MOSDPGPush(1.0 116/113 [PushOp(1:9f41ce81:::obj424:head, version: 111'424, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f41ce81:::obj424:head@111'424, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d8680 con 0x4acb000 2026-02-11T23:31:27.708+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:9f41ce81:::obj424:head, version: 111'424, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f41ce81:::obj424:head@111'424, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 116 2026-02-11T23:31:27.708+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:9f41ce81:::obj424:head, version: 111'424, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f41ce81:::obj424:head@111'424, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.708+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.708+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 764 ==== pg_backfill(progress 1.0 e 116/116 lb 1:9c9438f5:::obj513:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9a100 con 0x4acb000 2026-02-11T23:31:27.708+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:9c9438f5:::obj513:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 116 2026-02-11T23:31:27.708+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:9f41ce81:::obj424:head, version: 111'424, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f41ce81:::obj424:head@111'424, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.708+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:9c9438f5:::obj513:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.708+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.708+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:9f41ce81:::obj424:head, version: 111'424, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f41ce81:::obj424:head@111'424, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.708+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:9f41ce81:::obj424:head, version: 111'424, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f41ce81:::obj424:head@111'424, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000042 MOSDPGPush(1.0 116/113 [PushOp(1:9f41ce81:::obj424:head, version: 111'424, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f41ce81:::obj424:head@111'424, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:9c626db4:::obj119:head local-lis/les=113/114 n=363 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.708+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.708+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.708+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9c626db4:::obj119:head local-lis/les=113/114 n=363 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:9f41ce81:::obj424:head, version: 111'424, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f41ce81:::obj424:head@111'424, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.708+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:9c9438f5:::obj513:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.708+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9c626db4:::obj119:head local-lis/les=113/114 n=363 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:9f41ce81:::obj424:head@111'424, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.708+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9c626db4:::obj119:head local-lis/les=113/114 n=363 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:9f41ce81:::obj424:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.708+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9c626db4:::obj119:head local-lis/les=113/114 n=363 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:9f41ce81:::obj424:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.708+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9c626db4:::obj119:head local-lis/les=113/114 n=363 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:9f41ce81:::obj424:head 2026-02-11T23:31:27.708+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9c626db4:::obj119:head local-lis/les=113/114 n=363 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:9f41ce81:::obj424:head v 111'424 2026-02-11T23:31:27.708+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9c626db4:::obj119:head local-lis/les=113/114 n=363 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.708+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.708+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:9f41ce81:::obj424:head, version: 111'424, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f41ce81:::obj424:head@111'424, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.708+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.708+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.708+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:9c9438f5:::obj513:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.708+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:9c9438f5:::obj513:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 1.0 e 116/116 lb 1:9c9438f5:::obj513:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:9c626db4:::obj119:head local-lis/les=113/114 n=363 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.708+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.708+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9c626db4:::obj119:head local-lis/les=113/114 n=363 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:9c9438f5:::obj513:head) v3 2026-02-11T23:31:27.708+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9c626db4:::obj119:head local-lis/les=113/114 n=363 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:9c9438f5:::obj513:head) 2026-02-11T23:31:27.708+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.708+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:9c9438f5:::obj513:head) v3 finish latency 0.000009 2026-02-11T23:31:27.708+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9c9438f5:::obj513:head local-lis/les=113/114 n=364 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.708+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.708+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.708+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9c9438f5:::obj513:head local-lis/les=113/114 n=364 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.708+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.708+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.708+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.708+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fed800 con 0x4acb000 2026-02-11T23:31:27.708+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:9f41ce81:::obj424:head)]) -- 0x6004900 con 0x4acb000 2026-02-11T23:31:27.708+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.708+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 765 ==== MOSDPGPush(1.0 116/113 [PushOp(1:9f7e105e:::obj535:head, version: 111'535, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f7e105e:::obj535:head@111'535, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559dd40 con 0x4acb000 2026-02-11T23:31:27.708+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:9f7e105e:::obj535:head, version: 111'535, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f7e105e:::obj535:head@111'535, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.708+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:9f7e105e:::obj535:head, version: 111'535, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f7e105e:::obj535:head@111'535, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.708+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.708+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 766 ==== pg_backfill(progress 1.0 e 116/116 lb 1:9f41ce81:::obj424:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9a680 con 0x4acb000 2026-02-11T23:31:27.708+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:9f41ce81:::obj424:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.708+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:9f7e105e:::obj535:head, version: 111'535, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f7e105e:::obj535:head@111'535, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.708+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:9f41ce81:::obj424:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.708+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.708+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:9f7e105e:::obj535:head, version: 111'535, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f7e105e:::obj535:head@111'535, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.708+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:9f7e105e:::obj535:head, version: 111'535, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f7e105e:::obj535:head@111'535, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 116/113 [PushOp(1:9f7e105e:::obj535:head, version: 111'535, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f7e105e:::obj535:head@111'535, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:9c9438f5:::obj513:head local-lis/les=113/114 n=364 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.708+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.708+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.708+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9c9438f5:::obj513:head local-lis/les=113/114 n=364 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:9f7e105e:::obj535:head, version: 111'535, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f7e105e:::obj535:head@111'535, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.708+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:9f41ce81:::obj424:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.708+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9c9438f5:::obj513:head local-lis/les=113/114 n=364 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:9f7e105e:::obj535:head@111'535, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.708+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9c9438f5:::obj513:head local-lis/les=113/114 n=364 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:9f7e105e:::obj535:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.708+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9c9438f5:::obj513:head local-lis/les=113/114 n=364 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:9f7e105e:::obj535:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.708+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9c9438f5:::obj513:head local-lis/les=113/114 n=364 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:9f7e105e:::obj535:head 2026-02-11T23:31:27.708+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9c9438f5:::obj513:head local-lis/les=113/114 n=364 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:9f7e105e:::obj535:head v 111'535 2026-02-11T23:31:27.708+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9c9438f5:::obj513:head local-lis/les=113/114 n=364 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.708+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.709+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:9f7e105e:::obj535:head, version: 111'535, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f7e105e:::obj535:head@111'535, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:27.709+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.709+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.709+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:9f41ce81:::obj424:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.709+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:9f41ce81:::obj424:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 1.0 e 116/116 lb 1:9f41ce81:::obj424:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:9c9438f5:::obj513:head local-lis/les=113/114 n=364 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.709+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.709+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9c9438f5:::obj513:head local-lis/les=113/114 n=364 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:9f41ce81:::obj424:head) v3 2026-02-11T23:31:27.709+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9c9438f5:::obj513:head local-lis/les=113/114 n=364 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:9f41ce81:::obj424:head) 2026-02-11T23:31:27.709+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.709+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:9f41ce81:::obj424:head) v3 finish latency 0.000009 2026-02-11T23:31:27.709+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9f41ce81:::obj424:head local-lis/les=113/114 n=365 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.709+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.709+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.709+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9f41ce81:::obj424:head local-lis/les=113/114 n=365 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.709+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.709+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.709+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.709+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x612ec00 con 0x4acb000 2026-02-11T23:31:27.709+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:9f7e105e:::obj535:head)]) -- 0x6170600 con 0x4acb000 2026-02-11T23:31:27.709+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.709+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 767 ==== MOSDPGPush(1.0 116/113 [PushOp(1:9f8e934c:::obj456:head, version: 111'456, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f8e934c:::obj456:head@111'456, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c820 con 0x4acb000 2026-02-11T23:31:27.709+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:9f8e934c:::obj456:head, version: 111'456, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f8e934c:::obj456:head@111'456, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 116 2026-02-11T23:31:27.709+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:9f8e934c:::obj456:head, version: 111'456, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f8e934c:::obj456:head@111'456, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.709+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.709+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 768 ==== pg_backfill(progress 1.0 e 116/116 lb 1:9f7e105e:::obj535:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9ac00 con 0x4acb000 2026-02-11T23:31:27.709+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:9f8e934c:::obj456:head, version: 111'456, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f8e934c:::obj456:head@111'456, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.709+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:9f7e105e:::obj535:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.709+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:9f7e105e:::obj535:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.709+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.709+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:9f8e934c:::obj456:head, version: 111'456, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f8e934c:::obj456:head@111'456, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.709+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.709+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:9f8e934c:::obj456:head, version: 111'456, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f8e934c:::obj456:head@111'456, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(1.0 116/113 [PushOp(1:9f8e934c:::obj456:head, version: 111'456, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f8e934c:::obj456:head@111'456, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:9f41ce81:::obj424:head local-lis/les=113/114 n=365 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.709+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.709+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9f41ce81:::obj424:head local-lis/les=113/114 n=365 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:9f8e934c:::obj456:head, version: 111'456, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f8e934c:::obj456:head@111'456, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.709+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9f41ce81:::obj424:head local-lis/les=113/114 n=365 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:9f8e934c:::obj456:head@111'456, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.709+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.709+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:9f7e105e:::obj535:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.709+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9f41ce81:::obj424:head local-lis/les=113/114 n=365 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:9f8e934c:::obj456:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.709+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9f41ce81:::obj424:head local-lis/les=113/114 n=365 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:9f8e934c:::obj456:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.709+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9f41ce81:::obj424:head local-lis/les=113/114 n=365 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:9f8e934c:::obj456:head 2026-02-11T23:31:27.709+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9f41ce81:::obj424:head local-lis/les=113/114 n=365 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:9f8e934c:::obj456:head v 111'456 2026-02-11T23:31:27.709+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9f41ce81:::obj424:head local-lis/les=113/114 n=365 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.709+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.709+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:9f8e934c:::obj456:head, version: 111'456, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f8e934c:::obj456:head@111'456, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.709+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.709+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.709+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:9f7e105e:::obj535:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.709+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:9f7e105e:::obj535:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 1.0 e 116/116 lb 1:9f7e105e:::obj535:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:9f41ce81:::obj424:head local-lis/les=113/114 n=365 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.709+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.709+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9f41ce81:::obj424:head local-lis/les=113/114 n=365 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:9f7e105e:::obj535:head) v3 2026-02-11T23:31:27.709+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9f41ce81:::obj424:head local-lis/les=113/114 n=365 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:9f7e105e:::obj535:head) 2026-02-11T23:31:27.709+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.709+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:9f7e105e:::obj535:head) v3 finish latency 0.000009 2026-02-11T23:31:27.709+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9f7e105e:::obj535:head local-lis/les=113/114 n=366 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.709+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.709+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.709+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9f7e105e:::obj535:head local-lis/les=113/114 n=366 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.709+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.709+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.709+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.709+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5ca0480 con 0x4acb000 2026-02-11T23:31:27.709+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:9f8e934c:::obj456:head)]) -- 0x6015b00 con 0x4acb000 2026-02-11T23:31:27.709+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.710+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 769 ==== MOSDPGPush(1.0 116/113 [PushOp(1:a0060720:::obj493:head, version: 111'493, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0060720:::obj493:head@111'493, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c340 con 0x4acb000 2026-02-11T23:31:27.710+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:a0060720:::obj493:head, version: 111'493, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0060720:::obj493:head@111'493, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000016 epoch 116 2026-02-11T23:31:27.710+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a0060720:::obj493:head, version: 111'493, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0060720:::obj493:head@111'493, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.710+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.710+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 770 ==== pg_backfill(progress 1.0 e 116/116 lb 1:9f8e934c:::obj456:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9b180 con 0x4acb000 2026-02-11T23:31:27.710+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a0060720:::obj493:head, version: 111'493, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0060720:::obj493:head@111'493, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.710+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:9f8e934c:::obj456:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.710+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.710+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:9f8e934c:::obj456:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.710+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a0060720:::obj493:head, version: 111'493, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0060720:::obj493:head@111'493, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.710+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.710+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a0060720:::obj493:head, version: 111'493, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0060720:::obj493:head@111'493, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000043 MOSDPGPush(1.0 116/113 [PushOp(1:a0060720:::obj493:head, version: 111'493, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0060720:::obj493:head@111'493, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:9f7e105e:::obj535:head local-lis/les=113/114 n=366 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.710+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.710+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9f7e105e:::obj535:head local-lis/les=113/114 n=366 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:a0060720:::obj493:head, version: 111'493, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0060720:::obj493:head@111'493, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.710+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9f7e105e:::obj535:head local-lis/les=113/114 n=366 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a0060720:::obj493:head@111'493, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.710+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.710+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:9f8e934c:::obj456:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.710+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9f7e105e:::obj535:head local-lis/les=113/114 n=366 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:a0060720:::obj493:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.710+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9f7e105e:::obj535:head local-lis/les=113/114 n=366 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:a0060720:::obj493:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.710+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9f7e105e:::obj535:head local-lis/les=113/114 n=366 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:a0060720:::obj493:head 2026-02-11T23:31:27.710+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9f7e105e:::obj535:head local-lis/les=113/114 n=366 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:a0060720:::obj493:head v 111'493 2026-02-11T23:31:27.710+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9f7e105e:::obj535:head local-lis/les=113/114 n=366 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.710+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.710+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a0060720:::obj493:head, version: 111'493, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0060720:::obj493:head@111'493, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.710+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.710+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.710+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:9f8e934c:::obj456:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.710+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:9f8e934c:::obj456:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 116/116 lb 1:9f8e934c:::obj456:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:9f7e105e:::obj535:head local-lis/les=113/114 n=366 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.710+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.710+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9f7e105e:::obj535:head local-lis/les=113/114 n=366 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:9f8e934c:::obj456:head) v3 2026-02-11T23:31:27.710+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9f7e105e:::obj535:head local-lis/les=113/114 n=366 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:9f8e934c:::obj456:head) 2026-02-11T23:31:27.710+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.710+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:9f8e934c:::obj456:head) v3 finish latency 0.000009 2026-02-11T23:31:27.710+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9f8e934c:::obj456:head local-lis/les=113/114 n=367 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.710+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.710+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.710+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9f8e934c:::obj456:head local-lis/les=113/114 n=367 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.710+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.710+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.710+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.710+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fed680 con 0x4acb000 2026-02-11T23:31:27.710+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:a0060720:::obj493:head)]) -- 0x6170900 con 0x4acb000 2026-02-11T23:31:27.710+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.710+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 771 ==== MOSDPGPush(1.0 116/113 [PushOp(1:a0217245:::obj31:head, version: 111'31, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0217245:::obj31:head@111'31, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x559d1e0 con 0x4acb000 2026-02-11T23:31:27.710+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:a0217245:::obj31:head, version: 111'31, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0217245:::obj31:head@111'31, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 116 2026-02-11T23:31:27.710+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a0217245:::obj31:head, version: 111'31, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0217245:::obj31:head@111'31, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.710+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.710+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 772 ==== pg_backfill(progress 1.0 e 116/116 lb 1:a0060720:::obj493:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98000 con 0x4acb000 2026-02-11T23:31:27.710+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a0217245:::obj31:head, version: 111'31, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0217245:::obj31:head@111'31, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.710+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:a0060720:::obj493:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.710+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a0060720:::obj493:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.710+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.710+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a0217245:::obj31:head, version: 111'31, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0217245:::obj31:head@111'31, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.710+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.710+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a0217245:::obj31:head, version: 111'31, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0217245:::obj31:head@111'31, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 116/113 [PushOp(1:a0217245:::obj31:head, version: 111'31, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0217245:::obj31:head@111'31, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:9f8e934c:::obj456:head local-lis/les=113/114 n=367 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.710+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.710+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9f8e934c:::obj456:head local-lis/les=113/114 n=367 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:a0217245:::obj31:head, version: 111'31, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0217245:::obj31:head@111'31, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.710+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9f8e934c:::obj456:head local-lis/les=113/114 n=367 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a0217245:::obj31:head@111'31, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.710+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.710+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a0060720:::obj493:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.710+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9f8e934c:::obj456:head local-lis/les=113/114 n=367 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:a0217245:::obj31:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.710+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9f8e934c:::obj456:head local-lis/les=113/114 n=367 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:a0217245:::obj31:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.710+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9f8e934c:::obj456:head local-lis/les=113/114 n=367 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:a0217245:::obj31:head 2026-02-11T23:31:27.710+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9f8e934c:::obj456:head local-lis/les=113/114 n=367 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:a0217245:::obj31:head v 111'31 2026-02-11T23:31:27.710+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9f8e934c:::obj456:head local-lis/les=113/114 n=367 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.710+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.710+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a0217245:::obj31:head, version: 111'31, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0217245:::obj31:head@111'31, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.710+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.710+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.710+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a0060720:::obj493:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.710+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a0060720:::obj493:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 1.0 e 116/116 lb 1:a0060720:::obj493:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:9f8e934c:::obj456:head local-lis/les=113/114 n=367 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.710+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.710+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9f8e934c:::obj456:head local-lis/les=113/114 n=367 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:a0060720:::obj493:head) v3 2026-02-11T23:31:27.710+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:9f8e934c:::obj456:head local-lis/les=113/114 n=367 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:a0060720:::obj493:head) 2026-02-11T23:31:27.710+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.710+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a0060720:::obj493:head) v3 finish latency 0.000009 2026-02-11T23:31:27.710+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a0060720:::obj493:head local-lis/les=113/114 n=368 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.710+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.710+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.710+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a0060720:::obj493:head local-lis/les=113/114 n=368 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.710+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.710+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.710+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.710+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fed380 con 0x4acb000 2026-02-11T23:31:27.710+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:a0217245:::obj31:head)]) -- 0x6015800 con 0x4acb000 2026-02-11T23:31:27.710+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.711+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 773 ==== MOSDPGPush(1.0 116/113 [PushOp(1:a0e9da24:::obj323:head, version: 111'323, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0e9da24:::obj323:head@111'323, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559cd00 con 0x4acb000 2026-02-11T23:31:27.711+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:a0e9da24:::obj323:head, version: 111'323, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0e9da24:::obj323:head@111'323, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000016 epoch 116 2026-02-11T23:31:27.711+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a0e9da24:::obj323:head, version: 111'323, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0e9da24:::obj323:head@111'323, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.711+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.711+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 774 ==== pg_backfill(progress 1.0 e 116/116 lb 1:a0217245:::obj31:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d98580 con 0x4acb000 2026-02-11T23:31:27.711+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a0e9da24:::obj323:head, version: 111'323, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0e9da24:::obj323:head@111'323, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.711+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:a0217245:::obj31:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.711+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.711+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a0217245:::obj31:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.711+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a0e9da24:::obj323:head, version: 111'323, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0e9da24:::obj323:head@111'323, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.711+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.711+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a0e9da24:::obj323:head, version: 111'323, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0e9da24:::obj323:head@111'323, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000042 MOSDPGPush(1.0 116/113 [PushOp(1:a0e9da24:::obj323:head, version: 111'323, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0e9da24:::obj323:head@111'323, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a0060720:::obj493:head local-lis/les=113/114 n=368 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.711+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.711+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a0060720:::obj493:head local-lis/les=113/114 n=368 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:a0e9da24:::obj323:head, version: 111'323, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0e9da24:::obj323:head@111'323, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.711+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a0060720:::obj493:head local-lis/les=113/114 n=368 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a0e9da24:::obj323:head@111'323, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.711+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.711+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a0217245:::obj31:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.711+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a0060720:::obj493:head local-lis/les=113/114 n=368 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:a0e9da24:::obj323:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.711+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a0060720:::obj493:head local-lis/les=113/114 n=368 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:a0e9da24:::obj323:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.711+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a0060720:::obj493:head local-lis/les=113/114 n=368 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:a0e9da24:::obj323:head 2026-02-11T23:31:27.711+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a0060720:::obj493:head local-lis/les=113/114 n=368 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:a0e9da24:::obj323:head v 111'323 2026-02-11T23:31:27.711+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a0060720:::obj493:head local-lis/les=113/114 n=368 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.711+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.711+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a0e9da24:::obj323:head, version: 111'323, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0e9da24:::obj323:head@111'323, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.711+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.711+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.711+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a0217245:::obj31:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.711+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a0217245:::obj31:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 1.0 e 116/116 lb 1:a0217245:::obj31:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a0060720:::obj493:head local-lis/les=113/114 n=368 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.711+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.711+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a0060720:::obj493:head local-lis/les=113/114 n=368 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:a0217245:::obj31:head) v3 2026-02-11T23:31:27.711+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a0060720:::obj493:head local-lis/les=113/114 n=368 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:a0217245:::obj31:head) 2026-02-11T23:31:27.711+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.711+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a0217245:::obj31:head) v3 finish latency 0.000009 2026-02-11T23:31:27.711+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a0217245:::obj31:head local-lis/les=113/114 n=369 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.711+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.711+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.711+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a0217245:::obj31:head local-lis/les=113/114 n=369 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.711+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.711+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.711+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.711+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x612e900 con 0x4acb000 2026-02-11T23:31:27.711+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:a0e9da24:::obj323:head)]) -- 0x6170f00 con 0x4acb000 2026-02-11T23:31:27.711+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.711+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 775 ==== MOSDPGPush(1.0 116/113 [PushOp(1:a10cad2d:::obj340:head, version: 111'340, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a10cad2d:::obj340:head@111'340, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e261a0 con 0x4acb000 2026-02-11T23:31:27.711+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:a10cad2d:::obj340:head, version: 111'340, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a10cad2d:::obj340:head@111'340, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000016 epoch 116 2026-02-11T23:31:27.711+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a10cad2d:::obj340:head, version: 111'340, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a10cad2d:::obj340:head@111'340, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.711+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.711+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 776 ==== pg_backfill(progress 1.0 e 116/116 lb 1:a0e9da24:::obj323:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98b00 con 0x4acb000 2026-02-11T23:31:27.711+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a10cad2d:::obj340:head, version: 111'340, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a10cad2d:::obj340:head@111'340, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.711+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:a0e9da24:::obj323:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.711+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.711+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a0e9da24:::obj323:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.711+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a10cad2d:::obj340:head, version: 111'340, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a10cad2d:::obj340:head@111'340, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.711+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.711+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a10cad2d:::obj340:head, version: 111'340, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a10cad2d:::obj340:head@111'340, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000043 MOSDPGPush(1.0 116/113 [PushOp(1:a10cad2d:::obj340:head, version: 111'340, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a10cad2d:::obj340:head@111'340, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a0217245:::obj31:head local-lis/les=113/114 n=369 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.711+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.711+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a0217245:::obj31:head local-lis/les=113/114 n=369 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:a10cad2d:::obj340:head, version: 111'340, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a10cad2d:::obj340:head@111'340, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.711+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a0217245:::obj31:head local-lis/les=113/114 n=369 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a10cad2d:::obj340:head@111'340, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.711+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.711+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a0e9da24:::obj323:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.711+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a0217245:::obj31:head local-lis/les=113/114 n=369 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:a10cad2d:::obj340:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.711+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a0217245:::obj31:head local-lis/les=113/114 n=369 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:a10cad2d:::obj340:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.711+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a0217245:::obj31:head local-lis/les=113/114 n=369 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:a10cad2d:::obj340:head 2026-02-11T23:31:27.711+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a0217245:::obj31:head local-lis/les=113/114 n=369 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:a10cad2d:::obj340:head v 111'340 2026-02-11T23:31:27.711+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a0217245:::obj31:head local-lis/les=113/114 n=369 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.711+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.711+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a10cad2d:::obj340:head, version: 111'340, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a10cad2d:::obj340:head@111'340, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.711+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.711+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.711+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a0e9da24:::obj323:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.711+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a0e9da24:::obj323:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 1.0 e 116/116 lb 1:a0e9da24:::obj323:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a0217245:::obj31:head local-lis/les=113/114 n=369 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.711+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.711+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a0217245:::obj31:head local-lis/les=113/114 n=369 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:a0e9da24:::obj323:head) v3 2026-02-11T23:31:27.711+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a0217245:::obj31:head local-lis/les=113/114 n=369 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:a0e9da24:::obj323:head) 2026-02-11T23:31:27.711+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.712+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a0e9da24:::obj323:head) v3 finish latency 0.000013 2026-02-11T23:31:27.712+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a0e9da24:::obj323:head local-lis/les=113/114 n=370 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.712+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.712+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.712+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a0e9da24:::obj323:head local-lis/les=113/114 n=370 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.712+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.712+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.712+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.712+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5ca0a80 con 0x4acb000 2026-02-11T23:31:27.712+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:a10cad2d:::obj340:head)]) -- 0x6171800 con 0x4acb000 2026-02-11T23:31:27.712+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.712+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 777 ==== MOSDPGPush(1.0 116/113 [PushOp(1:a16a9a6a:::obj427:head, version: 111'427, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a16a9a6a:::obj427:head@111'427, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f1d40 con 0x4acb000 2026-02-11T23:31:27.712+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:a16a9a6a:::obj427:head, version: 111'427, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a16a9a6a:::obj427:head@111'427, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000016 epoch 116 2026-02-11T23:31:27.712+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a16a9a6a:::obj427:head, version: 111'427, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a16a9a6a:::obj427:head@111'427, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.712+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.712+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 778 ==== pg_backfill(progress 1.0 e 116/116 lb 1:a10cad2d:::obj340:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99080 con 0x4acb000 2026-02-11T23:31:27.712+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a16a9a6a:::obj427:head, version: 111'427, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a16a9a6a:::obj427:head@111'427, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.712+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:a10cad2d:::obj340:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.712+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.712+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a10cad2d:::obj340:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.712+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a16a9a6a:::obj427:head, version: 111'427, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a16a9a6a:::obj427:head@111'427, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.712+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.712+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a16a9a6a:::obj427:head, version: 111'427, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a16a9a6a:::obj427:head@111'427, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000042 MOSDPGPush(1.0 116/113 [PushOp(1:a16a9a6a:::obj427:head, version: 111'427, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a16a9a6a:::obj427:head@111'427, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a0e9da24:::obj323:head local-lis/les=113/114 n=370 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.712+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.712+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a0e9da24:::obj323:head local-lis/les=113/114 n=370 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:a16a9a6a:::obj427:head, version: 111'427, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a16a9a6a:::obj427:head@111'427, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.712+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a0e9da24:::obj323:head local-lis/les=113/114 n=370 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a16a9a6a:::obj427:head@111'427, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.712+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.712+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a10cad2d:::obj340:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.712+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a0e9da24:::obj323:head local-lis/les=113/114 n=370 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:a16a9a6a:::obj427:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.712+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a0e9da24:::obj323:head local-lis/les=113/114 n=370 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:a16a9a6a:::obj427:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.712+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a0e9da24:::obj323:head local-lis/les=113/114 n=370 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:a16a9a6a:::obj427:head 2026-02-11T23:31:27.712+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a0e9da24:::obj323:head local-lis/les=113/114 n=370 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:a16a9a6a:::obj427:head v 111'427 2026-02-11T23:31:27.712+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a0e9da24:::obj323:head local-lis/les=113/114 n=370 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.712+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.712+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a16a9a6a:::obj427:head, version: 111'427, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a16a9a6a:::obj427:head@111'427, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000043 2026-02-11T23:31:27.712+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.712+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.712+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a10cad2d:::obj340:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.712+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a10cad2d:::obj340:head) v3 prio 15 cost 0 latency 0.000058 pg_backfill(progress 1.0 e 116/116 lb 1:a10cad2d:::obj340:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a0e9da24:::obj323:head local-lis/les=113/114 n=370 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.712+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.712+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a0e9da24:::obj323:head local-lis/les=113/114 n=370 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:a10cad2d:::obj340:head) v3 2026-02-11T23:31:27.712+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a0e9da24:::obj323:head local-lis/les=113/114 n=370 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:a10cad2d:::obj340:head) 2026-02-11T23:31:27.712+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.712+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a10cad2d:::obj340:head) v3 finish latency 0.000010 2026-02-11T23:31:27.712+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a10cad2d:::obj340:head local-lis/les=113/114 n=371 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.712+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.712+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.712+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a10cad2d:::obj340:head local-lis/les=113/114 n=371 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.712+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.712+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.712+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.712+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x612e600 con 0x4acb000 2026-02-11T23:31:27.712+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:a16a9a6a:::obj427:head)]) -- 0x6171680 con 0x4acb000 2026-02-11T23:31:27.712+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.713+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 779 ==== MOSDPGPush(1.0 116/113 [PushOp(1:a16dc95c:::obj269:head, version: 111'269, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a16dc95c:::obj269:head@111'269, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f0b60 con 0x4acb000 2026-02-11T23:31:27.713+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:a16dc95c:::obj269:head, version: 111'269, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a16dc95c:::obj269:head@111'269, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 116 2026-02-11T23:31:27.713+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a16dc95c:::obj269:head, version: 111'269, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a16dc95c:::obj269:head@111'269, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.713+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.713+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 780 ==== pg_backfill(progress 1.0 e 116/116 lb 1:a16a9a6a:::obj427:head) ==== 1062+0+0 (unknown 0 0 0) 0x4a2cb00 con 0x4acb000 2026-02-11T23:31:27.713+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a16dc95c:::obj269:head, version: 111'269, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a16dc95c:::obj269:head@111'269, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.713+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:a16a9a6a:::obj427:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.713+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a16a9a6a:::obj427:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.713+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.713+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a16dc95c:::obj269:head, version: 111'269, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a16dc95c:::obj269:head@111'269, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.713+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.713+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a16dc95c:::obj269:head, version: 111'269, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a16dc95c:::obj269:head@111'269, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(1.0 116/113 [PushOp(1:a16dc95c:::obj269:head, version: 111'269, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a16dc95c:::obj269:head@111'269, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a10cad2d:::obj340:head local-lis/les=113/114 n=371 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.713+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.713+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a10cad2d:::obj340:head local-lis/les=113/114 n=371 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:a16dc95c:::obj269:head, version: 111'269, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a16dc95c:::obj269:head@111'269, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.713+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a10cad2d:::obj340:head local-lis/les=113/114 n=371 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a16dc95c:::obj269:head@111'269, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.713+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.713+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a16a9a6a:::obj427:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.713+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a10cad2d:::obj340:head local-lis/les=113/114 n=371 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:a16dc95c:::obj269:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.713+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a10cad2d:::obj340:head local-lis/les=113/114 n=371 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:a16dc95c:::obj269:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.713+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a10cad2d:::obj340:head local-lis/les=113/114 n=371 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:a16dc95c:::obj269:head 2026-02-11T23:31:27.713+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a10cad2d:::obj340:head local-lis/les=113/114 n=371 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:a16dc95c:::obj269:head v 111'269 2026-02-11T23:31:27.713+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a10cad2d:::obj340:head local-lis/les=113/114 n=371 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.713+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.713+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a16dc95c:::obj269:head, version: 111'269, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a16dc95c:::obj269:head@111'269, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000043 2026-02-11T23:31:27.713+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.713+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.713+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a16a9a6a:::obj427:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.713+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a16a9a6a:::obj427:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 1.0 e 116/116 lb 1:a16a9a6a:::obj427:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a10cad2d:::obj340:head local-lis/les=113/114 n=371 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.713+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.713+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a10cad2d:::obj340:head local-lis/les=113/114 n=371 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:a16a9a6a:::obj427:head) v3 2026-02-11T23:31:27.713+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a10cad2d:::obj340:head local-lis/les=113/114 n=371 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:a16a9a6a:::obj427:head) 2026-02-11T23:31:27.713+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.713+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a16a9a6a:::obj427:head) v3 finish latency 0.000010 2026-02-11T23:31:27.713+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a16a9a6a:::obj427:head local-lis/les=113/114 n=372 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.713+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.713+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.713+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a16a9a6a:::obj427:head local-lis/les=113/114 n=372 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.713+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.713+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.713+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.713+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d1de00 con 0x4acb000 2026-02-11T23:31:27.713+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:a16dc95c:::obj269:head)]) -- 0x6171380 con 0x4acb000 2026-02-11T23:31:27.713+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.713+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 781 ==== MOSDPGPush(1.0 116/113 [PushOp(1:a22120c4:::obj223:head, version: 111'223, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a22120c4:::obj223:head@111'223, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f16c0 con 0x4acb000 2026-02-11T23:31:27.713+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:a22120c4:::obj223:head, version: 111'223, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a22120c4:::obj223:head@111'223, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 116 2026-02-11T23:31:27.713+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a22120c4:::obj223:head, version: 111'223, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a22120c4:::obj223:head@111'223, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.713+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.713+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 782 ==== pg_backfill(progress 1.0 e 116/116 lb 1:a16dc95c:::obj269:head) ==== 1062+0+0 (unknown 0 0 0) 0x5982b00 con 0x4acb000 2026-02-11T23:31:27.713+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a22120c4:::obj223:head, version: 111'223, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a22120c4:::obj223:head@111'223, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.713+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:a16dc95c:::obj269:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.713+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.713+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a16dc95c:::obj269:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.713+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a22120c4:::obj223:head, version: 111'223, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a22120c4:::obj223:head@111'223, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.713+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.713+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a22120c4:::obj223:head, version: 111'223, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a22120c4:::obj223:head@111'223, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(1.0 116/113 [PushOp(1:a22120c4:::obj223:head, version: 111'223, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a22120c4:::obj223:head@111'223, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a16a9a6a:::obj427:head local-lis/les=113/114 n=372 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.713+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.713+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a16a9a6a:::obj427:head local-lis/les=113/114 n=372 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:a22120c4:::obj223:head, version: 111'223, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a22120c4:::obj223:head@111'223, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.713+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a16a9a6a:::obj427:head local-lis/les=113/114 n=372 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a22120c4:::obj223:head@111'223, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.713+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.713+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a16dc95c:::obj269:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.713+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a16a9a6a:::obj427:head local-lis/les=113/114 n=372 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:a22120c4:::obj223:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.713+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a16a9a6a:::obj427:head local-lis/les=113/114 n=372 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:a22120c4:::obj223:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.713+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a16a9a6a:::obj427:head local-lis/les=113/114 n=372 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:a22120c4:::obj223:head 2026-02-11T23:31:27.713+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a16a9a6a:::obj427:head local-lis/les=113/114 n=372 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:a22120c4:::obj223:head v 111'223 2026-02-11T23:31:27.713+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a16a9a6a:::obj427:head local-lis/les=113/114 n=372 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.713+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.713+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a22120c4:::obj223:head, version: 111'223, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a22120c4:::obj223:head@111'223, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.713+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.713+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.713+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a16dc95c:::obj269:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.713+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a16dc95c:::obj269:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 1.0 e 116/116 lb 1:a16dc95c:::obj269:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a16a9a6a:::obj427:head local-lis/les=113/114 n=372 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.713+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.713+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a16a9a6a:::obj427:head local-lis/les=113/114 n=372 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:a16dc95c:::obj269:head) v3 2026-02-11T23:31:27.713+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a16a9a6a:::obj427:head local-lis/les=113/114 n=372 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:a16dc95c:::obj269:head) 2026-02-11T23:31:27.713+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.713+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a16dc95c:::obj269:head) v3 finish latency 0.000009 2026-02-11T23:31:27.713+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a16dc95c:::obj269:head local-lis/les=113/114 n=373 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.713+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.713+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.713+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a16dc95c:::obj269:head local-lis/les=113/114 n=373 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.713+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.713+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.713+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.713+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x612e300 con 0x4acb000 2026-02-11T23:31:27.713+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:a22120c4:::obj223:head)]) -- 0x6171080 con 0x4acb000 2026-02-11T23:31:27.713+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.714+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 783 ==== MOSDPGPush(1.0 116/113 [PushOp(1:a25cd2bd:::obj42:head, version: 111'42, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a25cd2bd:::obj42:head@111'42, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x559c680 con 0x4acb000 2026-02-11T23:31:27.714+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:a25cd2bd:::obj42:head, version: 111'42, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a25cd2bd:::obj42:head@111'42, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 116 2026-02-11T23:31:27.714+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a25cd2bd:::obj42:head, version: 111'42, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a25cd2bd:::obj42:head@111'42, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.714+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.714+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 784 ==== pg_backfill(progress 1.0 e 116/116 lb 1:a22120c4:::obj223:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2d080 con 0x4acb000 2026-02-11T23:31:27.714+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:a22120c4:::obj223:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.714+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a25cd2bd:::obj42:head, version: 111'42, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a25cd2bd:::obj42:head@111'42, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.714+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a22120c4:::obj223:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.714+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.714+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a25cd2bd:::obj42:head, version: 111'42, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a25cd2bd:::obj42:head@111'42, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.714+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a25cd2bd:::obj42:head, version: 111'42, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a25cd2bd:::obj42:head@111'42, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 116/113 [PushOp(1:a25cd2bd:::obj42:head, version: 111'42, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a25cd2bd:::obj42:head@111'42, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a16dc95c:::obj269:head local-lis/les=113/114 n=373 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.714+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.714+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.714+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a16dc95c:::obj269:head local-lis/les=113/114 n=373 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:a25cd2bd:::obj42:head, version: 111'42, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a25cd2bd:::obj42:head@111'42, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.714+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a22120c4:::obj223:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.714+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a16dc95c:::obj269:head local-lis/les=113/114 n=373 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a25cd2bd:::obj42:head@111'42, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.714+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a16dc95c:::obj269:head local-lis/les=113/114 n=373 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:a25cd2bd:::obj42:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.714+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a16dc95c:::obj269:head local-lis/les=113/114 n=373 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:a25cd2bd:::obj42:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.714+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a16dc95c:::obj269:head local-lis/les=113/114 n=373 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:a25cd2bd:::obj42:head 2026-02-11T23:31:27.714+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a16dc95c:::obj269:head local-lis/les=113/114 n=373 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:a25cd2bd:::obj42:head v 111'42 2026-02-11T23:31:27.714+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a16dc95c:::obj269:head local-lis/les=113/114 n=373 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.714+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.714+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a25cd2bd:::obj42:head, version: 111'42, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a25cd2bd:::obj42:head@111'42, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.714+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.714+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.714+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a22120c4:::obj223:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.714+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a22120c4:::obj223:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 1.0 e 116/116 lb 1:a22120c4:::obj223:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a16dc95c:::obj269:head local-lis/les=113/114 n=373 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.714+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.714+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a16dc95c:::obj269:head local-lis/les=113/114 n=373 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:a22120c4:::obj223:head) v3 2026-02-11T23:31:27.714+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a16dc95c:::obj269:head local-lis/les=113/114 n=373 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:a22120c4:::obj223:head) 2026-02-11T23:31:27.714+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.714+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a22120c4:::obj223:head) v3 finish latency 0.000010 2026-02-11T23:31:27.714+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a22120c4:::obj223:head local-lis/les=113/114 n=374 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.714+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.714+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.714+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a22120c4:::obj223:head local-lis/les=113/114 n=374 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.714+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.714+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.714+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.714+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5ca0d80 con 0x4acb000 2026-02-11T23:31:27.714+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:a25cd2bd:::obj42:head)]) -- 0x614c780 con 0x4acb000 2026-02-11T23:31:27.714+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.714+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 785 ==== MOSDPGPush(1.0 116/113 [PushOp(1:a2bd80aa:::obj577:head, version: 111'577, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a2bd80aa:::obj577:head@111'577, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559dba0 con 0x4acb000 2026-02-11T23:31:27.714+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:a2bd80aa:::obj577:head, version: 111'577, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a2bd80aa:::obj577:head@111'577, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 116 2026-02-11T23:31:27.714+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a2bd80aa:::obj577:head, version: 111'577, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a2bd80aa:::obj577:head@111'577, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.714+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.714+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 786 ==== pg_backfill(progress 1.0 e 116/116 lb 1:a25cd2bd:::obj42:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d2e680 con 0x4acb000 2026-02-11T23:31:27.714+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:a25cd2bd:::obj42:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.714+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a2bd80aa:::obj577:head, version: 111'577, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a2bd80aa:::obj577:head@111'577, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.714+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a25cd2bd:::obj42:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.714+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.714+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a25cd2bd:::obj42:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.714+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.714+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a2bd80aa:::obj577:head, version: 111'577, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a2bd80aa:::obj577:head@111'577, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.714+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a2bd80aa:::obj577:head, version: 111'577, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a2bd80aa:::obj577:head@111'577, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000050 MOSDPGPush(1.0 116/113 [PushOp(1:a2bd80aa:::obj577:head, version: 111'577, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a2bd80aa:::obj577:head@111'577, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a22120c4:::obj223:head local-lis/les=113/114 n=374 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.714+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.714+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a22120c4:::obj223:head local-lis/les=113/114 n=374 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:a2bd80aa:::obj577:head, version: 111'577, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a2bd80aa:::obj577:head@111'577, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.714+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a22120c4:::obj223:head local-lis/les=113/114 n=374 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a2bd80aa:::obj577:head@111'577, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.714+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a22120c4:::obj223:head local-lis/les=113/114 n=374 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:a2bd80aa:::obj577:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.714+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a22120c4:::obj223:head local-lis/les=113/114 n=374 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:a2bd80aa:::obj577:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.714+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a22120c4:::obj223:head local-lis/les=113/114 n=374 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:a2bd80aa:::obj577:head 2026-02-11T23:31:27.714+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a22120c4:::obj223:head local-lis/les=113/114 n=374 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:a2bd80aa:::obj577:head v 111'577 2026-02-11T23:31:27.714+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a22120c4:::obj223:head local-lis/les=113/114 n=374 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.714+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.714+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a2bd80aa:::obj577:head, version: 111'577, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a2bd80aa:::obj577:head@111'577, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:27.714+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a22120c4:::obj223:head local-lis/les=113/114 n=374 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.714+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.714+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.714+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a25cd2bd:::obj42:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.714+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a25cd2bd:::obj42:head) v3 prio 15 cost 0 latency 0.000090 pg_backfill(progress 1.0 e 116/116 lb 1:a25cd2bd:::obj42:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a22120c4:::obj223:head local-lis/les=113/114 n=374 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.714+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.714+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a22120c4:::obj223:head local-lis/les=113/114 n=374 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:a25cd2bd:::obj42:head) v3 2026-02-11T23:31:27.714+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a22120c4:::obj223:head local-lis/les=113/114 n=374 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:a25cd2bd:::obj42:head) 2026-02-11T23:31:27.714+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.714+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a25cd2bd:::obj42:head) v3 finish latency 0.000011 2026-02-11T23:31:27.714+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.715+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.715+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a25cd2bd:::obj42:head local-lis/les=113/114 n=375 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.715+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.715+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.715+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.715+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5ca1c80 con 0x4acb000 2026-02-11T23:31:27.715+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:a2bd80aa:::obj577:head)]) -- 0x612e000 con 0x4acb000 2026-02-11T23:31:27.715+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.715+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 787 ==== MOSDPGPush(1.0 116/113 [PushOp(1:a3278361:::obj425:head, version: 111'425, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3278361:::obj425:head@111'425, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559d040 con 0x4acb000 2026-02-11T23:31:27.715+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:a3278361:::obj425:head, version: 111'425, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3278361:::obj425:head@111'425, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 116 2026-02-11T23:31:27.715+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a3278361:::obj425:head, version: 111'425, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3278361:::obj425:head@111'425, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.715+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.715+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 788 ==== pg_backfill(progress 1.0 e 116/116 lb 1:a2bd80aa:::obj577:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e100 con 0x4acb000 2026-02-11T23:31:27.715+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a3278361:::obj425:head, version: 111'425, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3278361:::obj425:head@111'425, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.715+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:a2bd80aa:::obj577:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.715+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a2bd80aa:::obj577:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.715+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.715+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a3278361:::obj425:head, version: 111'425, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3278361:::obj425:head@111'425, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.715+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.715+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a3278361:::obj425:head, version: 111'425, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3278361:::obj425:head@111'425, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(1.0 116/113 [PushOp(1:a3278361:::obj425:head, version: 111'425, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3278361:::obj425:head@111'425, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a25cd2bd:::obj42:head local-lis/les=113/114 n=375 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.715+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.715+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a25cd2bd:::obj42:head local-lis/les=113/114 n=375 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:a3278361:::obj425:head, version: 111'425, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3278361:::obj425:head@111'425, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.715+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a25cd2bd:::obj42:head local-lis/les=113/114 n=375 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a3278361:::obj425:head@111'425, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.715+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.715+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a2bd80aa:::obj577:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.715+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a25cd2bd:::obj42:head local-lis/les=113/114 n=375 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:a3278361:::obj425:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.715+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a25cd2bd:::obj42:head local-lis/les=113/114 n=375 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:a3278361:::obj425:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.715+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a25cd2bd:::obj42:head local-lis/les=113/114 n=375 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:a3278361:::obj425:head 2026-02-11T23:31:27.715+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a25cd2bd:::obj42:head local-lis/les=113/114 n=375 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:a3278361:::obj425:head v 111'425 2026-02-11T23:31:27.715+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a25cd2bd:::obj42:head local-lis/les=113/114 n=375 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.715+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.715+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a3278361:::obj425:head, version: 111'425, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3278361:::obj425:head@111'425, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000044 2026-02-11T23:31:27.715+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.715+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.715+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a2bd80aa:::obj577:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.715+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a2bd80aa:::obj577:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 116/116 lb 1:a2bd80aa:::obj577:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a25cd2bd:::obj42:head local-lis/les=113/114 n=375 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.715+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.715+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a25cd2bd:::obj42:head local-lis/les=113/114 n=375 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:a2bd80aa:::obj577:head) v3 2026-02-11T23:31:27.715+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a25cd2bd:::obj42:head local-lis/les=113/114 n=375 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:a2bd80aa:::obj577:head) 2026-02-11T23:31:27.715+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.715+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a2bd80aa:::obj577:head) v3 finish latency 0.000013 2026-02-11T23:31:27.715+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a2bd80aa:::obj577:head local-lis/les=113/114 n=376 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.715+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.715+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.715+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a2bd80aa:::obj577:head local-lis/les=113/114 n=376 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.715+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.715+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.715+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.715+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d1d680 con 0x4acb000 2026-02-11T23:31:27.715+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:a3278361:::obj425:head)]) -- 0x5ca0180 con 0x4acb000 2026-02-11T23:31:27.715+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.716+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 789 ==== MOSDPGPush(1.0 116/113 [PushOp(1:a3411a0c:::obj160:head, version: 111'160, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3411a0c:::obj160:head@111'160, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d89c0 con 0x4acb000 2026-02-11T23:31:27.716+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:a3411a0c:::obj160:head, version: 111'160, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3411a0c:::obj160:head@111'160, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 116 2026-02-11T23:31:27.716+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a3411a0c:::obj160:head, version: 111'160, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3411a0c:::obj160:head@111'160, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.716+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.716+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 790 ==== pg_backfill(progress 1.0 e 116/116 lb 1:a3278361:::obj425:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2c000 con 0x4acb000 2026-02-11T23:31:27.716+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a3411a0c:::obj160:head, version: 111'160, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3411a0c:::obj160:head@111'160, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.716+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:a3278361:::obj425:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.716+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a3278361:::obj425:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.716+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.716+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a3411a0c:::obj160:head, version: 111'160, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3411a0c:::obj160:head@111'160, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.716+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.716+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a3411a0c:::obj160:head, version: 111'160, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3411a0c:::obj160:head@111'160, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(1.0 116/113 [PushOp(1:a3411a0c:::obj160:head, version: 111'160, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3411a0c:::obj160:head@111'160, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a2bd80aa:::obj577:head local-lis/les=113/114 n=376 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.716+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.716+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a2bd80aa:::obj577:head local-lis/les=113/114 n=376 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:a3411a0c:::obj160:head, version: 111'160, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3411a0c:::obj160:head@111'160, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.716+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a2bd80aa:::obj577:head local-lis/les=113/114 n=376 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a3411a0c:::obj160:head@111'160, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.716+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.716+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a3278361:::obj425:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.716+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a2bd80aa:::obj577:head local-lis/les=113/114 n=376 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:a3411a0c:::obj160:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.716+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a2bd80aa:::obj577:head local-lis/les=113/114 n=376 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:a3411a0c:::obj160:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.716+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a2bd80aa:::obj577:head local-lis/les=113/114 n=376 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:a3411a0c:::obj160:head 2026-02-11T23:31:27.716+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a2bd80aa:::obj577:head local-lis/les=113/114 n=376 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:a3411a0c:::obj160:head v 111'160 2026-02-11T23:31:27.716+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a2bd80aa:::obj577:head local-lis/les=113/114 n=376 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.716+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.716+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a3411a0c:::obj160:head, version: 111'160, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3411a0c:::obj160:head@111'160, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.716+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.716+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.716+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a3278361:::obj425:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.716+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a3278361:::obj425:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 116/116 lb 1:a3278361:::obj425:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a2bd80aa:::obj577:head local-lis/les=113/114 n=376 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.716+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.716+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a2bd80aa:::obj577:head local-lis/les=113/114 n=376 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:a3278361:::obj425:head) v3 2026-02-11T23:31:27.716+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a2bd80aa:::obj577:head local-lis/les=113/114 n=376 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:a3278361:::obj425:head) 2026-02-11T23:31:27.716+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.716+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a3278361:::obj425:head) v3 finish latency 0.000011 2026-02-11T23:31:27.716+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a3278361:::obj425:head local-lis/les=113/114 n=377 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.716+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.716+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.716+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a3278361:::obj425:head local-lis/les=113/114 n=377 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.716+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.716+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.716+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.716+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5ca1980 con 0x4acb000 2026-02-11T23:31:27.716+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:a3411a0c:::obj160:head)]) -- 0x612ef00 con 0x4acb000 2026-02-11T23:31:27.716+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.716+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 791 ==== MOSDPGPush(1.0 116/113 [PushOp(1:a37ff878:::obj107:head, version: 111'107, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a37ff878:::obj107:head@111'107, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56cb1e0 con 0x4acb000 2026-02-11T23:31:27.716+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:a37ff878:::obj107:head, version: 111'107, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a37ff878:::obj107:head@111'107, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.716+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a37ff878:::obj107:head, version: 111'107, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a37ff878:::obj107:head@111'107, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.716+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.716+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 792 ==== pg_backfill(progress 1.0 e 116/116 lb 1:a3411a0c:::obj160:head) ==== 1062+0+0 (unknown 0 0 0) 0x6179700 con 0x4acb000 2026-02-11T23:31:27.716+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a37ff878:::obj107:head, version: 111'107, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a37ff878:::obj107:head@111'107, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.716+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:a3411a0c:::obj160:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.716+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a3411a0c:::obj160:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.716+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.716+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a37ff878:::obj107:head, version: 111'107, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a37ff878:::obj107:head@111'107, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.716+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.716+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a37ff878:::obj107:head, version: 111'107, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a37ff878:::obj107:head@111'107, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 116/113 [PushOp(1:a37ff878:::obj107:head, version: 111'107, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a37ff878:::obj107:head@111'107, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a3278361:::obj425:head local-lis/les=113/114 n=377 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.716+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.716+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a3278361:::obj425:head local-lis/les=113/114 n=377 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:a37ff878:::obj107:head, version: 111'107, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a37ff878:::obj107:head@111'107, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.716+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.716+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a3278361:::obj425:head local-lis/les=113/114 n=377 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a37ff878:::obj107:head@111'107, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.716+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a3411a0c:::obj160:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.716+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a3278361:::obj425:head local-lis/les=113/114 n=377 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:a37ff878:::obj107:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.716+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a3278361:::obj425:head local-lis/les=113/114 n=377 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:a37ff878:::obj107:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.716+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a3278361:::obj425:head local-lis/les=113/114 n=377 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:a37ff878:::obj107:head 2026-02-11T23:31:27.716+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a3278361:::obj425:head local-lis/les=113/114 n=377 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:a37ff878:::obj107:head v 111'107 2026-02-11T23:31:27.716+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a3278361:::obj425:head local-lis/les=113/114 n=377 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.716+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.716+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a37ff878:::obj107:head, version: 111'107, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a37ff878:::obj107:head@111'107, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.716+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.716+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.716+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a3411a0c:::obj160:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.716+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a3411a0c:::obj160:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:a3411a0c:::obj160:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a3278361:::obj425:head local-lis/les=113/114 n=377 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.716+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.716+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a3278361:::obj425:head local-lis/les=113/114 n=377 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:a3411a0c:::obj160:head) v3 2026-02-11T23:31:27.716+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a3278361:::obj425:head local-lis/les=113/114 n=377 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:a3411a0c:::obj160:head) 2026-02-11T23:31:27.716+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.716+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a3411a0c:::obj160:head) v3 finish latency 0.000011 2026-02-11T23:31:27.716+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a3411a0c:::obj160:head local-lis/les=113/114 n=378 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.716+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.716+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.716+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a3411a0c:::obj160:head local-lis/les=113/114 n=378 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.716+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.716+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.716+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.716+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fecf00 con 0x4acb000 2026-02-11T23:31:27.716+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:a37ff878:::obj107:head)]) -- 0x5fce180 con 0x4acb000 2026-02-11T23:31:27.716+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.717+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 793 ==== MOSDPGPush(1.0 116/113 [PushOp(1:a383681b:::obj41:head, version: 111'41, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a383681b:::obj41:head@111'41, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x55d9520 con 0x4acb000 2026-02-11T23:31:27.717+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:a383681b:::obj41:head, version: 111'41, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a383681b:::obj41:head@111'41, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000016 epoch 116 2026-02-11T23:31:27.717+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a383681b:::obj41:head, version: 111'41, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a383681b:::obj41:head@111'41, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.717+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.717+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 794 ==== pg_backfill(progress 1.0 e 116/116 lb 1:a37ff878:::obj107:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176000 con 0x4acb000 2026-02-11T23:31:27.717+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a383681b:::obj41:head, version: 111'41, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a383681b:::obj41:head@111'41, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.717+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:a37ff878:::obj107:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.717+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a37ff878:::obj107:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.717+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.717+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a383681b:::obj41:head, version: 111'41, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a383681b:::obj41:head@111'41, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.717+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.717+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a383681b:::obj41:head, version: 111'41, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a383681b:::obj41:head@111'41, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000042 MOSDPGPush(1.0 116/113 [PushOp(1:a383681b:::obj41:head, version: 111'41, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a383681b:::obj41:head@111'41, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a3411a0c:::obj160:head local-lis/les=113/114 n=378 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.717+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.717+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a3411a0c:::obj160:head local-lis/les=113/114 n=378 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:a383681b:::obj41:head, version: 111'41, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a383681b:::obj41:head@111'41, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.717+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a3411a0c:::obj160:head local-lis/les=113/114 n=378 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a383681b:::obj41:head@111'41, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.717+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.717+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a37ff878:::obj107:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.717+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a3411a0c:::obj160:head local-lis/les=113/114 n=378 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:a383681b:::obj41:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.717+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a3411a0c:::obj160:head local-lis/les=113/114 n=378 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:a383681b:::obj41:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.717+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a3411a0c:::obj160:head local-lis/les=113/114 n=378 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:a383681b:::obj41:head 2026-02-11T23:31:27.717+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a3411a0c:::obj160:head local-lis/les=113/114 n=378 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:a383681b:::obj41:head v 111'41 2026-02-11T23:31:27.717+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a3411a0c:::obj160:head local-lis/les=113/114 n=378 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.717+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.717+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a383681b:::obj41:head, version: 111'41, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a383681b:::obj41:head@111'41, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.717+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.717+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.717+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a37ff878:::obj107:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.717+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a37ff878:::obj107:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 1.0 e 116/116 lb 1:a37ff878:::obj107:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a3411a0c:::obj160:head local-lis/les=113/114 n=378 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.717+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.717+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a3411a0c:::obj160:head local-lis/les=113/114 n=378 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:a37ff878:::obj107:head) v3 2026-02-11T23:31:27.717+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a3411a0c:::obj160:head local-lis/les=113/114 n=378 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:a37ff878:::obj107:head) 2026-02-11T23:31:27.717+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.717+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a37ff878:::obj107:head) v3 finish latency 0.000008 2026-02-11T23:31:27.717+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a37ff878:::obj107:head local-lis/les=113/114 n=379 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.717+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.717+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.717+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a37ff878:::obj107:head local-lis/les=113/114 n=379 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.717+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.717+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.717+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.717+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5ca1680 con 0x4acb000 2026-02-11T23:31:27.717+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:a383681b:::obj41:head)]) -- 0x612f200 con 0x4acb000 2026-02-11T23:31:27.717+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.717+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 795 ==== MOSDPGPush(1.0 116/113 [PushOp(1:a39a40f0:::obj561:head, version: 111'561, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a39a40f0:::obj561:head@111'561, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d9040 con 0x4acb000 2026-02-11T23:31:27.717+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:a39a40f0:::obj561:head, version: 111'561, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a39a40f0:::obj561:head@111'561, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000016 epoch 116 2026-02-11T23:31:27.717+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a39a40f0:::obj561:head, version: 111'561, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a39a40f0:::obj561:head@111'561, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.717+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.717+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 796 ==== pg_backfill(progress 1.0 e 116/116 lb 1:a383681b:::obj41:head) ==== 1061+0+0 (unknown 0 0 0) 0x6176b00 con 0x4acb000 2026-02-11T23:31:27.717+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a39a40f0:::obj561:head, version: 111'561, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a39a40f0:::obj561:head@111'561, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.717+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:a383681b:::obj41:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.717+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.717+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a39a40f0:::obj561:head, version: 111'561, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a39a40f0:::obj561:head@111'561, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.717+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a383681b:::obj41:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.717+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.717+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a383681b:::obj41:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.717+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a39a40f0:::obj561:head, version: 111'561, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a39a40f0:::obj561:head@111'561, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000044 MOSDPGPush(1.0 116/113 [PushOp(1:a39a40f0:::obj561:head, version: 111'561, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a39a40f0:::obj561:head@111'561, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a37ff878:::obj107:head local-lis/les=113/114 n=379 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.717+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.717+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a37ff878:::obj107:head local-lis/les=113/114 n=379 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:a39a40f0:::obj561:head, version: 111'561, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a39a40f0:::obj561:head@111'561, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.717+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a37ff878:::obj107:head local-lis/les=113/114 n=379 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a39a40f0:::obj561:head@111'561, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.717+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a37ff878:::obj107:head local-lis/les=113/114 n=379 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:a39a40f0:::obj561:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.717+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a37ff878:::obj107:head local-lis/les=113/114 n=379 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:a39a40f0:::obj561:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.717+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a37ff878:::obj107:head local-lis/les=113/114 n=379 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:a39a40f0:::obj561:head 2026-02-11T23:31:27.717+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a37ff878:::obj107:head local-lis/les=113/114 n=379 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:a39a40f0:::obj561:head v 111'561 2026-02-11T23:31:27.717+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a37ff878:::obj107:head local-lis/les=113/114 n=379 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.717+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.717+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a39a40f0:::obj561:head, version: 111'561, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a39a40f0:::obj561:head@111'561, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.717+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.717+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.717+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a383681b:::obj41:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.717+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a383681b:::obj41:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:a383681b:::obj41:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a37ff878:::obj107:head local-lis/les=113/114 n=379 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.717+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.717+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a37ff878:::obj107:head local-lis/les=113/114 n=379 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:a383681b:::obj41:head) v3 2026-02-11T23:31:27.717+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a37ff878:::obj107:head local-lis/les=113/114 n=379 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:a383681b:::obj41:head) 2026-02-11T23:31:27.717+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.717+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a383681b:::obj41:head) v3 finish latency 0.000010 2026-02-11T23:31:27.717+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a383681b:::obj41:head local-lis/les=113/114 n=380 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.717+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.718+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.718+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a383681b:::obj41:head local-lis/les=113/114 n=380 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.718+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.718+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.718+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.718+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d1d800 con 0x4acb000 2026-02-11T23:31:27.718+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:a39a40f0:::obj561:head)]) -- 0x6004f00 con 0x4acb000 2026-02-11T23:31:27.718+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.718+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 797 ==== MOSDPGPush(1.0 116/113 [PushOp(1:a3cf8cd5:::obj363:head, version: 111'363, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3cf8cd5:::obj363:head@111'363, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d8b60 con 0x4acb000 2026-02-11T23:31:27.718+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:a3cf8cd5:::obj363:head, version: 111'363, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3cf8cd5:::obj363:head@111'363, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 116 2026-02-11T23:31:27.718+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a3cf8cd5:::obj363:head, version: 111'363, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3cf8cd5:::obj363:head@111'363, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.718+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.718+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 798 ==== pg_backfill(progress 1.0 e 116/116 lb 1:a39a40f0:::obj561:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177080 con 0x4acb000 2026-02-11T23:31:27.718+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:a39a40f0:::obj561:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.718+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a3cf8cd5:::obj363:head, version: 111'363, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3cf8cd5:::obj363:head@111'363, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.718+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a39a40f0:::obj561:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.718+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.718+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a3cf8cd5:::obj363:head, version: 111'363, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3cf8cd5:::obj363:head@111'363, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.718+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a3cf8cd5:::obj363:head, version: 111'363, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3cf8cd5:::obj363:head@111'363, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 116/113 [PushOp(1:a3cf8cd5:::obj363:head, version: 111'363, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3cf8cd5:::obj363:head@111'363, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a383681b:::obj41:head local-lis/les=113/114 n=380 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.718+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.718+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.718+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a383681b:::obj41:head local-lis/les=113/114 n=380 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:a3cf8cd5:::obj363:head, version: 111'363, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3cf8cd5:::obj363:head@111'363, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.718+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a39a40f0:::obj561:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.718+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a383681b:::obj41:head local-lis/les=113/114 n=380 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a3cf8cd5:::obj363:head@111'363, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.718+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a383681b:::obj41:head local-lis/les=113/114 n=380 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:a3cf8cd5:::obj363:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.718+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a383681b:::obj41:head local-lis/les=113/114 n=380 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:a3cf8cd5:::obj363:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.718+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a383681b:::obj41:head local-lis/les=113/114 n=380 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:a3cf8cd5:::obj363:head 2026-02-11T23:31:27.718+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a383681b:::obj41:head local-lis/les=113/114 n=380 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:a3cf8cd5:::obj363:head v 111'363 2026-02-11T23:31:27.718+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a383681b:::obj41:head local-lis/les=113/114 n=380 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.718+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.718+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a3cf8cd5:::obj363:head, version: 111'363, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3cf8cd5:::obj363:head@111'363, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:27.718+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.718+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.718+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a39a40f0:::obj561:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.718+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a39a40f0:::obj561:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 116/116 lb 1:a39a40f0:::obj561:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a383681b:::obj41:head local-lis/les=113/114 n=380 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.718+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.718+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a383681b:::obj41:head local-lis/les=113/114 n=380 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:a39a40f0:::obj561:head) v3 2026-02-11T23:31:27.718+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a383681b:::obj41:head local-lis/les=113/114 n=380 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:a39a40f0:::obj561:head) 2026-02-11T23:31:27.718+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.718+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a39a40f0:::obj561:head) v3 finish latency 0.000009 2026-02-11T23:31:27.718+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a39a40f0:::obj561:head local-lis/les=113/114 n=381 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.718+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.718+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.718+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a39a40f0:::obj561:head local-lis/les=113/114 n=381 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.718+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.718+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.718+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.718+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5ca1500 con 0x4acb000 2026-02-11T23:31:27.718+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:a3cf8cd5:::obj363:head)]) -- 0x612f680 con 0x4acb000 2026-02-11T23:31:27.718+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.718+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 799 ==== MOSDPGPush(1.0 116/113 [PushOp(1:a42b336e:::obj435:head, version: 111'435, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a42b336e:::obj435:head@111'435, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5647040 con 0x4acb000 2026-02-11T23:31:27.718+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:a42b336e:::obj435:head, version: 111'435, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a42b336e:::obj435:head@111'435, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 116 2026-02-11T23:31:27.718+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a42b336e:::obj435:head, version: 111'435, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a42b336e:::obj435:head@111'435, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.719+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.719+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 800 ==== pg_backfill(progress 1.0 e 116/116 lb 1:a3cf8cd5:::obj363:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177600 con 0x4acb000 2026-02-11T23:31:27.719+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a42b336e:::obj435:head, version: 111'435, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a42b336e:::obj435:head@111'435, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.719+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:a3cf8cd5:::obj363:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.719+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.719+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a42b336e:::obj435:head, version: 111'435, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a42b336e:::obj435:head@111'435, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.719+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a3cf8cd5:::obj363:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.719+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.719+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a42b336e:::obj435:head, version: 111'435, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a42b336e:::obj435:head@111'435, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000041 MOSDPGPush(1.0 116/113 [PushOp(1:a42b336e:::obj435:head, version: 111'435, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a42b336e:::obj435:head@111'435, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a39a40f0:::obj561:head local-lis/les=113/114 n=381 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.719+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a3cf8cd5:::obj363:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.719+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.719+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a39a40f0:::obj561:head local-lis/les=113/114 n=381 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:a42b336e:::obj435:head, version: 111'435, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a42b336e:::obj435:head@111'435, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.719+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a39a40f0:::obj561:head local-lis/les=113/114 n=381 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a42b336e:::obj435:head@111'435, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.719+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a39a40f0:::obj561:head local-lis/les=113/114 n=381 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:a42b336e:::obj435:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.719+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a39a40f0:::obj561:head local-lis/les=113/114 n=381 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:a42b336e:::obj435:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.719+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a39a40f0:::obj561:head local-lis/les=113/114 n=381 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:a42b336e:::obj435:head 2026-02-11T23:31:27.719+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a39a40f0:::obj561:head local-lis/les=113/114 n=381 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:a42b336e:::obj435:head v 111'435 2026-02-11T23:31:27.719+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a39a40f0:::obj561:head local-lis/les=113/114 n=381 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.719+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.719+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a42b336e:::obj435:head, version: 111'435, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a42b336e:::obj435:head@111'435, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:27.719+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a39a40f0:::obj561:head local-lis/les=113/114 n=381 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.719+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.719+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.719+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a3cf8cd5:::obj363:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.719+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a3cf8cd5:::obj363:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 1.0 e 116/116 lb 1:a3cf8cd5:::obj363:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a39a40f0:::obj561:head local-lis/les=113/114 n=381 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.719+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.719+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a39a40f0:::obj561:head local-lis/les=113/114 n=381 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:a3cf8cd5:::obj363:head) v3 2026-02-11T23:31:27.719+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a39a40f0:::obj561:head local-lis/les=113/114 n=381 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:a3cf8cd5:::obj363:head) 2026-02-11T23:31:27.719+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.719+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a3cf8cd5:::obj363:head) v3 finish latency 0.000011 2026-02-11T23:31:27.719+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.719+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.719+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a3cf8cd5:::obj363:head local-lis/les=113/114 n=382 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.719+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.719+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.719+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.719+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d1db00 con 0x4acb000 2026-02-11T23:31:27.719+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:a42b336e:::obj435:head)]) -- 0x5fed080 con 0x4acb000 2026-02-11T23:31:27.719+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.719+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 801 ==== MOSDPGPush(1.0 116/113 [PushOp(1:a4383432:::obj559:head, version: 111'559, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4383432:::obj559:head@111'559, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d81a0 con 0x4acb000 2026-02-11T23:31:27.719+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:a4383432:::obj559:head, version: 111'559, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4383432:::obj559:head@111'559, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.719+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a4383432:::obj559:head, version: 111'559, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4383432:::obj559:head@111'559, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.719+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.719+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 802 ==== pg_backfill(progress 1.0 e 116/116 lb 1:a42b336e:::obj435:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177b80 con 0x4acb000 2026-02-11T23:31:27.719+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:a42b336e:::obj435:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.719+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a4383432:::obj559:head, version: 111'559, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4383432:::obj559:head@111'559, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.719+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a42b336e:::obj435:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.719+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.719+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a4383432:::obj559:head, version: 111'559, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4383432:::obj559:head@111'559, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.719+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a4383432:::obj559:head, version: 111'559, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4383432:::obj559:head@111'559, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 116/113 [PushOp(1:a4383432:::obj559:head, version: 111'559, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4383432:::obj559:head@111'559, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a3cf8cd5:::obj363:head local-lis/les=113/114 n=382 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.719+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.719+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a3cf8cd5:::obj363:head local-lis/les=113/114 n=382 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:a4383432:::obj559:head, version: 111'559, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4383432:::obj559:head@111'559, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.719+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.719+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a42b336e:::obj435:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.719+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a3cf8cd5:::obj363:head local-lis/les=113/114 n=382 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a4383432:::obj559:head@111'559, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.719+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a3cf8cd5:::obj363:head local-lis/les=113/114 n=382 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:a4383432:::obj559:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.719+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a3cf8cd5:::obj363:head local-lis/les=113/114 n=382 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:a4383432:::obj559:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.719+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a3cf8cd5:::obj363:head local-lis/les=113/114 n=382 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:a4383432:::obj559:head 2026-02-11T23:31:27.719+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a3cf8cd5:::obj363:head local-lis/les=113/114 n=382 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:a4383432:::obj559:head v 111'559 2026-02-11T23:31:27.719+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a3cf8cd5:::obj363:head local-lis/les=113/114 n=382 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.719+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.719+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a4383432:::obj559:head, version: 111'559, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4383432:::obj559:head@111'559, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.719+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.719+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.719+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a42b336e:::obj435:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.719+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a42b336e:::obj435:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 1.0 e 116/116 lb 1:a42b336e:::obj435:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a3cf8cd5:::obj363:head local-lis/les=113/114 n=382 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.719+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.719+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a3cf8cd5:::obj363:head local-lis/les=113/114 n=382 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:a42b336e:::obj435:head) v3 2026-02-11T23:31:27.719+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a3cf8cd5:::obj363:head local-lis/les=113/114 n=382 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:a42b336e:::obj435:head) 2026-02-11T23:31:27.719+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.719+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a42b336e:::obj435:head) v3 finish latency 0.000011 2026-02-11T23:31:27.719+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a42b336e:::obj435:head local-lis/les=113/114 n=383 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.719+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.719+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.719+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a42b336e:::obj435:head local-lis/les=113/114 n=383 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.719+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.719+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.719+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.719+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5ca0f00 con 0x4acb000 2026-02-11T23:31:27.719+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:a4383432:::obj559:head)]) -- 0x615c480 con 0x4acb000 2026-02-11T23:31:27.719+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.720+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 803 ==== MOSDPGPush(1.0 116/113 [PushOp(1:a4518626:::obj581:head, version: 111'581, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4518626:::obj581:head@111'581, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559d860 con 0x4acb000 2026-02-11T23:31:27.720+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:a4518626:::obj581:head, version: 111'581, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4518626:::obj581:head@111'581, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000016 epoch 116 2026-02-11T23:31:27.720+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a4518626:::obj581:head, version: 111'581, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4518626:::obj581:head@111'581, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.720+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.720+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 804 ==== pg_backfill(progress 1.0 e 116/116 lb 1:a4383432:::obj559:head) ==== 1062+0+0 (unknown 0 0 0) 0x6178100 con 0x4acb000 2026-02-11T23:31:27.720+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a4518626:::obj581:head, version: 111'581, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4518626:::obj581:head@111'581, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.720+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:a4383432:::obj559:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.720+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.720+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a4383432:::obj559:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.720+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a4518626:::obj581:head, version: 111'581, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4518626:::obj581:head@111'581, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.720+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.720+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a4518626:::obj581:head, version: 111'581, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4518626:::obj581:head@111'581, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000041 MOSDPGPush(1.0 116/113 [PushOp(1:a4518626:::obj581:head, version: 111'581, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4518626:::obj581:head@111'581, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a42b336e:::obj435:head local-lis/les=113/114 n=383 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.720+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.720+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a42b336e:::obj435:head local-lis/les=113/114 n=383 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:a4518626:::obj581:head, version: 111'581, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4518626:::obj581:head@111'581, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.720+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a42b336e:::obj435:head local-lis/les=113/114 n=383 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a4518626:::obj581:head@111'581, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.720+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.720+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a4383432:::obj559:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.720+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a42b336e:::obj435:head local-lis/les=113/114 n=383 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:a4518626:::obj581:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.720+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a42b336e:::obj435:head local-lis/les=113/114 n=383 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:a4518626:::obj581:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.720+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a42b336e:::obj435:head local-lis/les=113/114 n=383 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:a4518626:::obj581:head 2026-02-11T23:31:27.720+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a42b336e:::obj435:head local-lis/les=113/114 n=383 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:a4518626:::obj581:head v 111'581 2026-02-11T23:31:27.720+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a42b336e:::obj435:head local-lis/les=113/114 n=383 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.720+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.720+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a4518626:::obj581:head, version: 111'581, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4518626:::obj581:head@111'581, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.720+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.720+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.720+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a4383432:::obj559:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.720+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a4383432:::obj559:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 1.0 e 116/116 lb 1:a4383432:::obj559:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a42b336e:::obj435:head local-lis/les=113/114 n=383 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.720+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.720+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a42b336e:::obj435:head local-lis/les=113/114 n=383 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:a4383432:::obj559:head) v3 2026-02-11T23:31:27.720+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a42b336e:::obj435:head local-lis/les=113/114 n=383 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:a4383432:::obj559:head) 2026-02-11T23:31:27.720+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.720+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a4383432:::obj559:head) v3 finish latency 0.000009 2026-02-11T23:31:27.720+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a4383432:::obj559:head local-lis/les=113/114 n=384 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.720+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.720+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.720+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a4383432:::obj559:head local-lis/les=113/114 n=384 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.720+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.720+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.720+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.720+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fed200 con 0x4acb000 2026-02-11T23:31:27.720+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:a4518626:::obj581:head)]) -- 0x615c780 con 0x4acb000 2026-02-11T23:31:27.720+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.720+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 805 ==== MOSDPGPush(1.0 116/113 [PushOp(1:a4664b16:::obj367:head, version: 111'367, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4664b16:::obj367:head@111'367, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d8820 con 0x4acb000 2026-02-11T23:31:27.720+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:a4664b16:::obj367:head, version: 111'367, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4664b16:::obj367:head@111'367, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 116 2026-02-11T23:31:27.720+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a4664b16:::obj367:head, version: 111'367, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4664b16:::obj367:head@111'367, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.720+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.720+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 806 ==== pg_backfill(progress 1.0 e 116/116 lb 1:a4518626:::obj581:head) ==== 1062+0+0 (unknown 0 0 0) 0x6178680 con 0x4acb000 2026-02-11T23:31:27.720+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a4664b16:::obj367:head, version: 111'367, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4664b16:::obj367:head@111'367, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.720+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:a4518626:::obj581:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.720+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.720+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a4518626:::obj581:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.720+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a4664b16:::obj367:head, version: 111'367, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4664b16:::obj367:head@111'367, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.720+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.720+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a4664b16:::obj367:head, version: 111'367, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4664b16:::obj367:head@111'367, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 116/113 [PushOp(1:a4664b16:::obj367:head, version: 111'367, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4664b16:::obj367:head@111'367, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a4383432:::obj559:head local-lis/les=113/114 n=384 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.720+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.720+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a4383432:::obj559:head local-lis/les=113/114 n=384 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:a4664b16:::obj367:head, version: 111'367, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4664b16:::obj367:head@111'367, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.720+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a4383432:::obj559:head local-lis/les=113/114 n=384 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a4664b16:::obj367:head@111'367, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.720+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.720+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a4518626:::obj581:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.720+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a4383432:::obj559:head local-lis/les=113/114 n=384 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:a4664b16:::obj367:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.720+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a4383432:::obj559:head local-lis/les=113/114 n=384 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:a4664b16:::obj367:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.720+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a4383432:::obj559:head local-lis/les=113/114 n=384 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:a4664b16:::obj367:head 2026-02-11T23:31:27.720+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a4383432:::obj559:head local-lis/les=113/114 n=384 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:a4664b16:::obj367:head v 111'367 2026-02-11T23:31:27.720+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a4383432:::obj559:head local-lis/les=113/114 n=384 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.720+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.720+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a4664b16:::obj367:head, version: 111'367, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4664b16:::obj367:head@111'367, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.720+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.720+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.720+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a4518626:::obj581:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.720+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a4518626:::obj581:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 116/116 lb 1:a4518626:::obj581:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a4383432:::obj559:head local-lis/les=113/114 n=384 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.720+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.720+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a4383432:::obj559:head local-lis/les=113/114 n=384 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:a4518626:::obj581:head) v3 2026-02-11T23:31:27.720+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a4383432:::obj559:head local-lis/les=113/114 n=384 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:a4518626:::obj581:head) 2026-02-11T23:31:27.720+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.720+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a4518626:::obj581:head) v3 finish latency 0.000009 2026-02-11T23:31:27.720+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a4518626:::obj581:head local-lis/les=113/114 n=385 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.720+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.720+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.720+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a4518626:::obj581:head local-lis/les=113/114 n=385 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.720+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.720+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.720+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.720+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fce180 con 0x4acb000 2026-02-11T23:31:27.720+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:a4664b16:::obj367:head)]) -- 0x615ca80 con 0x4acb000 2026-02-11T23:31:27.720+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.721+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 807 ==== MOSDPGPush(1.0 116/113 [PushOp(1:a52fdb40:::obj548:head, version: 111'548, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a52fdb40:::obj548:head@111'548, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559cea0 con 0x4acb000 2026-02-11T23:31:27.721+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:a52fdb40:::obj548:head, version: 111'548, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a52fdb40:::obj548:head@111'548, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000015 epoch 116 2026-02-11T23:31:27.721+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a52fdb40:::obj548:head, version: 111'548, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a52fdb40:::obj548:head@111'548, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.721+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.721+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 808 ==== pg_backfill(progress 1.0 e 116/116 lb 1:a4664b16:::obj367:head) ==== 1062+0+0 (unknown 0 0 0) 0x6178c00 con 0x4acb000 2026-02-11T23:31:27.721+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a52fdb40:::obj548:head, version: 111'548, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a52fdb40:::obj548:head@111'548, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.721+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:a4664b16:::obj367:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.721+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a4664b16:::obj367:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.721+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.721+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a52fdb40:::obj548:head, version: 111'548, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a52fdb40:::obj548:head@111'548, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.721+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a52fdb40:::obj548:head, version: 111'548, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a52fdb40:::obj548:head@111'548, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000042 MOSDPGPush(1.0 116/113 [PushOp(1:a52fdb40:::obj548:head, version: 111'548, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a52fdb40:::obj548:head@111'548, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a4518626:::obj581:head local-lis/les=113/114 n=385 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.721+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.721+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.721+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a4518626:::obj581:head local-lis/les=113/114 n=385 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:a52fdb40:::obj548:head, version: 111'548, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a52fdb40:::obj548:head@111'548, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.721+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a4664b16:::obj367:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.721+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a4518626:::obj581:head local-lis/les=113/114 n=385 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a52fdb40:::obj548:head@111'548, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.721+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a4518626:::obj581:head local-lis/les=113/114 n=385 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:a52fdb40:::obj548:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.721+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a4518626:::obj581:head local-lis/les=113/114 n=385 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:a52fdb40:::obj548:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.721+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a4518626:::obj581:head local-lis/les=113/114 n=385 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:a52fdb40:::obj548:head 2026-02-11T23:31:27.721+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a4518626:::obj581:head local-lis/les=113/114 n=385 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:a52fdb40:::obj548:head v 111'548 2026-02-11T23:31:27.721+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a4518626:::obj581:head local-lis/les=113/114 n=385 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.721+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.721+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a52fdb40:::obj548:head, version: 111'548, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a52fdb40:::obj548:head@111'548, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.721+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.721+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.721+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a4664b16:::obj367:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.721+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a4664b16:::obj367:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 1.0 e 116/116 lb 1:a4664b16:::obj367:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a4518626:::obj581:head local-lis/les=113/114 n=385 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.721+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.721+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a4518626:::obj581:head local-lis/les=113/114 n=385 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:a4664b16:::obj367:head) v3 2026-02-11T23:31:27.721+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a4518626:::obj581:head local-lis/les=113/114 n=385 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:a4664b16:::obj367:head) 2026-02-11T23:31:27.721+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.721+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a4664b16:::obj367:head) v3 finish latency 0.000009 2026-02-11T23:31:27.721+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a4664b16:::obj367:head local-lis/les=113/114 n=386 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.721+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.721+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.721+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a4664b16:::obj367:head local-lis/les=113/114 n=386 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.721+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.721+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.721+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.721+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fecf00 con 0x4acb000 2026-02-11T23:31:27.721+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:a52fdb40:::obj548:head)]) -- 0x6014000 con 0x4acb000 2026-02-11T23:31:27.721+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.721+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 809 ==== MOSDPGPush(1.0 116/113 [PushOp(1:a53b2263:::obj137:head, version: 111'137, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a53b2263:::obj137:head@111'137, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559da00 con 0x4acb000 2026-02-11T23:31:27.721+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:a53b2263:::obj137:head, version: 111'137, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a53b2263:::obj137:head@111'137, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000016 epoch 116 2026-02-11T23:31:27.721+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a53b2263:::obj137:head, version: 111'137, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a53b2263:::obj137:head@111'137, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.722+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.722+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 810 ==== pg_backfill(progress 1.0 e 116/116 lb 1:a52fdb40:::obj548:head) ==== 1062+0+0 (unknown 0 0 0) 0x6179180 con 0x4acb000 2026-02-11T23:31:27.722+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:a52fdb40:::obj548:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 116 2026-02-11T23:31:27.722+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a53b2263:::obj137:head, version: 111'137, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a53b2263:::obj137:head@111'137, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.722+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a52fdb40:::obj548:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.722+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.722+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a52fdb40:::obj548:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.722+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.722+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a53b2263:::obj137:head, version: 111'137, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a53b2263:::obj137:head@111'137, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.722+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a53b2263:::obj137:head, version: 111'137, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a53b2263:::obj137:head@111'137, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000061 MOSDPGPush(1.0 116/113 [PushOp(1:a53b2263:::obj137:head, version: 111'137, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a53b2263:::obj137:head@111'137, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a4664b16:::obj367:head local-lis/les=113/114 n=386 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.722+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.722+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a4664b16:::obj367:head local-lis/les=113/114 n=386 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:a53b2263:::obj137:head, version: 111'137, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a53b2263:::obj137:head@111'137, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.722+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a4664b16:::obj367:head local-lis/les=113/114 n=386 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a53b2263:::obj137:head@111'137, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.722+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a4664b16:::obj367:head local-lis/les=113/114 n=386 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:a53b2263:::obj137:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.722+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a4664b16:::obj367:head local-lis/les=113/114 n=386 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:a53b2263:::obj137:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.722+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a4664b16:::obj367:head local-lis/les=113/114 n=386 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:a53b2263:::obj137:head 2026-02-11T23:31:27.722+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a4664b16:::obj367:head local-lis/les=113/114 n=386 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:a53b2263:::obj137:head v 111'137 2026-02-11T23:31:27.722+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a4664b16:::obj367:head local-lis/les=113/114 n=386 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.722+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.722+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a53b2263:::obj137:head, version: 111'137, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a53b2263:::obj137:head@111'137, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.722+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.722+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.722+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a52fdb40:::obj548:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.722+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a52fdb40:::obj548:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 1.0 e 116/116 lb 1:a52fdb40:::obj548:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a4664b16:::obj367:head local-lis/les=113/114 n=386 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.722+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.722+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a4664b16:::obj367:head local-lis/les=113/114 n=386 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:a52fdb40:::obj548:head) v3 2026-02-11T23:31:27.722+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a4664b16:::obj367:head local-lis/les=113/114 n=386 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:a52fdb40:::obj548:head) 2026-02-11T23:31:27.722+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.722+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a52fdb40:::obj548:head) v3 finish latency 0.000009 2026-02-11T23:31:27.722+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a52fdb40:::obj548:head local-lis/les=113/114 n=387 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.722+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.722+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.722+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a52fdb40:::obj548:head local-lis/les=113/114 n=387 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.722+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.722+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.722+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.722+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x612ef00 con 0x4acb000 2026-02-11T23:31:27.722+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:a53b2263:::obj137:head)]) -- 0x6005800 con 0x4acb000 2026-02-11T23:31:27.722+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.722+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 811 ==== MOSDPGPush(1.0 116/113 [PushOp(1:a555be2b:::obj75:head, version: 111'75, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a555be2b:::obj75:head@111'75, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x559c4e0 con 0x4acb000 2026-02-11T23:31:27.722+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:a555be2b:::obj75:head, version: 111'75, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a555be2b:::obj75:head@111'75, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 116 2026-02-11T23:31:27.722+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a555be2b:::obj75:head, version: 111'75, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a555be2b:::obj75:head@111'75, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.722+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.722+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a555be2b:::obj75:head, version: 111'75, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a555be2b:::obj75:head@111'75, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.722+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 812 ==== pg_backfill(progress 1.0 e 116/116 lb 1:a53b2263:::obj137:head) ==== 1062+0+0 (unknown 0 0 0) 0x5ece000 con 0x4acb000 2026-02-11T23:31:27.722+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.722+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:a53b2263:::obj137:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 116 2026-02-11T23:31:27.722+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a555be2b:::obj75:head, version: 111'75, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a555be2b:::obj75:head@111'75, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.722+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a53b2263:::obj137:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.722+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.722+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a53b2263:::obj137:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.722+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a555be2b:::obj75:head, version: 111'75, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a555be2b:::obj75:head@111'75, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 116/113 [PushOp(1:a555be2b:::obj75:head, version: 111'75, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a555be2b:::obj75:head@111'75, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a52fdb40:::obj548:head local-lis/les=113/114 n=387 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.722+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.722+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a52fdb40:::obj548:head local-lis/les=113/114 n=387 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:a555be2b:::obj75:head, version: 111'75, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a555be2b:::obj75:head@111'75, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.722+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a52fdb40:::obj548:head local-lis/les=113/114 n=387 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a555be2b:::obj75:head@111'75, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.722+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a52fdb40:::obj548:head local-lis/les=113/114 n=387 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:a555be2b:::obj75:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.722+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a52fdb40:::obj548:head local-lis/les=113/114 n=387 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:a555be2b:::obj75:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.722+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a52fdb40:::obj548:head local-lis/les=113/114 n=387 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:a555be2b:::obj75:head 2026-02-11T23:31:27.722+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a52fdb40:::obj548:head local-lis/les=113/114 n=387 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:a555be2b:::obj75:head v 111'75 2026-02-11T23:31:27.722+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a52fdb40:::obj548:head local-lis/les=113/114 n=387 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.722+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.722+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a555be2b:::obj75:head, version: 111'75, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a555be2b:::obj75:head@111'75, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.722+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.722+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.722+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a53b2263:::obj137:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.722+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a53b2263:::obj137:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 1.0 e 116/116 lb 1:a53b2263:::obj137:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a52fdb40:::obj548:head local-lis/les=113/114 n=387 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.722+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.722+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a52fdb40:::obj548:head local-lis/les=113/114 n=387 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:a53b2263:::obj137:head) v3 2026-02-11T23:31:27.722+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a52fdb40:::obj548:head local-lis/les=113/114 n=387 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:a53b2263:::obj137:head) 2026-02-11T23:31:27.722+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.722+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a53b2263:::obj137:head) v3 finish latency 0.000009 2026-02-11T23:31:27.722+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a53b2263:::obj137:head local-lis/les=113/114 n=388 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.722+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.722+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.722+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a53b2263:::obj137:head local-lis/les=113/114 n=388 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.722+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.722+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.722+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.722+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5ca1980 con 0x4acb000 2026-02-11T23:31:27.722+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:a555be2b:::obj75:head)]) -- 0x612fc80 con 0x4acb000 2026-02-11T23:31:27.722+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.723+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 813 ==== MOSDPGPush(1.0 116/113 [PushOp(1:a557efb1:::obj8:head, version: 111'8, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a557efb1:::obj8:head@111'8, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 4999+0+0 (unknown 0 0 0) 0x55f1860 con 0x4acb000 2026-02-11T23:31:27.723+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:a557efb1:::obj8:head, version: 111'8, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a557efb1:::obj8:head@111'8, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.723+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a557efb1:::obj8:head, version: 111'8, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a557efb1:::obj8:head@111'8, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.723+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.723+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a557efb1:::obj8:head, version: 111'8, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a557efb1:::obj8:head@111'8, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.723+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 814 ==== pg_backfill(progress 1.0 e 116/116 lb 1:a555be2b:::obj75:head) ==== 1061+0+0 (unknown 0 0 0) 0x5ece580 con 0x4acb000 2026-02-11T23:31:27.723+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.723+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:a555be2b:::obj75:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 116 2026-02-11T23:31:27.723+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a557efb1:::obj8:head, version: 111'8, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a557efb1:::obj8:head@111'8, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.723+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a555be2b:::obj75:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.723+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.723+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a555be2b:::obj75:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.723+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a557efb1:::obj8:head, version: 111'8, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a557efb1:::obj8:head@111'8, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 116/113 [PushOp(1:a557efb1:::obj8:head, version: 111'8, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a557efb1:::obj8:head@111'8, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a53b2263:::obj137:head local-lis/les=113/114 n=388 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.723+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.723+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a53b2263:::obj137:head local-lis/les=113/114 n=388 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:a557efb1:::obj8:head, version: 111'8, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a557efb1:::obj8:head@111'8, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.723+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a53b2263:::obj137:head local-lis/les=113/114 n=388 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a557efb1:::obj8:head@111'8, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.723+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a53b2263:::obj137:head local-lis/les=113/114 n=388 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:a557efb1:::obj8:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.723+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a53b2263:::obj137:head local-lis/les=113/114 n=388 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:a557efb1:::obj8:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.723+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a53b2263:::obj137:head local-lis/les=113/114 n=388 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:a557efb1:::obj8:head 2026-02-11T23:31:27.723+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a53b2263:::obj137:head local-lis/les=113/114 n=388 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:a557efb1:::obj8:head v 111'8 2026-02-11T23:31:27.723+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a53b2263:::obj137:head local-lis/les=113/114 n=388 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.723+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.723+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a557efb1:::obj8:head, version: 111'8, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a557efb1:::obj8:head@111'8, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.723+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.723+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.723+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a555be2b:::obj75:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.723+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a555be2b:::obj75:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 116/116 lb 1:a555be2b:::obj75:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a53b2263:::obj137:head local-lis/les=113/114 n=388 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.723+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.723+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a53b2263:::obj137:head local-lis/les=113/114 n=388 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:a555be2b:::obj75:head) v3 2026-02-11T23:31:27.723+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a53b2263:::obj137:head local-lis/les=113/114 n=388 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:a555be2b:::obj75:head) 2026-02-11T23:31:27.723+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.723+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a555be2b:::obj75:head) v3 finish latency 0.000010 2026-02-11T23:31:27.723+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a555be2b:::obj75:head local-lis/les=113/114 n=389 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.723+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.723+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.723+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a555be2b:::obj75:head local-lis/les=113/114 n=389 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.723+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.723+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.723+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.723+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5ca0180 con 0x4acb000 2026-02-11T23:31:27.723+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:a557efb1:::obj8:head)]) -- 0x6005380 con 0x4acb000 2026-02-11T23:31:27.723+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.723+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 815 ==== MOSDPGPush(1.0 116/113 [PushOp(1:a5c2f6e2:::obj111:head, version: 111'111, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a5c2f6e2:::obj111:head@111'111, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f0d00 con 0x4acb000 2026-02-11T23:31:27.723+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:a5c2f6e2:::obj111:head, version: 111'111, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a5c2f6e2:::obj111:head@111'111, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.723+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a5c2f6e2:::obj111:head, version: 111'111, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a5c2f6e2:::obj111:head@111'111, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.723+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.723+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 816 ==== pg_backfill(progress 1.0 e 116/116 lb 1:a557efb1:::obj8:head) ==== 1060+0+0 (unknown 0 0 0) 0x5eceb00 con 0x4acb000 2026-02-11T23:31:27.723+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:a557efb1:::obj8:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.723+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a5c2f6e2:::obj111:head, version: 111'111, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a5c2f6e2:::obj111:head@111'111, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.723+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a557efb1:::obj8:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.723+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.723+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a5c2f6e2:::obj111:head, version: 111'111, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a5c2f6e2:::obj111:head@111'111, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.723+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a5c2f6e2:::obj111:head, version: 111'111, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a5c2f6e2:::obj111:head@111'111, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:a5c2f6e2:::obj111:head, version: 111'111, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a5c2f6e2:::obj111:head@111'111, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a555be2b:::obj75:head local-lis/les=113/114 n=389 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.723+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.723+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.723+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a557efb1:::obj8:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.723+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a555be2b:::obj75:head local-lis/les=113/114 n=389 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:a5c2f6e2:::obj111:head, version: 111'111, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a5c2f6e2:::obj111:head@111'111, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.723+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a555be2b:::obj75:head local-lis/les=113/114 n=389 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a5c2f6e2:::obj111:head@111'111, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.723+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a555be2b:::obj75:head local-lis/les=113/114 n=389 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:a5c2f6e2:::obj111:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.723+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a555be2b:::obj75:head local-lis/les=113/114 n=389 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:a5c2f6e2:::obj111:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.723+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a555be2b:::obj75:head local-lis/les=113/114 n=389 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:a5c2f6e2:::obj111:head 2026-02-11T23:31:27.723+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a555be2b:::obj75:head local-lis/les=113/114 n=389 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:a5c2f6e2:::obj111:head v 111'111 2026-02-11T23:31:27.723+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a555be2b:::obj75:head local-lis/les=113/114 n=389 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.723+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.723+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a5c2f6e2:::obj111:head, version: 111'111, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a5c2f6e2:::obj111:head@111'111, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.723+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.723+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.723+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a557efb1:::obj8:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.723+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a557efb1:::obj8:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 1.0 e 116/116 lb 1:a557efb1:::obj8:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a555be2b:::obj75:head local-lis/les=113/114 n=389 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.723+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.723+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a555be2b:::obj75:head local-lis/les=113/114 n=389 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:a557efb1:::obj8:head) v3 2026-02-11T23:31:27.723+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a555be2b:::obj75:head local-lis/les=113/114 n=389 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:a557efb1:::obj8:head) 2026-02-11T23:31:27.723+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.723+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a557efb1:::obj8:head) v3 finish latency 0.000009 2026-02-11T23:31:27.723+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a557efb1:::obj8:head local-lis/les=113/114 n=390 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.723+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.723+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.723+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a557efb1:::obj8:head local-lis/les=113/114 n=390 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.723+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.723+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.723+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.723+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d1d680 con 0x4acb000 2026-02-11T23:31:27.723+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:a5c2f6e2:::obj111:head)]) -- 0x5fce480 con 0x4acb000 2026-02-11T23:31:27.723+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.724+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 817 ==== MOSDPGPush(1.0 116/113 [PushOp(1:a5ea1435:::obj539:head, version: 111'539, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a5ea1435:::obj539:head@111'539, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c000 con 0x4acb000 2026-02-11T23:31:27.724+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:a5ea1435:::obj539:head, version: 111'539, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a5ea1435:::obj539:head@111'539, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.724+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a5ea1435:::obj539:head, version: 111'539, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a5ea1435:::obj539:head@111'539, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.724+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.724+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 818 ==== pg_backfill(progress 1.0 e 116/116 lb 1:a5c2f6e2:::obj111:head) ==== 1062+0+0 (unknown 0 0 0) 0x5ecf080 con 0x4acb000 2026-02-11T23:31:27.724+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:a5c2f6e2:::obj111:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.724+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a5ea1435:::obj539:head, version: 111'539, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a5ea1435:::obj539:head@111'539, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.724+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a5c2f6e2:::obj111:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.724+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.724+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a5ea1435:::obj539:head, version: 111'539, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a5ea1435:::obj539:head@111'539, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.724+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a5ea1435:::obj539:head, version: 111'539, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a5ea1435:::obj539:head@111'539, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 116/113 [PushOp(1:a5ea1435:::obj539:head, version: 111'539, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a5ea1435:::obj539:head@111'539, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a557efb1:::obj8:head local-lis/les=113/114 n=390 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.724+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.724+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.724+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a557efb1:::obj8:head local-lis/les=113/114 n=390 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:a5ea1435:::obj539:head, version: 111'539, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a5ea1435:::obj539:head@111'539, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.724+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a5c2f6e2:::obj111:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.724+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a557efb1:::obj8:head local-lis/les=113/114 n=390 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a5ea1435:::obj539:head@111'539, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.724+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a557efb1:::obj8:head local-lis/les=113/114 n=390 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:a5ea1435:::obj539:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.724+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a557efb1:::obj8:head local-lis/les=113/114 n=390 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:a5ea1435:::obj539:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.724+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a557efb1:::obj8:head local-lis/les=113/114 n=390 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:a5ea1435:::obj539:head 2026-02-11T23:31:27.724+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a557efb1:::obj8:head local-lis/les=113/114 n=390 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:a5ea1435:::obj539:head v 111'539 2026-02-11T23:31:27.724+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a557efb1:::obj8:head local-lis/les=113/114 n=390 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.724+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.724+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a5ea1435:::obj539:head, version: 111'539, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a5ea1435:::obj539:head@111'539, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.724+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.724+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.724+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a5c2f6e2:::obj111:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.724+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a5c2f6e2:::obj111:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:a5c2f6e2:::obj111:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a557efb1:::obj8:head local-lis/les=113/114 n=390 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.724+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.724+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a557efb1:::obj8:head local-lis/les=113/114 n=390 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:a5c2f6e2:::obj111:head) v3 2026-02-11T23:31:27.724+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a557efb1:::obj8:head local-lis/les=113/114 n=390 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:a5c2f6e2:::obj111:head) 2026-02-11T23:31:27.724+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.724+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a5c2f6e2:::obj111:head) v3 finish latency 0.000009 2026-02-11T23:31:27.724+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a5c2f6e2:::obj111:head local-lis/les=113/114 n=391 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.724+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.724+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.724+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a5c2f6e2:::obj111:head local-lis/les=113/114 n=391 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.724+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.724+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.724+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.724+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x612e000 con 0x4acb000 2026-02-11T23:31:27.724+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:a5ea1435:::obj539:head)]) -- 0x612f980 con 0x4acb000 2026-02-11T23:31:27.724+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.724+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 819 ==== MOSDPGPush(1.0 116/113 [PushOp(1:a601bbe9:::obj562:head, version: 111'562, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a601bbe9:::obj562:head@111'562, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c9c0 con 0x4acb000 2026-02-11T23:31:27.724+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:a601bbe9:::obj562:head, version: 111'562, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a601bbe9:::obj562:head@111'562, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 116 2026-02-11T23:31:27.724+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a601bbe9:::obj562:head, version: 111'562, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a601bbe9:::obj562:head@111'562, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.725+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.725+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 820 ==== pg_backfill(progress 1.0 e 116/116 lb 1:a5ea1435:::obj539:head) ==== 1062+0+0 (unknown 0 0 0) 0x5ecf600 con 0x4acb000 2026-02-11T23:31:27.725+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:a5ea1435:::obj539:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 116 2026-02-11T23:31:27.725+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a601bbe9:::obj562:head, version: 111'562, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a601bbe9:::obj562:head@111'562, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.725+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a5ea1435:::obj539:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.725+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.725+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a601bbe9:::obj562:head, version: 111'562, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a601bbe9:::obj562:head@111'562, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.725+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a601bbe9:::obj562:head, version: 111'562, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a601bbe9:::obj562:head@111'562, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000047 MOSDPGPush(1.0 116/113 [PushOp(1:a601bbe9:::obj562:head, version: 111'562, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a601bbe9:::obj562:head@111'562, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a5c2f6e2:::obj111:head local-lis/les=113/114 n=391 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.725+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.725+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a5c2f6e2:::obj111:head local-lis/les=113/114 n=391 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:a601bbe9:::obj562:head, version: 111'562, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a601bbe9:::obj562:head@111'562, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.725+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.725+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a5c2f6e2:::obj111:head local-lis/les=113/114 n=391 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a601bbe9:::obj562:head@111'562, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.725+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a5ea1435:::obj539:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.725+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a5c2f6e2:::obj111:head local-lis/les=113/114 n=391 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:a601bbe9:::obj562:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.725+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a5c2f6e2:::obj111:head local-lis/les=113/114 n=391 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:a601bbe9:::obj562:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.725+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a5c2f6e2:::obj111:head local-lis/les=113/114 n=391 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:a601bbe9:::obj562:head 2026-02-11T23:31:27.725+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a5c2f6e2:::obj111:head local-lis/les=113/114 n=391 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:a601bbe9:::obj562:head v 111'562 2026-02-11T23:31:27.725+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a5c2f6e2:::obj111:head local-lis/les=113/114 n=391 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.725+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.725+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a601bbe9:::obj562:head, version: 111'562, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a601bbe9:::obj562:head@111'562, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:27.725+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a5c2f6e2:::obj111:head local-lis/les=113/114 n=391 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.725+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.725+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.725+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a5ea1435:::obj539:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.725+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a5ea1435:::obj539:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 1.0 e 116/116 lb 1:a5ea1435:::obj539:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a5c2f6e2:::obj111:head local-lis/les=113/114 n=391 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.725+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.725+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a5c2f6e2:::obj111:head local-lis/les=113/114 n=391 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:a5ea1435:::obj539:head) v3 2026-02-11T23:31:27.725+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a5c2f6e2:::obj111:head local-lis/les=113/114 n=391 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:a5ea1435:::obj539:head) 2026-02-11T23:31:27.725+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.725+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a5ea1435:::obj539:head) v3 finish latency 0.000011 2026-02-11T23:31:27.725+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.725+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.725+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a5ea1435:::obj539:head local-lis/les=113/114 n=392 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.725+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.725+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.725+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.725+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5ca0d80 con 0x4acb000 2026-02-11T23:31:27.725+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:a601bbe9:::obj562:head)]) -- 0x5ca1c80 con 0x4acb000 2026-02-11T23:31:27.725+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.725+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 821 ==== MOSDPGPush(1.0 116/113 [PushOp(1:a61ad63e:::obj13:head, version: 111'13, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a61ad63e:::obj13:head@111'13, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x55f1a00 con 0x4acb000 2026-02-11T23:31:27.725+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:a61ad63e:::obj13:head, version: 111'13, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a61ad63e:::obj13:head@111'13, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000017 epoch 116 2026-02-11T23:31:27.725+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a61ad63e:::obj13:head, version: 111'13, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a61ad63e:::obj13:head@111'13, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.725+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.725+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 822 ==== pg_backfill(progress 1.0 e 116/116 lb 1:a601bbe9:::obj562:head) ==== 1062+0+0 (unknown 0 0 0) 0x5ecfb80 con 0x4acb000 2026-02-11T23:31:27.725+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a61ad63e:::obj13:head, version: 111'13, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a61ad63e:::obj13:head@111'13, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.725+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:a601bbe9:::obj562:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.725+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.725+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a601bbe9:::obj562:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.725+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a61ad63e:::obj13:head, version: 111'13, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a61ad63e:::obj13:head@111'13, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.725+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.725+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a61ad63e:::obj13:head, version: 111'13, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a61ad63e:::obj13:head@111'13, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000044 MOSDPGPush(1.0 116/113 [PushOp(1:a61ad63e:::obj13:head, version: 111'13, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a61ad63e:::obj13:head@111'13, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a5ea1435:::obj539:head local-lis/les=113/114 n=392 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.725+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.725+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a5ea1435:::obj539:head local-lis/les=113/114 n=392 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:a61ad63e:::obj13:head, version: 111'13, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a61ad63e:::obj13:head@111'13, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.725+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a5ea1435:::obj539:head local-lis/les=113/114 n=392 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a61ad63e:::obj13:head@111'13, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.725+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.725+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a601bbe9:::obj562:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.725+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a5ea1435:::obj539:head local-lis/les=113/114 n=392 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:a61ad63e:::obj13:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.725+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a5ea1435:::obj539:head local-lis/les=113/114 n=392 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:a61ad63e:::obj13:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.725+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a5ea1435:::obj539:head local-lis/les=113/114 n=392 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:a61ad63e:::obj13:head 2026-02-11T23:31:27.725+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a5ea1435:::obj539:head local-lis/les=113/114 n=392 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:a61ad63e:::obj13:head v 111'13 2026-02-11T23:31:27.725+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a5ea1435:::obj539:head local-lis/les=113/114 n=392 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.725+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.725+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a61ad63e:::obj13:head, version: 111'13, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a61ad63e:::obj13:head@111'13, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000044 2026-02-11T23:31:27.725+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.725+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.725+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a601bbe9:::obj562:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.725+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a601bbe9:::obj562:head) v3 prio 15 cost 0 latency 0.000059 pg_backfill(progress 1.0 e 116/116 lb 1:a601bbe9:::obj562:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a5ea1435:::obj539:head local-lis/les=113/114 n=392 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.725+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.725+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a5ea1435:::obj539:head local-lis/les=113/114 n=392 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:a601bbe9:::obj562:head) v3 2026-02-11T23:31:27.725+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a5ea1435:::obj539:head local-lis/les=113/114 n=392 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:a601bbe9:::obj562:head) 2026-02-11T23:31:27.725+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.725+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a601bbe9:::obj562:head) v3 finish latency 0.000011 2026-02-11T23:31:27.725+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a601bbe9:::obj562:head local-lis/les=113/114 n=393 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.725+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.725+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.725+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a601bbe9:::obj562:head local-lis/les=113/114 n=393 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.725+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.725+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.725+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.725+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6171080 con 0x4acb000 2026-02-11T23:31:27.725+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:a61ad63e:::obj13:head)]) -- 0x6014c00 con 0x4acb000 2026-02-11T23:31:27.725+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.726+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 823 ==== MOSDPGPush(1.0 116/113 [PushOp(1:a630f391:::obj375:head, version: 111'375, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a630f391:::obj375:head@111'375, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559d380 con 0x4acb000 2026-02-11T23:31:27.726+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:a630f391:::obj375:head, version: 111'375, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a630f391:::obj375:head@111'375, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000016 epoch 116 2026-02-11T23:31:27.726+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a630f391:::obj375:head, version: 111'375, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a630f391:::obj375:head@111'375, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.726+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.726+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 824 ==== pg_backfill(progress 1.0 e 116/116 lb 1:a61ad63e:::obj13:head) ==== 1061+0+0 (unknown 0 0 0) 0x5ed0100 con 0x4acb000 2026-02-11T23:31:27.726+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a630f391:::obj375:head, version: 111'375, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a630f391:::obj375:head@111'375, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.726+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:a61ad63e:::obj13:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.726+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a61ad63e:::obj13:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.726+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.726+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a630f391:::obj375:head, version: 111'375, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a630f391:::obj375:head@111'375, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.726+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a630f391:::obj375:head, version: 111'375, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a630f391:::obj375:head@111'375, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000043 MOSDPGPush(1.0 116/113 [PushOp(1:a630f391:::obj375:head, version: 111'375, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a630f391:::obj375:head@111'375, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a601bbe9:::obj562:head local-lis/les=113/114 n=393 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.726+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.726+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.726+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a601bbe9:::obj562:head local-lis/les=113/114 n=393 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:a630f391:::obj375:head, version: 111'375, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a630f391:::obj375:head@111'375, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.726+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a61ad63e:::obj13:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.726+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a601bbe9:::obj562:head local-lis/les=113/114 n=393 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a630f391:::obj375:head@111'375, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.726+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a601bbe9:::obj562:head local-lis/les=113/114 n=393 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:a630f391:::obj375:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.726+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a601bbe9:::obj562:head local-lis/les=113/114 n=393 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:a630f391:::obj375:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.726+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a601bbe9:::obj562:head local-lis/les=113/114 n=393 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:a630f391:::obj375:head 2026-02-11T23:31:27.726+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a601bbe9:::obj562:head local-lis/les=113/114 n=393 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:a630f391:::obj375:head v 111'375 2026-02-11T23:31:27.726+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a601bbe9:::obj562:head local-lis/les=113/114 n=393 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.726+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.726+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a630f391:::obj375:head, version: 111'375, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a630f391:::obj375:head@111'375, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.726+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.726+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.726+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a61ad63e:::obj13:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.726+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a61ad63e:::obj13:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 1.0 e 116/116 lb 1:a61ad63e:::obj13:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a601bbe9:::obj562:head local-lis/les=113/114 n=393 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.726+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.726+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a601bbe9:::obj562:head local-lis/les=113/114 n=393 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:a61ad63e:::obj13:head) v3 2026-02-11T23:31:27.726+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a601bbe9:::obj562:head local-lis/les=113/114 n=393 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:a61ad63e:::obj13:head) 2026-02-11T23:31:27.726+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.726+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a61ad63e:::obj13:head) v3 finish latency 0.000009 2026-02-11T23:31:27.726+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a61ad63e:::obj13:head local-lis/les=113/114 n=394 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.726+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.726+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.726+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a61ad63e:::obj13:head local-lis/les=113/114 n=394 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.726+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.726+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.726+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.726+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x612e300 con 0x4acb000 2026-02-11T23:31:27.726+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:a630f391:::obj375:head)]) -- 0x6014f00 con 0x4acb000 2026-02-11T23:31:27.726+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.726+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 825 ==== MOSDPGPush(1.0 116/113 [PushOp(1:a6b27c5f:::obj420:head, version: 111'420, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a6b27c5f:::obj420:head@111'420, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c4e0 con 0x4acb000 2026-02-11T23:31:27.726+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:a6b27c5f:::obj420:head, version: 111'420, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a6b27c5f:::obj420:head@111'420, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000017 epoch 116 2026-02-11T23:31:27.726+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a6b27c5f:::obj420:head, version: 111'420, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a6b27c5f:::obj420:head@111'420, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.726+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.726+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 826 ==== pg_backfill(progress 1.0 e 116/116 lb 1:a630f391:::obj375:head) ==== 1062+0+0 (unknown 0 0 0) 0x5ed0680 con 0x4acb000 2026-02-11T23:31:27.726+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a6b27c5f:::obj420:head, version: 111'420, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a6b27c5f:::obj420:head@111'420, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.726+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:a630f391:::obj375:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.726+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.726+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a630f391:::obj375:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.726+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a6b27c5f:::obj420:head, version: 111'420, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a6b27c5f:::obj420:head@111'420, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.726+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.726+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a6b27c5f:::obj420:head, version: 111'420, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a6b27c5f:::obj420:head@111'420, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000043 MOSDPGPush(1.0 116/113 [PushOp(1:a6b27c5f:::obj420:head, version: 111'420, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a6b27c5f:::obj420:head@111'420, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a61ad63e:::obj13:head local-lis/les=113/114 n=394 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.726+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.726+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a61ad63e:::obj13:head local-lis/les=113/114 n=394 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:a6b27c5f:::obj420:head, version: 111'420, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a6b27c5f:::obj420:head@111'420, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.726+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a61ad63e:::obj13:head local-lis/les=113/114 n=394 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a6b27c5f:::obj420:head@111'420, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.726+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.726+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a630f391:::obj375:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.726+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a61ad63e:::obj13:head local-lis/les=113/114 n=394 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:a6b27c5f:::obj420:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.726+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a61ad63e:::obj13:head local-lis/les=113/114 n=394 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:a6b27c5f:::obj420:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.726+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a61ad63e:::obj13:head local-lis/les=113/114 n=394 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:a6b27c5f:::obj420:head 2026-02-11T23:31:27.726+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a61ad63e:::obj13:head local-lis/les=113/114 n=394 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:a6b27c5f:::obj420:head v 111'420 2026-02-11T23:31:27.726+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a61ad63e:::obj13:head local-lis/les=113/114 n=394 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.726+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.726+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a6b27c5f:::obj420:head, version: 111'420, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a6b27c5f:::obj420:head@111'420, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.726+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.726+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.726+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a630f391:::obj375:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.726+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a630f391:::obj375:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 1.0 e 116/116 lb 1:a630f391:::obj375:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a61ad63e:::obj13:head local-lis/les=113/114 n=394 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.726+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.726+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a61ad63e:::obj13:head local-lis/les=113/114 n=394 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:a630f391:::obj375:head) v3 2026-02-11T23:31:27.726+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a61ad63e:::obj13:head local-lis/les=113/114 n=394 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:a630f391:::obj375:head) 2026-02-11T23:31:27.726+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.726+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a630f391:::obj375:head) v3 finish latency 0.000009 2026-02-11T23:31:27.726+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a630f391:::obj375:head local-lis/les=113/114 n=395 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.726+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.727+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.727+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a630f391:::obj375:head local-lis/les=113/114 n=395 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.727+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.727+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.727+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.727+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6171380 con 0x4acb000 2026-02-11T23:31:27.727+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:a6b27c5f:::obj420:head)]) -- 0x613f980 con 0x4acb000 2026-02-11T23:31:27.727+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.727+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 827 ==== MOSDPGPush(1.0 116/113 [PushOp(1:a6f174aa:::obj203:head, version: 111'203, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a6f174aa:::obj203:head@111'203, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559da00 con 0x4acb000 2026-02-11T23:31:27.727+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:a6f174aa:::obj203:head, version: 111'203, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a6f174aa:::obj203:head@111'203, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 116 2026-02-11T23:31:27.727+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a6f174aa:::obj203:head, version: 111'203, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a6f174aa:::obj203:head@111'203, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.727+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.727+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 828 ==== pg_backfill(progress 1.0 e 116/116 lb 1:a6b27c5f:::obj420:head) ==== 1062+0+0 (unknown 0 0 0) 0x5ed0c00 con 0x4acb000 2026-02-11T23:31:27.727+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a6f174aa:::obj203:head, version: 111'203, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a6f174aa:::obj203:head@111'203, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.727+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:a6b27c5f:::obj420:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.727+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a6b27c5f:::obj420:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.727+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.727+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a6f174aa:::obj203:head, version: 111'203, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a6f174aa:::obj203:head@111'203, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.727+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.727+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a6f174aa:::obj203:head, version: 111'203, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a6f174aa:::obj203:head@111'203, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 116/113 [PushOp(1:a6f174aa:::obj203:head, version: 111'203, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a6f174aa:::obj203:head@111'203, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a630f391:::obj375:head local-lis/les=113/114 n=395 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.727+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.727+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a630f391:::obj375:head local-lis/les=113/114 n=395 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:a6f174aa:::obj203:head, version: 111'203, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a6f174aa:::obj203:head@111'203, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.727+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a630f391:::obj375:head local-lis/les=113/114 n=395 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a6f174aa:::obj203:head@111'203, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.727+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.727+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a6b27c5f:::obj420:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.727+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a630f391:::obj375:head local-lis/les=113/114 n=395 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:a6f174aa:::obj203:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.727+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a630f391:::obj375:head local-lis/les=113/114 n=395 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:a6f174aa:::obj203:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.727+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a630f391:::obj375:head local-lis/les=113/114 n=395 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:a6f174aa:::obj203:head 2026-02-11T23:31:27.727+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a630f391:::obj375:head local-lis/les=113/114 n=395 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:a6f174aa:::obj203:head v 111'203 2026-02-11T23:31:27.727+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a630f391:::obj375:head local-lis/les=113/114 n=395 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.727+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.727+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a6f174aa:::obj203:head, version: 111'203, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a6f174aa:::obj203:head@111'203, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.727+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.727+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.727+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a6b27c5f:::obj420:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.727+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a6b27c5f:::obj420:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 1.0 e 116/116 lb 1:a6b27c5f:::obj420:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a630f391:::obj375:head local-lis/les=113/114 n=395 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.727+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.727+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a630f391:::obj375:head local-lis/les=113/114 n=395 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:a6b27c5f:::obj420:head) v3 2026-02-11T23:31:27.727+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a630f391:::obj375:head local-lis/les=113/114 n=395 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:a6b27c5f:::obj420:head) 2026-02-11T23:31:27.727+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.727+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a6b27c5f:::obj420:head) v3 finish latency 0.000009 2026-02-11T23:31:27.727+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a6b27c5f:::obj420:head local-lis/les=113/114 n=396 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.727+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.727+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.727+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a6b27c5f:::obj420:head local-lis/les=113/114 n=396 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.727+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.727+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.727+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.727+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d1de00 con 0x4acb000 2026-02-11T23:31:27.727+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:a6f174aa:::obj203:head)]) -- 0x612e300 con 0x4acb000 2026-02-11T23:31:27.727+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.727+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 829 ==== MOSDPGPush(1.0 116/113 [PushOp(1:a8709cbe:::obj458:head, version: 111'458, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a8709cbe:::obj458:head@111'458, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559cea0 con 0x4acb000 2026-02-11T23:31:27.727+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:a8709cbe:::obj458:head, version: 111'458, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a8709cbe:::obj458:head@111'458, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 116 2026-02-11T23:31:27.727+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a8709cbe:::obj458:head, version: 111'458, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a8709cbe:::obj458:head@111'458, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.727+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.727+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 830 ==== pg_backfill(progress 1.0 e 116/116 lb 1:a6f174aa:::obj203:head) ==== 1062+0+0 (unknown 0 0 0) 0x5ed1180 con 0x4acb000 2026-02-11T23:31:27.727+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a8709cbe:::obj458:head, version: 111'458, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a8709cbe:::obj458:head@111'458, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.727+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:a6f174aa:::obj203:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.727+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a6f174aa:::obj203:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.727+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.727+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a8709cbe:::obj458:head, version: 111'458, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a8709cbe:::obj458:head@111'458, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.727+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a8709cbe:::obj458:head, version: 111'458, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a8709cbe:::obj458:head@111'458, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 116/113 [PushOp(1:a8709cbe:::obj458:head, version: 111'458, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a8709cbe:::obj458:head@111'458, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a6b27c5f:::obj420:head local-lis/les=113/114 n=396 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.727+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.727+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a6b27c5f:::obj420:head local-lis/les=113/114 n=396 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:a8709cbe:::obj458:head, version: 111'458, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a8709cbe:::obj458:head@111'458, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.727+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.727+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a6f174aa:::obj203:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.727+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a6b27c5f:::obj420:head local-lis/les=113/114 n=396 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a8709cbe:::obj458:head@111'458, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.727+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a6b27c5f:::obj420:head local-lis/les=113/114 n=396 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:a8709cbe:::obj458:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.727+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a6b27c5f:::obj420:head local-lis/les=113/114 n=396 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:a8709cbe:::obj458:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.727+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a6b27c5f:::obj420:head local-lis/les=113/114 n=396 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:a8709cbe:::obj458:head 2026-02-11T23:31:27.727+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a6b27c5f:::obj420:head local-lis/les=113/114 n=396 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:a8709cbe:::obj458:head v 111'458 2026-02-11T23:31:27.727+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a6b27c5f:::obj420:head local-lis/les=113/114 n=396 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.727+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.728+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a8709cbe:::obj458:head, version: 111'458, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a8709cbe:::obj458:head@111'458, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:27.728+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.728+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.728+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a6f174aa:::obj203:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.728+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a6f174aa:::obj203:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 1.0 e 116/116 lb 1:a6f174aa:::obj203:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a6b27c5f:::obj420:head local-lis/les=113/114 n=396 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.728+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.728+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a6b27c5f:::obj420:head local-lis/les=113/114 n=396 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:a6f174aa:::obj203:head) v3 2026-02-11T23:31:27.728+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a6b27c5f:::obj420:head local-lis/les=113/114 n=396 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:a6f174aa:::obj203:head) 2026-02-11T23:31:27.728+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.728+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a6f174aa:::obj203:head) v3 finish latency 0.000010 2026-02-11T23:31:27.728+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a6f174aa:::obj203:head local-lis/les=113/114 n=397 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.728+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.728+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.728+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a6f174aa:::obj203:head local-lis/les=113/114 n=397 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.728+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.728+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.728+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.728+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6171680 con 0x4acb000 2026-02-11T23:31:27.728+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:a8709cbe:::obj458:head)]) -- 0x6171080 con 0x4acb000 2026-02-11T23:31:27.728+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.728+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 831 ==== MOSDPGPush(1.0 116/113 [PushOp(1:a87532ae:::obj194:head, version: 111'194, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a87532ae:::obj194:head@111'194, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d8820 con 0x4acb000 2026-02-11T23:31:27.728+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:a87532ae:::obj194:head, version: 111'194, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a87532ae:::obj194:head@111'194, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 116 2026-02-11T23:31:27.728+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a87532ae:::obj194:head, version: 111'194, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a87532ae:::obj194:head@111'194, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.728+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.728+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 832 ==== pg_backfill(progress 1.0 e 116/116 lb 1:a8709cbe:::obj458:head) ==== 1062+0+0 (unknown 0 0 0) 0x5ed1700 con 0x4acb000 2026-02-11T23:31:27.728+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a87532ae:::obj194:head, version: 111'194, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a87532ae:::obj194:head@111'194, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.728+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:a8709cbe:::obj458:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.728+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a8709cbe:::obj458:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.728+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.728+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a87532ae:::obj194:head, version: 111'194, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a87532ae:::obj194:head@111'194, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.728+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.728+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a87532ae:::obj194:head, version: 111'194, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a87532ae:::obj194:head@111'194, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 116/113 [PushOp(1:a87532ae:::obj194:head, version: 111'194, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a87532ae:::obj194:head@111'194, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a6f174aa:::obj203:head local-lis/les=113/114 n=397 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.728+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.728+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a6f174aa:::obj203:head local-lis/les=113/114 n=397 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:a87532ae:::obj194:head, version: 111'194, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a87532ae:::obj194:head@111'194, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.728+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a6f174aa:::obj203:head local-lis/les=113/114 n=397 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a87532ae:::obj194:head@111'194, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.728+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.728+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a8709cbe:::obj458:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.728+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a6f174aa:::obj203:head local-lis/les=113/114 n=397 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:a87532ae:::obj194:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.728+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a6f174aa:::obj203:head local-lis/les=113/114 n=397 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:a87532ae:::obj194:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.728+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a6f174aa:::obj203:head local-lis/les=113/114 n=397 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:a87532ae:::obj194:head 2026-02-11T23:31:27.728+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a6f174aa:::obj203:head local-lis/les=113/114 n=397 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:a87532ae:::obj194:head v 111'194 2026-02-11T23:31:27.728+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a6f174aa:::obj203:head local-lis/les=113/114 n=397 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.728+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.728+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a87532ae:::obj194:head, version: 111'194, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a87532ae:::obj194:head@111'194, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.728+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.728+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.728+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a8709cbe:::obj458:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.728+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a8709cbe:::obj458:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:a8709cbe:::obj458:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a6f174aa:::obj203:head local-lis/les=113/114 n=397 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.728+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.728+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a6f174aa:::obj203:head local-lis/les=113/114 n=397 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:a8709cbe:::obj458:head) v3 2026-02-11T23:31:27.728+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a6f174aa:::obj203:head local-lis/les=113/114 n=397 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:a8709cbe:::obj458:head) 2026-02-11T23:31:27.728+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.728+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a8709cbe:::obj458:head) v3 finish latency 0.000010 2026-02-11T23:31:27.728+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a8709cbe:::obj458:head local-lis/les=113/114 n=398 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.728+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.728+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.728+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a8709cbe:::obj458:head local-lis/les=113/114 n=398 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.728+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.728+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.728+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.728+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x612e600 con 0x4acb000 2026-02-11T23:31:27.728+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:a87532ae:::obj194:head)]) -- 0x5ca0d80 con 0x4acb000 2026-02-11T23:31:27.728+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.729+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 833 ==== MOSDPGPush(1.0 116/113 [PushOp(1:a8d6b24d:::obj229:head, version: 111'229, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a8d6b24d:::obj229:head@111'229, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559d860 con 0x4acb000 2026-02-11T23:31:27.729+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:a8d6b24d:::obj229:head, version: 111'229, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a8d6b24d:::obj229:head@111'229, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 116 2026-02-11T23:31:27.729+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a8d6b24d:::obj229:head, version: 111'229, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a8d6b24d:::obj229:head@111'229, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.729+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.729+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a8d6b24d:::obj229:head, version: 111'229, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a8d6b24d:::obj229:head@111'229, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.729+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 834 ==== pg_backfill(progress 1.0 e 116/116 lb 1:a87532ae:::obj194:head) ==== 1062+0+0 (unknown 0 0 0) 0x5ece000 con 0x4acb000 2026-02-11T23:31:27.729+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:a87532ae:::obj194:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 116 2026-02-11T23:31:27.729+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.729+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a87532ae:::obj194:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.729+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:a8d6b24d:::obj229:head, version: 111'229, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a8d6b24d:::obj229:head@111'229, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.729+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.729+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a8d6b24d:::obj229:head, version: 111'229, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a8d6b24d:::obj229:head@111'229, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 116/113 [PushOp(1:a8d6b24d:::obj229:head, version: 111'229, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a8d6b24d:::obj229:head@111'229, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a8709cbe:::obj458:head local-lis/les=113/114 n=398 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.729+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.729+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a8709cbe:::obj458:head local-lis/les=113/114 n=398 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:a8d6b24d:::obj229:head, version: 111'229, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a8d6b24d:::obj229:head@111'229, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.729+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a8709cbe:::obj458:head local-lis/les=113/114 n=398 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a8d6b24d:::obj229:head@111'229, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.729+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.729+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a87532ae:::obj194:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.729+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a8709cbe:::obj458:head local-lis/les=113/114 n=398 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:a8d6b24d:::obj229:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.729+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a8709cbe:::obj458:head local-lis/les=113/114 n=398 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:a8d6b24d:::obj229:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.729+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a8709cbe:::obj458:head local-lis/les=113/114 n=398 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:a8d6b24d:::obj229:head 2026-02-11T23:31:27.729+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a8709cbe:::obj458:head local-lis/les=113/114 n=398 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:a8d6b24d:::obj229:head v 111'229 2026-02-11T23:31:27.729+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a8709cbe:::obj458:head local-lis/les=113/114 n=398 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.729+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.729+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:a8d6b24d:::obj229:head, version: 111'229, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a8d6b24d:::obj229:head@111'229, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.729+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.729+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.729+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a87532ae:::obj194:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.729+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a87532ae:::obj194:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 1.0 e 116/116 lb 1:a87532ae:::obj194:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a8709cbe:::obj458:head local-lis/les=113/114 n=398 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.729+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.729+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a8709cbe:::obj458:head local-lis/les=113/114 n=398 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:a87532ae:::obj194:head) v3 2026-02-11T23:31:27.729+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a8709cbe:::obj458:head local-lis/les=113/114 n=398 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:a87532ae:::obj194:head) 2026-02-11T23:31:27.729+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.729+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a87532ae:::obj194:head) v3 finish latency 0.000009 2026-02-11T23:31:27.729+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a87532ae:::obj194:head local-lis/les=113/114 n=399 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.729+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.729+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.729+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a87532ae:::obj194:head local-lis/les=113/114 n=399 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.729+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.729+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.729+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.729+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6171800 con 0x4acb000 2026-02-11T23:31:27.729+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:a8d6b24d:::obj229:head)]) -- 0x612e000 con 0x4acb000 2026-02-11T23:31:27.729+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.729+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 835 ==== MOSDPGPush(1.0 116/113 [PushOp(1:aa1e2680:::obj587:head, version: 111'587, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aa1e2680:::obj587:head@111'587, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d81a0 con 0x4acb000 2026-02-11T23:31:27.729+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:aa1e2680:::obj587:head, version: 111'587, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aa1e2680:::obj587:head@111'587, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 116 2026-02-11T23:31:27.729+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:aa1e2680:::obj587:head, version: 111'587, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aa1e2680:::obj587:head@111'587, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.729+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.729+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 836 ==== pg_backfill(progress 1.0 e 116/116 lb 1:a8d6b24d:::obj229:head) ==== 1062+0+0 (unknown 0 0 0) 0x6179700 con 0x4acb000 2026-02-11T23:31:27.729+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:a8d6b24d:::obj229:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.729+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:aa1e2680:::obj587:head, version: 111'587, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aa1e2680:::obj587:head@111'587, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.729+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a8d6b24d:::obj229:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.729+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.729+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a8d6b24d:::obj229:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.729+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.729+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:aa1e2680:::obj587:head, version: 111'587, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aa1e2680:::obj587:head@111'587, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.729+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:aa1e2680:::obj587:head, version: 111'587, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aa1e2680:::obj587:head@111'587, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000045 MOSDPGPush(1.0 116/113 [PushOp(1:aa1e2680:::obj587:head, version: 111'587, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aa1e2680:::obj587:head@111'587, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a87532ae:::obj194:head local-lis/les=113/114 n=399 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.729+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.729+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a87532ae:::obj194:head local-lis/les=113/114 n=399 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:aa1e2680:::obj587:head, version: 111'587, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aa1e2680:::obj587:head@111'587, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.729+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a87532ae:::obj194:head local-lis/les=113/114 n=399 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:aa1e2680:::obj587:head@111'587, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.729+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a87532ae:::obj194:head local-lis/les=113/114 n=399 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:aa1e2680:::obj587:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.729+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a87532ae:::obj194:head local-lis/les=113/114 n=399 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:aa1e2680:::obj587:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.729+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a87532ae:::obj194:head local-lis/les=113/114 n=399 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:aa1e2680:::obj587:head 2026-02-11T23:31:27.729+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a87532ae:::obj194:head local-lis/les=113/114 n=399 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:aa1e2680:::obj587:head v 111'587 2026-02-11T23:31:27.729+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a87532ae:::obj194:head local-lis/les=113/114 n=399 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.729+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.729+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:aa1e2680:::obj587:head, version: 111'587, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aa1e2680:::obj587:head@111'587, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.729+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a87532ae:::obj194:head local-lis/les=113/114 n=399 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.729+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.729+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.729+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:a8d6b24d:::obj229:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.729+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a8d6b24d:::obj229:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 1.0 e 116/116 lb 1:a8d6b24d:::obj229:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a87532ae:::obj194:head local-lis/les=113/114 n=399 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.729+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.729+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a87532ae:::obj194:head local-lis/les=113/114 n=399 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:a8d6b24d:::obj229:head) v3 2026-02-11T23:31:27.729+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a87532ae:::obj194:head local-lis/les=113/114 n=399 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:a8d6b24d:::obj229:head) 2026-02-11T23:31:27.729+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.729+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:a8d6b24d:::obj229:head) v3 finish latency 0.000010 2026-02-11T23:31:27.729+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.729+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.729+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a8d6b24d:::obj229:head local-lis/les=113/114 n=400 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.729+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.729+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.729+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.729+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x612e900 con 0x4acb000 2026-02-11T23:31:27.729+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:aa1e2680:::obj587:head)]) -- 0x5ca0a80 con 0x4acb000 2026-02-11T23:31:27.729+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.730+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 837 ==== MOSDPGPush(1.0 116/113 [PushOp(1:ab0f5189:::obj85:head, version: 111'85, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ab0f5189:::obj85:head@111'85, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5647040 con 0x4acb000 2026-02-11T23:31:27.730+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:ab0f5189:::obj85:head, version: 111'85, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ab0f5189:::obj85:head@111'85, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.730+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:ab0f5189:::obj85:head, version: 111'85, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ab0f5189:::obj85:head@111'85, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.730+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.730+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 838 ==== pg_backfill(progress 1.0 e 116/116 lb 1:aa1e2680:::obj587:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176000 con 0x4acb000 2026-02-11T23:31:27.730+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:aa1e2680:::obj587:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.730+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:ab0f5189:::obj85:head, version: 111'85, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ab0f5189:::obj85:head@111'85, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.730+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:aa1e2680:::obj587:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.730+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.730+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:ab0f5189:::obj85:head, version: 111'85, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ab0f5189:::obj85:head@111'85, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.730+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:ab0f5189:::obj85:head, version: 111'85, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ab0f5189:::obj85:head@111'85, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:ab0f5189:::obj85:head, version: 111'85, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ab0f5189:::obj85:head@111'85, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a8d6b24d:::obj229:head local-lis/les=113/114 n=400 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.730+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.730+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.730+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:aa1e2680:::obj587:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.730+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a8d6b24d:::obj229:head local-lis/les=113/114 n=400 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:ab0f5189:::obj85:head, version: 111'85, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ab0f5189:::obj85:head@111'85, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.730+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a8d6b24d:::obj229:head local-lis/les=113/114 n=400 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:ab0f5189:::obj85:head@111'85, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.730+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a8d6b24d:::obj229:head local-lis/les=113/114 n=400 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:ab0f5189:::obj85:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.730+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a8d6b24d:::obj229:head local-lis/les=113/114 n=400 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:ab0f5189:::obj85:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.730+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a8d6b24d:::obj229:head local-lis/les=113/114 n=400 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:ab0f5189:::obj85:head 2026-02-11T23:31:27.730+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a8d6b24d:::obj229:head local-lis/les=113/114 n=400 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:ab0f5189:::obj85:head v 111'85 2026-02-11T23:31:27.730+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a8d6b24d:::obj229:head local-lis/les=113/114 n=400 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.730+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.730+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:ab0f5189:::obj85:head, version: 111'85, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ab0f5189:::obj85:head@111'85, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:27.730+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.730+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.730+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:aa1e2680:::obj587:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.730+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:aa1e2680:::obj587:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 1.0 e 116/116 lb 1:aa1e2680:::obj587:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:a8d6b24d:::obj229:head local-lis/les=113/114 n=400 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.730+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.730+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a8d6b24d:::obj229:head local-lis/les=113/114 n=400 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:aa1e2680:::obj587:head) v3 2026-02-11T23:31:27.730+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:a8d6b24d:::obj229:head local-lis/les=113/114 n=400 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:aa1e2680:::obj587:head) 2026-02-11T23:31:27.730+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.730+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:aa1e2680:::obj587:head) v3 finish latency 0.000013 2026-02-11T23:31:27.730+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:aa1e2680:::obj587:head local-lis/les=113/114 n=401 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.730+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.730+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.730+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:aa1e2680:::obj587:head local-lis/les=113/114 n=401 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.730+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.730+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.730+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.730+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6015800 con 0x4acb000 2026-02-11T23:31:27.730+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:ab0f5189:::obj85:head)]) -- 0x5d1d680 con 0x4acb000 2026-02-11T23:31:27.730+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.730+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 839 ==== MOSDPGPush(1.0 116/113 [PushOp(1:ab946124:::obj15:head, version: 111'15, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ab946124:::obj15:head@111'15, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x55d8b60 con 0x4acb000 2026-02-11T23:31:27.730+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:ab946124:::obj15:head, version: 111'15, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ab946124:::obj15:head@111'15, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 116 2026-02-11T23:31:27.730+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:ab946124:::obj15:head, version: 111'15, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ab946124:::obj15:head@111'15, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.730+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.730+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 840 ==== pg_backfill(progress 1.0 e 116/116 lb 1:ab0f5189:::obj85:head) ==== 1061+0+0 (unknown 0 0 0) 0x6176b00 con 0x4acb000 2026-02-11T23:31:27.730+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:ab0f5189:::obj85:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.730+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:ab946124:::obj15:head, version: 111'15, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ab946124:::obj15:head@111'15, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.730+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:ab0f5189:::obj85:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.730+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.730+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:ab946124:::obj15:head, version: 111'15, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ab946124:::obj15:head@111'15, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.730+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:ab946124:::obj15:head, version: 111'15, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ab946124:::obj15:head@111'15, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(1.0 116/113 [PushOp(1:ab946124:::obj15:head, version: 111'15, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ab946124:::obj15:head@111'15, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:aa1e2680:::obj587:head local-lis/les=113/114 n=401 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.730+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.730+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.730+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:aa1e2680:::obj587:head local-lis/les=113/114 n=401 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:ab946124:::obj15:head, version: 111'15, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ab946124:::obj15:head@111'15, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.730+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:ab0f5189:::obj85:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.730+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:aa1e2680:::obj587:head local-lis/les=113/114 n=401 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:ab946124:::obj15:head@111'15, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.730+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:aa1e2680:::obj587:head local-lis/les=113/114 n=401 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:ab946124:::obj15:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.730+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:aa1e2680:::obj587:head local-lis/les=113/114 n=401 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:ab946124:::obj15:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.730+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:aa1e2680:::obj587:head local-lis/les=113/114 n=401 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:ab946124:::obj15:head 2026-02-11T23:31:27.730+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:aa1e2680:::obj587:head local-lis/les=113/114 n=401 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:ab946124:::obj15:head v 111'15 2026-02-11T23:31:27.730+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:aa1e2680:::obj587:head local-lis/les=113/114 n=401 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.730+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.730+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:ab946124:::obj15:head, version: 111'15, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ab946124:::obj15:head@111'15, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.730+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.730+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.730+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:ab0f5189:::obj85:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.730+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:ab0f5189:::obj85:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:ab0f5189:::obj85:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:aa1e2680:::obj587:head local-lis/les=113/114 n=401 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.730+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.730+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:aa1e2680:::obj587:head local-lis/les=113/114 n=401 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:ab0f5189:::obj85:head) v3 2026-02-11T23:31:27.730+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:aa1e2680:::obj587:head local-lis/les=113/114 n=401 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:ab0f5189:::obj85:head) 2026-02-11T23:31:27.730+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.730+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:ab0f5189:::obj85:head) v3 finish latency 0.000010 2026-02-11T23:31:27.730+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ab0f5189:::obj85:head local-lis/les=113/114 n=402 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.730+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.731+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.731+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ab0f5189:::obj85:head local-lis/les=113/114 n=402 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.731+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.731+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.731+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.731+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fed380 con 0x4acb000 2026-02-11T23:31:27.731+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:ab946124:::obj15:head)]) -- 0x6005380 con 0x4acb000 2026-02-11T23:31:27.731+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.731+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 841 ==== MOSDPGPush(1.0 116/113 [PushOp(1:ac7304c6:::obj483:head, version: 111'483, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ac7304c6:::obj483:head@111'483, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d9040 con 0x4acb000 2026-02-11T23:31:27.731+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:ac7304c6:::obj483:head, version: 111'483, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ac7304c6:::obj483:head@111'483, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 116 2026-02-11T23:31:27.731+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:ac7304c6:::obj483:head, version: 111'483, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ac7304c6:::obj483:head@111'483, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.731+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.731+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 842 ==== pg_backfill(progress 1.0 e 116/116 lb 1:ab946124:::obj15:head) ==== 1061+0+0 (unknown 0 0 0) 0x6177080 con 0x4acb000 2026-02-11T23:31:27.731+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:ab946124:::obj15:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.731+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:ac7304c6:::obj483:head, version: 111'483, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ac7304c6:::obj483:head@111'483, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.731+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:ab946124:::obj15:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.731+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.731+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:ac7304c6:::obj483:head, version: 111'483, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ac7304c6:::obj483:head@111'483, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.731+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:ac7304c6:::obj483:head, version: 111'483, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ac7304c6:::obj483:head@111'483, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 116/113 [PushOp(1:ac7304c6:::obj483:head, version: 111'483, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ac7304c6:::obj483:head@111'483, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:ab0f5189:::obj85:head local-lis/les=113/114 n=402 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.731+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.731+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.731+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ab0f5189:::obj85:head local-lis/les=113/114 n=402 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:ac7304c6:::obj483:head, version: 111'483, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ac7304c6:::obj483:head@111'483, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.731+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:ab946124:::obj15:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.731+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ab0f5189:::obj85:head local-lis/les=113/114 n=402 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:ac7304c6:::obj483:head@111'483, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.731+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ab0f5189:::obj85:head local-lis/les=113/114 n=402 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:ac7304c6:::obj483:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.731+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ab0f5189:::obj85:head local-lis/les=113/114 n=402 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:ac7304c6:::obj483:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.731+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ab0f5189:::obj85:head local-lis/les=113/114 n=402 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:ac7304c6:::obj483:head 2026-02-11T23:31:27.731+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ab0f5189:::obj85:head local-lis/les=113/114 n=402 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:ac7304c6:::obj483:head v 111'483 2026-02-11T23:31:27.731+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ab0f5189:::obj85:head local-lis/les=113/114 n=402 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.731+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.731+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:ac7304c6:::obj483:head, version: 111'483, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ac7304c6:::obj483:head@111'483, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.731+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.731+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.731+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:ab946124:::obj15:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.731+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:ab946124:::obj15:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 1.0 e 116/116 lb 1:ab946124:::obj15:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:ab0f5189:::obj85:head local-lis/les=113/114 n=402 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.731+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.731+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ab0f5189:::obj85:head local-lis/les=113/114 n=402 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:ab946124:::obj15:head) v3 2026-02-11T23:31:27.731+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ab0f5189:::obj85:head local-lis/les=113/114 n=402 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:ab946124:::obj15:head) 2026-02-11T23:31:27.731+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.731+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:ab946124:::obj15:head) v3 finish latency 0.000009 2026-02-11T23:31:27.731+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ab946124:::obj15:head local-lis/les=113/114 n=403 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.731+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.731+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.731+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ab946124:::obj15:head local-lis/les=113/114 n=403 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.731+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.731+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.731+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.731+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6170900 con 0x4acb000 2026-02-11T23:31:27.731+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:ac7304c6:::obj483:head)]) -- 0x5ca0180 con 0x4acb000 2026-02-11T23:31:27.731+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.731+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 843 ==== MOSDPGPush(1.0 116/113 [PushOp(1:acb8a831:::obj238:head, version: 111'238, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:acb8a831:::obj238:head@111'238, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d9520 con 0x4acb000 2026-02-11T23:31:27.731+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:acb8a831:::obj238:head, version: 111'238, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:acb8a831:::obj238:head@111'238, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.731+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:acb8a831:::obj238:head, version: 111'238, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:acb8a831:::obj238:head@111'238, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.731+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.731+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 844 ==== pg_backfill(progress 1.0 e 116/116 lb 1:ac7304c6:::obj483:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177600 con 0x4acb000 2026-02-11T23:31:27.731+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:ac7304c6:::obj483:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.731+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:acb8a831:::obj238:head, version: 111'238, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:acb8a831:::obj238:head@111'238, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.731+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:ac7304c6:::obj483:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.731+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.731+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:ac7304c6:::obj483:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.731+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.731+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:acb8a831:::obj238:head, version: 111'238, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:acb8a831:::obj238:head@111'238, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.731+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:acb8a831:::obj238:head, version: 111'238, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:acb8a831:::obj238:head@111'238, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000042 MOSDPGPush(1.0 116/113 [PushOp(1:acb8a831:::obj238:head, version: 111'238, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:acb8a831:::obj238:head@111'238, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:ab946124:::obj15:head local-lis/les=113/114 n=403 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.731+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.731+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ab946124:::obj15:head local-lis/les=113/114 n=403 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:acb8a831:::obj238:head, version: 111'238, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:acb8a831:::obj238:head@111'238, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.731+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ab946124:::obj15:head local-lis/les=113/114 n=403 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:acb8a831:::obj238:head@111'238, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.731+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ab946124:::obj15:head local-lis/les=113/114 n=403 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:acb8a831:::obj238:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.732+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ab946124:::obj15:head local-lis/les=113/114 n=403 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:acb8a831:::obj238:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.732+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ab946124:::obj15:head local-lis/les=113/114 n=403 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:acb8a831:::obj238:head 2026-02-11T23:31:27.732+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ab946124:::obj15:head local-lis/les=113/114 n=403 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:acb8a831:::obj238:head v 111'238 2026-02-11T23:31:27.732+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ab946124:::obj15:head local-lis/les=113/114 n=403 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.732+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.732+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:acb8a831:::obj238:head, version: 111'238, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:acb8a831:::obj238:head@111'238, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:27.732+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.732+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.732+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:ac7304c6:::obj483:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.732+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:ac7304c6:::obj483:head) v3 prio 15 cost 0 latency 0.000081 pg_backfill(progress 1.0 e 116/116 lb 1:ac7304c6:::obj483:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:ab946124:::obj15:head local-lis/les=113/114 n=403 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.732+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.732+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ab946124:::obj15:head local-lis/les=113/114 n=403 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:ac7304c6:::obj483:head) v3 2026-02-11T23:31:27.732+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ab946124:::obj15:head local-lis/les=113/114 n=403 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:ac7304c6:::obj483:head) 2026-02-11T23:31:27.732+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.732+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:ac7304c6:::obj483:head) v3 finish latency 0.000010 2026-02-11T23:31:27.732+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ac7304c6:::obj483:head local-lis/les=113/114 n=404 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.732+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.732+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.732+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ac7304c6:::obj483:head local-lis/les=113/114 n=404 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.732+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.732+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.732+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.732+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fed680 con 0x4acb000 2026-02-11T23:31:27.732+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:acb8a831:::obj238:head)]) -- 0x5ca1980 con 0x4acb000 2026-02-11T23:31:27.732+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.732+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 845 ==== MOSDPGPush(1.0 116/113 [PushOp(1:acfc8dd9:::obj74:head, version: 111'74, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:acfc8dd9:::obj74:head@111'74, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x56cb1e0 con 0x4acb000 2026-02-11T23:31:27.732+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:acfc8dd9:::obj74:head, version: 111'74, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:acfc8dd9:::obj74:head@111'74, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000018 epoch 116 2026-02-11T23:31:27.732+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:acfc8dd9:::obj74:head, version: 111'74, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:acfc8dd9:::obj74:head@111'74, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.732+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.732+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 846 ==== pg_backfill(progress 1.0 e 116/116 lb 1:acb8a831:::obj238:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177b80 con 0x4acb000 2026-02-11T23:31:27.732+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:acfc8dd9:::obj74:head, version: 111'74, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:acfc8dd9:::obj74:head@111'74, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.732+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:acb8a831:::obj238:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.732+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.732+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:acb8a831:::obj238:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.732+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:acfc8dd9:::obj74:head, version: 111'74, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:acfc8dd9:::obj74:head@111'74, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.732+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.732+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:acfc8dd9:::obj74:head, version: 111'74, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:acfc8dd9:::obj74:head@111'74, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000047 MOSDPGPush(1.0 116/113 [PushOp(1:acfc8dd9:::obj74:head, version: 111'74, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:acfc8dd9:::obj74:head@111'74, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:ac7304c6:::obj483:head local-lis/les=113/114 n=404 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.732+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.732+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ac7304c6:::obj483:head local-lis/les=113/114 n=404 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:acfc8dd9:::obj74:head, version: 111'74, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:acfc8dd9:::obj74:head@111'74, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.732+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ac7304c6:::obj483:head local-lis/les=113/114 n=404 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:acfc8dd9:::obj74:head@111'74, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.732+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.732+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:acb8a831:::obj238:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.732+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ac7304c6:::obj483:head local-lis/les=113/114 n=404 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:acfc8dd9:::obj74:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.732+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ac7304c6:::obj483:head local-lis/les=113/114 n=404 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:acfc8dd9:::obj74:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.732+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ac7304c6:::obj483:head local-lis/les=113/114 n=404 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:acfc8dd9:::obj74:head 2026-02-11T23:31:27.732+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ac7304c6:::obj483:head local-lis/les=113/114 n=404 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:acfc8dd9:::obj74:head v 111'74 2026-02-11T23:31:27.732+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ac7304c6:::obj483:head local-lis/les=113/114 n=404 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.732+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.732+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:acfc8dd9:::obj74:head, version: 111'74, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:acfc8dd9:::obj74:head@111'74, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.732+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.732+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.732+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:acb8a831:::obj238:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.732+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:acb8a831:::obj238:head) v3 prio 15 cost 0 latency 0.000060 pg_backfill(progress 1.0 e 116/116 lb 1:acb8a831:::obj238:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:ac7304c6:::obj483:head local-lis/les=113/114 n=404 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.732+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.732+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ac7304c6:::obj483:head local-lis/les=113/114 n=404 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:acb8a831:::obj238:head) v3 2026-02-11T23:31:27.732+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ac7304c6:::obj483:head local-lis/les=113/114 n=404 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:acb8a831:::obj238:head) 2026-02-11T23:31:27.732+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.732+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:acb8a831:::obj238:head) v3 finish latency 0.000009 2026-02-11T23:31:27.732+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:acb8a831:::obj238:head local-lis/les=113/114 n=405 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.732+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.732+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.732+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:acb8a831:::obj238:head local-lis/les=113/114 n=405 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.732+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.732+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.732+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.732+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6015b00 con 0x4acb000 2026-02-11T23:31:27.732+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:acfc8dd9:::obj74:head)]) -- 0x612ef00 con 0x4acb000 2026-02-11T23:31:27.732+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.733+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 847 ==== MOSDPGPush(1.0 116/113 [PushOp(1:ad830abd:::obj496:head, version: 111'496, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ad830abd:::obj496:head@111'496, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d89c0 con 0x4acb000 2026-02-11T23:31:27.733+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:ad830abd:::obj496:head, version: 111'496, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ad830abd:::obj496:head@111'496, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000016 epoch 116 2026-02-11T23:31:27.733+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:ad830abd:::obj496:head, version: 111'496, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ad830abd:::obj496:head@111'496, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.733+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.733+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 848 ==== pg_backfill(progress 1.0 e 116/116 lb 1:acfc8dd9:::obj74:head) ==== 1061+0+0 (unknown 0 0 0) 0x6178100 con 0x4acb000 2026-02-11T23:31:27.733+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:ad830abd:::obj496:head, version: 111'496, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ad830abd:::obj496:head@111'496, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.733+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:acfc8dd9:::obj74:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.733+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:acfc8dd9:::obj74:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.733+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.733+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:ad830abd:::obj496:head, version: 111'496, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ad830abd:::obj496:head@111'496, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.733+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:ad830abd:::obj496:head, version: 111'496, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ad830abd:::obj496:head@111'496, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000042 MOSDPGPush(1.0 116/113 [PushOp(1:ad830abd:::obj496:head, version: 111'496, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ad830abd:::obj496:head@111'496, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:acb8a831:::obj238:head local-lis/les=113/114 n=405 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.733+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.733+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.733+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:acfc8dd9:::obj74:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.733+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:acb8a831:::obj238:head local-lis/les=113/114 n=405 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:ad830abd:::obj496:head, version: 111'496, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ad830abd:::obj496:head@111'496, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.733+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:acb8a831:::obj238:head local-lis/les=113/114 n=405 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:ad830abd:::obj496:head@111'496, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.733+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:acb8a831:::obj238:head local-lis/les=113/114 n=405 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:ad830abd:::obj496:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.733+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:acb8a831:::obj238:head local-lis/les=113/114 n=405 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:ad830abd:::obj496:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.733+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:acb8a831:::obj238:head local-lis/les=113/114 n=405 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:ad830abd:::obj496:head 2026-02-11T23:31:27.733+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:acb8a831:::obj238:head local-lis/les=113/114 n=405 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:ad830abd:::obj496:head v 111'496 2026-02-11T23:31:27.733+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:acb8a831:::obj238:head local-lis/les=113/114 n=405 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.733+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.733+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:ad830abd:::obj496:head, version: 111'496, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ad830abd:::obj496:head@111'496, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.733+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.733+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.733+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:acfc8dd9:::obj74:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.733+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:acfc8dd9:::obj74:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 116/116 lb 1:acfc8dd9:::obj74:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:acb8a831:::obj238:head local-lis/les=113/114 n=405 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.733+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.733+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:acb8a831:::obj238:head local-lis/les=113/114 n=405 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:acfc8dd9:::obj74:head) v3 2026-02-11T23:31:27.733+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:acb8a831:::obj238:head local-lis/les=113/114 n=405 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:acfc8dd9:::obj74:head) 2026-02-11T23:31:27.733+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.733+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:acfc8dd9:::obj74:head) v3 finish latency 0.000009 2026-02-11T23:31:27.733+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:acfc8dd9:::obj74:head local-lis/les=113/114 n=406 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.733+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.733+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.733+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:acfc8dd9:::obj74:head local-lis/les=113/114 n=406 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.733+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.733+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.733+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.733+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5ca0480 con 0x4acb000 2026-02-11T23:31:27.733+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:ad830abd:::obj496:head)]) -- 0x6014000 con 0x4acb000 2026-02-11T23:31:27.733+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.733+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 849 ==== MOSDPGPush(1.0 116/113 [PushOp(1:adc1b1a4:::obj497:head, version: 111'497, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:adc1b1a4:::obj497:head@111'497, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559d040 con 0x4acb000 2026-02-11T23:31:27.733+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:adc1b1a4:::obj497:head, version: 111'497, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:adc1b1a4:::obj497:head@111'497, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000016 epoch 116 2026-02-11T23:31:27.733+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:adc1b1a4:::obj497:head, version: 111'497, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:adc1b1a4:::obj497:head@111'497, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.733+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.733+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 850 ==== pg_backfill(progress 1.0 e 116/116 lb 1:ad830abd:::obj496:head) ==== 1062+0+0 (unknown 0 0 0) 0x6178680 con 0x4acb000 2026-02-11T23:31:27.733+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:adc1b1a4:::obj497:head, version: 111'497, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:adc1b1a4:::obj497:head@111'497, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.733+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:ad830abd:::obj496:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.733+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.733+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:ad830abd:::obj496:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.733+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:adc1b1a4:::obj497:head, version: 111'497, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:adc1b1a4:::obj497:head@111'497, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.733+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.733+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:adc1b1a4:::obj497:head, version: 111'497, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:adc1b1a4:::obj497:head@111'497, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000044 MOSDPGPush(1.0 116/113 [PushOp(1:adc1b1a4:::obj497:head, version: 111'497, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:adc1b1a4:::obj497:head@111'497, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:acfc8dd9:::obj74:head local-lis/les=113/114 n=406 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.733+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.733+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:acfc8dd9:::obj74:head local-lis/les=113/114 n=406 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:adc1b1a4:::obj497:head, version: 111'497, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:adc1b1a4:::obj497:head@111'497, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.733+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:acfc8dd9:::obj74:head local-lis/les=113/114 n=406 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:adc1b1a4:::obj497:head@111'497, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.733+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.733+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:ad830abd:::obj496:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.733+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:acfc8dd9:::obj74:head local-lis/les=113/114 n=406 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:adc1b1a4:::obj497:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.733+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:acfc8dd9:::obj74:head local-lis/les=113/114 n=406 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:adc1b1a4:::obj497:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.733+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:acfc8dd9:::obj74:head local-lis/les=113/114 n=406 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:adc1b1a4:::obj497:head 2026-02-11T23:31:27.733+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:acfc8dd9:::obj74:head local-lis/les=113/114 n=406 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:adc1b1a4:::obj497:head v 111'497 2026-02-11T23:31:27.733+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:acfc8dd9:::obj74:head local-lis/les=113/114 n=406 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.733+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.733+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:adc1b1a4:::obj497:head, version: 111'497, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:adc1b1a4:::obj497:head@111'497, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.733+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.733+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.733+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:ad830abd:::obj496:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.733+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:ad830abd:::obj496:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 1.0 e 116/116 lb 1:ad830abd:::obj496:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:acfc8dd9:::obj74:head local-lis/les=113/114 n=406 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.733+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.733+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:acfc8dd9:::obj74:head local-lis/les=113/114 n=406 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:ad830abd:::obj496:head) v3 2026-02-11T23:31:27.733+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:acfc8dd9:::obj74:head local-lis/les=113/114 n=406 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:ad830abd:::obj496:head) 2026-02-11T23:31:27.733+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.733+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:ad830abd:::obj496:head) v3 finish latency 0.000010 2026-02-11T23:31:27.733+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ad830abd:::obj496:head local-lis/les=113/114 n=407 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.733+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.734+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.734+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ad830abd:::obj496:head local-lis/les=113/114 n=407 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.734+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.734+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.734+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.734+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6170600 con 0x4acb000 2026-02-11T23:31:27.734+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:adc1b1a4:::obj497:head)]) -- 0x615ca80 con 0x4acb000 2026-02-11T23:31:27.734+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.734+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 851 ==== MOSDPGPush(1.0 116/113 [PushOp(1:ae2ef588:::obj234:head, version: 111'234, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ae2ef588:::obj234:head@111'234, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559dba0 con 0x4acb000 2026-02-11T23:31:27.734+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:ae2ef588:::obj234:head, version: 111'234, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ae2ef588:::obj234:head@111'234, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 116 2026-02-11T23:31:27.734+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:ae2ef588:::obj234:head, version: 111'234, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ae2ef588:::obj234:head@111'234, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.734+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.734+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 852 ==== pg_backfill(progress 1.0 e 116/116 lb 1:adc1b1a4:::obj497:head) ==== 1062+0+0 (unknown 0 0 0) 0x6178c00 con 0x4acb000 2026-02-11T23:31:27.734+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:ae2ef588:::obj234:head, version: 111'234, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ae2ef588:::obj234:head@111'234, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.734+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:adc1b1a4:::obj497:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.734+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.734+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:adc1b1a4:::obj497:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.734+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:ae2ef588:::obj234:head, version: 111'234, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ae2ef588:::obj234:head@111'234, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.734+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.734+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:ae2ef588:::obj234:head, version: 111'234, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ae2ef588:::obj234:head@111'234, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 116/113 [PushOp(1:ae2ef588:::obj234:head, version: 111'234, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ae2ef588:::obj234:head@111'234, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:ad830abd:::obj496:head local-lis/les=113/114 n=407 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.734+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.734+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ad830abd:::obj496:head local-lis/les=113/114 n=407 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:ae2ef588:::obj234:head, version: 111'234, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ae2ef588:::obj234:head@111'234, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.734+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ad830abd:::obj496:head local-lis/les=113/114 n=407 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:ae2ef588:::obj234:head@111'234, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.734+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.734+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:adc1b1a4:::obj497:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.734+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ad830abd:::obj496:head local-lis/les=113/114 n=407 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:ae2ef588:::obj234:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.734+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ad830abd:::obj496:head local-lis/les=113/114 n=407 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:ae2ef588:::obj234:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.734+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ad830abd:::obj496:head local-lis/les=113/114 n=407 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:ae2ef588:::obj234:head 2026-02-11T23:31:27.734+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ad830abd:::obj496:head local-lis/les=113/114 n=407 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:ae2ef588:::obj234:head v 111'234 2026-02-11T23:31:27.734+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ad830abd:::obj496:head local-lis/les=113/114 n=407 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.734+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.734+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:ae2ef588:::obj234:head, version: 111'234, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ae2ef588:::obj234:head@111'234, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:27.734+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.734+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.734+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:adc1b1a4:::obj497:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.734+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:adc1b1a4:::obj497:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 116/116 lb 1:adc1b1a4:::obj497:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:ad830abd:::obj496:head local-lis/les=113/114 n=407 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.734+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.734+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ad830abd:::obj496:head local-lis/les=113/114 n=407 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:adc1b1a4:::obj497:head) v3 2026-02-11T23:31:27.734+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ad830abd:::obj496:head local-lis/les=113/114 n=407 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:adc1b1a4:::obj497:head) 2026-02-11T23:31:27.734+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.734+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:adc1b1a4:::obj497:head) v3 finish latency 0.000010 2026-02-11T23:31:27.734+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:adc1b1a4:::obj497:head local-lis/les=113/114 n=408 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.734+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.734+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.734+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:adc1b1a4:::obj497:head local-lis/les=113/114 n=408 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.734+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.734+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.734+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.734+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x612ec00 con 0x4acb000 2026-02-11T23:31:27.734+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:ae2ef588:::obj234:head)]) -- 0x615c780 con 0x4acb000 2026-02-11T23:31:27.734+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.735+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 853 ==== MOSDPGPush(1.0 116/113 [PushOp(1:aec2b241:::obj347:head, version: 111'347, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aec2b241:::obj347:head@111'347, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c680 con 0x4acb000 2026-02-11T23:31:27.735+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:aec2b241:::obj347:head, version: 111'347, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aec2b241:::obj347:head@111'347, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 116 2026-02-11T23:31:27.735+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:aec2b241:::obj347:head, version: 111'347, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aec2b241:::obj347:head@111'347, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.735+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.735+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 854 ==== pg_backfill(progress 1.0 e 116/116 lb 1:ae2ef588:::obj234:head) ==== 1062+0+0 (unknown 0 0 0) 0x6179180 con 0x4acb000 2026-02-11T23:31:27.735+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:aec2b241:::obj347:head, version: 111'347, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aec2b241:::obj347:head@111'347, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.735+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:ae2ef588:::obj234:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.735+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:ae2ef588:::obj234:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.735+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.735+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:aec2b241:::obj347:head, version: 111'347, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aec2b241:::obj347:head@111'347, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.735+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.735+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:aec2b241:::obj347:head, version: 111'347, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aec2b241:::obj347:head@111'347, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(1.0 116/113 [PushOp(1:aec2b241:::obj347:head, version: 111'347, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aec2b241:::obj347:head@111'347, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:adc1b1a4:::obj497:head local-lis/les=113/114 n=408 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.735+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.735+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:adc1b1a4:::obj497:head local-lis/les=113/114 n=408 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:aec2b241:::obj347:head, version: 111'347, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aec2b241:::obj347:head@111'347, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.735+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:adc1b1a4:::obj497:head local-lis/les=113/114 n=408 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:aec2b241:::obj347:head@111'347, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.735+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.735+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:ae2ef588:::obj234:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.735+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:adc1b1a4:::obj497:head local-lis/les=113/114 n=408 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:aec2b241:::obj347:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.735+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:adc1b1a4:::obj497:head local-lis/les=113/114 n=408 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:aec2b241:::obj347:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.735+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:adc1b1a4:::obj497:head local-lis/les=113/114 n=408 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:aec2b241:::obj347:head 2026-02-11T23:31:27.735+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:adc1b1a4:::obj497:head local-lis/les=113/114 n=408 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:aec2b241:::obj347:head v 111'347 2026-02-11T23:31:27.735+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:adc1b1a4:::obj497:head local-lis/les=113/114 n=408 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.735+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.735+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:aec2b241:::obj347:head, version: 111'347, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aec2b241:::obj347:head@111'347, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.735+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.735+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.735+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:ae2ef588:::obj234:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.735+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:ae2ef588:::obj234:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:ae2ef588:::obj234:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:adc1b1a4:::obj497:head local-lis/les=113/114 n=408 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.735+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.735+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:adc1b1a4:::obj497:head local-lis/les=113/114 n=408 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:ae2ef588:::obj234:head) v3 2026-02-11T23:31:27.735+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:adc1b1a4:::obj497:head local-lis/les=113/114 n=408 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:ae2ef588:::obj234:head) 2026-02-11T23:31:27.735+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.735+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:ae2ef588:::obj234:head) v3 finish latency 0.000009 2026-02-11T23:31:27.735+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ae2ef588:::obj234:head local-lis/les=113/114 n=409 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.735+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.735+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.735+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ae2ef588:::obj234:head local-lis/les=113/114 n=409 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.735+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.735+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.735+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.735+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6004900 con 0x4acb000 2026-02-11T23:31:27.735+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:aec2b241:::obj347:head)]) -- 0x615c480 con 0x4acb000 2026-02-11T23:31:27.735+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.735+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 855 ==== MOSDPGPush(1.0 116/113 [PushOp(1:aecaa835:::obj157:head, version: 111'157, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aecaa835:::obj157:head@111'157, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559d520 con 0x4acb000 2026-02-11T23:31:27.735+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:aecaa835:::obj157:head, version: 111'157, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aecaa835:::obj157:head@111'157, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 116 2026-02-11T23:31:27.735+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:aecaa835:::obj157:head, version: 111'157, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aecaa835:::obj157:head@111'157, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.735+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.735+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 856 ==== pg_backfill(progress 1.0 e 116/116 lb 1:aec2b241:::obj347:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2d080 con 0x4acb000 2026-02-11T23:31:27.735+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:aec2b241:::obj347:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.735+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:aecaa835:::obj157:head, version: 111'157, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aecaa835:::obj157:head@111'157, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.735+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:aec2b241:::obj347:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.735+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.735+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:aecaa835:::obj157:head, version: 111'157, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aecaa835:::obj157:head@111'157, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.735+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:aecaa835:::obj157:head, version: 111'157, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aecaa835:::obj157:head@111'157, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 116/113 [PushOp(1:aecaa835:::obj157:head, version: 111'157, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aecaa835:::obj157:head@111'157, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:ae2ef588:::obj234:head local-lis/les=113/114 n=409 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.735+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.735+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ae2ef588:::obj234:head local-lis/les=113/114 n=409 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:aecaa835:::obj157:head, version: 111'157, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aecaa835:::obj157:head@111'157, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.735+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.735+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:aec2b241:::obj347:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.735+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ae2ef588:::obj234:head local-lis/les=113/114 n=409 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:aecaa835:::obj157:head@111'157, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.735+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ae2ef588:::obj234:head local-lis/les=113/114 n=409 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:aecaa835:::obj157:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.735+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ae2ef588:::obj234:head local-lis/les=113/114 n=409 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:aecaa835:::obj157:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.735+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ae2ef588:::obj234:head local-lis/les=113/114 n=409 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:aecaa835:::obj157:head 2026-02-11T23:31:27.735+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ae2ef588:::obj234:head local-lis/les=113/114 n=409 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:aecaa835:::obj157:head v 111'157 2026-02-11T23:31:27.735+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ae2ef588:::obj234:head local-lis/les=113/114 n=409 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.735+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.735+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:aecaa835:::obj157:head, version: 111'157, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aecaa835:::obj157:head@111'157, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:27.735+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.735+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.735+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:aec2b241:::obj347:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.735+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:aec2b241:::obj347:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 116/116 lb 1:aec2b241:::obj347:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:ae2ef588:::obj234:head local-lis/les=113/114 n=409 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.735+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.735+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ae2ef588:::obj234:head local-lis/les=113/114 n=409 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:aec2b241:::obj347:head) v3 2026-02-11T23:31:27.735+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ae2ef588:::obj234:head local-lis/les=113/114 n=409 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:aec2b241:::obj347:head) 2026-02-11T23:31:27.735+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.735+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:aec2b241:::obj347:head) v3 finish latency 0.000009 2026-02-11T23:31:27.735+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:aec2b241:::obj347:head local-lis/les=113/114 n=410 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.735+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.735+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.735+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:aec2b241:::obj347:head local-lis/les=113/114 n=410 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.735+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.735+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.735+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.735+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fed800 con 0x4acb000 2026-02-11T23:31:27.735+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:aecaa835:::obj157:head)]) -- 0x5fed080 con 0x4acb000 2026-02-11T23:31:27.735+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.736+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 857 ==== MOSDPGPush(1.0 116/113 [PushOp(1:af2b834d:::obj153:head, version: 111'153, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af2b834d:::obj153:head@111'153, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f1ba0 con 0x4acb000 2026-02-11T23:31:27.736+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:af2b834d:::obj153:head, version: 111'153, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af2b834d:::obj153:head@111'153, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000016 epoch 116 2026-02-11T23:31:27.736+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:af2b834d:::obj153:head, version: 111'153, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af2b834d:::obj153:head@111'153, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.736+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.736+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 858 ==== pg_backfill(progress 1.0 e 116/116 lb 1:aecaa835:::obj157:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e680 con 0x4acb000 2026-02-11T23:31:27.736+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:af2b834d:::obj153:head, version: 111'153, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af2b834d:::obj153:head@111'153, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.736+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:aecaa835:::obj157:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.736+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:aecaa835:::obj157:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.736+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.736+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:af2b834d:::obj153:head, version: 111'153, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af2b834d:::obj153:head@111'153, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.736+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.736+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:af2b834d:::obj153:head, version: 111'153, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af2b834d:::obj153:head@111'153, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000045 MOSDPGPush(1.0 116/113 [PushOp(1:af2b834d:::obj153:head, version: 111'153, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af2b834d:::obj153:head@111'153, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:aec2b241:::obj347:head local-lis/les=113/114 n=410 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.736+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.736+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:aec2b241:::obj347:head local-lis/les=113/114 n=410 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:af2b834d:::obj153:head, version: 111'153, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af2b834d:::obj153:head@111'153, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.736+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:aec2b241:::obj347:head local-lis/les=113/114 n=410 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:af2b834d:::obj153:head@111'153, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.736+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.736+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:aecaa835:::obj157:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.736+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:aec2b241:::obj347:head local-lis/les=113/114 n=410 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:af2b834d:::obj153:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.736+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:aec2b241:::obj347:head local-lis/les=113/114 n=410 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:af2b834d:::obj153:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.736+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:aec2b241:::obj347:head local-lis/les=113/114 n=410 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:af2b834d:::obj153:head 2026-02-11T23:31:27.736+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:aec2b241:::obj347:head local-lis/les=113/114 n=410 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:af2b834d:::obj153:head v 111'153 2026-02-11T23:31:27.736+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:aec2b241:::obj347:head local-lis/les=113/114 n=410 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.736+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.736+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:af2b834d:::obj153:head, version: 111'153, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af2b834d:::obj153:head@111'153, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.736+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.736+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.736+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:aecaa835:::obj157:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.736+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:aecaa835:::obj157:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 1.0 e 116/116 lb 1:aecaa835:::obj157:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:aec2b241:::obj347:head local-lis/les=113/114 n=410 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.736+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.736+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:aec2b241:::obj347:head local-lis/les=113/114 n=410 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:aecaa835:::obj157:head) v3 2026-02-11T23:31:27.736+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:aec2b241:::obj347:head local-lis/les=113/114 n=410 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:aecaa835:::obj157:head) 2026-02-11T23:31:27.736+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.736+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:aecaa835:::obj157:head) v3 finish latency 0.000009 2026-02-11T23:31:27.736+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:aecaa835:::obj157:head local-lis/les=113/114 n=411 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.736+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.736+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.736+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:aecaa835:::obj157:head local-lis/les=113/114 n=411 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.736+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.736+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.736+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.736+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6170300 con 0x4acb000 2026-02-11T23:31:27.736+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:af2b834d:::obj153:head)]) -- 0x612f680 con 0x4acb000 2026-02-11T23:31:27.736+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.736+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 859 ==== MOSDPGPush(1.0 116/113 [PushOp(1:af2dbe17:::obj414:head, version: 111'414, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af2dbe17:::obj414:head@111'414, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559cb60 con 0x4acb000 2026-02-11T23:31:27.736+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:af2dbe17:::obj414:head, version: 111'414, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af2dbe17:::obj414:head@111'414, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 116 2026-02-11T23:31:27.736+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:af2dbe17:::obj414:head, version: 111'414, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af2dbe17:::obj414:head@111'414, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.736+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.736+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 860 ==== pg_backfill(progress 1.0 e 116/116 lb 1:af2b834d:::obj153:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e100 con 0x4acb000 2026-02-11T23:31:27.736+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:af2dbe17:::obj414:head, version: 111'414, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af2dbe17:::obj414:head@111'414, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.736+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:af2b834d:::obj153:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.736+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:af2b834d:::obj153:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.736+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.736+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:af2dbe17:::obj414:head, version: 111'414, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af2dbe17:::obj414:head@111'414, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.736+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.736+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:af2dbe17:::obj414:head, version: 111'414, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af2dbe17:::obj414:head@111'414, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 116/113 [PushOp(1:af2dbe17:::obj414:head, version: 111'414, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af2dbe17:::obj414:head@111'414, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:aecaa835:::obj157:head local-lis/les=113/114 n=411 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.736+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.736+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:aecaa835:::obj157:head local-lis/les=113/114 n=411 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:af2dbe17:::obj414:head, version: 111'414, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af2dbe17:::obj414:head@111'414, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.736+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:aecaa835:::obj157:head local-lis/les=113/114 n=411 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:af2dbe17:::obj414:head@111'414, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.736+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.736+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:af2b834d:::obj153:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.736+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:aecaa835:::obj157:head local-lis/les=113/114 n=411 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:af2dbe17:::obj414:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.736+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:aecaa835:::obj157:head local-lis/les=113/114 n=411 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:af2dbe17:::obj414:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.736+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:aecaa835:::obj157:head local-lis/les=113/114 n=411 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:af2dbe17:::obj414:head 2026-02-11T23:31:27.736+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:aecaa835:::obj157:head local-lis/les=113/114 n=411 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:af2dbe17:::obj414:head v 111'414 2026-02-11T23:31:27.736+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:aecaa835:::obj157:head local-lis/les=113/114 n=411 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.736+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.736+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:af2dbe17:::obj414:head, version: 111'414, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af2dbe17:::obj414:head@111'414, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.736+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.736+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.736+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:af2b834d:::obj153:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.736+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:af2b834d:::obj153:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:af2b834d:::obj153:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:aecaa835:::obj157:head local-lis/les=113/114 n=411 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.736+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.736+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:aecaa835:::obj157:head local-lis/les=113/114 n=411 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:af2b834d:::obj153:head) v3 2026-02-11T23:31:27.736+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:aecaa835:::obj157:head local-lis/les=113/114 n=411 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:af2b834d:::obj153:head) 2026-02-11T23:31:27.736+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.736+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:af2b834d:::obj153:head) v3 finish latency 0.000009 2026-02-11T23:31:27.736+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af2b834d:::obj153:head local-lis/les=113/114 n=412 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.736+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.737+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.737+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af2b834d:::obj153:head local-lis/les=113/114 n=412 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.737+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.737+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.737+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.737+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x612ed80 con 0x4acb000 2026-02-11T23:31:27.737+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:af2dbe17:::obj414:head)]) -- 0x6004f00 con 0x4acb000 2026-02-11T23:31:27.737+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.737+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 861 ==== MOSDPGPush(1.0 116/113 [PushOp(1:af44a1d6:::obj35:head, version: 111'35, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af44a1d6:::obj35:head@111'35, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5e26680 con 0x4acb000 2026-02-11T23:31:27.737+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:af44a1d6:::obj35:head, version: 111'35, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af44a1d6:::obj35:head@111'35, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 116 2026-02-11T23:31:27.737+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:af44a1d6:::obj35:head, version: 111'35, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af44a1d6:::obj35:head@111'35, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.737+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.737+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 862 ==== pg_backfill(progress 1.0 e 116/116 lb 1:af2dbe17:::obj414:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2c000 con 0x4acb000 2026-02-11T23:31:27.737+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:af2dbe17:::obj414:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.737+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:af44a1d6:::obj35:head, version: 111'35, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af44a1d6:::obj35:head@111'35, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.737+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:af2dbe17:::obj414:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.737+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.737+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:af44a1d6:::obj35:head, version: 111'35, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af44a1d6:::obj35:head@111'35, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.737+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:af44a1d6:::obj35:head, version: 111'35, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af44a1d6:::obj35:head@111'35, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000040 MOSDPGPush(1.0 116/113 [PushOp(1:af44a1d6:::obj35:head, version: 111'35, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af44a1d6:::obj35:head@111'35, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:af2b834d:::obj153:head local-lis/les=113/114 n=412 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.737+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.737+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.737+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af2b834d:::obj153:head local-lis/les=113/114 n=412 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:af44a1d6:::obj35:head, version: 111'35, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af44a1d6:::obj35:head@111'35, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.737+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:af2dbe17:::obj414:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.737+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af2b834d:::obj153:head local-lis/les=113/114 n=412 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:af44a1d6:::obj35:head@111'35, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.737+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af2b834d:::obj153:head local-lis/les=113/114 n=412 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:af44a1d6:::obj35:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.737+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af2b834d:::obj153:head local-lis/les=113/114 n=412 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:af44a1d6:::obj35:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.737+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af2b834d:::obj153:head local-lis/les=113/114 n=412 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:af44a1d6:::obj35:head 2026-02-11T23:31:27.737+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af2b834d:::obj153:head local-lis/les=113/114 n=412 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:af44a1d6:::obj35:head v 111'35 2026-02-11T23:31:27.737+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af2b834d:::obj153:head local-lis/les=113/114 n=412 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.737+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.737+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:af44a1d6:::obj35:head, version: 111'35, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af44a1d6:::obj35:head@111'35, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.737+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.737+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.737+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:af2dbe17:::obj414:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.737+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:af2dbe17:::obj414:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 116/116 lb 1:af2dbe17:::obj414:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:af2b834d:::obj153:head local-lis/les=113/114 n=412 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.737+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.737+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af2b834d:::obj153:head local-lis/les=113/114 n=412 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:af2dbe17:::obj414:head) v3 2026-02-11T23:31:27.737+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af2b834d:::obj153:head local-lis/les=113/114 n=412 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:af2dbe17:::obj414:head) 2026-02-11T23:31:27.737+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.737+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:af2dbe17:::obj414:head) v3 finish latency 0.000009 2026-02-11T23:31:27.737+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af2dbe17:::obj414:head local-lis/les=113/114 n=413 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.737+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.737+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.737+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af2dbe17:::obj414:head local-lis/les=113/114 n=413 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.737+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.737+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.737+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.737+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fed080 con 0x4acb000 2026-02-11T23:31:27.737+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:af44a1d6:::obj35:head)]) -- 0x612f200 con 0x4acb000 2026-02-11T23:31:27.737+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.737+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 863 ==== MOSDPGPush(1.0 116/113 [PushOp(1:af5f95cb:::obj12:head, version: 111'12, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af5f95cb:::obj12:head@111'12, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x55f0ea0 con 0x4acb000 2026-02-11T23:31:27.737+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:af5f95cb:::obj12:head, version: 111'12, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af5f95cb:::obj12:head@111'12, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 116 2026-02-11T23:31:27.737+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:af5f95cb:::obj12:head, version: 111'12, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af5f95cb:::obj12:head@111'12, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.737+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.737+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 864 ==== pg_backfill(progress 1.0 e 116/116 lb 1:af44a1d6:::obj35:head) ==== 1061+0+0 (unknown 0 0 0) 0x5982b00 con 0x4acb000 2026-02-11T23:31:27.737+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:af44a1d6:::obj35:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.737+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:af5f95cb:::obj12:head, version: 111'12, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af5f95cb:::obj12:head@111'12, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.737+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:af44a1d6:::obj35:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.737+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.737+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:af44a1d6:::obj35:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.737+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.737+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:af5f95cb:::obj12:head, version: 111'12, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af5f95cb:::obj12:head@111'12, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.737+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:af5f95cb:::obj12:head, version: 111'12, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af5f95cb:::obj12:head@111'12, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000045 MOSDPGPush(1.0 116/113 [PushOp(1:af5f95cb:::obj12:head, version: 111'12, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af5f95cb:::obj12:head@111'12, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:af2dbe17:::obj414:head local-lis/les=113/114 n=413 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.737+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.737+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af2dbe17:::obj414:head local-lis/les=113/114 n=413 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:af5f95cb:::obj12:head, version: 111'12, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af5f95cb:::obj12:head@111'12, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.737+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af2dbe17:::obj414:head local-lis/les=113/114 n=413 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:af5f95cb:::obj12:head@111'12, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.737+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af2dbe17:::obj414:head local-lis/les=113/114 n=413 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:af5f95cb:::obj12:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.737+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af2dbe17:::obj414:head local-lis/les=113/114 n=413 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:af5f95cb:::obj12:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.737+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af2dbe17:::obj414:head local-lis/les=113/114 n=413 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:af5f95cb:::obj12:head 2026-02-11T23:31:27.738+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af2dbe17:::obj414:head local-lis/les=113/114 n=413 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:af5f95cb:::obj12:head v 111'12 2026-02-11T23:31:27.738+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af2dbe17:::obj414:head local-lis/les=113/114 n=413 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.738+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.738+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:af5f95cb:::obj12:head, version: 111'12, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af5f95cb:::obj12:head@111'12, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:27.738+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.738+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.738+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:af44a1d6:::obj35:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.738+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:af44a1d6:::obj35:head) v3 prio 15 cost 0 latency 0.000092 pg_backfill(progress 1.0 e 116/116 lb 1:af44a1d6:::obj35:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:af2dbe17:::obj414:head local-lis/les=113/114 n=413 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.738+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.738+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af2dbe17:::obj414:head local-lis/les=113/114 n=413 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:af44a1d6:::obj35:head) v3 2026-02-11T23:31:27.738+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af2dbe17:::obj414:head local-lis/les=113/114 n=413 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:af44a1d6:::obj35:head) 2026-02-11T23:31:27.738+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.738+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:af44a1d6:::obj35:head) v3 finish latency 0.000023 2026-02-11T23:31:27.738+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af44a1d6:::obj35:head local-lis/les=113/114 n=414 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.738+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.738+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.738+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af44a1d6:::obj35:head local-lis/les=113/114 n=414 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.738+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.738+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.738+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.738+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fed800 con 0x4acb000 2026-02-11T23:31:27.738+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:af5f95cb:::obj12:head)]) -- 0x5ca1680 con 0x4acb000 2026-02-11T23:31:27.738+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.738+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.738+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.738+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 865 ==== MOSDPGPush(1.0 116/113 [PushOp(1:af93a04b:::obj437:head, version: 111'437, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af93a04b:::obj437:head@111'437, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f09c0 con 0x4acb000 2026-02-11T23:31:27.738+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:af93a04b:::obj437:head, version: 111'437, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af93a04b:::obj437:head@111'437, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 116 2026-02-11T23:31:27.738+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:af93a04b:::obj437:head, version: 111'437, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af93a04b:::obj437:head@111'437, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.738+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.738+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 866 ==== pg_backfill(progress 1.0 e 116/116 lb 1:af5f95cb:::obj12:head) ==== 1061+0+0 (unknown 0 0 0) 0x4a2cb00 con 0x4acb000 2026-02-11T23:31:27.738+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:af5f95cb:::obj12:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.738+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:af93a04b:::obj437:head, version: 111'437, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af93a04b:::obj437:head@111'437, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.738+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:af5f95cb:::obj12:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.738+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.738+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:af93a04b:::obj437:head, version: 111'437, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af93a04b:::obj437:head@111'437, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.738+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.738+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:af93a04b:::obj437:head, version: 111'437, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af93a04b:::obj437:head@111'437, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 116/113 [PushOp(1:af93a04b:::obj437:head, version: 111'437, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af93a04b:::obj437:head@111'437, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:af44a1d6:::obj35:head local-lis/les=113/114 n=414 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.738+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.738+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af44a1d6:::obj35:head local-lis/les=113/114 n=414 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:af93a04b:::obj437:head, version: 111'437, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af93a04b:::obj437:head@111'437, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.738+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.738+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:af5f95cb:::obj12:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.738+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af44a1d6:::obj35:head local-lis/les=113/114 n=414 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:af93a04b:::obj437:head@111'437, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.738+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af44a1d6:::obj35:head local-lis/les=113/114 n=414 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:af93a04b:::obj437:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.738+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af44a1d6:::obj35:head local-lis/les=113/114 n=414 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:af93a04b:::obj437:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.738+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af44a1d6:::obj35:head local-lis/les=113/114 n=414 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:af93a04b:::obj437:head 2026-02-11T23:31:27.738+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af44a1d6:::obj35:head local-lis/les=113/114 n=414 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:af93a04b:::obj437:head v 111'437 2026-02-11T23:31:27.738+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af44a1d6:::obj35:head local-lis/les=113/114 n=414 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.738+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.738+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:af93a04b:::obj437:head, version: 111'437, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af93a04b:::obj437:head@111'437, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000070 2026-02-11T23:31:27.738+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af44a1d6:::obj35:head local-lis/les=113/114 n=414 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.738+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.738+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.738+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:af5f95cb:::obj12:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.738+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:af5f95cb:::obj12:head) v3 prio 15 cost 0 latency 0.000092 pg_backfill(progress 1.0 e 116/116 lb 1:af5f95cb:::obj12:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:af44a1d6:::obj35:head local-lis/les=113/114 n=414 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.738+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.738+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af44a1d6:::obj35:head local-lis/les=113/114 n=414 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:af5f95cb:::obj12:head) v3 2026-02-11T23:31:27.738+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af44a1d6:::obj35:head local-lis/les=113/114 n=414 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:af5f95cb:::obj12:head) 2026-02-11T23:31:27.738+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.738+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:af5f95cb:::obj12:head) v3 finish latency 0.000010 2026-02-11T23:31:27.738+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.738+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.738+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af5f95cb:::obj12:head local-lis/les=113/114 n=415 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.738+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.738+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.738+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.738+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x615c780 con 0x4acb000 2026-02-11T23:31:27.738+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:af93a04b:::obj437:head)]) -- 0x615c480 con 0x4acb000 2026-02-11T23:31:27.738+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.739+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 867 ==== MOSDPGPush(1.0 116/113 [PushOp(1:af9ff462:::obj165:head, version: 111'165, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af9ff462:::obj165:head@111'165, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f1520 con 0x4acb000 2026-02-11T23:31:27.739+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:af9ff462:::obj165:head, version: 111'165, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af9ff462:::obj165:head@111'165, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 116 2026-02-11T23:31:27.739+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:af9ff462:::obj165:head, version: 111'165, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af9ff462:::obj165:head@111'165, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.739+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.739+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 868 ==== pg_backfill(progress 1.0 e 116/116 lb 1:af93a04b:::obj437:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99600 con 0x4acb000 2026-02-11T23:31:27.739+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:af9ff462:::obj165:head, version: 111'165, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af9ff462:::obj165:head@111'165, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.739+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.739+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:af9ff462:::obj165:head, version: 111'165, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af9ff462:::obj165:head@111'165, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.739+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:af93a04b:::obj437:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.739+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:af93a04b:::obj437:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.739+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:af9ff462:::obj165:head, version: 111'165, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af9ff462:::obj165:head@111'165, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000041 MOSDPGPush(1.0 116/113 [PushOp(1:af9ff462:::obj165:head, version: 111'165, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af9ff462:::obj165:head@111'165, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:af5f95cb:::obj12:head local-lis/les=113/114 n=415 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.739+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.739+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af5f95cb:::obj12:head local-lis/les=113/114 n=415 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:af9ff462:::obj165:head, version: 111'165, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af9ff462:::obj165:head@111'165, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.739+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.739+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af5f95cb:::obj12:head local-lis/les=113/114 n=415 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:af9ff462:::obj165:head@111'165, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.739+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:af93a04b:::obj437:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.739+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af5f95cb:::obj12:head local-lis/les=113/114 n=415 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:af9ff462:::obj165:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.739+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af5f95cb:::obj12:head local-lis/les=113/114 n=415 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:af9ff462:::obj165:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.739+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af5f95cb:::obj12:head local-lis/les=113/114 n=415 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:af9ff462:::obj165:head 2026-02-11T23:31:27.739+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af5f95cb:::obj12:head local-lis/les=113/114 n=415 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:af9ff462:::obj165:head v 111'165 2026-02-11T23:31:27.739+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af5f95cb:::obj12:head local-lis/les=113/114 n=415 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.739+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.739+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:af9ff462:::obj165:head, version: 111'165, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af9ff462:::obj165:head@111'165, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:27.739+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.739+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.739+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:af93a04b:::obj437:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.739+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:af93a04b:::obj437:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 1.0 e 116/116 lb 1:af93a04b:::obj437:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:af5f95cb:::obj12:head local-lis/les=113/114 n=415 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.739+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.739+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af5f95cb:::obj12:head local-lis/les=113/114 n=415 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:af93a04b:::obj437:head) v3 2026-02-11T23:31:27.739+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af5f95cb:::obj12:head local-lis/les=113/114 n=415 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:af93a04b:::obj437:head) 2026-02-11T23:31:27.739+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.739+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:af93a04b:::obj437:head) v3 finish latency 0.000013 2026-02-11T23:31:27.739+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af93a04b:::obj437:head local-lis/les=113/114 n=416 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.739+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.739+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.739+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af93a04b:::obj437:head local-lis/les=113/114 n=416 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.739+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.739+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.739+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.739+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x612ec00 con 0x4acb000 2026-02-11T23:31:27.739+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:af9ff462:::obj165:head)]) -- 0x6004180 con 0x4acb000 2026-02-11T23:31:27.739+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.739+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 869 ==== MOSDPGPush(1.0 116/113 [PushOp(1:afacbcad:::obj315:head, version: 111'315, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afacbcad:::obj315:head@111'315, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c820 con 0x4acb000 2026-02-11T23:31:27.739+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:afacbcad:::obj315:head, version: 111'315, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afacbcad:::obj315:head@111'315, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.739+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:afacbcad:::obj315:head, version: 111'315, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afacbcad:::obj315:head@111'315, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.739+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.739+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 870 ==== pg_backfill(progress 1.0 e 116/116 lb 1:af9ff462:::obj165:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99b80 con 0x4acb000 2026-02-11T23:31:27.739+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:af9ff462:::obj165:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.739+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:afacbcad:::obj315:head, version: 111'315, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afacbcad:::obj315:head@111'315, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.739+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:af9ff462:::obj165:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.739+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.739+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:afacbcad:::obj315:head, version: 111'315, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afacbcad:::obj315:head@111'315, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.739+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:afacbcad:::obj315:head, version: 111'315, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afacbcad:::obj315:head@111'315, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 116/113 [PushOp(1:afacbcad:::obj315:head, version: 111'315, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afacbcad:::obj315:head@111'315, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:af93a04b:::obj437:head local-lis/les=113/114 n=416 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.739+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.739+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.739+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af93a04b:::obj437:head local-lis/les=113/114 n=416 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:afacbcad:::obj315:head, version: 111'315, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afacbcad:::obj315:head@111'315, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.739+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:af9ff462:::obj165:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.739+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af93a04b:::obj437:head local-lis/les=113/114 n=416 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:afacbcad:::obj315:head@111'315, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.739+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af93a04b:::obj437:head local-lis/les=113/114 n=416 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:afacbcad:::obj315:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.739+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af93a04b:::obj437:head local-lis/les=113/114 n=416 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:afacbcad:::obj315:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.739+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af93a04b:::obj437:head local-lis/les=113/114 n=416 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:afacbcad:::obj315:head 2026-02-11T23:31:27.739+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af93a04b:::obj437:head local-lis/les=113/114 n=416 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:afacbcad:::obj315:head v 111'315 2026-02-11T23:31:27.739+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af93a04b:::obj437:head local-lis/les=113/114 n=416 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.739+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.739+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:afacbcad:::obj315:head, version: 111'315, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afacbcad:::obj315:head@111'315, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.739+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.739+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.739+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:af9ff462:::obj165:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.739+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:af9ff462:::obj165:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 1.0 e 116/116 lb 1:af9ff462:::obj165:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:af93a04b:::obj437:head local-lis/les=113/114 n=416 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.739+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.739+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af93a04b:::obj437:head local-lis/les=113/114 n=416 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:af9ff462:::obj165:head) v3 2026-02-11T23:31:27.739+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af93a04b:::obj437:head local-lis/les=113/114 n=416 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:af9ff462:::obj165:head) 2026-02-11T23:31:27.739+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.739+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:af9ff462:::obj165:head) v3 finish latency 0.000011 2026-02-11T23:31:27.739+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af9ff462:::obj165:head local-lis/les=113/114 n=417 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.739+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.739+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.739+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af9ff462:::obj165:head local-lis/les=113/114 n=417 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.739+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.739+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.739+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.739+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x615ca80 con 0x4acb000 2026-02-11T23:31:27.739+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:afacbcad:::obj315:head)]) -- 0x6004000 con 0x4acb000 2026-02-11T23:31:27.740+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.740+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 871 ==== MOSDPGPush(1.0 116/113 [PushOp(1:afbdc8ea:::obj55:head, version: 111'55, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afbdc8ea:::obj55:head@111'55, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x559dd40 con 0x4acb000 2026-02-11T23:31:27.740+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:afbdc8ea:::obj55:head, version: 111'55, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afbdc8ea:::obj55:head@111'55, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000015 epoch 116 2026-02-11T23:31:27.740+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:afbdc8ea:::obj55:head, version: 111'55, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afbdc8ea:::obj55:head@111'55, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.740+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.740+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 872 ==== pg_backfill(progress 1.0 e 116/116 lb 1:afacbcad:::obj315:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9a100 con 0x4acb000 2026-02-11T23:31:27.740+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:afbdc8ea:::obj55:head, version: 111'55, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afbdc8ea:::obj55:head@111'55, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.740+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:afacbcad:::obj315:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.740+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:afacbcad:::obj315:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.740+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.740+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:afbdc8ea:::obj55:head, version: 111'55, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afbdc8ea:::obj55:head@111'55, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.740+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:afbdc8ea:::obj55:head, version: 111'55, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afbdc8ea:::obj55:head@111'55, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(1.0 116/113 [PushOp(1:afbdc8ea:::obj55:head, version: 111'55, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afbdc8ea:::obj55:head@111'55, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:af9ff462:::obj165:head local-lis/les=113/114 n=417 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.740+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.740+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.740+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af9ff462:::obj165:head local-lis/les=113/114 n=417 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:afbdc8ea:::obj55:head, version: 111'55, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afbdc8ea:::obj55:head@111'55, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.740+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:afacbcad:::obj315:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.740+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af9ff462:::obj165:head local-lis/les=113/114 n=417 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:afbdc8ea:::obj55:head@111'55, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.740+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af9ff462:::obj165:head local-lis/les=113/114 n=417 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:afbdc8ea:::obj55:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.740+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af9ff462:::obj165:head local-lis/les=113/114 n=417 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:afbdc8ea:::obj55:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.740+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af9ff462:::obj165:head local-lis/les=113/114 n=417 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:afbdc8ea:::obj55:head 2026-02-11T23:31:27.740+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af9ff462:::obj165:head local-lis/les=113/114 n=417 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:afbdc8ea:::obj55:head v 111'55 2026-02-11T23:31:27.740+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af9ff462:::obj165:head local-lis/les=113/114 n=417 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.740+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.740+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:afbdc8ea:::obj55:head, version: 111'55, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afbdc8ea:::obj55:head@111'55, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.740+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.740+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.740+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:afacbcad:::obj315:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.740+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:afacbcad:::obj315:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 1.0 e 116/116 lb 1:afacbcad:::obj315:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:af9ff462:::obj165:head local-lis/les=113/114 n=417 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.740+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.740+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af9ff462:::obj165:head local-lis/les=113/114 n=417 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:afacbcad:::obj315:head) v3 2026-02-11T23:31:27.740+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:af9ff462:::obj165:head local-lis/les=113/114 n=417 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:afacbcad:::obj315:head) 2026-02-11T23:31:27.740+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.740+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:afacbcad:::obj315:head) v3 finish latency 0.000016 2026-02-11T23:31:27.740+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:afacbcad:::obj315:head local-lis/les=113/114 n=418 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.740+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.740+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.740+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:afacbcad:::obj315:head local-lis/les=113/114 n=418 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.740+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.740+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.740+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.740+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6170600 con 0x4acb000 2026-02-11T23:31:27.740+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:afbdc8ea:::obj55:head)]) -- 0x6001c80 con 0x4acb000 2026-02-11T23:31:27.740+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.740+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 873 ==== MOSDPGPush(1.0 116/113 [PushOp(1:afd348d8:::obj184:head, version: 111'184, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afd348d8:::obj184:head@111'184, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d8680 con 0x4acb000 2026-02-11T23:31:27.740+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:afd348d8:::obj184:head, version: 111'184, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afd348d8:::obj184:head@111'184, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 116 2026-02-11T23:31:27.740+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:afd348d8:::obj184:head, version: 111'184, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afd348d8:::obj184:head@111'184, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.740+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.740+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 874 ==== pg_backfill(progress 1.0 e 116/116 lb 1:afbdc8ea:::obj55:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d9a680 con 0x4acb000 2026-02-11T23:31:27.740+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:afbdc8ea:::obj55:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.740+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:afd348d8:::obj184:head, version: 111'184, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afd348d8:::obj184:head@111'184, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.740+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:afbdc8ea:::obj55:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.740+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.740+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:afd348d8:::obj184:head, version: 111'184, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afd348d8:::obj184:head@111'184, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.740+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:afd348d8:::obj184:head, version: 111'184, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afd348d8:::obj184:head@111'184, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 116/113 [PushOp(1:afd348d8:::obj184:head, version: 111'184, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afd348d8:::obj184:head@111'184, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:afacbcad:::obj315:head local-lis/les=113/114 n=418 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.740+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.740+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.740+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:afacbcad:::obj315:head local-lis/les=113/114 n=418 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:afd348d8:::obj184:head, version: 111'184, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afd348d8:::obj184:head@111'184, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.740+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:afbdc8ea:::obj55:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.740+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:afacbcad:::obj315:head local-lis/les=113/114 n=418 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:afd348d8:::obj184:head@111'184, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.740+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:afacbcad:::obj315:head local-lis/les=113/114 n=418 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:afd348d8:::obj184:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.740+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:afacbcad:::obj315:head local-lis/les=113/114 n=418 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:afd348d8:::obj184:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.740+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:afacbcad:::obj315:head local-lis/les=113/114 n=418 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:afd348d8:::obj184:head 2026-02-11T23:31:27.740+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:afacbcad:::obj315:head local-lis/les=113/114 n=418 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:afd348d8:::obj184:head v 111'184 2026-02-11T23:31:27.740+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:afacbcad:::obj315:head local-lis/les=113/114 n=418 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.740+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.740+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:afd348d8:::obj184:head, version: 111'184, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afd348d8:::obj184:head@111'184, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.740+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.740+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.740+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:afbdc8ea:::obj55:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.740+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:afbdc8ea:::obj55:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 1.0 e 116/116 lb 1:afbdc8ea:::obj55:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:afacbcad:::obj315:head local-lis/les=113/114 n=418 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.740+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.740+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:afacbcad:::obj315:head local-lis/les=113/114 n=418 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:afbdc8ea:::obj55:head) v3 2026-02-11T23:31:27.740+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:afacbcad:::obj315:head local-lis/les=113/114 n=418 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:afbdc8ea:::obj55:head) 2026-02-11T23:31:27.740+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.740+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:afbdc8ea:::obj55:head) v3 finish latency 0.000011 2026-02-11T23:31:27.740+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:afbdc8ea:::obj55:head local-lis/les=113/114 n=419 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.740+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.741+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.741+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:afbdc8ea:::obj55:head local-lis/les=113/114 n=419 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.741+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.741+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.741+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.741+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6014000 con 0x4acb000 2026-02-11T23:31:27.741+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:afd348d8:::obj184:head)]) -- 0x5fcf680 con 0x4acb000 2026-02-11T23:31:27.741+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.741+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 875 ==== MOSDPGPush(1.0 116/113 [PushOp(1:b0395c4b:::obj158:head, version: 111'158, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b0395c4b:::obj158:head@111'158, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56ca1a0 con 0x4acb000 2026-02-11T23:31:27.741+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:b0395c4b:::obj158:head, version: 111'158, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b0395c4b:::obj158:head@111'158, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 116 2026-02-11T23:31:27.741+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b0395c4b:::obj158:head, version: 111'158, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b0395c4b:::obj158:head@111'158, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.741+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.741+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 876 ==== pg_backfill(progress 1.0 e 116/116 lb 1:afd348d8:::obj184:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9ac00 con 0x4acb000 2026-02-11T23:31:27.741+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:afd348d8:::obj184:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.741+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b0395c4b:::obj158:head, version: 111'158, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b0395c4b:::obj158:head@111'158, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.741+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:afd348d8:::obj184:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.741+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.741+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:afd348d8:::obj184:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.741+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.741+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b0395c4b:::obj158:head, version: 111'158, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b0395c4b:::obj158:head@111'158, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.741+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b0395c4b:::obj158:head, version: 111'158, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b0395c4b:::obj158:head@111'158, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000048 MOSDPGPush(1.0 116/113 [PushOp(1:b0395c4b:::obj158:head, version: 111'158, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b0395c4b:::obj158:head@111'158, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:afbdc8ea:::obj55:head local-lis/les=113/114 n=419 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.741+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.741+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:afbdc8ea:::obj55:head local-lis/les=113/114 n=419 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:b0395c4b:::obj158:head, version: 111'158, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b0395c4b:::obj158:head@111'158, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.741+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:afbdc8ea:::obj55:head local-lis/les=113/114 n=419 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b0395c4b:::obj158:head@111'158, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.741+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:afbdc8ea:::obj55:head local-lis/les=113/114 n=419 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:b0395c4b:::obj158:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.741+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:afbdc8ea:::obj55:head local-lis/les=113/114 n=419 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:b0395c4b:::obj158:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.741+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:afbdc8ea:::obj55:head local-lis/les=113/114 n=419 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:b0395c4b:::obj158:head 2026-02-11T23:31:27.741+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:afbdc8ea:::obj55:head local-lis/les=113/114 n=419 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:b0395c4b:::obj158:head v 111'158 2026-02-11T23:31:27.741+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:afbdc8ea:::obj55:head local-lis/les=113/114 n=419 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.741+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.741+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b0395c4b:::obj158:head, version: 111'158, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b0395c4b:::obj158:head@111'158, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.741+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.741+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.741+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:afd348d8:::obj184:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.741+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:afd348d8:::obj184:head) v3 prio 15 cost 0 latency 0.000080 pg_backfill(progress 1.0 e 116/116 lb 1:afd348d8:::obj184:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:afbdc8ea:::obj55:head local-lis/les=113/114 n=419 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.741+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.741+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:afbdc8ea:::obj55:head local-lis/les=113/114 n=419 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:afd348d8:::obj184:head) v3 2026-02-11T23:31:27.741+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:afbdc8ea:::obj55:head local-lis/les=113/114 n=419 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:afd348d8:::obj184:head) 2026-02-11T23:31:27.741+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.741+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:afd348d8:::obj184:head) v3 finish latency 0.000010 2026-02-11T23:31:27.741+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:afd348d8:::obj184:head local-lis/les=113/114 n=420 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.741+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.741+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.741+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:afd348d8:::obj184:head local-lis/les=113/114 n=420 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.741+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.741+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.741+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.741+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5ca0480 con 0x4acb000 2026-02-11T23:31:27.741+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:b0395c4b:::obj158:head)]) -- 0x600a480 con 0x4acb000 2026-02-11T23:31:27.741+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.741+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 877 ==== MOSDPGPush(1.0 116/113 [PushOp(1:b133d16d:::obj222:head, version: 111'222, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b133d16d:::obj222:head@111'222, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56cb040 con 0x4acb000 2026-02-11T23:31:27.741+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:b133d16d:::obj222:head, version: 111'222, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b133d16d:::obj222:head@111'222, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.741+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b133d16d:::obj222:head, version: 111'222, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b133d16d:::obj222:head@111'222, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.741+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.741+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 878 ==== pg_backfill(progress 1.0 e 116/116 lb 1:b0395c4b:::obj158:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9b180 con 0x4acb000 2026-02-11T23:31:27.741+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:b0395c4b:::obj158:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.741+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b133d16d:::obj222:head, version: 111'222, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b133d16d:::obj222:head@111'222, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.741+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b0395c4b:::obj158:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.741+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.741+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b133d16d:::obj222:head, version: 111'222, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b133d16d:::obj222:head@111'222, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.742+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.741+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b133d16d:::obj222:head, version: 111'222, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b133d16d:::obj222:head@111'222, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 116/113 [PushOp(1:b133d16d:::obj222:head, version: 111'222, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b133d16d:::obj222:head@111'222, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:afd348d8:::obj184:head local-lis/les=113/114 n=420 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.742+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b0395c4b:::obj158:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.742+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.742+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:afd348d8:::obj184:head local-lis/les=113/114 n=420 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:b133d16d:::obj222:head, version: 111'222, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b133d16d:::obj222:head@111'222, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.742+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:afd348d8:::obj184:head local-lis/les=113/114 n=420 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b133d16d:::obj222:head@111'222, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.742+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:afd348d8:::obj184:head local-lis/les=113/114 n=420 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:b133d16d:::obj222:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.742+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:afd348d8:::obj184:head local-lis/les=113/114 n=420 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:b133d16d:::obj222:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.742+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:afd348d8:::obj184:head local-lis/les=113/114 n=420 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:b133d16d:::obj222:head 2026-02-11T23:31:27.742+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:afd348d8:::obj184:head local-lis/les=113/114 n=420 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:b133d16d:::obj222:head v 111'222 2026-02-11T23:31:27.742+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:afd348d8:::obj184:head local-lis/les=113/114 n=420 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.742+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.742+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b133d16d:::obj222:head, version: 111'222, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b133d16d:::obj222:head@111'222, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:27.742+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.742+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.742+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b0395c4b:::obj158:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.742+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b0395c4b:::obj158:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 1.0 e 116/116 lb 1:b0395c4b:::obj158:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:afd348d8:::obj184:head local-lis/les=113/114 n=420 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.742+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.742+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:afd348d8:::obj184:head local-lis/les=113/114 n=420 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:b0395c4b:::obj158:head) v3 2026-02-11T23:31:27.742+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:afd348d8:::obj184:head local-lis/les=113/114 n=420 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:b0395c4b:::obj158:head) 2026-02-11T23:31:27.742+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.742+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b0395c4b:::obj158:head) v3 finish latency 0.000010 2026-02-11T23:31:27.742+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b0395c4b:::obj158:head local-lis/les=113/114 n=421 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.742+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.742+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.742+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b0395c4b:::obj158:head local-lis/les=113/114 n=421 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.742+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.742+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.742+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.742+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x612ef00 con 0x4acb000 2026-02-11T23:31:27.742+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:b133d16d:::obj222:head)]) -- 0x5fcf800 con 0x4acb000 2026-02-11T23:31:27.742+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.742+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 879 ==== MOSDPGPush(1.0 116/113 [PushOp(1:b164ceff:::obj421:head, version: 111'421, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b164ceff:::obj421:head@111'421, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d9380 con 0x4acb000 2026-02-11T23:31:27.742+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:b164ceff:::obj421:head, version: 111'421, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b164ceff:::obj421:head@111'421, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 116 2026-02-11T23:31:27.742+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b164ceff:::obj421:head, version: 111'421, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b164ceff:::obj421:head@111'421, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.742+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.742+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 880 ==== pg_backfill(progress 1.0 e 116/116 lb 1:b133d16d:::obj222:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98000 con 0x4acb000 2026-02-11T23:31:27.742+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:b133d16d:::obj222:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.742+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b164ceff:::obj421:head, version: 111'421, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b164ceff:::obj421:head@111'421, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.742+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b133d16d:::obj222:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.742+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.742+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b133d16d:::obj222:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.742+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.742+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b164ceff:::obj421:head, version: 111'421, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b164ceff:::obj421:head@111'421, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.742+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b164ceff:::obj421:head, version: 111'421, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b164ceff:::obj421:head@111'421, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000048 MOSDPGPush(1.0 116/113 [PushOp(1:b164ceff:::obj421:head, version: 111'421, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b164ceff:::obj421:head@111'421, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b0395c4b:::obj158:head local-lis/les=113/114 n=421 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.742+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.742+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b0395c4b:::obj158:head local-lis/les=113/114 n=421 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:b164ceff:::obj421:head, version: 111'421, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b164ceff:::obj421:head@111'421, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.742+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b0395c4b:::obj158:head local-lis/les=113/114 n=421 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b164ceff:::obj421:head@111'421, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.742+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b0395c4b:::obj158:head local-lis/les=113/114 n=421 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:b164ceff:::obj421:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.742+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b0395c4b:::obj158:head local-lis/les=113/114 n=421 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:b164ceff:::obj421:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.742+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b0395c4b:::obj158:head local-lis/les=113/114 n=421 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:b164ceff:::obj421:head 2026-02-11T23:31:27.742+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b0395c4b:::obj158:head local-lis/les=113/114 n=421 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:b164ceff:::obj421:head v 111'421 2026-02-11T23:31:27.742+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b0395c4b:::obj158:head local-lis/les=113/114 n=421 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.742+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.742+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b164ceff:::obj421:head, version: 111'421, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b164ceff:::obj421:head@111'421, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.742+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.742+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.742+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b133d16d:::obj222:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.742+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b133d16d:::obj222:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 1.0 e 116/116 lb 1:b133d16d:::obj222:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b0395c4b:::obj158:head local-lis/les=113/114 n=421 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.742+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.742+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b0395c4b:::obj158:head local-lis/les=113/114 n=421 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:b133d16d:::obj222:head) v3 2026-02-11T23:31:27.742+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b0395c4b:::obj158:head local-lis/les=113/114 n=421 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:b133d16d:::obj222:head) 2026-02-11T23:31:27.742+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.742+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b133d16d:::obj222:head) v3 finish latency 0.000009 2026-02-11T23:31:27.742+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b133d16d:::obj222:head local-lis/les=113/114 n=422 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.742+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.742+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.742+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b133d16d:::obj222:head local-lis/les=113/114 n=422 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.742+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.742+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.742+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.742+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6015b00 con 0x4acb000 2026-02-11T23:31:27.742+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:b164ceff:::obj421:head)]) -- 0x600a900 con 0x4acb000 2026-02-11T23:31:27.742+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.743+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 881 ==== MOSDPGPush(1.0 116/113 [PushOp(1:b1721c5a:::obj526:head, version: 111'526, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1721c5a:::obj526:head@111'526, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d8ea0 con 0x4acb000 2026-02-11T23:31:27.743+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:b1721c5a:::obj526:head, version: 111'526, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1721c5a:::obj526:head@111'526, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.743+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b1721c5a:::obj526:head, version: 111'526, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1721c5a:::obj526:head@111'526, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.743+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.743+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 882 ==== pg_backfill(progress 1.0 e 116/116 lb 1:b164ceff:::obj421:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98580 con 0x4acb000 2026-02-11T23:31:27.743+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:b164ceff:::obj421:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.743+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b1721c5a:::obj526:head, version: 111'526, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1721c5a:::obj526:head@111'526, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.743+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b164ceff:::obj421:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.743+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.743+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b164ceff:::obj421:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.743+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.743+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b1721c5a:::obj526:head, version: 111'526, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1721c5a:::obj526:head@111'526, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.743+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b1721c5a:::obj526:head, version: 111'526, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1721c5a:::obj526:head@111'526, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(1.0 116/113 [PushOp(1:b1721c5a:::obj526:head, version: 111'526, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1721c5a:::obj526:head@111'526, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b133d16d:::obj222:head local-lis/les=113/114 n=422 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.743+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.743+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b133d16d:::obj222:head local-lis/les=113/114 n=422 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:b1721c5a:::obj526:head, version: 111'526, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1721c5a:::obj526:head@111'526, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.743+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b133d16d:::obj222:head local-lis/les=113/114 n=422 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b1721c5a:::obj526:head@111'526, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.743+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b133d16d:::obj222:head local-lis/les=113/114 n=422 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:b1721c5a:::obj526:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.743+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b133d16d:::obj222:head local-lis/les=113/114 n=422 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:b1721c5a:::obj526:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.743+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b133d16d:::obj222:head local-lis/les=113/114 n=422 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:b1721c5a:::obj526:head 2026-02-11T23:31:27.743+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b133d16d:::obj222:head local-lis/les=113/114 n=422 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:b1721c5a:::obj526:head v 111'526 2026-02-11T23:31:27.743+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b133d16d:::obj222:head local-lis/les=113/114 n=422 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.743+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.743+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b1721c5a:::obj526:head, version: 111'526, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1721c5a:::obj526:head@111'526, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:27.743+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.743+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.743+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b164ceff:::obj421:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.743+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b164ceff:::obj421:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 1.0 e 116/116 lb 1:b164ceff:::obj421:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b133d16d:::obj222:head local-lis/les=113/114 n=422 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.743+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.743+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b133d16d:::obj222:head local-lis/les=113/114 n=422 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:b164ceff:::obj421:head) v3 2026-02-11T23:31:27.743+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b133d16d:::obj222:head local-lis/les=113/114 n=422 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:b164ceff:::obj421:head) 2026-02-11T23:31:27.743+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.743+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b164ceff:::obj421:head) v3 finish latency 0.000009 2026-02-11T23:31:27.743+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b164ceff:::obj421:head local-lis/les=113/114 n=423 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.743+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.743+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.743+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b164ceff:::obj421:head local-lis/les=113/114 n=423 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.743+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.743+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.743+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.743+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5ca1980 con 0x4acb000 2026-02-11T23:31:27.743+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:b1721c5a:::obj526:head)]) -- 0x5fcfb00 con 0x4acb000 2026-02-11T23:31:27.743+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.743+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 883 ==== MOSDPGPush(1.0 116/113 [PushOp(1:b1ca68cf:::obj358:head, version: 111'358, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1ca68cf:::obj358:head@111'358, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5647380 con 0x4acb000 2026-02-11T23:31:27.743+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:b1ca68cf:::obj358:head, version: 111'358, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1ca68cf:::obj358:head@111'358, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.743+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b1ca68cf:::obj358:head, version: 111'358, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1ca68cf:::obj358:head@111'358, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.743+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.743+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 884 ==== pg_backfill(progress 1.0 e 116/116 lb 1:b1721c5a:::obj526:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98b00 con 0x4acb000 2026-02-11T23:31:27.743+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:b1721c5a:::obj526:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.743+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b1ca68cf:::obj358:head, version: 111'358, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1ca68cf:::obj358:head@111'358, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.743+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b1721c5a:::obj526:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.743+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.743+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b1721c5a:::obj526:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.743+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.743+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b1ca68cf:::obj358:head, version: 111'358, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1ca68cf:::obj358:head@111'358, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.743+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b1ca68cf:::obj358:head, version: 111'358, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1ca68cf:::obj358:head@111'358, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000043 MOSDPGPush(1.0 116/113 [PushOp(1:b1ca68cf:::obj358:head, version: 111'358, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1ca68cf:::obj358:head@111'358, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b164ceff:::obj421:head local-lis/les=113/114 n=423 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.743+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.743+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b164ceff:::obj421:head local-lis/les=113/114 n=423 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:b1ca68cf:::obj358:head, version: 111'358, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1ca68cf:::obj358:head@111'358, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.743+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b164ceff:::obj421:head local-lis/les=113/114 n=423 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b1ca68cf:::obj358:head@111'358, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.743+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b164ceff:::obj421:head local-lis/les=113/114 n=423 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:b1ca68cf:::obj358:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.743+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b164ceff:::obj421:head local-lis/les=113/114 n=423 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:b1ca68cf:::obj358:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.743+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b164ceff:::obj421:head local-lis/les=113/114 n=423 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:b1ca68cf:::obj358:head 2026-02-11T23:31:27.743+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b164ceff:::obj421:head local-lis/les=113/114 n=423 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:b1ca68cf:::obj358:head v 111'358 2026-02-11T23:31:27.743+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b164ceff:::obj421:head local-lis/les=113/114 n=423 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.743+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.743+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b1ca68cf:::obj358:head, version: 111'358, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1ca68cf:::obj358:head@111'358, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:27.743+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.743+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.743+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b1721c5a:::obj526:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.743+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b1721c5a:::obj526:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 1.0 e 116/116 lb 1:b1721c5a:::obj526:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b164ceff:::obj421:head local-lis/les=113/114 n=423 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.743+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.743+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b164ceff:::obj421:head local-lis/les=113/114 n=423 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:b1721c5a:::obj526:head) v3 2026-02-11T23:31:27.743+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b164ceff:::obj421:head local-lis/les=113/114 n=423 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:b1721c5a:::obj526:head) 2026-02-11T23:31:27.743+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.743+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b1721c5a:::obj526:head) v3 finish latency 0.000012 2026-02-11T23:31:27.743+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b1721c5a:::obj526:head local-lis/les=113/114 n=424 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.743+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.743+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.743+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b1721c5a:::obj526:head local-lis/les=113/114 n=424 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.743+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.743+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.743+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.743+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fed680 con 0x4acb000 2026-02-11T23:31:27.743+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:b1ca68cf:::obj358:head)]) -- 0x600be00 con 0x4acb000 2026-02-11T23:31:27.743+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.744+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 885 ==== MOSDPGPush(1.0 116/113 [PushOp(1:b1e2ea00:::obj289:head, version: 111'289, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1e2ea00:::obj289:head@111'289, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d84e0 con 0x4acb000 2026-02-11T23:31:27.744+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:b1e2ea00:::obj289:head, version: 111'289, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1e2ea00:::obj289:head@111'289, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 116 2026-02-11T23:31:27.744+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b1e2ea00:::obj289:head, version: 111'289, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1e2ea00:::obj289:head@111'289, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.744+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.744+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 886 ==== pg_backfill(progress 1.0 e 116/116 lb 1:b1ca68cf:::obj358:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99080 con 0x4acb000 2026-02-11T23:31:27.744+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b1e2ea00:::obj289:head, version: 111'289, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1e2ea00:::obj289:head@111'289, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.744+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:b1ca68cf:::obj358:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.744+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.744+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b1ca68cf:::obj358:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.744+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b1e2ea00:::obj289:head, version: 111'289, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1e2ea00:::obj289:head@111'289, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.744+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.744+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b1e2ea00:::obj289:head, version: 111'289, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1e2ea00:::obj289:head@111'289, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 116/113 [PushOp(1:b1e2ea00:::obj289:head, version: 111'289, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1e2ea00:::obj289:head@111'289, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b1721c5a:::obj526:head local-lis/les=113/114 n=424 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.744+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.744+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b1721c5a:::obj526:head local-lis/les=113/114 n=424 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:b1e2ea00:::obj289:head, version: 111'289, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1e2ea00:::obj289:head@111'289, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.744+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b1721c5a:::obj526:head local-lis/les=113/114 n=424 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b1e2ea00:::obj289:head@111'289, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.744+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.744+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b1ca68cf:::obj358:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.744+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b1721c5a:::obj526:head local-lis/les=113/114 n=424 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:b1e2ea00:::obj289:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.744+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b1721c5a:::obj526:head local-lis/les=113/114 n=424 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:b1e2ea00:::obj289:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.744+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b1721c5a:::obj526:head local-lis/les=113/114 n=424 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:b1e2ea00:::obj289:head 2026-02-11T23:31:27.744+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b1721c5a:::obj526:head local-lis/les=113/114 n=424 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:b1e2ea00:::obj289:head v 111'289 2026-02-11T23:31:27.744+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b1721c5a:::obj526:head local-lis/les=113/114 n=424 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.744+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.744+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b1e2ea00:::obj289:head, version: 111'289, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1e2ea00:::obj289:head@111'289, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.744+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.744+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.744+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b1ca68cf:::obj358:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.744+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b1ca68cf:::obj358:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 1.0 e 116/116 lb 1:b1ca68cf:::obj358:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b1721c5a:::obj526:head local-lis/les=113/114 n=424 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.744+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.744+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b1721c5a:::obj526:head local-lis/les=113/114 n=424 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:b1ca68cf:::obj358:head) v3 2026-02-11T23:31:27.744+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b1721c5a:::obj526:head local-lis/les=113/114 n=424 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:b1ca68cf:::obj358:head) 2026-02-11T23:31:27.744+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.744+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b1ca68cf:::obj358:head) v3 finish latency 0.000009 2026-02-11T23:31:27.744+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b1ca68cf:::obj358:head local-lis/les=113/114 n=425 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.744+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.744+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.744+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b1ca68cf:::obj358:head local-lis/les=113/114 n=425 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.744+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.744+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.744+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.744+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5ca0180 con 0x4acb000 2026-02-11T23:31:27.744+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:b1e2ea00:::obj289:head)]) -- 0x600bc80 con 0x4acb000 2026-02-11T23:31:27.744+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.744+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 887 ==== MOSDPGPush(1.0 116/113 [PushOp(1:b26ae718:::obj317:head, version: 111'317, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b26ae718:::obj317:head@111'317, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d8000 con 0x4acb000 2026-02-11T23:31:27.744+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:b26ae718:::obj317:head, version: 111'317, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b26ae718:::obj317:head@111'317, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.744+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b26ae718:::obj317:head, version: 111'317, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b26ae718:::obj317:head@111'317, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.744+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.744+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 888 ==== pg_backfill(progress 1.0 e 116/116 lb 1:b1e2ea00:::obj289:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99600 con 0x4acb000 2026-02-11T23:31:27.744+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b26ae718:::obj317:head, version: 111'317, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b26ae718:::obj317:head@111'317, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.744+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:b1e2ea00:::obj289:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.744+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b1e2ea00:::obj289:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.744+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.744+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b26ae718:::obj317:head, version: 111'317, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b26ae718:::obj317:head@111'317, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.744+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b26ae718:::obj317:head, version: 111'317, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b26ae718:::obj317:head@111'317, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 116/113 [PushOp(1:b26ae718:::obj317:head, version: 111'317, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b26ae718:::obj317:head@111'317, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b1ca68cf:::obj358:head local-lis/les=113/114 n=425 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.744+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.744+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.744+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b1ca68cf:::obj358:head local-lis/les=113/114 n=425 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:b26ae718:::obj317:head, version: 111'317, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b26ae718:::obj317:head@111'317, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.744+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b1e2ea00:::obj289:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.744+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b1ca68cf:::obj358:head local-lis/les=113/114 n=425 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b26ae718:::obj317:head@111'317, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.744+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b1ca68cf:::obj358:head local-lis/les=113/114 n=425 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:b26ae718:::obj317:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.744+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b1ca68cf:::obj358:head local-lis/les=113/114 n=425 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:b26ae718:::obj317:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.744+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b1ca68cf:::obj358:head local-lis/les=113/114 n=425 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:b26ae718:::obj317:head 2026-02-11T23:31:27.744+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b1ca68cf:::obj358:head local-lis/les=113/114 n=425 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:b26ae718:::obj317:head v 111'317 2026-02-11T23:31:27.744+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b1ca68cf:::obj358:head local-lis/les=113/114 n=425 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.744+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.744+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b26ae718:::obj317:head, version: 111'317, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b26ae718:::obj317:head@111'317, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.744+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.744+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.744+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b1e2ea00:::obj289:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.744+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b1e2ea00:::obj289:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:b1e2ea00:::obj289:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b1ca68cf:::obj358:head local-lis/les=113/114 n=425 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.744+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.744+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b1ca68cf:::obj358:head local-lis/les=113/114 n=425 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:b1e2ea00:::obj289:head) v3 2026-02-11T23:31:27.744+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b1ca68cf:::obj358:head local-lis/les=113/114 n=425 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:b1e2ea00:::obj289:head) 2026-02-11T23:31:27.744+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.745+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b1e2ea00:::obj289:head) v3 finish latency 0.000012 2026-02-11T23:31:27.745+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b1e2ea00:::obj289:head local-lis/les=113/114 n=426 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.745+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.745+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.745+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b1e2ea00:::obj289:head local-lis/les=113/114 n=426 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.745+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.745+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.745+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.745+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6170900 con 0x4acb000 2026-02-11T23:31:27.745+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:b26ae718:::obj317:head)]) -- 0x600b800 con 0x4acb000 2026-02-11T23:31:27.745+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.745+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 889 ==== MOSDPGPush(1.0 116/113 [PushOp(1:b2b26856:::obj586:head, version: 111'586, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b2b26856:::obj586:head@111'586, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d89c0 con 0x4acb000 2026-02-11T23:31:27.745+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:b2b26856:::obj586:head, version: 111'586, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b2b26856:::obj586:head@111'586, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 116 2026-02-11T23:31:27.745+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b2b26856:::obj586:head, version: 111'586, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b2b26856:::obj586:head@111'586, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.745+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.745+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 890 ==== pg_backfill(progress 1.0 e 116/116 lb 1:b26ae718:::obj317:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99b80 con 0x4acb000 2026-02-11T23:31:27.745+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:b26ae718:::obj317:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.745+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b2b26856:::obj586:head, version: 111'586, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b2b26856:::obj586:head@111'586, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.745+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b26ae718:::obj317:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.745+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.745+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b2b26856:::obj586:head, version: 111'586, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b2b26856:::obj586:head@111'586, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.745+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b2b26856:::obj586:head, version: 111'586, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b2b26856:::obj586:head@111'586, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(1.0 116/113 [PushOp(1:b2b26856:::obj586:head, version: 111'586, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b2b26856:::obj586:head@111'586, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b1e2ea00:::obj289:head local-lis/les=113/114 n=426 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.745+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.745+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.745+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b1e2ea00:::obj289:head local-lis/les=113/114 n=426 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:b2b26856:::obj586:head, version: 111'586, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b2b26856:::obj586:head@111'586, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.745+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b26ae718:::obj317:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.745+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b1e2ea00:::obj289:head local-lis/les=113/114 n=426 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b2b26856:::obj586:head@111'586, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.745+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b1e2ea00:::obj289:head local-lis/les=113/114 n=426 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:b2b26856:::obj586:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.745+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b1e2ea00:::obj289:head local-lis/les=113/114 n=426 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:b2b26856:::obj586:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.745+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b1e2ea00:::obj289:head local-lis/les=113/114 n=426 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:b2b26856:::obj586:head 2026-02-11T23:31:27.745+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b1e2ea00:::obj289:head local-lis/les=113/114 n=426 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:b2b26856:::obj586:head v 111'586 2026-02-11T23:31:27.745+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b1e2ea00:::obj289:head local-lis/les=113/114 n=426 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.745+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.745+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b2b26856:::obj586:head, version: 111'586, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b2b26856:::obj586:head@111'586, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.745+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.745+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.745+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b26ae718:::obj317:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.745+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b26ae718:::obj317:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 1.0 e 116/116 lb 1:b26ae718:::obj317:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b1e2ea00:::obj289:head local-lis/les=113/114 n=426 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.745+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.745+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b1e2ea00:::obj289:head local-lis/les=113/114 n=426 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:b26ae718:::obj317:head) v3 2026-02-11T23:31:27.745+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b1e2ea00:::obj289:head local-lis/les=113/114 n=426 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:b26ae718:::obj317:head) 2026-02-11T23:31:27.745+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.745+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b26ae718:::obj317:head) v3 finish latency 0.000011 2026-02-11T23:31:27.745+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b26ae718:::obj317:head local-lis/les=113/114 n=427 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.745+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.745+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.745+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b26ae718:::obj317:head local-lis/les=113/114 n=427 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.745+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.745+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.745+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.745+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6005380 con 0x4acb000 2026-02-11T23:31:27.745+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:b2b26856:::obj586:head)]) -- 0x600b680 con 0x4acb000 2026-02-11T23:31:27.745+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.745+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 891 ==== MOSDPGPush(1.0 116/113 [PushOp(1:b2ec4718:::obj500:head, version: 111'500, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b2ec4718:::obj500:head@111'500, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559d040 con 0x4acb000 2026-02-11T23:31:27.745+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:b2ec4718:::obj500:head, version: 111'500, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b2ec4718:::obj500:head@111'500, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 116 2026-02-11T23:31:27.745+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b2ec4718:::obj500:head, version: 111'500, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b2ec4718:::obj500:head@111'500, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.745+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.745+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 892 ==== pg_backfill(progress 1.0 e 116/116 lb 1:b2b26856:::obj586:head) ==== 1062+0+0 (unknown 0 0 0) 0x4a2cb00 con 0x4acb000 2026-02-11T23:31:27.745+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b2ec4718:::obj500:head, version: 111'500, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b2ec4718:::obj500:head@111'500, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.745+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.745+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:b2b26856:::obj586:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.746+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b2ec4718:::obj500:head, version: 111'500, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b2ec4718:::obj500:head@111'500, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.746+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b2b26856:::obj586:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.746+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.746+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b2b26856:::obj586:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.746+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b2ec4718:::obj500:head, version: 111'500, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b2ec4718:::obj500:head@111'500, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(1.0 116/113 [PushOp(1:b2ec4718:::obj500:head, version: 111'500, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b2ec4718:::obj500:head@111'500, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b26ae718:::obj317:head local-lis/les=113/114 n=427 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.746+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.746+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b26ae718:::obj317:head local-lis/les=113/114 n=427 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:b2ec4718:::obj500:head, version: 111'500, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b2ec4718:::obj500:head@111'500, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.746+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b26ae718:::obj317:head local-lis/les=113/114 n=427 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b2ec4718:::obj500:head@111'500, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.746+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b26ae718:::obj317:head local-lis/les=113/114 n=427 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:b2ec4718:::obj500:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.746+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b26ae718:::obj317:head local-lis/les=113/114 n=427 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:b2ec4718:::obj500:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.746+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b26ae718:::obj317:head local-lis/les=113/114 n=427 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:b2ec4718:::obj500:head 2026-02-11T23:31:27.746+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b26ae718:::obj317:head local-lis/les=113/114 n=427 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:b2ec4718:::obj500:head v 111'500 2026-02-11T23:31:27.746+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b26ae718:::obj317:head local-lis/les=113/114 n=427 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.746+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.746+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b2ec4718:::obj500:head, version: 111'500, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b2ec4718:::obj500:head@111'500, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:27.746+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.746+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.746+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b2b26856:::obj586:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.746+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b2b26856:::obj586:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 1.0 e 116/116 lb 1:b2b26856:::obj586:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b26ae718:::obj317:head local-lis/les=113/114 n=427 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.746+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.746+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b26ae718:::obj317:head local-lis/les=113/114 n=427 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:b2b26856:::obj586:head) v3 2026-02-11T23:31:27.746+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b26ae718:::obj317:head local-lis/les=113/114 n=427 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:b2b26856:::obj586:head) 2026-02-11T23:31:27.746+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.746+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b2b26856:::obj586:head) v3 finish latency 0.000009 2026-02-11T23:31:27.746+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b2b26856:::obj586:head local-lis/les=113/114 n=428 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.746+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.746+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b2b26856:::obj586:head local-lis/les=113/114 n=428 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.746+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.746+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.746+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.746+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.746+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fed380 con 0x4acb000 2026-02-11T23:31:27.746+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:b2ec4718:::obj500:head)]) -- 0x600b500 con 0x4acb000 2026-02-11T23:31:27.746+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.746+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 893 ==== MOSDPGPush(1.0 116/113 [PushOp(1:b33086a8:::obj36:head, version: 111'36, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b33086a8:::obj36:head@111'36, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x559dba0 con 0x4acb000 2026-02-11T23:31:27.746+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:b33086a8:::obj36:head, version: 111'36, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b33086a8:::obj36:head@111'36, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.746+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b33086a8:::obj36:head, version: 111'36, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b33086a8:::obj36:head@111'36, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.746+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.746+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 894 ==== pg_backfill(progress 1.0 e 116/116 lb 1:b2ec4718:::obj500:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2d080 con 0x4acb000 2026-02-11T23:31:27.746+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:b2ec4718:::obj500:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.746+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b33086a8:::obj36:head, version: 111'36, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b33086a8:::obj36:head@111'36, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.746+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b2ec4718:::obj500:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.746+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.746+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b33086a8:::obj36:head, version: 111'36, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b33086a8:::obj36:head@111'36, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.746+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b33086a8:::obj36:head, version: 111'36, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b33086a8:::obj36:head@111'36, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 116/113 [PushOp(1:b33086a8:::obj36:head, version: 111'36, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b33086a8:::obj36:head@111'36, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b2b26856:::obj586:head local-lis/les=113/114 n=428 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.746+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.746+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.746+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b2ec4718:::obj500:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.746+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b2b26856:::obj586:head local-lis/les=113/114 n=428 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:b33086a8:::obj36:head, version: 111'36, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b33086a8:::obj36:head@111'36, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.746+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b2b26856:::obj586:head local-lis/les=113/114 n=428 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b33086a8:::obj36:head@111'36, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.746+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b2b26856:::obj586:head local-lis/les=113/114 n=428 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:b33086a8:::obj36:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.746+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b2b26856:::obj586:head local-lis/les=113/114 n=428 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:b33086a8:::obj36:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.746+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b2b26856:::obj586:head local-lis/les=113/114 n=428 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:b33086a8:::obj36:head 2026-02-11T23:31:27.746+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b2b26856:::obj586:head local-lis/les=113/114 n=428 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:b33086a8:::obj36:head v 111'36 2026-02-11T23:31:27.746+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b2b26856:::obj586:head local-lis/les=113/114 n=428 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.746+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.746+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b33086a8:::obj36:head, version: 111'36, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b33086a8:::obj36:head@111'36, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.746+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.746+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.746+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b2ec4718:::obj500:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.746+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b2ec4718:::obj500:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 1.0 e 116/116 lb 1:b2ec4718:::obj500:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b2b26856:::obj586:head local-lis/les=113/114 n=428 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.746+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.746+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b2b26856:::obj586:head local-lis/les=113/114 n=428 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:b2ec4718:::obj500:head) v3 2026-02-11T23:31:27.746+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b2b26856:::obj586:head local-lis/les=113/114 n=428 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:b2ec4718:::obj500:head) 2026-02-11T23:31:27.746+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.746+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b2ec4718:::obj500:head) v3 finish latency 0.000009 2026-02-11T23:31:27.746+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b2ec4718:::obj500:head local-lis/les=113/114 n=429 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.746+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.746+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.746+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b2ec4718:::obj500:head local-lis/les=113/114 n=429 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.746+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.746+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.746+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.746+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d1d680 con 0x4acb000 2026-02-11T23:31:27.746+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:b33086a8:::obj36:head)]) -- 0x600b380 con 0x4acb000 2026-02-11T23:31:27.746+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.747+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 895 ==== MOSDPGPush(1.0 116/113 [PushOp(1:b4726009:::obj503:head, version: 111'503, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b4726009:::obj503:head@111'503, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c680 con 0x4acb000 2026-02-11T23:31:27.747+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:b4726009:::obj503:head, version: 111'503, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b4726009:::obj503:head@111'503, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 116 2026-02-11T23:31:27.747+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b4726009:::obj503:head, version: 111'503, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b4726009:::obj503:head@111'503, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.747+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.747+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b4726009:::obj503:head, version: 111'503, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b4726009:::obj503:head@111'503, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.747+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 896 ==== pg_backfill(progress 1.0 e 116/116 lb 1:b33086a8:::obj36:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d2e680 con 0x4acb000 2026-02-11T23:31:27.747+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.747+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:b33086a8:::obj36:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.747+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b4726009:::obj503:head, version: 111'503, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b4726009:::obj503:head@111'503, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.747+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b33086a8:::obj36:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.747+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b4726009:::obj503:head, version: 111'503, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b4726009:::obj503:head@111'503, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000041 MOSDPGPush(1.0 116/113 [PushOp(1:b4726009:::obj503:head, version: 111'503, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b4726009:::obj503:head@111'503, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b2ec4718:::obj500:head local-lis/les=113/114 n=429 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.747+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.747+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.747+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b2ec4718:::obj500:head local-lis/les=113/114 n=429 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:b4726009:::obj503:head, version: 111'503, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b4726009:::obj503:head@111'503, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.747+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b33086a8:::obj36:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.747+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b2ec4718:::obj500:head local-lis/les=113/114 n=429 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b4726009:::obj503:head@111'503, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.747+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b2ec4718:::obj500:head local-lis/les=113/114 n=429 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:b4726009:::obj503:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.747+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b2ec4718:::obj500:head local-lis/les=113/114 n=429 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:b4726009:::obj503:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.747+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b2ec4718:::obj500:head local-lis/les=113/114 n=429 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:b4726009:::obj503:head 2026-02-11T23:31:27.747+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b2ec4718:::obj500:head local-lis/les=113/114 n=429 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:b4726009:::obj503:head v 111'503 2026-02-11T23:31:27.747+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b2ec4718:::obj500:head local-lis/les=113/114 n=429 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.747+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.747+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b4726009:::obj503:head, version: 111'503, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b4726009:::obj503:head@111'503, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:27.747+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.747+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.747+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b33086a8:::obj36:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.747+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b33086a8:::obj36:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:b33086a8:::obj36:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b2ec4718:::obj500:head local-lis/les=113/114 n=429 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.747+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.747+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b2ec4718:::obj500:head local-lis/les=113/114 n=429 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:b33086a8:::obj36:head) v3 2026-02-11T23:31:27.747+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b2ec4718:::obj500:head local-lis/les=113/114 n=429 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:b33086a8:::obj36:head) 2026-02-11T23:31:27.747+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.747+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b33086a8:::obj36:head) v3 finish latency 0.000009 2026-02-11T23:31:27.747+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b33086a8:::obj36:head local-lis/les=113/114 n=430 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.747+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.747+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.747+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b33086a8:::obj36:head local-lis/les=113/114 n=430 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.747+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.747+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.747+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.747+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6015800 con 0x4acb000 2026-02-11T23:31:27.747+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:b4726009:::obj503:head)]) -- 0x6000180 con 0x4acb000 2026-02-11T23:31:27.747+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.747+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 897 ==== MOSDPGPush(1.0 116/113 [PushOp(1:b5298834:::obj386:head, version: 111'386, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5298834:::obj386:head@111'386, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f16c0 con 0x4acb000 2026-02-11T23:31:27.747+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:b5298834:::obj386:head, version: 111'386, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5298834:::obj386:head@111'386, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000015 epoch 116 2026-02-11T23:31:27.747+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b5298834:::obj386:head, version: 111'386, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5298834:::obj386:head@111'386, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.747+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.747+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 898 ==== pg_backfill(progress 1.0 e 116/116 lb 1:b4726009:::obj503:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e100 con 0x4acb000 2026-02-11T23:31:27.747+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b5298834:::obj386:head, version: 111'386, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5298834:::obj386:head@111'386, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.747+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:b4726009:::obj503:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.747+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b4726009:::obj503:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.747+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.747+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b5298834:::obj386:head, version: 111'386, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5298834:::obj386:head@111'386, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.747+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.747+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b5298834:::obj386:head, version: 111'386, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5298834:::obj386:head@111'386, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(1.0 116/113 [PushOp(1:b5298834:::obj386:head, version: 111'386, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5298834:::obj386:head@111'386, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b33086a8:::obj36:head local-lis/les=113/114 n=430 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.747+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.747+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b33086a8:::obj36:head local-lis/les=113/114 n=430 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:b5298834:::obj386:head, version: 111'386, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5298834:::obj386:head@111'386, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.747+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b33086a8:::obj36:head local-lis/les=113/114 n=430 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b5298834:::obj386:head@111'386, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.747+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.747+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b4726009:::obj503:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.747+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b33086a8:::obj36:head local-lis/les=113/114 n=430 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:b5298834:::obj386:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.747+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b33086a8:::obj36:head local-lis/les=113/114 n=430 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:b5298834:::obj386:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.747+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b33086a8:::obj36:head local-lis/les=113/114 n=430 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:b5298834:::obj386:head 2026-02-11T23:31:27.747+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b33086a8:::obj36:head local-lis/les=113/114 n=430 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:b5298834:::obj386:head v 111'386 2026-02-11T23:31:27.747+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b33086a8:::obj36:head local-lis/les=113/114 n=430 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.747+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.747+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b5298834:::obj386:head, version: 111'386, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5298834:::obj386:head@111'386, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.747+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.747+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.747+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b4726009:::obj503:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.747+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b4726009:::obj503:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 1.0 e 116/116 lb 1:b4726009:::obj503:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b33086a8:::obj36:head local-lis/les=113/114 n=430 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.747+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.747+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b33086a8:::obj36:head local-lis/les=113/114 n=430 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:b4726009:::obj503:head) v3 2026-02-11T23:31:27.747+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b33086a8:::obj36:head local-lis/les=113/114 n=430 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:b4726009:::obj503:head) 2026-02-11T23:31:27.747+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.747+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b4726009:::obj503:head) v3 finish latency 0.000009 2026-02-11T23:31:27.747+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b4726009:::obj503:head local-lis/les=113/114 n=431 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.747+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.747+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.747+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b4726009:::obj503:head local-lis/les=113/114 n=431 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.747+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.747+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.747+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.747+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5ca0a80 con 0x4acb000 2026-02-11T23:31:27.747+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:b5298834:::obj386:head)]) -- 0x600ad80 con 0x4acb000 2026-02-11T23:31:27.747+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.748+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 899 ==== MOSDPGPush(1.0 116/113 [PushOp(1:b5339e65:::obj355:head, version: 111'355, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5339e65:::obj355:head@111'355, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x555d040 con 0x4acb000 2026-02-11T23:31:27.748+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:b5339e65:::obj355:head, version: 111'355, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5339e65:::obj355:head@111'355, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.748+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b5339e65:::obj355:head, version: 111'355, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5339e65:::obj355:head@111'355, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.748+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.748+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 900 ==== pg_backfill(progress 1.0 e 116/116 lb 1:b5298834:::obj386:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2c000 con 0x4acb000 2026-02-11T23:31:27.748+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b5339e65:::obj355:head, version: 111'355, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5339e65:::obj355:head@111'355, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.748+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:b5298834:::obj386:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.748+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.748+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b5298834:::obj386:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.748+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b5339e65:::obj355:head, version: 111'355, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5339e65:::obj355:head@111'355, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.748+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.748+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b5339e65:::obj355:head, version: 111'355, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5339e65:::obj355:head@111'355, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:b5339e65:::obj355:head, version: 111'355, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5339e65:::obj355:head@111'355, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b4726009:::obj503:head local-lis/les=113/114 n=431 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.748+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.748+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b4726009:::obj503:head local-lis/les=113/114 n=431 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:b5339e65:::obj355:head, version: 111'355, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5339e65:::obj355:head@111'355, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.748+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b4726009:::obj503:head local-lis/les=113/114 n=431 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b5339e65:::obj355:head@111'355, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.748+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.748+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b5298834:::obj386:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.748+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b4726009:::obj503:head local-lis/les=113/114 n=431 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:b5339e65:::obj355:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.748+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b4726009:::obj503:head local-lis/les=113/114 n=431 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:b5339e65:::obj355:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.748+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b4726009:::obj503:head local-lis/les=113/114 n=431 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:b5339e65:::obj355:head 2026-02-11T23:31:27.748+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b4726009:::obj503:head local-lis/les=113/114 n=431 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:b5339e65:::obj355:head v 111'355 2026-02-11T23:31:27.748+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b4726009:::obj503:head local-lis/les=113/114 n=431 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.748+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.748+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b5339e65:::obj355:head, version: 111'355, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5339e65:::obj355:head@111'355, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.748+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.748+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.748+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b5298834:::obj386:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.748+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b5298834:::obj386:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:b5298834:::obj386:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b4726009:::obj503:head local-lis/les=113/114 n=431 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.748+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.748+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b4726009:::obj503:head local-lis/les=113/114 n=431 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:b5298834:::obj386:head) v3 2026-02-11T23:31:27.748+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b4726009:::obj503:head local-lis/les=113/114 n=431 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:b5298834:::obj386:head) 2026-02-11T23:31:27.748+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.748+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b5298834:::obj386:head) v3 finish latency 0.000010 2026-02-11T23:31:27.748+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b5298834:::obj386:head local-lis/les=113/114 n=432 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.748+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.748+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.748+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b5298834:::obj386:head local-lis/les=113/114 n=432 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.748+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.748+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.748+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.748+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x612e900 con 0x4acb000 2026-02-11T23:31:27.748+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:b5339e65:::obj355:head)]) -- 0x5fcfe00 con 0x4acb000 2026-02-11T23:31:27.748+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.748+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 901 ==== MOSDPGPush(1.0 116/113 [PushOp(1:b5445b0d:::obj455:head, version: 111'455, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5445b0d:::obj455:head@111'455, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f0ea0 con 0x4acb000 2026-02-11T23:31:27.748+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:b5445b0d:::obj455:head, version: 111'455, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5445b0d:::obj455:head@111'455, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 116 2026-02-11T23:31:27.748+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b5445b0d:::obj455:head, version: 111'455, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5445b0d:::obj455:head@111'455, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.748+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.748+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 902 ==== pg_backfill(progress 1.0 e 116/116 lb 1:b5339e65:::obj355:head) ==== 1062+0+0 (unknown 0 0 0) 0x6179700 con 0x4acb000 2026-02-11T23:31:27.748+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:b5339e65:::obj355:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.748+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b5445b0d:::obj455:head, version: 111'455, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5445b0d:::obj455:head@111'455, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.748+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b5339e65:::obj355:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.748+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.748+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b5445b0d:::obj455:head, version: 111'455, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5445b0d:::obj455:head@111'455, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.748+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b5445b0d:::obj455:head, version: 111'455, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5445b0d:::obj455:head@111'455, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(1.0 116/113 [PushOp(1:b5445b0d:::obj455:head, version: 111'455, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5445b0d:::obj455:head@111'455, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b5298834:::obj386:head local-lis/les=113/114 n=432 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.748+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.748+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.748+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b5298834:::obj386:head local-lis/les=113/114 n=432 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:b5445b0d:::obj455:head, version: 111'455, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5445b0d:::obj455:head@111'455, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.748+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b5339e65:::obj355:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.748+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b5298834:::obj386:head local-lis/les=113/114 n=432 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b5445b0d:::obj455:head@111'455, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.748+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b5298834:::obj386:head local-lis/les=113/114 n=432 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:b5445b0d:::obj455:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.748+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b5298834:::obj386:head local-lis/les=113/114 n=432 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:b5445b0d:::obj455:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.748+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b5298834:::obj386:head local-lis/les=113/114 n=432 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:b5445b0d:::obj455:head 2026-02-11T23:31:27.748+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b5298834:::obj386:head local-lis/les=113/114 n=432 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:b5445b0d:::obj455:head v 111'455 2026-02-11T23:31:27.748+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b5298834:::obj386:head local-lis/les=113/114 n=432 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.748+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.748+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b5445b0d:::obj455:head, version: 111'455, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5445b0d:::obj455:head@111'455, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.748+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.748+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.748+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b5339e65:::obj355:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.748+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b5339e65:::obj355:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 116/116 lb 1:b5339e65:::obj355:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b5298834:::obj386:head local-lis/les=113/114 n=432 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.748+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.748+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b5298834:::obj386:head local-lis/les=113/114 n=432 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:b5339e65:::obj355:head) v3 2026-02-11T23:31:27.748+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b5298834:::obj386:head local-lis/les=113/114 n=432 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:b5339e65:::obj355:head) 2026-02-11T23:31:27.748+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.748+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b5339e65:::obj355:head) v3 finish latency 0.000010 2026-02-11T23:31:27.748+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b5339e65:::obj355:head local-lis/les=113/114 n=433 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.748+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.748+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.748+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b5339e65:::obj355:head local-lis/les=113/114 n=433 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.748+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.748+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.748+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.748+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x612e000 con 0x4acb000 2026-02-11T23:31:27.748+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:b5445b0d:::obj455:head)]) -- 0x600b800 con 0x4acb000 2026-02-11T23:31:27.748+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.749+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 903 ==== MOSDPGPush(1.0 116/113 [PushOp(1:b59cea89:::obj183:head, version: 111'183, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b59cea89:::obj183:head@111'183, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e26680 con 0x4acb000 2026-02-11T23:31:27.749+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:b59cea89:::obj183:head, version: 111'183, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b59cea89:::obj183:head@111'183, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 116 2026-02-11T23:31:27.749+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b59cea89:::obj183:head, version: 111'183, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b59cea89:::obj183:head@111'183, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.749+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.749+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 904 ==== pg_backfill(progress 1.0 e 116/116 lb 1:b5445b0d:::obj455:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176000 con 0x4acb000 2026-02-11T23:31:27.749+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:b5445b0d:::obj455:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.749+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b59cea89:::obj183:head, version: 111'183, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b59cea89:::obj183:head@111'183, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.749+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b5445b0d:::obj455:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.749+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.749+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b59cea89:::obj183:head, version: 111'183, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b59cea89:::obj183:head@111'183, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.749+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b59cea89:::obj183:head, version: 111'183, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b59cea89:::obj183:head@111'183, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 116/113 [PushOp(1:b59cea89:::obj183:head, version: 111'183, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b59cea89:::obj183:head@111'183, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b5339e65:::obj355:head local-lis/les=113/114 n=433 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.749+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.749+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.749+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b5339e65:::obj355:head local-lis/les=113/114 n=433 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:b59cea89:::obj183:head, version: 111'183, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b59cea89:::obj183:head@111'183, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.749+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b5445b0d:::obj455:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.749+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b5339e65:::obj355:head local-lis/les=113/114 n=433 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b59cea89:::obj183:head@111'183, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.749+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b5339e65:::obj355:head local-lis/les=113/114 n=433 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:b59cea89:::obj183:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.749+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b5339e65:::obj355:head local-lis/les=113/114 n=433 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:b59cea89:::obj183:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.749+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b5339e65:::obj355:head local-lis/les=113/114 n=433 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:b59cea89:::obj183:head 2026-02-11T23:31:27.749+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b5339e65:::obj355:head local-lis/les=113/114 n=433 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:b59cea89:::obj183:head v 111'183 2026-02-11T23:31:27.749+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b5339e65:::obj355:head local-lis/les=113/114 n=433 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.749+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.749+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b59cea89:::obj183:head, version: 111'183, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b59cea89:::obj183:head@111'183, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.749+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.749+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.749+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b5445b0d:::obj455:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.749+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b5445b0d:::obj455:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 1.0 e 116/116 lb 1:b5445b0d:::obj455:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b5339e65:::obj355:head local-lis/les=113/114 n=433 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.749+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.749+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b5339e65:::obj355:head local-lis/les=113/114 n=433 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:b5445b0d:::obj455:head) v3 2026-02-11T23:31:27.749+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b5339e65:::obj355:head local-lis/les=113/114 n=433 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:b5445b0d:::obj455:head) 2026-02-11T23:31:27.749+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.749+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b5445b0d:::obj455:head) v3 finish latency 0.000009 2026-02-11T23:31:27.749+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b5445b0d:::obj455:head local-lis/les=113/114 n=434 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.749+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.749+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.749+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b5445b0d:::obj455:head local-lis/les=113/114 n=434 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.749+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.749+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.749+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.749+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6171800 con 0x4acb000 2026-02-11T23:31:27.749+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:b59cea89:::obj183:head)]) -- 0x600bc80 con 0x4acb000 2026-02-11T23:31:27.749+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.749+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 905 ==== MOSDPGPush(1.0 116/113 [PushOp(1:b5a67629:::obj219:head, version: 111'219, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5a67629:::obj219:head@111'219, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559cb60 con 0x4acb000 2026-02-11T23:31:27.749+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:b5a67629:::obj219:head, version: 111'219, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5a67629:::obj219:head@111'219, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 116 2026-02-11T23:31:27.749+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b5a67629:::obj219:head, version: 111'219, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5a67629:::obj219:head@111'219, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.749+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.749+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 906 ==== pg_backfill(progress 1.0 e 116/116 lb 1:b59cea89:::obj183:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176b00 con 0x4acb000 2026-02-11T23:31:27.749+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b5a67629:::obj219:head, version: 111'219, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5a67629:::obj219:head@111'219, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.749+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:b59cea89:::obj183:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.749+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.749+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b59cea89:::obj183:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.749+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b5a67629:::obj219:head, version: 111'219, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5a67629:::obj219:head@111'219, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.749+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.749+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b5a67629:::obj219:head, version: 111'219, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5a67629:::obj219:head@111'219, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 116/113 [PushOp(1:b5a67629:::obj219:head, version: 111'219, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5a67629:::obj219:head@111'219, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b5445b0d:::obj455:head local-lis/les=113/114 n=434 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.749+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.749+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b5445b0d:::obj455:head local-lis/les=113/114 n=434 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:b5a67629:::obj219:head, version: 111'219, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5a67629:::obj219:head@111'219, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.749+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b5445b0d:::obj455:head local-lis/les=113/114 n=434 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b5a67629:::obj219:head@111'219, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.749+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.749+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b59cea89:::obj183:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.749+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b5445b0d:::obj455:head local-lis/les=113/114 n=434 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:b5a67629:::obj219:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.749+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b5445b0d:::obj455:head local-lis/les=113/114 n=434 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:b5a67629:::obj219:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.749+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b5445b0d:::obj455:head local-lis/les=113/114 n=434 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:b5a67629:::obj219:head 2026-02-11T23:31:27.749+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b5445b0d:::obj455:head local-lis/les=113/114 n=434 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:b5a67629:::obj219:head v 111'219 2026-02-11T23:31:27.749+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b5445b0d:::obj455:head local-lis/les=113/114 n=434 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.749+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.749+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b5a67629:::obj219:head, version: 111'219, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5a67629:::obj219:head@111'219, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000044 2026-02-11T23:31:27.749+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.749+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.749+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b59cea89:::obj183:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.749+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b59cea89:::obj183:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 116/116 lb 1:b59cea89:::obj183:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b5445b0d:::obj455:head local-lis/les=113/114 n=434 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.749+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.749+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b5445b0d:::obj455:head local-lis/les=113/114 n=434 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:b59cea89:::obj183:head) v3 2026-02-11T23:31:27.749+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b5445b0d:::obj455:head local-lis/les=113/114 n=434 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:b59cea89:::obj183:head) 2026-02-11T23:31:27.749+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.749+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b59cea89:::obj183:head) v3 finish latency 0.000009 2026-02-11T23:31:27.749+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b59cea89:::obj183:head local-lis/les=113/114 n=435 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.749+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.750+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.750+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b59cea89:::obj183:head local-lis/les=113/114 n=435 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.750+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.750+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.750+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.750+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5ca0d80 con 0x4acb000 2026-02-11T23:31:27.750+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:b5a67629:::obj219:head)]) -- 0x600be00 con 0x4acb000 2026-02-11T23:31:27.750+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.750+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 907 ==== MOSDPGPush(1.0 116/113 [PushOp(1:b609c5a3:::obj261:head, version: 111'261, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b609c5a3:::obj261:head@111'261, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f1ba0 con 0x4acb000 2026-02-11T23:31:27.750+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:b609c5a3:::obj261:head, version: 111'261, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b609c5a3:::obj261:head@111'261, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.750+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b609c5a3:::obj261:head, version: 111'261, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b609c5a3:::obj261:head@111'261, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.750+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.750+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 908 ==== pg_backfill(progress 1.0 e 116/116 lb 1:b5a67629:::obj219:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177080 con 0x4acb000 2026-02-11T23:31:27.750+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:b5a67629:::obj219:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.750+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b609c5a3:::obj261:head, version: 111'261, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b609c5a3:::obj261:head@111'261, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.750+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b5a67629:::obj219:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.750+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.750+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b609c5a3:::obj261:head, version: 111'261, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b609c5a3:::obj261:head@111'261, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.750+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b609c5a3:::obj261:head, version: 111'261, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b609c5a3:::obj261:head@111'261, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 116/113 [PushOp(1:b609c5a3:::obj261:head, version: 111'261, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b609c5a3:::obj261:head@111'261, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b59cea89:::obj183:head local-lis/les=113/114 n=435 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.750+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.750+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.750+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b5a67629:::obj219:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.750+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b59cea89:::obj183:head local-lis/les=113/114 n=435 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:b609c5a3:::obj261:head, version: 111'261, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b609c5a3:::obj261:head@111'261, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.750+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b59cea89:::obj183:head local-lis/les=113/114 n=435 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b609c5a3:::obj261:head@111'261, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.750+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b59cea89:::obj183:head local-lis/les=113/114 n=435 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:b609c5a3:::obj261:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.750+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b59cea89:::obj183:head local-lis/les=113/114 n=435 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:b609c5a3:::obj261:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.750+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b59cea89:::obj183:head local-lis/les=113/114 n=435 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:b609c5a3:::obj261:head 2026-02-11T23:31:27.750+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b59cea89:::obj183:head local-lis/les=113/114 n=435 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:b609c5a3:::obj261:head v 111'261 2026-02-11T23:31:27.750+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b59cea89:::obj183:head local-lis/les=113/114 n=435 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.750+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.750+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b609c5a3:::obj261:head, version: 111'261, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b609c5a3:::obj261:head@111'261, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:27.750+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.750+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.750+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b5a67629:::obj219:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.750+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b5a67629:::obj219:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 1.0 e 116/116 lb 1:b5a67629:::obj219:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b59cea89:::obj183:head local-lis/les=113/114 n=435 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.750+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.750+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b59cea89:::obj183:head local-lis/les=113/114 n=435 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:b5a67629:::obj219:head) v3 2026-02-11T23:31:27.750+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b59cea89:::obj183:head local-lis/les=113/114 n=435 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:b5a67629:::obj219:head) 2026-02-11T23:31:27.750+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.750+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b5a67629:::obj219:head) v3 finish latency 0.000009 2026-02-11T23:31:27.750+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b5a67629:::obj219:head local-lis/les=113/114 n=436 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.750+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.750+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.750+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b5a67629:::obj219:head local-lis/les=113/114 n=436 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.750+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.750+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.750+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.750+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x612e600 con 0x4acb000 2026-02-11T23:31:27.750+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:b609c5a3:::obj261:head)]) -- 0x5fcfb00 con 0x4acb000 2026-02-11T23:31:27.750+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.750+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 909 ==== MOSDPGPush(1.0 116/113 [PushOp(1:b66eab34:::obj112:head, version: 111'112, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b66eab34:::obj112:head@111'112, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559d520 con 0x4acb000 2026-02-11T23:31:27.750+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:b66eab34:::obj112:head, version: 111'112, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b66eab34:::obj112:head@111'112, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000015 epoch 116 2026-02-11T23:31:27.750+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b66eab34:::obj112:head, version: 111'112, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b66eab34:::obj112:head@111'112, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.750+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.750+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 910 ==== pg_backfill(progress 1.0 e 116/116 lb 1:b609c5a3:::obj261:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177600 con 0x4acb000 2026-02-11T23:31:27.750+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b66eab34:::obj112:head, version: 111'112, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b66eab34:::obj112:head@111'112, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.750+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:b609c5a3:::obj261:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.750+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b609c5a3:::obj261:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.750+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.750+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b66eab34:::obj112:head, version: 111'112, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b66eab34:::obj112:head@111'112, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.750+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.750+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b66eab34:::obj112:head, version: 111'112, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b66eab34:::obj112:head@111'112, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000040 MOSDPGPush(1.0 116/113 [PushOp(1:b66eab34:::obj112:head, version: 111'112, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b66eab34:::obj112:head@111'112, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b5a67629:::obj219:head local-lis/les=113/114 n=436 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.750+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.750+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b5a67629:::obj219:head local-lis/les=113/114 n=436 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:b66eab34:::obj112:head, version: 111'112, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b66eab34:::obj112:head@111'112, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.750+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b5a67629:::obj219:head local-lis/les=113/114 n=436 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b66eab34:::obj112:head@111'112, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.750+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.750+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b609c5a3:::obj261:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.750+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b5a67629:::obj219:head local-lis/les=113/114 n=436 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:b66eab34:::obj112:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.750+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b5a67629:::obj219:head local-lis/les=113/114 n=436 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:b66eab34:::obj112:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.750+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b5a67629:::obj219:head local-lis/les=113/114 n=436 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:b66eab34:::obj112:head 2026-02-11T23:31:27.750+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b5a67629:::obj219:head local-lis/les=113/114 n=436 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:b66eab34:::obj112:head v 111'112 2026-02-11T23:31:27.750+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b5a67629:::obj219:head local-lis/les=113/114 n=436 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.750+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.751+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b66eab34:::obj112:head, version: 111'112, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b66eab34:::obj112:head@111'112, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:27.751+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.751+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.751+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b609c5a3:::obj261:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.751+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b609c5a3:::obj261:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 1.0 e 116/116 lb 1:b609c5a3:::obj261:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b5a67629:::obj219:head local-lis/les=113/114 n=436 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.751+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.751+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b5a67629:::obj219:head local-lis/les=113/114 n=436 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:b609c5a3:::obj261:head) v3 2026-02-11T23:31:27.751+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b5a67629:::obj219:head local-lis/les=113/114 n=436 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:b609c5a3:::obj261:head) 2026-02-11T23:31:27.751+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.751+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b609c5a3:::obj261:head) v3 finish latency 0.000014 2026-02-11T23:31:27.751+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b609c5a3:::obj261:head local-lis/les=113/114 n=437 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.751+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.751+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.751+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b609c5a3:::obj261:head local-lis/les=113/114 n=437 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.751+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.751+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.751+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.751+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6171080 con 0x4acb000 2026-02-11T23:31:27.751+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:b66eab34:::obj112:head)]) -- 0x600a900 con 0x4acb000 2026-02-11T23:31:27.751+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.751+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 911 ==== MOSDPGPush(1.0 116/113 [PushOp(1:b6a90992:::obj552:head, version: 111'552, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b6a90992:::obj552:head@111'552, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c680 con 0x4acb000 2026-02-11T23:31:27.751+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:b6a90992:::obj552:head, version: 111'552, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b6a90992:::obj552:head@111'552, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 116 2026-02-11T23:31:27.751+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b6a90992:::obj552:head, version: 111'552, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b6a90992:::obj552:head@111'552, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.751+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.751+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 912 ==== pg_backfill(progress 1.0 e 116/116 lb 1:b66eab34:::obj112:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177b80 con 0x4acb000 2026-02-11T23:31:27.751+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:b66eab34:::obj112:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.751+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b6a90992:::obj552:head, version: 111'552, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b6a90992:::obj552:head@111'552, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.751+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b66eab34:::obj112:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.751+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.751+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b6a90992:::obj552:head, version: 111'552, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b6a90992:::obj552:head@111'552, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.751+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.751+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b6a90992:::obj552:head, version: 111'552, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b6a90992:::obj552:head@111'552, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 116/113 [PushOp(1:b6a90992:::obj552:head, version: 111'552, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b6a90992:::obj552:head@111'552, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b609c5a3:::obj261:head local-lis/les=113/114 n=437 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.751+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.751+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b66eab34:::obj112:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.751+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b609c5a3:::obj261:head local-lis/les=113/114 n=437 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:b6a90992:::obj552:head, version: 111'552, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b6a90992:::obj552:head@111'552, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.751+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b609c5a3:::obj261:head local-lis/les=113/114 n=437 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b6a90992:::obj552:head@111'552, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.751+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b609c5a3:::obj261:head local-lis/les=113/114 n=437 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:b6a90992:::obj552:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.751+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b609c5a3:::obj261:head local-lis/les=113/114 n=437 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:b6a90992:::obj552:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.751+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b609c5a3:::obj261:head local-lis/les=113/114 n=437 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:b6a90992:::obj552:head 2026-02-11T23:31:27.751+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b609c5a3:::obj261:head local-lis/les=113/114 n=437 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:b6a90992:::obj552:head v 111'552 2026-02-11T23:31:27.751+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b609c5a3:::obj261:head local-lis/les=113/114 n=437 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.751+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.751+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b6a90992:::obj552:head, version: 111'552, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b6a90992:::obj552:head@111'552, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:27.751+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.751+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.751+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b66eab34:::obj112:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.751+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b66eab34:::obj112:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 116/116 lb 1:b66eab34:::obj112:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b609c5a3:::obj261:head local-lis/les=113/114 n=437 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.751+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.751+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b609c5a3:::obj261:head local-lis/les=113/114 n=437 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:b66eab34:::obj112:head) v3 2026-02-11T23:31:27.751+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b609c5a3:::obj261:head local-lis/les=113/114 n=437 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:b66eab34:::obj112:head) 2026-02-11T23:31:27.751+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.751+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b66eab34:::obj112:head) v3 finish latency 0.000010 2026-02-11T23:31:27.751+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b66eab34:::obj112:head local-lis/les=113/114 n=438 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.751+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.751+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.751+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b66eab34:::obj112:head local-lis/les=113/114 n=438 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.751+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.751+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.751+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.751+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6171680 con 0x4acb000 2026-02-11T23:31:27.751+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:b6a90992:::obj552:head)]) -- 0x5fcf800 con 0x4acb000 2026-02-11T23:31:27.751+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.752+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 913 ==== MOSDPGPush(1.0 116/113 [PushOp(1:b6dfbaf7:::obj426:head, version: 111'426, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b6dfbaf7:::obj426:head@111'426, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559dba0 con 0x4acb000 2026-02-11T23:31:27.752+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:b6dfbaf7:::obj426:head, version: 111'426, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b6dfbaf7:::obj426:head@111'426, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 116 2026-02-11T23:31:27.752+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b6dfbaf7:::obj426:head, version: 111'426, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b6dfbaf7:::obj426:head@111'426, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.752+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.752+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 914 ==== pg_backfill(progress 1.0 e 116/116 lb 1:b6a90992:::obj552:head) ==== 1062+0+0 (unknown 0 0 0) 0x6178100 con 0x4acb000 2026-02-11T23:31:27.752+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:b6a90992:::obj552:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.752+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b6dfbaf7:::obj426:head, version: 111'426, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b6dfbaf7:::obj426:head@111'426, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.752+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b6a90992:::obj552:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.752+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.752+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b6dfbaf7:::obj426:head, version: 111'426, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b6dfbaf7:::obj426:head@111'426, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.752+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.752+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b6dfbaf7:::obj426:head, version: 111'426, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b6dfbaf7:::obj426:head@111'426, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000043 MOSDPGPush(1.0 116/113 [PushOp(1:b6dfbaf7:::obj426:head, version: 111'426, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b6dfbaf7:::obj426:head@111'426, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b66eab34:::obj112:head local-lis/les=113/114 n=438 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.752+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.752+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b66eab34:::obj112:head local-lis/les=113/114 n=438 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:b6dfbaf7:::obj426:head, version: 111'426, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b6dfbaf7:::obj426:head@111'426, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.752+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b66eab34:::obj112:head local-lis/les=113/114 n=438 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b6dfbaf7:::obj426:head@111'426, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.752+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.752+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b6a90992:::obj552:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.752+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b66eab34:::obj112:head local-lis/les=113/114 n=438 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:b6dfbaf7:::obj426:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.752+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b66eab34:::obj112:head local-lis/les=113/114 n=438 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:b6dfbaf7:::obj426:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.752+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b66eab34:::obj112:head local-lis/les=113/114 n=438 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:b6dfbaf7:::obj426:head 2026-02-11T23:31:27.752+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b66eab34:::obj112:head local-lis/les=113/114 n=438 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:b6dfbaf7:::obj426:head v 111'426 2026-02-11T23:31:27.752+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b66eab34:::obj112:head local-lis/les=113/114 n=438 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.752+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.752+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b6dfbaf7:::obj426:head, version: 111'426, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b6dfbaf7:::obj426:head@111'426, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:27.752+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.752+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.752+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b6a90992:::obj552:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.752+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b6a90992:::obj552:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 1.0 e 116/116 lb 1:b6a90992:::obj552:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b66eab34:::obj112:head local-lis/les=113/114 n=438 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.752+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.752+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b66eab34:::obj112:head local-lis/les=113/114 n=438 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:b6a90992:::obj552:head) v3 2026-02-11T23:31:27.752+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b66eab34:::obj112:head local-lis/les=113/114 n=438 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:b6a90992:::obj552:head) 2026-02-11T23:31:27.752+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.752+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b6a90992:::obj552:head) v3 finish latency 0.000009 2026-02-11T23:31:27.752+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b6a90992:::obj552:head local-lis/les=113/114 n=439 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.752+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.752+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.752+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b6a90992:::obj552:head local-lis/les=113/114 n=439 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.752+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.752+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.752+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.752+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x612e300 con 0x4acb000 2026-02-11T23:31:27.752+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:b6dfbaf7:::obj426:head)]) -- 0x600a480 con 0x4acb000 2026-02-11T23:31:27.752+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.752+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 915 ==== MOSDPGPush(1.0 116/113 [PushOp(1:b70eecc2:::obj59:head, version: 111'59, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b70eecc2:::obj59:head@111'59, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x559d040 con 0x4acb000 2026-02-11T23:31:27.752+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:b70eecc2:::obj59:head, version: 111'59, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b70eecc2:::obj59:head@111'59, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.752+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b70eecc2:::obj59:head, version: 111'59, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b70eecc2:::obj59:head@111'59, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.752+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.752+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 916 ==== pg_backfill(progress 1.0 e 116/116 lb 1:b6dfbaf7:::obj426:head) ==== 1062+0+0 (unknown 0 0 0) 0x6178680 con 0x4acb000 2026-02-11T23:31:27.752+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:b6dfbaf7:::obj426:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.752+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b70eecc2:::obj59:head, version: 111'59, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b70eecc2:::obj59:head@111'59, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.752+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b6dfbaf7:::obj426:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.752+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.752+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b70eecc2:::obj59:head, version: 111'59, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b70eecc2:::obj59:head@111'59, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.752+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.752+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b70eecc2:::obj59:head, version: 111'59, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b70eecc2:::obj59:head@111'59, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 116/113 [PushOp(1:b70eecc2:::obj59:head, version: 111'59, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b70eecc2:::obj59:head@111'59, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b6a90992:::obj552:head local-lis/les=113/114 n=439 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.752+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.752+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b6a90992:::obj552:head local-lis/les=113/114 n=439 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:b70eecc2:::obj59:head, version: 111'59, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b70eecc2:::obj59:head@111'59, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.752+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b6a90992:::obj552:head local-lis/les=113/114 n=439 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b70eecc2:::obj59:head@111'59, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.752+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.752+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b6dfbaf7:::obj426:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.752+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b6a90992:::obj552:head local-lis/les=113/114 n=439 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:b70eecc2:::obj59:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.752+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b6a90992:::obj552:head local-lis/les=113/114 n=439 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:b70eecc2:::obj59:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.752+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b6a90992:::obj552:head local-lis/les=113/114 n=439 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:b70eecc2:::obj59:head 2026-02-11T23:31:27.752+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b6a90992:::obj552:head local-lis/les=113/114 n=439 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:b70eecc2:::obj59:head v 111'59 2026-02-11T23:31:27.752+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b6a90992:::obj552:head local-lis/les=113/114 n=439 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.752+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.752+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b70eecc2:::obj59:head, version: 111'59, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b70eecc2:::obj59:head@111'59, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.752+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.752+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.752+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b6dfbaf7:::obj426:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.752+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b6dfbaf7:::obj426:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 116/116 lb 1:b6dfbaf7:::obj426:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b6a90992:::obj552:head local-lis/les=113/114 n=439 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.752+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.752+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b6a90992:::obj552:head local-lis/les=113/114 n=439 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:b6dfbaf7:::obj426:head) v3 2026-02-11T23:31:27.752+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b6a90992:::obj552:head local-lis/les=113/114 n=439 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:b6dfbaf7:::obj426:head) 2026-02-11T23:31:27.752+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.752+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b6dfbaf7:::obj426:head) v3 finish latency 0.000010 2026-02-11T23:31:27.752+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b6dfbaf7:::obj426:head local-lis/les=113/114 n=440 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.752+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.752+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.752+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b6dfbaf7:::obj426:head local-lis/les=113/114 n=440 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.752+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.752+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.752+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.752+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d1de00 con 0x4acb000 2026-02-11T23:31:27.752+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:b70eecc2:::obj59:head)]) -- 0x5fcf680 con 0x4acb000 2026-02-11T23:31:27.752+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.753+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 917 ==== MOSDPGPush(1.0 116/113 [PushOp(1:b75b9722:::obj495:head, version: 111'495, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b75b9722:::obj495:head@111'495, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d89c0 con 0x4acb000 2026-02-11T23:31:27.753+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:b75b9722:::obj495:head, version: 111'495, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b75b9722:::obj495:head@111'495, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.753+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b75b9722:::obj495:head, version: 111'495, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b75b9722:::obj495:head@111'495, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.753+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.753+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 918 ==== pg_backfill(progress 1.0 e 116/116 lb 1:b70eecc2:::obj59:head) ==== 1061+0+0 (unknown 0 0 0) 0x6178c00 con 0x4acb000 2026-02-11T23:31:27.753+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:b70eecc2:::obj59:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.753+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b75b9722:::obj495:head, version: 111'495, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b75b9722:::obj495:head@111'495, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.753+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b70eecc2:::obj59:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.753+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.753+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b75b9722:::obj495:head, version: 111'495, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b75b9722:::obj495:head@111'495, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.753+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b75b9722:::obj495:head, version: 111'495, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b75b9722:::obj495:head@111'495, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 116/113 [PushOp(1:b75b9722:::obj495:head, version: 111'495, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b75b9722:::obj495:head@111'495, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b6dfbaf7:::obj426:head local-lis/les=113/114 n=440 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.753+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.753+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.753+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b6dfbaf7:::obj426:head local-lis/les=113/114 n=440 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:b75b9722:::obj495:head, version: 111'495, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b75b9722:::obj495:head@111'495, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.753+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b70eecc2:::obj59:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.753+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b6dfbaf7:::obj426:head local-lis/les=113/114 n=440 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b75b9722:::obj495:head@111'495, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.753+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b6dfbaf7:::obj426:head local-lis/les=113/114 n=440 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:b75b9722:::obj495:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.753+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b6dfbaf7:::obj426:head local-lis/les=113/114 n=440 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:b75b9722:::obj495:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.753+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b6dfbaf7:::obj426:head local-lis/les=113/114 n=440 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:b75b9722:::obj495:head 2026-02-11T23:31:27.753+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b6dfbaf7:::obj426:head local-lis/les=113/114 n=440 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:b75b9722:::obj495:head v 111'495 2026-02-11T23:31:27.753+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b6dfbaf7:::obj426:head local-lis/les=113/114 n=440 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.753+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.753+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b75b9722:::obj495:head, version: 111'495, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b75b9722:::obj495:head@111'495, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.753+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.753+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.753+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b70eecc2:::obj59:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.753+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b70eecc2:::obj59:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 116/116 lb 1:b70eecc2:::obj59:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b6dfbaf7:::obj426:head local-lis/les=113/114 n=440 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.753+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.753+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b6dfbaf7:::obj426:head local-lis/les=113/114 n=440 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:b70eecc2:::obj59:head) v3 2026-02-11T23:31:27.753+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b6dfbaf7:::obj426:head local-lis/les=113/114 n=440 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:b70eecc2:::obj59:head) 2026-02-11T23:31:27.753+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.753+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b70eecc2:::obj59:head) v3 finish latency 0.000009 2026-02-11T23:31:27.753+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b70eecc2:::obj59:head local-lis/les=113/114 n=441 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.753+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.753+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.753+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b70eecc2:::obj59:head local-lis/les=113/114 n=441 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.753+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.753+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.753+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.753+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x613f980 con 0x4acb000 2026-02-11T23:31:27.753+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:b75b9722:::obj495:head)]) -- 0x6014000 con 0x4acb000 2026-02-11T23:31:27.753+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.753+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 919 ==== MOSDPGPush(1.0 116/113 [PushOp(1:b841f58a:::obj361:head, version: 111'361, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b841f58a:::obj361:head@111'361, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d8000 con 0x4acb000 2026-02-11T23:31:27.753+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:b841f58a:::obj361:head, version: 111'361, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b841f58a:::obj361:head@111'361, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 116 2026-02-11T23:31:27.753+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b841f58a:::obj361:head, version: 111'361, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b841f58a:::obj361:head@111'361, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.753+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.753+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 920 ==== pg_backfill(progress 1.0 e 116/116 lb 1:b75b9722:::obj495:head) ==== 1062+0+0 (unknown 0 0 0) 0x6179180 con 0x4acb000 2026-02-11T23:31:27.753+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:b75b9722:::obj495:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.753+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b841f58a:::obj361:head, version: 111'361, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b841f58a:::obj361:head@111'361, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.753+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b75b9722:::obj495:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.753+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.753+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b841f58a:::obj361:head, version: 111'361, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b841f58a:::obj361:head@111'361, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.753+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b841f58a:::obj361:head, version: 111'361, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b841f58a:::obj361:head@111'361, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(1.0 116/113 [PushOp(1:b841f58a:::obj361:head, version: 111'361, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b841f58a:::obj361:head@111'361, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b70eecc2:::obj59:head local-lis/les=113/114 n=441 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.753+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.753+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.753+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b70eecc2:::obj59:head local-lis/les=113/114 n=441 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:b841f58a:::obj361:head, version: 111'361, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b841f58a:::obj361:head@111'361, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.753+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b75b9722:::obj495:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.753+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b70eecc2:::obj59:head local-lis/les=113/114 n=441 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b841f58a:::obj361:head@111'361, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.753+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b70eecc2:::obj59:head local-lis/les=113/114 n=441 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:b841f58a:::obj361:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.753+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b70eecc2:::obj59:head local-lis/les=113/114 n=441 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:b841f58a:::obj361:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.753+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b70eecc2:::obj59:head local-lis/les=113/114 n=441 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:b841f58a:::obj361:head 2026-02-11T23:31:27.753+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b70eecc2:::obj59:head local-lis/les=113/114 n=441 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:b841f58a:::obj361:head v 111'361 2026-02-11T23:31:27.753+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b70eecc2:::obj59:head local-lis/les=113/114 n=441 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.753+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.753+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b841f58a:::obj361:head, version: 111'361, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b841f58a:::obj361:head@111'361, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.753+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.753+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.753+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b75b9722:::obj495:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.753+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b75b9722:::obj495:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 116/116 lb 1:b75b9722:::obj495:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b70eecc2:::obj59:head local-lis/les=113/114 n=441 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.753+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.753+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b70eecc2:::obj59:head local-lis/les=113/114 n=441 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:b75b9722:::obj495:head) v3 2026-02-11T23:31:27.753+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b70eecc2:::obj59:head local-lis/les=113/114 n=441 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:b75b9722:::obj495:head) 2026-02-11T23:31:27.753+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.753+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b75b9722:::obj495:head) v3 finish latency 0.000009 2026-02-11T23:31:27.753+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b75b9722:::obj495:head local-lis/les=113/114 n=442 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.753+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.753+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.753+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b75b9722:::obj495:head local-lis/les=113/114 n=442 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.753+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.753+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.753+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.753+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6171380 con 0x4acb000 2026-02-11T23:31:27.753+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:b841f58a:::obj361:head)]) -- 0x6170600 con 0x4acb000 2026-02-11T23:31:27.753+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.754+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 921 ==== MOSDPGPush(1.0 116/113 [PushOp(1:b8510467:::obj40:head, version: 111'40, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b8510467:::obj40:head@111'40, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x55d84e0 con 0x4acb000 2026-02-11T23:31:27.754+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:b8510467:::obj40:head, version: 111'40, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b8510467:::obj40:head@111'40, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 116 2026-02-11T23:31:27.754+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b8510467:::obj40:head, version: 111'40, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b8510467:::obj40:head@111'40, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.754+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.754+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 922 ==== pg_backfill(progress 1.0 e 116/116 lb 1:b841f58a:::obj361:head) ==== 1062+0+0 (unknown 0 0 0) 0x5fbe000 con 0x4acb000 2026-02-11T23:31:27.754+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b8510467:::obj40:head, version: 111'40, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b8510467:::obj40:head@111'40, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.754+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:b841f58a:::obj361:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 116 2026-02-11T23:31:27.754+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b841f58a:::obj361:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.754+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.754+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b8510467:::obj40:head, version: 111'40, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b8510467:::obj40:head@111'40, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.754+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.754+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b8510467:::obj40:head, version: 111'40, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b8510467:::obj40:head@111'40, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000040 MOSDPGPush(1.0 116/113 [PushOp(1:b8510467:::obj40:head, version: 111'40, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b8510467:::obj40:head@111'40, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b75b9722:::obj495:head local-lis/les=113/114 n=442 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.754+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.754+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b75b9722:::obj495:head local-lis/les=113/114 n=442 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:b8510467:::obj40:head, version: 111'40, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b8510467:::obj40:head@111'40, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.754+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b75b9722:::obj495:head local-lis/les=113/114 n=442 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b8510467:::obj40:head@111'40, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.754+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.754+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b841f58a:::obj361:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.754+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b75b9722:::obj495:head local-lis/les=113/114 n=442 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:b8510467:::obj40:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.754+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b75b9722:::obj495:head local-lis/les=113/114 n=442 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:b8510467:::obj40:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.754+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b75b9722:::obj495:head local-lis/les=113/114 n=442 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:b8510467:::obj40:head 2026-02-11T23:31:27.754+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b75b9722:::obj495:head local-lis/les=113/114 n=442 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:b8510467:::obj40:head v 111'40 2026-02-11T23:31:27.754+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b75b9722:::obj495:head local-lis/les=113/114 n=442 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.754+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.754+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b8510467:::obj40:head, version: 111'40, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b8510467:::obj40:head@111'40, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:27.754+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.754+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.754+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b841f58a:::obj361:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.754+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b841f58a:::obj361:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 1.0 e 116/116 lb 1:b841f58a:::obj361:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b75b9722:::obj495:head local-lis/les=113/114 n=442 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.754+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.754+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b75b9722:::obj495:head local-lis/les=113/114 n=442 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:b841f58a:::obj361:head) v3 2026-02-11T23:31:27.754+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b75b9722:::obj495:head local-lis/les=113/114 n=442 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:b841f58a:::obj361:head) 2026-02-11T23:31:27.754+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.754+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b841f58a:::obj361:head) v3 finish latency 0.000009 2026-02-11T23:31:27.754+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b841f58a:::obj361:head local-lis/les=113/114 n=443 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.754+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.754+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.754+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b841f58a:::obj361:head local-lis/les=113/114 n=443 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.754+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.754+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.754+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.754+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6170600 con 0x4acb000 2026-02-11T23:31:27.754+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:b8510467:::obj40:head)]) -- 0x615ca80 con 0x4acb000 2026-02-11T23:31:27.754+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.754+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 923 ==== MOSDPGPush(1.0 116/113 [PushOp(1:b8f63b78:::obj291:head, version: 111'291, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b8f63b78:::obj291:head@111'291, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5647380 con 0x4acb000 2026-02-11T23:31:27.754+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:b8f63b78:::obj291:head, version: 111'291, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b8f63b78:::obj291:head@111'291, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.754+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b8f63b78:::obj291:head, version: 111'291, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b8f63b78:::obj291:head@111'291, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.754+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.754+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 924 ==== pg_backfill(progress 1.0 e 116/116 lb 1:b8510467:::obj40:head) ==== 1061+0+0 (unknown 0 0 0) 0x5fbe580 con 0x4acb000 2026-02-11T23:31:27.754+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:b8510467:::obj40:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.754+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b8f63b78:::obj291:head, version: 111'291, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b8f63b78:::obj291:head@111'291, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.754+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b8510467:::obj40:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.754+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.754+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b8f63b78:::obj291:head, version: 111'291, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b8f63b78:::obj291:head@111'291, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.754+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b8f63b78:::obj291:head, version: 111'291, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b8f63b78:::obj291:head@111'291, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 116/113 [PushOp(1:b8f63b78:::obj291:head, version: 111'291, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b8f63b78:::obj291:head@111'291, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b841f58a:::obj361:head local-lis/les=113/114 n=443 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.754+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.754+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.754+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b8510467:::obj40:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.754+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b841f58a:::obj361:head local-lis/les=113/114 n=443 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:b8f63b78:::obj291:head, version: 111'291, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b8f63b78:::obj291:head@111'291, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.754+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b841f58a:::obj361:head local-lis/les=113/114 n=443 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b8f63b78:::obj291:head@111'291, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.754+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b841f58a:::obj361:head local-lis/les=113/114 n=443 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:b8f63b78:::obj291:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.754+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b841f58a:::obj361:head local-lis/les=113/114 n=443 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:b8f63b78:::obj291:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.754+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b841f58a:::obj361:head local-lis/les=113/114 n=443 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:b8f63b78:::obj291:head 2026-02-11T23:31:27.754+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b841f58a:::obj361:head local-lis/les=113/114 n=443 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:b8f63b78:::obj291:head v 111'291 2026-02-11T23:31:27.754+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b841f58a:::obj361:head local-lis/les=113/114 n=443 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.754+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.754+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b8f63b78:::obj291:head, version: 111'291, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b8f63b78:::obj291:head@111'291, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.754+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.754+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.754+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b8510467:::obj40:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.754+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b8510467:::obj40:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 1.0 e 116/116 lb 1:b8510467:::obj40:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b841f58a:::obj361:head local-lis/les=113/114 n=443 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.754+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.754+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b841f58a:::obj361:head local-lis/les=113/114 n=443 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:b8510467:::obj40:head) v3 2026-02-11T23:31:27.754+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b841f58a:::obj361:head local-lis/les=113/114 n=443 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:b8510467:::obj40:head) 2026-02-11T23:31:27.754+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.754+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b8510467:::obj40:head) v3 finish latency 0.000010 2026-02-11T23:31:27.754+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b8510467:::obj40:head local-lis/les=113/114 n=444 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.754+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.755+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.755+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b8510467:::obj40:head local-lis/les=113/114 n=444 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.755+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.755+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.755+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.755+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6171380 con 0x4acb000 2026-02-11T23:31:27.755+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:b8f63b78:::obj291:head)]) -- 0x6004180 con 0x4acb000 2026-02-11T23:31:27.755+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.755+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 925 ==== MOSDPGPush(1.0 116/113 [PushOp(1:b92d6f18:::obj505:head, version: 111'505, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b92d6f18:::obj505:head@111'505, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d8ea0 con 0x4acb000 2026-02-11T23:31:27.755+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:b92d6f18:::obj505:head, version: 111'505, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b92d6f18:::obj505:head@111'505, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.755+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b92d6f18:::obj505:head, version: 111'505, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b92d6f18:::obj505:head@111'505, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.755+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.755+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 926 ==== pg_backfill(progress 1.0 e 116/116 lb 1:b8f63b78:::obj291:head) ==== 1062+0+0 (unknown 0 0 0) 0x5fbeb00 con 0x4acb000 2026-02-11T23:31:27.755+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:b8f63b78:::obj291:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.755+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b92d6f18:::obj505:head, version: 111'505, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b92d6f18:::obj505:head@111'505, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.755+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b8f63b78:::obj291:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.755+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.755+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b92d6f18:::obj505:head, version: 111'505, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b92d6f18:::obj505:head@111'505, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.755+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b92d6f18:::obj505:head, version: 111'505, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b92d6f18:::obj505:head@111'505, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:b92d6f18:::obj505:head, version: 111'505, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b92d6f18:::obj505:head@111'505, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b8510467:::obj40:head local-lis/les=113/114 n=444 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.755+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.755+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.755+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b8510467:::obj40:head local-lis/les=113/114 n=444 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:b92d6f18:::obj505:head, version: 111'505, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b92d6f18:::obj505:head@111'505, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.755+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b8f63b78:::obj291:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.755+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b8510467:::obj40:head local-lis/les=113/114 n=444 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b92d6f18:::obj505:head@111'505, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.755+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b8510467:::obj40:head local-lis/les=113/114 n=444 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:b92d6f18:::obj505:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.755+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b8510467:::obj40:head local-lis/les=113/114 n=444 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:b92d6f18:::obj505:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.755+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b8510467:::obj40:head local-lis/les=113/114 n=444 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:b92d6f18:::obj505:head 2026-02-11T23:31:27.755+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b8510467:::obj40:head local-lis/les=113/114 n=444 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:b92d6f18:::obj505:head v 111'505 2026-02-11T23:31:27.755+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b8510467:::obj40:head local-lis/les=113/114 n=444 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.755+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.755+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b92d6f18:::obj505:head, version: 111'505, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b92d6f18:::obj505:head@111'505, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:27.755+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.755+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.755+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b8f63b78:::obj291:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.755+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b8f63b78:::obj291:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 116/116 lb 1:b8f63b78:::obj291:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b8510467:::obj40:head local-lis/les=113/114 n=444 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.755+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.755+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b8510467:::obj40:head local-lis/les=113/114 n=444 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:b8f63b78:::obj291:head) v3 2026-02-11T23:31:27.755+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b8510467:::obj40:head local-lis/les=113/114 n=444 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:b8f63b78:::obj291:head) 2026-02-11T23:31:27.755+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.755+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b8f63b78:::obj291:head) v3 finish latency 0.000009 2026-02-11T23:31:27.755+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b8f63b78:::obj291:head local-lis/les=113/114 n=445 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.755+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.755+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.755+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b8f63b78:::obj291:head local-lis/les=113/114 n=445 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.755+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.755+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.755+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.755+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6014000 con 0x4acb000 2026-02-11T23:31:27.755+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:b92d6f18:::obj505:head)]) -- 0x615c480 con 0x4acb000 2026-02-11T23:31:27.755+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.755+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 927 ==== MOSDPGPush(1.0 116/113 [PushOp(1:b9836a99:::obj18:head, version: 111'18, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b9836a99:::obj18:head@111'18, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x55d9380 con 0x4acb000 2026-02-11T23:31:27.755+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:b9836a99:::obj18:head, version: 111'18, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b9836a99:::obj18:head@111'18, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.755+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b9836a99:::obj18:head, version: 111'18, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b9836a99:::obj18:head@111'18, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.755+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.755+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 928 ==== pg_backfill(progress 1.0 e 116/116 lb 1:b92d6f18:::obj505:head) ==== 1062+0+0 (unknown 0 0 0) 0x5fbf080 con 0x4acb000 2026-02-11T23:31:27.755+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:b92d6f18:::obj505:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.755+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b9836a99:::obj18:head, version: 111'18, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b9836a99:::obj18:head@111'18, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.755+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b92d6f18:::obj505:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.755+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.755+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:b9836a99:::obj18:head, version: 111'18, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b9836a99:::obj18:head@111'18, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.755+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b9836a99:::obj18:head, version: 111'18, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b9836a99:::obj18:head@111'18, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 116/113 [PushOp(1:b9836a99:::obj18:head, version: 111'18, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b9836a99:::obj18:head@111'18, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b8f63b78:::obj291:head local-lis/les=113/114 n=445 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.755+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.755+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.755+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b8f63b78:::obj291:head local-lis/les=113/114 n=445 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:b9836a99:::obj18:head, version: 111'18, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b9836a99:::obj18:head@111'18, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.755+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b92d6f18:::obj505:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.755+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b8f63b78:::obj291:head local-lis/les=113/114 n=445 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b9836a99:::obj18:head@111'18, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.755+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b8f63b78:::obj291:head local-lis/les=113/114 n=445 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:b9836a99:::obj18:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.755+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b8f63b78:::obj291:head local-lis/les=113/114 n=445 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:b9836a99:::obj18:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.755+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b8f63b78:::obj291:head local-lis/les=113/114 n=445 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:b9836a99:::obj18:head 2026-02-11T23:31:27.755+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b8f63b78:::obj291:head local-lis/les=113/114 n=445 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:b9836a99:::obj18:head v 111'18 2026-02-11T23:31:27.755+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b8f63b78:::obj291:head local-lis/les=113/114 n=445 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.755+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.755+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:b9836a99:::obj18:head, version: 111'18, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b9836a99:::obj18:head@111'18, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.755+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.755+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.755+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b92d6f18:::obj505:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.755+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b92d6f18:::obj505:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 1.0 e 116/116 lb 1:b92d6f18:::obj505:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b8f63b78:::obj291:head local-lis/les=113/114 n=445 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.755+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.755+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b8f63b78:::obj291:head local-lis/les=113/114 n=445 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:b92d6f18:::obj505:head) v3 2026-02-11T23:31:27.755+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b8f63b78:::obj291:head local-lis/les=113/114 n=445 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:b92d6f18:::obj505:head) 2026-02-11T23:31:27.755+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.755+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b92d6f18:::obj505:head) v3 finish latency 0.000009 2026-02-11T23:31:27.755+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b92d6f18:::obj505:head local-lis/les=113/114 n=446 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.755+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.756+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.756+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b92d6f18:::obj505:head local-lis/les=113/114 n=446 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.756+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.756+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.756+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.756+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x613f980 con 0x4acb000 2026-02-11T23:31:27.756+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:b9836a99:::obj18:head)]) -- 0x5ca1680 con 0x4acb000 2026-02-11T23:31:27.756+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.756+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 929 ==== MOSDPGPush(1.0 116/113 [PushOp(1:ba083999:::obj295:head, version: 111'295, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba083999:::obj295:head@111'295, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56cb040 con 0x4acb000 2026-02-11T23:31:27.756+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:ba083999:::obj295:head, version: 111'295, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba083999:::obj295:head@111'295, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000015 epoch 116 2026-02-11T23:31:27.756+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:ba083999:::obj295:head, version: 111'295, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba083999:::obj295:head@111'295, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.756+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.756+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 930 ==== pg_backfill(progress 1.0 e 116/116 lb 1:b9836a99:::obj18:head) ==== 1061+0+0 (unknown 0 0 0) 0x5fbf600 con 0x4acb000 2026-02-11T23:31:27.756+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:ba083999:::obj295:head, version: 111'295, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba083999:::obj295:head@111'295, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.756+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:b9836a99:::obj18:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.756+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b9836a99:::obj18:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.756+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.756+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:ba083999:::obj295:head, version: 111'295, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba083999:::obj295:head@111'295, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.756+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:ba083999:::obj295:head, version: 111'295, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba083999:::obj295:head@111'295, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 116/113 [PushOp(1:ba083999:::obj295:head, version: 111'295, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba083999:::obj295:head@111'295, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b92d6f18:::obj505:head local-lis/les=113/114 n=446 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.756+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.756+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.756+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b92d6f18:::obj505:head local-lis/les=113/114 n=446 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:ba083999:::obj295:head, version: 111'295, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba083999:::obj295:head@111'295, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.756+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b9836a99:::obj18:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.756+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b92d6f18:::obj505:head local-lis/les=113/114 n=446 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:ba083999:::obj295:head@111'295, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.756+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b92d6f18:::obj505:head local-lis/les=113/114 n=446 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:ba083999:::obj295:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.756+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b92d6f18:::obj505:head local-lis/les=113/114 n=446 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:ba083999:::obj295:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.756+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b92d6f18:::obj505:head local-lis/les=113/114 n=446 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:ba083999:::obj295:head 2026-02-11T23:31:27.756+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b92d6f18:::obj505:head local-lis/les=113/114 n=446 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:ba083999:::obj295:head v 111'295 2026-02-11T23:31:27.756+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b92d6f18:::obj505:head local-lis/les=113/114 n=446 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.756+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.756+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:ba083999:::obj295:head, version: 111'295, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba083999:::obj295:head@111'295, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:27.756+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.756+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.756+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:b9836a99:::obj18:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.756+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b9836a99:::obj18:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 1.0 e 116/116 lb 1:b9836a99:::obj18:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b92d6f18:::obj505:head local-lis/les=113/114 n=446 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.756+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.756+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b92d6f18:::obj505:head local-lis/les=113/114 n=446 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:b9836a99:::obj18:head) v3 2026-02-11T23:31:27.756+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b92d6f18:::obj505:head local-lis/les=113/114 n=446 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:b9836a99:::obj18:head) 2026-02-11T23:31:27.756+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.756+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:b9836a99:::obj18:head) v3 finish latency 0.000009 2026-02-11T23:31:27.756+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b9836a99:::obj18:head local-lis/les=113/114 n=447 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.756+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.756+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.756+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b9836a99:::obj18:head local-lis/les=113/114 n=447 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.756+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.756+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.756+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.756+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fcf680 con 0x4acb000 2026-02-11T23:31:27.756+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:ba083999:::obj295:head)]) -- 0x5fed800 con 0x4acb000 2026-02-11T23:31:27.756+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.757+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 931 ==== MOSDPGPush(1.0 116/113 [PushOp(1:ba3d236d:::obj529:head, version: 111'529, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba3d236d:::obj529:head@111'529, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56ca1a0 con 0x4acb000 2026-02-11T23:31:27.757+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:ba3d236d:::obj529:head, version: 111'529, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba3d236d:::obj529:head@111'529, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 116 2026-02-11T23:31:27.757+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:ba3d236d:::obj529:head, version: 111'529, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba3d236d:::obj529:head@111'529, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.757+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.757+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 932 ==== pg_backfill(progress 1.0 e 116/116 lb 1:ba083999:::obj295:head) ==== 1062+0+0 (unknown 0 0 0) 0x5fbfb80 con 0x4acb000 2026-02-11T23:31:27.757+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:ba083999:::obj295:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.757+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:ba3d236d:::obj529:head, version: 111'529, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba3d236d:::obj529:head@111'529, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.757+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:ba083999:::obj295:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.757+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.757+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:ba3d236d:::obj529:head, version: 111'529, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba3d236d:::obj529:head@111'529, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.757+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:ba3d236d:::obj529:head, version: 111'529, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba3d236d:::obj529:head@111'529, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 116/113 [PushOp(1:ba3d236d:::obj529:head, version: 111'529, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba3d236d:::obj529:head@111'529, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b9836a99:::obj18:head local-lis/les=113/114 n=447 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.757+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.757+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.757+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:ba083999:::obj295:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.757+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b9836a99:::obj18:head local-lis/les=113/114 n=447 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:ba3d236d:::obj529:head, version: 111'529, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba3d236d:::obj529:head@111'529, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.757+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b9836a99:::obj18:head local-lis/les=113/114 n=447 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:ba3d236d:::obj529:head@111'529, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.757+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b9836a99:::obj18:head local-lis/les=113/114 n=447 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:ba3d236d:::obj529:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.757+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b9836a99:::obj18:head local-lis/les=113/114 n=447 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:ba3d236d:::obj529:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.757+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b9836a99:::obj18:head local-lis/les=113/114 n=447 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:ba3d236d:::obj529:head 2026-02-11T23:31:27.757+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b9836a99:::obj18:head local-lis/les=113/114 n=447 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:ba3d236d:::obj529:head v 111'529 2026-02-11T23:31:27.757+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b9836a99:::obj18:head local-lis/les=113/114 n=447 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.757+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.757+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:ba3d236d:::obj529:head, version: 111'529, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba3d236d:::obj529:head@111'529, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:27.757+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.757+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.757+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:ba083999:::obj295:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.757+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:ba083999:::obj295:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 1.0 e 116/116 lb 1:ba083999:::obj295:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:b9836a99:::obj18:head local-lis/les=113/114 n=447 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.757+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.757+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b9836a99:::obj18:head local-lis/les=113/114 n=447 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:ba083999:::obj295:head) v3 2026-02-11T23:31:27.757+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:b9836a99:::obj18:head local-lis/les=113/114 n=447 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:ba083999:::obj295:head) 2026-02-11T23:31:27.757+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.757+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:ba083999:::obj295:head) v3 finish latency 0.000009 2026-02-11T23:31:27.757+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ba083999:::obj295:head local-lis/les=113/114 n=448 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.757+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.757+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.757+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ba083999:::obj295:head local-lis/les=113/114 n=448 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.757+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.757+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.757+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.757+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d1de00 con 0x4acb000 2026-02-11T23:31:27.757+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:ba3d236d:::obj529:head)]) -- 0x5fed080 con 0x4acb000 2026-02-11T23:31:27.757+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.757+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 933 ==== MOSDPGPush(1.0 116/113 [PushOp(1:ba98db39:::obj542:head, version: 111'542, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba98db39:::obj542:head@111'542, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d8680 con 0x4acb000 2026-02-11T23:31:27.757+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:ba98db39:::obj542:head, version: 111'542, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba98db39:::obj542:head@111'542, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000016 epoch 116 2026-02-11T23:31:27.757+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:ba98db39:::obj542:head, version: 111'542, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba98db39:::obj542:head@111'542, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.757+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.757+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 934 ==== pg_backfill(progress 1.0 e 116/116 lb 1:ba3d236d:::obj529:head) ==== 1062+0+0 (unknown 0 0 0) 0x5fc0100 con 0x4acb000 2026-02-11T23:31:27.757+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:ba98db39:::obj542:head, version: 111'542, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba98db39:::obj542:head@111'542, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.757+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:ba3d236d:::obj529:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.757+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.757+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:ba3d236d:::obj529:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.757+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:ba98db39:::obj542:head, version: 111'542, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba98db39:::obj542:head@111'542, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.757+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.757+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:ba98db39:::obj542:head, version: 111'542, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba98db39:::obj542:head@111'542, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000043 MOSDPGPush(1.0 116/113 [PushOp(1:ba98db39:::obj542:head, version: 111'542, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba98db39:::obj542:head@111'542, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:ba083999:::obj295:head local-lis/les=113/114 n=448 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.757+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.757+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ba083999:::obj295:head local-lis/les=113/114 n=448 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:ba98db39:::obj542:head, version: 111'542, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba98db39:::obj542:head@111'542, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.757+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ba083999:::obj295:head local-lis/les=113/114 n=448 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:ba98db39:::obj542:head@111'542, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.757+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.757+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:ba3d236d:::obj529:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.757+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ba083999:::obj295:head local-lis/les=113/114 n=448 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:ba98db39:::obj542:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.757+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ba083999:::obj295:head local-lis/les=113/114 n=448 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:ba98db39:::obj542:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.757+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ba083999:::obj295:head local-lis/les=113/114 n=448 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:ba98db39:::obj542:head 2026-02-11T23:31:27.757+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ba083999:::obj295:head local-lis/les=113/114 n=448 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:ba98db39:::obj542:head v 111'542 2026-02-11T23:31:27.757+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ba083999:::obj295:head local-lis/les=113/114 n=448 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.757+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.757+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:ba98db39:::obj542:head, version: 111'542, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba98db39:::obj542:head@111'542, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.757+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.757+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.757+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:ba3d236d:::obj529:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.757+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:ba3d236d:::obj529:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 1.0 e 116/116 lb 1:ba3d236d:::obj529:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:ba083999:::obj295:head local-lis/les=113/114 n=448 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.757+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.757+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ba083999:::obj295:head local-lis/les=113/114 n=448 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:ba3d236d:::obj529:head) v3 2026-02-11T23:31:27.757+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ba083999:::obj295:head local-lis/les=113/114 n=448 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:ba3d236d:::obj529:head) 2026-02-11T23:31:27.757+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.757+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:ba3d236d:::obj529:head) v3 finish latency 0.000009 2026-02-11T23:31:27.757+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ba3d236d:::obj529:head local-lis/les=113/114 n=449 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.757+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.757+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.757+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ba3d236d:::obj529:head local-lis/les=113/114 n=449 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.757+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.757+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.757+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.757+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x600a480 con 0x4acb000 2026-02-11T23:31:27.757+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:ba98db39:::obj542:head)]) -- 0x612ed80 con 0x4acb000 2026-02-11T23:31:27.757+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.758+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 935 ==== MOSDPGPush(1.0 116/113 [PushOp(1:bb038e88:::obj54:head, version: 111'54, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bb038e88:::obj54:head@111'54, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x559dd40 con 0x4acb000 2026-02-11T23:31:27.758+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:bb038e88:::obj54:head, version: 111'54, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bb038e88:::obj54:head@111'54, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 116 2026-02-11T23:31:27.758+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:bb038e88:::obj54:head, version: 111'54, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bb038e88:::obj54:head@111'54, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.758+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.758+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 936 ==== pg_backfill(progress 1.0 e 116/116 lb 1:ba98db39:::obj542:head) ==== 1062+0+0 (unknown 0 0 0) 0x5fc0680 con 0x4acb000 2026-02-11T23:31:27.758+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:ba98db39:::obj542:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.758+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:bb038e88:::obj54:head, version: 111'54, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bb038e88:::obj54:head@111'54, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.758+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.758+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:ba98db39:::obj542:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.758+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:bb038e88:::obj54:head, version: 111'54, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bb038e88:::obj54:head@111'54, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.758+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.758+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:bb038e88:::obj54:head, version: 111'54, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bb038e88:::obj54:head@111'54, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(1.0 116/113 [PushOp(1:bb038e88:::obj54:head, version: 111'54, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bb038e88:::obj54:head@111'54, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:ba3d236d:::obj529:head local-lis/les=113/114 n=449 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.758+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.758+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ba3d236d:::obj529:head local-lis/les=113/114 n=449 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:bb038e88:::obj54:head, version: 111'54, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bb038e88:::obj54:head@111'54, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.758+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ba3d236d:::obj529:head local-lis/les=113/114 n=449 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:bb038e88:::obj54:head@111'54, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.758+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.758+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:ba98db39:::obj542:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.758+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ba3d236d:::obj529:head local-lis/les=113/114 n=449 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:bb038e88:::obj54:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.758+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ba3d236d:::obj529:head local-lis/les=113/114 n=449 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:bb038e88:::obj54:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.758+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ba3d236d:::obj529:head local-lis/les=113/114 n=449 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:bb038e88:::obj54:head 2026-02-11T23:31:27.758+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ba3d236d:::obj529:head local-lis/les=113/114 n=449 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:bb038e88:::obj54:head v 111'54 2026-02-11T23:31:27.758+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ba3d236d:::obj529:head local-lis/les=113/114 n=449 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.758+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.758+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:bb038e88:::obj54:head, version: 111'54, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bb038e88:::obj54:head@111'54, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:27.758+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.758+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.758+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:ba98db39:::obj542:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.758+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:ba98db39:::obj542:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 1.0 e 116/116 lb 1:ba98db39:::obj542:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:ba3d236d:::obj529:head local-lis/les=113/114 n=449 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.758+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.758+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ba3d236d:::obj529:head local-lis/les=113/114 n=449 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:ba98db39:::obj542:head) v3 2026-02-11T23:31:27.758+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ba3d236d:::obj529:head local-lis/les=113/114 n=449 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:ba98db39:::obj542:head) 2026-02-11T23:31:27.758+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.758+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:ba98db39:::obj542:head) v3 finish latency 0.000010 2026-02-11T23:31:27.758+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ba98db39:::obj542:head local-lis/les=113/114 n=450 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.758+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.758+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.758+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ba98db39:::obj542:head local-lis/les=113/114 n=450 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.758+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.758+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.758+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.758+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x612e300 con 0x4acb000 2026-02-11T23:31:27.758+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:bb038e88:::obj54:head)]) -- 0x6170300 con 0x4acb000 2026-02-11T23:31:27.758+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.758+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 937 ==== MOSDPGPush(1.0 116/113 [PushOp(1:bbf67374:::obj187:head, version: 111'187, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bbf67374:::obj187:head@111'187, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c820 con 0x4acb000 2026-02-11T23:31:27.758+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:bbf67374:::obj187:head, version: 111'187, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bbf67374:::obj187:head@111'187, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 116 2026-02-11T23:31:27.758+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:bbf67374:::obj187:head, version: 111'187, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bbf67374:::obj187:head@111'187, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.758+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.758+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 938 ==== pg_backfill(progress 1.0 e 116/116 lb 1:bb038e88:::obj54:head) ==== 1061+0+0 (unknown 0 0 0) 0x5fc0c00 con 0x4acb000 2026-02-11T23:31:27.758+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:bbf67374:::obj187:head, version: 111'187, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bbf67374:::obj187:head@111'187, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.758+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:bb038e88:::obj54:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.758+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:bb038e88:::obj54:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.758+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.758+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:bbf67374:::obj187:head, version: 111'187, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bbf67374:::obj187:head@111'187, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.758+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:bbf67374:::obj187:head, version: 111'187, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bbf67374:::obj187:head@111'187, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(1.0 116/113 [PushOp(1:bbf67374:::obj187:head, version: 111'187, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bbf67374:::obj187:head@111'187, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:ba98db39:::obj542:head local-lis/les=113/114 n=450 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.758+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.758+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.758+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ba98db39:::obj542:head local-lis/les=113/114 n=450 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:bbf67374:::obj187:head, version: 111'187, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bbf67374:::obj187:head@111'187, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.758+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:bb038e88:::obj54:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.758+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ba98db39:::obj542:head local-lis/les=113/114 n=450 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:bbf67374:::obj187:head@111'187, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.758+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ba98db39:::obj542:head local-lis/les=113/114 n=450 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:bbf67374:::obj187:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.758+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ba98db39:::obj542:head local-lis/les=113/114 n=450 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:bbf67374:::obj187:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.758+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ba98db39:::obj542:head local-lis/les=113/114 n=450 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:bbf67374:::obj187:head 2026-02-11T23:31:27.758+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ba98db39:::obj542:head local-lis/les=113/114 n=450 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:bbf67374:::obj187:head v 111'187 2026-02-11T23:31:27.758+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ba98db39:::obj542:head local-lis/les=113/114 n=450 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.758+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.758+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:bbf67374:::obj187:head, version: 111'187, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bbf67374:::obj187:head@111'187, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.758+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.758+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.758+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:bb038e88:::obj54:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.758+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:bb038e88:::obj54:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 1.0 e 116/116 lb 1:bb038e88:::obj54:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:ba98db39:::obj542:head local-lis/les=113/114 n=450 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.758+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.758+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ba98db39:::obj542:head local-lis/les=113/114 n=450 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:bb038e88:::obj54:head) v3 2026-02-11T23:31:27.758+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ba98db39:::obj542:head local-lis/les=113/114 n=450 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:bb038e88:::obj54:head) 2026-02-11T23:31:27.758+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.758+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:bb038e88:::obj54:head) v3 finish latency 0.000010 2026-02-11T23:31:27.758+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bb038e88:::obj54:head local-lis/les=113/114 n=451 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.758+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.759+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.759+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bb038e88:::obj54:head local-lis/les=113/114 n=451 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.759+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.759+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.759+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.759+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fcf800 con 0x4acb000 2026-02-11T23:31:27.759+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:bbf67374:::obj187:head)]) -- 0x5d1d380 con 0x4acb000 2026-02-11T23:31:27.759+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.759+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 939 ==== MOSDPGPush(1.0 116/113 [PushOp(1:bc55eba4:::obj534:head, version: 111'534, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bc55eba4:::obj534:head@111'534, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c340 con 0x4acb000 2026-02-11T23:31:27.759+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:bc55eba4:::obj534:head, version: 111'534, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bc55eba4:::obj534:head@111'534, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.759+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:bc55eba4:::obj534:head, version: 111'534, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bc55eba4:::obj534:head@111'534, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.759+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.759+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 940 ==== pg_backfill(progress 1.0 e 116/116 lb 1:bbf67374:::obj187:head) ==== 1062+0+0 (unknown 0 0 0) 0x5fc1180 con 0x4acb000 2026-02-11T23:31:27.759+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:bbf67374:::obj187:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.759+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:bc55eba4:::obj534:head, version: 111'534, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bc55eba4:::obj534:head@111'534, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.759+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:bbf67374:::obj187:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.759+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.759+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:bc55eba4:::obj534:head, version: 111'534, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bc55eba4:::obj534:head@111'534, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.759+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:bc55eba4:::obj534:head, version: 111'534, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bc55eba4:::obj534:head@111'534, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 116/113 [PushOp(1:bc55eba4:::obj534:head, version: 111'534, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bc55eba4:::obj534:head@111'534, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:bb038e88:::obj54:head local-lis/les=113/114 n=451 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.759+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.759+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.759+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:bbf67374:::obj187:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.759+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bb038e88:::obj54:head local-lis/les=113/114 n=451 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:bc55eba4:::obj534:head, version: 111'534, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bc55eba4:::obj534:head@111'534, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.759+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bb038e88:::obj54:head local-lis/les=113/114 n=451 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:bc55eba4:::obj534:head@111'534, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.759+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bb038e88:::obj54:head local-lis/les=113/114 n=451 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:bc55eba4:::obj534:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.759+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bb038e88:::obj54:head local-lis/les=113/114 n=451 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:bc55eba4:::obj534:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.759+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bb038e88:::obj54:head local-lis/les=113/114 n=451 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:bc55eba4:::obj534:head 2026-02-11T23:31:27.759+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bb038e88:::obj54:head local-lis/les=113/114 n=451 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:bc55eba4:::obj534:head v 111'534 2026-02-11T23:31:27.759+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bb038e88:::obj54:head local-lis/les=113/114 n=451 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.759+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.759+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:bc55eba4:::obj534:head, version: 111'534, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bc55eba4:::obj534:head@111'534, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.759+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.759+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.759+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:bbf67374:::obj187:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.759+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:bbf67374:::obj187:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 1.0 e 116/116 lb 1:bbf67374:::obj187:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:bb038e88:::obj54:head local-lis/les=113/114 n=451 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.759+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.759+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bb038e88:::obj54:head local-lis/les=113/114 n=451 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:bbf67374:::obj187:head) v3 2026-02-11T23:31:27.759+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bb038e88:::obj54:head local-lis/les=113/114 n=451 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:bbf67374:::obj187:head) 2026-02-11T23:31:27.759+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.759+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:bbf67374:::obj187:head) v3 finish latency 0.000009 2026-02-11T23:31:27.759+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bbf67374:::obj187:head local-lis/les=113/114 n=452 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.759+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.759+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.759+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bbf67374:::obj187:head local-lis/les=113/114 n=452 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.759+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.759+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.759+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.759+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6171680 con 0x4acb000 2026-02-11T23:31:27.759+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:bc55eba4:::obj534:head)]) -- 0x5d1d080 con 0x4acb000 2026-02-11T23:31:27.759+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.759+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 941 ==== MOSDPGPush(1.0 116/113 [PushOp(1:bd84d997:::obj292:head, version: 111'292, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bd84d997:::obj292:head@111'292, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559d1e0 con 0x4acb000 2026-02-11T23:31:27.759+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:bd84d997:::obj292:head, version: 111'292, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bd84d997:::obj292:head@111'292, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 116 2026-02-11T23:31:27.759+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:bd84d997:::obj292:head, version: 111'292, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bd84d997:::obj292:head@111'292, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.759+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.759+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 942 ==== pg_backfill(progress 1.0 e 116/116 lb 1:bc55eba4:::obj534:head) ==== 1062+0+0 (unknown 0 0 0) 0x5fc1700 con 0x4acb000 2026-02-11T23:31:27.759+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:bc55eba4:::obj534:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.759+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:bd84d997:::obj292:head, version: 111'292, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bd84d997:::obj292:head@111'292, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.759+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:bc55eba4:::obj534:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.759+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.759+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:bd84d997:::obj292:head, version: 111'292, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bd84d997:::obj292:head@111'292, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.759+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:bd84d997:::obj292:head, version: 111'292, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bd84d997:::obj292:head@111'292, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(1.0 116/113 [PushOp(1:bd84d997:::obj292:head, version: 111'292, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bd84d997:::obj292:head@111'292, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:bbf67374:::obj187:head local-lis/les=113/114 n=452 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.759+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.759+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.759+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:bc55eba4:::obj534:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.759+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bbf67374:::obj187:head local-lis/les=113/114 n=452 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:bd84d997:::obj292:head, version: 111'292, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bd84d997:::obj292:head@111'292, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.759+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bbf67374:::obj187:head local-lis/les=113/114 n=452 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:bd84d997:::obj292:head@111'292, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.759+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bbf67374:::obj187:head local-lis/les=113/114 n=452 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:bd84d997:::obj292:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.759+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bbf67374:::obj187:head local-lis/les=113/114 n=452 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:bd84d997:::obj292:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.759+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bbf67374:::obj187:head local-lis/les=113/114 n=452 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:bd84d997:::obj292:head 2026-02-11T23:31:27.759+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bbf67374:::obj187:head local-lis/les=113/114 n=452 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:bd84d997:::obj292:head v 111'292 2026-02-11T23:31:27.759+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bbf67374:::obj187:head local-lis/les=113/114 n=452 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.759+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.759+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:bd84d997:::obj292:head, version: 111'292, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bd84d997:::obj292:head@111'292, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.759+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.759+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.760+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:bc55eba4:::obj534:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.760+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:bc55eba4:::obj534:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:bc55eba4:::obj534:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:bbf67374:::obj187:head local-lis/les=113/114 n=452 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.760+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.760+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bbf67374:::obj187:head local-lis/les=113/114 n=452 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:bc55eba4:::obj534:head) v3 2026-02-11T23:31:27.760+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bbf67374:::obj187:head local-lis/les=113/114 n=452 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:bc55eba4:::obj534:head) 2026-02-11T23:31:27.760+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.760+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:bc55eba4:::obj534:head) v3 finish latency 0.000011 2026-02-11T23:31:27.760+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bc55eba4:::obj534:head local-lis/les=113/114 n=453 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.760+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.760+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.760+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bc55eba4:::obj534:head local-lis/les=113/114 n=453 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.760+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.760+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.760+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.760+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x600a900 con 0x4acb000 2026-02-11T23:31:27.760+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:bd84d997:::obj292:head)]) -- 0x5d1cd80 con 0x4acb000 2026-02-11T23:31:27.760+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.760+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 943 ==== MOSDPGPush(1.0 116/113 [PushOp(1:bdc5b82e:::obj52:head, version: 111'52, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bdc5b82e:::obj52:head@111'52, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x559cd00 con 0x4acb000 2026-02-11T23:31:27.760+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:bdc5b82e:::obj52:head, version: 111'52, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bdc5b82e:::obj52:head@111'52, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 116 2026-02-11T23:31:27.760+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:bdc5b82e:::obj52:head, version: 111'52, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bdc5b82e:::obj52:head@111'52, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.760+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.760+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:bdc5b82e:::obj52:head, version: 111'52, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bdc5b82e:::obj52:head@111'52, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.760+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 944 ==== pg_backfill(progress 1.0 e 116/116 lb 1:bd84d997:::obj292:head) ==== 1062+0+0 (unknown 0 0 0) 0x5fbe000 con 0x4acb000 2026-02-11T23:31:27.760+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.760+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:bd84d997:::obj292:head) v3 prio 15 type 95 cost 0 latency 0.000010 epoch 116 2026-02-11T23:31:27.760+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:bdc5b82e:::obj52:head, version: 111'52, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bdc5b82e:::obj52:head@111'52, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.760+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:bd84d997:::obj292:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.760+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.760+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:bd84d997:::obj292:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.760+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:bdc5b82e:::obj52:head, version: 111'52, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bdc5b82e:::obj52:head@111'52, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(1.0 116/113 [PushOp(1:bdc5b82e:::obj52:head, version: 111'52, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bdc5b82e:::obj52:head@111'52, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:bc55eba4:::obj534:head local-lis/les=113/114 n=453 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.760+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.760+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bc55eba4:::obj534:head local-lis/les=113/114 n=453 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:bdc5b82e:::obj52:head, version: 111'52, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bdc5b82e:::obj52:head@111'52, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.760+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bc55eba4:::obj534:head local-lis/les=113/114 n=453 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:bdc5b82e:::obj52:head@111'52, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.760+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bc55eba4:::obj534:head local-lis/les=113/114 n=453 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:bdc5b82e:::obj52:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.760+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bc55eba4:::obj534:head local-lis/les=113/114 n=453 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:bdc5b82e:::obj52:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.760+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bc55eba4:::obj534:head local-lis/les=113/114 n=453 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:bdc5b82e:::obj52:head 2026-02-11T23:31:27.760+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bc55eba4:::obj534:head local-lis/les=113/114 n=453 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:bdc5b82e:::obj52:head v 111'52 2026-02-11T23:31:27.760+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bc55eba4:::obj534:head local-lis/les=113/114 n=453 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.760+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.760+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:bdc5b82e:::obj52:head, version: 111'52, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bdc5b82e:::obj52:head@111'52, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000044 2026-02-11T23:31:27.760+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.760+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.760+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:bd84d997:::obj292:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.760+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:bd84d997:::obj292:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 1.0 e 116/116 lb 1:bd84d997:::obj292:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:bc55eba4:::obj534:head local-lis/les=113/114 n=453 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.760+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.760+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bc55eba4:::obj534:head local-lis/les=113/114 n=453 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:bd84d997:::obj292:head) v3 2026-02-11T23:31:27.760+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bc55eba4:::obj534:head local-lis/les=113/114 n=453 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:bd84d997:::obj292:head) 2026-02-11T23:31:27.760+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.760+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:bd84d997:::obj292:head) v3 finish latency 0.000010 2026-02-11T23:31:27.760+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bd84d997:::obj292:head local-lis/les=113/114 n=454 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.760+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.760+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.760+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bd84d997:::obj292:head local-lis/les=113/114 n=454 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.760+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.760+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.760+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.760+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6171080 con 0x4acb000 2026-02-11T23:31:27.760+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:bdc5b82e:::obj52:head)]) -- 0x5d1ca80 con 0x4acb000 2026-02-11T23:31:27.760+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.760+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 945 ==== MOSDPGPush(1.0 116/113 [PushOp(1:bdf2c185:::obj359:head, version: 111'359, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bdf2c185:::obj359:head@111'359, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e261a0 con 0x4acb000 2026-02-11T23:31:27.760+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:bdf2c185:::obj359:head, version: 111'359, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bdf2c185:::obj359:head@111'359, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 116 2026-02-11T23:31:27.760+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:bdf2c185:::obj359:head, version: 111'359, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bdf2c185:::obj359:head@111'359, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.761+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.761+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 946 ==== pg_backfill(progress 1.0 e 116/116 lb 1:bdc5b82e:::obj52:head) ==== 1061+0+0 (unknown 0 0 0) 0x5fbe580 con 0x4acb000 2026-02-11T23:31:27.761+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:bdc5b82e:::obj52:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.761+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:bdf2c185:::obj359:head, version: 111'359, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bdf2c185:::obj359:head@111'359, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.761+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:bdc5b82e:::obj52:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.761+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.761+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:bdc5b82e:::obj52:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.761+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.761+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:bdf2c185:::obj359:head, version: 111'359, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bdf2c185:::obj359:head@111'359, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.761+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:bdf2c185:::obj359:head, version: 111'359, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bdf2c185:::obj359:head@111'359, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000047 MOSDPGPush(1.0 116/113 [PushOp(1:bdf2c185:::obj359:head, version: 111'359, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bdf2c185:::obj359:head@111'359, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:bd84d997:::obj292:head local-lis/les=113/114 n=454 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.761+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.761+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bd84d997:::obj292:head local-lis/les=113/114 n=454 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:bdf2c185:::obj359:head, version: 111'359, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bdf2c185:::obj359:head@111'359, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.761+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bd84d997:::obj292:head local-lis/les=113/114 n=454 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:bdf2c185:::obj359:head@111'359, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.761+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bd84d997:::obj292:head local-lis/les=113/114 n=454 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:bdf2c185:::obj359:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.761+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bd84d997:::obj292:head local-lis/les=113/114 n=454 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:bdf2c185:::obj359:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.761+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bd84d997:::obj292:head local-lis/les=113/114 n=454 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:bdf2c185:::obj359:head 2026-02-11T23:31:27.761+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bd84d997:::obj292:head local-lis/les=113/114 n=454 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:bdf2c185:::obj359:head v 111'359 2026-02-11T23:31:27.761+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bd84d997:::obj292:head local-lis/les=113/114 n=454 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.761+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.761+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:bdf2c185:::obj359:head, version: 111'359, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bdf2c185:::obj359:head@111'359, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:27.761+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.761+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.761+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:bdc5b82e:::obj52:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.761+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:bdc5b82e:::obj52:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 1.0 e 116/116 lb 1:bdc5b82e:::obj52:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:bd84d997:::obj292:head local-lis/les=113/114 n=454 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.761+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.761+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bd84d997:::obj292:head local-lis/les=113/114 n=454 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:bdc5b82e:::obj52:head) v3 2026-02-11T23:31:27.761+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bd84d997:::obj292:head local-lis/les=113/114 n=454 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:bdc5b82e:::obj52:head) 2026-02-11T23:31:27.761+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.761+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:bdc5b82e:::obj52:head) v3 finish latency 0.000010 2026-02-11T23:31:27.761+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bdc5b82e:::obj52:head local-lis/les=113/114 n=455 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.761+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.761+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.761+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bdc5b82e:::obj52:head local-lis/les=113/114 n=455 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.761+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.761+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.761+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.761+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fcfb00 con 0x4acb000 2026-02-11T23:31:27.761+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:bdf2c185:::obj359:head)]) -- 0x5fec780 con 0x4acb000 2026-02-11T23:31:27.761+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.761+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 947 ==== MOSDPGPush(1.0 116/113 [PushOp(1:be10f7c9:::obj475:head, version: 111'475, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be10f7c9:::obj475:head@111'475, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f1d40 con 0x4acb000 2026-02-11T23:31:27.761+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:be10f7c9:::obj475:head, version: 111'475, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be10f7c9:::obj475:head@111'475, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.761+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:be10f7c9:::obj475:head, version: 111'475, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be10f7c9:::obj475:head@111'475, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.761+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.761+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 948 ==== pg_backfill(progress 1.0 e 116/116 lb 1:bdf2c185:::obj359:head) ==== 1062+0+0 (unknown 0 0 0) 0x5fbeb00 con 0x4acb000 2026-02-11T23:31:27.761+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:be10f7c9:::obj475:head, version: 111'475, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be10f7c9:::obj475:head@111'475, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.761+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:bdf2c185:::obj359:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.761+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:bdf2c185:::obj359:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.761+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.761+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:be10f7c9:::obj475:head, version: 111'475, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be10f7c9:::obj475:head@111'475, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.761+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.761+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:be10f7c9:::obj475:head, version: 111'475, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be10f7c9:::obj475:head@111'475, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 116/113 [PushOp(1:be10f7c9:::obj475:head, version: 111'475, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be10f7c9:::obj475:head@111'475, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:bdc5b82e:::obj52:head local-lis/les=113/114 n=455 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.761+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.761+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bdc5b82e:::obj52:head local-lis/les=113/114 n=455 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:be10f7c9:::obj475:head, version: 111'475, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be10f7c9:::obj475:head@111'475, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.761+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bdc5b82e:::obj52:head local-lis/les=113/114 n=455 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:be10f7c9:::obj475:head@111'475, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.761+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.761+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:bdf2c185:::obj359:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.761+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bdc5b82e:::obj52:head local-lis/les=113/114 n=455 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:be10f7c9:::obj475:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.761+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bdc5b82e:::obj52:head local-lis/les=113/114 n=455 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:be10f7c9:::obj475:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.761+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bdc5b82e:::obj52:head local-lis/les=113/114 n=455 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:be10f7c9:::obj475:head 2026-02-11T23:31:27.761+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bdc5b82e:::obj52:head local-lis/les=113/114 n=455 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:be10f7c9:::obj475:head v 111'475 2026-02-11T23:31:27.761+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bdc5b82e:::obj52:head local-lis/les=113/114 n=455 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.761+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.761+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:be10f7c9:::obj475:head, version: 111'475, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be10f7c9:::obj475:head@111'475, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.761+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.761+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.761+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:bdf2c185:::obj359:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.761+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:bdf2c185:::obj359:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 1.0 e 116/116 lb 1:bdf2c185:::obj359:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:bdc5b82e:::obj52:head local-lis/les=113/114 n=455 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.761+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.761+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bdc5b82e:::obj52:head local-lis/les=113/114 n=455 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:bdf2c185:::obj359:head) v3 2026-02-11T23:31:27.761+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bdc5b82e:::obj52:head local-lis/les=113/114 n=455 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:bdf2c185:::obj359:head) 2026-02-11T23:31:27.761+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.761+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:bdf2c185:::obj359:head) v3 finish latency 0.000009 2026-02-11T23:31:27.761+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bdf2c185:::obj359:head local-lis/les=113/114 n=456 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.761+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.761+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.761+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bdf2c185:::obj359:head local-lis/les=113/114 n=456 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.761+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.761+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.761+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.761+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x612e600 con 0x4acb000 2026-02-11T23:31:27.761+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:be10f7c9:::obj475:head)]) -- 0x5fec600 con 0x4acb000 2026-02-11T23:31:27.761+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.762+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 949 ==== MOSDPGPush(1.0 116/113 [PushOp(1:be1f0c3b:::obj423:head, version: 111'423, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be1f0c3b:::obj423:head@111'423, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x555cea0 con 0x4acb000 2026-02-11T23:31:27.762+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:be1f0c3b:::obj423:head, version: 111'423, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be1f0c3b:::obj423:head@111'423, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.762+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:be1f0c3b:::obj423:head, version: 111'423, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be1f0c3b:::obj423:head@111'423, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.762+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.762+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 950 ==== pg_backfill(progress 1.0 e 116/116 lb 1:be10f7c9:::obj475:head) ==== 1062+0+0 (unknown 0 0 0) 0x5fbf080 con 0x4acb000 2026-02-11T23:31:27.762+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:be10f7c9:::obj475:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.762+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:be1f0c3b:::obj423:head, version: 111'423, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be1f0c3b:::obj423:head@111'423, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.762+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:be10f7c9:::obj475:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.762+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.762+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:be1f0c3b:::obj423:head, version: 111'423, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be1f0c3b:::obj423:head@111'423, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.762+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:be1f0c3b:::obj423:head, version: 111'423, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be1f0c3b:::obj423:head@111'423, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:be1f0c3b:::obj423:head, version: 111'423, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be1f0c3b:::obj423:head@111'423, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:bdf2c185:::obj359:head local-lis/les=113/114 n=456 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.762+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.762+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.762+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:be10f7c9:::obj475:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.762+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bdf2c185:::obj359:head local-lis/les=113/114 n=456 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:be1f0c3b:::obj423:head, version: 111'423, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be1f0c3b:::obj423:head@111'423, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.762+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bdf2c185:::obj359:head local-lis/les=113/114 n=456 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:be1f0c3b:::obj423:head@111'423, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.762+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bdf2c185:::obj359:head local-lis/les=113/114 n=456 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:be1f0c3b:::obj423:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.762+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bdf2c185:::obj359:head local-lis/les=113/114 n=456 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:be1f0c3b:::obj423:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.762+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bdf2c185:::obj359:head local-lis/les=113/114 n=456 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:be1f0c3b:::obj423:head 2026-02-11T23:31:27.762+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bdf2c185:::obj359:head local-lis/les=113/114 n=456 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:be1f0c3b:::obj423:head v 111'423 2026-02-11T23:31:27.762+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bdf2c185:::obj359:head local-lis/les=113/114 n=456 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.762+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.762+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:be1f0c3b:::obj423:head, version: 111'423, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be1f0c3b:::obj423:head@111'423, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:27.762+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.762+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.762+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:be10f7c9:::obj475:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.762+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:be10f7c9:::obj475:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 1.0 e 116/116 lb 1:be10f7c9:::obj475:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:bdf2c185:::obj359:head local-lis/les=113/114 n=456 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.762+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.762+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bdf2c185:::obj359:head local-lis/les=113/114 n=456 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:be10f7c9:::obj475:head) v3 2026-02-11T23:31:27.762+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bdf2c185:::obj359:head local-lis/les=113/114 n=456 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:be10f7c9:::obj475:head) 2026-02-11T23:31:27.762+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.762+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:be10f7c9:::obj475:head) v3 finish latency 0.000009 2026-02-11T23:31:27.762+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:be10f7c9:::obj475:head local-lis/les=113/114 n=457 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.762+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.762+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.762+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:be10f7c9:::obj475:head local-lis/les=113/114 n=457 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.762+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.762+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.762+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.762+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x600be00 con 0x4acb000 2026-02-11T23:31:27.762+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:be1f0c3b:::obj423:head)]) -- 0x5fec180 con 0x4acb000 2026-02-11T23:31:27.762+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.762+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 951 ==== MOSDPGPush(1.0 116/113 [PushOp(1:be604f9b:::obj512:head, version: 111'512, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be604f9b:::obj512:head@111'512, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f1860 con 0x4acb000 2026-02-11T23:31:27.762+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:be604f9b:::obj512:head, version: 111'512, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be604f9b:::obj512:head@111'512, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 116 2026-02-11T23:31:27.762+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:be604f9b:::obj512:head, version: 111'512, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be604f9b:::obj512:head@111'512, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.762+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.762+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 952 ==== pg_backfill(progress 1.0 e 116/116 lb 1:be1f0c3b:::obj423:head) ==== 1062+0+0 (unknown 0 0 0) 0x6179700 con 0x4acb000 2026-02-11T23:31:27.762+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:be604f9b:::obj512:head, version: 111'512, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be604f9b:::obj512:head@111'512, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.762+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:be1f0c3b:::obj423:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.762+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:be1f0c3b:::obj423:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.762+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.762+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:be604f9b:::obj512:head, version: 111'512, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be604f9b:::obj512:head@111'512, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.762+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:be604f9b:::obj512:head, version: 111'512, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be604f9b:::obj512:head@111'512, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(1.0 116/113 [PushOp(1:be604f9b:::obj512:head, version: 111'512, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be604f9b:::obj512:head@111'512, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:be10f7c9:::obj475:head local-lis/les=113/114 n=457 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.762+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.762+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.762+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:be10f7c9:::obj475:head local-lis/les=113/114 n=457 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:be604f9b:::obj512:head, version: 111'512, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be604f9b:::obj512:head@111'512, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.762+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:be1f0c3b:::obj423:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.762+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:be10f7c9:::obj475:head local-lis/les=113/114 n=457 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:be604f9b:::obj512:head@111'512, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.762+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:be10f7c9:::obj475:head local-lis/les=113/114 n=457 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:be604f9b:::obj512:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.762+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:be10f7c9:::obj475:head local-lis/les=113/114 n=457 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:be604f9b:::obj512:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.762+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:be10f7c9:::obj475:head local-lis/les=113/114 n=457 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:be604f9b:::obj512:head 2026-02-11T23:31:27.762+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:be10f7c9:::obj475:head local-lis/les=113/114 n=457 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:be604f9b:::obj512:head v 111'512 2026-02-11T23:31:27.762+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:be10f7c9:::obj475:head local-lis/les=113/114 n=457 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.762+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.762+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:be604f9b:::obj512:head, version: 111'512, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be604f9b:::obj512:head@111'512, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:27.762+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.762+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.762+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:be1f0c3b:::obj423:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.762+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:be1f0c3b:::obj423:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:be1f0c3b:::obj423:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:be10f7c9:::obj475:head local-lis/les=113/114 n=457 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.762+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.762+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:be10f7c9:::obj475:head local-lis/les=113/114 n=457 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:be1f0c3b:::obj423:head) v3 2026-02-11T23:31:27.762+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:be10f7c9:::obj475:head local-lis/les=113/114 n=457 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:be1f0c3b:::obj423:head) 2026-02-11T23:31:27.762+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.762+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:be1f0c3b:::obj423:head) v3 finish latency 0.000009 2026-02-11T23:31:27.762+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:be1f0c3b:::obj423:head local-lis/les=113/114 n=458 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.762+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.762+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.762+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:be1f0c3b:::obj423:head local-lis/les=113/114 n=458 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.762+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.762+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.762+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.762+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5ca0d80 con 0x4acb000 2026-02-11T23:31:27.762+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:be604f9b:::obj512:head)]) -- 0x5fec000 con 0x4acb000 2026-02-11T23:31:27.762+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.763+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 953 ==== MOSDPGPush(1.0 116/113 [PushOp(1:beac82f5:::obj51:head, version: 111'51, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:beac82f5:::obj51:head@111'51, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x559c4e0 con 0x4acb000 2026-02-11T23:31:27.763+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:beac82f5:::obj51:head, version: 111'51, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:beac82f5:::obj51:head@111'51, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000016 epoch 116 2026-02-11T23:31:27.763+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:beac82f5:::obj51:head, version: 111'51, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:beac82f5:::obj51:head@111'51, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.763+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.763+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 954 ==== pg_backfill(progress 1.0 e 116/116 lb 1:be604f9b:::obj512:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176000 con 0x4acb000 2026-02-11T23:31:27.763+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:beac82f5:::obj51:head, version: 111'51, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:beac82f5:::obj51:head@111'51, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.763+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:be604f9b:::obj512:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.763+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:be604f9b:::obj512:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.763+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.763+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:beac82f5:::obj51:head, version: 111'51, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:beac82f5:::obj51:head@111'51, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.763+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.763+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:beac82f5:::obj51:head, version: 111'51, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:beac82f5:::obj51:head@111'51, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000041 MOSDPGPush(1.0 116/113 [PushOp(1:beac82f5:::obj51:head, version: 111'51, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:beac82f5:::obj51:head@111'51, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:be1f0c3b:::obj423:head local-lis/les=113/114 n=458 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.763+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.763+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:be1f0c3b:::obj423:head local-lis/les=113/114 n=458 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:beac82f5:::obj51:head, version: 111'51, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:beac82f5:::obj51:head@111'51, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.763+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:be1f0c3b:::obj423:head local-lis/les=113/114 n=458 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:beac82f5:::obj51:head@111'51, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.763+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.763+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:be604f9b:::obj512:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.763+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:be1f0c3b:::obj423:head local-lis/les=113/114 n=458 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:beac82f5:::obj51:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.763+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:be1f0c3b:::obj423:head local-lis/les=113/114 n=458 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:beac82f5:::obj51:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.763+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:be1f0c3b:::obj423:head local-lis/les=113/114 n=458 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:beac82f5:::obj51:head 2026-02-11T23:31:27.763+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:be1f0c3b:::obj423:head local-lis/les=113/114 n=458 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:beac82f5:::obj51:head v 111'51 2026-02-11T23:31:27.763+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:be1f0c3b:::obj423:head local-lis/les=113/114 n=458 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.763+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.763+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:beac82f5:::obj51:head, version: 111'51, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:beac82f5:::obj51:head@111'51, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:27.763+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:be1f0c3b:::obj423:head local-lis/les=113/114 n=458 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.763+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.763+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.763+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:be604f9b:::obj512:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.763+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:be604f9b:::obj512:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 1.0 e 116/116 lb 1:be604f9b:::obj512:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:be1f0c3b:::obj423:head local-lis/les=113/114 n=458 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.763+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.763+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:be1f0c3b:::obj423:head local-lis/les=113/114 n=458 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:be604f9b:::obj512:head) v3 2026-02-11T23:31:27.763+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:be1f0c3b:::obj423:head local-lis/les=113/114 n=458 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:be604f9b:::obj512:head) 2026-02-11T23:31:27.763+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.763+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:be604f9b:::obj512:head) v3 finish latency 0.000011 2026-02-11T23:31:27.763+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.763+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.763+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:be604f9b:::obj512:head local-lis/les=113/114 n=459 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.763+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.763+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.763+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.763+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6171800 con 0x4acb000 2026-02-11T23:31:27.763+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:beac82f5:::obj51:head)]) -- 0x600bc80 con 0x4acb000 2026-02-11T23:31:27.763+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.763+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 955 ==== MOSDPGPush(1.0 116/113 [PushOp(1:bec6553a:::obj65:head, version: 111'65, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bec6553a:::obj65:head@111'65, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x559da00 con 0x4acb000 2026-02-11T23:31:27.763+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:bec6553a:::obj65:head, version: 111'65, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bec6553a:::obj65:head@111'65, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 116 2026-02-11T23:31:27.763+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:bec6553a:::obj65:head, version: 111'65, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bec6553a:::obj65:head@111'65, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.763+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.763+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 956 ==== pg_backfill(progress 1.0 e 116/116 lb 1:beac82f5:::obj51:head) ==== 1061+0+0 (unknown 0 0 0) 0x6176b00 con 0x4acb000 2026-02-11T23:31:27.763+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:beac82f5:::obj51:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.763+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:bec6553a:::obj65:head, version: 111'65, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bec6553a:::obj65:head@111'65, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.763+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:beac82f5:::obj51:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.763+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.763+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:bec6553a:::obj65:head, version: 111'65, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bec6553a:::obj65:head@111'65, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.763+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:bec6553a:::obj65:head, version: 111'65, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bec6553a:::obj65:head@111'65, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 116/113 [PushOp(1:bec6553a:::obj65:head, version: 111'65, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bec6553a:::obj65:head@111'65, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:be604f9b:::obj512:head local-lis/les=113/114 n=459 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.763+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.763+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:be604f9b:::obj512:head local-lis/les=113/114 n=459 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:bec6553a:::obj65:head, version: 111'65, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bec6553a:::obj65:head@111'65, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.763+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.763+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:beac82f5:::obj51:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.763+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:be604f9b:::obj512:head local-lis/les=113/114 n=459 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:bec6553a:::obj65:head@111'65, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.763+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:be604f9b:::obj512:head local-lis/les=113/114 n=459 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:bec6553a:::obj65:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.763+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:be604f9b:::obj512:head local-lis/les=113/114 n=459 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:bec6553a:::obj65:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.763+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:be604f9b:::obj512:head local-lis/les=113/114 n=459 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:bec6553a:::obj65:head 2026-02-11T23:31:27.763+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:be604f9b:::obj512:head local-lis/les=113/114 n=459 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:bec6553a:::obj65:head v 111'65 2026-02-11T23:31:27.763+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:be604f9b:::obj512:head local-lis/les=113/114 n=459 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.763+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.763+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:bec6553a:::obj65:head, version: 111'65, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bec6553a:::obj65:head@111'65, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:27.763+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.763+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.763+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:beac82f5:::obj51:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.763+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:beac82f5:::obj51:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 116/116 lb 1:beac82f5:::obj51:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:be604f9b:::obj512:head local-lis/les=113/114 n=459 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.763+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.763+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:be604f9b:::obj512:head local-lis/les=113/114 n=459 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:beac82f5:::obj51:head) v3 2026-02-11T23:31:27.763+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:be604f9b:::obj512:head local-lis/les=113/114 n=459 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:beac82f5:::obj51:head) 2026-02-11T23:31:27.763+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.763+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:beac82f5:::obj51:head) v3 finish latency 0.000011 2026-02-11T23:31:27.763+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:beac82f5:::obj51:head local-lis/les=113/114 n=460 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.763+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.763+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.763+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:beac82f5:::obj51:head local-lis/les=113/114 n=460 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.763+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.763+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.763+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.763+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x600b800 con 0x4acb000 2026-02-11T23:31:27.763+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:bec6553a:::obj65:head)]) -- 0x5dabe00 con 0x4acb000 2026-02-11T23:31:27.763+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.764+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 957 ==== MOSDPGPush(1.0 116/113 [PushOp(1:bf41e297:::obj573:head, version: 111'573, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bf41e297:::obj573:head@111'573, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559cea0 con 0x4acb000 2026-02-11T23:31:27.764+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:bf41e297:::obj573:head, version: 111'573, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bf41e297:::obj573:head@111'573, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.764+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:bf41e297:::obj573:head, version: 111'573, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bf41e297:::obj573:head@111'573, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.764+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.764+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 958 ==== pg_backfill(progress 1.0 e 116/116 lb 1:bec6553a:::obj65:head) ==== 1061+0+0 (unknown 0 0 0) 0x6177080 con 0x4acb000 2026-02-11T23:31:27.764+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:bec6553a:::obj65:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.764+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:bf41e297:::obj573:head, version: 111'573, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bf41e297:::obj573:head@111'573, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.764+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:bec6553a:::obj65:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.764+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.764+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:bec6553a:::obj65:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.764+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.764+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:bf41e297:::obj573:head, version: 111'573, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bf41e297:::obj573:head@111'573, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.764+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:bf41e297:::obj573:head, version: 111'573, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bf41e297:::obj573:head@111'573, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000040 MOSDPGPush(1.0 116/113 [PushOp(1:bf41e297:::obj573:head, version: 111'573, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bf41e297:::obj573:head@111'573, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:beac82f5:::obj51:head local-lis/les=113/114 n=460 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.764+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.764+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:beac82f5:::obj51:head local-lis/les=113/114 n=460 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:bf41e297:::obj573:head, version: 111'573, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bf41e297:::obj573:head@111'573, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.764+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:beac82f5:::obj51:head local-lis/les=113/114 n=460 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:bf41e297:::obj573:head@111'573, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.764+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:beac82f5:::obj51:head local-lis/les=113/114 n=460 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:bf41e297:::obj573:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.764+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:beac82f5:::obj51:head local-lis/les=113/114 n=460 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:bf41e297:::obj573:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.764+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:beac82f5:::obj51:head local-lis/les=113/114 n=460 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:bf41e297:::obj573:head 2026-02-11T23:31:27.764+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:beac82f5:::obj51:head local-lis/les=113/114 n=460 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:bf41e297:::obj573:head v 111'573 2026-02-11T23:31:27.764+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:beac82f5:::obj51:head local-lis/les=113/114 n=460 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.764+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.764+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:bf41e297:::obj573:head, version: 111'573, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bf41e297:::obj573:head@111'573, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.764+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.764+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.764+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:bec6553a:::obj65:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.764+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:bec6553a:::obj65:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 1.0 e 116/116 lb 1:bec6553a:::obj65:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:beac82f5:::obj51:head local-lis/les=113/114 n=460 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.764+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.764+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:beac82f5:::obj51:head local-lis/les=113/114 n=460 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:bec6553a:::obj65:head) v3 2026-02-11T23:31:27.764+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:beac82f5:::obj51:head local-lis/les=113/114 n=460 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:bec6553a:::obj65:head) 2026-02-11T23:31:27.764+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.764+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:bec6553a:::obj65:head) v3 finish latency 0.000010 2026-02-11T23:31:27.764+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bec6553a:::obj65:head local-lis/les=113/114 n=461 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.764+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.764+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.764+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bec6553a:::obj65:head local-lis/les=113/114 n=461 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.764+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.764+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.764+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.764+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x612e000 con 0x4acb000 2026-02-11T23:31:27.764+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:bf41e297:::obj573:head)]) -- 0x5e0ac00 con 0x4acb000 2026-02-11T23:31:27.764+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.764+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 959 ==== MOSDPGPush(1.0 116/113 [PushOp(1:bf770068:::obj268:head, version: 111'268, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bf770068:::obj268:head@111'268, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d8820 con 0x4acb000 2026-02-11T23:31:27.764+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:bf770068:::obj268:head, version: 111'268, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bf770068:::obj268:head@111'268, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 116 2026-02-11T23:31:27.764+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:bf770068:::obj268:head, version: 111'268, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bf770068:::obj268:head@111'268, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.764+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.764+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 960 ==== pg_backfill(progress 1.0 e 116/116 lb 1:bf41e297:::obj573:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177600 con 0x4acb000 2026-02-11T23:31:27.764+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:bf770068:::obj268:head, version: 111'268, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bf770068:::obj268:head@111'268, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.764+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:bf41e297:::obj573:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.764+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:bf41e297:::obj573:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.764+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.764+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:bf770068:::obj268:head, version: 111'268, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bf770068:::obj268:head@111'268, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.764+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:bf770068:::obj268:head, version: 111'268, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bf770068:::obj268:head@111'268, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(1.0 116/113 [PushOp(1:bf770068:::obj268:head, version: 111'268, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bf770068:::obj268:head@111'268, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:bec6553a:::obj65:head local-lis/les=113/114 n=461 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.764+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.764+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.764+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:bf41e297:::obj573:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.764+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bec6553a:::obj65:head local-lis/les=113/114 n=461 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:bf770068:::obj268:head, version: 111'268, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bf770068:::obj268:head@111'268, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.764+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bec6553a:::obj65:head local-lis/les=113/114 n=461 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:bf770068:::obj268:head@111'268, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.764+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bec6553a:::obj65:head local-lis/les=113/114 n=461 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:bf770068:::obj268:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.764+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bec6553a:::obj65:head local-lis/les=113/114 n=461 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:bf770068:::obj268:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.764+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bec6553a:::obj65:head local-lis/les=113/114 n=461 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:bf770068:::obj268:head 2026-02-11T23:31:27.764+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bec6553a:::obj65:head local-lis/les=113/114 n=461 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:bf770068:::obj268:head v 111'268 2026-02-11T23:31:27.764+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bec6553a:::obj65:head local-lis/les=113/114 n=461 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.764+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.764+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:bf770068:::obj268:head, version: 111'268, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bf770068:::obj268:head@111'268, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.764+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.764+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.764+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:bf41e297:::obj573:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.764+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:bf41e297:::obj573:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 1.0 e 116/116 lb 1:bf41e297:::obj573:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:bec6553a:::obj65:head local-lis/les=113/114 n=461 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.764+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.764+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bec6553a:::obj65:head local-lis/les=113/114 n=461 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:bf41e297:::obj573:head) v3 2026-02-11T23:31:27.764+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bec6553a:::obj65:head local-lis/les=113/114 n=461 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:bf41e297:::obj573:head) 2026-02-11T23:31:27.764+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.764+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:bf41e297:::obj573:head) v3 finish latency 0.000010 2026-02-11T23:31:27.764+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bf41e297:::obj573:head local-lis/les=113/114 n=462 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.764+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.765+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.765+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bf41e297:::obj573:head local-lis/les=113/114 n=462 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.765+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.765+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.765+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.765+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fcfe00 con 0x4acb000 2026-02-11T23:31:27.765+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:bf770068:::obj268:head)]) -- 0x5e0aa80 con 0x4acb000 2026-02-11T23:31:27.765+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.765+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 961 ==== MOSDPGPush(1.0 116/113 [PushOp(1:c0848ef4:::obj545:head, version: 111'545, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c0848ef4:::obj545:head@111'545, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56ca000 con 0x4acb000 2026-02-11T23:31:27.765+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:c0848ef4:::obj545:head, version: 111'545, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c0848ef4:::obj545:head@111'545, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 116 2026-02-11T23:31:27.765+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c0848ef4:::obj545:head, version: 111'545, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c0848ef4:::obj545:head@111'545, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.765+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.765+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 962 ==== pg_backfill(progress 1.0 e 116/116 lb 1:bf770068:::obj268:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177b80 con 0x4acb000 2026-02-11T23:31:27.765+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:bf770068:::obj268:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.765+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c0848ef4:::obj545:head, version: 111'545, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c0848ef4:::obj545:head@111'545, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.765+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:bf770068:::obj268:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.765+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.765+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:bf770068:::obj268:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.765+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.765+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c0848ef4:::obj545:head, version: 111'545, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c0848ef4:::obj545:head@111'545, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.765+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:c0848ef4:::obj545:head, version: 111'545, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c0848ef4:::obj545:head@111'545, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000043 MOSDPGPush(1.0 116/113 [PushOp(1:c0848ef4:::obj545:head, version: 111'545, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c0848ef4:::obj545:head@111'545, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:bf41e297:::obj573:head local-lis/les=113/114 n=462 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.765+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.765+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bf41e297:::obj573:head local-lis/les=113/114 n=462 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:c0848ef4:::obj545:head, version: 111'545, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c0848ef4:::obj545:head@111'545, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.765+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bf41e297:::obj573:head local-lis/les=113/114 n=462 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:c0848ef4:::obj545:head@111'545, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.765+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bf41e297:::obj573:head local-lis/les=113/114 n=462 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:c0848ef4:::obj545:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.765+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bf41e297:::obj573:head local-lis/les=113/114 n=462 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:c0848ef4:::obj545:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.765+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bf41e297:::obj573:head local-lis/les=113/114 n=462 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:c0848ef4:::obj545:head 2026-02-11T23:31:27.765+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bf41e297:::obj573:head local-lis/les=113/114 n=462 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:c0848ef4:::obj545:head v 111'545 2026-02-11T23:31:27.765+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bf41e297:::obj573:head local-lis/les=113/114 n=462 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.765+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.765+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:c0848ef4:::obj545:head, version: 111'545, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c0848ef4:::obj545:head@111'545, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:27.765+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bf41e297:::obj573:head local-lis/les=113/114 n=462 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.765+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.765+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.765+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:bf770068:::obj268:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.765+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:bf770068:::obj268:head) v3 prio 15 cost 0 latency 0.000082 pg_backfill(progress 1.0 e 116/116 lb 1:bf770068:::obj268:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:bf41e297:::obj573:head local-lis/les=113/114 n=462 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.765+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.765+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bf41e297:::obj573:head local-lis/les=113/114 n=462 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:bf770068:::obj268:head) v3 2026-02-11T23:31:27.765+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bf41e297:::obj573:head local-lis/les=113/114 n=462 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:bf770068:::obj268:head) 2026-02-11T23:31:27.765+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.765+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:bf770068:::obj268:head) v3 finish latency 0.000010 2026-02-11T23:31:27.765+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.765+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.765+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bf770068:::obj268:head local-lis/les=113/114 n=463 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.765+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.765+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.765+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.765+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5ca0a80 con 0x4acb000 2026-02-11T23:31:27.765+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:c0848ef4:::obj545:head)]) -- 0x612e900 con 0x4acb000 2026-02-11T23:31:27.765+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.765+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 963 ==== MOSDPGPush(1.0 116/113 [PushOp(1:c0c49def:::obj22:head, version: 111'22, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c0c49def:::obj22:head@111'22, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x55d96c0 con 0x4acb000 2026-02-11T23:31:27.765+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:c0c49def:::obj22:head, version: 111'22, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c0c49def:::obj22:head@111'22, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.765+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c0c49def:::obj22:head, version: 111'22, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c0c49def:::obj22:head@111'22, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.765+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.765+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 964 ==== pg_backfill(progress 1.0 e 116/116 lb 1:c0848ef4:::obj545:head) ==== 1062+0+0 (unknown 0 0 0) 0x6178100 con 0x4acb000 2026-02-11T23:31:27.765+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c0c49def:::obj22:head, version: 111'22, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c0c49def:::obj22:head@111'22, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.765+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:c0848ef4:::obj545:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.765+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c0848ef4:::obj545:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.765+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.765+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c0c49def:::obj22:head, version: 111'22, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c0c49def:::obj22:head@111'22, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.765+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:c0c49def:::obj22:head, version: 111'22, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c0c49def:::obj22:head@111'22, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:c0c49def:::obj22:head, version: 111'22, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c0c49def:::obj22:head@111'22, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:bf770068:::obj268:head local-lis/les=113/114 n=463 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.765+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.765+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bf770068:::obj268:head local-lis/les=113/114 n=463 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:c0c49def:::obj22:head, version: 111'22, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c0c49def:::obj22:head@111'22, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.765+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.765+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bf770068:::obj268:head local-lis/les=113/114 n=463 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:c0c49def:::obj22:head@111'22, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.765+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c0848ef4:::obj545:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.765+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bf770068:::obj268:head local-lis/les=113/114 n=463 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:c0c49def:::obj22:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.765+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bf770068:::obj268:head local-lis/les=113/114 n=463 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:c0c49def:::obj22:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.765+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bf770068:::obj268:head local-lis/les=113/114 n=463 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:c0c49def:::obj22:head 2026-02-11T23:31:27.765+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bf770068:::obj268:head local-lis/les=113/114 n=463 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:c0c49def:::obj22:head v 111'22 2026-02-11T23:31:27.765+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bf770068:::obj268:head local-lis/les=113/114 n=463 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.765+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.765+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.4 127.0.0.1:0/3331151400 18 ==== osd_ping(ping e114 up_from 23 ping_stamp 2026-02-11T23:31:27.767230+0000/60.605995178s send_stamp 60.605995178s delta_ub -9.836483955s) ==== 2033+0+0 (crc 0 0 0) 0x4ecca80 con 0x507a800 2026-02-11T23:31:27.765+0000 7f416c3fd640 20 osd.0 116 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836697578s,-9.836483955s]) 2026-02-11T23:31:27.765+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.4 127.0.0.1:0/3331151400 18 ==== osd_ping(ping e114 up_from 23 ping_stamp 2026-02-11T23:31:27.767230+0000/60.605995178s send_stamp 60.605995178s delta_ub -9.836483955s) ==== 2033+0+0 (crc 0 0 0) 0x612e900 con 0x507a400 2026-02-11T23:31:27.765+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e116 up_from 5 ping_stamp 2026-02-11T23:31:27.767230+0000/60.605995178s send_stamp 70.442695618s delta_ub 9.836697578s) -- 0x4ecc780 con 0x507a800 2026-02-11T23:31:27.765+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:c0c49def:::obj22:head, version: 111'22, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c0c49def:::obj22:head@111'22, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.765+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.765+0000 7f416c3fd640 20 osd.0 116 get_con_osd_cluster to osd.4 from_epoch 116 2026-02-11T23:31:27.765+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.765+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c0848ef4:::obj545:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.766+0000 7f416c3fd640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.766+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:c0848ef4:::obj545:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:c0848ef4:::obj545:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:bf770068:::obj268:head local-lis/les=113/114 n=463 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.766+0000 7f416c3fd640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.766+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.766+0000 7f416c3fd640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6836/3331151400 updating session's projected_epoch from 113 to ping map epoch of 114 2026-02-11T23:31:27.766+0000 7f416c3fd640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6836/3331151400 map epoch 114 -> 116 (shared) 2026-02-11T23:31:27.766+0000 7f416c3fd640 10 osd.0 116 send_incremental_map epoch range: (115, 116] to v2:127.0.0.1:6836/3331151400 2026-02-11T23:31:27.766+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bf770068:::obj268:head local-lis/les=113/114 n=463 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:c0848ef4:::obj545:head) v3 2026-02-11T23:31:27.766+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:bf770068:::obj268:head local-lis/les=113/114 n=463 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:c0848ef4:::obj545:head) 2026-02-11T23:31:27.766+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.766+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:c0848ef4:::obj545:head) v3 finish latency 0.000028 2026-02-11T23:31:27.766+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c0848ef4:::obj545:head local-lis/les=113/114 n=464 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.766+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.766+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.766+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c0848ef4:::obj545:head local-lis/les=113/114 n=464 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.766+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.766+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.766+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.766+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6000180 con 0x4acb000 2026-02-11T23:31:27.766+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:c0c49def:::obj22:head)]) -- 0x5e0a900 con 0x4acb000 2026-02-11T23:31:27.766+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.766+0000 7f416c3fd640 10 osd.0 116 add_map_inc_bl 116 574 bytes 2026-02-11T23:31:27.766+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- osd_map(115..116 src has 1..116) -- 0x4a31880 con 0x507ac00 2026-02-11T23:31:27.766+0000 7f416abfa640 20 osd.0 116 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836852074s,-9.836483955s]) 2026-02-11T23:31:27.766+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e116 up_from 5 ping_stamp 2026-02-11T23:31:27.767230+0000/60.605995178s send_stamp 70.442855835s delta_ub 9.836852074s) -- 0x5ca0a80 con 0x507a400 2026-02-11T23:31:27.766+0000 7f416abfa640 20 osd.0 116 get_con_osd_cluster to osd.4 from_epoch 116 2026-02-11T23:31:27.766+0000 7f416abfa640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.766+0000 7f416abfa640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.766+0000 7f416abfa640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.766+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 965 ==== MOSDPGPush(1.0 116/113 [PushOp(1:c138e0af:::obj576:head, version: 111'576, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c138e0af:::obj576:head@111'576, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d91e0 con 0x4acb000 2026-02-11T23:31:27.766+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:c138e0af:::obj576:head, version: 111'576, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c138e0af:::obj576:head@111'576, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.766+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c138e0af:::obj576:head, version: 111'576, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c138e0af:::obj576:head@111'576, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.766+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.766+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 966 ==== pg_backfill(progress 1.0 e 116/116 lb 1:c0c49def:::obj22:head) ==== 1061+0+0 (unknown 0 0 0) 0x6178680 con 0x4acb000 2026-02-11T23:31:27.766+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:c0c49def:::obj22:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.766+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c138e0af:::obj576:head, version: 111'576, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c138e0af:::obj576:head@111'576, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.766+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c0c49def:::obj22:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.766+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.766+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c0c49def:::obj22:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.766+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.766+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c138e0af:::obj576:head, version: 111'576, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c138e0af:::obj576:head@111'576, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.766+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:c138e0af:::obj576:head, version: 111'576, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c138e0af:::obj576:head@111'576, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000055 MOSDPGPush(1.0 116/113 [PushOp(1:c138e0af:::obj576:head, version: 111'576, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c138e0af:::obj576:head@111'576, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:c0848ef4:::obj545:head local-lis/les=113/114 n=464 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.766+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.766+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c0848ef4:::obj545:head local-lis/les=113/114 n=464 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:c138e0af:::obj576:head, version: 111'576, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c138e0af:::obj576:head@111'576, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.766+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c0848ef4:::obj545:head local-lis/les=113/114 n=464 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:c138e0af:::obj576:head@111'576, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.766+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c0848ef4:::obj545:head local-lis/les=113/114 n=464 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:c138e0af:::obj576:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.766+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c0848ef4:::obj545:head local-lis/les=113/114 n=464 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:c138e0af:::obj576:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.766+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c0848ef4:::obj545:head local-lis/les=113/114 n=464 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:c138e0af:::obj576:head 2026-02-11T23:31:27.766+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c0848ef4:::obj545:head local-lis/les=113/114 n=464 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:c138e0af:::obj576:head v 111'576 2026-02-11T23:31:27.766+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c0848ef4:::obj545:head local-lis/les=113/114 n=464 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.766+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.766+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:c138e0af:::obj576:head, version: 111'576, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c138e0af:::obj576:head@111'576, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000103 2026-02-11T23:31:27.766+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.766+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.766+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c0c49def:::obj22:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.766+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:c0c49def:::obj22:head) v3 prio 15 cost 0 latency 0.000149 pg_backfill(progress 1.0 e 116/116 lb 1:c0c49def:::obj22:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:c0848ef4:::obj545:head local-lis/les=113/114 n=464 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.766+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.766+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c0848ef4:::obj545:head local-lis/les=113/114 n=464 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:c0c49def:::obj22:head) v3 2026-02-11T23:31:27.766+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c0848ef4:::obj545:head local-lis/les=113/114 n=464 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:c0c49def:::obj22:head) 2026-02-11T23:31:27.766+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.766+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:c0c49def:::obj22:head) v3 finish latency 0.000018 2026-02-11T23:31:27.766+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c0c49def:::obj22:head local-lis/les=113/114 n=465 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.766+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.766+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.766+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c0c49def:::obj22:head local-lis/les=113/114 n=465 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.766+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.766+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.766+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.766+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6015800 con 0x4acb000 2026-02-11T23:31:27.766+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:c138e0af:::obj576:head)]) -- 0x5dab800 con 0x4acb000 2026-02-11T23:31:27.766+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.767+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 31 ==== pg_query2(4.0 4.0 query(info 0'0 epoch_sent 116) e116/116) ==== 158+0+0 (unknown 0 0 0) 0x5d66000 con 0x507ac00 2026-02-11T23:31:27.767+0000 7f416abfa640 15 osd.0 116 enqueue_peering_evt 4.0 epoch_sent: 116 epoch_requested: 116 MQuery 4.0 from 4 query_epoch 116 query: query(info 0'0 epoch_sent 116) 2026-02-11T23:31:27.767+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 116 epoch_requested: 116 MQuery 4.0 from 4 query_epoch 116 query: query(info 0'0 epoch_sent 116)) class_id 2 prio 255 cost 10 e116) 2026-02-11T23:31:27.767+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 (new) to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.767+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 116 epoch_requested: 116 MQuery 4.0 from 4 query_epoch 116 query: query(info 0'0 epoch_sent 116)) class_id 2immediate prio 255 cost 10 e116) queued 2026-02-11T23:31:27.767+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.767+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 116 epoch_requested: 116 MQuery 4.0 from 4 query_epoch 116 query: query(info 0'0 epoch_sent 116)) class_id 2immediate prio 255 cost 10 e116) pg 0 2026-02-11T23:31:27.767+0000 7f414b98b640 10 osd.0 116 handle_pg_query_nopg 4.0 2026-02-11T23:31:27.767+0000 7f414b98b640 10 osd.0 116 pg 4.0 dne 2026-02-11T23:31:27.767+0000 7f414b98b640 20 osd.0 116 get_con_osd_cluster to osd.4 from_epoch 116 2026-02-11T23:31:27.767+0000 7f414b98b640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.767+0000 7f414b98b640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.767+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.767+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_notify2(4.0 4.0 (query:116 sent:116 4.0( DNE empty local-lis/les=0/0 n=0 ec=0/0 lis/c=0/0 les/c/f=0/0/0 sis=0) ([0,0] all_participants={} intervals=[])) e116/116) -- 0x5ef9800 con 0x507ac00 2026-02-11T23:31:27.767+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.767+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.767+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 967 ==== MOSDPGPush(1.0 116/113 [PushOp(1:c19e5add:::obj152:head, version: 111'152, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c19e5add:::obj152:head@111'152, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d8d00 con 0x4acb000 2026-02-11T23:31:27.767+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:c19e5add:::obj152:head, version: 111'152, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c19e5add:::obj152:head@111'152, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 116 2026-02-11T23:31:27.767+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c19e5add:::obj152:head, version: 111'152, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c19e5add:::obj152:head@111'152, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.767+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.767+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 968 ==== pg_backfill(progress 1.0 e 116/116 lb 1:c138e0af:::obj576:head) ==== 1062+0+0 (unknown 0 0 0) 0x6178c00 con 0x4acb000 2026-02-11T23:31:27.767+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c19e5add:::obj152:head, version: 111'152, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c19e5add:::obj152:head@111'152, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.767+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:c138e0af:::obj576:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.767+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.767+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c138e0af:::obj576:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.767+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c19e5add:::obj152:head, version: 111'152, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c19e5add:::obj152:head@111'152, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.767+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:c19e5add:::obj152:head, version: 111'152, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c19e5add:::obj152:head@111'152, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(1.0 116/113 [PushOp(1:c19e5add:::obj152:head, version: 111'152, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c19e5add:::obj152:head@111'152, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:c0c49def:::obj22:head local-lis/les=113/114 n=465 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.767+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.767+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.767+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c0c49def:::obj22:head local-lis/les=113/114 n=465 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:c19e5add:::obj152:head, version: 111'152, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c19e5add:::obj152:head@111'152, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.767+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c0c49def:::obj22:head local-lis/les=113/114 n=465 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:c19e5add:::obj152:head@111'152, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.767+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.767+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c138e0af:::obj576:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.767+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c0c49def:::obj22:head local-lis/les=113/114 n=465 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:c19e5add:::obj152:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.767+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c0c49def:::obj22:head local-lis/les=113/114 n=465 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:c19e5add:::obj152:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.767+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c0c49def:::obj22:head local-lis/les=113/114 n=465 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:c19e5add:::obj152:head 2026-02-11T23:31:27.767+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c0c49def:::obj22:head local-lis/les=113/114 n=465 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:c19e5add:::obj152:head v 111'152 2026-02-11T23:31:27.767+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c0c49def:::obj22:head local-lis/les=113/114 n=465 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.767+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.767+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:c19e5add:::obj152:head, version: 111'152, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c19e5add:::obj152:head@111'152, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000064 2026-02-11T23:31:27.767+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c0c49def:::obj22:head local-lis/les=113/114 n=465 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.767+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.767+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.767+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c138e0af:::obj576:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.767+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:c138e0af:::obj576:head) v3 prio 15 cost 0 latency 0.000088 pg_backfill(progress 1.0 e 116/116 lb 1:c138e0af:::obj576:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:c0c49def:::obj22:head local-lis/les=113/114 n=465 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.767+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.767+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c0c49def:::obj22:head local-lis/les=113/114 n=465 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:c138e0af:::obj576:head) v3 2026-02-11T23:31:27.767+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c0c49def:::obj22:head local-lis/les=113/114 n=465 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:c138e0af:::obj576:head) 2026-02-11T23:31:27.767+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.767+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:c138e0af:::obj576:head) v3 finish latency 0.000025 2026-02-11T23:31:27.767+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.767+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.767+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c138e0af:::obj576:head local-lis/les=113/114 n=466 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.767+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.767+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.767+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.767+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x600b500 con 0x4acb000 2026-02-11T23:31:27.767+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:c19e5add:::obj152:head)]) -- 0x600b380 con 0x4acb000 2026-02-11T23:31:27.767+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.767+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 969 ==== MOSDPGPush(1.0 116/113 [PushOp(1:c275c16a:::obj32:head, version: 111'32, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c275c16a:::obj32:head@111'32, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x56471e0 con 0x4acb000 2026-02-11T23:31:27.767+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:c275c16a:::obj32:head, version: 111'32, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c275c16a:::obj32:head@111'32, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.767+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c275c16a:::obj32:head, version: 111'32, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c275c16a:::obj32:head@111'32, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.767+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.767+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 970 ==== pg_backfill(progress 1.0 e 116/116 lb 1:c19e5add:::obj152:head) ==== 1062+0+0 (unknown 0 0 0) 0x6179180 con 0x4acb000 2026-02-11T23:31:27.767+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:c19e5add:::obj152:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.767+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c275c16a:::obj32:head, version: 111'32, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c275c16a:::obj32:head@111'32, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.767+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c19e5add:::obj152:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.767+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.767+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c275c16a:::obj32:head, version: 111'32, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c275c16a:::obj32:head@111'32, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.767+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.767+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:c275c16a:::obj32:head, version: 111'32, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c275c16a:::obj32:head@111'32, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(1.0 116/113 [PushOp(1:c275c16a:::obj32:head, version: 111'32, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c275c16a:::obj32:head@111'32, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:c138e0af:::obj576:head local-lis/les=113/114 n=466 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.767+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c19e5add:::obj152:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.767+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.767+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c138e0af:::obj576:head local-lis/les=113/114 n=466 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:c275c16a:::obj32:head, version: 111'32, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c275c16a:::obj32:head@111'32, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.767+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c138e0af:::obj576:head local-lis/les=113/114 n=466 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:c275c16a:::obj32:head@111'32, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.767+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c138e0af:::obj576:head local-lis/les=113/114 n=466 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:c275c16a:::obj32:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.767+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c138e0af:::obj576:head local-lis/les=113/114 n=466 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:c275c16a:::obj32:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.767+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c138e0af:::obj576:head local-lis/les=113/114 n=466 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:c275c16a:::obj32:head 2026-02-11T23:31:27.767+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c138e0af:::obj576:head local-lis/les=113/114 n=466 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:c275c16a:::obj32:head v 111'32 2026-02-11T23:31:27.767+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c138e0af:::obj576:head local-lis/les=113/114 n=466 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.767+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.767+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:c275c16a:::obj32:head, version: 111'32, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c275c16a:::obj32:head@111'32, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000073 2026-02-11T23:31:27.767+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.767+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.767+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c19e5add:::obj152:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.767+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:c19e5add:::obj152:head) v3 prio 15 cost 0 latency 0.000095 pg_backfill(progress 1.0 e 116/116 lb 1:c19e5add:::obj152:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:c138e0af:::obj576:head local-lis/les=113/114 n=466 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.767+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.767+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c138e0af:::obj576:head local-lis/les=113/114 n=466 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:c19e5add:::obj152:head) v3 2026-02-11T23:31:27.767+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c138e0af:::obj576:head local-lis/les=113/114 n=466 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:c19e5add:::obj152:head) 2026-02-11T23:31:27.767+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.767+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:c19e5add:::obj152:head) v3 finish latency 0.000015 2026-02-11T23:31:27.767+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c19e5add:::obj152:head local-lis/les=113/114 n=467 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.767+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.768+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.768+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c19e5add:::obj152:head local-lis/les=113/114 n=467 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.768+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.768+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.768+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.768+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fed380 con 0x4acb000 2026-02-11T23:31:27.768+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:c275c16a:::obj32:head)]) -- 0x5dab500 con 0x4acb000 2026-02-11T23:31:27.768+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.768+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 971 ==== MOSDPGPush(1.0 116/113 [PushOp(1:c290fff4:::obj320:head, version: 111'320, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c290fff4:::obj320:head@111'320, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d8340 con 0x4acb000 2026-02-11T23:31:27.768+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:c290fff4:::obj320:head, version: 111'320, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c290fff4:::obj320:head@111'320, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.768+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c290fff4:::obj320:head, version: 111'320, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c290fff4:::obj320:head@111'320, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.768+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 972 ==== pg_backfill(progress 1.0 e 116/116 lb 1:c275c16a:::obj32:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d2d080 con 0x4acb000 2026-02-11T23:31:27.768+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.768+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:c275c16a:::obj32:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 116 2026-02-11T23:31:27.768+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c275c16a:::obj32:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.768+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c290fff4:::obj320:head, version: 111'320, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c290fff4:::obj320:head@111'320, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.768+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.768+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c290fff4:::obj320:head, version: 111'320, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c290fff4:::obj320:head@111'320, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.768+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:c290fff4:::obj320:head, version: 111'320, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c290fff4:::obj320:head@111'320, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 116/113 [PushOp(1:c290fff4:::obj320:head, version: 111'320, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c290fff4:::obj320:head@111'320, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:c19e5add:::obj152:head local-lis/les=113/114 n=467 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.768+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.768+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.768+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c275c16a:::obj32:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.768+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c19e5add:::obj152:head local-lis/les=113/114 n=467 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:c290fff4:::obj320:head, version: 111'320, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c290fff4:::obj320:head@111'320, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.768+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c19e5add:::obj152:head local-lis/les=113/114 n=467 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:c290fff4:::obj320:head@111'320, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.768+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c19e5add:::obj152:head local-lis/les=113/114 n=467 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:c290fff4:::obj320:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.768+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c19e5add:::obj152:head local-lis/les=113/114 n=467 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:c290fff4:::obj320:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.768+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c19e5add:::obj152:head local-lis/les=113/114 n=467 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:c290fff4:::obj320:head 2026-02-11T23:31:27.768+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c19e5add:::obj152:head local-lis/les=113/114 n=467 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:c290fff4:::obj320:head v 111'320 2026-02-11T23:31:27.768+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c19e5add:::obj152:head local-lis/les=113/114 n=467 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.768+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.768+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:c290fff4:::obj320:head, version: 111'320, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c290fff4:::obj320:head@111'320, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:27.768+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.768+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.768+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c275c16a:::obj32:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.768+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:c275c16a:::obj32:head) v3 prio 15 cost 0 latency 0.000088 pg_backfill(progress 1.0 e 116/116 lb 1:c275c16a:::obj32:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:c19e5add:::obj152:head local-lis/les=113/114 n=467 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.768+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.768+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c19e5add:::obj152:head local-lis/les=113/114 n=467 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:c275c16a:::obj32:head) v3 2026-02-11T23:31:27.768+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c19e5add:::obj152:head local-lis/les=113/114 n=467 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:c275c16a:::obj32:head) 2026-02-11T23:31:27.768+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.768+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:c275c16a:::obj32:head) v3 finish latency 0.000014 2026-02-11T23:31:27.768+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c275c16a:::obj32:head local-lis/les=113/114 n=468 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.768+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.768+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.768+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c275c16a:::obj32:head local-lis/les=113/114 n=468 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.768+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.768+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.768+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.768+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x600b680 con 0x4acb000 2026-02-11T23:31:27.768+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:c290fff4:::obj320:head)]) -- 0x5e0bc80 con 0x4acb000 2026-02-11T23:31:27.768+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.768+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 973 ==== MOSDPGPush(1.0 116/113 [PushOp(1:c3033278:::obj208:head, version: 111'208, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3033278:::obj208:head@111'208, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d9860 con 0x4acb000 2026-02-11T23:31:27.768+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:c3033278:::obj208:head, version: 111'208, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3033278:::obj208:head@111'208, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.768+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c3033278:::obj208:head, version: 111'208, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3033278:::obj208:head@111'208, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.768+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.768+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c3033278:::obj208:head, version: 111'208, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3033278:::obj208:head@111'208, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.768+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 974 ==== pg_backfill(progress 1.0 e 116/116 lb 1:c290fff4:::obj320:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e680 con 0x4acb000 2026-02-11T23:31:27.769+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.769+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c3033278:::obj208:head, version: 111'208, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3033278:::obj208:head@111'208, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.769+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:c290fff4:::obj320:head) v3 prio 15 type 95 cost 0 latency 0.000010 epoch 116 2026-02-11T23:31:27.769+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.769+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:c3033278:::obj208:head, version: 111'208, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3033278:::obj208:head@111'208, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(1.0 116/113 [PushOp(1:c3033278:::obj208:head, version: 111'208, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3033278:::obj208:head@111'208, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:c275c16a:::obj32:head local-lis/les=113/114 n=468 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.769+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.769+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c290fff4:::obj320:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.769+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c275c16a:::obj32:head local-lis/les=113/114 n=468 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:c3033278:::obj208:head, version: 111'208, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3033278:::obj208:head@111'208, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.769+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c275c16a:::obj32:head local-lis/les=113/114 n=468 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:c3033278:::obj208:head@111'208, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.769+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.769+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c290fff4:::obj320:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.769+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c275c16a:::obj32:head local-lis/les=113/114 n=468 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:c3033278:::obj208:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.769+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c275c16a:::obj32:head local-lis/les=113/114 n=468 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:c3033278:::obj208:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.769+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c275c16a:::obj32:head local-lis/les=113/114 n=468 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:c3033278:::obj208:head 2026-02-11T23:31:27.769+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c275c16a:::obj32:head local-lis/les=113/114 n=468 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:c3033278:::obj208:head v 111'208 2026-02-11T23:31:27.769+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c275c16a:::obj32:head local-lis/les=113/114 n=468 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.769+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.769+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:c3033278:::obj208:head, version: 111'208, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3033278:::obj208:head@111'208, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000064 2026-02-11T23:31:27.769+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.769+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.769+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c290fff4:::obj320:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.769+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:c290fff4:::obj320:head) v3 prio 15 cost 0 latency 0.000089 pg_backfill(progress 1.0 e 116/116 lb 1:c290fff4:::obj320:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:c275c16a:::obj32:head local-lis/les=113/114 n=468 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.769+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.769+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c275c16a:::obj32:head local-lis/les=113/114 n=468 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:c290fff4:::obj320:head) v3 2026-02-11T23:31:27.769+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c275c16a:::obj32:head local-lis/les=113/114 n=468 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:c290fff4:::obj320:head) 2026-02-11T23:31:27.769+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.769+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:c290fff4:::obj320:head) v3 finish latency 0.000013 2026-02-11T23:31:27.769+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c290fff4:::obj320:head local-lis/les=113/114 n=469 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.769+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.769+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.769+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c290fff4:::obj320:head local-lis/les=113/114 n=469 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.769+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.769+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.769+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.769+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6005380 con 0x4acb000 2026-02-11T23:31:27.769+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:c3033278:::obj208:head)]) -- 0x5e0bb00 con 0x4acb000 2026-02-11T23:31:27.769+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.769+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 975 ==== MOSDPGPush(1.0 116/113 [PushOp(1:c3ae426e:::obj537:head, version: 111'537, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3ae426e:::obj537:head@111'537, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d8820 con 0x4acb000 2026-02-11T23:31:27.769+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:c3ae426e:::obj537:head, version: 111'537, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3ae426e:::obj537:head@111'537, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000016 epoch 116 2026-02-11T23:31:27.769+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c3ae426e:::obj537:head, version: 111'537, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3ae426e:::obj537:head@111'537, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.769+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.769+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c3ae426e:::obj537:head, version: 111'537, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3ae426e:::obj537:head@111'537, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.769+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 976 ==== pg_backfill(progress 1.0 e 116/116 lb 1:c3033278:::obj208:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e100 con 0x4acb000 2026-02-11T23:31:27.769+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.769+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c3ae426e:::obj537:head, version: 111'537, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3ae426e:::obj537:head@111'537, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.769+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:c3033278:::obj208:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 116 2026-02-11T23:31:27.769+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c3033278:::obj208:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.769+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:c3ae426e:::obj537:head, version: 111'537, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3ae426e:::obj537:head@111'537, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000046 MOSDPGPush(1.0 116/113 [PushOp(1:c3ae426e:::obj537:head, version: 111'537, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3ae426e:::obj537:head@111'537, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:c290fff4:::obj320:head local-lis/les=113/114 n=469 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.769+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.769+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c290fff4:::obj320:head local-lis/les=113/114 n=469 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:c3ae426e:::obj537:head, version: 111'537, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3ae426e:::obj537:head@111'537, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.769+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.769+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c3033278:::obj208:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.769+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c290fff4:::obj320:head local-lis/les=113/114 n=469 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:c3ae426e:::obj537:head@111'537, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.769+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c290fff4:::obj320:head local-lis/les=113/114 n=469 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:c3ae426e:::obj537:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.769+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c290fff4:::obj320:head local-lis/les=113/114 n=469 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:c3ae426e:::obj537:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.769+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c290fff4:::obj320:head local-lis/les=113/114 n=469 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:c3ae426e:::obj537:head 2026-02-11T23:31:27.769+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c290fff4:::obj320:head local-lis/les=113/114 n=469 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:c3ae426e:::obj537:head v 111'537 2026-02-11T23:31:27.769+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c290fff4:::obj320:head local-lis/les=113/114 n=469 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.769+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.769+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:c3ae426e:::obj537:head, version: 111'537, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3ae426e:::obj537:head@111'537, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:27.769+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c290fff4:::obj320:head local-lis/les=113/114 n=469 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.769+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.769+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c3033278:::obj208:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.769+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.769+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:c3033278:::obj208:head) v3 prio 15 cost 0 latency 0.000079 pg_backfill(progress 1.0 e 116/116 lb 1:c3033278:::obj208:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:c290fff4:::obj320:head local-lis/les=113/114 n=469 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.769+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.769+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c290fff4:::obj320:head local-lis/les=113/114 n=469 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:c3033278:::obj208:head) v3 2026-02-11T23:31:27.769+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c290fff4:::obj320:head local-lis/les=113/114 n=469 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:c3033278:::obj208:head) 2026-02-11T23:31:27.769+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.769+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:c3033278:::obj208:head) v3 finish latency 0.000020 2026-02-11T23:31:27.769+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.769+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.769+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3033278:::obj208:head local-lis/les=113/114 n=470 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.769+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.769+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.769+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.769+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fec180 con 0x4acb000 2026-02-11T23:31:27.769+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:c3ae426e:::obj537:head)]) -- 0x5fec000 con 0x4acb000 2026-02-11T23:31:27.769+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.770+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 977 ==== MOSDPGPush(1.0 116/113 [PushOp(1:c3ba172d:::obj29:head, version: 111'29, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3ba172d:::obj29:head@111'29, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x559cea0 con 0x4acb000 2026-02-11T23:31:27.770+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:c3ba172d:::obj29:head, version: 111'29, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3ba172d:::obj29:head@111'29, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000016 epoch 116 2026-02-11T23:31:27.770+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c3ba172d:::obj29:head, version: 111'29, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3ba172d:::obj29:head@111'29, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.770+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.770+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 978 ==== pg_backfill(progress 1.0 e 116/116 lb 1:c3ae426e:::obj537:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2c000 con 0x4acb000 2026-02-11T23:31:27.770+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:c3ae426e:::obj537:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.770+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c3ba172d:::obj29:head, version: 111'29, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3ba172d:::obj29:head@111'29, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.770+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c3ae426e:::obj537:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.770+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.770+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c3ba172d:::obj29:head, version: 111'29, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3ba172d:::obj29:head@111'29, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.770+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:c3ba172d:::obj29:head, version: 111'29, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3ba172d:::obj29:head@111'29, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000045 MOSDPGPush(1.0 116/113 [PushOp(1:c3ba172d:::obj29:head, version: 111'29, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3ba172d:::obj29:head@111'29, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:c3033278:::obj208:head local-lis/les=113/114 n=470 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.770+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.770+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.770+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3033278:::obj208:head local-lis/les=113/114 n=470 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:c3ba172d:::obj29:head, version: 111'29, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3ba172d:::obj29:head@111'29, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.770+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c3ae426e:::obj537:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.770+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3033278:::obj208:head local-lis/les=113/114 n=470 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:c3ba172d:::obj29:head@111'29, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.770+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3033278:::obj208:head local-lis/les=113/114 n=470 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:c3ba172d:::obj29:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.770+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3033278:::obj208:head local-lis/les=113/114 n=470 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:c3ba172d:::obj29:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.770+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3033278:::obj208:head local-lis/les=113/114 n=470 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:c3ba172d:::obj29:head 2026-02-11T23:31:27.770+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3033278:::obj208:head local-lis/les=113/114 n=470 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:c3ba172d:::obj29:head v 111'29 2026-02-11T23:31:27.770+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3033278:::obj208:head local-lis/les=113/114 n=470 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.770+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.770+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:c3ba172d:::obj29:head, version: 111'29, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3ba172d:::obj29:head@111'29, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000062 2026-02-11T23:31:27.770+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.770+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.770+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c3ae426e:::obj537:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.770+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:c3ae426e:::obj537:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 1.0 e 116/116 lb 1:c3ae426e:::obj537:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:c3033278:::obj208:head local-lis/les=113/114 n=470 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.770+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.770+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3033278:::obj208:head local-lis/les=113/114 n=470 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:c3ae426e:::obj537:head) v3 2026-02-11T23:31:27.770+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3033278:::obj208:head local-lis/les=113/114 n=470 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:c3ae426e:::obj537:head) 2026-02-11T23:31:27.770+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.770+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:c3ae426e:::obj537:head) v3 finish latency 0.000013 2026-02-11T23:31:27.770+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3ae426e:::obj537:head local-lis/les=113/114 n=471 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.770+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.770+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.770+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3ae426e:::obj537:head local-lis/les=113/114 n=471 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.770+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.770+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.770+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.770+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x600be00 con 0x4acb000 2026-02-11T23:31:27.770+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:c3ba172d:::obj29:head)]) -- 0x5e6b980 con 0x4acb000 2026-02-11T23:31:27.770+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.770+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 979 ==== MOSDPGPush(1.0 116/113 [PushOp(1:c3bae056:::obj148:head, version: 111'148, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3bae056:::obj148:head@111'148, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559da00 con 0x4acb000 2026-02-11T23:31:27.770+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:c3bae056:::obj148:head, version: 111'148, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3bae056:::obj148:head@111'148, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 116 2026-02-11T23:31:27.770+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c3bae056:::obj148:head, version: 111'148, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3bae056:::obj148:head@111'148, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.770+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.770+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 980 ==== pg_backfill(progress 1.0 e 116/116 lb 1:c3ba172d:::obj29:head) ==== 1061+0+0 (unknown 0 0 0) 0x4a2cb00 con 0x4acb000 2026-02-11T23:31:27.770+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c3bae056:::obj148:head, version: 111'148, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3bae056:::obj148:head@111'148, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.770+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:c3ba172d:::obj29:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.770+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c3ba172d:::obj29:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.770+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.770+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c3bae056:::obj148:head, version: 111'148, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3bae056:::obj148:head@111'148, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.770+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:c3bae056:::obj148:head, version: 111'148, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3bae056:::obj148:head@111'148, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000042 MOSDPGPush(1.0 116/113 [PushOp(1:c3bae056:::obj148:head, version: 111'148, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3bae056:::obj148:head@111'148, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:c3ae426e:::obj537:head local-lis/les=113/114 n=471 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.770+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.770+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.770+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3ae426e:::obj537:head local-lis/les=113/114 n=471 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:c3bae056:::obj148:head, version: 111'148, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3bae056:::obj148:head@111'148, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.770+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3ae426e:::obj537:head local-lis/les=113/114 n=471 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:c3bae056:::obj148:head@111'148, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.770+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.770+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3ae426e:::obj537:head local-lis/les=113/114 n=471 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:c3bae056:::obj148:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.770+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c3ba172d:::obj29:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.770+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3ae426e:::obj537:head local-lis/les=113/114 n=471 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:c3bae056:::obj148:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.770+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3ae426e:::obj537:head local-lis/les=113/114 n=471 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:c3bae056:::obj148:head 2026-02-11T23:31:27.770+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3ae426e:::obj537:head local-lis/les=113/114 n=471 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:c3bae056:::obj148:head v 111'148 2026-02-11T23:31:27.770+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3ae426e:::obj537:head local-lis/les=113/114 n=471 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.770+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.770+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:c3bae056:::obj148:head, version: 111'148, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3bae056:::obj148:head@111'148, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:27.770+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3ae426e:::obj537:head local-lis/les=113/114 n=471 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.770+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.770+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c3ba172d:::obj29:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.770+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.770+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:c3ba172d:::obj29:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 1.0 e 116/116 lb 1:c3ba172d:::obj29:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:c3ae426e:::obj537:head local-lis/les=113/114 n=471 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.770+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.770+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3ae426e:::obj537:head local-lis/les=113/114 n=471 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:c3ba172d:::obj29:head) v3 2026-02-11T23:31:27.770+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3ae426e:::obj537:head local-lis/les=113/114 n=471 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:c3ba172d:::obj29:head) 2026-02-11T23:31:27.770+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.770+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:c3ba172d:::obj29:head) v3 finish latency 0.000025 2026-02-11T23:31:27.770+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.770+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.770+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3ba172d:::obj29:head local-lis/les=113/114 n=472 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.770+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.770+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.770+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.770+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fec780 con 0x4acb000 2026-02-11T23:31:27.770+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:c3bae056:::obj148:head)]) -- 0x5fec600 con 0x4acb000 2026-02-11T23:31:27.770+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.771+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 981 ==== MOSDPGPush(1.0 116/113 [PushOp(1:c3bcd33c:::obj192:head, version: 111'192, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3bcd33c:::obj192:head@111'192, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c4e0 con 0x4acb000 2026-02-11T23:31:27.771+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:c3bcd33c:::obj192:head, version: 111'192, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3bcd33c:::obj192:head@111'192, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000015 epoch 116 2026-02-11T23:31:27.771+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c3bcd33c:::obj192:head, version: 111'192, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3bcd33c:::obj192:head@111'192, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.771+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.771+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 982 ==== pg_backfill(progress 1.0 e 116/116 lb 1:c3bae056:::obj148:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9a100 con 0x4acb000 2026-02-11T23:31:27.771+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:c3bae056:::obj148:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.771+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c3bcd33c:::obj192:head, version: 111'192, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3bcd33c:::obj192:head@111'192, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.771+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c3bae056:::obj148:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.771+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.771+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c3bcd33c:::obj192:head, version: 111'192, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3bcd33c:::obj192:head@111'192, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.771+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.771+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c3bae056:::obj148:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.771+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:c3bcd33c:::obj192:head, version: 111'192, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3bcd33c:::obj192:head@111'192, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000042 MOSDPGPush(1.0 116/113 [PushOp(1:c3bcd33c:::obj192:head, version: 111'192, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3bcd33c:::obj192:head@111'192, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:c3ba172d:::obj29:head local-lis/les=113/114 n=472 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.771+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.771+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3ba172d:::obj29:head local-lis/les=113/114 n=472 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:c3bcd33c:::obj192:head, version: 111'192, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3bcd33c:::obj192:head@111'192, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.771+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3ba172d:::obj29:head local-lis/les=113/114 n=472 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:c3bcd33c:::obj192:head@111'192, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.771+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3ba172d:::obj29:head local-lis/les=113/114 n=472 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:c3bcd33c:::obj192:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.771+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3ba172d:::obj29:head local-lis/les=113/114 n=472 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:c3bcd33c:::obj192:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.771+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3ba172d:::obj29:head local-lis/les=113/114 n=472 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:c3bcd33c:::obj192:head 2026-02-11T23:31:27.771+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3ba172d:::obj29:head local-lis/les=113/114 n=472 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:c3bcd33c:::obj192:head v 111'192 2026-02-11T23:31:27.771+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3ba172d:::obj29:head local-lis/les=113/114 n=472 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.771+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.771+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:c3bcd33c:::obj192:head, version: 111'192, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3bcd33c:::obj192:head@111'192, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000070 2026-02-11T23:31:27.771+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.771+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.771+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c3bae056:::obj148:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.771+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:c3bae056:::obj148:head) v3 prio 15 cost 0 latency 0.000091 pg_backfill(progress 1.0 e 116/116 lb 1:c3bae056:::obj148:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:c3ba172d:::obj29:head local-lis/les=113/114 n=472 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.771+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.771+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3ba172d:::obj29:head local-lis/les=113/114 n=472 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:c3bae056:::obj148:head) v3 2026-02-11T23:31:27.771+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3ba172d:::obj29:head local-lis/les=113/114 n=472 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:c3bae056:::obj148:head) 2026-02-11T23:31:27.771+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.771+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:c3bae056:::obj148:head) v3 finish latency 0.000015 2026-02-11T23:31:27.771+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3bae056:::obj148:head local-lis/les=113/114 n=473 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.771+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.771+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3bae056:::obj148:head local-lis/les=113/114 n=473 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.771+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.771+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.771+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.771+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.771+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fcfb00 con 0x4acb000 2026-02-11T23:31:27.771+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:c3bcd33c:::obj192:head)]) -- 0x5daac00 con 0x4acb000 2026-02-11T23:31:27.771+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.771+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 983 ==== MOSDPGPush(1.0 116/113 [PushOp(1:c41d6f6c:::obj298:head, version: 111'298, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c41d6f6c:::obj298:head@111'298, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f1860 con 0x4acb000 2026-02-11T23:31:27.771+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:c41d6f6c:::obj298:head, version: 111'298, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c41d6f6c:::obj298:head@111'298, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 116 2026-02-11T23:31:27.771+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c41d6f6c:::obj298:head, version: 111'298, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c41d6f6c:::obj298:head@111'298, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.771+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.771+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 984 ==== pg_backfill(progress 1.0 e 116/116 lb 1:c3bcd33c:::obj192:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9a680 con 0x4acb000 2026-02-11T23:31:27.771+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:c3bcd33c:::obj192:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.771+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c41d6f6c:::obj298:head, version: 111'298, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c41d6f6c:::obj298:head@111'298, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.771+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c3bcd33c:::obj192:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.771+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.771+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c3bcd33c:::obj192:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.771+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.771+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c41d6f6c:::obj298:head, version: 111'298, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c41d6f6c:::obj298:head@111'298, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.771+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:c41d6f6c:::obj298:head, version: 111'298, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c41d6f6c:::obj298:head@111'298, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000054 MOSDPGPush(1.0 116/113 [PushOp(1:c41d6f6c:::obj298:head, version: 111'298, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c41d6f6c:::obj298:head@111'298, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:c3bae056:::obj148:head local-lis/les=113/114 n=473 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.771+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.771+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3bae056:::obj148:head local-lis/les=113/114 n=473 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:c41d6f6c:::obj298:head, version: 111'298, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c41d6f6c:::obj298:head@111'298, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.771+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3bae056:::obj148:head local-lis/les=113/114 n=473 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:c41d6f6c:::obj298:head@111'298, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.771+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3bae056:::obj148:head local-lis/les=113/114 n=473 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:c41d6f6c:::obj298:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.771+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3bae056:::obj148:head local-lis/les=113/114 n=473 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:c41d6f6c:::obj298:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.771+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3bae056:::obj148:head local-lis/les=113/114 n=473 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:c41d6f6c:::obj298:head 2026-02-11T23:31:27.771+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3bae056:::obj148:head local-lis/les=113/114 n=473 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:c41d6f6c:::obj298:head v 111'298 2026-02-11T23:31:27.771+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3bae056:::obj148:head local-lis/les=113/114 n=473 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.771+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.772+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:c41d6f6c:::obj298:head, version: 111'298, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c41d6f6c:::obj298:head@111'298, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000068 2026-02-11T23:31:27.772+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.772+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.772+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c3bcd33c:::obj192:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.772+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:c3bcd33c:::obj192:head) v3 prio 15 cost 0 latency 0.000105 pg_backfill(progress 1.0 e 116/116 lb 1:c3bcd33c:::obj192:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:c3bae056:::obj148:head local-lis/les=113/114 n=473 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.772+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.772+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3bae056:::obj148:head local-lis/les=113/114 n=473 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:c3bcd33c:::obj192:head) v3 2026-02-11T23:31:27.772+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3bae056:::obj148:head local-lis/les=113/114 n=473 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:c3bcd33c:::obj192:head) 2026-02-11T23:31:27.772+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.772+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:c3bcd33c:::obj192:head) v3 finish latency 0.000013 2026-02-11T23:31:27.772+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3bcd33c:::obj192:head local-lis/les=113/114 n=474 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.772+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.772+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.772+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3bcd33c:::obj192:head local-lis/les=113/114 n=474 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.772+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.772+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.772+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.772+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d1ca80 con 0x4acb000 2026-02-11T23:31:27.772+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:c41d6f6c:::obj298:head)]) -- 0x5df9980 con 0x4acb000 2026-02-11T23:31:27.772+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.772+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 985 ==== MOSDPGPush(1.0 116/113 [PushOp(1:c4f18846:::obj81:head, version: 111'81, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c4f18846:::obj81:head@111'81, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x555cea0 con 0x4acb000 2026-02-11T23:31:27.772+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:c4f18846:::obj81:head, version: 111'81, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c4f18846:::obj81:head@111'81, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 116 2026-02-11T23:31:27.772+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c4f18846:::obj81:head, version: 111'81, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c4f18846:::obj81:head@111'81, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.772+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.772+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 986 ==== pg_backfill(progress 1.0 e 116/116 lb 1:c41d6f6c:::obj298:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9ac00 con 0x4acb000 2026-02-11T23:31:27.772+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c4f18846:::obj81:head, version: 111'81, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c4f18846:::obj81:head@111'81, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.772+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:c41d6f6c:::obj298:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.772+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c41d6f6c:::obj298:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.772+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.772+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c4f18846:::obj81:head, version: 111'81, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c4f18846:::obj81:head@111'81, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.772+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.772+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:c4f18846:::obj81:head, version: 111'81, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c4f18846:::obj81:head@111'81, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000048 MOSDPGPush(1.0 116/113 [PushOp(1:c4f18846:::obj81:head, version: 111'81, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c4f18846:::obj81:head@111'81, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:c3bcd33c:::obj192:head local-lis/les=113/114 n=474 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.772+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.772+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3bcd33c:::obj192:head local-lis/les=113/114 n=474 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:c4f18846:::obj81:head, version: 111'81, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c4f18846:::obj81:head@111'81, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.772+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.772+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c41d6f6c:::obj298:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.772+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3bcd33c:::obj192:head local-lis/les=113/114 n=474 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:c4f18846:::obj81:head@111'81, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.772+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3bcd33c:::obj192:head local-lis/les=113/114 n=474 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:c4f18846:::obj81:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.772+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3bcd33c:::obj192:head local-lis/les=113/114 n=474 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:c4f18846:::obj81:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.772+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3bcd33c:::obj192:head local-lis/les=113/114 n=474 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:c4f18846:::obj81:head 2026-02-11T23:31:27.772+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3bcd33c:::obj192:head local-lis/les=113/114 n=474 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:c4f18846:::obj81:head v 111'81 2026-02-11T23:31:27.772+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3bcd33c:::obj192:head local-lis/les=113/114 n=474 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.772+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.772+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:c4f18846:::obj81:head, version: 111'81, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c4f18846:::obj81:head@111'81, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000068 2026-02-11T23:31:27.772+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.772+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.772+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c41d6f6c:::obj298:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.772+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:c41d6f6c:::obj298:head) v3 prio 15 cost 0 latency 0.000095 pg_backfill(progress 1.0 e 116/116 lb 1:c41d6f6c:::obj298:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:c3bcd33c:::obj192:head local-lis/les=113/114 n=474 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.772+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.772+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3bcd33c:::obj192:head local-lis/les=113/114 n=474 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:c41d6f6c:::obj298:head) v3 2026-02-11T23:31:27.772+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c3bcd33c:::obj192:head local-lis/les=113/114 n=474 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:c41d6f6c:::obj298:head) 2026-02-11T23:31:27.772+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.772+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:c41d6f6c:::obj298:head) v3 finish latency 0.000013 2026-02-11T23:31:27.772+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c41d6f6c:::obj298:head local-lis/les=113/114 n=475 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.772+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.772+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c41d6f6c:::obj298:head local-lis/les=113/114 n=475 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.772+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.772+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.772+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.772+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.772+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6171080 con 0x4acb000 2026-02-11T23:31:27.772+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:c4f18846:::obj81:head)]) -- 0x5e7ed80 con 0x4acb000 2026-02-11T23:31:27.772+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.773+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 987 ==== MOSDPGPush(1.0 116/113 [PushOp(1:c52c9666:::obj5:head, version: 111'5, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c52c9666:::obj5:head@111'5, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 4999+0+0 (unknown 0 0 0) 0x55f1d40 con 0x4acb000 2026-02-11T23:31:27.773+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:c52c9666:::obj5:head, version: 111'5, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c52c9666:::obj5:head@111'5, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000016 epoch 116 2026-02-11T23:31:27.773+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c52c9666:::obj5:head, version: 111'5, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c52c9666:::obj5:head@111'5, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.773+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.773+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 988 ==== pg_backfill(progress 1.0 e 116/116 lb 1:c4f18846:::obj81:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d9b180 con 0x4acb000 2026-02-11T23:31:27.773+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c52c9666:::obj5:head, version: 111'5, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c52c9666:::obj5:head@111'5, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.773+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:c4f18846:::obj81:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 116 2026-02-11T23:31:27.773+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c4f18846:::obj81:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.773+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.773+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c52c9666:::obj5:head, version: 111'5, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c52c9666:::obj5:head@111'5, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.773+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:c52c9666:::obj5:head, version: 111'5, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c52c9666:::obj5:head@111'5, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000052 MOSDPGPush(1.0 116/113 [PushOp(1:c52c9666:::obj5:head, version: 111'5, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c52c9666:::obj5:head@111'5, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:c41d6f6c:::obj298:head local-lis/les=113/114 n=475 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.773+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.773+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.773+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c4f18846:::obj81:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.773+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c41d6f6c:::obj298:head local-lis/les=113/114 n=475 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:c52c9666:::obj5:head, version: 111'5, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c52c9666:::obj5:head@111'5, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.773+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c41d6f6c:::obj298:head local-lis/les=113/114 n=475 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:c52c9666:::obj5:head@111'5, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.773+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c41d6f6c:::obj298:head local-lis/les=113/114 n=475 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:c52c9666:::obj5:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.773+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c41d6f6c:::obj298:head local-lis/les=113/114 n=475 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:c52c9666:::obj5:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.773+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c41d6f6c:::obj298:head local-lis/les=113/114 n=475 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:c52c9666:::obj5:head 2026-02-11T23:31:27.773+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c41d6f6c:::obj298:head local-lis/les=113/114 n=475 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:c52c9666:::obj5:head v 111'5 2026-02-11T23:31:27.773+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c41d6f6c:::obj298:head local-lis/les=113/114 n=475 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.773+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.773+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:c52c9666:::obj5:head, version: 111'5, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c52c9666:::obj5:head@111'5, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000069 2026-02-11T23:31:27.773+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.773+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.773+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c4f18846:::obj81:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.773+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:c4f18846:::obj81:head) v3 prio 15 cost 0 latency 0.000094 pg_backfill(progress 1.0 e 116/116 lb 1:c4f18846:::obj81:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:c41d6f6c:::obj298:head local-lis/les=113/114 n=475 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.773+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.773+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c41d6f6c:::obj298:head local-lis/les=113/114 n=475 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:c4f18846:::obj81:head) v3 2026-02-11T23:31:27.773+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c41d6f6c:::obj298:head local-lis/les=113/114 n=475 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:c4f18846:::obj81:head) 2026-02-11T23:31:27.773+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.773+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:c4f18846:::obj81:head) v3 finish latency 0.000012 2026-02-11T23:31:27.773+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c4f18846:::obj81:head local-lis/les=113/114 n=476 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.773+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.773+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.773+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c4f18846:::obj81:head local-lis/les=113/114 n=476 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.773+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.773+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.773+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.773+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d1cd80 con 0x4acb000 2026-02-11T23:31:27.773+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:c52c9666:::obj5:head)]) -- 0x5fd3c80 con 0x4acb000 2026-02-11T23:31:27.773+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.773+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 989 ==== MOSDPGPush(1.0 116/113 [PushOp(1:c57e020a:::obj395:head, version: 111'395, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c57e020a:::obj395:head@111'395, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e261a0 con 0x4acb000 2026-02-11T23:31:27.773+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:c57e020a:::obj395:head, version: 111'395, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c57e020a:::obj395:head@111'395, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000015 epoch 116 2026-02-11T23:31:27.773+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c57e020a:::obj395:head, version: 111'395, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c57e020a:::obj395:head@111'395, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.773+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.773+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 990 ==== pg_backfill(progress 1.0 e 116/116 lb 1:c52c9666:::obj5:head) ==== 1060+0+0 (unknown 0 0 0) 0x5d98000 con 0x4acb000 2026-02-11T23:31:27.773+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:c52c9666:::obj5:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.773+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c57e020a:::obj395:head, version: 111'395, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c57e020a:::obj395:head@111'395, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.773+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c52c9666:::obj5:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.773+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.773+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c52c9666:::obj5:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.773+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.773+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c57e020a:::obj395:head, version: 111'395, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c57e020a:::obj395:head@111'395, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.773+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:c57e020a:::obj395:head, version: 111'395, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c57e020a:::obj395:head@111'395, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000053 MOSDPGPush(1.0 116/113 [PushOp(1:c57e020a:::obj395:head, version: 111'395, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c57e020a:::obj395:head@111'395, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:c4f18846:::obj81:head local-lis/les=113/114 n=476 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.773+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.773+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c4f18846:::obj81:head local-lis/les=113/114 n=476 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:c57e020a:::obj395:head, version: 111'395, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c57e020a:::obj395:head@111'395, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.773+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c4f18846:::obj81:head local-lis/les=113/114 n=476 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:c57e020a:::obj395:head@111'395, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.773+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c4f18846:::obj81:head local-lis/les=113/114 n=476 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:c57e020a:::obj395:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.773+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c4f18846:::obj81:head local-lis/les=113/114 n=476 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:c57e020a:::obj395:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.773+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c4f18846:::obj81:head local-lis/les=113/114 n=476 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:c57e020a:::obj395:head 2026-02-11T23:31:27.773+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c4f18846:::obj81:head local-lis/les=113/114 n=476 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:c57e020a:::obj395:head v 111'395 2026-02-11T23:31:27.773+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c4f18846:::obj81:head local-lis/les=113/114 n=476 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.773+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.773+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:c57e020a:::obj395:head, version: 111'395, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c57e020a:::obj395:head@111'395, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.773+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c4f18846:::obj81:head local-lis/les=113/114 n=476 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.773+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.773+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.773+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c52c9666:::obj5:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.773+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:c52c9666:::obj5:head) v3 prio 15 cost 0 latency 0.000081 pg_backfill(progress 1.0 e 116/116 lb 1:c52c9666:::obj5:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:c4f18846:::obj81:head local-lis/les=113/114 n=476 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.773+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.773+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c4f18846:::obj81:head local-lis/les=113/114 n=476 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:c52c9666:::obj5:head) v3 2026-02-11T23:31:27.773+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c4f18846:::obj81:head local-lis/les=113/114 n=476 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:c52c9666:::obj5:head) 2026-02-11T23:31:27.773+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.773+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:c52c9666:::obj5:head) v3 finish latency 0.000023 2026-02-11T23:31:27.773+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.774+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.774+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c52c9666:::obj5:head local-lis/les=113/114 n=477 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.774+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.774+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.774+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.774+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6171680 con 0x4acb000 2026-02-11T23:31:27.774+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:c57e020a:::obj395:head)]) -- 0x600a900 con 0x4acb000 2026-02-11T23:31:27.774+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.774+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 991 ==== MOSDPGPush(1.0 116/113 [PushOp(1:c59209e1:::obj523:head, version: 111'523, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c59209e1:::obj523:head@111'523, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559cd00 con 0x4acb000 2026-02-11T23:31:27.774+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:c59209e1:::obj523:head, version: 111'523, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c59209e1:::obj523:head@111'523, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 116 2026-02-11T23:31:27.774+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c59209e1:::obj523:head, version: 111'523, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c59209e1:::obj523:head@111'523, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.774+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.774+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 992 ==== pg_backfill(progress 1.0 e 116/116 lb 1:c57e020a:::obj395:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98580 con 0x4acb000 2026-02-11T23:31:27.774+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:c57e020a:::obj395:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.774+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c59209e1:::obj523:head, version: 111'523, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c59209e1:::obj523:head@111'523, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.774+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c57e020a:::obj395:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.774+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.774+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c57e020a:::obj395:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.774+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.774+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c59209e1:::obj523:head, version: 111'523, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c59209e1:::obj523:head@111'523, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.774+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:c59209e1:::obj523:head, version: 111'523, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c59209e1:::obj523:head@111'523, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000054 MOSDPGPush(1.0 116/113 [PushOp(1:c59209e1:::obj523:head, version: 111'523, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c59209e1:::obj523:head@111'523, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:c52c9666:::obj5:head local-lis/les=113/114 n=477 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.774+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.774+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c52c9666:::obj5:head local-lis/les=113/114 n=477 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:c59209e1:::obj523:head, version: 111'523, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c59209e1:::obj523:head@111'523, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.774+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c52c9666:::obj5:head local-lis/les=113/114 n=477 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:c59209e1:::obj523:head@111'523, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.774+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c52c9666:::obj5:head local-lis/les=113/114 n=477 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:c59209e1:::obj523:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.774+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c52c9666:::obj5:head local-lis/les=113/114 n=477 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:c59209e1:::obj523:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.774+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c52c9666:::obj5:head local-lis/les=113/114 n=477 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:c59209e1:::obj523:head 2026-02-11T23:31:27.774+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c52c9666:::obj5:head local-lis/les=113/114 n=477 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:c59209e1:::obj523:head v 111'523 2026-02-11T23:31:27.774+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c52c9666:::obj5:head local-lis/les=113/114 n=477 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.774+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.774+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:c59209e1:::obj523:head, version: 111'523, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c59209e1:::obj523:head@111'523, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:27.774+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.774+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.774+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c57e020a:::obj395:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.774+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:c57e020a:::obj395:head) v3 prio 15 cost 0 latency 0.000097 pg_backfill(progress 1.0 e 116/116 lb 1:c57e020a:::obj395:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:c52c9666:::obj5:head local-lis/les=113/114 n=477 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.774+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.774+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c52c9666:::obj5:head local-lis/les=113/114 n=477 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:c57e020a:::obj395:head) v3 2026-02-11T23:31:27.774+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c52c9666:::obj5:head local-lis/les=113/114 n=477 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:c57e020a:::obj395:head) 2026-02-11T23:31:27.774+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.774+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:c57e020a:::obj395:head) v3 finish latency 0.000018 2026-02-11T23:31:27.774+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c57e020a:::obj395:head local-lis/les=113/114 n=478 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.774+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.774+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.774+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c57e020a:::obj395:head local-lis/les=113/114 n=478 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.774+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.774+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.774+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.774+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d1d380 con 0x4acb000 2026-02-11T23:31:27.774+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:c59209e1:::obj523:head)]) -- 0x5daa900 con 0x4acb000 2026-02-11T23:31:27.774+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.774+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 993 ==== MOSDPGPush(1.0 116/113 [PushOp(1:c5df6474:::obj459:head, version: 111'459, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c5df6474:::obj459:head@111'459, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559d1e0 con 0x4acb000 2026-02-11T23:31:27.774+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:c5df6474:::obj459:head, version: 111'459, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c5df6474:::obj459:head@111'459, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 116 2026-02-11T23:31:27.774+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c5df6474:::obj459:head, version: 111'459, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c5df6474:::obj459:head@111'459, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.774+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.774+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 994 ==== pg_backfill(progress 1.0 e 116/116 lb 1:c59209e1:::obj523:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98b00 con 0x4acb000 2026-02-11T23:31:27.774+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:c59209e1:::obj523:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.774+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c5df6474:::obj459:head, version: 111'459, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c5df6474:::obj459:head@111'459, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.774+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c59209e1:::obj523:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.774+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.774+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c5df6474:::obj459:head, version: 111'459, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c5df6474:::obj459:head@111'459, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.774+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:c5df6474:::obj459:head, version: 111'459, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c5df6474:::obj459:head@111'459, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000040 MOSDPGPush(1.0 116/113 [PushOp(1:c5df6474:::obj459:head, version: 111'459, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c5df6474:::obj459:head@111'459, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:c57e020a:::obj395:head local-lis/les=113/114 n=478 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.774+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.774+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c59209e1:::obj523:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.774+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.774+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c57e020a:::obj395:head local-lis/les=113/114 n=478 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:c5df6474:::obj459:head, version: 111'459, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c5df6474:::obj459:head@111'459, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.774+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c57e020a:::obj395:head local-lis/les=113/114 n=478 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:c5df6474:::obj459:head@111'459, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.774+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c57e020a:::obj395:head local-lis/les=113/114 n=478 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:c5df6474:::obj459:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.774+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c57e020a:::obj395:head local-lis/les=113/114 n=478 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:c5df6474:::obj459:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.774+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c57e020a:::obj395:head local-lis/les=113/114 n=478 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:c5df6474:::obj459:head 2026-02-11T23:31:27.774+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c57e020a:::obj395:head local-lis/les=113/114 n=478 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:c5df6474:::obj459:head v 111'459 2026-02-11T23:31:27.774+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c57e020a:::obj395:head local-lis/les=113/114 n=478 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.774+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.774+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:c5df6474:::obj459:head, version: 111'459, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c5df6474:::obj459:head@111'459, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000069 2026-02-11T23:31:27.775+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.775+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.775+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c59209e1:::obj523:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.775+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:c59209e1:::obj523:head) v3 prio 15 cost 0 latency 0.000096 pg_backfill(progress 1.0 e 116/116 lb 1:c59209e1:::obj523:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:c57e020a:::obj395:head local-lis/les=113/114 n=478 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.775+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.775+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c57e020a:::obj395:head local-lis/les=113/114 n=478 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:c59209e1:::obj523:head) v3 2026-02-11T23:31:27.775+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c57e020a:::obj395:head local-lis/les=113/114 n=478 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:c59209e1:::obj523:head) 2026-02-11T23:31:27.775+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.775+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:c59209e1:::obj523:head) v3 finish latency 0.000024 2026-02-11T23:31:27.775+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c59209e1:::obj523:head local-lis/les=113/114 n=479 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.775+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.775+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.775+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c59209e1:::obj523:head local-lis/les=113/114 n=479 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.775+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.775+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.775+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.775+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fcf800 con 0x4acb000 2026-02-11T23:31:27.775+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:c5df6474:::obj459:head)]) -- 0x5daa300 con 0x4acb000 2026-02-11T23:31:27.775+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.775+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.775+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.775+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 995 ==== MOSDPGPush(1.0 116/113 [PushOp(1:c5e569a1:::obj444:head, version: 111'444, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c5e569a1:::obj444:head@111'444, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c340 con 0x4acb000 2026-02-11T23:31:27.775+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:c5e569a1:::obj444:head, version: 111'444, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c5e569a1:::obj444:head@111'444, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000018 epoch 116 2026-02-11T23:31:27.775+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c5e569a1:::obj444:head, version: 111'444, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c5e569a1:::obj444:head@111'444, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.775+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.775+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c5e569a1:::obj444:head, version: 111'444, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c5e569a1:::obj444:head@111'444, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.775+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 996 ==== pg_backfill(progress 1.0 e 116/116 lb 1:c5df6474:::obj459:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99080 con 0x4acb000 2026-02-11T23:31:27.775+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:c5df6474:::obj459:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.775+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.775+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c5e569a1:::obj444:head, version: 111'444, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c5e569a1:::obj444:head@111'444, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.775+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c5df6474:::obj459:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.775+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.775+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:c5e569a1:::obj444:head, version: 111'444, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c5e569a1:::obj444:head@111'444, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000044 MOSDPGPush(1.0 116/113 [PushOp(1:c5e569a1:::obj444:head, version: 111'444, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c5e569a1:::obj444:head@111'444, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:c59209e1:::obj523:head local-lis/les=113/114 n=479 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.775+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c5df6474:::obj459:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.775+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.775+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c59209e1:::obj523:head local-lis/les=113/114 n=479 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:c5e569a1:::obj444:head, version: 111'444, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c5e569a1:::obj444:head@111'444, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.775+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c59209e1:::obj523:head local-lis/les=113/114 n=479 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:c5e569a1:::obj444:head@111'444, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.775+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c59209e1:::obj523:head local-lis/les=113/114 n=479 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:c5e569a1:::obj444:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.775+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c59209e1:::obj523:head local-lis/les=113/114 n=479 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:c5e569a1:::obj444:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.775+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c59209e1:::obj523:head local-lis/les=113/114 n=479 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:c5e569a1:::obj444:head 2026-02-11T23:31:27.775+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c59209e1:::obj523:head local-lis/les=113/114 n=479 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:c5e569a1:::obj444:head v 111'444 2026-02-11T23:31:27.775+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c59209e1:::obj523:head local-lis/les=113/114 n=479 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.775+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.775+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:c5e569a1:::obj444:head, version: 111'444, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c5e569a1:::obj444:head@111'444, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:27.775+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c59209e1:::obj523:head local-lis/les=113/114 n=479 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.775+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.775+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.775+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c5df6474:::obj459:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.775+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:c5df6474:::obj459:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 1.0 e 116/116 lb 1:c5df6474:::obj459:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:c59209e1:::obj523:head local-lis/les=113/114 n=479 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.775+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.775+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c59209e1:::obj523:head local-lis/les=113/114 n=479 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:c5df6474:::obj459:head) v3 2026-02-11T23:31:27.775+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c59209e1:::obj523:head local-lis/les=113/114 n=479 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:c5df6474:::obj459:head) 2026-02-11T23:31:27.775+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.775+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:c5df6474:::obj459:head) v3 finish latency 0.000019 2026-02-11T23:31:27.775+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.775+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.775+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c5df6474:::obj459:head local-lis/les=113/114 n=480 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.775+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.775+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.775+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.775+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x612ed80 con 0x4acb000 2026-02-11T23:31:27.775+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:c5e569a1:::obj444:head)]) -- 0x6170300 con 0x4acb000 2026-02-11T23:31:27.775+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.776+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 997 ==== MOSDPGPush(1.0 116/113 [PushOp(1:c64b60bf:::obj563:head, version: 111'563, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c64b60bf:::obj563:head@111'563, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559da00 con 0x4acb000 2026-02-11T23:31:27.776+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:c64b60bf:::obj563:head, version: 111'563, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c64b60bf:::obj563:head@111'563, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.776+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c64b60bf:::obj563:head, version: 111'563, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c64b60bf:::obj563:head@111'563, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.776+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.776+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 998 ==== pg_backfill(progress 1.0 e 116/116 lb 1:c5e569a1:::obj444:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99600 con 0x4acb000 2026-02-11T23:31:27.776+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:c5e569a1:::obj444:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.776+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c64b60bf:::obj563:head, version: 111'563, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c64b60bf:::obj563:head@111'563, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.776+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c5e569a1:::obj444:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.776+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.776+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c5e569a1:::obj444:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.776+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.776+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c64b60bf:::obj563:head, version: 111'563, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c64b60bf:::obj563:head@111'563, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.776+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:c64b60bf:::obj563:head, version: 111'563, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c64b60bf:::obj563:head@111'563, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000047 MOSDPGPush(1.0 116/113 [PushOp(1:c64b60bf:::obj563:head, version: 111'563, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c64b60bf:::obj563:head@111'563, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:c5df6474:::obj459:head local-lis/les=113/114 n=480 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.776+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.776+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c5df6474:::obj459:head local-lis/les=113/114 n=480 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:c64b60bf:::obj563:head, version: 111'563, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c64b60bf:::obj563:head@111'563, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.776+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c5df6474:::obj459:head local-lis/les=113/114 n=480 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:c64b60bf:::obj563:head@111'563, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.776+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c5df6474:::obj459:head local-lis/les=113/114 n=480 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:c64b60bf:::obj563:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.776+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c5df6474:::obj459:head local-lis/les=113/114 n=480 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:c64b60bf:::obj563:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.776+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c5df6474:::obj459:head local-lis/les=113/114 n=480 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:c64b60bf:::obj563:head 2026-02-11T23:31:27.776+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c5df6474:::obj459:head local-lis/les=113/114 n=480 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:c64b60bf:::obj563:head v 111'563 2026-02-11T23:31:27.776+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c5df6474:::obj459:head local-lis/les=113/114 n=480 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.776+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.776+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:c64b60bf:::obj563:head, version: 111'563, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c64b60bf:::obj563:head@111'563, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:27.776+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.776+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.776+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c5e569a1:::obj444:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.776+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:c5e569a1:::obj444:head) v3 prio 15 cost 0 latency 0.000080 pg_backfill(progress 1.0 e 116/116 lb 1:c5e569a1:::obj444:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:c5df6474:::obj459:head local-lis/les=113/114 n=480 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.776+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.776+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c5df6474:::obj459:head local-lis/les=113/114 n=480 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:c5e569a1:::obj444:head) v3 2026-02-11T23:31:27.776+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c5df6474:::obj459:head local-lis/les=113/114 n=480 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:c5e569a1:::obj444:head) 2026-02-11T23:31:27.776+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.776+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:c5e569a1:::obj444:head) v3 finish latency 0.000010 2026-02-11T23:31:27.776+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c5e569a1:::obj444:head local-lis/les=113/114 n=481 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.776+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.776+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.776+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c5e569a1:::obj444:head local-lis/les=113/114 n=481 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.776+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.776+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.776+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.776+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x600a480 con 0x4acb000 2026-02-11T23:31:27.776+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:c64b60bf:::obj563:head)]) -- 0x5fd3b00 con 0x4acb000 2026-02-11T23:31:27.776+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.776+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 999 ==== MOSDPGPush(1.0 116/113 [PushOp(1:c68dea80:::obj235:head, version: 111'235, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c68dea80:::obj235:head@111'235, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559cea0 con 0x4acb000 2026-02-11T23:31:27.776+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:c68dea80:::obj235:head, version: 111'235, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c68dea80:::obj235:head@111'235, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.776+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c68dea80:::obj235:head, version: 111'235, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c68dea80:::obj235:head@111'235, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.776+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.776+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1000 ==== pg_backfill(progress 1.0 e 116/116 lb 1:c64b60bf:::obj563:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99b80 con 0x4acb000 2026-02-11T23:31:27.776+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:c64b60bf:::obj563:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.776+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c68dea80:::obj235:head, version: 111'235, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c68dea80:::obj235:head@111'235, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.776+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c64b60bf:::obj563:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.776+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.776+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c68dea80:::obj235:head, version: 111'235, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c68dea80:::obj235:head@111'235, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.776+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:c68dea80:::obj235:head, version: 111'235, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c68dea80:::obj235:head@111'235, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 116/113 [PushOp(1:c68dea80:::obj235:head, version: 111'235, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c68dea80:::obj235:head@111'235, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:c5e569a1:::obj444:head local-lis/les=113/114 n=481 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.776+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.776+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.776+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c5e569a1:::obj444:head local-lis/les=113/114 n=481 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:c68dea80:::obj235:head, version: 111'235, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c68dea80:::obj235:head@111'235, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.776+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c64b60bf:::obj563:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.776+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c5e569a1:::obj444:head local-lis/les=113/114 n=481 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:c68dea80:::obj235:head@111'235, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.776+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c5e569a1:::obj444:head local-lis/les=113/114 n=481 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:c68dea80:::obj235:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.776+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c5e569a1:::obj444:head local-lis/les=113/114 n=481 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:c68dea80:::obj235:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.776+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c5e569a1:::obj444:head local-lis/les=113/114 n=481 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:c68dea80:::obj235:head 2026-02-11T23:31:27.776+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c5e569a1:::obj444:head local-lis/les=113/114 n=481 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:c68dea80:::obj235:head v 111'235 2026-02-11T23:31:27.776+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c5e569a1:::obj444:head local-lis/les=113/114 n=481 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.776+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.776+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:c68dea80:::obj235:head, version: 111'235, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c68dea80:::obj235:head@111'235, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:27.776+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.776+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.776+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c64b60bf:::obj563:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.776+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:c64b60bf:::obj563:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 116/116 lb 1:c64b60bf:::obj563:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:c5e569a1:::obj444:head local-lis/les=113/114 n=481 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.776+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.776+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c5e569a1:::obj444:head local-lis/les=113/114 n=481 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:c64b60bf:::obj563:head) v3 2026-02-11T23:31:27.776+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c5e569a1:::obj444:head local-lis/les=113/114 n=481 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:c64b60bf:::obj563:head) 2026-02-11T23:31:27.776+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.776+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:c64b60bf:::obj563:head) v3 finish latency 0.000010 2026-02-11T23:31:27.776+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c64b60bf:::obj563:head local-lis/les=113/114 n=482 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.776+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.776+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.776+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c64b60bf:::obj563:head local-lis/les=113/114 n=482 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.776+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.776+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.776+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.776+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fed080 con 0x4acb000 2026-02-11T23:31:27.776+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:c68dea80:::obj235:head)]) -- 0x5fd3800 con 0x4acb000 2026-02-11T23:31:27.776+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.777+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1001 ==== MOSDPGPush(1.0 116/113 [PushOp(1:c71bdb5a:::obj349:head, version: 111'349, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c71bdb5a:::obj349:head@111'349, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d8820 con 0x4acb000 2026-02-11T23:31:27.777+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:c71bdb5a:::obj349:head, version: 111'349, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c71bdb5a:::obj349:head@111'349, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000016 epoch 116 2026-02-11T23:31:27.777+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c71bdb5a:::obj349:head, version: 111'349, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c71bdb5a:::obj349:head@111'349, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.777+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.777+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c71bdb5a:::obj349:head, version: 111'349, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c71bdb5a:::obj349:head@111'349, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.777+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1002 ==== pg_backfill(progress 1.0 e 116/116 lb 1:c68dea80:::obj235:head) ==== 1062+0+0 (unknown 0 0 0) 0x4a2cb00 con 0x4acb000 2026-02-11T23:31:27.777+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.777+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:c68dea80:::obj235:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 116 2026-02-11T23:31:27.777+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c71bdb5a:::obj349:head, version: 111'349, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c71bdb5a:::obj349:head@111'349, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.777+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c68dea80:::obj235:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.777+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.777+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:c71bdb5a:::obj349:head, version: 111'349, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c71bdb5a:::obj349:head@111'349, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000042 MOSDPGPush(1.0 116/113 [PushOp(1:c71bdb5a:::obj349:head, version: 111'349, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c71bdb5a:::obj349:head@111'349, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:c64b60bf:::obj563:head local-lis/les=113/114 n=482 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.777+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c68dea80:::obj235:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.777+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.777+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c64b60bf:::obj563:head local-lis/les=113/114 n=482 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:c71bdb5a:::obj349:head, version: 111'349, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c71bdb5a:::obj349:head@111'349, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.777+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c64b60bf:::obj563:head local-lis/les=113/114 n=482 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:c71bdb5a:::obj349:head@111'349, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.777+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c64b60bf:::obj563:head local-lis/les=113/114 n=482 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:c71bdb5a:::obj349:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.777+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c64b60bf:::obj563:head local-lis/les=113/114 n=482 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:c71bdb5a:::obj349:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.777+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c64b60bf:::obj563:head local-lis/les=113/114 n=482 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:c71bdb5a:::obj349:head 2026-02-11T23:31:27.777+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c64b60bf:::obj563:head local-lis/les=113/114 n=482 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:c71bdb5a:::obj349:head v 111'349 2026-02-11T23:31:27.777+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c64b60bf:::obj563:head local-lis/les=113/114 n=482 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.777+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.777+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:c71bdb5a:::obj349:head, version: 111'349, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c71bdb5a:::obj349:head@111'349, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:27.777+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c64b60bf:::obj563:head local-lis/les=113/114 n=482 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.777+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.777+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.777+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c68dea80:::obj235:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.777+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:c68dea80:::obj235:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 1.0 e 116/116 lb 1:c68dea80:::obj235:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:c64b60bf:::obj563:head local-lis/les=113/114 n=482 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.777+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.777+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c64b60bf:::obj563:head local-lis/les=113/114 n=482 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:c68dea80:::obj235:head) v3 2026-02-11T23:31:27.777+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c64b60bf:::obj563:head local-lis/les=113/114 n=482 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:c68dea80:::obj235:head) 2026-02-11T23:31:27.777+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.777+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:c68dea80:::obj235:head) v3 finish latency 0.000011 2026-02-11T23:31:27.777+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.777+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.777+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c68dea80:::obj235:head local-lis/les=113/114 n=483 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.777+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.777+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.777+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.777+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fcf680 con 0x4acb000 2026-02-11T23:31:27.777+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:c71bdb5a:::obj349:head)]) -- 0x5d1de00 con 0x4acb000 2026-02-11T23:31:27.777+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.777+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1003 ==== MOSDPGPush(1.0 116/113 [PushOp(1:c7481937:::obj131:head, version: 111'131, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c7481937:::obj131:head@111'131, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d9860 con 0x4acb000 2026-02-11T23:31:27.777+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:c7481937:::obj131:head, version: 111'131, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c7481937:::obj131:head@111'131, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000018 epoch 116 2026-02-11T23:31:27.777+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c7481937:::obj131:head, version: 111'131, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c7481937:::obj131:head@111'131, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.777+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.777+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c7481937:::obj131:head, version: 111'131, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c7481937:::obj131:head@111'131, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.777+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1004 ==== pg_backfill(progress 1.0 e 116/116 lb 1:c71bdb5a:::obj349:head) ==== 1062+0+0 (unknown 0 0 0) 0x4a2c000 con 0x4acb000 2026-02-11T23:31:27.777+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.777+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:c71bdb5a:::obj349:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 116 2026-02-11T23:31:27.777+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c7481937:::obj131:head, version: 111'131, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c7481937:::obj131:head@111'131, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.777+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c71bdb5a:::obj349:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.777+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.777+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c71bdb5a:::obj349:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.777+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:c7481937:::obj131:head, version: 111'131, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c7481937:::obj131:head@111'131, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000048 MOSDPGPush(1.0 116/113 [PushOp(1:c7481937:::obj131:head, version: 111'131, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c7481937:::obj131:head@111'131, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:c68dea80:::obj235:head local-lis/les=113/114 n=483 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.777+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.777+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c68dea80:::obj235:head local-lis/les=113/114 n=483 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:c7481937:::obj131:head, version: 111'131, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c7481937:::obj131:head@111'131, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.777+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c68dea80:::obj235:head local-lis/les=113/114 n=483 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:c7481937:::obj131:head@111'131, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.777+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c68dea80:::obj235:head local-lis/les=113/114 n=483 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:c7481937:::obj131:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.777+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c68dea80:::obj235:head local-lis/les=113/114 n=483 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:c7481937:::obj131:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.777+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c68dea80:::obj235:head local-lis/les=113/114 n=483 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:c7481937:::obj131:head 2026-02-11T23:31:27.777+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c68dea80:::obj235:head local-lis/les=113/114 n=483 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:c7481937:::obj131:head v 111'131 2026-02-11T23:31:27.777+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c68dea80:::obj235:head local-lis/les=113/114 n=483 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.777+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.777+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:c7481937:::obj131:head, version: 111'131, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c7481937:::obj131:head@111'131, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:27.777+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.777+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.777+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c71bdb5a:::obj349:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.777+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:c71bdb5a:::obj349:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 1.0 e 116/116 lb 1:c71bdb5a:::obj349:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:c68dea80:::obj235:head local-lis/les=113/114 n=483 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.777+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.777+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c68dea80:::obj235:head local-lis/les=113/114 n=483 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:c71bdb5a:::obj349:head) v3 2026-02-11T23:31:27.777+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c68dea80:::obj235:head local-lis/les=113/114 n=483 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:c71bdb5a:::obj349:head) 2026-02-11T23:31:27.777+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.777+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:c71bdb5a:::obj349:head) v3 finish latency 0.000013 2026-02-11T23:31:27.777+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c71bdb5a:::obj349:head local-lis/les=113/114 n=484 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.777+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.778+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.778+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c71bdb5a:::obj349:head local-lis/les=113/114 n=484 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.778+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.778+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.778+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.778+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5ca1680 con 0x4acb000 2026-02-11T23:31:27.778+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:c7481937:::obj131:head)]) -- 0x5fd3500 con 0x4acb000 2026-02-11T23:31:27.778+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.778+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1005 ==== MOSDPGPush(1.0 116/113 [PushOp(1:c8495ae6:::obj149:head, version: 111'149, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c8495ae6:::obj149:head@111'149, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d8340 con 0x4acb000 2026-02-11T23:31:27.778+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:c8495ae6:::obj149:head, version: 111'149, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c8495ae6:::obj149:head@111'149, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 116 2026-02-11T23:31:27.778+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c8495ae6:::obj149:head, version: 111'149, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c8495ae6:::obj149:head@111'149, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.778+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.778+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1006 ==== pg_backfill(progress 1.0 e 116/116 lb 1:c7481937:::obj131:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9a100 con 0x4acb000 2026-02-11T23:31:27.778+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:c7481937:::obj131:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.778+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c8495ae6:::obj149:head, version: 111'149, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c8495ae6:::obj149:head@111'149, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.778+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c7481937:::obj131:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.778+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.778+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:c8495ae6:::obj149:head, version: 111'149, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c8495ae6:::obj149:head@111'149, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.778+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.778+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:c8495ae6:::obj149:head, version: 111'149, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c8495ae6:::obj149:head@111'149, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(1.0 116/113 [PushOp(1:c8495ae6:::obj149:head, version: 111'149, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c8495ae6:::obj149:head@111'149, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:c71bdb5a:::obj349:head local-lis/les=113/114 n=484 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.778+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.778+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c71bdb5a:::obj349:head local-lis/les=113/114 n=484 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:c8495ae6:::obj149:head, version: 111'149, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c8495ae6:::obj149:head@111'149, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.778+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c71bdb5a:::obj349:head local-lis/les=113/114 n=484 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:c8495ae6:::obj149:head@111'149, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.778+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.778+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c7481937:::obj131:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.778+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c71bdb5a:::obj349:head local-lis/les=113/114 n=484 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:c8495ae6:::obj149:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.778+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c71bdb5a:::obj349:head local-lis/les=113/114 n=484 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:c8495ae6:::obj149:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.778+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c71bdb5a:::obj349:head local-lis/les=113/114 n=484 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:c8495ae6:::obj149:head 2026-02-11T23:31:27.778+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c71bdb5a:::obj349:head local-lis/les=113/114 n=484 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:c8495ae6:::obj149:head v 111'149 2026-02-11T23:31:27.778+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c71bdb5a:::obj349:head local-lis/les=113/114 n=484 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.778+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.778+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:c8495ae6:::obj149:head, version: 111'149, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c8495ae6:::obj149:head@111'149, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:27.778+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.778+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.778+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c7481937:::obj131:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.778+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:c7481937:::obj131:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 1.0 e 116/116 lb 1:c7481937:::obj131:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:c71bdb5a:::obj349:head local-lis/les=113/114 n=484 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.778+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.778+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c71bdb5a:::obj349:head local-lis/les=113/114 n=484 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:c7481937:::obj131:head) v3 2026-02-11T23:31:27.778+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c71bdb5a:::obj349:head local-lis/les=113/114 n=484 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:c7481937:::obj131:head) 2026-02-11T23:31:27.778+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.778+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:c7481937:::obj131:head) v3 finish latency 0.000010 2026-02-11T23:31:27.778+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c7481937:::obj131:head local-lis/les=113/114 n=485 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.778+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.778+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.778+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c7481937:::obj131:head local-lis/les=113/114 n=485 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.778+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.778+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.778+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.778+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x613f980 con 0x4acb000 2026-02-11T23:31:27.778+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:c8495ae6:::obj149:head)]) -- 0x5d8e900 con 0x4acb000 2026-02-11T23:31:27.778+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.779+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1007 ==== MOSDPGPush(1.0 116/113 [PushOp(1:ca82ed30:::obj135:head, version: 111'135, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ca82ed30:::obj135:head@111'135, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56471e0 con 0x4acb000 2026-02-11T23:31:27.779+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:ca82ed30:::obj135:head, version: 111'135, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ca82ed30:::obj135:head@111'135, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.779+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:ca82ed30:::obj135:head, version: 111'135, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ca82ed30:::obj135:head@111'135, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.779+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.779+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1008 ==== pg_backfill(progress 1.0 e 116/116 lb 1:c8495ae6:::obj149:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9ac00 con 0x4acb000 2026-02-11T23:31:27.779+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:ca82ed30:::obj135:head, version: 111'135, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ca82ed30:::obj135:head@111'135, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.779+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:c8495ae6:::obj149:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.779+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.779+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c8495ae6:::obj149:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.779+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:ca82ed30:::obj135:head, version: 111'135, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ca82ed30:::obj135:head@111'135, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.779+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.779+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:ca82ed30:::obj135:head, version: 111'135, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ca82ed30:::obj135:head@111'135, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 116/113 [PushOp(1:ca82ed30:::obj135:head, version: 111'135, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ca82ed30:::obj135:head@111'135, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:c7481937:::obj131:head local-lis/les=113/114 n=485 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.779+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.779+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c7481937:::obj131:head local-lis/les=113/114 n=485 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:ca82ed30:::obj135:head, version: 111'135, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ca82ed30:::obj135:head@111'135, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.779+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c7481937:::obj131:head local-lis/les=113/114 n=485 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:ca82ed30:::obj135:head@111'135, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.779+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.779+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c8495ae6:::obj149:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.779+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c7481937:::obj131:head local-lis/les=113/114 n=485 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:ca82ed30:::obj135:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.779+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c7481937:::obj131:head local-lis/les=113/114 n=485 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:ca82ed30:::obj135:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.779+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c7481937:::obj131:head local-lis/les=113/114 n=485 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:ca82ed30:::obj135:head 2026-02-11T23:31:27.779+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c7481937:::obj131:head local-lis/les=113/114 n=485 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:ca82ed30:::obj135:head v 111'135 2026-02-11T23:31:27.779+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c7481937:::obj131:head local-lis/les=113/114 n=485 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.779+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.779+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:ca82ed30:::obj135:head, version: 111'135, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ca82ed30:::obj135:head@111'135, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:27.779+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.779+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.779+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:c8495ae6:::obj149:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.779+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:c8495ae6:::obj149:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 116/116 lb 1:c8495ae6:::obj149:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:c7481937:::obj131:head local-lis/les=113/114 n=485 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.779+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.779+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c7481937:::obj131:head local-lis/les=113/114 n=485 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:c8495ae6:::obj149:head) v3 2026-02-11T23:31:27.779+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c7481937:::obj131:head local-lis/les=113/114 n=485 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:c8495ae6:::obj149:head) 2026-02-11T23:31:27.779+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.779+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:c8495ae6:::obj149:head) v3 finish latency 0.000009 2026-02-11T23:31:27.779+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c8495ae6:::obj149:head local-lis/les=113/114 n=486 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.779+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.779+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.779+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c8495ae6:::obj149:head local-lis/les=113/114 n=486 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.779+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.779+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.779+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.779+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x615c480 con 0x4acb000 2026-02-11T23:31:27.779+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:ca82ed30:::obj135:head)]) -- 0x5fd3380 con 0x4acb000 2026-02-11T23:31:27.779+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.779+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1009 ==== MOSDPGPush(1.0 116/113 [PushOp(1:cadaa6e3:::obj220:head, version: 111'220, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cadaa6e3:::obj220:head@111'220, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d8b60 con 0x4acb000 2026-02-11T23:31:27.779+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:cadaa6e3:::obj220:head, version: 111'220, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cadaa6e3:::obj220:head@111'220, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.779+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:cadaa6e3:::obj220:head, version: 111'220, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cadaa6e3:::obj220:head@111'220, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.779+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.779+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1010 ==== pg_backfill(progress 1.0 e 116/116 lb 1:ca82ed30:::obj135:head) ==== 1062+0+0 (unknown 0 0 0) 0x6179700 con 0x4acb000 2026-02-11T23:31:27.779+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:cadaa6e3:::obj220:head, version: 111'220, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cadaa6e3:::obj220:head@111'220, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.779+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:ca82ed30:::obj135:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.779+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.779+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:ca82ed30:::obj135:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.779+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:cadaa6e3:::obj220:head, version: 111'220, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cadaa6e3:::obj220:head@111'220, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.779+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.779+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:cadaa6e3:::obj220:head, version: 111'220, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cadaa6e3:::obj220:head@111'220, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 116/113 [PushOp(1:cadaa6e3:::obj220:head, version: 111'220, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cadaa6e3:::obj220:head@111'220, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:c8495ae6:::obj149:head local-lis/les=113/114 n=486 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.779+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.779+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c8495ae6:::obj149:head local-lis/les=113/114 n=486 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:cadaa6e3:::obj220:head, version: 111'220, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cadaa6e3:::obj220:head@111'220, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.779+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c8495ae6:::obj149:head local-lis/les=113/114 n=486 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:cadaa6e3:::obj220:head@111'220, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.779+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.779+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:ca82ed30:::obj135:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.779+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c8495ae6:::obj149:head local-lis/les=113/114 n=486 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:cadaa6e3:::obj220:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.779+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c8495ae6:::obj149:head local-lis/les=113/114 n=486 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:cadaa6e3:::obj220:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.779+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c8495ae6:::obj149:head local-lis/les=113/114 n=486 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:cadaa6e3:::obj220:head 2026-02-11T23:31:27.779+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c8495ae6:::obj149:head local-lis/les=113/114 n=486 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:cadaa6e3:::obj220:head v 111'220 2026-02-11T23:31:27.779+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c8495ae6:::obj149:head local-lis/les=113/114 n=486 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.779+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.779+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:cadaa6e3:::obj220:head, version: 111'220, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cadaa6e3:::obj220:head@111'220, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.779+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.779+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.779+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:ca82ed30:::obj135:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.779+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:ca82ed30:::obj135:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 116/116 lb 1:ca82ed30:::obj135:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:c8495ae6:::obj149:head local-lis/les=113/114 n=486 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.779+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.779+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c8495ae6:::obj149:head local-lis/les=113/114 n=486 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:ca82ed30:::obj135:head) v3 2026-02-11T23:31:27.779+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:c8495ae6:::obj149:head local-lis/les=113/114 n=486 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:ca82ed30:::obj135:head) 2026-02-11T23:31:27.779+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.779+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:ca82ed30:::obj135:head) v3 finish latency 0.000009 2026-02-11T23:31:27.779+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ca82ed30:::obj135:head local-lis/les=113/114 n=487 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.779+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.779+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.779+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ca82ed30:::obj135:head local-lis/les=113/114 n=487 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.779+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.779+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.779+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.779+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6014000 con 0x4acb000 2026-02-11T23:31:27.779+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:cadaa6e3:::obj220:head)]) -- 0x5d78d80 con 0x4acb000 2026-02-11T23:31:27.779+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.780+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1011 ==== MOSDPGPush(1.0 116/113 [PushOp(1:cb1a5635:::obj233:head, version: 111'233, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb1a5635:::obj233:head@111'233, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e269c0 con 0x4acb000 2026-02-11T23:31:27.780+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:cb1a5635:::obj233:head, version: 111'233, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb1a5635:::obj233:head@111'233, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.780+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:cb1a5635:::obj233:head, version: 111'233, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb1a5635:::obj233:head@111'233, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.780+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.780+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1012 ==== pg_backfill(progress 1.0 e 116/116 lb 1:cadaa6e3:::obj220:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176000 con 0x4acb000 2026-02-11T23:31:27.780+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:cb1a5635:::obj233:head, version: 111'233, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb1a5635:::obj233:head@111'233, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.780+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:cadaa6e3:::obj220:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.780+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.780+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:cadaa6e3:::obj220:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.780+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:cb1a5635:::obj233:head, version: 111'233, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb1a5635:::obj233:head@111'233, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.780+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.780+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:cb1a5635:::obj233:head, version: 111'233, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb1a5635:::obj233:head@111'233, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:cb1a5635:::obj233:head, version: 111'233, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb1a5635:::obj233:head@111'233, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:ca82ed30:::obj135:head local-lis/les=113/114 n=487 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.780+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.780+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ca82ed30:::obj135:head local-lis/les=113/114 n=487 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:cb1a5635:::obj233:head, version: 111'233, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb1a5635:::obj233:head@111'233, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.780+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ca82ed30:::obj135:head local-lis/les=113/114 n=487 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:cb1a5635:::obj233:head@111'233, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.780+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.780+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:cadaa6e3:::obj220:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.780+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ca82ed30:::obj135:head local-lis/les=113/114 n=487 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:cb1a5635:::obj233:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.780+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ca82ed30:::obj135:head local-lis/les=113/114 n=487 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:cb1a5635:::obj233:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.780+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ca82ed30:::obj135:head local-lis/les=113/114 n=487 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:cb1a5635:::obj233:head 2026-02-11T23:31:27.780+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ca82ed30:::obj135:head local-lis/les=113/114 n=487 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:cb1a5635:::obj233:head v 111'233 2026-02-11T23:31:27.780+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ca82ed30:::obj135:head local-lis/les=113/114 n=487 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.780+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.780+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:cb1a5635:::obj233:head, version: 111'233, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb1a5635:::obj233:head@111'233, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:27.780+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.780+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.780+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:cadaa6e3:::obj220:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.780+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:cadaa6e3:::obj220:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 116/116 lb 1:cadaa6e3:::obj220:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:ca82ed30:::obj135:head local-lis/les=113/114 n=487 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.780+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.780+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ca82ed30:::obj135:head local-lis/les=113/114 n=487 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:cadaa6e3:::obj220:head) v3 2026-02-11T23:31:27.780+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ca82ed30:::obj135:head local-lis/les=113/114 n=487 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:cadaa6e3:::obj220:head) 2026-02-11T23:31:27.780+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.780+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:cadaa6e3:::obj220:head) v3 finish latency 0.000009 2026-02-11T23:31:27.780+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cadaa6e3:::obj220:head local-lis/les=113/114 n=488 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.780+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.780+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.780+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cadaa6e3:::obj220:head local-lis/les=113/114 n=488 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.780+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.780+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.780+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.780+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6004180 con 0x4acb000 2026-02-11T23:31:27.780+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:cb1a5635:::obj233:head)]) -- 0x5fd3080 con 0x4acb000 2026-02-11T23:31:27.780+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.780+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1013 ==== MOSDPGPush(1.0 116/113 [PushOp(1:cb1be4ca:::obj406:head, version: 111'406, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb1be4ca:::obj406:head@111'406, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d9520 con 0x4acb000 2026-02-11T23:31:27.780+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:cb1be4ca:::obj406:head, version: 111'406, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb1be4ca:::obj406:head@111'406, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.780+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:cb1be4ca:::obj406:head, version: 111'406, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb1be4ca:::obj406:head@111'406, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.780+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.780+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1014 ==== pg_backfill(progress 1.0 e 116/116 lb 1:cb1a5635:::obj233:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176b00 con 0x4acb000 2026-02-11T23:31:27.780+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:cb1be4ca:::obj406:head, version: 111'406, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb1be4ca:::obj406:head@111'406, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.780+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:cb1a5635:::obj233:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.780+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.780+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:cb1a5635:::obj233:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.780+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:cb1be4ca:::obj406:head, version: 111'406, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb1be4ca:::obj406:head@111'406, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.780+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.780+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:cb1be4ca:::obj406:head, version: 111'406, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb1be4ca:::obj406:head@111'406, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:cb1be4ca:::obj406:head, version: 111'406, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb1be4ca:::obj406:head@111'406, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:cadaa6e3:::obj220:head local-lis/les=113/114 n=488 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.780+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.780+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cadaa6e3:::obj220:head local-lis/les=113/114 n=488 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:cb1be4ca:::obj406:head, version: 111'406, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb1be4ca:::obj406:head@111'406, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.780+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cadaa6e3:::obj220:head local-lis/les=113/114 n=488 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:cb1be4ca:::obj406:head@111'406, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.780+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.780+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:cb1a5635:::obj233:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.780+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cadaa6e3:::obj220:head local-lis/les=113/114 n=488 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:cb1be4ca:::obj406:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.780+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cadaa6e3:::obj220:head local-lis/les=113/114 n=488 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:cb1be4ca:::obj406:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.780+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cadaa6e3:::obj220:head local-lis/les=113/114 n=488 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:cb1be4ca:::obj406:head 2026-02-11T23:31:27.780+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cadaa6e3:::obj220:head local-lis/les=113/114 n=488 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:cb1be4ca:::obj406:head v 111'406 2026-02-11T23:31:27.780+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cadaa6e3:::obj220:head local-lis/les=113/114 n=488 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.780+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.780+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:cb1be4ca:::obj406:head, version: 111'406, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb1be4ca:::obj406:head@111'406, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:27.780+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.780+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.780+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:cb1a5635:::obj233:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.780+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:cb1a5635:::obj233:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 1.0 e 116/116 lb 1:cb1a5635:::obj233:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:cadaa6e3:::obj220:head local-lis/les=113/114 n=488 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.780+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.780+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cadaa6e3:::obj220:head local-lis/les=113/114 n=488 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:cb1a5635:::obj233:head) v3 2026-02-11T23:31:27.780+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cadaa6e3:::obj220:head local-lis/les=113/114 n=488 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:cb1a5635:::obj233:head) 2026-02-11T23:31:27.780+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.780+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:cb1a5635:::obj233:head) v3 finish latency 0.000009 2026-02-11T23:31:27.780+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cb1a5635:::obj233:head local-lis/les=113/114 n=489 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.780+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.780+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.780+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cb1a5635:::obj233:head local-lis/les=113/114 n=489 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.780+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.780+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.780+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.780+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6171380 con 0x4acb000 2026-02-11T23:31:27.780+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:cb1be4ca:::obj406:head)]) -- 0x5fd2f00 con 0x4acb000 2026-02-11T23:31:27.780+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.781+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1015 ==== MOSDPGPush(1.0 116/113 [PushOp(1:cb4c4ca4:::obj480:head, version: 111'480, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb4c4ca4:::obj480:head@111'480, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56cb1e0 con 0x4acb000 2026-02-11T23:31:27.781+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:cb4c4ca4:::obj480:head, version: 111'480, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb4c4ca4:::obj480:head@111'480, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.781+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:cb4c4ca4:::obj480:head, version: 111'480, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb4c4ca4:::obj480:head@111'480, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.781+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.781+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1016 ==== pg_backfill(progress 1.0 e 116/116 lb 1:cb1be4ca:::obj406:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177080 con 0x4acb000 2026-02-11T23:31:27.781+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:cb1be4ca:::obj406:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.781+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:cb4c4ca4:::obj480:head, version: 111'480, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb4c4ca4:::obj480:head@111'480, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.781+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:cb1be4ca:::obj406:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.781+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.781+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:cb4c4ca4:::obj480:head, version: 111'480, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb4c4ca4:::obj480:head@111'480, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.781+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.781+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:cb4c4ca4:::obj480:head, version: 111'480, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb4c4ca4:::obj480:head@111'480, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 116/113 [PushOp(1:cb4c4ca4:::obj480:head, version: 111'480, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb4c4ca4:::obj480:head@111'480, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:cb1a5635:::obj233:head local-lis/les=113/114 n=489 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.781+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.781+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cb1a5635:::obj233:head local-lis/les=113/114 n=489 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:cb4c4ca4:::obj480:head, version: 111'480, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb4c4ca4:::obj480:head@111'480, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.781+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cb1a5635:::obj233:head local-lis/les=113/114 n=489 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:cb4c4ca4:::obj480:head@111'480, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.781+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.781+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:cb1be4ca:::obj406:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.781+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cb1a5635:::obj233:head local-lis/les=113/114 n=489 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:cb4c4ca4:::obj480:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.781+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cb1a5635:::obj233:head local-lis/les=113/114 n=489 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:cb4c4ca4:::obj480:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.781+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cb1a5635:::obj233:head local-lis/les=113/114 n=489 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:cb4c4ca4:::obj480:head 2026-02-11T23:31:27.781+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cb1a5635:::obj233:head local-lis/les=113/114 n=489 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:cb4c4ca4:::obj480:head v 111'480 2026-02-11T23:31:27.781+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cb1a5635:::obj233:head local-lis/les=113/114 n=489 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.781+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.781+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:cb4c4ca4:::obj480:head, version: 111'480, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb4c4ca4:::obj480:head@111'480, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.781+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.781+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.781+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:cb1be4ca:::obj406:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.781+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:cb1be4ca:::obj406:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 1.0 e 116/116 lb 1:cb1be4ca:::obj406:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:cb1a5635:::obj233:head local-lis/les=113/114 n=489 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.781+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.781+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cb1a5635:::obj233:head local-lis/les=113/114 n=489 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:cb1be4ca:::obj406:head) v3 2026-02-11T23:31:27.781+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cb1a5635:::obj233:head local-lis/les=113/114 n=489 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:cb1be4ca:::obj406:head) 2026-02-11T23:31:27.781+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.781+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:cb1be4ca:::obj406:head) v3 finish latency 0.000009 2026-02-11T23:31:27.781+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cb1be4ca:::obj406:head local-lis/les=113/114 n=490 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.781+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.781+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.781+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cb1be4ca:::obj406:head local-lis/les=113/114 n=490 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.781+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.781+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.781+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.781+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x615ca80 con 0x4acb000 2026-02-11T23:31:27.781+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:cb4c4ca4:::obj480:head)]) -- 0x5fd2c00 con 0x4acb000 2026-02-11T23:31:27.781+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.781+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1017 ==== MOSDPGPush(1.0 116/113 [PushOp(1:cb843a8e:::obj34:head, version: 111'34, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb843a8e:::obj34:head@111'34, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x55d9860 con 0x4acb000 2026-02-11T23:31:27.781+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:cb843a8e:::obj34:head, version: 111'34, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb843a8e:::obj34:head@111'34, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.781+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:cb843a8e:::obj34:head, version: 111'34, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb843a8e:::obj34:head@111'34, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.781+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.781+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1018 ==== pg_backfill(progress 1.0 e 116/116 lb 1:cb4c4ca4:::obj480:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177600 con 0x4acb000 2026-02-11T23:31:27.781+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:cb843a8e:::obj34:head, version: 111'34, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb843a8e:::obj34:head@111'34, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.781+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:cb4c4ca4:::obj480:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 116 2026-02-11T23:31:27.781+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:cb4c4ca4:::obj480:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.781+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.781+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:cb843a8e:::obj34:head, version: 111'34, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb843a8e:::obj34:head@111'34, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.781+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.781+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:cb843a8e:::obj34:head, version: 111'34, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb843a8e:::obj34:head@111'34, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 116/113 [PushOp(1:cb843a8e:::obj34:head, version: 111'34, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb843a8e:::obj34:head@111'34, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:cb1be4ca:::obj406:head local-lis/les=113/114 n=490 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.781+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.781+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cb1be4ca:::obj406:head local-lis/les=113/114 n=490 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:cb843a8e:::obj34:head, version: 111'34, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb843a8e:::obj34:head@111'34, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.781+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.781+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cb1be4ca:::obj406:head local-lis/les=113/114 n=490 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:cb843a8e:::obj34:head@111'34, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.781+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:cb4c4ca4:::obj480:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.781+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cb1be4ca:::obj406:head local-lis/les=113/114 n=490 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:cb843a8e:::obj34:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.781+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cb1be4ca:::obj406:head local-lis/les=113/114 n=490 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:cb843a8e:::obj34:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.781+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cb1be4ca:::obj406:head local-lis/les=113/114 n=490 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:cb843a8e:::obj34:head 2026-02-11T23:31:27.781+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cb1be4ca:::obj406:head local-lis/les=113/114 n=490 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:cb843a8e:::obj34:head v 111'34 2026-02-11T23:31:27.781+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cb1be4ca:::obj406:head local-lis/les=113/114 n=490 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.781+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.781+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:cb843a8e:::obj34:head, version: 111'34, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb843a8e:::obj34:head@111'34, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:27.781+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cb1be4ca:::obj406:head local-lis/les=113/114 n=490 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.781+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.781+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.781+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:cb4c4ca4:::obj480:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.781+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:cb4c4ca4:::obj480:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 1.0 e 116/116 lb 1:cb4c4ca4:::obj480:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:cb1be4ca:::obj406:head local-lis/les=113/114 n=490 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.781+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.781+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cb1be4ca:::obj406:head local-lis/les=113/114 n=490 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:cb4c4ca4:::obj480:head) v3 2026-02-11T23:31:27.781+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cb1be4ca:::obj406:head local-lis/les=113/114 n=490 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:cb4c4ca4:::obj480:head) 2026-02-11T23:31:27.781+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.781+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:cb4c4ca4:::obj480:head) v3 finish latency 0.000011 2026-02-11T23:31:27.781+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.782+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.782+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cb4c4ca4:::obj480:head local-lis/les=113/114 n=491 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.782+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.782+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.782+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.782+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fd2c00 con 0x4acb000 2026-02-11T23:31:27.782+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:cb843a8e:::obj34:head)]) -- 0x6170600 con 0x4acb000 2026-02-11T23:31:27.782+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.782+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1019 ==== MOSDPGPush(1.0 116/113 [PushOp(1:cbce4a0b:::obj95:head, version: 111'95, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cbce4a0b:::obj95:head@111'95, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x55d8820 con 0x4acb000 2026-02-11T23:31:27.782+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:cbce4a0b:::obj95:head, version: 111'95, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cbce4a0b:::obj95:head@111'95, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.782+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:cbce4a0b:::obj95:head, version: 111'95, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cbce4a0b:::obj95:head@111'95, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.782+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.782+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1020 ==== pg_backfill(progress 1.0 e 116/116 lb 1:cb843a8e:::obj34:head) ==== 1061+0+0 (unknown 0 0 0) 0x6177b80 con 0x4acb000 2026-02-11T23:31:27.782+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:cb843a8e:::obj34:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.782+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:cbce4a0b:::obj95:head, version: 111'95, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cbce4a0b:::obj95:head@111'95, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.782+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:cb843a8e:::obj34:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.782+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.782+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:cbce4a0b:::obj95:head, version: 111'95, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cbce4a0b:::obj95:head@111'95, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.782+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:cbce4a0b:::obj95:head, version: 111'95, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cbce4a0b:::obj95:head@111'95, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 116/113 [PushOp(1:cbce4a0b:::obj95:head, version: 111'95, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cbce4a0b:::obj95:head@111'95, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:cb4c4ca4:::obj480:head local-lis/les=113/114 n=491 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.782+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.782+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.782+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cb4c4ca4:::obj480:head local-lis/les=113/114 n=491 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:cbce4a0b:::obj95:head, version: 111'95, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cbce4a0b:::obj95:head@111'95, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.782+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:cb843a8e:::obj34:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.782+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cb4c4ca4:::obj480:head local-lis/les=113/114 n=491 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:cbce4a0b:::obj95:head@111'95, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.782+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cb4c4ca4:::obj480:head local-lis/les=113/114 n=491 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:cbce4a0b:::obj95:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.782+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cb4c4ca4:::obj480:head local-lis/les=113/114 n=491 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:cbce4a0b:::obj95:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.782+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cb4c4ca4:::obj480:head local-lis/les=113/114 n=491 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:cbce4a0b:::obj95:head 2026-02-11T23:31:27.782+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cb4c4ca4:::obj480:head local-lis/les=113/114 n=491 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:cbce4a0b:::obj95:head v 111'95 2026-02-11T23:31:27.782+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cb4c4ca4:::obj480:head local-lis/les=113/114 n=491 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.782+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.782+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:cbce4a0b:::obj95:head, version: 111'95, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cbce4a0b:::obj95:head@111'95, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.782+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.782+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.782+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:cb843a8e:::obj34:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.782+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:cb843a8e:::obj34:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 1.0 e 116/116 lb 1:cb843a8e:::obj34:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:cb4c4ca4:::obj480:head local-lis/les=113/114 n=491 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.782+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.782+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cb4c4ca4:::obj480:head local-lis/les=113/114 n=491 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:cb843a8e:::obj34:head) v3 2026-02-11T23:31:27.782+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cb4c4ca4:::obj480:head local-lis/les=113/114 n=491 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:cb843a8e:::obj34:head) 2026-02-11T23:31:27.782+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.782+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:cb843a8e:::obj34:head) v3 finish latency 0.000012 2026-02-11T23:31:27.782+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cb843a8e:::obj34:head local-lis/les=113/114 n=492 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.782+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.782+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.782+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cb843a8e:::obj34:head local-lis/les=113/114 n=492 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.782+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.782+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.782+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.782+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x615ca80 con 0x4acb000 2026-02-11T23:31:27.782+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:cbce4a0b:::obj95:head)]) -- 0x5fd2a80 con 0x4acb000 2026-02-11T23:31:27.782+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.782+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1021 ==== MOSDPGPush(1.0 116/113 [PushOp(1:cc461e5d:::obj45:head, version: 111'45, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cc461e5d:::obj45:head@111'45, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x559cea0 con 0x4acb000 2026-02-11T23:31:27.782+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:cc461e5d:::obj45:head, version: 111'45, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cc461e5d:::obj45:head@111'45, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.782+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:cc461e5d:::obj45:head, version: 111'45, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cc461e5d:::obj45:head@111'45, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.782+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.782+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1022 ==== pg_backfill(progress 1.0 e 116/116 lb 1:cbce4a0b:::obj95:head) ==== 1061+0+0 (unknown 0 0 0) 0x6178100 con 0x4acb000 2026-02-11T23:31:27.782+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:cc461e5d:::obj45:head, version: 111'45, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cc461e5d:::obj45:head@111'45, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.782+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:cbce4a0b:::obj95:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.782+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:cbce4a0b:::obj95:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.782+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.782+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:cc461e5d:::obj45:head, version: 111'45, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cc461e5d:::obj45:head@111'45, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.782+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:cc461e5d:::obj45:head, version: 111'45, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cc461e5d:::obj45:head@111'45, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 116/113 [PushOp(1:cc461e5d:::obj45:head, version: 111'45, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cc461e5d:::obj45:head@111'45, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:cb843a8e:::obj34:head local-lis/les=113/114 n=492 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.782+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.782+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.782+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cb843a8e:::obj34:head local-lis/les=113/114 n=492 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:cc461e5d:::obj45:head, version: 111'45, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cc461e5d:::obj45:head@111'45, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.782+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:cbce4a0b:::obj95:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.782+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cb843a8e:::obj34:head local-lis/les=113/114 n=492 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:cc461e5d:::obj45:head@111'45, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.782+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cb843a8e:::obj34:head local-lis/les=113/114 n=492 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:cc461e5d:::obj45:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.782+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cb843a8e:::obj34:head local-lis/les=113/114 n=492 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:cc461e5d:::obj45:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.782+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cb843a8e:::obj34:head local-lis/les=113/114 n=492 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:cc461e5d:::obj45:head 2026-02-11T23:31:27.782+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cb843a8e:::obj34:head local-lis/les=113/114 n=492 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:cc461e5d:::obj45:head v 111'45 2026-02-11T23:31:27.782+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cb843a8e:::obj34:head local-lis/les=113/114 n=492 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.782+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.783+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:cc461e5d:::obj45:head, version: 111'45, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cc461e5d:::obj45:head@111'45, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000064 2026-02-11T23:31:27.783+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.783+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.783+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:cbce4a0b:::obj95:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.783+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:cbce4a0b:::obj95:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 1.0 e 116/116 lb 1:cbce4a0b:::obj95:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:cb843a8e:::obj34:head local-lis/les=113/114 n=492 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.783+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.783+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cb843a8e:::obj34:head local-lis/les=113/114 n=492 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:cbce4a0b:::obj95:head) v3 2026-02-11T23:31:27.783+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cb843a8e:::obj34:head local-lis/les=113/114 n=492 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:cbce4a0b:::obj95:head) 2026-02-11T23:31:27.783+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.783+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:cbce4a0b:::obj95:head) v3 finish latency 0.000029 2026-02-11T23:31:27.783+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cbce4a0b:::obj95:head local-lis/les=113/114 n=493 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.783+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.783+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.783+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cbce4a0b:::obj95:head local-lis/les=113/114 n=493 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.783+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.783+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.783+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.783+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fd2f00 con 0x4acb000 2026-02-11T23:31:27.783+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:cc461e5d:::obj45:head)]) -- 0x5df8a80 con 0x4acb000 2026-02-11T23:31:27.783+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.783+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.783+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.783+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1023 ==== MOSDPGPush(1.0 116/113 [PushOp(1:cdb46b38:::obj588:head, version: 111'588, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cdb46b38:::obj588:head@111'588, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559da00 con 0x4acb000 2026-02-11T23:31:27.783+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:cdb46b38:::obj588:head, version: 111'588, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cdb46b38:::obj588:head@111'588, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.783+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:cdb46b38:::obj588:head, version: 111'588, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cdb46b38:::obj588:head@111'588, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.783+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.783+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1024 ==== pg_backfill(progress 1.0 e 116/116 lb 1:cc461e5d:::obj45:head) ==== 1061+0+0 (unknown 0 0 0) 0x6179180 con 0x4acb000 2026-02-11T23:31:27.783+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:cc461e5d:::obj45:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.783+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:cdb46b38:::obj588:head, version: 111'588, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cdb46b38:::obj588:head@111'588, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.783+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:cc461e5d:::obj45:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.783+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.783+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:cdb46b38:::obj588:head, version: 111'588, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cdb46b38:::obj588:head@111'588, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.783+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:cdb46b38:::obj588:head, version: 111'588, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cdb46b38:::obj588:head@111'588, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 116/113 [PushOp(1:cdb46b38:::obj588:head, version: 111'588, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cdb46b38:::obj588:head@111'588, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:cbce4a0b:::obj95:head local-lis/les=113/114 n=493 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.783+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.783+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cbce4a0b:::obj95:head local-lis/les=113/114 n=493 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:cdb46b38:::obj588:head, version: 111'588, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cdb46b38:::obj588:head@111'588, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.783+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.783+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cbce4a0b:::obj95:head local-lis/les=113/114 n=493 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:cdb46b38:::obj588:head@111'588, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.783+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:cc461e5d:::obj45:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.783+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cbce4a0b:::obj95:head local-lis/les=113/114 n=493 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:cdb46b38:::obj588:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.783+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cbce4a0b:::obj95:head local-lis/les=113/114 n=493 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:cdb46b38:::obj588:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.783+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cbce4a0b:::obj95:head local-lis/les=113/114 n=493 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:cdb46b38:::obj588:head 2026-02-11T23:31:27.783+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cbce4a0b:::obj95:head local-lis/les=113/114 n=493 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:cdb46b38:::obj588:head v 111'588 2026-02-11T23:31:27.783+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cbce4a0b:::obj95:head local-lis/les=113/114 n=493 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.783+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.783+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:cdb46b38:::obj588:head, version: 111'588, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cdb46b38:::obj588:head@111'588, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.783+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cbce4a0b:::obj95:head local-lis/les=113/114 n=493 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.783+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.783+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.783+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:cc461e5d:::obj45:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.783+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:cc461e5d:::obj45:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 1.0 e 116/116 lb 1:cc461e5d:::obj45:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:cbce4a0b:::obj95:head local-lis/les=113/114 n=493 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.783+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.783+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cbce4a0b:::obj95:head local-lis/les=113/114 n=493 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:cc461e5d:::obj45:head) v3 2026-02-11T23:31:27.783+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cbce4a0b:::obj95:head local-lis/les=113/114 n=493 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:cc461e5d:::obj45:head) 2026-02-11T23:31:27.783+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.783+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:cc461e5d:::obj45:head) v3 finish latency 0.000019 2026-02-11T23:31:27.783+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.783+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cc461e5d:::obj45:head local-lis/les=113/114 n=494 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.783+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.783+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.783+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.783+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.783+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6004180 con 0x4acb000 2026-02-11T23:31:27.783+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:cdb46b38:::obj588:head)]) -- 0x6171380 con 0x4acb000 2026-02-11T23:31:27.783+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.783+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1025 ==== MOSDPGPush(1.0 116/113 [PushOp(1:cde49336:::obj263:head, version: 111'263, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cde49336:::obj263:head@111'263, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c340 con 0x4acb000 2026-02-11T23:31:27.783+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:cde49336:::obj263:head, version: 111'263, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cde49336:::obj263:head@111'263, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.783+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:cde49336:::obj263:head, version: 111'263, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cde49336:::obj263:head@111'263, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.783+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.783+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1026 ==== pg_backfill(progress 1.0 e 116/116 lb 1:cdb46b38:::obj588:head) ==== 1062+0+0 (unknown 0 0 0) 0x6178c00 con 0x4acb000 2026-02-11T23:31:27.783+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:cdb46b38:::obj588:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.783+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:cde49336:::obj263:head, version: 111'263, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cde49336:::obj263:head@111'263, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.783+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:cdb46b38:::obj588:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.783+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.783+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:cde49336:::obj263:head, version: 111'263, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cde49336:::obj263:head@111'263, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.783+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:cde49336:::obj263:head, version: 111'263, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cde49336:::obj263:head@111'263, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 116/113 [PushOp(1:cde49336:::obj263:head, version: 111'263, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cde49336:::obj263:head@111'263, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:cc461e5d:::obj45:head local-lis/les=113/114 n=494 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.783+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.783+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cc461e5d:::obj45:head local-lis/les=113/114 n=494 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:cde49336:::obj263:head, version: 111'263, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cde49336:::obj263:head@111'263, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.783+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.783+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cc461e5d:::obj45:head local-lis/les=113/114 n=494 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:cde49336:::obj263:head@111'263, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.783+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:cdb46b38:::obj588:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.783+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cc461e5d:::obj45:head local-lis/les=113/114 n=494 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:cde49336:::obj263:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.783+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cc461e5d:::obj45:head local-lis/les=113/114 n=494 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:cde49336:::obj263:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.783+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cc461e5d:::obj45:head local-lis/les=113/114 n=494 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:cde49336:::obj263:head 2026-02-11T23:31:27.783+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cc461e5d:::obj45:head local-lis/les=113/114 n=494 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:cde49336:::obj263:head v 111'263 2026-02-11T23:31:27.783+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cc461e5d:::obj45:head local-lis/les=113/114 n=494 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.783+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.783+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:cde49336:::obj263:head, version: 111'263, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cde49336:::obj263:head@111'263, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:27.783+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cc461e5d:::obj45:head local-lis/les=113/114 n=494 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.783+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.783+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.783+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:cdb46b38:::obj588:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.783+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:cdb46b38:::obj588:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 1.0 e 116/116 lb 1:cdb46b38:::obj588:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:cc461e5d:::obj45:head local-lis/les=113/114 n=494 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.783+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.783+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cc461e5d:::obj45:head local-lis/les=113/114 n=494 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:cdb46b38:::obj588:head) v3 2026-02-11T23:31:27.783+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cc461e5d:::obj45:head local-lis/les=113/114 n=494 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:cdb46b38:::obj588:head) 2026-02-11T23:31:27.783+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.783+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:cdb46b38:::obj588:head) v3 finish latency 0.000015 2026-02-11T23:31:27.783+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.784+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.784+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cdb46b38:::obj588:head local-lis/les=113/114 n=495 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.784+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.784+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.784+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.784+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fd3380 con 0x4acb000 2026-02-11T23:31:27.784+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:cde49336:::obj263:head)]) -- 0x5d78d80 con 0x4acb000 2026-02-11T23:31:27.784+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.784+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1027 ==== MOSDPGPush(1.0 116/113 [PushOp(1:cf1e3413:::obj224:head, version: 111'224, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cf1e3413:::obj224:head@111'224, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3815380 con 0x4acb000 2026-02-11T23:31:27.784+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:cf1e3413:::obj224:head, version: 111'224, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cf1e3413:::obj224:head@111'224, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.784+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:cf1e3413:::obj224:head, version: 111'224, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cf1e3413:::obj224:head@111'224, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.784+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.784+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1028 ==== pg_backfill(progress 1.0 e 116/116 lb 1:cde49336:::obj263:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2f700 con 0x4acb000 2026-02-11T23:31:27.784+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:cde49336:::obj263:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.784+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:cf1e3413:::obj224:head, version: 111'224, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cf1e3413:::obj224:head@111'224, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.784+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:cde49336:::obj263:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.784+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.784+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:cf1e3413:::obj224:head, version: 111'224, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cf1e3413:::obj224:head@111'224, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.784+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:cf1e3413:::obj224:head, version: 111'224, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cf1e3413:::obj224:head@111'224, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 116/113 [PushOp(1:cf1e3413:::obj224:head, version: 111'224, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cf1e3413:::obj224:head@111'224, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:cdb46b38:::obj588:head local-lis/les=113/114 n=495 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.784+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.784+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.784+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cdb46b38:::obj588:head local-lis/les=113/114 n=495 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:cf1e3413:::obj224:head, version: 111'224, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cf1e3413:::obj224:head@111'224, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.784+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:cde49336:::obj263:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.784+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cdb46b38:::obj588:head local-lis/les=113/114 n=495 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:cf1e3413:::obj224:head@111'224, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.784+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cdb46b38:::obj588:head local-lis/les=113/114 n=495 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:cf1e3413:::obj224:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.784+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cdb46b38:::obj588:head local-lis/les=113/114 n=495 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:cf1e3413:::obj224:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.784+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cdb46b38:::obj588:head local-lis/les=113/114 n=495 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:cf1e3413:::obj224:head 2026-02-11T23:31:27.784+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cdb46b38:::obj588:head local-lis/les=113/114 n=495 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:cf1e3413:::obj224:head v 111'224 2026-02-11T23:31:27.784+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cdb46b38:::obj588:head local-lis/les=113/114 n=495 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.784+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.784+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:cf1e3413:::obj224:head, version: 111'224, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cf1e3413:::obj224:head@111'224, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:27.784+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.784+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.784+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:cde49336:::obj263:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.784+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:cde49336:::obj263:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 1.0 e 116/116 lb 1:cde49336:::obj263:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:cdb46b38:::obj588:head local-lis/les=113/114 n=495 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.784+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.784+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cdb46b38:::obj588:head local-lis/les=113/114 n=495 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:cde49336:::obj263:head) v3 2026-02-11T23:31:27.784+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cdb46b38:::obj588:head local-lis/les=113/114 n=495 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:cde49336:::obj263:head) 2026-02-11T23:31:27.784+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.784+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:cde49336:::obj263:head) v3 finish latency 0.000012 2026-02-11T23:31:27.784+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cde49336:::obj263:head local-lis/les=113/114 n=496 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.784+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.784+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.784+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cde49336:::obj263:head local-lis/les=113/114 n=496 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.784+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.784+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.784+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.784+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x615c480 con 0x4acb000 2026-02-11T23:31:27.784+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:cf1e3413:::obj224:head)]) -- 0x5fd2780 con 0x4acb000 2026-02-11T23:31:27.784+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.785+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1029 ==== MOSDPGPush(1.0 116/113 [PushOp(1:d05ea685:::obj543:head, version: 111'543, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d05ea685:::obj543:head@111'543, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3814d00 con 0x4acb000 2026-02-11T23:31:27.785+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:d05ea685:::obj543:head, version: 111'543, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d05ea685:::obj543:head@111'543, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.785+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d05ea685:::obj543:head, version: 111'543, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d05ea685:::obj543:head@111'543, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.785+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.785+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1030 ==== pg_backfill(progress 1.0 e 116/116 lb 1:cf1e3413:::obj224:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2d080 con 0x4acb000 2026-02-11T23:31:27.785+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:cf1e3413:::obj224:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.785+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d05ea685:::obj543:head, version: 111'543, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d05ea685:::obj543:head@111'543, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.785+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:cf1e3413:::obj224:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.785+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.785+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d05ea685:::obj543:head, version: 111'543, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d05ea685:::obj543:head@111'543, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.785+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:d05ea685:::obj543:head, version: 111'543, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d05ea685:::obj543:head@111'543, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:d05ea685:::obj543:head, version: 111'543, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d05ea685:::obj543:head@111'543, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:cde49336:::obj263:head local-lis/les=113/114 n=496 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.785+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.785+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.785+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cde49336:::obj263:head local-lis/les=113/114 n=496 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:d05ea685:::obj543:head, version: 111'543, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d05ea685:::obj543:head@111'543, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.785+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:cf1e3413:::obj224:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.785+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cde49336:::obj263:head local-lis/les=113/114 n=496 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:d05ea685:::obj543:head@111'543, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.785+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cde49336:::obj263:head local-lis/les=113/114 n=496 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:d05ea685:::obj543:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.785+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cde49336:::obj263:head local-lis/les=113/114 n=496 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:d05ea685:::obj543:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.785+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cde49336:::obj263:head local-lis/les=113/114 n=496 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:d05ea685:::obj543:head 2026-02-11T23:31:27.785+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cde49336:::obj263:head local-lis/les=113/114 n=496 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:d05ea685:::obj543:head v 111'543 2026-02-11T23:31:27.785+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cde49336:::obj263:head local-lis/les=113/114 n=496 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.785+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.785+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:d05ea685:::obj543:head, version: 111'543, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d05ea685:::obj543:head@111'543, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:27.785+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.785+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.785+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:cf1e3413:::obj224:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.785+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:cf1e3413:::obj224:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 1.0 e 116/116 lb 1:cf1e3413:::obj224:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:cde49336:::obj263:head local-lis/les=113/114 n=496 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.785+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.785+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cde49336:::obj263:head local-lis/les=113/114 n=496 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:cf1e3413:::obj224:head) v3 2026-02-11T23:31:27.785+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cde49336:::obj263:head local-lis/les=113/114 n=496 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:cf1e3413:::obj224:head) 2026-02-11T23:31:27.785+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.785+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:cf1e3413:::obj224:head) v3 finish latency 0.000011 2026-02-11T23:31:27.785+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cf1e3413:::obj224:head local-lis/les=113/114 n=497 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.785+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.785+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.785+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cf1e3413:::obj224:head local-lis/les=113/114 n=497 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.785+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.785+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.785+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.785+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d8e900 con 0x4acb000 2026-02-11T23:31:27.785+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:d05ea685:::obj543:head)]) -- 0x5fd2480 con 0x4acb000 2026-02-11T23:31:27.785+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.785+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1031 ==== MOSDPGPush(1.0 116/113 [PushOp(1:d06c69cb:::obj532:head, version: 111'532, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d06c69cb:::obj532:head@111'532, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e261a0 con 0x4acb000 2026-02-11T23:31:27.785+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:d06c69cb:::obj532:head, version: 111'532, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d06c69cb:::obj532:head@111'532, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.785+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d06c69cb:::obj532:head, version: 111'532, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d06c69cb:::obj532:head@111'532, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.785+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.785+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1032 ==== pg_backfill(progress 1.0 e 116/116 lb 1:d05ea685:::obj543:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e680 con 0x4acb000 2026-02-11T23:31:27.785+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d06c69cb:::obj532:head, version: 111'532, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d06c69cb:::obj532:head@111'532, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.785+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:d05ea685:::obj543:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.785+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.785+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d05ea685:::obj543:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.785+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d06c69cb:::obj532:head, version: 111'532, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d06c69cb:::obj532:head@111'532, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.785+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.785+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:d06c69cb:::obj532:head, version: 111'532, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d06c69cb:::obj532:head@111'532, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:d06c69cb:::obj532:head, version: 111'532, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d06c69cb:::obj532:head@111'532, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:cf1e3413:::obj224:head local-lis/les=113/114 n=497 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.785+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.785+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cf1e3413:::obj224:head local-lis/les=113/114 n=497 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:d06c69cb:::obj532:head, version: 111'532, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d06c69cb:::obj532:head@111'532, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.785+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cf1e3413:::obj224:head local-lis/les=113/114 n=497 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:d06c69cb:::obj532:head@111'532, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.785+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.785+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d05ea685:::obj543:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.785+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cf1e3413:::obj224:head local-lis/les=113/114 n=497 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:d06c69cb:::obj532:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.785+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cf1e3413:::obj224:head local-lis/les=113/114 n=497 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:d06c69cb:::obj532:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.785+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cf1e3413:::obj224:head local-lis/les=113/114 n=497 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:d06c69cb:::obj532:head 2026-02-11T23:31:27.785+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cf1e3413:::obj224:head local-lis/les=113/114 n=497 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:d06c69cb:::obj532:head v 111'532 2026-02-11T23:31:27.785+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cf1e3413:::obj224:head local-lis/les=113/114 n=497 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.785+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.785+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:d06c69cb:::obj532:head, version: 111'532, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d06c69cb:::obj532:head@111'532, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.785+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.785+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.785+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d05ea685:::obj543:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.785+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:d05ea685:::obj543:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:d05ea685:::obj543:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:cf1e3413:::obj224:head local-lis/les=113/114 n=497 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.785+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.785+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cf1e3413:::obj224:head local-lis/les=113/114 n=497 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:d05ea685:::obj543:head) v3 2026-02-11T23:31:27.785+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:cf1e3413:::obj224:head local-lis/les=113/114 n=497 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:d05ea685:::obj543:head) 2026-02-11T23:31:27.785+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.785+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:d05ea685:::obj543:head) v3 finish latency 0.000011 2026-02-11T23:31:27.785+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d05ea685:::obj543:head local-lis/les=113/114 n=498 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.785+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.785+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.785+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d05ea685:::obj543:head local-lis/les=113/114 n=498 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.785+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.785+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.785+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.785+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x613f980 con 0x4acb000 2026-02-11T23:31:27.785+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:d06c69cb:::obj532:head)]) -- 0x5fd2180 con 0x4acb000 2026-02-11T23:31:27.785+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.786+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1033 ==== MOSDPGPush(1.0 116/113 [PushOp(1:d0e799f1:::obj299:head, version: 111'299, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d0e799f1:::obj299:head@111'299, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f0ea0 con 0x4acb000 2026-02-11T23:31:27.786+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:d0e799f1:::obj299:head, version: 111'299, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d0e799f1:::obj299:head@111'299, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.786+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d0e799f1:::obj299:head, version: 111'299, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d0e799f1:::obj299:head@111'299, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.786+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.786+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1034 ==== pg_backfill(progress 1.0 e 116/116 lb 1:d06c69cb:::obj532:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e100 con 0x4acb000 2026-02-11T23:31:27.786+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:d06c69cb:::obj532:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.786+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d0e799f1:::obj299:head, version: 111'299, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d0e799f1:::obj299:head@111'299, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.786+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.786+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d0e799f1:::obj299:head, version: 111'299, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d0e799f1:::obj299:head@111'299, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.786+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d06c69cb:::obj532:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.786+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.786+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:d0e799f1:::obj299:head, version: 111'299, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d0e799f1:::obj299:head@111'299, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 116/113 [PushOp(1:d0e799f1:::obj299:head, version: 111'299, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d0e799f1:::obj299:head@111'299, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:d05ea685:::obj543:head local-lis/les=113/114 n=498 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.786+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d06c69cb:::obj532:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.786+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.786+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d05ea685:::obj543:head local-lis/les=113/114 n=498 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:d0e799f1:::obj299:head, version: 111'299, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d0e799f1:::obj299:head@111'299, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.786+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d05ea685:::obj543:head local-lis/les=113/114 n=498 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:d0e799f1:::obj299:head@111'299, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.786+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d05ea685:::obj543:head local-lis/les=113/114 n=498 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:d0e799f1:::obj299:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.786+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d05ea685:::obj543:head local-lis/les=113/114 n=498 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:d0e799f1:::obj299:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.786+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d05ea685:::obj543:head local-lis/les=113/114 n=498 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:d0e799f1:::obj299:head 2026-02-11T23:31:27.786+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d05ea685:::obj543:head local-lis/les=113/114 n=498 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:d0e799f1:::obj299:head v 111'299 2026-02-11T23:31:27.786+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d05ea685:::obj543:head local-lis/les=113/114 n=498 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.786+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.786+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:d0e799f1:::obj299:head, version: 111'299, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d0e799f1:::obj299:head@111'299, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.786+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.786+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.786+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d06c69cb:::obj532:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.786+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:d06c69cb:::obj532:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 1.0 e 116/116 lb 1:d06c69cb:::obj532:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:d05ea685:::obj543:head local-lis/les=113/114 n=498 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.786+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.786+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d05ea685:::obj543:head local-lis/les=113/114 n=498 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:d06c69cb:::obj532:head) v3 2026-02-11T23:31:27.786+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d05ea685:::obj543:head local-lis/les=113/114 n=498 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:d06c69cb:::obj532:head) 2026-02-11T23:31:27.786+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.786+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:d06c69cb:::obj532:head) v3 finish latency 0.000010 2026-02-11T23:31:27.786+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d06c69cb:::obj532:head local-lis/les=113/114 n=499 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.786+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.786+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.786+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d06c69cb:::obj532:head local-lis/les=113/114 n=499 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.786+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.786+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.786+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.786+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fd3500 con 0x4acb000 2026-02-11T23:31:27.786+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:d0e799f1:::obj299:head)]) -- 0x5d1ca80 con 0x4acb000 2026-02-11T23:31:27.786+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.786+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1035 ==== MOSDPGPush(1.0 116/113 [PushOp(1:d0f049f1:::obj387:head, version: 111'387, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d0f049f1:::obj387:head@111'387, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c4e0 con 0x4acb000 2026-02-11T23:31:27.786+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:d0f049f1:::obj387:head, version: 111'387, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d0f049f1:::obj387:head@111'387, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.786+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d0f049f1:::obj387:head, version: 111'387, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d0f049f1:::obj387:head@111'387, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.786+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.786+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1036 ==== pg_backfill(progress 1.0 e 116/116 lb 1:d0e799f1:::obj299:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2c000 con 0x4acb000 2026-02-11T23:31:27.786+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:d0e799f1:::obj299:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.786+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d0f049f1:::obj387:head, version: 111'387, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d0f049f1:::obj387:head@111'387, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.786+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d0e799f1:::obj299:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.786+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.786+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d0f049f1:::obj387:head, version: 111'387, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d0f049f1:::obj387:head@111'387, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.786+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:d0f049f1:::obj387:head, version: 111'387, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d0f049f1:::obj387:head@111'387, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:d0f049f1:::obj387:head, version: 111'387, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d0f049f1:::obj387:head@111'387, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:d06c69cb:::obj532:head local-lis/les=113/114 n=499 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.786+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.786+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d06c69cb:::obj532:head local-lis/les=113/114 n=499 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:d0f049f1:::obj387:head, version: 111'387, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d0f049f1:::obj387:head@111'387, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.786+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.786+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d06c69cb:::obj532:head local-lis/les=113/114 n=499 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:d0f049f1:::obj387:head@111'387, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.786+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d0e799f1:::obj299:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.786+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d06c69cb:::obj532:head local-lis/les=113/114 n=499 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:d0f049f1:::obj387:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.786+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d06c69cb:::obj532:head local-lis/les=113/114 n=499 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:d0f049f1:::obj387:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.786+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d06c69cb:::obj532:head local-lis/les=113/114 n=499 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:d0f049f1:::obj387:head 2026-02-11T23:31:27.786+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d06c69cb:::obj532:head local-lis/les=113/114 n=499 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:d0f049f1:::obj387:head v 111'387 2026-02-11T23:31:27.786+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d06c69cb:::obj532:head local-lis/les=113/114 n=499 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.786+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.786+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:d0f049f1:::obj387:head, version: 111'387, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d0f049f1:::obj387:head@111'387, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.786+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d06c69cb:::obj532:head local-lis/les=113/114 n=499 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.786+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.786+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.786+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d0e799f1:::obj299:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.786+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:d0e799f1:::obj299:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 1.0 e 116/116 lb 1:d0e799f1:::obj299:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:d06c69cb:::obj532:head local-lis/les=113/114 n=499 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.786+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.786+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d06c69cb:::obj532:head local-lis/les=113/114 n=499 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:d0e799f1:::obj299:head) v3 2026-02-11T23:31:27.786+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d06c69cb:::obj532:head local-lis/les=113/114 n=499 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:d0e799f1:::obj299:head) 2026-02-11T23:31:27.786+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.786+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:d0e799f1:::obj299:head) v3 finish latency 0.000012 2026-02-11T23:31:27.786+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.786+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.786+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d0e799f1:::obj299:head local-lis/les=113/114 n=500 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.786+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.786+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.786+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.786+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fcf680 con 0x4acb000 2026-02-11T23:31:27.786+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:d0f049f1:::obj387:head)]) -- 0x5ca1680 con 0x4acb000 2026-02-11T23:31:27.786+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.787+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1037 ==== MOSDPGPush(1.0 116/113 [PushOp(1:d11b9b34:::obj309:head, version: 111'309, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d11b9b34:::obj309:head@111'309, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f16c0 con 0x4acb000 2026-02-11T23:31:27.787+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:d11b9b34:::obj309:head, version: 111'309, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d11b9b34:::obj309:head@111'309, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.787+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d11b9b34:::obj309:head, version: 111'309, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d11b9b34:::obj309:head@111'309, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.787+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.787+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1038 ==== pg_backfill(progress 1.0 e 116/116 lb 1:d0f049f1:::obj387:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2cb00 con 0x4acb000 2026-02-11T23:31:27.787+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:d0f049f1:::obj387:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.787+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d11b9b34:::obj309:head, version: 111'309, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d11b9b34:::obj309:head@111'309, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.787+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d0f049f1:::obj387:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.787+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.787+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d11b9b34:::obj309:head, version: 111'309, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d11b9b34:::obj309:head@111'309, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.787+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.787+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:d11b9b34:::obj309:head, version: 111'309, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d11b9b34:::obj309:head@111'309, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(1.0 116/113 [PushOp(1:d11b9b34:::obj309:head, version: 111'309, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d11b9b34:::obj309:head@111'309, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:d0e799f1:::obj299:head local-lis/les=113/114 n=500 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.787+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.787+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d0e799f1:::obj299:head local-lis/les=113/114 n=500 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:d11b9b34:::obj309:head, version: 111'309, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d11b9b34:::obj309:head@111'309, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.787+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d0e799f1:::obj299:head local-lis/les=113/114 n=500 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:d11b9b34:::obj309:head@111'309, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.787+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.787+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d0f049f1:::obj387:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.787+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d0e799f1:::obj299:head local-lis/les=113/114 n=500 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:d11b9b34:::obj309:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.787+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d0e799f1:::obj299:head local-lis/les=113/114 n=500 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:d11b9b34:::obj309:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.787+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d0e799f1:::obj299:head local-lis/les=113/114 n=500 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:d11b9b34:::obj309:head 2026-02-11T23:31:27.787+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d0e799f1:::obj299:head local-lis/les=113/114 n=500 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:d11b9b34:::obj309:head v 111'309 2026-02-11T23:31:27.787+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d0e799f1:::obj299:head local-lis/les=113/114 n=500 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.787+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.787+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:d11b9b34:::obj309:head, version: 111'309, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d11b9b34:::obj309:head@111'309, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.787+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.787+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.787+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d0f049f1:::obj387:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.787+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:d0f049f1:::obj387:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 1.0 e 116/116 lb 1:d0f049f1:::obj387:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:d0e799f1:::obj299:head local-lis/les=113/114 n=500 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.787+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.787+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d0e799f1:::obj299:head local-lis/les=113/114 n=500 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:d0f049f1:::obj387:head) v3 2026-02-11T23:31:27.787+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d0e799f1:::obj299:head local-lis/les=113/114 n=500 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:d0f049f1:::obj387:head) 2026-02-11T23:31:27.787+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.787+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:d0f049f1:::obj387:head) v3 finish latency 0.000011 2026-02-11T23:31:27.787+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d0f049f1:::obj387:head local-lis/les=113/114 n=501 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.787+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.787+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.787+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d0f049f1:::obj387:head local-lis/les=113/114 n=501 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.787+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.787+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.787+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.787+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fd3800 con 0x4acb000 2026-02-11T23:31:27.787+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:d11b9b34:::obj309:head)]) -- 0x5fcfb00 con 0x4acb000 2026-02-11T23:31:27.787+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.787+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1039 ==== MOSDPGPush(1.0 116/113 [PushOp(1:d1337354:::obj10:head, version: 111'10, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d1337354:::obj10:head@111'10, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x559c820 con 0x4acb000 2026-02-11T23:31:27.787+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:d1337354:::obj10:head, version: 111'10, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d1337354:::obj10:head@111'10, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.787+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d1337354:::obj10:head, version: 111'10, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d1337354:::obj10:head@111'10, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.787+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.787+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1040 ==== pg_backfill(progress 1.0 e 116/116 lb 1:d11b9b34:::obj309:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f10000 con 0x4acb000 2026-02-11T23:31:27.787+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d1337354:::obj10:head, version: 111'10, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d1337354:::obj10:head@111'10, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.787+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:d11b9b34:::obj309:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.787+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.787+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d11b9b34:::obj309:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.787+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d1337354:::obj10:head, version: 111'10, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d1337354:::obj10:head@111'10, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.787+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.787+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:d1337354:::obj10:head, version: 111'10, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d1337354:::obj10:head@111'10, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 116/113 [PushOp(1:d1337354:::obj10:head, version: 111'10, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d1337354:::obj10:head@111'10, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:d0f049f1:::obj387:head local-lis/les=113/114 n=501 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.787+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.787+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d0f049f1:::obj387:head local-lis/les=113/114 n=501 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:d1337354:::obj10:head, version: 111'10, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d1337354:::obj10:head@111'10, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.787+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d0f049f1:::obj387:head local-lis/les=113/114 n=501 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:d1337354:::obj10:head@111'10, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.787+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.787+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d11b9b34:::obj309:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.787+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d0f049f1:::obj387:head local-lis/les=113/114 n=501 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:d1337354:::obj10:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.787+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d0f049f1:::obj387:head local-lis/les=113/114 n=501 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:d1337354:::obj10:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.787+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d0f049f1:::obj387:head local-lis/les=113/114 n=501 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:d1337354:::obj10:head 2026-02-11T23:31:27.787+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d0f049f1:::obj387:head local-lis/les=113/114 n=501 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:d1337354:::obj10:head v 111'10 2026-02-11T23:31:27.787+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d0f049f1:::obj387:head local-lis/les=113/114 n=501 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.787+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.787+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:d1337354:::obj10:head, version: 111'10, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d1337354:::obj10:head@111'10, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.787+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.787+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.787+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d11b9b34:::obj309:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.787+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:d11b9b34:::obj309:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 116/116 lb 1:d11b9b34:::obj309:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:d0f049f1:::obj387:head local-lis/les=113/114 n=501 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.787+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.787+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d0f049f1:::obj387:head local-lis/les=113/114 n=501 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:d11b9b34:::obj309:head) v3 2026-02-11T23:31:27.787+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d0f049f1:::obj387:head local-lis/les=113/114 n=501 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:d11b9b34:::obj309:head) 2026-02-11T23:31:27.787+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.787+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:d11b9b34:::obj309:head) v3 finish latency 0.000013 2026-02-11T23:31:27.787+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d11b9b34:::obj309:head local-lis/les=113/114 n=502 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.787+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.787+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.787+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d11b9b34:::obj309:head local-lis/les=113/114 n=502 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.787+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.787+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.787+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.787+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fed080 con 0x4acb000 2026-02-11T23:31:27.787+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:d1337354:::obj10:head)]) -- 0x5fec780 con 0x4acb000 2026-02-11T23:31:27.787+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.788+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1041 ==== MOSDPGPush(1.0 116/113 [PushOp(1:d1c8135f:::obj67:head, version: 111'67, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d1c8135f:::obj67:head@111'67, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x559dd40 con 0x4acb000 2026-02-11T23:31:27.788+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:d1c8135f:::obj67:head, version: 111'67, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d1c8135f:::obj67:head@111'67, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.788+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d1c8135f:::obj67:head, version: 111'67, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d1c8135f:::obj67:head@111'67, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.788+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.788+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1042 ==== pg_backfill(progress 1.0 e 116/116 lb 1:d1337354:::obj10:head) ==== 1061+0+0 (unknown 0 0 0) 0x5f10580 con 0x4acb000 2026-02-11T23:31:27.788+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d1c8135f:::obj67:head, version: 111'67, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d1c8135f:::obj67:head@111'67, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.788+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:d1337354:::obj10:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.788+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d1337354:::obj10:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.788+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.788+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d1c8135f:::obj67:head, version: 111'67, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d1c8135f:::obj67:head@111'67, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.788+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.788+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:d1c8135f:::obj67:head, version: 111'67, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d1c8135f:::obj67:head@111'67, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:d1c8135f:::obj67:head, version: 111'67, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d1c8135f:::obj67:head@111'67, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:d11b9b34:::obj309:head local-lis/les=113/114 n=502 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.788+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.788+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d11b9b34:::obj309:head local-lis/les=113/114 n=502 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:d1c8135f:::obj67:head, version: 111'67, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d1c8135f:::obj67:head@111'67, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.788+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d11b9b34:::obj309:head local-lis/les=113/114 n=502 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:d1c8135f:::obj67:head@111'67, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.788+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.788+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d1337354:::obj10:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.788+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d11b9b34:::obj309:head local-lis/les=113/114 n=502 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:d1c8135f:::obj67:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.788+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d11b9b34:::obj309:head local-lis/les=113/114 n=502 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:d1c8135f:::obj67:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.788+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d11b9b34:::obj309:head local-lis/les=113/114 n=502 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:d1c8135f:::obj67:head 2026-02-11T23:31:27.788+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d11b9b34:::obj309:head local-lis/les=113/114 n=502 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:d1c8135f:::obj67:head v 111'67 2026-02-11T23:31:27.788+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d11b9b34:::obj309:head local-lis/les=113/114 n=502 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.788+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.788+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:d1c8135f:::obj67:head, version: 111'67, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d1c8135f:::obj67:head@111'67, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.788+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.788+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.788+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d1337354:::obj10:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.788+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:d1337354:::obj10:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 1.0 e 116/116 lb 1:d1337354:::obj10:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:d11b9b34:::obj309:head local-lis/les=113/114 n=502 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.788+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.788+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d11b9b34:::obj309:head local-lis/les=113/114 n=502 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:d1337354:::obj10:head) v3 2026-02-11T23:31:27.788+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d11b9b34:::obj309:head local-lis/les=113/114 n=502 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:d1337354:::obj10:head) 2026-02-11T23:31:27.788+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.788+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:d1337354:::obj10:head) v3 finish latency 0.000010 2026-02-11T23:31:27.788+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d1337354:::obj10:head local-lis/les=113/114 n=503 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.788+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.788+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.788+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d1337354:::obj10:head local-lis/les=113/114 n=503 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.788+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.788+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.788+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.788+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fd3b00 con 0x4acb000 2026-02-11T23:31:27.788+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:d1c8135f:::obj67:head)]) -- 0x5e6b980 con 0x4acb000 2026-02-11T23:31:27.788+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.788+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1043 ==== MOSDPGPush(1.0 116/113 [PushOp(1:d33f403b:::obj434:head, version: 111'434, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d33f403b:::obj434:head@111'434, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d8680 con 0x4acb000 2026-02-11T23:31:27.788+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:d33f403b:::obj434:head, version: 111'434, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d33f403b:::obj434:head@111'434, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.788+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d33f403b:::obj434:head, version: 111'434, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d33f403b:::obj434:head@111'434, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.788+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.788+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1044 ==== pg_backfill(progress 1.0 e 116/116 lb 1:d1c8135f:::obj67:head) ==== 1061+0+0 (unknown 0 0 0) 0x5f10b00 con 0x4acb000 2026-02-11T23:31:27.788+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:d1c8135f:::obj67:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.788+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d33f403b:::obj434:head, version: 111'434, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d33f403b:::obj434:head@111'434, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.788+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d1c8135f:::obj67:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.788+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.788+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d33f403b:::obj434:head, version: 111'434, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d33f403b:::obj434:head@111'434, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.788+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:d33f403b:::obj434:head, version: 111'434, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d33f403b:::obj434:head@111'434, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 116/113 [PushOp(1:d33f403b:::obj434:head, version: 111'434, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d33f403b:::obj434:head@111'434, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:d1337354:::obj10:head local-lis/les=113/114 n=503 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.788+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.788+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.788+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d1337354:::obj10:head local-lis/les=113/114 n=503 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:d33f403b:::obj434:head, version: 111'434, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d33f403b:::obj434:head@111'434, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.788+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d1c8135f:::obj67:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.788+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d1337354:::obj10:head local-lis/les=113/114 n=503 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:d33f403b:::obj434:head@111'434, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.788+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d1337354:::obj10:head local-lis/les=113/114 n=503 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:d33f403b:::obj434:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.788+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d1337354:::obj10:head local-lis/les=113/114 n=503 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:d33f403b:::obj434:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.788+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d1337354:::obj10:head local-lis/les=113/114 n=503 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:d33f403b:::obj434:head 2026-02-11T23:31:27.788+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d1337354:::obj10:head local-lis/les=113/114 n=503 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:d33f403b:::obj434:head v 111'434 2026-02-11T23:31:27.788+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d1337354:::obj10:head local-lis/les=113/114 n=503 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.788+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.788+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:d33f403b:::obj434:head, version: 111'434, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d33f403b:::obj434:head@111'434, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.788+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.788+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.788+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d1c8135f:::obj67:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.788+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:d1c8135f:::obj67:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 116/116 lb 1:d1c8135f:::obj67:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:d1337354:::obj10:head local-lis/les=113/114 n=503 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.788+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.788+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d1337354:::obj10:head local-lis/les=113/114 n=503 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:d1c8135f:::obj67:head) v3 2026-02-11T23:31:27.788+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d1337354:::obj10:head local-lis/les=113/114 n=503 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:d1c8135f:::obj67:head) 2026-02-11T23:31:27.788+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.788+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:d1c8135f:::obj67:head) v3 finish latency 0.000009 2026-02-11T23:31:27.788+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d1c8135f:::obj67:head local-lis/les=113/114 n=504 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.788+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.789+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.789+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d1c8135f:::obj67:head local-lis/les=113/114 n=504 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.789+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.789+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.789+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.789+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x600a480 con 0x4acb000 2026-02-11T23:31:27.789+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:d33f403b:::obj434:head)]) -- 0x600be00 con 0x4acb000 2026-02-11T23:31:27.789+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.789+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1045 ==== MOSDPGPush(1.0 116/113 [PushOp(1:d3f6948e:::obj218:head, version: 111'218, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d3f6948e:::obj218:head@111'218, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559d860 con 0x4acb000 2026-02-11T23:31:27.789+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:d3f6948e:::obj218:head, version: 111'218, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d3f6948e:::obj218:head@111'218, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.789+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d3f6948e:::obj218:head, version: 111'218, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d3f6948e:::obj218:head@111'218, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.789+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.789+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1046 ==== pg_backfill(progress 1.0 e 116/116 lb 1:d33f403b:::obj434:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f11080 con 0x4acb000 2026-02-11T23:31:27.789+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:d33f403b:::obj434:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.789+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d3f6948e:::obj218:head, version: 111'218, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d3f6948e:::obj218:head@111'218, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.789+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d33f403b:::obj434:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.789+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.789+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d3f6948e:::obj218:head, version: 111'218, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d3f6948e:::obj218:head@111'218, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.789+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:d3f6948e:::obj218:head, version: 111'218, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d3f6948e:::obj218:head@111'218, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 116/113 [PushOp(1:d3f6948e:::obj218:head, version: 111'218, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d3f6948e:::obj218:head@111'218, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:d1c8135f:::obj67:head local-lis/les=113/114 n=504 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.789+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.789+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.789+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d1c8135f:::obj67:head local-lis/les=113/114 n=504 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:d3f6948e:::obj218:head, version: 111'218, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d3f6948e:::obj218:head@111'218, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.789+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d33f403b:::obj434:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.789+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d1c8135f:::obj67:head local-lis/les=113/114 n=504 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:d3f6948e:::obj218:head@111'218, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.789+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d1c8135f:::obj67:head local-lis/les=113/114 n=504 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:d3f6948e:::obj218:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.789+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d1c8135f:::obj67:head local-lis/les=113/114 n=504 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:d3f6948e:::obj218:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.789+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d1c8135f:::obj67:head local-lis/les=113/114 n=504 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:d3f6948e:::obj218:head 2026-02-11T23:31:27.789+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d1c8135f:::obj67:head local-lis/les=113/114 n=504 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:d3f6948e:::obj218:head v 111'218 2026-02-11T23:31:27.789+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d1c8135f:::obj67:head local-lis/les=113/114 n=504 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.789+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.789+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:d3f6948e:::obj218:head, version: 111'218, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d3f6948e:::obj218:head@111'218, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:27.789+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.789+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.789+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d33f403b:::obj434:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.789+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:d33f403b:::obj434:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 1.0 e 116/116 lb 1:d33f403b:::obj434:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:d1c8135f:::obj67:head local-lis/les=113/114 n=504 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.789+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.789+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d1c8135f:::obj67:head local-lis/les=113/114 n=504 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:d33f403b:::obj434:head) v3 2026-02-11T23:31:27.789+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d1c8135f:::obj67:head local-lis/les=113/114 n=504 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:d33f403b:::obj434:head) 2026-02-11T23:31:27.789+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.789+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:d33f403b:::obj434:head) v3 finish latency 0.000011 2026-02-11T23:31:27.789+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d33f403b:::obj434:head local-lis/les=113/114 n=505 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.789+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.789+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.789+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d33f403b:::obj434:head local-lis/les=113/114 n=505 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.789+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.789+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.789+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.789+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6170300 con 0x4acb000 2026-02-11T23:31:27.789+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:d3f6948e:::obj218:head)]) -- 0x5fec180 con 0x4acb000 2026-02-11T23:31:27.789+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.789+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1047 ==== MOSDPGPush(1.0 116/113 [PushOp(1:d4006106:::obj329:head, version: 111'329, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d4006106:::obj329:head@111'329, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56ca000 con 0x4acb000 2026-02-11T23:31:27.789+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:d4006106:::obj329:head, version: 111'329, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d4006106:::obj329:head@111'329, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 116 2026-02-11T23:31:27.789+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d4006106:::obj329:head, version: 111'329, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d4006106:::obj329:head@111'329, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.789+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.789+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1048 ==== pg_backfill(progress 1.0 e 116/116 lb 1:d3f6948e:::obj218:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f11600 con 0x4acb000 2026-02-11T23:31:27.789+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:d3f6948e:::obj218:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.789+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d4006106:::obj329:head, version: 111'329, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d4006106:::obj329:head@111'329, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.789+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.789+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d3f6948e:::obj218:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.789+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d4006106:::obj329:head, version: 111'329, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d4006106:::obj329:head@111'329, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.789+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.789+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d3f6948e:::obj218:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.789+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:d4006106:::obj329:head, version: 111'329, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d4006106:::obj329:head@111'329, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:d4006106:::obj329:head, version: 111'329, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d4006106:::obj329:head@111'329, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:d33f403b:::obj434:head local-lis/les=113/114 n=505 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.789+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.789+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d33f403b:::obj434:head local-lis/les=113/114 n=505 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:d4006106:::obj329:head, version: 111'329, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d4006106:::obj329:head@111'329, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.789+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d33f403b:::obj434:head local-lis/les=113/114 n=505 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:d4006106:::obj329:head@111'329, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.789+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d33f403b:::obj434:head local-lis/les=113/114 n=505 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:d4006106:::obj329:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.789+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d33f403b:::obj434:head local-lis/les=113/114 n=505 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:d4006106:::obj329:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.789+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d33f403b:::obj434:head local-lis/les=113/114 n=505 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:d4006106:::obj329:head 2026-02-11T23:31:27.789+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d33f403b:::obj434:head local-lis/les=113/114 n=505 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:d4006106:::obj329:head v 111'329 2026-02-11T23:31:27.789+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d33f403b:::obj434:head local-lis/les=113/114 n=505 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.789+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.789+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:d4006106:::obj329:head, version: 111'329, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d4006106:::obj329:head@111'329, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.789+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.789+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.789+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d3f6948e:::obj218:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.789+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:d3f6948e:::obj218:head) v3 prio 15 cost 0 latency 0.000060 pg_backfill(progress 1.0 e 116/116 lb 1:d3f6948e:::obj218:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:d33f403b:::obj434:head local-lis/les=113/114 n=505 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.789+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.789+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d33f403b:::obj434:head local-lis/les=113/114 n=505 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:d3f6948e:::obj218:head) v3 2026-02-11T23:31:27.789+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d33f403b:::obj434:head local-lis/les=113/114 n=505 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:d3f6948e:::obj218:head) 2026-02-11T23:31:27.789+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.789+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:d3f6948e:::obj218:head) v3 finish latency 0.000011 2026-02-11T23:31:27.789+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d3f6948e:::obj218:head local-lis/les=113/114 n=506 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.789+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.790+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.790+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d3f6948e:::obj218:head local-lis/les=113/114 n=506 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.790+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.790+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.790+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.790+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x612ed80 con 0x4acb000 2026-02-11T23:31:27.790+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:d4006106:::obj329:head)]) -- 0x6005380 con 0x4acb000 2026-02-11T23:31:27.790+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.790+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1049 ==== MOSDPGPush(1.0 116/113 [PushOp(1:d43593c2:::obj365:head, version: 111'365, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d43593c2:::obj365:head@111'365, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d96c0 con 0x4acb000 2026-02-11T23:31:27.790+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:d43593c2:::obj365:head, version: 111'365, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d43593c2:::obj365:head@111'365, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.790+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d43593c2:::obj365:head, version: 111'365, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d43593c2:::obj365:head@111'365, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.790+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.790+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1050 ==== pg_backfill(progress 1.0 e 116/116 lb 1:d4006106:::obj329:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f11b80 con 0x4acb000 2026-02-11T23:31:27.790+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d43593c2:::obj365:head, version: 111'365, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d43593c2:::obj365:head@111'365, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.790+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:d4006106:::obj329:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.790+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d4006106:::obj329:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.790+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.790+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d43593c2:::obj365:head, version: 111'365, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d43593c2:::obj365:head@111'365, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.790+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.790+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:d43593c2:::obj365:head, version: 111'365, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d43593c2:::obj365:head@111'365, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 116/113 [PushOp(1:d43593c2:::obj365:head, version: 111'365, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d43593c2:::obj365:head@111'365, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:d3f6948e:::obj218:head local-lis/les=113/114 n=506 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.790+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.790+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d3f6948e:::obj218:head local-lis/les=113/114 n=506 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:d43593c2:::obj365:head, version: 111'365, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d43593c2:::obj365:head@111'365, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.790+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d3f6948e:::obj218:head local-lis/les=113/114 n=506 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:d43593c2:::obj365:head@111'365, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.790+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.790+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d4006106:::obj329:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.790+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d3f6948e:::obj218:head local-lis/les=113/114 n=506 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:d43593c2:::obj365:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.790+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d3f6948e:::obj218:head local-lis/les=113/114 n=506 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:d43593c2:::obj365:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.790+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d3f6948e:::obj218:head local-lis/les=113/114 n=506 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:d43593c2:::obj365:head 2026-02-11T23:31:27.790+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d3f6948e:::obj218:head local-lis/les=113/114 n=506 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:d43593c2:::obj365:head v 111'365 2026-02-11T23:31:27.790+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d3f6948e:::obj218:head local-lis/les=113/114 n=506 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.790+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.790+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:d43593c2:::obj365:head, version: 111'365, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d43593c2:::obj365:head@111'365, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:27.790+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.790+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.790+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d4006106:::obj329:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.790+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:d4006106:::obj329:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 1.0 e 116/116 lb 1:d4006106:::obj329:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:d3f6948e:::obj218:head local-lis/les=113/114 n=506 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.790+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.790+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d3f6948e:::obj218:head local-lis/les=113/114 n=506 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:d4006106:::obj329:head) v3 2026-02-11T23:31:27.790+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d3f6948e:::obj218:head local-lis/les=113/114 n=506 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:d4006106:::obj329:head) 2026-02-11T23:31:27.790+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.790+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:d4006106:::obj329:head) v3 finish latency 0.000012 2026-02-11T23:31:27.790+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d4006106:::obj329:head local-lis/les=113/114 n=507 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.790+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.790+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.790+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d4006106:::obj329:head local-lis/les=113/114 n=507 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.790+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.790+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.790+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.790+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5daa300 con 0x4acb000 2026-02-11T23:31:27.790+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:d43593c2:::obj365:head)]) -- 0x600b680 con 0x4acb000 2026-02-11T23:31:27.790+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.790+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1051 ==== MOSDPGPush(1.0 116/113 [PushOp(1:d4c3bba7:::obj106:head, version: 111'106, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d4c3bba7:::obj106:head@111'106, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e26820 con 0x4acb000 2026-02-11T23:31:27.790+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:d4c3bba7:::obj106:head, version: 111'106, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d4c3bba7:::obj106:head@111'106, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.790+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d4c3bba7:::obj106:head, version: 111'106, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d4c3bba7:::obj106:head@111'106, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.790+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.790+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1052 ==== pg_backfill(progress 1.0 e 116/116 lb 1:d43593c2:::obj365:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f12100 con 0x4acb000 2026-02-11T23:31:27.790+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:d43593c2:::obj365:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.790+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d4c3bba7:::obj106:head, version: 111'106, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d4c3bba7:::obj106:head@111'106, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.790+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d43593c2:::obj365:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.790+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.790+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d43593c2:::obj365:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.790+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.790+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d4c3bba7:::obj106:head, version: 111'106, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d4c3bba7:::obj106:head@111'106, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.790+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:d4c3bba7:::obj106:head, version: 111'106, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d4c3bba7:::obj106:head@111'106, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000042 MOSDPGPush(1.0 116/113 [PushOp(1:d4c3bba7:::obj106:head, version: 111'106, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d4c3bba7:::obj106:head@111'106, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:d4006106:::obj329:head local-lis/les=113/114 n=507 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.790+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.790+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d4006106:::obj329:head local-lis/les=113/114 n=507 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:d4c3bba7:::obj106:head, version: 111'106, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d4c3bba7:::obj106:head@111'106, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.790+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d4006106:::obj329:head local-lis/les=113/114 n=507 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:d4c3bba7:::obj106:head@111'106, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.790+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d4006106:::obj329:head local-lis/les=113/114 n=507 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:d4c3bba7:::obj106:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.790+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d4006106:::obj329:head local-lis/les=113/114 n=507 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:d4c3bba7:::obj106:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.790+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d4006106:::obj329:head local-lis/les=113/114 n=507 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:d4c3bba7:::obj106:head 2026-02-11T23:31:27.790+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d4006106:::obj329:head local-lis/les=113/114 n=507 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:d4c3bba7:::obj106:head v 111'106 2026-02-11T23:31:27.790+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d4006106:::obj329:head local-lis/les=113/114 n=507 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.790+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.790+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:d4c3bba7:::obj106:head, version: 111'106, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d4c3bba7:::obj106:head@111'106, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.790+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.790+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.790+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d43593c2:::obj365:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.790+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:d43593c2:::obj365:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 1.0 e 116/116 lb 1:d43593c2:::obj365:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:d4006106:::obj329:head local-lis/les=113/114 n=507 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.790+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.790+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d4006106:::obj329:head local-lis/les=113/114 n=507 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:d43593c2:::obj365:head) v3 2026-02-11T23:31:27.790+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d4006106:::obj329:head local-lis/les=113/114 n=507 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:d43593c2:::obj365:head) 2026-02-11T23:31:27.790+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.791+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:d43593c2:::obj365:head) v3 finish latency 0.000014 2026-02-11T23:31:27.791+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d43593c2:::obj365:head local-lis/les=113/114 n=508 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.791+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.791+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.791+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d43593c2:::obj365:head local-lis/les=113/114 n=508 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.791+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.791+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.791+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.791+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fcf800 con 0x4acb000 2026-02-11T23:31:27.791+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:d4c3bba7:::obj106:head)]) -- 0x5fed380 con 0x4acb000 2026-02-11T23:31:27.791+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.791+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1053 ==== MOSDPGPush(1.0 116/113 [PushOp(1:d5f69272:::obj407:head, version: 111'407, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d5f69272:::obj407:head@111'407, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d8d00 con 0x4acb000 2026-02-11T23:31:27.791+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:d5f69272:::obj407:head, version: 111'407, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d5f69272:::obj407:head@111'407, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.791+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d5f69272:::obj407:head, version: 111'407, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d5f69272:::obj407:head@111'407, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.791+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.791+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1054 ==== pg_backfill(progress 1.0 e 116/116 lb 1:d4c3bba7:::obj106:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f12680 con 0x4acb000 2026-02-11T23:31:27.791+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d5f69272:::obj407:head, version: 111'407, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d5f69272:::obj407:head@111'407, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.791+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:d4c3bba7:::obj106:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.791+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.791+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d4c3bba7:::obj106:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.791+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d5f69272:::obj407:head, version: 111'407, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d5f69272:::obj407:head@111'407, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.791+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.791+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:d5f69272:::obj407:head, version: 111'407, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d5f69272:::obj407:head@111'407, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:d5f69272:::obj407:head, version: 111'407, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d5f69272:::obj407:head@111'407, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:d43593c2:::obj365:head local-lis/les=113/114 n=508 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.791+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.791+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d43593c2:::obj365:head local-lis/les=113/114 n=508 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:d5f69272:::obj407:head, version: 111'407, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d5f69272:::obj407:head@111'407, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.791+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d43593c2:::obj365:head local-lis/les=113/114 n=508 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:d5f69272:::obj407:head@111'407, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.791+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.791+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d4c3bba7:::obj106:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.791+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d43593c2:::obj365:head local-lis/les=113/114 n=508 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:d5f69272:::obj407:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.791+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d43593c2:::obj365:head local-lis/les=113/114 n=508 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:d5f69272:::obj407:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.791+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d43593c2:::obj365:head local-lis/les=113/114 n=508 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:d5f69272:::obj407:head 2026-02-11T23:31:27.791+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d43593c2:::obj365:head local-lis/les=113/114 n=508 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:d5f69272:::obj407:head v 111'407 2026-02-11T23:31:27.791+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d43593c2:::obj365:head local-lis/les=113/114 n=508 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.791+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.791+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:d5f69272:::obj407:head, version: 111'407, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d5f69272:::obj407:head@111'407, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000042 2026-02-11T23:31:27.791+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.791+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.791+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d4c3bba7:::obj106:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.791+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:d4c3bba7:::obj106:head) v3 prio 15 cost 0 latency 0.000058 pg_backfill(progress 1.0 e 116/116 lb 1:d4c3bba7:::obj106:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:d43593c2:::obj365:head local-lis/les=113/114 n=508 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.791+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.791+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d43593c2:::obj365:head local-lis/les=113/114 n=508 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:d4c3bba7:::obj106:head) v3 2026-02-11T23:31:27.791+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d43593c2:::obj365:head local-lis/les=113/114 n=508 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:d4c3bba7:::obj106:head) 2026-02-11T23:31:27.791+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.791+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:d4c3bba7:::obj106:head) v3 finish latency 0.000010 2026-02-11T23:31:27.791+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d4c3bba7:::obj106:head local-lis/les=113/114 n=509 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.791+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.791+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.791+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d4c3bba7:::obj106:head local-lis/les=113/114 n=509 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.791+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.791+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.791+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.791+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5daa900 con 0x4acb000 2026-02-11T23:31:27.791+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:d5f69272:::obj407:head)]) -- 0x600b500 con 0x4acb000 2026-02-11T23:31:27.791+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.791+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1055 ==== MOSDPGPush(1.0 116/113 [PushOp(1:d6df9426:::obj469:head, version: 111'469, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d6df9426:::obj469:head@111'469, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db0820 con 0x4acb000 2026-02-11T23:31:27.791+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:d6df9426:::obj469:head, version: 111'469, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d6df9426:::obj469:head@111'469, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.791+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d6df9426:::obj469:head, version: 111'469, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d6df9426:::obj469:head@111'469, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.791+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.791+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1056 ==== pg_backfill(progress 1.0 e 116/116 lb 1:d5f69272:::obj407:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f12c00 con 0x4acb000 2026-02-11T23:31:27.791+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:d5f69272:::obj407:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.791+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d6df9426:::obj469:head, version: 111'469, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d6df9426:::obj469:head@111'469, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.791+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d5f69272:::obj407:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.791+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.791+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d6df9426:::obj469:head, version: 111'469, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d6df9426:::obj469:head@111'469, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.791+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:d6df9426:::obj469:head, version: 111'469, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d6df9426:::obj469:head@111'469, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 116/113 [PushOp(1:d6df9426:::obj469:head, version: 111'469, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d6df9426:::obj469:head@111'469, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:d4c3bba7:::obj106:head local-lis/les=113/114 n=509 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.791+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.791+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.791+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d4c3bba7:::obj106:head local-lis/les=113/114 n=509 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:d6df9426:::obj469:head, version: 111'469, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d6df9426:::obj469:head@111'469, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.791+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d5f69272:::obj407:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.791+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d4c3bba7:::obj106:head local-lis/les=113/114 n=509 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:d6df9426:::obj469:head@111'469, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.791+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d4c3bba7:::obj106:head local-lis/les=113/114 n=509 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:d6df9426:::obj469:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.791+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d4c3bba7:::obj106:head local-lis/les=113/114 n=509 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:d6df9426:::obj469:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.791+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d4c3bba7:::obj106:head local-lis/les=113/114 n=509 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:d6df9426:::obj469:head 2026-02-11T23:31:27.791+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d4c3bba7:::obj106:head local-lis/les=113/114 n=509 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:d6df9426:::obj469:head v 111'469 2026-02-11T23:31:27.791+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d4c3bba7:::obj106:head local-lis/les=113/114 n=509 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.791+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.791+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:d6df9426:::obj469:head, version: 111'469, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d6df9426:::obj469:head@111'469, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000044 2026-02-11T23:31:27.791+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.791+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.791+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d5f69272:::obj407:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.791+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:d5f69272:::obj407:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 1.0 e 116/116 lb 1:d5f69272:::obj407:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:d4c3bba7:::obj106:head local-lis/les=113/114 n=509 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.791+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.791+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d4c3bba7:::obj106:head local-lis/les=113/114 n=509 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:d5f69272:::obj407:head) v3 2026-02-11T23:31:27.791+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d4c3bba7:::obj106:head local-lis/les=113/114 n=509 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:d5f69272:::obj407:head) 2026-02-11T23:31:27.791+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.791+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:d5f69272:::obj407:head) v3 finish latency 0.000010 2026-02-11T23:31:27.791+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d5f69272:::obj407:head local-lis/les=113/114 n=510 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.791+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.792+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.792+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d5f69272:::obj407:head local-lis/les=113/114 n=510 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.792+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.792+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.792+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.792+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d1d380 con 0x4acb000 2026-02-11T23:31:27.792+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:d6df9426:::obj469:head)]) -- 0x6015800 con 0x4acb000 2026-02-11T23:31:27.792+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.792+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1057 ==== MOSDPGPush(1.0 116/113 [PushOp(1:d80bb902:::obj418:head, version: 111'418, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d80bb902:::obj418:head@111'418, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d84e0 con 0x4acb000 2026-02-11T23:31:27.792+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:d80bb902:::obj418:head, version: 111'418, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d80bb902:::obj418:head@111'418, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.792+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d80bb902:::obj418:head, version: 111'418, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d80bb902:::obj418:head@111'418, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.792+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.792+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1058 ==== pg_backfill(progress 1.0 e 116/116 lb 1:d6df9426:::obj469:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f13180 con 0x4acb000 2026-02-11T23:31:27.792+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d80bb902:::obj418:head, version: 111'418, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d80bb902:::obj418:head@111'418, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.792+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:d6df9426:::obj469:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.792+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d6df9426:::obj469:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.792+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.792+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d80bb902:::obj418:head, version: 111'418, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d80bb902:::obj418:head@111'418, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.792+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:d80bb902:::obj418:head, version: 111'418, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d80bb902:::obj418:head@111'418, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 116/113 [PushOp(1:d80bb902:::obj418:head, version: 111'418, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d80bb902:::obj418:head@111'418, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:d5f69272:::obj407:head local-lis/les=113/114 n=510 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.792+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.792+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.792+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d6df9426:::obj469:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.792+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d5f69272:::obj407:head local-lis/les=113/114 n=510 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:d80bb902:::obj418:head, version: 111'418, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d80bb902:::obj418:head@111'418, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.792+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d5f69272:::obj407:head local-lis/les=113/114 n=510 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:d80bb902:::obj418:head@111'418, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.792+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d5f69272:::obj407:head local-lis/les=113/114 n=510 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:d80bb902:::obj418:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.792+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d5f69272:::obj407:head local-lis/les=113/114 n=510 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:d80bb902:::obj418:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.792+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d5f69272:::obj407:head local-lis/les=113/114 n=510 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:d80bb902:::obj418:head 2026-02-11T23:31:27.792+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d5f69272:::obj407:head local-lis/les=113/114 n=510 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:d80bb902:::obj418:head v 111'418 2026-02-11T23:31:27.792+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d5f69272:::obj407:head local-lis/les=113/114 n=510 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.792+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.792+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:d80bb902:::obj418:head, version: 111'418, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d80bb902:::obj418:head@111'418, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:27.792+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.792+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.792+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d6df9426:::obj469:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.792+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:d6df9426:::obj469:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 116/116 lb 1:d6df9426:::obj469:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:d5f69272:::obj407:head local-lis/les=113/114 n=510 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.792+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.792+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d5f69272:::obj407:head local-lis/les=113/114 n=510 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:d6df9426:::obj469:head) v3 2026-02-11T23:31:27.792+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d5f69272:::obj407:head local-lis/les=113/114 n=510 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:d6df9426:::obj469:head) 2026-02-11T23:31:27.792+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.792+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:d6df9426:::obj469:head) v3 finish latency 0.000009 2026-02-11T23:31:27.792+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d6df9426:::obj469:head local-lis/les=113/114 n=511 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.792+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.792+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.792+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d6df9426:::obj469:head local-lis/les=113/114 n=511 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.792+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.792+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.792+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.792+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x600a900 con 0x4acb000 2026-02-11T23:31:27.792+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:d80bb902:::obj418:head)]) -- 0x6000180 con 0x4acb000 2026-02-11T23:31:27.792+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.793+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1059 ==== MOSDPGPush(1.0 116/113 [PushOp(1:d83b1cbf:::obj272:head, version: 111'272, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d83b1cbf:::obj272:head@111'272, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d8000 con 0x4acb000 2026-02-11T23:31:27.793+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:d83b1cbf:::obj272:head, version: 111'272, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d83b1cbf:::obj272:head@111'272, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.793+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d83b1cbf:::obj272:head, version: 111'272, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d83b1cbf:::obj272:head@111'272, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.793+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.793+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1060 ==== pg_backfill(progress 1.0 e 116/116 lb 1:d80bb902:::obj418:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f13700 con 0x4acb000 2026-02-11T23:31:27.793+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d83b1cbf:::obj272:head, version: 111'272, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d83b1cbf:::obj272:head@111'272, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.793+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:d80bb902:::obj418:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.793+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.793+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d80bb902:::obj418:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.793+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d83b1cbf:::obj272:head, version: 111'272, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d83b1cbf:::obj272:head@111'272, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.793+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.793+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:d83b1cbf:::obj272:head, version: 111'272, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d83b1cbf:::obj272:head@111'272, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:d83b1cbf:::obj272:head, version: 111'272, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d83b1cbf:::obj272:head@111'272, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:d6df9426:::obj469:head local-lis/les=113/114 n=511 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.793+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.793+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d6df9426:::obj469:head local-lis/les=113/114 n=511 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:d83b1cbf:::obj272:head, version: 111'272, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d83b1cbf:::obj272:head@111'272, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.793+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d6df9426:::obj469:head local-lis/les=113/114 n=511 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:d83b1cbf:::obj272:head@111'272, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.793+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.793+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d80bb902:::obj418:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.793+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d6df9426:::obj469:head local-lis/les=113/114 n=511 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:d83b1cbf:::obj272:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.793+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d6df9426:::obj469:head local-lis/les=113/114 n=511 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:d83b1cbf:::obj272:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.793+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d6df9426:::obj469:head local-lis/les=113/114 n=511 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:d83b1cbf:::obj272:head 2026-02-11T23:31:27.793+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d6df9426:::obj469:head local-lis/les=113/114 n=511 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:d83b1cbf:::obj272:head v 111'272 2026-02-11T23:31:27.793+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d6df9426:::obj469:head local-lis/les=113/114 n=511 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.793+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.793+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:d83b1cbf:::obj272:head, version: 111'272, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d83b1cbf:::obj272:head@111'272, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:27.793+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.793+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.793+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d80bb902:::obj418:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.793+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:d80bb902:::obj418:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:d80bb902:::obj418:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:d6df9426:::obj469:head local-lis/les=113/114 n=511 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.793+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.793+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d6df9426:::obj469:head local-lis/les=113/114 n=511 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:d80bb902:::obj418:head) v3 2026-02-11T23:31:27.793+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d6df9426:::obj469:head local-lis/les=113/114 n=511 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:d80bb902:::obj418:head) 2026-02-11T23:31:27.793+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.793+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:d80bb902:::obj418:head) v3 finish latency 0.000010 2026-02-11T23:31:27.793+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d80bb902:::obj418:head local-lis/les=113/114 n=512 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.793+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.793+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.793+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d80bb902:::obj418:head local-lis/les=113/114 n=512 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.793+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.793+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.793+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.793+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6171680 con 0x4acb000 2026-02-11T23:31:27.793+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:d83b1cbf:::obj272:head)]) -- 0x612e900 con 0x4acb000 2026-02-11T23:31:27.793+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.793+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1061 ==== MOSDPGPush(1.0 116/113 [PushOp(1:d85a09a6:::obj441:head, version: 111'441, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d85a09a6:::obj441:head@111'441, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d89c0 con 0x4acb000 2026-02-11T23:31:27.793+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:d85a09a6:::obj441:head, version: 111'441, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d85a09a6:::obj441:head@111'441, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.793+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d85a09a6:::obj441:head, version: 111'441, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d85a09a6:::obj441:head@111'441, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.793+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.793+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1062 ==== pg_backfill(progress 1.0 e 116/116 lb 1:d83b1cbf:::obj272:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f10000 con 0x4acb000 2026-02-11T23:31:27.793+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d85a09a6:::obj441:head, version: 111'441, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d85a09a6:::obj441:head@111'441, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.793+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:d83b1cbf:::obj272:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.793+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d83b1cbf:::obj272:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.793+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.793+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d85a09a6:::obj441:head, version: 111'441, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d85a09a6:::obj441:head@111'441, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.793+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.793+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:d85a09a6:::obj441:head, version: 111'441, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d85a09a6:::obj441:head@111'441, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(1.0 116/113 [PushOp(1:d85a09a6:::obj441:head, version: 111'441, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d85a09a6:::obj441:head@111'441, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:d80bb902:::obj418:head local-lis/les=113/114 n=512 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.793+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.793+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d80bb902:::obj418:head local-lis/les=113/114 n=512 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:d85a09a6:::obj441:head, version: 111'441, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d85a09a6:::obj441:head@111'441, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.793+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d80bb902:::obj418:head local-lis/les=113/114 n=512 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:d85a09a6:::obj441:head@111'441, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.793+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.793+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d83b1cbf:::obj272:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.793+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d80bb902:::obj418:head local-lis/les=113/114 n=512 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:d85a09a6:::obj441:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.793+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d80bb902:::obj418:head local-lis/les=113/114 n=512 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:d85a09a6:::obj441:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.793+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d80bb902:::obj418:head local-lis/les=113/114 n=512 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:d85a09a6:::obj441:head 2026-02-11T23:31:27.793+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d80bb902:::obj418:head local-lis/les=113/114 n=512 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:d85a09a6:::obj441:head v 111'441 2026-02-11T23:31:27.793+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d80bb902:::obj418:head local-lis/les=113/114 n=512 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.793+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.793+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:d85a09a6:::obj441:head, version: 111'441, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d85a09a6:::obj441:head@111'441, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.793+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.793+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.793+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d83b1cbf:::obj272:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.793+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:d83b1cbf:::obj272:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 1.0 e 116/116 lb 1:d83b1cbf:::obj272:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:d80bb902:::obj418:head local-lis/les=113/114 n=512 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.793+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.793+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d80bb902:::obj418:head local-lis/les=113/114 n=512 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:d83b1cbf:::obj272:head) v3 2026-02-11T23:31:27.793+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d80bb902:::obj418:head local-lis/les=113/114 n=512 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:d83b1cbf:::obj272:head) 2026-02-11T23:31:27.793+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.793+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:d83b1cbf:::obj272:head) v3 finish latency 0.000009 2026-02-11T23:31:27.793+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d83b1cbf:::obj272:head local-lis/les=113/114 n=513 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.793+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.794+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.794+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d83b1cbf:::obj272:head local-lis/les=113/114 n=513 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.794+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.794+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.794+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.794+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fd3c80 con 0x4acb000 2026-02-11T23:31:27.794+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:d85a09a6:::obj441:head)]) -- 0x5fcfe00 con 0x4acb000 2026-02-11T23:31:27.794+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.794+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1063 ==== MOSDPGPush(1.0 116/113 [PushOp(1:d881fc7d:::obj242:head, version: 111'242, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d881fc7d:::obj242:head@111'242, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559d040 con 0x4acb000 2026-02-11T23:31:27.794+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:d881fc7d:::obj242:head, version: 111'242, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d881fc7d:::obj242:head@111'242, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.794+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d881fc7d:::obj242:head, version: 111'242, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d881fc7d:::obj242:head@111'242, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.794+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.794+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1064 ==== pg_backfill(progress 1.0 e 116/116 lb 1:d85a09a6:::obj441:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f10580 con 0x4acb000 2026-02-11T23:31:27.794+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d881fc7d:::obj242:head, version: 111'242, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d881fc7d:::obj242:head@111'242, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.794+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:d85a09a6:::obj441:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.794+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d85a09a6:::obj441:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.794+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.794+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d881fc7d:::obj242:head, version: 111'242, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d881fc7d:::obj242:head@111'242, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.794+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:d881fc7d:::obj242:head, version: 111'242, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d881fc7d:::obj242:head@111'242, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 116/113 [PushOp(1:d881fc7d:::obj242:head, version: 111'242, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d881fc7d:::obj242:head@111'242, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:d83b1cbf:::obj272:head local-lis/les=113/114 n=513 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.794+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.794+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.794+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d83b1cbf:::obj272:head local-lis/les=113/114 n=513 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:d881fc7d:::obj242:head, version: 111'242, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d881fc7d:::obj242:head@111'242, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.794+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d85a09a6:::obj441:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.794+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d83b1cbf:::obj272:head local-lis/les=113/114 n=513 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:d881fc7d:::obj242:head@111'242, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.794+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d83b1cbf:::obj272:head local-lis/les=113/114 n=513 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:d881fc7d:::obj242:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.794+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d83b1cbf:::obj272:head local-lis/les=113/114 n=513 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:d881fc7d:::obj242:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.794+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d83b1cbf:::obj272:head local-lis/les=113/114 n=513 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:d881fc7d:::obj242:head 2026-02-11T23:31:27.794+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d83b1cbf:::obj272:head local-lis/les=113/114 n=513 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:d881fc7d:::obj242:head v 111'242 2026-02-11T23:31:27.794+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d83b1cbf:::obj272:head local-lis/les=113/114 n=513 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.794+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.794+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:d881fc7d:::obj242:head, version: 111'242, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d881fc7d:::obj242:head@111'242, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:27.794+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.794+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.794+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d85a09a6:::obj441:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.794+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:d85a09a6:::obj441:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:d85a09a6:::obj441:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:d83b1cbf:::obj272:head local-lis/les=113/114 n=513 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.794+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.794+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d83b1cbf:::obj272:head local-lis/les=113/114 n=513 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:d85a09a6:::obj441:head) v3 2026-02-11T23:31:27.794+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d83b1cbf:::obj272:head local-lis/les=113/114 n=513 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:d85a09a6:::obj441:head) 2026-02-11T23:31:27.794+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.794+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:d85a09a6:::obj441:head) v3 finish latency 0.000010 2026-02-11T23:31:27.794+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d85a09a6:::obj441:head local-lis/les=113/114 n=514 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.794+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.794+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.794+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d85a09a6:::obj441:head local-lis/les=113/114 n=514 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.794+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.794+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.794+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.794+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d1cd80 con 0x4acb000 2026-02-11T23:31:27.794+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:d881fc7d:::obj242:head)]) -- 0x612e000 con 0x4acb000 2026-02-11T23:31:27.794+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.794+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1065 ==== MOSDPGPush(1.0 116/113 [PushOp(1:d9239af3:::obj409:head, version: 111'409, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9239af3:::obj409:head@111'409, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559dba0 con 0x4acb000 2026-02-11T23:31:27.794+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:d9239af3:::obj409:head, version: 111'409, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9239af3:::obj409:head@111'409, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.794+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d9239af3:::obj409:head, version: 111'409, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9239af3:::obj409:head@111'409, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.794+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.794+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1066 ==== pg_backfill(progress 1.0 e 116/116 lb 1:d881fc7d:::obj242:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f10b00 con 0x4acb000 2026-02-11T23:31:27.794+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d9239af3:::obj409:head, version: 111'409, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9239af3:::obj409:head@111'409, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.794+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:d881fc7d:::obj242:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.794+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.794+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d881fc7d:::obj242:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.794+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d9239af3:::obj409:head, version: 111'409, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9239af3:::obj409:head@111'409, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.794+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.794+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d881fc7d:::obj242:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.794+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:d9239af3:::obj409:head, version: 111'409, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9239af3:::obj409:head@111'409, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 116/113 [PushOp(1:d9239af3:::obj409:head, version: 111'409, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9239af3:::obj409:head@111'409, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:d85a09a6:::obj441:head local-lis/les=113/114 n=514 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.794+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.794+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d85a09a6:::obj441:head local-lis/les=113/114 n=514 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:d9239af3:::obj409:head, version: 111'409, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9239af3:::obj409:head@111'409, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.794+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d85a09a6:::obj441:head local-lis/les=113/114 n=514 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:d9239af3:::obj409:head@111'409, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.794+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d85a09a6:::obj441:head local-lis/les=113/114 n=514 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:d9239af3:::obj409:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.794+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d85a09a6:::obj441:head local-lis/les=113/114 n=514 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:d9239af3:::obj409:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.794+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d85a09a6:::obj441:head local-lis/les=113/114 n=514 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:d9239af3:::obj409:head 2026-02-11T23:31:27.794+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d85a09a6:::obj441:head local-lis/les=113/114 n=514 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:d9239af3:::obj409:head v 111'409 2026-02-11T23:31:27.794+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d85a09a6:::obj441:head local-lis/les=113/114 n=514 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.794+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.794+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:d9239af3:::obj409:head, version: 111'409, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9239af3:::obj409:head@111'409, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.794+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.794+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.794+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d881fc7d:::obj242:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.794+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:d881fc7d:::obj242:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 1.0 e 116/116 lb 1:d881fc7d:::obj242:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:d85a09a6:::obj441:head local-lis/les=113/114 n=514 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.794+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.794+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d85a09a6:::obj441:head local-lis/les=113/114 n=514 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:d881fc7d:::obj242:head) v3 2026-02-11T23:31:27.794+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d85a09a6:::obj441:head local-lis/les=113/114 n=514 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:d881fc7d:::obj242:head) 2026-02-11T23:31:27.794+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.794+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:d881fc7d:::obj242:head) v3 finish latency 0.000009 2026-02-11T23:31:27.794+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d881fc7d:::obj242:head local-lis/les=113/114 n=515 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.794+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.795+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.795+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d881fc7d:::obj242:head local-lis/les=113/114 n=515 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.795+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.795+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.795+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.795+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e7ed80 con 0x4acb000 2026-02-11T23:31:27.795+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:d9239af3:::obj409:head)]) -- 0x600b800 con 0x4acb000 2026-02-11T23:31:27.795+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.795+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1067 ==== MOSDPGPush(1.0 116/113 [PushOp(1:d99b7c45:::obj564:head, version: 111'564, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d99b7c45:::obj564:head@111'564, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c680 con 0x4acb000 2026-02-11T23:31:27.795+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:d99b7c45:::obj564:head, version: 111'564, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d99b7c45:::obj564:head@111'564, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.795+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d99b7c45:::obj564:head, version: 111'564, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d99b7c45:::obj564:head@111'564, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.795+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.795+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1068 ==== pg_backfill(progress 1.0 e 116/116 lb 1:d9239af3:::obj409:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f11080 con 0x4acb000 2026-02-11T23:31:27.795+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d99b7c45:::obj564:head, version: 111'564, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d99b7c45:::obj564:head@111'564, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.795+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:d9239af3:::obj409:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.795+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.795+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d9239af3:::obj409:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.795+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d99b7c45:::obj564:head, version: 111'564, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d99b7c45:::obj564:head@111'564, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.795+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.795+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:d99b7c45:::obj564:head, version: 111'564, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d99b7c45:::obj564:head@111'564, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 116/113 [PushOp(1:d99b7c45:::obj564:head, version: 111'564, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d99b7c45:::obj564:head@111'564, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:d881fc7d:::obj242:head local-lis/les=113/114 n=515 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.795+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.795+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d881fc7d:::obj242:head local-lis/les=113/114 n=515 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:d99b7c45:::obj564:head, version: 111'564, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d99b7c45:::obj564:head@111'564, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.795+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d881fc7d:::obj242:head local-lis/les=113/114 n=515 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:d99b7c45:::obj564:head@111'564, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.795+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.795+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d9239af3:::obj409:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.795+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d881fc7d:::obj242:head local-lis/les=113/114 n=515 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:d99b7c45:::obj564:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.795+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d881fc7d:::obj242:head local-lis/les=113/114 n=515 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:d99b7c45:::obj564:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.795+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d881fc7d:::obj242:head local-lis/les=113/114 n=515 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:d99b7c45:::obj564:head 2026-02-11T23:31:27.795+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d881fc7d:::obj242:head local-lis/les=113/114 n=515 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:d99b7c45:::obj564:head v 111'564 2026-02-11T23:31:27.795+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d881fc7d:::obj242:head local-lis/les=113/114 n=515 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.795+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.795+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:d99b7c45:::obj564:head, version: 111'564, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d99b7c45:::obj564:head@111'564, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000043 2026-02-11T23:31:27.795+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.795+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.795+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d9239af3:::obj409:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.795+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:d9239af3:::obj409:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 1.0 e 116/116 lb 1:d9239af3:::obj409:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:d881fc7d:::obj242:head local-lis/les=113/114 n=515 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.795+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.795+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d881fc7d:::obj242:head local-lis/les=113/114 n=515 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:d9239af3:::obj409:head) v3 2026-02-11T23:31:27.795+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d881fc7d:::obj242:head local-lis/les=113/114 n=515 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:d9239af3:::obj409:head) 2026-02-11T23:31:27.795+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.795+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:d9239af3:::obj409:head) v3 finish latency 0.000010 2026-02-11T23:31:27.795+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d9239af3:::obj409:head local-lis/les=113/114 n=516 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.795+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.795+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.795+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d9239af3:::obj409:head local-lis/les=113/114 n=516 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.795+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.795+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.795+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.795+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6171080 con 0x4acb000 2026-02-11T23:31:27.795+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:d99b7c45:::obj564:head)]) -- 0x6171800 con 0x4acb000 2026-02-11T23:31:27.795+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.795+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1069 ==== MOSDPGPush(1.0 116/113 [PushOp(1:d9c98c7a:::obj262:head, version: 111'262, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9c98c7a:::obj262:head@111'262, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3815860 con 0x4acb000 2026-02-11T23:31:27.795+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:d9c98c7a:::obj262:head, version: 111'262, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9c98c7a:::obj262:head@111'262, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.795+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d9c98c7a:::obj262:head, version: 111'262, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9c98c7a:::obj262:head@111'262, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.795+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.795+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1070 ==== pg_backfill(progress 1.0 e 116/116 lb 1:d99b7c45:::obj564:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f11600 con 0x4acb000 2026-02-11T23:31:27.795+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d9c98c7a:::obj262:head, version: 111'262, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9c98c7a:::obj262:head@111'262, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.795+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:d99b7c45:::obj564:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.795+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d99b7c45:::obj564:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.795+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.796+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d9c98c7a:::obj262:head, version: 111'262, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9c98c7a:::obj262:head@111'262, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.796+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.796+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:d9c98c7a:::obj262:head, version: 111'262, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9c98c7a:::obj262:head@111'262, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(1.0 116/113 [PushOp(1:d9c98c7a:::obj262:head, version: 111'262, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9c98c7a:::obj262:head@111'262, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:d9239af3:::obj409:head local-lis/les=113/114 n=516 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.796+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.796+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d9239af3:::obj409:head local-lis/les=113/114 n=516 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:d9c98c7a:::obj262:head, version: 111'262, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9c98c7a:::obj262:head@111'262, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.796+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d9239af3:::obj409:head local-lis/les=113/114 n=516 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:d9c98c7a:::obj262:head@111'262, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.796+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.796+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d99b7c45:::obj564:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.796+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d9239af3:::obj409:head local-lis/les=113/114 n=516 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:d9c98c7a:::obj262:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.796+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d9239af3:::obj409:head local-lis/les=113/114 n=516 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:d9c98c7a:::obj262:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.796+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d9239af3:::obj409:head local-lis/les=113/114 n=516 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:d9c98c7a:::obj262:head 2026-02-11T23:31:27.796+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d9239af3:::obj409:head local-lis/les=113/114 n=516 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:d9c98c7a:::obj262:head v 111'262 2026-02-11T23:31:27.796+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d9239af3:::obj409:head local-lis/les=113/114 n=516 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.796+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.796+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:d9c98c7a:::obj262:head, version: 111'262, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9c98c7a:::obj262:head@111'262, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.796+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.796+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.796+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d99b7c45:::obj564:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.796+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:d99b7c45:::obj564:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 1.0 e 116/116 lb 1:d99b7c45:::obj564:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:d9239af3:::obj409:head local-lis/les=113/114 n=516 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.796+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.796+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d9239af3:::obj409:head local-lis/les=113/114 n=516 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:d99b7c45:::obj564:head) v3 2026-02-11T23:31:27.796+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d9239af3:::obj409:head local-lis/les=113/114 n=516 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:d99b7c45:::obj564:head) 2026-02-11T23:31:27.796+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.796+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:d99b7c45:::obj564:head) v3 finish latency 0.000009 2026-02-11T23:31:27.796+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d99b7c45:::obj564:head local-lis/les=113/114 n=517 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.796+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.796+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.796+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d99b7c45:::obj564:head local-lis/les=113/114 n=517 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.796+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.796+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.796+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.796+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5f1c000 con 0x4acb000 2026-02-11T23:31:27.796+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:d9c98c7a:::obj262:head)]) -- 0x612ed80 con 0x4acb000 2026-02-11T23:31:27.796+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.796+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1071 ==== MOSDPGPush(1.0 116/113 [PushOp(1:d9da6b63:::obj302:head, version: 111'302, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9da6b63:::obj302:head@111'302, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f1520 con 0x4acb000 2026-02-11T23:31:27.796+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:d9da6b63:::obj302:head, version: 111'302, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9da6b63:::obj302:head@111'302, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.796+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d9da6b63:::obj302:head, version: 111'302, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9da6b63:::obj302:head@111'302, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.796+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.796+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1072 ==== pg_backfill(progress 1.0 e 116/116 lb 1:d9c98c7a:::obj262:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2f700 con 0x4acb000 2026-02-11T23:31:27.796+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:d9c98c7a:::obj262:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.796+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d9da6b63:::obj302:head, version: 111'302, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9da6b63:::obj302:head@111'302, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.796+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d9c98c7a:::obj262:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.796+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.796+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:d9da6b63:::obj302:head, version: 111'302, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9da6b63:::obj302:head@111'302, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.796+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.796+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:d9da6b63:::obj302:head, version: 111'302, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9da6b63:::obj302:head@111'302, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 116/113 [PushOp(1:d9da6b63:::obj302:head, version: 111'302, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9da6b63:::obj302:head@111'302, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:d99b7c45:::obj564:head local-lis/les=113/114 n=517 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.796+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.796+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d99b7c45:::obj564:head local-lis/les=113/114 n=517 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:d9da6b63:::obj302:head, version: 111'302, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9da6b63:::obj302:head@111'302, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.796+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d99b7c45:::obj564:head local-lis/les=113/114 n=517 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:d9da6b63:::obj302:head@111'302, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.796+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.796+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d99b7c45:::obj564:head local-lis/les=113/114 n=517 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:d9da6b63:::obj302:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.796+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d9c98c7a:::obj262:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.796+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d99b7c45:::obj564:head local-lis/les=113/114 n=517 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:d9da6b63:::obj302:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.796+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d99b7c45:::obj564:head local-lis/les=113/114 n=517 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:d9da6b63:::obj302:head 2026-02-11T23:31:27.796+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d99b7c45:::obj564:head local-lis/les=113/114 n=517 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:d9da6b63:::obj302:head v 111'302 2026-02-11T23:31:27.796+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d99b7c45:::obj564:head local-lis/les=113/114 n=517 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.796+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.796+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:d9da6b63:::obj302:head, version: 111'302, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9da6b63:::obj302:head@111'302, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000043 2026-02-11T23:31:27.796+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.796+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.796+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d9c98c7a:::obj262:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.796+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:d9c98c7a:::obj262:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 1.0 e 116/116 lb 1:d9c98c7a:::obj262:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:d99b7c45:::obj564:head local-lis/les=113/114 n=517 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.796+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.796+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d99b7c45:::obj564:head local-lis/les=113/114 n=517 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:d9c98c7a:::obj262:head) v3 2026-02-11T23:31:27.796+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d99b7c45:::obj564:head local-lis/les=113/114 n=517 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:d9c98c7a:::obj262:head) 2026-02-11T23:31:27.796+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.796+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:d9c98c7a:::obj262:head) v3 finish latency 0.000011 2026-02-11T23:31:27.796+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d9c98c7a:::obj262:head local-lis/les=113/114 n=518 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.796+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.796+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.796+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d9c98c7a:::obj262:head local-lis/les=113/114 n=518 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.796+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.796+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.796+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.796+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5f1c180 con 0x4acb000 2026-02-11T23:31:27.796+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:d9da6b63:::obj302:head)]) -- 0x6170300 con 0x4acb000 2026-02-11T23:31:27.796+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.797+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1073 ==== MOSDPGPush(1.0 116/113 [PushOp(1:da0e2ac9:::obj69:head, version: 111'69, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da0e2ac9:::obj69:head@111'69, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x38156c0 con 0x4acb000 2026-02-11T23:31:27.797+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:da0e2ac9:::obj69:head, version: 111'69, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da0e2ac9:::obj69:head@111'69, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.797+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:da0e2ac9:::obj69:head, version: 111'69, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da0e2ac9:::obj69:head@111'69, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.797+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.797+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1074 ==== pg_backfill(progress 1.0 e 116/116 lb 1:d9da6b63:::obj302:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2d080 con 0x4acb000 2026-02-11T23:31:27.797+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:da0e2ac9:::obj69:head, version: 111'69, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da0e2ac9:::obj69:head@111'69, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.797+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:d9da6b63:::obj302:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.797+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d9da6b63:::obj302:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.797+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.797+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:da0e2ac9:::obj69:head, version: 111'69, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da0e2ac9:::obj69:head@111'69, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.797+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.797+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:da0e2ac9:::obj69:head, version: 111'69, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da0e2ac9:::obj69:head@111'69, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:da0e2ac9:::obj69:head, version: 111'69, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da0e2ac9:::obj69:head@111'69, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:d9c98c7a:::obj262:head local-lis/les=113/114 n=518 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.797+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.797+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d9c98c7a:::obj262:head local-lis/les=113/114 n=518 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:da0e2ac9:::obj69:head, version: 111'69, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da0e2ac9:::obj69:head@111'69, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.797+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d9c98c7a:::obj262:head local-lis/les=113/114 n=518 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:da0e2ac9:::obj69:head@111'69, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.797+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.797+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d9da6b63:::obj302:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.797+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d9c98c7a:::obj262:head local-lis/les=113/114 n=518 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:da0e2ac9:::obj69:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.797+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d9c98c7a:::obj262:head local-lis/les=113/114 n=518 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:da0e2ac9:::obj69:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.797+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d9c98c7a:::obj262:head local-lis/les=113/114 n=518 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:da0e2ac9:::obj69:head 2026-02-11T23:31:27.797+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d9c98c7a:::obj262:head local-lis/les=113/114 n=518 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:da0e2ac9:::obj69:head v 111'69 2026-02-11T23:31:27.797+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d9c98c7a:::obj262:head local-lis/les=113/114 n=518 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.797+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.797+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:da0e2ac9:::obj69:head, version: 111'69, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da0e2ac9:::obj69:head@111'69, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.797+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.797+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.797+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:d9da6b63:::obj302:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.797+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:d9da6b63:::obj302:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 1.0 e 116/116 lb 1:d9da6b63:::obj302:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:d9c98c7a:::obj262:head local-lis/les=113/114 n=518 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.797+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.797+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d9c98c7a:::obj262:head local-lis/les=113/114 n=518 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:d9da6b63:::obj302:head) v3 2026-02-11T23:31:27.797+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d9c98c7a:::obj262:head local-lis/les=113/114 n=518 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:d9da6b63:::obj302:head) 2026-02-11T23:31:27.797+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.797+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:d9da6b63:::obj302:head) v3 finish latency 0.000010 2026-02-11T23:31:27.797+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d9da6b63:::obj302:head local-lis/les=113/114 n=519 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.797+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.797+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.797+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d9da6b63:::obj302:head local-lis/les=113/114 n=519 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.797+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.797+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.797+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.797+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5f1c300 con 0x4acb000 2026-02-11T23:31:27.797+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:da0e2ac9:::obj69:head)]) -- 0x600a480 con 0x4acb000 2026-02-11T23:31:27.797+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.797+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1075 ==== MOSDPGPush(1.0 116/113 [PushOp(1:da920364:::obj185:head, version: 111'185, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da920364:::obj185:head@111'185, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c9c0 con 0x4acb000 2026-02-11T23:31:27.797+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:da920364:::obj185:head, version: 111'185, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da920364:::obj185:head@111'185, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.797+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:da920364:::obj185:head, version: 111'185, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da920364:::obj185:head@111'185, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.797+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.797+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1076 ==== pg_backfill(progress 1.0 e 116/116 lb 1:da0e2ac9:::obj69:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d2e680 con 0x4acb000 2026-02-11T23:31:27.797+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:da920364:::obj185:head, version: 111'185, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da920364:::obj185:head@111'185, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.797+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:da0e2ac9:::obj69:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.797+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:da0e2ac9:::obj69:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.797+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.797+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:da920364:::obj185:head, version: 111'185, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da920364:::obj185:head@111'185, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.797+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.797+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:da920364:::obj185:head, version: 111'185, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da920364:::obj185:head@111'185, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 116/113 [PushOp(1:da920364:::obj185:head, version: 111'185, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da920364:::obj185:head@111'185, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:d9da6b63:::obj302:head local-lis/les=113/114 n=519 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.797+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.797+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d9da6b63:::obj302:head local-lis/les=113/114 n=519 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:da920364:::obj185:head, version: 111'185, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da920364:::obj185:head@111'185, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.797+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d9da6b63:::obj302:head local-lis/les=113/114 n=519 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:da920364:::obj185:head@111'185, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.797+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.797+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:da0e2ac9:::obj69:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.797+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d9da6b63:::obj302:head local-lis/les=113/114 n=519 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:da920364:::obj185:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.797+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d9da6b63:::obj302:head local-lis/les=113/114 n=519 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:da920364:::obj185:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.797+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d9da6b63:::obj302:head local-lis/les=113/114 n=519 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:da920364:::obj185:head 2026-02-11T23:31:27.797+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d9da6b63:::obj302:head local-lis/les=113/114 n=519 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:da920364:::obj185:head v 111'185 2026-02-11T23:31:27.797+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d9da6b63:::obj302:head local-lis/les=113/114 n=519 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.797+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.797+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:da920364:::obj185:head, version: 111'185, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da920364:::obj185:head@111'185, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.797+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.797+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.797+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:da0e2ac9:::obj69:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.797+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:da0e2ac9:::obj69:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 1.0 e 116/116 lb 1:da0e2ac9:::obj69:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:d9da6b63:::obj302:head local-lis/les=113/114 n=519 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.797+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.797+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d9da6b63:::obj302:head local-lis/les=113/114 n=519 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:da0e2ac9:::obj69:head) v3 2026-02-11T23:31:27.797+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:d9da6b63:::obj302:head local-lis/les=113/114 n=519 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:da0e2ac9:::obj69:head) 2026-02-11T23:31:27.797+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.797+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:da0e2ac9:::obj69:head) v3 finish latency 0.000009 2026-02-11T23:31:27.797+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:da0e2ac9:::obj69:head local-lis/les=113/114 n=520 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.797+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.797+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.797+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:da0e2ac9:::obj69:head local-lis/les=113/114 n=520 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.797+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.797+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.797+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.797+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5f1c480 con 0x4acb000 2026-02-11T23:31:27.797+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:da920364:::obj185:head)]) -- 0x5e6b980 con 0x4acb000 2026-02-11T23:31:27.797+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.798+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1077 ==== MOSDPGPush(1.0 116/113 [PushOp(1:da932e1e:::obj333:head, version: 111'333, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da932e1e:::obj333:head@111'333, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3814ea0 con 0x4acb000 2026-02-11T23:31:27.798+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:da932e1e:::obj333:head, version: 111'333, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da932e1e:::obj333:head@111'333, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.798+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:da932e1e:::obj333:head, version: 111'333, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da932e1e:::obj333:head@111'333, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.798+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.798+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1078 ==== pg_backfill(progress 1.0 e 116/116 lb 1:da920364:::obj185:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e100 con 0x4acb000 2026-02-11T23:31:27.798+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:da920364:::obj185:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.798+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:da932e1e:::obj333:head, version: 111'333, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da932e1e:::obj333:head@111'333, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.798+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:da920364:::obj185:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.798+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.798+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:da932e1e:::obj333:head, version: 111'333, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da932e1e:::obj333:head@111'333, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.798+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:da932e1e:::obj333:head, version: 111'333, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da932e1e:::obj333:head@111'333, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 116/113 [PushOp(1:da932e1e:::obj333:head, version: 111'333, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da932e1e:::obj333:head@111'333, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:da0e2ac9:::obj69:head local-lis/les=113/114 n=520 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.798+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.798+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.798+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:da0e2ac9:::obj69:head local-lis/les=113/114 n=520 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:da932e1e:::obj333:head, version: 111'333, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da932e1e:::obj333:head@111'333, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.798+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:da920364:::obj185:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.798+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:da0e2ac9:::obj69:head local-lis/les=113/114 n=520 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:da932e1e:::obj333:head@111'333, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.798+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:da0e2ac9:::obj69:head local-lis/les=113/114 n=520 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:da932e1e:::obj333:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.798+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:da0e2ac9:::obj69:head local-lis/les=113/114 n=520 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:da932e1e:::obj333:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.798+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:da0e2ac9:::obj69:head local-lis/les=113/114 n=520 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:da932e1e:::obj333:head 2026-02-11T23:31:27.798+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:da0e2ac9:::obj69:head local-lis/les=113/114 n=520 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:da932e1e:::obj333:head v 111'333 2026-02-11T23:31:27.798+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:da0e2ac9:::obj69:head local-lis/les=113/114 n=520 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.798+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.798+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:da932e1e:::obj333:head, version: 111'333, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da932e1e:::obj333:head@111'333, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.798+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.798+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.798+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:da920364:::obj185:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.798+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:da920364:::obj185:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:da920364:::obj185:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:da0e2ac9:::obj69:head local-lis/les=113/114 n=520 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.798+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.798+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:da0e2ac9:::obj69:head local-lis/les=113/114 n=520 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:da920364:::obj185:head) v3 2026-02-11T23:31:27.798+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:da0e2ac9:::obj69:head local-lis/les=113/114 n=520 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:da920364:::obj185:head) 2026-02-11T23:31:27.798+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.798+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:da920364:::obj185:head) v3 finish latency 0.000009 2026-02-11T23:31:27.798+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:da920364:::obj185:head local-lis/les=113/114 n=521 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.798+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.798+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.798+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:da920364:::obj185:head local-lis/les=113/114 n=521 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.798+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.798+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.798+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.798+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5f1c600 con 0x4acb000 2026-02-11T23:31:27.798+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:da932e1e:::obj333:head)]) -- 0x5fec780 con 0x4acb000 2026-02-11T23:31:27.798+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.798+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1079 ==== MOSDPGPush(1.0 116/113 [PushOp(1:dbca42a3:::obj382:head, version: 111'382, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dbca42a3:::obj382:head@111'382, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3815520 con 0x4acb000 2026-02-11T23:31:27.798+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:dbca42a3:::obj382:head, version: 111'382, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dbca42a3:::obj382:head@111'382, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.798+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:dbca42a3:::obj382:head, version: 111'382, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dbca42a3:::obj382:head@111'382, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.798+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.798+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1080 ==== pg_backfill(progress 1.0 e 116/116 lb 1:da932e1e:::obj333:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2c000 con 0x4acb000 2026-02-11T23:31:27.798+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:da932e1e:::obj333:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.798+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:dbca42a3:::obj382:head, version: 111'382, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dbca42a3:::obj382:head@111'382, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.798+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:da932e1e:::obj333:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.798+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.798+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:dbca42a3:::obj382:head, version: 111'382, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dbca42a3:::obj382:head@111'382, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.798+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.798+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:da932e1e:::obj333:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.798+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:dbca42a3:::obj382:head, version: 111'382, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dbca42a3:::obj382:head@111'382, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 116/113 [PushOp(1:dbca42a3:::obj382:head, version: 111'382, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dbca42a3:::obj382:head@111'382, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:da920364:::obj185:head local-lis/les=113/114 n=521 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.798+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.798+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:da920364:::obj185:head local-lis/les=113/114 n=521 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:dbca42a3:::obj382:head, version: 111'382, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dbca42a3:::obj382:head@111'382, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.798+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:da920364:::obj185:head local-lis/les=113/114 n=521 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:dbca42a3:::obj382:head@111'382, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.798+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:da920364:::obj185:head local-lis/les=113/114 n=521 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:dbca42a3:::obj382:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.798+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:da920364:::obj185:head local-lis/les=113/114 n=521 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:dbca42a3:::obj382:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.798+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:da920364:::obj185:head local-lis/les=113/114 n=521 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:dbca42a3:::obj382:head 2026-02-11T23:31:27.798+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:da920364:::obj185:head local-lis/les=113/114 n=521 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:dbca42a3:::obj382:head v 111'382 2026-02-11T23:31:27.798+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:da920364:::obj185:head local-lis/les=113/114 n=521 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.798+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.798+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:dbca42a3:::obj382:head, version: 111'382, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dbca42a3:::obj382:head@111'382, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.798+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.798+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.798+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:da932e1e:::obj333:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.798+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:da932e1e:::obj333:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 116/116 lb 1:da932e1e:::obj333:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:da920364:::obj185:head local-lis/les=113/114 n=521 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.798+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.798+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:da920364:::obj185:head local-lis/les=113/114 n=521 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:da932e1e:::obj333:head) v3 2026-02-11T23:31:27.798+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:da920364:::obj185:head local-lis/les=113/114 n=521 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:da932e1e:::obj333:head) 2026-02-11T23:31:27.798+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.798+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:da932e1e:::obj333:head) v3 finish latency 0.000010 2026-02-11T23:31:27.798+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:da932e1e:::obj333:head local-lis/les=113/114 n=522 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.798+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.798+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.798+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:da932e1e:::obj333:head local-lis/les=113/114 n=522 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.798+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.798+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.798+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.798+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5f1c780 con 0x4acb000 2026-02-11T23:31:27.798+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:dbca42a3:::obj382:head)]) -- 0x5fcfb00 con 0x4acb000 2026-02-11T23:31:27.798+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.799+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1081 ==== MOSDPGPush(1.0 116/113 [PushOp(1:dc49c339:::obj378:head, version: 111'378, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc49c339:::obj378:head@111'378, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c680 con 0x4acb000 2026-02-11T23:31:27.799+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:dc49c339:::obj378:head, version: 111'378, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc49c339:::obj378:head@111'378, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.799+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:dc49c339:::obj378:head, version: 111'378, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc49c339:::obj378:head@111'378, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.799+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.799+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1082 ==== pg_backfill(progress 1.0 e 116/116 lb 1:dbca42a3:::obj382:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2cb00 con 0x4acb000 2026-02-11T23:31:27.799+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:dbca42a3:::obj382:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.799+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:dc49c339:::obj378:head, version: 111'378, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc49c339:::obj378:head@111'378, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.799+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:dbca42a3:::obj382:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.799+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.799+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:dc49c339:::obj378:head, version: 111'378, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc49c339:::obj378:head@111'378, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.799+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.799+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:dc49c339:::obj378:head, version: 111'378, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc49c339:::obj378:head@111'378, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:dc49c339:::obj378:head, version: 111'378, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc49c339:::obj378:head@111'378, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:da932e1e:::obj333:head local-lis/les=113/114 n=522 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.799+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.799+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:da932e1e:::obj333:head local-lis/les=113/114 n=522 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:dc49c339:::obj378:head, version: 111'378, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc49c339:::obj378:head@111'378, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.799+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:da932e1e:::obj333:head local-lis/les=113/114 n=522 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:dc49c339:::obj378:head@111'378, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.799+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.799+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:dbca42a3:::obj382:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.799+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:da932e1e:::obj333:head local-lis/les=113/114 n=522 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:dc49c339:::obj378:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.799+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:da932e1e:::obj333:head local-lis/les=113/114 n=522 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:dc49c339:::obj378:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.799+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:da932e1e:::obj333:head local-lis/les=113/114 n=522 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:dc49c339:::obj378:head 2026-02-11T23:31:27.799+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:da932e1e:::obj333:head local-lis/les=113/114 n=522 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:dc49c339:::obj378:head v 111'378 2026-02-11T23:31:27.799+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:da932e1e:::obj333:head local-lis/les=113/114 n=522 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.799+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.799+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:dc49c339:::obj378:head, version: 111'378, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc49c339:::obj378:head@111'378, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.799+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.799+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.799+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:dbca42a3:::obj382:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.799+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:dbca42a3:::obj382:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:dbca42a3:::obj382:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:da932e1e:::obj333:head local-lis/les=113/114 n=522 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.799+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.799+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:da932e1e:::obj333:head local-lis/les=113/114 n=522 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:dbca42a3:::obj382:head) v3 2026-02-11T23:31:27.799+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:da932e1e:::obj333:head local-lis/les=113/114 n=522 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:dbca42a3:::obj382:head) 2026-02-11T23:31:27.799+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.799+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:dbca42a3:::obj382:head) v3 finish latency 0.000009 2026-02-11T23:31:27.799+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dbca42a3:::obj382:head local-lis/les=113/114 n=523 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.799+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.799+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.799+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dbca42a3:::obj382:head local-lis/les=113/114 n=523 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.799+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.799+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.799+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.799+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5f1c900 con 0x4acb000 2026-02-11T23:31:27.799+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:dc49c339:::obj378:head)]) -- 0x5ca1680 con 0x4acb000 2026-02-11T23:31:27.799+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.799+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1083 ==== MOSDPGPush(1.0 116/113 [PushOp(1:dc6ed3c7:::obj113:head, version: 111'113, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc6ed3c7:::obj113:head@111'113, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559dba0 con 0x4acb000 2026-02-11T23:31:27.799+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:dc6ed3c7:::obj113:head, version: 111'113, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc6ed3c7:::obj113:head@111'113, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.799+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:dc6ed3c7:::obj113:head, version: 111'113, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc6ed3c7:::obj113:head@111'113, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.799+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.799+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1084 ==== pg_backfill(progress 1.0 e 116/116 lb 1:dc49c339:::obj378:head) ==== 1062+0+0 (unknown 0 0 0) 0x6179700 con 0x4acb000 2026-02-11T23:31:27.799+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:dc6ed3c7:::obj113:head, version: 111'113, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc6ed3c7:::obj113:head@111'113, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.799+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:dc49c339:::obj378:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.799+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:dc49c339:::obj378:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.799+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.799+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:dc6ed3c7:::obj113:head, version: 111'113, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc6ed3c7:::obj113:head@111'113, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.799+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.799+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:dc6ed3c7:::obj113:head, version: 111'113, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc6ed3c7:::obj113:head@111'113, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 116/113 [PushOp(1:dc6ed3c7:::obj113:head, version: 111'113, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc6ed3c7:::obj113:head@111'113, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:dbca42a3:::obj382:head local-lis/les=113/114 n=523 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.799+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.799+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dbca42a3:::obj382:head local-lis/les=113/114 n=523 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:dc6ed3c7:::obj113:head, version: 111'113, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc6ed3c7:::obj113:head@111'113, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.799+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dbca42a3:::obj382:head local-lis/les=113/114 n=523 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:dc6ed3c7:::obj113:head@111'113, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.799+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.799+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:dc49c339:::obj378:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.799+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dbca42a3:::obj382:head local-lis/les=113/114 n=523 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:dc6ed3c7:::obj113:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.799+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dbca42a3:::obj382:head local-lis/les=113/114 n=523 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:dc6ed3c7:::obj113:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.799+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dbca42a3:::obj382:head local-lis/les=113/114 n=523 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:dc6ed3c7:::obj113:head 2026-02-11T23:31:27.799+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dbca42a3:::obj382:head local-lis/les=113/114 n=523 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:dc6ed3c7:::obj113:head v 111'113 2026-02-11T23:31:27.799+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dbca42a3:::obj382:head local-lis/les=113/114 n=523 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.799+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.799+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:dc6ed3c7:::obj113:head, version: 111'113, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc6ed3c7:::obj113:head@111'113, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000044 2026-02-11T23:31:27.799+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.799+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.799+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:dc49c339:::obj378:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.799+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:dc49c339:::obj378:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 1.0 e 116/116 lb 1:dc49c339:::obj378:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:dbca42a3:::obj382:head local-lis/les=113/114 n=523 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.799+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.799+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dbca42a3:::obj382:head local-lis/les=113/114 n=523 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:dc49c339:::obj378:head) v3 2026-02-11T23:31:27.799+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dbca42a3:::obj382:head local-lis/les=113/114 n=523 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:dc49c339:::obj378:head) 2026-02-11T23:31:27.799+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.799+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:dc49c339:::obj378:head) v3 finish latency 0.000010 2026-02-11T23:31:27.799+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc49c339:::obj378:head local-lis/les=113/114 n=524 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.799+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.800+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.800+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc49c339:::obj378:head local-lis/les=113/114 n=524 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.800+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.800+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.800+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.800+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5f1ca80 con 0x4acb000 2026-02-11T23:31:27.800+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:dc6ed3c7:::obj113:head)]) -- 0x5d1ca80 con 0x4acb000 2026-02-11T23:31:27.800+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.800+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1085 ==== MOSDPGPush(1.0 116/113 [PushOp(1:dc6fad89:::obj337:head, version: 111'337, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc6fad89:::obj337:head@111'337, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559d040 con 0x4acb000 2026-02-11T23:31:27.800+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:dc6fad89:::obj337:head, version: 111'337, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc6fad89:::obj337:head@111'337, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.800+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:dc6fad89:::obj337:head, version: 111'337, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc6fad89:::obj337:head@111'337, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.800+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.800+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1086 ==== pg_backfill(progress 1.0 e 116/116 lb 1:dc6ed3c7:::obj113:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176000 con 0x4acb000 2026-02-11T23:31:27.800+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:dc6fad89:::obj337:head, version: 111'337, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc6fad89:::obj337:head@111'337, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.800+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:dc6ed3c7:::obj113:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.800+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.800+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:dc6ed3c7:::obj113:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.800+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:dc6fad89:::obj337:head, version: 111'337, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc6fad89:::obj337:head@111'337, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.800+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.800+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:dc6fad89:::obj337:head, version: 111'337, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc6fad89:::obj337:head@111'337, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 116/113 [PushOp(1:dc6fad89:::obj337:head, version: 111'337, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc6fad89:::obj337:head@111'337, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:dc49c339:::obj378:head local-lis/les=113/114 n=524 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.800+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.800+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc49c339:::obj378:head local-lis/les=113/114 n=524 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:dc6fad89:::obj337:head, version: 111'337, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc6fad89:::obj337:head@111'337, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.800+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc49c339:::obj378:head local-lis/les=113/114 n=524 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:dc6fad89:::obj337:head@111'337, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.800+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.800+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:dc6ed3c7:::obj113:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.800+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc49c339:::obj378:head local-lis/les=113/114 n=524 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:dc6fad89:::obj337:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.800+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc49c339:::obj378:head local-lis/les=113/114 n=524 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:dc6fad89:::obj337:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.800+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc49c339:::obj378:head local-lis/les=113/114 n=524 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:dc6fad89:::obj337:head 2026-02-11T23:31:27.800+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc49c339:::obj378:head local-lis/les=113/114 n=524 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:dc6fad89:::obj337:head v 111'337 2026-02-11T23:31:27.800+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc49c339:::obj378:head local-lis/les=113/114 n=524 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.800+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.800+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:dc6fad89:::obj337:head, version: 111'337, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc6fad89:::obj337:head@111'337, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000042 2026-02-11T23:31:27.800+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.800+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.800+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:dc6ed3c7:::obj113:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.800+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:dc6ed3c7:::obj113:head) v3 prio 15 cost 0 latency 0.000057 pg_backfill(progress 1.0 e 116/116 lb 1:dc6ed3c7:::obj113:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:dc49c339:::obj378:head local-lis/les=113/114 n=524 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.800+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.800+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc49c339:::obj378:head local-lis/les=113/114 n=524 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:dc6ed3c7:::obj113:head) v3 2026-02-11T23:31:27.800+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc49c339:::obj378:head local-lis/les=113/114 n=524 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:dc6ed3c7:::obj113:head) 2026-02-11T23:31:27.800+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.800+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:dc6ed3c7:::obj113:head) v3 finish latency 0.000012 2026-02-11T23:31:27.800+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc6ed3c7:::obj113:head local-lis/les=113/114 n=525 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.800+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.800+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.800+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc6ed3c7:::obj113:head local-lis/les=113/114 n=525 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.800+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.800+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.800+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.800+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5f1cc00 con 0x4acb000 2026-02-11T23:31:27.800+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:dc6fad89:::obj337:head)]) -- 0x5fd2180 con 0x4acb000 2026-02-11T23:31:27.800+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.800+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1087 ==== MOSDPGPush(1.0 116/113 [PushOp(1:dc7d10b2:::obj533:head, version: 111'533, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc7d10b2:::obj533:head@111'533, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d89c0 con 0x4acb000 2026-02-11T23:31:27.800+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:dc7d10b2:::obj533:head, version: 111'533, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc7d10b2:::obj533:head@111'533, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.800+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:dc7d10b2:::obj533:head, version: 111'533, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc7d10b2:::obj533:head@111'533, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.800+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.800+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1088 ==== pg_backfill(progress 1.0 e 116/116 lb 1:dc6fad89:::obj337:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176b00 con 0x4acb000 2026-02-11T23:31:27.800+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:dc7d10b2:::obj533:head, version: 111'533, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc7d10b2:::obj533:head@111'533, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.800+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:dc6fad89:::obj337:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.800+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:dc6fad89:::obj337:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.800+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.800+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:dc7d10b2:::obj533:head, version: 111'533, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc7d10b2:::obj533:head@111'533, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.800+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:dc7d10b2:::obj533:head, version: 111'533, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc7d10b2:::obj533:head@111'533, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 116/113 [PushOp(1:dc7d10b2:::obj533:head, version: 111'533, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc7d10b2:::obj533:head@111'533, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:dc6ed3c7:::obj113:head local-lis/les=113/114 n=525 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.800+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.800+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.800+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc6ed3c7:::obj113:head local-lis/les=113/114 n=525 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:dc7d10b2:::obj533:head, version: 111'533, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc7d10b2:::obj533:head@111'533, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.800+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:dc6fad89:::obj337:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.800+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc6ed3c7:::obj113:head local-lis/les=113/114 n=525 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:dc7d10b2:::obj533:head@111'533, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.801+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc6ed3c7:::obj113:head local-lis/les=113/114 n=525 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:dc7d10b2:::obj533:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.801+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc6ed3c7:::obj113:head local-lis/les=113/114 n=525 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:dc7d10b2:::obj533:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.801+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc6ed3c7:::obj113:head local-lis/les=113/114 n=525 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:dc7d10b2:::obj533:head 2026-02-11T23:31:27.801+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc6ed3c7:::obj113:head local-lis/les=113/114 n=525 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:dc7d10b2:::obj533:head v 111'533 2026-02-11T23:31:27.801+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc6ed3c7:::obj113:head local-lis/les=113/114 n=525 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.801+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.801+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:dc7d10b2:::obj533:head, version: 111'533, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc7d10b2:::obj533:head@111'533, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:27.801+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc6ed3c7:::obj113:head local-lis/les=113/114 n=525 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.801+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.801+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.801+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:dc6fad89:::obj337:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.801+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:dc6fad89:::obj337:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 1.0 e 116/116 lb 1:dc6fad89:::obj337:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:dc6ed3c7:::obj113:head local-lis/les=113/114 n=525 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.801+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.801+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc6ed3c7:::obj113:head local-lis/les=113/114 n=525 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:dc6fad89:::obj337:head) v3 2026-02-11T23:31:27.801+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc6ed3c7:::obj113:head local-lis/les=113/114 n=525 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:dc6fad89:::obj337:head) 2026-02-11T23:31:27.801+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.801+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:dc6fad89:::obj337:head) v3 finish latency 0.000010 2026-02-11T23:31:27.801+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.801+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.801+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc6fad89:::obj337:head local-lis/les=113/114 n=526 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.801+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.801+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.801+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.801+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5f1d080 con 0x4acb000 2026-02-11T23:31:27.801+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:dc7d10b2:::obj533:head)]) -- 0x5f1cd80 con 0x4acb000 2026-02-11T23:31:27.801+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.801+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1089 ==== MOSDPGPush(1.0 116/113 [PushOp(1:dc9e4e8c:::obj567:head, version: 111'567, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc9e4e8c:::obj567:head@111'567, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d8000 con 0x4acb000 2026-02-11T23:31:27.801+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:dc9e4e8c:::obj567:head, version: 111'567, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc9e4e8c:::obj567:head@111'567, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.801+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:dc9e4e8c:::obj567:head, version: 111'567, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc9e4e8c:::obj567:head@111'567, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.801+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.801+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1090 ==== pg_backfill(progress 1.0 e 116/116 lb 1:dc7d10b2:::obj533:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177080 con 0x4acb000 2026-02-11T23:31:27.801+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:dc7d10b2:::obj533:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.801+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:dc9e4e8c:::obj567:head, version: 111'567, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc9e4e8c:::obj567:head@111'567, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.801+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:dc7d10b2:::obj533:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.801+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.801+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:dc9e4e8c:::obj567:head, version: 111'567, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc9e4e8c:::obj567:head@111'567, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.801+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:dc9e4e8c:::obj567:head, version: 111'567, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc9e4e8c:::obj567:head@111'567, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 116/113 [PushOp(1:dc9e4e8c:::obj567:head, version: 111'567, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc9e4e8c:::obj567:head@111'567, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:dc6fad89:::obj337:head local-lis/les=113/114 n=526 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.801+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.801+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.801+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:dc7d10b2:::obj533:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.801+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc6fad89:::obj337:head local-lis/les=113/114 n=526 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:dc9e4e8c:::obj567:head, version: 111'567, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc9e4e8c:::obj567:head@111'567, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.801+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc6fad89:::obj337:head local-lis/les=113/114 n=526 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:dc9e4e8c:::obj567:head@111'567, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.801+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc6fad89:::obj337:head local-lis/les=113/114 n=526 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:dc9e4e8c:::obj567:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.801+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc6fad89:::obj337:head local-lis/les=113/114 n=526 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:dc9e4e8c:::obj567:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.801+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc6fad89:::obj337:head local-lis/les=113/114 n=526 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:dc9e4e8c:::obj567:head 2026-02-11T23:31:27.801+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc6fad89:::obj337:head local-lis/les=113/114 n=526 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:dc9e4e8c:::obj567:head v 111'567 2026-02-11T23:31:27.801+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc6fad89:::obj337:head local-lis/les=113/114 n=526 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.801+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.801+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:dc9e4e8c:::obj567:head, version: 111'567, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc9e4e8c:::obj567:head@111'567, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:27.801+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc6fad89:::obj337:head local-lis/les=113/114 n=526 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.801+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.801+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.801+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:dc7d10b2:::obj533:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.801+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:dc7d10b2:::obj533:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 1.0 e 116/116 lb 1:dc7d10b2:::obj533:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:dc6fad89:::obj337:head local-lis/les=113/114 n=526 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.801+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.801+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc6fad89:::obj337:head local-lis/les=113/114 n=526 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:dc7d10b2:::obj533:head) v3 2026-02-11T23:31:27.801+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc6fad89:::obj337:head local-lis/les=113/114 n=526 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:dc7d10b2:::obj533:head) 2026-02-11T23:31:27.801+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.801+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:dc7d10b2:::obj533:head) v3 finish latency 0.000009 2026-02-11T23:31:27.801+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.801+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.801+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc7d10b2:::obj533:head local-lis/les=113/114 n=527 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.801+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.801+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.801+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.801+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5f1d500 con 0x4acb000 2026-02-11T23:31:27.801+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:dc9e4e8c:::obj567:head)]) -- 0x5f1d200 con 0x4acb000 2026-02-11T23:31:27.801+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.802+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1091 ==== MOSDPGPush(1.0 116/113 [PushOp(1:dcf61aca:::obj332:head, version: 111'332, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dcf61aca:::obj332:head@111'332, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d84e0 con 0x4acb000 2026-02-11T23:31:27.802+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:dcf61aca:::obj332:head, version: 111'332, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dcf61aca:::obj332:head@111'332, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.802+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:dcf61aca:::obj332:head, version: 111'332, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dcf61aca:::obj332:head@111'332, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.802+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.802+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1092 ==== pg_backfill(progress 1.0 e 116/116 lb 1:dc9e4e8c:::obj567:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177600 con 0x4acb000 2026-02-11T23:31:27.802+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:dc9e4e8c:::obj567:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.802+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:dcf61aca:::obj332:head, version: 111'332, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dcf61aca:::obj332:head@111'332, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.802+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:dc9e4e8c:::obj567:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.802+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.802+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:dcf61aca:::obj332:head, version: 111'332, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dcf61aca:::obj332:head@111'332, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.802+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:dcf61aca:::obj332:head, version: 111'332, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dcf61aca:::obj332:head@111'332, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:dcf61aca:::obj332:head, version: 111'332, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dcf61aca:::obj332:head@111'332, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:dc7d10b2:::obj533:head local-lis/les=113/114 n=527 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.802+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.802+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc7d10b2:::obj533:head local-lis/les=113/114 n=527 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:dcf61aca:::obj332:head, version: 111'332, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dcf61aca:::obj332:head@111'332, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.802+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.802+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:dc9e4e8c:::obj567:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.802+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc7d10b2:::obj533:head local-lis/les=113/114 n=527 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:dcf61aca:::obj332:head@111'332, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.802+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc7d10b2:::obj533:head local-lis/les=113/114 n=527 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:dcf61aca:::obj332:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.802+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc7d10b2:::obj533:head local-lis/les=113/114 n=527 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:dcf61aca:::obj332:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.802+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc7d10b2:::obj533:head local-lis/les=113/114 n=527 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:dcf61aca:::obj332:head 2026-02-11T23:31:27.802+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc7d10b2:::obj533:head local-lis/les=113/114 n=527 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:dcf61aca:::obj332:head v 111'332 2026-02-11T23:31:27.802+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc7d10b2:::obj533:head local-lis/les=113/114 n=527 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.802+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.802+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:dcf61aca:::obj332:head, version: 111'332, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dcf61aca:::obj332:head@111'332, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.802+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.802+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.802+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:dc9e4e8c:::obj567:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.802+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:dc9e4e8c:::obj567:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 1.0 e 116/116 lb 1:dc9e4e8c:::obj567:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:dc7d10b2:::obj533:head local-lis/les=113/114 n=527 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.802+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.802+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc7d10b2:::obj533:head local-lis/les=113/114 n=527 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:dc9e4e8c:::obj567:head) v3 2026-02-11T23:31:27.802+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc7d10b2:::obj533:head local-lis/les=113/114 n=527 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:dc9e4e8c:::obj567:head) 2026-02-11T23:31:27.802+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.802+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:dc9e4e8c:::obj567:head) v3 finish latency 0.000012 2026-02-11T23:31:27.802+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc9e4e8c:::obj567:head local-lis/les=113/114 n=528 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.802+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.802+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.802+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc9e4e8c:::obj567:head local-lis/les=113/114 n=528 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.802+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.802+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.802+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.802+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5f1d680 con 0x4acb000 2026-02-11T23:31:27.802+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:dcf61aca:::obj332:head)]) -- 0x5fd2480 con 0x4acb000 2026-02-11T23:31:27.802+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.802+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1093 ==== MOSDPGPush(1.0 116/113 [PushOp(1:ddce93c9:::obj164:head, version: 111'164, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ddce93c9:::obj164:head@111'164, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db0820 con 0x4acb000 2026-02-11T23:31:27.802+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:ddce93c9:::obj164:head, version: 111'164, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ddce93c9:::obj164:head@111'164, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.802+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:ddce93c9:::obj164:head, version: 111'164, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ddce93c9:::obj164:head@111'164, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.802+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.802+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1094 ==== pg_backfill(progress 1.0 e 116/116 lb 1:dcf61aca:::obj332:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177b80 con 0x4acb000 2026-02-11T23:31:27.802+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:dcf61aca:::obj332:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.802+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:ddce93c9:::obj164:head, version: 111'164, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ddce93c9:::obj164:head@111'164, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.802+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:dcf61aca:::obj332:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.802+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.802+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:ddce93c9:::obj164:head, version: 111'164, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ddce93c9:::obj164:head@111'164, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.802+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.802+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:ddce93c9:::obj164:head, version: 111'164, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ddce93c9:::obj164:head@111'164, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:ddce93c9:::obj164:head, version: 111'164, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ddce93c9:::obj164:head@111'164, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:dc9e4e8c:::obj567:head local-lis/les=113/114 n=528 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.802+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.802+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc9e4e8c:::obj567:head local-lis/les=113/114 n=528 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:ddce93c9:::obj164:head, version: 111'164, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ddce93c9:::obj164:head@111'164, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.802+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc9e4e8c:::obj567:head local-lis/les=113/114 n=528 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:ddce93c9:::obj164:head@111'164, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.802+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.802+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:dcf61aca:::obj332:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.802+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc9e4e8c:::obj567:head local-lis/les=113/114 n=528 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:ddce93c9:::obj164:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.802+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc9e4e8c:::obj567:head local-lis/les=113/114 n=528 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:ddce93c9:::obj164:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.802+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc9e4e8c:::obj567:head local-lis/les=113/114 n=528 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:ddce93c9:::obj164:head 2026-02-11T23:31:27.802+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc9e4e8c:::obj567:head local-lis/les=113/114 n=528 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:ddce93c9:::obj164:head v 111'164 2026-02-11T23:31:27.802+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc9e4e8c:::obj567:head local-lis/les=113/114 n=528 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.802+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.802+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:ddce93c9:::obj164:head, version: 111'164, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ddce93c9:::obj164:head@111'164, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.802+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.802+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.802+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:dcf61aca:::obj332:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.802+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:dcf61aca:::obj332:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 1.0 e 116/116 lb 1:dcf61aca:::obj332:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:dc9e4e8c:::obj567:head local-lis/les=113/114 n=528 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.802+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.802+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc9e4e8c:::obj567:head local-lis/les=113/114 n=528 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:dcf61aca:::obj332:head) v3 2026-02-11T23:31:27.802+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dc9e4e8c:::obj567:head local-lis/les=113/114 n=528 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:dcf61aca:::obj332:head) 2026-02-11T23:31:27.802+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.802+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:dcf61aca:::obj332:head) v3 finish latency 0.000009 2026-02-11T23:31:27.802+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dcf61aca:::obj332:head local-lis/les=113/114 n=529 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.802+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.802+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.802+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dcf61aca:::obj332:head local-lis/les=113/114 n=529 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.802+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.802+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.802+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.802+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5f1d800 con 0x4acb000 2026-02-11T23:31:27.802+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:ddce93c9:::obj164:head)]) -- 0x5fd2780 con 0x4acb000 2026-02-11T23:31:27.802+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.803+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1095 ==== MOSDPGPush(1.0 116/113 [PushOp(1:de0491ba:::obj374:head, version: 111'374, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:de0491ba:::obj374:head@111'374, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d8d00 con 0x4acb000 2026-02-11T23:31:27.803+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:de0491ba:::obj374:head, version: 111'374, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:de0491ba:::obj374:head@111'374, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.803+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:de0491ba:::obj374:head, version: 111'374, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:de0491ba:::obj374:head@111'374, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.803+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.803+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1096 ==== pg_backfill(progress 1.0 e 116/116 lb 1:ddce93c9:::obj164:head) ==== 1062+0+0 (unknown 0 0 0) 0x6178100 con 0x4acb000 2026-02-11T23:31:27.803+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:de0491ba:::obj374:head, version: 111'374, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:de0491ba:::obj374:head@111'374, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.803+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:ddce93c9:::obj164:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.803+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.803+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:ddce93c9:::obj164:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.803+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:de0491ba:::obj374:head, version: 111'374, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:de0491ba:::obj374:head@111'374, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.803+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.803+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:de0491ba:::obj374:head, version: 111'374, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:de0491ba:::obj374:head@111'374, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:de0491ba:::obj374:head, version: 111'374, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:de0491ba:::obj374:head@111'374, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:dcf61aca:::obj332:head local-lis/les=113/114 n=529 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.803+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.803+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dcf61aca:::obj332:head local-lis/les=113/114 n=529 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:de0491ba:::obj374:head, version: 111'374, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:de0491ba:::obj374:head@111'374, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.803+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dcf61aca:::obj332:head local-lis/les=113/114 n=529 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:de0491ba:::obj374:head@111'374, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.803+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.803+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:ddce93c9:::obj164:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.803+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dcf61aca:::obj332:head local-lis/les=113/114 n=529 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:de0491ba:::obj374:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.803+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dcf61aca:::obj332:head local-lis/les=113/114 n=529 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:de0491ba:::obj374:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.803+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dcf61aca:::obj332:head local-lis/les=113/114 n=529 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:de0491ba:::obj374:head 2026-02-11T23:31:27.803+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dcf61aca:::obj332:head local-lis/les=113/114 n=529 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:de0491ba:::obj374:head v 111'374 2026-02-11T23:31:27.803+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dcf61aca:::obj332:head local-lis/les=113/114 n=529 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.803+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.803+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:de0491ba:::obj374:head, version: 111'374, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:de0491ba:::obj374:head@111'374, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.803+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.803+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.803+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:ddce93c9:::obj164:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.803+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:ddce93c9:::obj164:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 116/116 lb 1:ddce93c9:::obj164:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:dcf61aca:::obj332:head local-lis/les=113/114 n=529 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.803+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.803+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dcf61aca:::obj332:head local-lis/les=113/114 n=529 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:ddce93c9:::obj164:head) v3 2026-02-11T23:31:27.803+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dcf61aca:::obj332:head local-lis/les=113/114 n=529 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:ddce93c9:::obj164:head) 2026-02-11T23:31:27.803+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.803+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:ddce93c9:::obj164:head) v3 finish latency 0.000009 2026-02-11T23:31:27.803+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ddce93c9:::obj164:head local-lis/les=113/114 n=530 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.803+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.803+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.803+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ddce93c9:::obj164:head local-lis/les=113/114 n=530 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.803+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.803+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.803+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.803+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5f1d980 con 0x4acb000 2026-02-11T23:31:27.803+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:de0491ba:::obj374:head)]) -- 0x5d78d80 con 0x4acb000 2026-02-11T23:31:27.803+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.803+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1097 ==== MOSDPGPush(1.0 116/113 [PushOp(1:deffd1ec:::obj134:head, version: 111'134, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:deffd1ec:::obj134:head@111'134, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e26820 con 0x4acb000 2026-02-11T23:31:27.803+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:deffd1ec:::obj134:head, version: 111'134, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:deffd1ec:::obj134:head@111'134, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.803+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:deffd1ec:::obj134:head, version: 111'134, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:deffd1ec:::obj134:head@111'134, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.803+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.803+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1098 ==== pg_backfill(progress 1.0 e 116/116 lb 1:de0491ba:::obj374:head) ==== 1062+0+0 (unknown 0 0 0) 0x6179180 con 0x4acb000 2026-02-11T23:31:27.803+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:deffd1ec:::obj134:head, version: 111'134, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:deffd1ec:::obj134:head@111'134, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.803+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.803+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:de0491ba:::obj374:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.803+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:deffd1ec:::obj134:head, version: 111'134, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:deffd1ec:::obj134:head@111'134, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.803+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:de0491ba:::obj374:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.803+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:deffd1ec:::obj134:head, version: 111'134, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:deffd1ec:::obj134:head@111'134, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 116/113 [PushOp(1:deffd1ec:::obj134:head, version: 111'134, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:deffd1ec:::obj134:head@111'134, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:ddce93c9:::obj164:head local-lis/les=113/114 n=530 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.803+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.803+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ddce93c9:::obj164:head local-lis/les=113/114 n=530 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:deffd1ec:::obj134:head, version: 111'134, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:deffd1ec:::obj134:head@111'134, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.803+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.803+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:de0491ba:::obj374:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.803+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ddce93c9:::obj164:head local-lis/les=113/114 n=530 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:deffd1ec:::obj134:head@111'134, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.803+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ddce93c9:::obj164:head local-lis/les=113/114 n=530 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:deffd1ec:::obj134:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.803+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ddce93c9:::obj164:head local-lis/les=113/114 n=530 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:deffd1ec:::obj134:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.803+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ddce93c9:::obj164:head local-lis/les=113/114 n=530 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:deffd1ec:::obj134:head 2026-02-11T23:31:27.803+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ddce93c9:::obj164:head local-lis/les=113/114 n=530 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:deffd1ec:::obj134:head v 111'134 2026-02-11T23:31:27.803+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ddce93c9:::obj164:head local-lis/les=113/114 n=530 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.803+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.803+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:deffd1ec:::obj134:head, version: 111'134, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:deffd1ec:::obj134:head@111'134, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000044 2026-02-11T23:31:27.803+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.803+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.803+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:de0491ba:::obj374:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.803+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:de0491ba:::obj374:head) v3 prio 15 cost 0 latency 0.000059 pg_backfill(progress 1.0 e 116/116 lb 1:de0491ba:::obj374:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:ddce93c9:::obj164:head local-lis/les=113/114 n=530 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.803+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.803+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ddce93c9:::obj164:head local-lis/les=113/114 n=530 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:de0491ba:::obj374:head) v3 2026-02-11T23:31:27.803+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ddce93c9:::obj164:head local-lis/les=113/114 n=530 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:de0491ba:::obj374:head) 2026-02-11T23:31:27.803+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.803+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:de0491ba:::obj374:head) v3 finish latency 0.000012 2026-02-11T23:31:27.803+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:de0491ba:::obj374:head local-lis/les=113/114 n=531 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.803+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.804+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.804+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:de0491ba:::obj374:head local-lis/les=113/114 n=531 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.804+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.804+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.804+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.804+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5f1db00 con 0x4acb000 2026-02-11T23:31:27.804+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:deffd1ec:::obj134:head)]) -- 0x6171380 con 0x4acb000 2026-02-11T23:31:27.804+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.804+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1099 ==== MOSDPGPush(1.0 116/113 [PushOp(1:dfa74a43:::obj27:head, version: 111'27, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dfa74a43:::obj27:head@111'27, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x55d96c0 con 0x4acb000 2026-02-11T23:31:27.804+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:dfa74a43:::obj27:head, version: 111'27, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dfa74a43:::obj27:head@111'27, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.804+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:dfa74a43:::obj27:head, version: 111'27, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dfa74a43:::obj27:head@111'27, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.804+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.804+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1100 ==== pg_backfill(progress 1.0 e 116/116 lb 1:deffd1ec:::obj134:head) ==== 1062+0+0 (unknown 0 0 0) 0x6178c00 con 0x4acb000 2026-02-11T23:31:27.804+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:dfa74a43:::obj27:head, version: 111'27, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dfa74a43:::obj27:head@111'27, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.804+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:deffd1ec:::obj134:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.804+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.804+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:deffd1ec:::obj134:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.804+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:dfa74a43:::obj27:head, version: 111'27, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dfa74a43:::obj27:head@111'27, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.804+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.804+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:dfa74a43:::obj27:head, version: 111'27, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dfa74a43:::obj27:head@111'27, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 116/113 [PushOp(1:dfa74a43:::obj27:head, version: 111'27, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dfa74a43:::obj27:head@111'27, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:de0491ba:::obj374:head local-lis/les=113/114 n=531 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.804+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.804+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:de0491ba:::obj374:head local-lis/les=113/114 n=531 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:dfa74a43:::obj27:head, version: 111'27, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dfa74a43:::obj27:head@111'27, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.804+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:de0491ba:::obj374:head local-lis/les=113/114 n=531 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:dfa74a43:::obj27:head@111'27, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.804+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.804+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:deffd1ec:::obj134:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.804+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:de0491ba:::obj374:head local-lis/les=113/114 n=531 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:dfa74a43:::obj27:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.804+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:de0491ba:::obj374:head local-lis/les=113/114 n=531 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:dfa74a43:::obj27:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.804+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:de0491ba:::obj374:head local-lis/les=113/114 n=531 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:dfa74a43:::obj27:head 2026-02-11T23:31:27.804+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:de0491ba:::obj374:head local-lis/les=113/114 n=531 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:dfa74a43:::obj27:head v 111'27 2026-02-11T23:31:27.804+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:de0491ba:::obj374:head local-lis/les=113/114 n=531 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.804+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.804+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:dfa74a43:::obj27:head, version: 111'27, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dfa74a43:::obj27:head@111'27, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.804+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.804+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.804+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:deffd1ec:::obj134:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.804+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:deffd1ec:::obj134:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 1.0 e 116/116 lb 1:deffd1ec:::obj134:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:de0491ba:::obj374:head local-lis/les=113/114 n=531 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.804+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.804+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:de0491ba:::obj374:head local-lis/les=113/114 n=531 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:deffd1ec:::obj134:head) v3 2026-02-11T23:31:27.804+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:de0491ba:::obj374:head local-lis/les=113/114 n=531 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:deffd1ec:::obj134:head) 2026-02-11T23:31:27.804+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.804+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:deffd1ec:::obj134:head) v3 finish latency 0.000010 2026-02-11T23:31:27.804+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:deffd1ec:::obj134:head local-lis/les=113/114 n=532 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.804+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.804+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.804+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:deffd1ec:::obj134:head local-lis/les=113/114 n=532 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.804+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.804+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.804+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.804+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5f1dc80 con 0x4acb000 2026-02-11T23:31:27.804+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:dfa74a43:::obj27:head)]) -- 0x5df8a80 con 0x4acb000 2026-02-11T23:31:27.804+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.804+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1101 ==== MOSDPGPush(1.0 116/113 [PushOp(1:e03f6950:::obj575:head, version: 111'575, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e03f6950:::obj575:head@111'575, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56ca000 con 0x4acb000 2026-02-11T23:31:27.804+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:e03f6950:::obj575:head, version: 111'575, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e03f6950:::obj575:head@111'575, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.804+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e03f6950:::obj575:head, version: 111'575, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e03f6950:::obj575:head@111'575, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.804+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.804+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1102 ==== pg_backfill(progress 1.0 e 116/116 lb 1:dfa74a43:::obj27:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d9b180 con 0x4acb000 2026-02-11T23:31:27.804+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:dfa74a43:::obj27:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.804+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:dfa74a43:::obj27:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.804+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e03f6950:::obj575:head, version: 111'575, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e03f6950:::obj575:head@111'575, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.804+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.804+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e03f6950:::obj575:head, version: 111'575, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e03f6950:::obj575:head@111'575, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.804+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:e03f6950:::obj575:head, version: 111'575, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e03f6950:::obj575:head@111'575, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000040 MOSDPGPush(1.0 116/113 [PushOp(1:e03f6950:::obj575:head, version: 111'575, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e03f6950:::obj575:head@111'575, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:deffd1ec:::obj134:head local-lis/les=113/114 n=532 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.804+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.804+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.804+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:deffd1ec:::obj134:head local-lis/les=113/114 n=532 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:e03f6950:::obj575:head, version: 111'575, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e03f6950:::obj575:head@111'575, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.804+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:dfa74a43:::obj27:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.804+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:deffd1ec:::obj134:head local-lis/les=113/114 n=532 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:e03f6950:::obj575:head@111'575, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.804+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:deffd1ec:::obj134:head local-lis/les=113/114 n=532 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:e03f6950:::obj575:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.804+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:deffd1ec:::obj134:head local-lis/les=113/114 n=532 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:e03f6950:::obj575:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.804+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:deffd1ec:::obj134:head local-lis/les=113/114 n=532 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:e03f6950:::obj575:head 2026-02-11T23:31:27.804+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:deffd1ec:::obj134:head local-lis/les=113/114 n=532 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:e03f6950:::obj575:head v 111'575 2026-02-11T23:31:27.804+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:deffd1ec:::obj134:head local-lis/les=113/114 n=532 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.804+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.804+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:e03f6950:::obj575:head, version: 111'575, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e03f6950:::obj575:head@111'575, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.804+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.804+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.804+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:dfa74a43:::obj27:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.804+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:dfa74a43:::obj27:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 1.0 e 116/116 lb 1:dfa74a43:::obj27:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:deffd1ec:::obj134:head local-lis/les=113/114 n=532 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.804+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.804+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:deffd1ec:::obj134:head local-lis/les=113/114 n=532 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:dfa74a43:::obj27:head) v3 2026-02-11T23:31:27.804+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:deffd1ec:::obj134:head local-lis/les=113/114 n=532 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:dfa74a43:::obj27:head) 2026-02-11T23:31:27.804+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.804+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:dfa74a43:::obj27:head) v3 finish latency 0.000010 2026-02-11T23:31:27.804+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dfa74a43:::obj27:head local-lis/les=113/114 n=533 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.804+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.805+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.805+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dfa74a43:::obj27:head local-lis/les=113/114 n=533 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.805+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.805+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.805+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.805+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5f1de00 con 0x4acb000 2026-02-11T23:31:27.805+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:e03f6950:::obj575:head)]) -- 0x5fd2f00 con 0x4acb000 2026-02-11T23:31:27.805+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.805+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1103 ==== MOSDPGPush(1.0 116/113 [PushOp(1:e05d79f1:::obj206:head, version: 111'206, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e05d79f1:::obj206:head@111'206, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559d860 con 0x4acb000 2026-02-11T23:31:27.805+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:e05d79f1:::obj206:head, version: 111'206, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e05d79f1:::obj206:head@111'206, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.805+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e05d79f1:::obj206:head, version: 111'206, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e05d79f1:::obj206:head@111'206, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.805+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.805+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1104 ==== pg_backfill(progress 1.0 e 116/116 lb 1:e03f6950:::obj575:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98000 con 0x4acb000 2026-02-11T23:31:27.805+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:e03f6950:::obj575:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.805+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e03f6950:::obj575:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.805+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e05d79f1:::obj206:head, version: 111'206, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e05d79f1:::obj206:head@111'206, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.805+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.805+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e05d79f1:::obj206:head, version: 111'206, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e05d79f1:::obj206:head@111'206, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.805+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:e05d79f1:::obj206:head, version: 111'206, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e05d79f1:::obj206:head@111'206, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 116/113 [PushOp(1:e05d79f1:::obj206:head, version: 111'206, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e05d79f1:::obj206:head@111'206, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:dfa74a43:::obj27:head local-lis/les=113/114 n=533 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.805+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.805+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.805+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dfa74a43:::obj27:head local-lis/les=113/114 n=533 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:e05d79f1:::obj206:head, version: 111'206, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e05d79f1:::obj206:head@111'206, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.805+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e03f6950:::obj575:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.805+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dfa74a43:::obj27:head local-lis/les=113/114 n=533 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:e05d79f1:::obj206:head@111'206, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.805+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dfa74a43:::obj27:head local-lis/les=113/114 n=533 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:e05d79f1:::obj206:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.805+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dfa74a43:::obj27:head local-lis/les=113/114 n=533 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:e05d79f1:::obj206:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.805+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dfa74a43:::obj27:head local-lis/les=113/114 n=533 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:e05d79f1:::obj206:head 2026-02-11T23:31:27.805+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dfa74a43:::obj27:head local-lis/les=113/114 n=533 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:e05d79f1:::obj206:head v 111'206 2026-02-11T23:31:27.805+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dfa74a43:::obj27:head local-lis/les=113/114 n=533 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.805+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.805+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:e05d79f1:::obj206:head, version: 111'206, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e05d79f1:::obj206:head@111'206, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.805+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.805+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.805+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e03f6950:::obj575:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.805+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:e03f6950:::obj575:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 1.0 e 116/116 lb 1:e03f6950:::obj575:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:dfa74a43:::obj27:head local-lis/les=113/114 n=533 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.805+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.805+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dfa74a43:::obj27:head local-lis/les=113/114 n=533 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:e03f6950:::obj575:head) v3 2026-02-11T23:31:27.805+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:dfa74a43:::obj27:head local-lis/les=113/114 n=533 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:e03f6950:::obj575:head) 2026-02-11T23:31:27.805+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.805+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:e03f6950:::obj575:head) v3 finish latency 0.000010 2026-02-11T23:31:27.805+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e03f6950:::obj575:head local-lis/les=113/114 n=534 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.805+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.805+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.805+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e03f6950:::obj575:head local-lis/les=113/114 n=534 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.805+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.805+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.805+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.805+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fcfb00 con 0x4acb000 2026-02-11T23:31:27.805+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:e05d79f1:::obj206:head)]) -- 0x615ca80 con 0x4acb000 2026-02-11T23:31:27.805+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.806+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1105 ==== MOSDPGPush(1.0 116/113 [PushOp(1:e0a44829:::obj14:head, version: 111'14, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e0a44829:::obj14:head@111'14, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x55d8680 con 0x4acb000 2026-02-11T23:31:27.806+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:e0a44829:::obj14:head, version: 111'14, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e0a44829:::obj14:head@111'14, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.806+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e0a44829:::obj14:head, version: 111'14, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e0a44829:::obj14:head@111'14, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.806+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.806+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1106 ==== pg_backfill(progress 1.0 e 116/116 lb 1:e05d79f1:::obj206:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98580 con 0x4acb000 2026-02-11T23:31:27.806+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e0a44829:::obj14:head, version: 111'14, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e0a44829:::obj14:head@111'14, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.806+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:e05d79f1:::obj206:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.806+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e05d79f1:::obj206:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.806+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.806+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e0a44829:::obj14:head, version: 111'14, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e0a44829:::obj14:head@111'14, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.806+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.806+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:e0a44829:::obj14:head, version: 111'14, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e0a44829:::obj14:head@111'14, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:e0a44829:::obj14:head, version: 111'14, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e0a44829:::obj14:head@111'14, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:e03f6950:::obj575:head local-lis/les=113/114 n=534 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.806+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.806+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e03f6950:::obj575:head local-lis/les=113/114 n=534 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:e0a44829:::obj14:head, version: 111'14, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e0a44829:::obj14:head@111'14, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.806+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e03f6950:::obj575:head local-lis/les=113/114 n=534 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:e0a44829:::obj14:head@111'14, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.806+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.806+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e05d79f1:::obj206:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.806+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e03f6950:::obj575:head local-lis/les=113/114 n=534 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:e0a44829:::obj14:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.806+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e03f6950:::obj575:head local-lis/les=113/114 n=534 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:e0a44829:::obj14:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.806+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e03f6950:::obj575:head local-lis/les=113/114 n=534 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:e0a44829:::obj14:head 2026-02-11T23:31:27.806+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e03f6950:::obj575:head local-lis/les=113/114 n=534 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:e0a44829:::obj14:head v 111'14 2026-02-11T23:31:27.806+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e03f6950:::obj575:head local-lis/les=113/114 n=534 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.806+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.806+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:e0a44829:::obj14:head, version: 111'14, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e0a44829:::obj14:head@111'14, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.806+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.806+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.806+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e05d79f1:::obj206:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.806+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:e05d79f1:::obj206:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:e05d79f1:::obj206:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:e03f6950:::obj575:head local-lis/les=113/114 n=534 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.806+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.806+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e03f6950:::obj575:head local-lis/les=113/114 n=534 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:e05d79f1:::obj206:head) v3 2026-02-11T23:31:27.806+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e03f6950:::obj575:head local-lis/les=113/114 n=534 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:e05d79f1:::obj206:head) 2026-02-11T23:31:27.806+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.806+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:e05d79f1:::obj206:head) v3 finish latency 0.000010 2026-02-11T23:31:27.806+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e05d79f1:::obj206:head local-lis/les=113/114 n=535 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.806+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.806+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.806+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e05d79f1:::obj206:head local-lis/les=113/114 n=535 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.806+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.806+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.806+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.806+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5f1c780 con 0x4acb000 2026-02-11T23:31:27.806+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:e0a44829:::obj14:head)]) -- 0x5fd2c00 con 0x4acb000 2026-02-11T23:31:27.806+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.806+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1107 ==== MOSDPGPush(1.0 116/113 [PushOp(1:e10533c6:::obj450:head, version: 111'450, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e10533c6:::obj450:head@111'450, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559dd40 con 0x4acb000 2026-02-11T23:31:27.806+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:e10533c6:::obj450:head, version: 111'450, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e10533c6:::obj450:head@111'450, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.806+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e10533c6:::obj450:head, version: 111'450, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e10533c6:::obj450:head@111'450, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.806+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.806+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1108 ==== pg_backfill(progress 1.0 e 116/116 lb 1:e0a44829:::obj14:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d99080 con 0x4acb000 2026-02-11T23:31:27.806+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:e0a44829:::obj14:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.806+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e0a44829:::obj14:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.806+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e10533c6:::obj450:head, version: 111'450, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e10533c6:::obj450:head@111'450, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.806+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.806+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e10533c6:::obj450:head, version: 111'450, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e10533c6:::obj450:head@111'450, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.806+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:e10533c6:::obj450:head, version: 111'450, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e10533c6:::obj450:head@111'450, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(1.0 116/113 [PushOp(1:e10533c6:::obj450:head, version: 111'450, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e10533c6:::obj450:head@111'450, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:e05d79f1:::obj206:head local-lis/les=113/114 n=535 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.806+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.806+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.806+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e05d79f1:::obj206:head local-lis/les=113/114 n=535 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:e10533c6:::obj450:head, version: 111'450, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e10533c6:::obj450:head@111'450, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.806+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e0a44829:::obj14:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.806+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e05d79f1:::obj206:head local-lis/les=113/114 n=535 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:e10533c6:::obj450:head@111'450, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.806+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e05d79f1:::obj206:head local-lis/les=113/114 n=535 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:e10533c6:::obj450:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.806+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e05d79f1:::obj206:head local-lis/les=113/114 n=535 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:e10533c6:::obj450:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.806+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e05d79f1:::obj206:head local-lis/les=113/114 n=535 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:e10533c6:::obj450:head 2026-02-11T23:31:27.806+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e05d79f1:::obj206:head local-lis/les=113/114 n=535 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:e10533c6:::obj450:head v 111'450 2026-02-11T23:31:27.806+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e05d79f1:::obj206:head local-lis/les=113/114 n=535 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.806+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.806+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:e10533c6:::obj450:head, version: 111'450, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e10533c6:::obj450:head@111'450, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.806+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.806+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.806+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e0a44829:::obj14:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.806+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:e0a44829:::obj14:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 1.0 e 116/116 lb 1:e0a44829:::obj14:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:e05d79f1:::obj206:head local-lis/les=113/114 n=535 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.806+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.806+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e05d79f1:::obj206:head local-lis/les=113/114 n=535 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:e0a44829:::obj14:head) v3 2026-02-11T23:31:27.806+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e05d79f1:::obj206:head local-lis/les=113/114 n=535 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:e0a44829:::obj14:head) 2026-02-11T23:31:27.806+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.806+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:e0a44829:::obj14:head) v3 finish latency 0.000009 2026-02-11T23:31:27.806+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e0a44829:::obj14:head local-lis/les=113/114 n=536 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.806+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.806+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.806+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e0a44829:::obj14:head local-lis/les=113/114 n=536 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.806+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.806+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.806+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.806+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fec780 con 0x4acb000 2026-02-11T23:31:27.806+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:e10533c6:::obj450:head)]) -- 0x5f2c000 con 0x4acb000 2026-02-11T23:31:27.806+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.807+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1109 ==== MOSDPGPush(1.0 116/113 [PushOp(1:e196274b:::obj304:head, version: 111'304, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e196274b:::obj304:head@111'304, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c820 con 0x4acb000 2026-02-11T23:31:27.807+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:e196274b:::obj304:head, version: 111'304, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e196274b:::obj304:head@111'304, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.807+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e196274b:::obj304:head, version: 111'304, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e196274b:::obj304:head@111'304, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.807+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.807+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1110 ==== pg_backfill(progress 1.0 e 116/116 lb 1:e10533c6:::obj450:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99600 con 0x4acb000 2026-02-11T23:31:27.807+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:e10533c6:::obj450:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.807+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e196274b:::obj304:head, version: 111'304, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e196274b:::obj304:head@111'304, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.807+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e10533c6:::obj450:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.807+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.807+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e196274b:::obj304:head, version: 111'304, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e196274b:::obj304:head@111'304, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.807+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:e196274b:::obj304:head, version: 111'304, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e196274b:::obj304:head@111'304, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:e196274b:::obj304:head, version: 111'304, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e196274b:::obj304:head@111'304, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:e0a44829:::obj14:head local-lis/les=113/114 n=536 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.807+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.807+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.807+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e0a44829:::obj14:head local-lis/les=113/114 n=536 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:e196274b:::obj304:head, version: 111'304, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e196274b:::obj304:head@111'304, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.807+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e10533c6:::obj450:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.807+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e0a44829:::obj14:head local-lis/les=113/114 n=536 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:e196274b:::obj304:head@111'304, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.807+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e0a44829:::obj14:head local-lis/les=113/114 n=536 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:e196274b:::obj304:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.807+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e0a44829:::obj14:head local-lis/les=113/114 n=536 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:e196274b:::obj304:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.807+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e0a44829:::obj14:head local-lis/les=113/114 n=536 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:e196274b:::obj304:head 2026-02-11T23:31:27.807+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e0a44829:::obj14:head local-lis/les=113/114 n=536 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:e196274b:::obj304:head v 111'304 2026-02-11T23:31:27.807+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e0a44829:::obj14:head local-lis/les=113/114 n=536 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.807+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.807+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:e196274b:::obj304:head, version: 111'304, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e196274b:::obj304:head@111'304, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:27.807+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.807+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.807+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e10533c6:::obj450:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.807+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:e10533c6:::obj450:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:e10533c6:::obj450:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:e0a44829:::obj14:head local-lis/les=113/114 n=536 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.807+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.807+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e0a44829:::obj14:head local-lis/les=113/114 n=536 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:e10533c6:::obj450:head) v3 2026-02-11T23:31:27.807+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e0a44829:::obj14:head local-lis/les=113/114 n=536 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:e10533c6:::obj450:head) 2026-02-11T23:31:27.807+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.807+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:e10533c6:::obj450:head) v3 finish latency 0.000011 2026-02-11T23:31:27.807+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e10533c6:::obj450:head local-lis/les=113/114 n=537 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.807+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.807+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.807+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e10533c6:::obj450:head local-lis/les=113/114 n=537 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.807+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.807+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.807+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.807+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5f1c600 con 0x4acb000 2026-02-11T23:31:27.807+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:e196274b:::obj304:head)]) -- 0x5f2c300 con 0x4acb000 2026-02-11T23:31:27.807+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.807+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1111 ==== MOSDPGPush(1.0 116/113 [PushOp(1:e1d56785:::obj141:head, version: 111'141, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e1d56785:::obj141:head@111'141, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f1ba0 con 0x4acb000 2026-02-11T23:31:27.807+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:e1d56785:::obj141:head, version: 111'141, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e1d56785:::obj141:head@111'141, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.807+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e1d56785:::obj141:head, version: 111'141, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e1d56785:::obj141:head@111'141, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.807+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.807+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1112 ==== pg_backfill(progress 1.0 e 116/116 lb 1:e196274b:::obj304:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99b80 con 0x4acb000 2026-02-11T23:31:27.807+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:e196274b:::obj304:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.807+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e1d56785:::obj141:head, version: 111'141, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e1d56785:::obj141:head@111'141, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.807+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e196274b:::obj304:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.807+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.807+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e1d56785:::obj141:head, version: 111'141, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e1d56785:::obj141:head@111'141, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.807+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:e1d56785:::obj141:head, version: 111'141, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e1d56785:::obj141:head@111'141, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:e1d56785:::obj141:head, version: 111'141, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e1d56785:::obj141:head@111'141, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:e10533c6:::obj450:head local-lis/les=113/114 n=537 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.807+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.807+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.807+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e196274b:::obj304:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.807+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e10533c6:::obj450:head local-lis/les=113/114 n=537 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:e1d56785:::obj141:head, version: 111'141, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e1d56785:::obj141:head@111'141, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.807+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e10533c6:::obj450:head local-lis/les=113/114 n=537 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:e1d56785:::obj141:head@111'141, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.807+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e10533c6:::obj450:head local-lis/les=113/114 n=537 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:e1d56785:::obj141:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.807+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e10533c6:::obj450:head local-lis/les=113/114 n=537 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:e1d56785:::obj141:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.807+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e10533c6:::obj450:head local-lis/les=113/114 n=537 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:e1d56785:::obj141:head 2026-02-11T23:31:27.807+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e10533c6:::obj450:head local-lis/les=113/114 n=537 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:e1d56785:::obj141:head v 111'141 2026-02-11T23:31:27.807+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e10533c6:::obj450:head local-lis/les=113/114 n=537 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.807+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.807+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:e1d56785:::obj141:head, version: 111'141, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e1d56785:::obj141:head@111'141, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000044 2026-02-11T23:31:27.807+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.807+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.807+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e196274b:::obj304:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.807+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:e196274b:::obj304:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 1.0 e 116/116 lb 1:e196274b:::obj304:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:e10533c6:::obj450:head local-lis/les=113/114 n=537 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.807+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.807+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e10533c6:::obj450:head local-lis/les=113/114 n=537 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:e196274b:::obj304:head) v3 2026-02-11T23:31:27.807+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e10533c6:::obj450:head local-lis/les=113/114 n=537 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:e196274b:::obj304:head) 2026-02-11T23:31:27.807+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.807+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:e196274b:::obj304:head) v3 finish latency 0.000010 2026-02-11T23:31:27.807+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e196274b:::obj304:head local-lis/les=113/114 n=538 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.807+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.807+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.807+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e196274b:::obj304:head local-lis/les=113/114 n=538 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.807+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.807+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.807+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.807+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e6b980 con 0x4acb000 2026-02-11T23:31:27.807+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:e1d56785:::obj141:head)]) -- 0x5f2c600 con 0x4acb000 2026-02-11T23:31:27.807+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.808+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1113 ==== MOSDPGPush(1.0 116/113 [PushOp(1:e25e0189:::obj579:head, version: 111'579, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e25e0189:::obj579:head@111'579, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f0b60 con 0x4acb000 2026-02-11T23:31:27.808+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:e25e0189:::obj579:head, version: 111'579, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e25e0189:::obj579:head@111'579, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.808+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e25e0189:::obj579:head, version: 111'579, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e25e0189:::obj579:head@111'579, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.808+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.808+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1114 ==== pg_backfill(progress 1.0 e 116/116 lb 1:e1d56785:::obj141:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9a100 con 0x4acb000 2026-02-11T23:31:27.808+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e25e0189:::obj579:head, version: 111'579, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e25e0189:::obj579:head@111'579, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.808+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:e1d56785:::obj141:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.808+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e1d56785:::obj141:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.808+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.808+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e25e0189:::obj579:head, version: 111'579, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e25e0189:::obj579:head@111'579, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.808+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:e25e0189:::obj579:head, version: 111'579, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e25e0189:::obj579:head@111'579, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:e25e0189:::obj579:head, version: 111'579, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e25e0189:::obj579:head@111'579, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:e196274b:::obj304:head local-lis/les=113/114 n=538 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.808+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.808+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.808+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e196274b:::obj304:head local-lis/les=113/114 n=538 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:e25e0189:::obj579:head, version: 111'579, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e25e0189:::obj579:head@111'579, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.808+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e1d56785:::obj141:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.808+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e196274b:::obj304:head local-lis/les=113/114 n=538 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:e25e0189:::obj579:head@111'579, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.808+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e196274b:::obj304:head local-lis/les=113/114 n=538 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:e25e0189:::obj579:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.808+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e196274b:::obj304:head local-lis/les=113/114 n=538 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:e25e0189:::obj579:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.808+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e196274b:::obj304:head local-lis/les=113/114 n=538 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:e25e0189:::obj579:head 2026-02-11T23:31:27.808+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e196274b:::obj304:head local-lis/les=113/114 n=538 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:e25e0189:::obj579:head v 111'579 2026-02-11T23:31:27.808+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e196274b:::obj304:head local-lis/les=113/114 n=538 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.808+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.808+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:e25e0189:::obj579:head, version: 111'579, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e25e0189:::obj579:head@111'579, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:27.808+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.808+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.808+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e1d56785:::obj141:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.808+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:e1d56785:::obj141:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:e1d56785:::obj141:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:e196274b:::obj304:head local-lis/les=113/114 n=538 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.808+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.808+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e196274b:::obj304:head local-lis/les=113/114 n=538 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:e1d56785:::obj141:head) v3 2026-02-11T23:31:27.808+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e196274b:::obj304:head local-lis/les=113/114 n=538 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:e1d56785:::obj141:head) 2026-02-11T23:31:27.808+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.808+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:e1d56785:::obj141:head) v3 finish latency 0.000009 2026-02-11T23:31:27.808+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e1d56785:::obj141:head local-lis/les=113/114 n=539 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.808+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.808+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.808+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e1d56785:::obj141:head local-lis/les=113/114 n=539 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.808+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.808+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.808+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.808+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5f1c480 con 0x4acb000 2026-02-11T23:31:27.808+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:e25e0189:::obj579:head)]) -- 0x5f2c900 con 0x4acb000 2026-02-11T23:31:27.808+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.808+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1115 ==== MOSDPGPush(1.0 116/113 [PushOp(1:e26151b8:::obj492:head, version: 111'492, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e26151b8:::obj492:head@111'492, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559cb60 con 0x4acb000 2026-02-11T23:31:27.808+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:e26151b8:::obj492:head, version: 111'492, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e26151b8:::obj492:head@111'492, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.808+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e26151b8:::obj492:head, version: 111'492, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e26151b8:::obj492:head@111'492, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.808+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.808+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1116 ==== pg_backfill(progress 1.0 e 116/116 lb 1:e25e0189:::obj579:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9ac00 con 0x4acb000 2026-02-11T23:31:27.808+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e26151b8:::obj492:head, version: 111'492, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e26151b8:::obj492:head@111'492, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.808+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:e25e0189:::obj579:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.808+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e25e0189:::obj579:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.808+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.808+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e26151b8:::obj492:head, version: 111'492, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e26151b8:::obj492:head@111'492, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.808+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.808+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:e26151b8:::obj492:head, version: 111'492, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e26151b8:::obj492:head@111'492, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 116/113 [PushOp(1:e26151b8:::obj492:head, version: 111'492, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e26151b8:::obj492:head@111'492, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:e1d56785:::obj141:head local-lis/les=113/114 n=539 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.808+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.808+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e1d56785:::obj141:head local-lis/les=113/114 n=539 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:e26151b8:::obj492:head, version: 111'492, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e26151b8:::obj492:head@111'492, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.808+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e1d56785:::obj141:head local-lis/les=113/114 n=539 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:e26151b8:::obj492:head@111'492, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.808+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.808+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e25e0189:::obj579:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.808+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e1d56785:::obj141:head local-lis/les=113/114 n=539 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:e26151b8:::obj492:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.808+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e1d56785:::obj141:head local-lis/les=113/114 n=539 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:e26151b8:::obj492:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.808+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e1d56785:::obj141:head local-lis/les=113/114 n=539 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:e26151b8:::obj492:head 2026-02-11T23:31:27.808+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e1d56785:::obj141:head local-lis/les=113/114 n=539 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:e26151b8:::obj492:head v 111'492 2026-02-11T23:31:27.808+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e1d56785:::obj141:head local-lis/les=113/114 n=539 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.808+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.808+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:e26151b8:::obj492:head, version: 111'492, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e26151b8:::obj492:head@111'492, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:27.808+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.808+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.808+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e25e0189:::obj579:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.808+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:e25e0189:::obj579:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 1.0 e 116/116 lb 1:e25e0189:::obj579:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:e1d56785:::obj141:head local-lis/les=113/114 n=539 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.808+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.808+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e1d56785:::obj141:head local-lis/les=113/114 n=539 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:e25e0189:::obj579:head) v3 2026-02-11T23:31:27.808+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e1d56785:::obj141:head local-lis/les=113/114 n=539 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:e25e0189:::obj579:head) 2026-02-11T23:31:27.808+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.808+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:e25e0189:::obj579:head) v3 finish latency 0.000009 2026-02-11T23:31:27.808+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e25e0189:::obj579:head local-lis/les=113/114 n=540 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.808+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.809+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.809+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e25e0189:::obj579:head local-lis/les=113/114 n=540 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.809+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.809+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.809+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.809+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x600a480 con 0x4acb000 2026-02-11T23:31:27.809+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:e26151b8:::obj492:head)]) -- 0x5f2cc00 con 0x4acb000 2026-02-11T23:31:27.809+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.809+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1117 ==== MOSDPGPush(1.0 116/113 [PushOp(1:e3cd6c93:::obj259:head, version: 111'259, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e3cd6c93:::obj259:head@111'259, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e26680 con 0x4acb000 2026-02-11T23:31:27.809+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:e3cd6c93:::obj259:head, version: 111'259, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e3cd6c93:::obj259:head@111'259, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.809+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e3cd6c93:::obj259:head, version: 111'259, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e3cd6c93:::obj259:head@111'259, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.809+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.809+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1118 ==== pg_backfill(progress 1.0 e 116/116 lb 1:e26151b8:::obj492:head) ==== 1062+0+0 (unknown 0 0 0) 0x4a2cb00 con 0x4acb000 2026-02-11T23:31:27.809+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e3cd6c93:::obj259:head, version: 111'259, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e3cd6c93:::obj259:head@111'259, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.809+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:e26151b8:::obj492:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.809+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e26151b8:::obj492:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.809+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.809+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e3cd6c93:::obj259:head, version: 111'259, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e3cd6c93:::obj259:head@111'259, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.809+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:e3cd6c93:::obj259:head, version: 111'259, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e3cd6c93:::obj259:head@111'259, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:e3cd6c93:::obj259:head, version: 111'259, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e3cd6c93:::obj259:head@111'259, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:e25e0189:::obj579:head local-lis/les=113/114 n=540 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.809+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.809+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e25e0189:::obj579:head local-lis/les=113/114 n=540 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:e3cd6c93:::obj259:head, version: 111'259, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e3cd6c93:::obj259:head@111'259, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.809+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.809+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e26151b8:::obj492:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.809+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e25e0189:::obj579:head local-lis/les=113/114 n=540 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:e3cd6c93:::obj259:head@111'259, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.809+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e25e0189:::obj579:head local-lis/les=113/114 n=540 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:e3cd6c93:::obj259:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.809+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e25e0189:::obj579:head local-lis/les=113/114 n=540 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:e3cd6c93:::obj259:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.809+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e25e0189:::obj579:head local-lis/les=113/114 n=540 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:e3cd6c93:::obj259:head 2026-02-11T23:31:27.809+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e25e0189:::obj579:head local-lis/les=113/114 n=540 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:e3cd6c93:::obj259:head v 111'259 2026-02-11T23:31:27.809+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e25e0189:::obj579:head local-lis/les=113/114 n=540 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.809+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.809+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:e3cd6c93:::obj259:head, version: 111'259, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e3cd6c93:::obj259:head@111'259, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.809+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.809+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.809+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e26151b8:::obj492:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.809+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:e26151b8:::obj492:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 1.0 e 116/116 lb 1:e26151b8:::obj492:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:e25e0189:::obj579:head local-lis/les=113/114 n=540 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.809+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.809+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e25e0189:::obj579:head local-lis/les=113/114 n=540 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:e26151b8:::obj492:head) v3 2026-02-11T23:31:27.809+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e25e0189:::obj579:head local-lis/les=113/114 n=540 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:e26151b8:::obj492:head) 2026-02-11T23:31:27.809+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.809+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:e26151b8:::obj492:head) v3 finish latency 0.000009 2026-02-11T23:31:27.809+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e26151b8:::obj492:head local-lis/les=113/114 n=541 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.809+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.809+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.809+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e26151b8:::obj492:head local-lis/les=113/114 n=541 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.809+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.809+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.809+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.809+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5f1c300 con 0x4acb000 2026-02-11T23:31:27.809+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:e3cd6c93:::obj259:head)]) -- 0x5f2cf00 con 0x4acb000 2026-02-11T23:31:27.809+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.809+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1119 ==== MOSDPGPush(1.0 116/113 [PushOp(1:e4109927:::obj482:head, version: 111'482, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e4109927:::obj482:head@111'482, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5647380 con 0x4acb000 2026-02-11T23:31:27.809+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:e4109927:::obj482:head, version: 111'482, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e4109927:::obj482:head@111'482, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.809+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e4109927:::obj482:head, version: 111'482, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e4109927:::obj482:head@111'482, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.809+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.809+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1120 ==== pg_backfill(progress 1.0 e 116/116 lb 1:e3cd6c93:::obj259:head) ==== 1062+0+0 (unknown 0 0 0) 0x4a2c000 con 0x4acb000 2026-02-11T23:31:27.809+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:e3cd6c93:::obj259:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.809+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e4109927:::obj482:head, version: 111'482, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e4109927:::obj482:head@111'482, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.809+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e3cd6c93:::obj259:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.809+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.809+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e4109927:::obj482:head, version: 111'482, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e4109927:::obj482:head@111'482, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.809+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:e4109927:::obj482:head, version: 111'482, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e4109927:::obj482:head@111'482, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 116/113 [PushOp(1:e4109927:::obj482:head, version: 111'482, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e4109927:::obj482:head@111'482, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:e26151b8:::obj492:head local-lis/les=113/114 n=541 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.809+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.809+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.809+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e26151b8:::obj492:head local-lis/les=113/114 n=541 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:e4109927:::obj482:head, version: 111'482, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e4109927:::obj482:head@111'482, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.809+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e3cd6c93:::obj259:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.809+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e26151b8:::obj492:head local-lis/les=113/114 n=541 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:e4109927:::obj482:head@111'482, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.809+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e26151b8:::obj492:head local-lis/les=113/114 n=541 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:e4109927:::obj482:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.809+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e26151b8:::obj492:head local-lis/les=113/114 n=541 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:e4109927:::obj482:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.809+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e26151b8:::obj492:head local-lis/les=113/114 n=541 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:e4109927:::obj482:head 2026-02-11T23:31:27.809+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e26151b8:::obj492:head local-lis/les=113/114 n=541 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:e4109927:::obj482:head v 111'482 2026-02-11T23:31:27.809+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e26151b8:::obj492:head local-lis/les=113/114 n=541 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.809+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.810+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:e4109927:::obj482:head, version: 111'482, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e4109927:::obj482:head@111'482, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:27.810+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.810+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.810+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e3cd6c93:::obj259:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.810+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:e3cd6c93:::obj259:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 1.0 e 116/116 lb 1:e3cd6c93:::obj259:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:e26151b8:::obj492:head local-lis/les=113/114 n=541 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.810+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.810+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e26151b8:::obj492:head local-lis/les=113/114 n=541 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:e3cd6c93:::obj259:head) v3 2026-02-11T23:31:27.810+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e26151b8:::obj492:head local-lis/les=113/114 n=541 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:e3cd6c93:::obj259:head) 2026-02-11T23:31:27.810+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.810+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:e3cd6c93:::obj259:head) v3 finish latency 0.000009 2026-02-11T23:31:27.810+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e3cd6c93:::obj259:head local-lis/les=113/114 n=542 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.810+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.810+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.810+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e3cd6c93:::obj259:head local-lis/les=113/114 n=542 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.810+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.810+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.810+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.810+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6170300 con 0x4acb000 2026-02-11T23:31:27.810+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:e4109927:::obj482:head)]) -- 0x5f2d200 con 0x4acb000 2026-02-11T23:31:27.810+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.810+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1121 ==== MOSDPGPush(1.0 116/113 [PushOp(1:e43ffde6:::obj285:head, version: 111'285, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e43ffde6:::obj285:head@111'285, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3815a00 con 0x4acb000 2026-02-11T23:31:27.810+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:e43ffde6:::obj285:head, version: 111'285, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e43ffde6:::obj285:head@111'285, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 116 2026-02-11T23:31:27.810+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e43ffde6:::obj285:head, version: 111'285, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e43ffde6:::obj285:head@111'285, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.810+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.810+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1122 ==== pg_backfill(progress 1.0 e 116/116 lb 1:e4109927:::obj482:head) ==== 1062+0+0 (unknown 0 0 0) 0x5982b00 con 0x4acb000 2026-02-11T23:31:27.810+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:e4109927:::obj482:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.810+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e43ffde6:::obj285:head, version: 111'285, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e43ffde6:::obj285:head@111'285, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.810+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e4109927:::obj482:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.810+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.810+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e43ffde6:::obj285:head, version: 111'285, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e43ffde6:::obj285:head@111'285, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.810+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:e43ffde6:::obj285:head, version: 111'285, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e43ffde6:::obj285:head@111'285, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 116/113 [PushOp(1:e43ffde6:::obj285:head, version: 111'285, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e43ffde6:::obj285:head@111'285, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:e3cd6c93:::obj259:head local-lis/les=113/114 n=542 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.810+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.810+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.810+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e3cd6c93:::obj259:head local-lis/les=113/114 n=542 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:e43ffde6:::obj285:head, version: 111'285, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e43ffde6:::obj285:head@111'285, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.810+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e4109927:::obj482:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.810+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e3cd6c93:::obj259:head local-lis/les=113/114 n=542 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:e43ffde6:::obj285:head@111'285, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.810+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e3cd6c93:::obj259:head local-lis/les=113/114 n=542 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:e43ffde6:::obj285:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.810+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e3cd6c93:::obj259:head local-lis/les=113/114 n=542 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:e43ffde6:::obj285:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.810+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e3cd6c93:::obj259:head local-lis/les=113/114 n=542 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:e43ffde6:::obj285:head 2026-02-11T23:31:27.810+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e3cd6c93:::obj259:head local-lis/les=113/114 n=542 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:e43ffde6:::obj285:head v 111'285 2026-02-11T23:31:27.810+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e3cd6c93:::obj259:head local-lis/les=113/114 n=542 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.810+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.810+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:e43ffde6:::obj285:head, version: 111'285, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e43ffde6:::obj285:head@111'285, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.810+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.810+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.810+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e4109927:::obj482:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.810+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:e4109927:::obj482:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 1.0 e 116/116 lb 1:e4109927:::obj482:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:e3cd6c93:::obj259:head local-lis/les=113/114 n=542 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.810+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.810+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e3cd6c93:::obj259:head local-lis/les=113/114 n=542 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:e4109927:::obj482:head) v3 2026-02-11T23:31:27.810+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e3cd6c93:::obj259:head local-lis/les=113/114 n=542 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:e4109927:::obj482:head) 2026-02-11T23:31:27.810+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.810+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:e4109927:::obj482:head) v3 finish latency 0.000010 2026-02-11T23:31:27.810+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e4109927:::obj482:head local-lis/les=113/114 n=543 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.810+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.810+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.810+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e4109927:::obj482:head local-lis/les=113/114 n=543 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.810+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.810+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.810+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.810+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5f1c180 con 0x4acb000 2026-02-11T23:31:27.810+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:e43ffde6:::obj285:head)]) -- 0x5f2d500 con 0x4acb000 2026-02-11T23:31:27.810+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.810+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1123 ==== MOSDPGPush(1.0 116/113 [PushOp(1:e46d208d:::obj89:head, version: 111'89, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e46d208d:::obj89:head@111'89, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5e27040 con 0x4acb000 2026-02-11T23:31:27.810+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:e46d208d:::obj89:head, version: 111'89, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e46d208d:::obj89:head@111'89, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.810+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e46d208d:::obj89:head, version: 111'89, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e46d208d:::obj89:head@111'89, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.811+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.811+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1124 ==== pg_backfill(progress 1.0 e 116/116 lb 1:e43ffde6:::obj285:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99b80 con 0x4acb000 2026-02-11T23:31:27.811+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e46d208d:::obj89:head, version: 111'89, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e46d208d:::obj89:head@111'89, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.811+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:e43ffde6:::obj285:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.811+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.811+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e43ffde6:::obj285:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.811+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e46d208d:::obj89:head, version: 111'89, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e46d208d:::obj89:head@111'89, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.811+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.811+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:e46d208d:::obj89:head, version: 111'89, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e46d208d:::obj89:head@111'89, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 116/113 [PushOp(1:e46d208d:::obj89:head, version: 111'89, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e46d208d:::obj89:head@111'89, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:e4109927:::obj482:head local-lis/les=113/114 n=543 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.811+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.811+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e4109927:::obj482:head local-lis/les=113/114 n=543 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:e46d208d:::obj89:head, version: 111'89, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e46d208d:::obj89:head@111'89, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.811+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e4109927:::obj482:head local-lis/les=113/114 n=543 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:e46d208d:::obj89:head@111'89, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.811+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.811+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e43ffde6:::obj285:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.811+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e4109927:::obj482:head local-lis/les=113/114 n=543 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:e46d208d:::obj89:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.811+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e4109927:::obj482:head local-lis/les=113/114 n=543 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:e46d208d:::obj89:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.811+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e4109927:::obj482:head local-lis/les=113/114 n=543 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:e46d208d:::obj89:head 2026-02-11T23:31:27.811+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e4109927:::obj482:head local-lis/les=113/114 n=543 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:e46d208d:::obj89:head v 111'89 2026-02-11T23:31:27.811+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e4109927:::obj482:head local-lis/les=113/114 n=543 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.811+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.811+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:e46d208d:::obj89:head, version: 111'89, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e46d208d:::obj89:head@111'89, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:27.811+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.811+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.811+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e43ffde6:::obj285:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.811+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:e43ffde6:::obj285:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 1.0 e 116/116 lb 1:e43ffde6:::obj285:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:e4109927:::obj482:head local-lis/les=113/114 n=543 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.811+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.811+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e4109927:::obj482:head local-lis/les=113/114 n=543 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:e43ffde6:::obj285:head) v3 2026-02-11T23:31:27.811+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e4109927:::obj482:head local-lis/les=113/114 n=543 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:e43ffde6:::obj285:head) 2026-02-11T23:31:27.811+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.811+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:e43ffde6:::obj285:head) v3 finish latency 0.000008 2026-02-11T23:31:27.811+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e43ffde6:::obj285:head local-lis/les=113/114 n=544 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.811+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.811+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.811+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e43ffde6:::obj285:head local-lis/les=113/114 n=544 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.811+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.811+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.811+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.811+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x612ed80 con 0x4acb000 2026-02-11T23:31:27.811+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:e46d208d:::obj89:head)]) -- 0x5f2d800 con 0x4acb000 2026-02-11T23:31:27.811+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.811+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1125 ==== MOSDPGPush(1.0 116/113 [PushOp(1:e48b6d56:::obj389:head, version: 111'389, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e48b6d56:::obj389:head@111'389, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559da00 con 0x4acb000 2026-02-11T23:31:27.811+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:e48b6d56:::obj389:head, version: 111'389, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e48b6d56:::obj389:head@111'389, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.811+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e48b6d56:::obj389:head, version: 111'389, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e48b6d56:::obj389:head@111'389, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.811+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.811+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1126 ==== pg_backfill(progress 1.0 e 116/116 lb 1:e46d208d:::obj89:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d99600 con 0x4acb000 2026-02-11T23:31:27.811+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:e46d208d:::obj89:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.811+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e48b6d56:::obj389:head, version: 111'389, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e48b6d56:::obj389:head@111'389, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.811+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e46d208d:::obj89:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.811+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.811+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e48b6d56:::obj389:head, version: 111'389, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e48b6d56:::obj389:head@111'389, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.811+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:e48b6d56:::obj389:head, version: 111'389, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e48b6d56:::obj389:head@111'389, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 116/113 [PushOp(1:e48b6d56:::obj389:head, version: 111'389, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e48b6d56:::obj389:head@111'389, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:e43ffde6:::obj285:head local-lis/les=113/114 n=544 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.811+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.811+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.811+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e43ffde6:::obj285:head local-lis/les=113/114 n=544 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:e48b6d56:::obj389:head, version: 111'389, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e48b6d56:::obj389:head@111'389, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.811+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e46d208d:::obj89:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.811+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e43ffde6:::obj285:head local-lis/les=113/114 n=544 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:e48b6d56:::obj389:head@111'389, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.811+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e43ffde6:::obj285:head local-lis/les=113/114 n=544 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:e48b6d56:::obj389:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.811+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e43ffde6:::obj285:head local-lis/les=113/114 n=544 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:e48b6d56:::obj389:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.811+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e43ffde6:::obj285:head local-lis/les=113/114 n=544 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:e48b6d56:::obj389:head 2026-02-11T23:31:27.811+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e43ffde6:::obj285:head local-lis/les=113/114 n=544 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:e48b6d56:::obj389:head v 111'389 2026-02-11T23:31:27.811+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e43ffde6:::obj285:head local-lis/les=113/114 n=544 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.811+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.811+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:e48b6d56:::obj389:head, version: 111'389, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e48b6d56:::obj389:head@111'389, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.811+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.811+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.811+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e46d208d:::obj89:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.811+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:e46d208d:::obj89:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 116/116 lb 1:e46d208d:::obj89:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:e43ffde6:::obj285:head local-lis/les=113/114 n=544 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.811+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.811+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e43ffde6:::obj285:head local-lis/les=113/114 n=544 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:e46d208d:::obj89:head) v3 2026-02-11T23:31:27.811+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e43ffde6:::obj285:head local-lis/les=113/114 n=544 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:e46d208d:::obj89:head) 2026-02-11T23:31:27.811+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.811+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:e46d208d:::obj89:head) v3 finish latency 0.000010 2026-02-11T23:31:27.811+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e46d208d:::obj89:head local-lis/les=113/114 n=545 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.811+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.811+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.811+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e46d208d:::obj89:head local-lis/les=113/114 n=545 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.811+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.811+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.811+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.811+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5f1c000 con 0x4acb000 2026-02-11T23:31:27.811+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:e48b6d56:::obj389:head)]) -- 0x5f2d980 con 0x4acb000 2026-02-11T23:31:27.811+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.812+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1127 ==== MOSDPGPush(1.0 116/113 [PushOp(1:e5124c98:::obj133:head, version: 111'133, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e5124c98:::obj133:head@111'133, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559cea0 con 0x4acb000 2026-02-11T23:31:27.812+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:e5124c98:::obj133:head, version: 111'133, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e5124c98:::obj133:head@111'133, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.812+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e5124c98:::obj133:head, version: 111'133, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e5124c98:::obj133:head@111'133, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.812+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.812+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1128 ==== pg_backfill(progress 1.0 e 116/116 lb 1:e48b6d56:::obj389:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99080 con 0x4acb000 2026-02-11T23:31:27.812+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:e48b6d56:::obj389:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.812+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e5124c98:::obj133:head, version: 111'133, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e5124c98:::obj133:head@111'133, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.812+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e48b6d56:::obj389:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.812+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.812+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e5124c98:::obj133:head, version: 111'133, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e5124c98:::obj133:head@111'133, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.812+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:e5124c98:::obj133:head, version: 111'133, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e5124c98:::obj133:head@111'133, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:e5124c98:::obj133:head, version: 111'133, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e5124c98:::obj133:head@111'133, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:e46d208d:::obj89:head local-lis/les=113/114 n=545 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.812+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.812+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.812+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e46d208d:::obj89:head local-lis/les=113/114 n=545 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:e5124c98:::obj133:head, version: 111'133, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e5124c98:::obj133:head@111'133, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.812+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e48b6d56:::obj389:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.812+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e46d208d:::obj89:head local-lis/les=113/114 n=545 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:e5124c98:::obj133:head@111'133, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.812+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e46d208d:::obj89:head local-lis/les=113/114 n=545 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:e5124c98:::obj133:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.812+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e46d208d:::obj89:head local-lis/les=113/114 n=545 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:e5124c98:::obj133:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.812+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e46d208d:::obj89:head local-lis/les=113/114 n=545 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:e5124c98:::obj133:head 2026-02-11T23:31:27.812+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e46d208d:::obj89:head local-lis/les=113/114 n=545 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:e5124c98:::obj133:head v 111'133 2026-02-11T23:31:27.812+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e46d208d:::obj89:head local-lis/les=113/114 n=545 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.812+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.812+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:e5124c98:::obj133:head, version: 111'133, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e5124c98:::obj133:head@111'133, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:27.812+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e46d208d:::obj89:head local-lis/les=113/114 n=545 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.812+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.812+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.812+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e48b6d56:::obj389:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.812+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:e48b6d56:::obj389:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 1.0 e 116/116 lb 1:e48b6d56:::obj389:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:e46d208d:::obj89:head local-lis/les=113/114 n=545 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.812+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.812+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e46d208d:::obj89:head local-lis/les=113/114 n=545 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:e48b6d56:::obj389:head) v3 2026-02-11T23:31:27.812+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e46d208d:::obj89:head local-lis/les=113/114 n=545 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:e48b6d56:::obj389:head) 2026-02-11T23:31:27.812+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.812+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:e48b6d56:::obj389:head) v3 finish latency 0.000010 2026-02-11T23:31:27.812+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.812+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.812+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e48b6d56:::obj389:head local-lis/les=113/114 n=546 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.812+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.812+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.812+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.812+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x600b800 con 0x4acb000 2026-02-11T23:31:27.812+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:e5124c98:::obj133:head)]) -- 0x6171800 con 0x4acb000 2026-02-11T23:31:27.812+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.812+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1129 ==== MOSDPGPush(1.0 116/113 [PushOp(1:e6508d05:::obj595:head, version: 111'595, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e6508d05:::obj595:head@111'595, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d8820 con 0x4acb000 2026-02-11T23:31:27.812+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:e6508d05:::obj595:head, version: 111'595, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e6508d05:::obj595:head@111'595, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 116 2026-02-11T23:31:27.812+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e6508d05:::obj595:head, version: 111'595, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e6508d05:::obj595:head@111'595, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.812+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.812+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1130 ==== pg_backfill(progress 1.0 e 116/116 lb 1:e5124c98:::obj133:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98580 con 0x4acb000 2026-02-11T23:31:27.812+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e6508d05:::obj595:head, version: 111'595, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e6508d05:::obj595:head@111'595, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.812+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:e5124c98:::obj133:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.812+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.812+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e5124c98:::obj133:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.812+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e6508d05:::obj595:head, version: 111'595, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e6508d05:::obj595:head@111'595, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.812+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.812+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:e6508d05:::obj595:head, version: 111'595, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e6508d05:::obj595:head@111'595, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 116/113 [PushOp(1:e6508d05:::obj595:head, version: 111'595, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e6508d05:::obj595:head@111'595, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:e48b6d56:::obj389:head local-lis/les=113/114 n=546 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.812+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.812+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e48b6d56:::obj389:head local-lis/les=113/114 n=546 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:e6508d05:::obj595:head, version: 111'595, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e6508d05:::obj595:head@111'595, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.812+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e48b6d56:::obj389:head local-lis/les=113/114 n=546 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:e6508d05:::obj595:head@111'595, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.812+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.812+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e5124c98:::obj133:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.812+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e48b6d56:::obj389:head local-lis/les=113/114 n=546 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:e6508d05:::obj595:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.812+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e48b6d56:::obj389:head local-lis/les=113/114 n=546 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:e6508d05:::obj595:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.812+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e48b6d56:::obj389:head local-lis/les=113/114 n=546 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:e6508d05:::obj595:head 2026-02-11T23:31:27.812+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e48b6d56:::obj389:head local-lis/les=113/114 n=546 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:e6508d05:::obj595:head v 111'595 2026-02-11T23:31:27.812+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e48b6d56:::obj389:head local-lis/les=113/114 n=546 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.812+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.812+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:e6508d05:::obj595:head, version: 111'595, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e6508d05:::obj595:head@111'595, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000042 2026-02-11T23:31:27.812+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.812+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.812+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e5124c98:::obj133:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.812+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:e5124c98:::obj133:head) v3 prio 15 cost 0 latency 0.000057 pg_backfill(progress 1.0 e 116/116 lb 1:e5124c98:::obj133:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:e48b6d56:::obj389:head local-lis/les=113/114 n=546 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.812+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.812+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e48b6d56:::obj389:head local-lis/les=113/114 n=546 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:e5124c98:::obj133:head) v3 2026-02-11T23:31:27.812+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e48b6d56:::obj389:head local-lis/les=113/114 n=546 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:e5124c98:::obj133:head) 2026-02-11T23:31:27.812+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.812+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:e5124c98:::obj133:head) v3 finish latency 0.000012 2026-02-11T23:31:27.812+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e5124c98:::obj133:head local-lis/les=113/114 n=547 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.812+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.812+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.812+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e5124c98:::obj133:head local-lis/les=113/114 n=547 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.812+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.812+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.812+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.812+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5e7ed80 con 0x4acb000 2026-02-11T23:31:27.812+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:e6508d05:::obj595:head)]) -- 0x5f2dc80 con 0x4acb000 2026-02-11T23:31:27.812+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.813+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1131 ==== MOSDPGPush(1.0 116/113 [PushOp(1:e6e28dcd:::obj87:head, version: 111'87, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e6e28dcd:::obj87:head@111'87, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x55d9860 con 0x4acb000 2026-02-11T23:31:27.813+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:e6e28dcd:::obj87:head, version: 111'87, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e6e28dcd:::obj87:head@111'87, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.813+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e6e28dcd:::obj87:head, version: 111'87, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e6e28dcd:::obj87:head@111'87, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.813+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.813+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1132 ==== pg_backfill(progress 1.0 e 116/116 lb 1:e6508d05:::obj595:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98000 con 0x4acb000 2026-02-11T23:31:27.813+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:e6508d05:::obj595:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.813+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e6e28dcd:::obj87:head, version: 111'87, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e6e28dcd:::obj87:head@111'87, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.813+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e6508d05:::obj595:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.813+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.813+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e6e28dcd:::obj87:head, version: 111'87, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e6e28dcd:::obj87:head@111'87, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.813+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:e6e28dcd:::obj87:head, version: 111'87, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e6e28dcd:::obj87:head@111'87, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 116/113 [PushOp(1:e6e28dcd:::obj87:head, version: 111'87, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e6e28dcd:::obj87:head@111'87, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:e5124c98:::obj133:head local-lis/les=113/114 n=547 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.813+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.813+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e5124c98:::obj133:head local-lis/les=113/114 n=547 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:e6e28dcd:::obj87:head, version: 111'87, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e6e28dcd:::obj87:head@111'87, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.813+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.813+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e6508d05:::obj595:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.813+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e5124c98:::obj133:head local-lis/les=113/114 n=547 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:e6e28dcd:::obj87:head@111'87, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.813+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e5124c98:::obj133:head local-lis/les=113/114 n=547 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:e6e28dcd:::obj87:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.813+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e5124c98:::obj133:head local-lis/les=113/114 n=547 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:e6e28dcd:::obj87:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.813+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e5124c98:::obj133:head local-lis/les=113/114 n=547 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:e6e28dcd:::obj87:head 2026-02-11T23:31:27.813+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e5124c98:::obj133:head local-lis/les=113/114 n=547 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:e6e28dcd:::obj87:head v 111'87 2026-02-11T23:31:27.813+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e5124c98:::obj133:head local-lis/les=113/114 n=547 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.813+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.813+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:e6e28dcd:::obj87:head, version: 111'87, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e6e28dcd:::obj87:head@111'87, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:27.813+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e5124c98:::obj133:head local-lis/les=113/114 n=547 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.813+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.813+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.813+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e6508d05:::obj595:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.813+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:e6508d05:::obj595:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 1.0 e 116/116 lb 1:e6508d05:::obj595:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:e5124c98:::obj133:head local-lis/les=113/114 n=547 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.813+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.813+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e5124c98:::obj133:head local-lis/les=113/114 n=547 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:e6508d05:::obj595:head) v3 2026-02-11T23:31:27.813+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e5124c98:::obj133:head local-lis/les=113/114 n=547 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:e6508d05:::obj595:head) 2026-02-11T23:31:27.813+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.813+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:e6508d05:::obj595:head) v3 finish latency 0.000010 2026-02-11T23:31:27.813+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.813+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.813+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e6508d05:::obj595:head local-lis/les=113/114 n=548 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.813+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.813+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.813+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.813+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d1cd80 con 0x4acb000 2026-02-11T23:31:27.813+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:e6e28dcd:::obj87:head)]) -- 0x612e000 con 0x4acb000 2026-02-11T23:31:27.813+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.813+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1133 ==== MOSDPGPush(1.0 116/113 [PushOp(1:e74a4ee3:::obj396:head, version: 111'396, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e74a4ee3:::obj396:head@111'396, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56cb1e0 con 0x4acb000 2026-02-11T23:31:27.813+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:e74a4ee3:::obj396:head, version: 111'396, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e74a4ee3:::obj396:head@111'396, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.813+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e74a4ee3:::obj396:head, version: 111'396, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e74a4ee3:::obj396:head@111'396, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.813+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.813+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1134 ==== pg_backfill(progress 1.0 e 116/116 lb 1:e6e28dcd:::obj87:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d9b180 con 0x4acb000 2026-02-11T23:31:27.813+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:e6e28dcd:::obj87:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 116 2026-02-11T23:31:27.813+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e74a4ee3:::obj396:head, version: 111'396, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e74a4ee3:::obj396:head@111'396, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.813+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e6e28dcd:::obj87:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.813+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.813+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e74a4ee3:::obj396:head, version: 111'396, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e74a4ee3:::obj396:head@111'396, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.813+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:e74a4ee3:::obj396:head, version: 111'396, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e74a4ee3:::obj396:head@111'396, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:e74a4ee3:::obj396:head, version: 111'396, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e74a4ee3:::obj396:head@111'396, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:e6508d05:::obj595:head local-lis/les=113/114 n=548 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.813+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.813+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.813+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e6508d05:::obj595:head local-lis/les=113/114 n=548 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:e74a4ee3:::obj396:head, version: 111'396, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e74a4ee3:::obj396:head@111'396, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.813+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e6e28dcd:::obj87:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.813+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e6508d05:::obj595:head local-lis/les=113/114 n=548 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:e74a4ee3:::obj396:head@111'396, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.813+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e6508d05:::obj595:head local-lis/les=113/114 n=548 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:e74a4ee3:::obj396:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.813+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e6508d05:::obj595:head local-lis/les=113/114 n=548 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:e74a4ee3:::obj396:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.813+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e6508d05:::obj595:head local-lis/les=113/114 n=548 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:e74a4ee3:::obj396:head 2026-02-11T23:31:27.813+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e6508d05:::obj595:head local-lis/les=113/114 n=548 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:e74a4ee3:::obj396:head v 111'396 2026-02-11T23:31:27.813+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e6508d05:::obj595:head local-lis/les=113/114 n=548 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.813+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.813+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:e74a4ee3:::obj396:head, version: 111'396, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e74a4ee3:::obj396:head@111'396, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.813+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.813+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.813+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e6e28dcd:::obj87:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.813+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:e6e28dcd:::obj87:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 1.0 e 116/116 lb 1:e6e28dcd:::obj87:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:e6508d05:::obj595:head local-lis/les=113/114 n=548 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.813+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.813+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e6508d05:::obj595:head local-lis/les=113/114 n=548 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:e6e28dcd:::obj87:head) v3 2026-02-11T23:31:27.813+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e6508d05:::obj595:head local-lis/les=113/114 n=548 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:e6e28dcd:::obj87:head) 2026-02-11T23:31:27.813+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.813+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:e6e28dcd:::obj87:head) v3 finish latency 0.000011 2026-02-11T23:31:27.813+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e6e28dcd:::obj87:head local-lis/les=113/114 n=549 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.813+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.813+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.813+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e6e28dcd:::obj87:head local-lis/les=113/114 n=549 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.813+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.813+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.813+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.813+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fcfe00 con 0x4acb000 2026-02-11T23:31:27.813+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:e74a4ee3:::obj396:head)]) -- 0x5f2c600 con 0x4acb000 2026-02-11T23:31:27.813+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.814+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1135 ==== MOSDPGPush(1.0 116/113 [PushOp(1:e7b9c028:::obj510:head, version: 111'510, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e7b9c028:::obj510:head@111'510, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d9520 con 0x4acb000 2026-02-11T23:31:27.814+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:e7b9c028:::obj510:head, version: 111'510, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e7b9c028:::obj510:head@111'510, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.814+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e7b9c028:::obj510:head, version: 111'510, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e7b9c028:::obj510:head@111'510, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.814+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.814+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1136 ==== pg_backfill(progress 1.0 e 116/116 lb 1:e74a4ee3:::obj396:head) ==== 1062+0+0 (unknown 0 0 0) 0x6178c00 con 0x4acb000 2026-02-11T23:31:27.814+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e7b9c028:::obj510:head, version: 111'510, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e7b9c028:::obj510:head@111'510, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.814+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:e74a4ee3:::obj396:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.814+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.814+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e74a4ee3:::obj396:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.814+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e7b9c028:::obj510:head, version: 111'510, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e7b9c028:::obj510:head@111'510, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.814+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.814+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:e7b9c028:::obj510:head, version: 111'510, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e7b9c028:::obj510:head@111'510, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:e7b9c028:::obj510:head, version: 111'510, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e7b9c028:::obj510:head@111'510, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:e6e28dcd:::obj87:head local-lis/les=113/114 n=549 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.814+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e74a4ee3:::obj396:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.814+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.814+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e6e28dcd:::obj87:head local-lis/les=113/114 n=549 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:e7b9c028:::obj510:head, version: 111'510, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e7b9c028:::obj510:head@111'510, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.814+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e6e28dcd:::obj87:head local-lis/les=113/114 n=549 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:e7b9c028:::obj510:head@111'510, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.814+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e6e28dcd:::obj87:head local-lis/les=113/114 n=549 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:e7b9c028:::obj510:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.814+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e6e28dcd:::obj87:head local-lis/les=113/114 n=549 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:e7b9c028:::obj510:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.814+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e6e28dcd:::obj87:head local-lis/les=113/114 n=549 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:e7b9c028:::obj510:head 2026-02-11T23:31:27.814+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e6e28dcd:::obj87:head local-lis/les=113/114 n=549 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:e7b9c028:::obj510:head v 111'510 2026-02-11T23:31:27.814+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e6e28dcd:::obj87:head local-lis/les=113/114 n=549 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.814+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.814+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:e7b9c028:::obj510:head, version: 111'510, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e7b9c028:::obj510:head@111'510, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:27.814+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e6e28dcd:::obj87:head local-lis/les=113/114 n=549 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.814+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.814+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.814+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e74a4ee3:::obj396:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.814+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:e74a4ee3:::obj396:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 1.0 e 116/116 lb 1:e74a4ee3:::obj396:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:e6e28dcd:::obj87:head local-lis/les=113/114 n=549 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.814+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.814+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e6e28dcd:::obj87:head local-lis/les=113/114 n=549 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:e74a4ee3:::obj396:head) v3 2026-02-11T23:31:27.814+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e6e28dcd:::obj87:head local-lis/les=113/114 n=549 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:e74a4ee3:::obj396:head) 2026-02-11T23:31:27.814+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.814+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:e74a4ee3:::obj396:head) v3 finish latency 0.000011 2026-02-11T23:31:27.814+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.814+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.814+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e74a4ee3:::obj396:head local-lis/les=113/114 n=550 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.814+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.814+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.814+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.814+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6171680 con 0x4acb000 2026-02-11T23:31:27.814+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:e7b9c028:::obj510:head)]) -- 0x5fd3c80 con 0x4acb000 2026-02-11T23:31:27.814+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.814+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1137 ==== MOSDPGPush(1.0 116/113 [PushOp(1:e7f87738:::obj356:head, version: 111'356, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e7f87738:::obj356:head@111'356, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e269c0 con 0x4acb000 2026-02-11T23:31:27.814+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:e7f87738:::obj356:head, version: 111'356, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e7f87738:::obj356:head@111'356, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.814+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e7f87738:::obj356:head, version: 111'356, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e7f87738:::obj356:head@111'356, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.814+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.814+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1138 ==== pg_backfill(progress 1.0 e 116/116 lb 1:e7b9c028:::obj510:head) ==== 1062+0+0 (unknown 0 0 0) 0x6179180 con 0x4acb000 2026-02-11T23:31:27.814+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e7f87738:::obj356:head, version: 111'356, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e7f87738:::obj356:head@111'356, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.814+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.814+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:e7b9c028:::obj510:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.814+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e7f87738:::obj356:head, version: 111'356, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e7f87738:::obj356:head@111'356, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.814+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e7b9c028:::obj510:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.814+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.814+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:e7f87738:::obj356:head, version: 111'356, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e7f87738:::obj356:head@111'356, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 116/113 [PushOp(1:e7f87738:::obj356:head, version: 111'356, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e7f87738:::obj356:head@111'356, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:e74a4ee3:::obj396:head local-lis/les=113/114 n=550 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.814+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.814+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e74a4ee3:::obj396:head local-lis/les=113/114 n=550 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:e7f87738:::obj356:head, version: 111'356, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e7f87738:::obj356:head@111'356, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.814+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e74a4ee3:::obj396:head local-lis/les=113/114 n=550 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:e7f87738:::obj356:head@111'356, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.814+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.814+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e7b9c028:::obj510:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.814+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e74a4ee3:::obj396:head local-lis/les=113/114 n=550 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:e7f87738:::obj356:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.814+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e74a4ee3:::obj396:head local-lis/les=113/114 n=550 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:e7f87738:::obj356:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.814+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e74a4ee3:::obj396:head local-lis/les=113/114 n=550 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:e7f87738:::obj356:head 2026-02-11T23:31:27.814+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e74a4ee3:::obj396:head local-lis/les=113/114 n=550 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:e7f87738:::obj356:head v 111'356 2026-02-11T23:31:27.814+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e74a4ee3:::obj396:head local-lis/les=113/114 n=550 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.814+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.814+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:e7f87738:::obj356:head, version: 111'356, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e7f87738:::obj356:head@111'356, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.814+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.814+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.814+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e7b9c028:::obj510:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.814+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:e7b9c028:::obj510:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 116/116 lb 1:e7b9c028:::obj510:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:e74a4ee3:::obj396:head local-lis/les=113/114 n=550 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.814+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.814+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e74a4ee3:::obj396:head local-lis/les=113/114 n=550 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:e7b9c028:::obj510:head) v3 2026-02-11T23:31:27.814+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e74a4ee3:::obj396:head local-lis/les=113/114 n=550 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:e7b9c028:::obj510:head) 2026-02-11T23:31:27.814+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.814+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:e7b9c028:::obj510:head) v3 finish latency 0.000011 2026-02-11T23:31:27.814+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e7b9c028:::obj510:head local-lis/les=113/114 n=551 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.814+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.814+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.814+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e7b9c028:::obj510:head local-lis/les=113/114 n=551 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.814+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.814+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.814+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.814+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x6000180 con 0x4acb000 2026-02-11T23:31:27.814+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:e7f87738:::obj356:head)]) -- 0x5f2c300 con 0x4acb000 2026-02-11T23:31:27.814+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.815+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1139 ==== MOSDPGPush(1.0 116/113 [PushOp(1:e8366861:::obj250:head, version: 111'250, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e8366861:::obj250:head@111'250, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d8b60 con 0x4acb000 2026-02-11T23:31:27.815+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:e8366861:::obj250:head, version: 111'250, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e8366861:::obj250:head@111'250, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.815+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e8366861:::obj250:head, version: 111'250, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e8366861:::obj250:head@111'250, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.815+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.815+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1140 ==== pg_backfill(progress 1.0 e 116/116 lb 1:e7f87738:::obj356:head) ==== 1062+0+0 (unknown 0 0 0) 0x6178100 con 0x4acb000 2026-02-11T23:31:27.815+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e8366861:::obj250:head, version: 111'250, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e8366861:::obj250:head@111'250, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.815+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:e7f87738:::obj356:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.815+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e7f87738:::obj356:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.815+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.815+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e8366861:::obj250:head, version: 111'250, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e8366861:::obj250:head@111'250, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.815+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.815+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:e8366861:::obj250:head, version: 111'250, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e8366861:::obj250:head@111'250, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 116/113 [PushOp(1:e8366861:::obj250:head, version: 111'250, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e8366861:::obj250:head@111'250, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:e7b9c028:::obj510:head local-lis/les=113/114 n=551 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.815+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.815+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e7b9c028:::obj510:head local-lis/les=113/114 n=551 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:e8366861:::obj250:head, version: 111'250, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e8366861:::obj250:head@111'250, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.815+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e7b9c028:::obj510:head local-lis/les=113/114 n=551 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:e8366861:::obj250:head@111'250, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.815+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.815+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e7f87738:::obj356:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.815+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e7b9c028:::obj510:head local-lis/les=113/114 n=551 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:e8366861:::obj250:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.815+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e7b9c028:::obj510:head local-lis/les=113/114 n=551 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:e8366861:::obj250:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.815+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e7b9c028:::obj510:head local-lis/les=113/114 n=551 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:e8366861:::obj250:head 2026-02-11T23:31:27.815+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e7b9c028:::obj510:head local-lis/les=113/114 n=551 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:e8366861:::obj250:head v 111'250 2026-02-11T23:31:27.815+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e7b9c028:::obj510:head local-lis/les=113/114 n=551 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.815+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.815+0000 7f414f993640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:e8366861:::obj250:head, version: 111'250, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e8366861:::obj250:head@111'250, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:27.815+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e7b9c028:::obj510:head local-lis/les=113/114 n=551 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.815+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.815+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.815+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e7f87738:::obj356:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.815+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:e7f87738:::obj356:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 1.0 e 116/116 lb 1:e7f87738:::obj356:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:e7b9c028:::obj510:head local-lis/les=113/114 n=551 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.815+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.815+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e7b9c028:::obj510:head local-lis/les=113/114 n=551 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:e7f87738:::obj356:head) v3 2026-02-11T23:31:27.815+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e7b9c028:::obj510:head local-lis/les=113/114 n=551 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:e7f87738:::obj356:head) 2026-02-11T23:31:27.815+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.815+0000 7f414b98b640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:e7f87738:::obj356:head) v3 finish latency 0.000011 2026-02-11T23:31:27.815+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.815+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.815+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e7f87738:::obj356:head local-lis/les=113/114 n=552 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.815+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.815+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.815+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.815+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5d1d380 con 0x4acb000 2026-02-11T23:31:27.815+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:e8366861:::obj250:head)]) -- 0x600a900 con 0x4acb000 2026-02-11T23:31:27.815+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.815+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1141 ==== MOSDPGPush(1.0 116/113 [PushOp(1:e9ac1484:::obj417:head, version: 111'417, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e9ac1484:::obj417:head@111'417, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56471e0 con 0x4acb000 2026-02-11T23:31:27.815+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:e9ac1484:::obj417:head, version: 111'417, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e9ac1484:::obj417:head@111'417, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 116 2026-02-11T23:31:27.815+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e9ac1484:::obj417:head, version: 111'417, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e9ac1484:::obj417:head@111'417, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.815+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.815+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1142 ==== pg_backfill(progress 1.0 e 116/116 lb 1:e8366861:::obj250:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177b80 con 0x4acb000 2026-02-11T23:31:27.815+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:e8366861:::obj250:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 116 2026-02-11T23:31:27.815+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e9ac1484:::obj417:head, version: 111'417, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e9ac1484:::obj417:head@111'417, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.815+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e8366861:::obj250:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.815+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.815+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:e9ac1484:::obj417:head, version: 111'417, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e9ac1484:::obj417:head@111'417, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.815+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:e9ac1484:::obj417:head, version: 111'417, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e9ac1484:::obj417:head@111'417, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 116/113 [PushOp(1:e9ac1484:::obj417:head, version: 111'417, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e9ac1484:::obj417:head@111'417, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:e7f87738:::obj356:head local-lis/les=113/114 n=552 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.815+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.815+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.815+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e7f87738:::obj356:head local-lis/les=113/114 n=552 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:e9ac1484:::obj417:head, version: 111'417, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e9ac1484:::obj417:head@111'417, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.815+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e8366861:::obj250:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.815+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e7f87738:::obj356:head local-lis/les=113/114 n=552 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:e9ac1484:::obj417:head@111'417, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.815+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e7f87738:::obj356:head local-lis/les=113/114 n=552 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:e9ac1484:::obj417:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.815+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e7f87738:::obj356:head local-lis/les=113/114 n=552 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:e9ac1484:::obj417:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.815+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e7f87738:::obj356:head local-lis/les=113/114 n=552 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:e9ac1484:::obj417:head 2026-02-11T23:31:27.815+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e7f87738:::obj356:head local-lis/les=113/114 n=552 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:e9ac1484:::obj417:head v 111'417 2026-02-11T23:31:27.815+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e7f87738:::obj356:head local-lis/les=113/114 n=552 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.815+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.815+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:e9ac1484:::obj417:head, version: 111'417, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e9ac1484:::obj417:head@111'417, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.815+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.815+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.815+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e8366861:::obj250:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.815+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:e8366861:::obj250:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 116/116 lb 1:e8366861:::obj250:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:e7f87738:::obj356:head local-lis/les=113/114 n=552 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.815+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.815+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e7f87738:::obj356:head local-lis/les=113/114 n=552 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:e8366861:::obj250:head) v3 2026-02-11T23:31:27.815+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e7f87738:::obj356:head local-lis/les=113/114 n=552 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:e8366861:::obj250:head) 2026-02-11T23:31:27.815+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.815+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:e8366861:::obj250:head) v3 finish latency 0.000011 2026-02-11T23:31:27.815+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e8366861:::obj250:head local-lis/les=113/114 n=553 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.815+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.816+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.816+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e8366861:::obj250:head local-lis/les=113/114 n=553 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.816+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.816+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.816+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.816+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x600b500 con 0x4acb000 2026-02-11T23:31:27.816+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:e9ac1484:::obj417:head)]) -- 0x5f2c000 con 0x4acb000 2026-02-11T23:31:27.816+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.816+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1143 ==== MOSDPGPush(1.0 116/113 [PushOp(1:ea485012:::obj312:head, version: 111'312, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ea485012:::obj312:head@111'312, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d8340 con 0x4acb000 2026-02-11T23:31:27.816+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:ea485012:::obj312:head, version: 111'312, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ea485012:::obj312:head@111'312, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000015 epoch 116 2026-02-11T23:31:27.816+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:ea485012:::obj312:head, version: 111'312, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ea485012:::obj312:head@111'312, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.816+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.816+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:ea485012:::obj312:head, version: 111'312, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ea485012:::obj312:head@111'312, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.816+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1144 ==== pg_backfill(progress 1.0 e 116/116 lb 1:e9ac1484:::obj417:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177600 con 0x4acb000 2026-02-11T23:31:27.816+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.816+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:e9ac1484:::obj417:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.816+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:ea485012:::obj312:head, version: 111'312, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ea485012:::obj312:head@111'312, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.816+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e9ac1484:::obj417:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.816+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:ea485012:::obj312:head, version: 111'312, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ea485012:::obj312:head@111'312, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000044 MOSDPGPush(1.0 116/113 [PushOp(1:ea485012:::obj312:head, version: 111'312, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ea485012:::obj312:head@111'312, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:e8366861:::obj250:head local-lis/les=113/114 n=553 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.816+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.816+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.816+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e9ac1484:::obj417:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.816+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e8366861:::obj250:head local-lis/les=113/114 n=553 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:ea485012:::obj312:head, version: 111'312, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ea485012:::obj312:head@111'312, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.816+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e8366861:::obj250:head local-lis/les=113/114 n=553 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:ea485012:::obj312:head@111'312, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.816+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e8366861:::obj250:head local-lis/les=113/114 n=553 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:ea485012:::obj312:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.816+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e8366861:::obj250:head local-lis/les=113/114 n=553 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:ea485012:::obj312:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.816+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e8366861:::obj250:head local-lis/les=113/114 n=553 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:ea485012:::obj312:head 2026-02-11T23:31:27.816+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e8366861:::obj250:head local-lis/les=113/114 n=553 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:ea485012:::obj312:head v 111'312 2026-02-11T23:31:27.816+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e8366861:::obj250:head local-lis/les=113/114 n=553 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.816+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.816+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:ea485012:::obj312:head, version: 111'312, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ea485012:::obj312:head@111'312, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000043 2026-02-11T23:31:27.816+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.816+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.816+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:e9ac1484:::obj417:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.816+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:e9ac1484:::obj417:head) v3 prio 15 cost 0 latency 0.000058 pg_backfill(progress 1.0 e 116/116 lb 1:e9ac1484:::obj417:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:e8366861:::obj250:head local-lis/les=113/114 n=553 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.816+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.816+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e8366861:::obj250:head local-lis/les=113/114 n=553 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:e9ac1484:::obj417:head) v3 2026-02-11T23:31:27.816+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e8366861:::obj250:head local-lis/les=113/114 n=553 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:e9ac1484:::obj417:head) 2026-02-11T23:31:27.816+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.816+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:e9ac1484:::obj417:head) v3 finish latency 0.000010 2026-02-11T23:31:27.816+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e9ac1484:::obj417:head local-lis/les=113/114 n=554 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.816+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.816+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.816+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e9ac1484:::obj417:head local-lis/les=113/114 n=554 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.816+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.816+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.816+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.816+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5daa900 con 0x4acb000 2026-02-11T23:31:27.816+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:ea485012:::obj312:head)]) -- 0x5fd2c00 con 0x4acb000 2026-02-11T23:31:27.816+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.816+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1145 ==== MOSDPGPush(1.0 116/113 [PushOp(1:ea54a43e:::obj502:head, version: 111'502, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ea54a43e:::obj502:head@111'502, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d9860 con 0x4acb000 2026-02-11T23:31:27.816+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:ea54a43e:::obj502:head, version: 111'502, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ea54a43e:::obj502:head@111'502, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000015 epoch 116 2026-02-11T23:31:27.816+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:ea54a43e:::obj502:head, version: 111'502, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ea54a43e:::obj502:head@111'502, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.816+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.816+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:ea54a43e:::obj502:head, version: 111'502, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ea54a43e:::obj502:head@111'502, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.816+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1146 ==== pg_backfill(progress 1.0 e 116/116 lb 1:ea485012:::obj312:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177080 con 0x4acb000 2026-02-11T23:31:27.816+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.816+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:ea485012:::obj312:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 116 2026-02-11T23:31:27.816+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:ea54a43e:::obj502:head, version: 111'502, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ea54a43e:::obj502:head@111'502, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.816+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:ea485012:::obj312:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.816+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:ea54a43e:::obj502:head, version: 111'502, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ea54a43e:::obj502:head@111'502, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000043 MOSDPGPush(1.0 116/113 [PushOp(1:ea54a43e:::obj502:head, version: 111'502, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ea54a43e:::obj502:head@111'502, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:e9ac1484:::obj417:head local-lis/les=113/114 n=554 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.816+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.816+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.816+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:ea485012:::obj312:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.816+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e9ac1484:::obj417:head local-lis/les=113/114 n=554 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:ea54a43e:::obj502:head, version: 111'502, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ea54a43e:::obj502:head@111'502, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.816+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e9ac1484:::obj417:head local-lis/les=113/114 n=554 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:ea54a43e:::obj502:head@111'502, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.816+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e9ac1484:::obj417:head local-lis/les=113/114 n=554 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:ea54a43e:::obj502:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.816+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e9ac1484:::obj417:head local-lis/les=113/114 n=554 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:ea54a43e:::obj502:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.816+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e9ac1484:::obj417:head local-lis/les=113/114 n=554 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:ea54a43e:::obj502:head 2026-02-11T23:31:27.816+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e9ac1484:::obj417:head local-lis/les=113/114 n=554 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:ea54a43e:::obj502:head v 111'502 2026-02-11T23:31:27.816+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e9ac1484:::obj417:head local-lis/les=113/114 n=554 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.816+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.816+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:ea54a43e:::obj502:head, version: 111'502, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ea54a43e:::obj502:head@111'502, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000040 2026-02-11T23:31:27.816+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.816+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.816+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:ea485012:::obj312:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.816+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:ea485012:::obj312:head) v3 prio 15 cost 0 latency 0.000053 pg_backfill(progress 1.0 e 116/116 lb 1:ea485012:::obj312:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:e9ac1484:::obj417:head local-lis/les=113/114 n=554 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.816+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.816+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e9ac1484:::obj417:head local-lis/les=113/114 n=554 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:ea485012:::obj312:head) v3 2026-02-11T23:31:27.816+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:e9ac1484:::obj417:head local-lis/les=113/114 n=554 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:ea485012:::obj312:head) 2026-02-11T23:31:27.816+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.816+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:ea485012:::obj312:head) v3 finish latency 0.000009 2026-02-11T23:31:27.816+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ea485012:::obj312:head local-lis/les=113/114 n=555 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.816+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.829+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.829+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ea485012:::obj312:head local-lis/les=113/114 n=555 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.829+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.829+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.829+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.829+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fed380 con 0x4acb000 2026-02-11T23:31:27.829+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:ea54a43e:::obj502:head)]) -- 0x615ca80 con 0x4acb000 2026-02-11T23:31:27.829+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.829+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1147 ==== MOSDPGPush(1.0 116/113 [PushOp(1:eaf65c1c:::obj380:head, version: 111'380, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eaf65c1c:::obj380:head@111'380, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55d8820 con 0x4acb000 2026-02-11T23:31:27.829+0000 7f416abfa640 15 osd.0 116 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:eaf65c1c:::obj380:head, version: 111'380, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eaf65c1c:::obj380:head@111'380, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 116 2026-02-11T23:31:27.829+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:eaf65c1c:::obj380:head, version: 111'380, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eaf65c1c:::obj380:head@111'380, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.829+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.829+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1148 ==== pg_backfill(progress 1.0 e 116/116 lb 1:ea54a43e:::obj502:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176b00 con 0x4acb000 2026-02-11T23:31:27.829+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:eaf65c1c:::obj380:head, version: 111'380, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eaf65c1c:::obj380:head@111'380, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.829+0000 7f416abfa640 15 osd.0 116 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:ea54a43e:::obj502:head) v3 prio 15 type 95 cost 0 latency 0.000010 epoch 116 2026-02-11T23:31:27.829+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.829+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:ea54a43e:::obj502:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.829+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:eaf65c1c:::obj380:head, version: 111'380, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eaf65c1c:::obj380:head@111'380, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.829+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.829+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:eaf65c1c:::obj380:head, version: 111'380, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eaf65c1c:::obj380:head@111'380, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000042 MOSDPGPush(1.0 116/113 [PushOp(1:eaf65c1c:::obj380:head, version: 111'380, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eaf65c1c:::obj380:head@111'380, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:ea485012:::obj312:head local-lis/les=113/114 n=555 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.829+0000 7f414b98b640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.829+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ea485012:::obj312:head local-lis/les=113/114 n=555 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:eaf65c1c:::obj380:head, version: 111'380, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eaf65c1c:::obj380:head@111'380, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.829+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.829+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ea485012:::obj312:head local-lis/les=113/114 n=555 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:eaf65c1c:::obj380:head@111'380, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.829+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:ea54a43e:::obj502:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.829+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ea485012:::obj312:head local-lis/les=113/114 n=555 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:eaf65c1c:::obj380:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.829+0000 7f414b98b640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ea485012:::obj312:head local-lis/les=113/114 n=555 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:eaf65c1c:::obj380:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.829+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ea485012:::obj312:head local-lis/les=113/114 n=555 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:eaf65c1c:::obj380:head 2026-02-11T23:31:27.829+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ea485012:::obj312:head local-lis/les=113/114 n=555 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:eaf65c1c:::obj380:head v 111'380 2026-02-11T23:31:27.829+0000 7f414b98b640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ea485012:::obj312:head local-lis/les=113/114 n=555 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.829+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.829+0000 7f414b98b640 10 osd.0 116 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:eaf65c1c:::obj380:head, version: 111'380, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eaf65c1c:::obj380:head@111'380, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:27.829+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.829+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.829+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:ea54a43e:::obj502:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.829+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:ea54a43e:::obj502:head) v3 prio 15 cost 0 latency 0.000082 pg_backfill(progress 1.0 e 116/116 lb 1:ea54a43e:::obj502:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:ea485012:::obj312:head local-lis/les=113/114 n=555 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.829+0000 7f414f993640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 116 is not newer than session's projected_epoch of 116 2026-02-11T23:31:27.829+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ea485012:::obj312:head local-lis/les=113/114 n=555 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:ea54a43e:::obj502:head) v3 2026-02-11T23:31:27.829+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ea485012:::obj312:head local-lis/les=113/114 n=555 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:ea54a43e:::obj502:head) 2026-02-11T23:31:27.829+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.829+0000 7f414f993640 10 osd.0 116 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:ea54a43e:::obj502:head) v3 finish latency 0.000013 2026-02-11T23:31:27.829+0000 7f414f993640 20 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ea54a43e:::obj502:head local-lis/les=113/114 n=556 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.829+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.830+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 53 ==== osd_map(117..117 src has 1..117) ==== 321+0+0 (crc 0 0 0) 0x5954000 con 0x4acbc00 2026-02-11T23:31:27.830+0000 7f415cf02640 20 osd.0 116 OSD::ms_dispatch: osd_map(117..117 src has 1..117) 2026-02-11T23:31:27.830+0000 7f415cf02640 20 osd.0 116 _dispatch 0x5954000 osd_map(117..117 src has 1..117) 2026-02-11T23:31:27.830+0000 7f415cf02640 3 osd.0 116 handle_osd_map epochs [117,117], i have 116, src has [1,117] 2026-02-11T23:31:27.830+0000 7f415cf02640 10 osd.0 116 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:31:27.830+0000 7f415cf02640 10 osd.0 116 handle_osd_map got inc map for epoch 117 2026-02-11T23:31:27.830+0000 7f415cf02640 20 osd.0 116 got_full_map 117, nothing requested 2026-02-11T23:31:27.830+0000 7f415cf02640 20 osd.0 116 trim_maps: min=1 oldest_map=1 2026-02-11T23:31:27.830+0000 7f415cf02640 20 osd.0 116 handle_osd_map pg_num_history pg_num_history(e117 pg_nums {1={27=1},2={34=1},3={42=1},4={49=1},5={56=1},6={64=1},7={70=1},8={77=1},9={85=1},10={92=1}} deleted_pools 99,2,100,3,101,5,102,6,103,7,104,8,105,9,106,10) 2026-02-11T23:31:27.830+0000 7f415cf02640 10 snap_mapper record_purged_snaps purged_snaps {117={}} 2026-02-11T23:31:27.830+0000 7f415cf02640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:31:27.830+0000 7f415cf02640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e117 maps [1~117] lci=[5,117] tlb=1) 2026-02-11T23:31:27.830+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.830+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ea54a43e:::obj502:head local-lis/les=113/114 n=556 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.830+0000 7f414f993640 20 osd.0 116 get_con_osd_cluster to osd.1 from_epoch 116 2026-02-11T23:31:27.830+0000 7f414f993640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:27.830+0000 7f414f993640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:27.830+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e116/116) -- 0x5fcf800 con 0x4acb000 2026-02-11T23:31:27.830+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:eaf65c1c:::obj380:head)]) -- 0x5fd2f00 con 0x4acb000 2026-02-11T23:31:27.830+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.830+0000 7f41591a6640 10 osd.0 116 _committed_osd_maps 117..117 2026-02-11T23:31:27.830+0000 7f41591a6640 10 osd.0 116 advance to epoch 117 (<= last 117 <= newest_map 117) 2026-02-11T23:31:27.830+0000 7f41591a6640 20 osd.0 117 consume_map version 117 2026-02-11T23:31:27.830+0000 7f41591a6640 20 osd.0 116 await_reserved_maps epoch:117 2026-02-11T23:31:27.830+0000 7f41591a6640 20 osd.0 116 await_reserved_maps done 0 2026-02-11T23:31:27.830+0000 7f41591a6640 20 osd.0 117 consume_map 117 -- publish done 2026-02-11T23:31:27.830+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:31:27.830+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x5efc160 2026-02-11T23:31:27.830+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot 4.0 has no pg and waiting_for_split 2026-02-11T23:31:27.830+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3701340 2026-02-11T23:31:27.830+0000 7f41591a6640 20 osd.0 117 identify_splits_and_merges 1.0 e116 to e117 2026-02-11T23:31:27.830+0000 7f41591a6640 20 osd.0 117 identify_splits_and_merges 1.0 e116 to e117 pg_nums {27=1} 2026-02-11T23:31:27.830+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:27.830+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:27.830+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:27.830+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:27.830+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:27.830+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:27.830+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:27.830+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:27.830+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:27.830+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:27.830+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:27.830+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:27.830+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:27.830+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:27.830+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:27.830+0000 7f41591a6640 20 osd.0 117 prune_pg_created 2026-02-11T23:31:27.830+0000 7f41591a6640 10 osd.0:0.consume_map 117 (was 116) 2026-02-11T23:31:27.830+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:31:27.830+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 empty, pruning 2026-02-11T23:31:27.830+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:31:27.830+0000 7f41591a6640 10 osd.0:1.consume_map 117 (was 116) 2026-02-11T23:31:27.830+0000 7f41591a6640 10 osd.0:2.consume_map 117 (was 116) 2026-02-11T23:31:27.830+0000 7f41591a6640 10 osd.0:3.consume_map 117 (was 116) 2026-02-11T23:31:27.830+0000 7f41591a6640 10 osd.0:4.consume_map 117 (was 116) 2026-02-11T23:31:27.830+0000 7f41591a6640 10 osd.0:5.consume_map 117 (was 116) 2026-02-11T23:31:27.830+0000 7f41591a6640 10 osd.0:6.consume_map 117 (was 116) 2026-02-11T23:31:27.830+0000 7f41591a6640 10 osd.0:7.consume_map 117 (was 116) 2026-02-11T23:31:27.830+0000 7f41591a6640 10 osd.0 117 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:31:27.830+0000 7f41591a6640 15 osd.0 117 enqueue_peering_evt 1.0 epoch_sent: 117 epoch_requested: 117 NullEvt 2026-02-11T23:31:27.830+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 117 epoch_requested: 117 NullEvt) class_id 2 prio 255 cost 10 e117) 2026-02-11T23:31:27.830+0000 7f41591a6640 7 osd.0 117 activate_map version 117 2026-02-11T23:31:27.830+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.830+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 117 epoch_requested: 117 NullEvt) class_id 2immediate prio 255 cost 10 e117) queued 2026-02-11T23:31:27.830+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.830+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 117 epoch_requested: 117 NullEvt) class_id 2immediate prio 255 cost 10 e117) pg 0x4ef0000 2026-02-11T23:31:27.830+0000 7f4150194640 20 osd.0 117 agent_entry empty queue 2026-02-11T23:31:27.830+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.830+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ea54a43e:::obj502:head local-lis/les=113/114 n=556 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_advance_map: 117 2026-02-11T23:31:27.830+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 116 -> 117 2026-02-11T23:31:27.830+0000 7f414f993640 10 osd.0 pg_epoch: 116 pg[1.0( v 111'600 (111'590,111'600] lb 1:ea54a43e:::obj502:head local-lis/les=113/114 n=556 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1] -- 1/1 2026-02-11T23:31:27.830+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ea54a43e:::obj502:head local-lis/les=113/114 n=556 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] state: Started advmap 2026-02-11T23:31:27.830+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ea54a43e:::obj502:head local-lis/les=113/114 n=556 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:31:27.830+0000 7f414f993640 20 osd.0 117 advance_pg new pool opts old pool opts 2026-02-11T23:31:27.830+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ea54a43e:::obj502:head local-lis/les=113/114 n=556 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_activate_map: epoch range: 117..117 2026-02-11T23:31:27.830+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ea54a43e:::obj502:head local-lis/les=113/114 n=556 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::activate_map 2026-02-11T23:31:27.830+0000 7f414f993640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ea54a43e:::obj502:head local-lis/les=113/114 n=556 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 116 while current is 117 2026-02-11T23:31:27.830+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.830+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ea54a43e:::obj502:head local-lis/les=113/114 n=556 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_peering_event: epoch_sent: 117 epoch_requested: 117 NullEvt 2026-02-11T23:31:27.830+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.830+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.830+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1149 ==== MOSDPGPush(1.0 116/113 [PushOp(1:eb0c167c:::obj188:head, version: 111'188, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb0c167c:::obj188:head@111'188, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559cea0 con 0x4acb000 2026-02-11T23:31:27.830+0000 7f416abfa640 15 osd.0 117 enqueue_op MOSDPGPush(1.0 116/113 [PushOp(1:eb0c167c:::obj188:head, version: 111'188, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb0c167c:::obj188:head@111'188, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 116 2026-02-11T23:31:27.830+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:eb0c167c:::obj188:head, version: 111'188, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb0c167c:::obj188:head@111'188, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e116) 2026-02-11T23:31:27.830+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.830+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:eb0c167c:::obj188:head, version: 111'188, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb0c167c:::obj188:head@111'188, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) queued 2026-02-11T23:31:27.830+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.830+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 116/113 [PushOp(1:eb0c167c:::obj188:head, version: 111'188, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb0c167c:::obj188:head@111'188, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e116) pg 0x4ef0000 2026-02-11T23:31:27.830+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.830+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:eb0c167c:::obj188:head, version: 111'188, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb0c167c:::obj188:head@111'188, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000040 MOSDPGPush(1.0 116/113 [PushOp(1:eb0c167c:::obj188:head, version: 111'188, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb0c167c:::obj188:head@111'188, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:ea54a43e:::obj502:head local-lis/les=113/114 n=556 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.830+0000 7f414b98b640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 map epoch 116 -> 117 (shared) 2026-02-11T23:31:27.830+0000 7f414b98b640 10 osd.0 117 send_incremental_map epoch range: (117, 117] to v2:127.0.0.1:6812/3043114013 2026-02-11T23:31:27.830+0000 7f415aefe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1150 ==== osd_map(117..117 src has 1..117) ==== 321+0+0 (unknown 0 0 0) 0x4a31500 con 0x4acb000 2026-02-11T23:31:27.830+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1151 ==== pg_backfill(progress 1.0 e 116/116 lb 1:eaf65c1c:::obj380:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176000 con 0x4acb000 2026-02-11T23:31:27.830+0000 7f416abfa640 15 osd.0 117 enqueue_op pg_backfill(progress 1.0 e 116/116 lb 1:eaf65c1c:::obj380:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 116 2026-02-11T23:31:27.830+0000 7f415aefe640 20 osd.0 117 OSD::ms_dispatch: osd_map(117..117 src has 1..117) 2026-02-11T23:31:27.830+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:eaf65c1c:::obj380:head) v3) class_id 0 prio 15 cost 0 e116) 2026-02-11T23:31:27.830+0000 7f415aefe640 20 osd.0 117 _dispatch 0x4a31500 osd_map(117..117 src has 1..117) 2026-02-11T23:31:27.830+0000 7f415aefe640 3 osd.0 117 handle_osd_map epochs [117,117], i have 117, src has [1,117] 2026-02-11T23:31:27.830+0000 7f415aefe640 10 osd.0 117 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:31:27.830+0000 7f415aefe640 10 osd.0 117 no new maps here, dropping 2026-02-11T23:31:27.830+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.830+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:eaf65c1c:::obj380:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) queued 2026-02-11T23:31:27.830+0000 7f414b98b640 10 osd.0 117 add_map_inc_bl 117 277 bytes 2026-02-11T23:31:27.830+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- osd_map(117..117 src has 1..117) -- 0x5954380 con 0x4acb000 2026-02-11T23:31:27.830+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ea54a43e:::obj502:head local-lis/les=113/114 n=556 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 116/113 [PushOp(1:eb0c167c:::obj188:head, version: 111'188, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb0c167c:::obj188:head@111'188, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.830+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ea54a43e:::obj502:head local-lis/les=113/114 n=556 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:eb0c167c:::obj188:head@111'188, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.830+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ea54a43e:::obj502:head local-lis/les=113/114 n=556 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:eb0c167c:::obj188:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.830+0000 7f414b98b640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ea54a43e:::obj502:head local-lis/les=113/114 n=556 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:eb0c167c:::obj188:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.830+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ea54a43e:::obj502:head local-lis/les=113/114 n=556 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:eb0c167c:::obj188:head 2026-02-11T23:31:27.830+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ea54a43e:::obj502:head local-lis/les=113/114 n=556 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:eb0c167c:::obj188:head v 111'188 2026-02-11T23:31:27.830+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ea54a43e:::obj502:head local-lis/les=113/114 n=556 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.830+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.830+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 116/113 [PushOp(1:eb0c167c:::obj188:head, version: 111'188, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb0c167c:::obj188:head@111'188, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000117 2026-02-11T23:31:27.830+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.830+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.830+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 116/116 lb 1:eaf65c1c:::obj380:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e116) pg 0x4ef0000 2026-02-11T23:31:27.830+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:eaf65c1c:::obj380:head) v3 prio 15 cost 0 latency 0.000097 pg_backfill(progress 1.0 e 116/116 lb 1:eaf65c1c:::obj380:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:ea54a43e:::obj502:head local-lis/les=113/114 n=556 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.830+0000 7f414f993640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.830+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ea54a43e:::obj502:head local-lis/les=113/114 n=556 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 116/116 lb 1:eaf65c1c:::obj380:head) v3 2026-02-11T23:31:27.830+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ea54a43e:::obj502:head local-lis/les=113/114 n=556 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 116/116 lb 1:eaf65c1c:::obj380:head) 2026-02-11T23:31:27.830+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.830+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 116/116 lb 1:eaf65c1c:::obj380:head) v3 finish latency 0.000014 2026-02-11T23:31:27.830+0000 7f414f993640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eaf65c1c:::obj380:head local-lis/les=113/114 n=557 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.830+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.830+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.830+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eaf65c1c:::obj380:head local-lis/les=113/114 n=557 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.830+0000 7f414f993640 20 osd.0 117 get_con_osd_cluster to osd.1 from_epoch 117 2026-02-11T23:31:27.830+0000 7f414f993640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:27.830+0000 7f414f993640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:27.830+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e117/117) -- 0x600b680 con 0x4acb000 2026-02-11T23:31:27.830+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 116/113 [PushReplyOp(1:eb0c167c:::obj188:head)]) -- 0x5df8a80 con 0x4acb000 2026-02-11T23:31:27.830+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.831+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1152 ==== MOSDPGPush(1.0 117/113 [PushOp(1:eb29d295:::obj554:head, version: 111'554, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb29d295:::obj554:head@111'554, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c820 con 0x4acb000 2026-02-11T23:31:27.831+0000 7f416abfa640 15 osd.0 117 enqueue_op MOSDPGPush(1.0 117/113 [PushOp(1:eb29d295:::obj554:head, version: 111'554, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb29d295:::obj554:head@111'554, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000018 epoch 117 2026-02-11T23:31:27.831+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:eb29d295:::obj554:head, version: 111'554, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb29d295:::obj554:head@111'554, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e117) 2026-02-11T23:31:27.831+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.831+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1153 ==== pg_backfill(progress 1.0 e 117/117 lb 1:eb0c167c:::obj188:head) ==== 1062+0+0 (unknown 0 0 0) 0x6179700 con 0x4acb000 2026-02-11T23:31:27.831+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:eb29d295:::obj554:head, version: 111'554, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb29d295:::obj554:head@111'554, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) queued 2026-02-11T23:31:27.831+0000 7f416abfa640 15 osd.0 117 enqueue_op pg_backfill(progress 1.0 e 117/117 lb 1:eb0c167c:::obj188:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 117 2026-02-11T23:31:27.831+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.831+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:eb0c167c:::obj188:head) v3) class_id 0 prio 15 cost 0 e117) 2026-02-11T23:31:27.831+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:eb29d295:::obj554:head, version: 111'554, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb29d295:::obj554:head@111'554, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) pg 0x4ef0000 2026-02-11T23:31:27.831+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.831+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:eb29d295:::obj554:head, version: 111'554, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb29d295:::obj554:head@111'554, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000045 MOSDPGPush(1.0 117/113 [PushOp(1:eb29d295:::obj554:head, version: 111'554, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb29d295:::obj554:head@111'554, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:eaf65c1c:::obj380:head local-lis/les=113/114 n=557 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.831+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:eb0c167c:::obj188:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) queued 2026-02-11T23:31:27.831+0000 7f414b98b640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.831+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eaf65c1c:::obj380:head local-lis/les=113/114 n=557 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 117/113 [PushOp(1:eb29d295:::obj554:head, version: 111'554, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb29d295:::obj554:head@111'554, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.831+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eaf65c1c:::obj380:head local-lis/les=113/114 n=557 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:eb29d295:::obj554:head@111'554, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.831+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eaf65c1c:::obj380:head local-lis/les=113/114 n=557 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:eb29d295:::obj554:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.831+0000 7f414b98b640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eaf65c1c:::obj380:head local-lis/les=113/114 n=557 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:eb29d295:::obj554:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.831+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eaf65c1c:::obj380:head local-lis/les=113/114 n=557 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:eb29d295:::obj554:head 2026-02-11T23:31:27.831+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eaf65c1c:::obj380:head local-lis/les=113/114 n=557 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:eb29d295:::obj554:head v 111'554 2026-02-11T23:31:27.831+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eaf65c1c:::obj380:head local-lis/les=113/114 n=557 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.831+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.831+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:eb29d295:::obj554:head, version: 111'554, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb29d295:::obj554:head@111'554, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:27.831+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.831+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.831+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:eb0c167c:::obj188:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) pg 0x4ef0000 2026-02-11T23:31:27.831+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:eb0c167c:::obj188:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 1.0 e 117/117 lb 1:eb0c167c:::obj188:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:eaf65c1c:::obj380:head local-lis/les=113/114 n=557 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.831+0000 7f414f993640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.831+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eaf65c1c:::obj380:head local-lis/les=113/114 n=557 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 117/117 lb 1:eb0c167c:::obj188:head) v3 2026-02-11T23:31:27.831+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eaf65c1c:::obj380:head local-lis/les=113/114 n=557 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 117/117 lb 1:eb0c167c:::obj188:head) 2026-02-11T23:31:27.831+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.831+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:eb0c167c:::obj188:head) v3 finish latency 0.000015 2026-02-11T23:31:27.831+0000 7f414f993640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eb0c167c:::obj188:head local-lis/les=113/114 n=558 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.831+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.831+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.831+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eb0c167c:::obj188:head local-lis/les=113/114 n=558 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.831+0000 7f414f993640 20 osd.0 117 get_con_osd_cluster to osd.1 from_epoch 117 2026-02-11T23:31:27.831+0000 7f414f993640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:27.831+0000 7f414f993640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:27.831+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e117/117) -- 0x5daa300 con 0x4acb000 2026-02-11T23:31:27.831+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 117/113 [PushReplyOp(1:eb29d295:::obj554:head)]) -- 0x6171380 con 0x4acb000 2026-02-11T23:31:27.831+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.831+0000 7f415aefe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 32 ==== osd_map(117..117 src has 1..117) ==== 321+0+0 (unknown 0 0 0) 0x5954380 con 0x507ac00 2026-02-11T23:31:27.831+0000 7f415aefe640 20 osd.0 117 OSD::ms_dispatch: osd_map(117..117 src has 1..117) 2026-02-11T23:31:27.831+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 33 ==== pg_query2(4.0 4.0 query(info 0'0 epoch_sent 117) e117/117) ==== 158+0+0 (unknown 0 0 0) 0x4fcb4a0 con 0x507ac00 2026-02-11T23:31:27.831+0000 7f415aefe640 20 osd.0 117 _dispatch 0x5954380 osd_map(117..117 src has 1..117) 2026-02-11T23:31:27.831+0000 7f416abfa640 15 osd.0 117 enqueue_peering_evt 4.0 epoch_sent: 117 epoch_requested: 117 MQuery 4.0 from 4 query_epoch 117 query: query(info 0'0 epoch_sent 117) 2026-02-11T23:31:27.831+0000 7f415aefe640 3 osd.0 117 handle_osd_map epochs [117,117], i have 117, src has [1,117] 2026-02-11T23:31:27.831+0000 7f415aefe640 10 osd.0 117 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:31:27.831+0000 7f415aefe640 10 osd.0 117 no new maps here, dropping 2026-02-11T23:31:27.831+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 117 epoch_requested: 117 MQuery 4.0 from 4 query_epoch 117 query: query(info 0'0 epoch_sent 117)) class_id 2 prio 255 cost 10 e117) 2026-02-11T23:31:27.831+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 (new) to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.831+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 117 epoch_requested: 117 MQuery 4.0 from 4 query_epoch 117 query: query(info 0'0 epoch_sent 117)) class_id 2immediate prio 255 cost 10 e117) queued 2026-02-11T23:31:27.831+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.831+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 117 epoch_requested: 117 MQuery 4.0 from 4 query_epoch 117 query: query(info 0'0 epoch_sent 117)) class_id 2immediate prio 255 cost 10 e117) pg 0 2026-02-11T23:31:27.831+0000 7f414b98b640 10 osd.0 117 handle_pg_query_nopg 4.0 2026-02-11T23:31:27.831+0000 7f414b98b640 10 osd.0 117 pg 4.0 dne 2026-02-11T23:31:27.831+0000 7f414b98b640 20 osd.0 117 get_con_osd_cluster to osd.4 from_epoch 117 2026-02-11T23:31:27.831+0000 7f414b98b640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:27.831+0000 7f414b98b640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:27.831+0000 7f414b98b640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6836/3331151400 map epoch 116 -> 117 (shared) 2026-02-11T23:31:27.831+0000 7f414b98b640 10 osd.0 117 send_incremental_map epoch range: (117, 117] to v2:127.0.0.1:6836/3331151400 2026-02-11T23:31:27.831+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- osd_map(117..117 src has 1..117) -- 0x5955a40 con 0x507ac00 2026-02-11T23:31:27.831+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_notify2(4.0 4.0 (query:117 sent:117 4.0( DNE empty local-lis/les=0/0 n=0 ec=0/0 lis/c=0/0 les/c/f=0/0/0 sis=0) ([0,0] all_participants={} intervals=[])) e117/117) -- 0x5f39100 con 0x507ac00 2026-02-11T23:31:27.831+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.831+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.832+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1154 ==== MOSDPGPush(1.0 117/113 [PushOp(1:eb806c57:::obj373:head, version: 111'373, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb806c57:::obj373:head@111'373, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3815860 con 0x4acb000 2026-02-11T23:31:27.832+0000 7f416abfa640 15 osd.0 117 enqueue_op MOSDPGPush(1.0 117/113 [PushOp(1:eb806c57:::obj373:head, version: 111'373, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb806c57:::obj373:head@111'373, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 117 2026-02-11T23:31:27.832+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:eb806c57:::obj373:head, version: 111'373, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb806c57:::obj373:head@111'373, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e117) 2026-02-11T23:31:27.832+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.832+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1155 ==== pg_backfill(progress 1.0 e 117/117 lb 1:eb29d295:::obj554:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2cb00 con 0x4acb000 2026-02-11T23:31:27.832+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:eb806c57:::obj373:head, version: 111'373, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb806c57:::obj373:head@111'373, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) queued 2026-02-11T23:31:27.832+0000 7f416abfa640 15 osd.0 117 enqueue_op pg_backfill(progress 1.0 e 117/117 lb 1:eb29d295:::obj554:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 117 2026-02-11T23:31:27.832+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:eb29d295:::obj554:head) v3) class_id 0 prio 15 cost 0 e117) 2026-02-11T23:31:27.832+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.832+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:eb806c57:::obj373:head, version: 111'373, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb806c57:::obj373:head@111'373, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) pg 0x4ef0000 2026-02-11T23:31:27.832+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.832+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:eb806c57:::obj373:head, version: 111'373, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb806c57:::obj373:head@111'373, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 117/113 [PushOp(1:eb806c57:::obj373:head, version: 111'373, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb806c57:::obj373:head@111'373, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:eb0c167c:::obj188:head local-lis/les=113/114 n=558 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.832+0000 7f414b98b640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.832+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eb0c167c:::obj188:head local-lis/les=113/114 n=558 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 117/113 [PushOp(1:eb806c57:::obj373:head, version: 111'373, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb806c57:::obj373:head@111'373, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.832+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eb0c167c:::obj188:head local-lis/les=113/114 n=558 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:eb806c57:::obj373:head@111'373, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.832+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.832+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:eb29d295:::obj554:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) queued 2026-02-11T23:31:27.832+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eb0c167c:::obj188:head local-lis/les=113/114 n=558 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:eb806c57:::obj373:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.832+0000 7f414b98b640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eb0c167c:::obj188:head local-lis/les=113/114 n=558 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:eb806c57:::obj373:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.832+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eb0c167c:::obj188:head local-lis/les=113/114 n=558 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:eb806c57:::obj373:head 2026-02-11T23:31:27.832+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eb0c167c:::obj188:head local-lis/les=113/114 n=558 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:eb806c57:::obj373:head v 111'373 2026-02-11T23:31:27.832+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eb0c167c:::obj188:head local-lis/les=113/114 n=558 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.832+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.832+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:eb806c57:::obj373:head, version: 111'373, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb806c57:::obj373:head@111'373, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:27.832+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.832+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.832+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:eb29d295:::obj554:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) pg 0x4ef0000 2026-02-11T23:31:27.832+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:eb29d295:::obj554:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 1.0 e 117/117 lb 1:eb29d295:::obj554:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:eb0c167c:::obj188:head local-lis/les=113/114 n=558 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.832+0000 7f414f993640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.832+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eb0c167c:::obj188:head local-lis/les=113/114 n=558 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 117/117 lb 1:eb29d295:::obj554:head) v3 2026-02-11T23:31:27.832+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eb0c167c:::obj188:head local-lis/les=113/114 n=558 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 117/117 lb 1:eb29d295:::obj554:head) 2026-02-11T23:31:27.832+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.832+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:eb29d295:::obj554:head) v3 finish latency 0.000011 2026-02-11T23:31:27.832+0000 7f414f993640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eb29d295:::obj554:head local-lis/les=113/114 n=559 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.832+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.832+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.832+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eb29d295:::obj554:head local-lis/les=113/114 n=559 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.832+0000 7f414f993640 20 osd.0 117 get_con_osd_cluster to osd.1 from_epoch 117 2026-02-11T23:31:27.832+0000 7f414f993640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:27.832+0000 7f414f993640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:27.832+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e117/117) -- 0x5fd2c00 con 0x4acb000 2026-02-11T23:31:27.832+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 117/113 [PushReplyOp(1:eb806c57:::obj373:head)]) -- 0x5d78d80 con 0x4acb000 2026-02-11T23:31:27.832+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.832+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1156 ==== MOSDPGPush(1.0 117/113 [PushOp(1:ecbe4ab7:::obj125:head, version: 111'125, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ecbe4ab7:::obj125:head@111'125, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f1520 con 0x4acb000 2026-02-11T23:31:27.832+0000 7f416abfa640 15 osd.0 117 enqueue_op MOSDPGPush(1.0 117/113 [PushOp(1:ecbe4ab7:::obj125:head, version: 111'125, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ecbe4ab7:::obj125:head@111'125, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 117 2026-02-11T23:31:27.832+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:ecbe4ab7:::obj125:head, version: 111'125, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ecbe4ab7:::obj125:head@111'125, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e117) 2026-02-11T23:31:27.832+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.832+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1157 ==== pg_backfill(progress 1.0 e 117/117 lb 1:eb806c57:::obj373:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2c000 con 0x4acb000 2026-02-11T23:31:27.832+0000 7f416abfa640 15 osd.0 117 enqueue_op pg_backfill(progress 1.0 e 117/117 lb 1:eb806c57:::obj373:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 117 2026-02-11T23:31:27.832+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:ecbe4ab7:::obj125:head, version: 111'125, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ecbe4ab7:::obj125:head@111'125, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) queued 2026-02-11T23:31:27.832+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:eb806c57:::obj373:head) v3) class_id 0 prio 15 cost 0 e117) 2026-02-11T23:31:27.832+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.832+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:ecbe4ab7:::obj125:head, version: 111'125, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ecbe4ab7:::obj125:head@111'125, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) pg 0x4ef0000 2026-02-11T23:31:27.832+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.832+0000 7f414f993640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:ecbe4ab7:::obj125:head, version: 111'125, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ecbe4ab7:::obj125:head@111'125, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 117/113 [PushOp(1:ecbe4ab7:::obj125:head, version: 111'125, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ecbe4ab7:::obj125:head@111'125, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:eb29d295:::obj554:head local-lis/les=113/114 n=559 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.832+0000 7f414f993640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.832+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eb29d295:::obj554:head local-lis/les=113/114 n=559 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 117/113 [PushOp(1:ecbe4ab7:::obj125:head, version: 111'125, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ecbe4ab7:::obj125:head@111'125, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.832+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eb29d295:::obj554:head local-lis/les=113/114 n=559 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:ecbe4ab7:::obj125:head@111'125, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.832+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.832+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:eb806c57:::obj373:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) queued 2026-02-11T23:31:27.832+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eb29d295:::obj554:head local-lis/les=113/114 n=559 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:ecbe4ab7:::obj125:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.832+0000 7f414f993640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eb29d295:::obj554:head local-lis/les=113/114 n=559 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:ecbe4ab7:::obj125:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.832+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eb29d295:::obj554:head local-lis/les=113/114 n=559 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:ecbe4ab7:::obj125:head 2026-02-11T23:31:27.832+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eb29d295:::obj554:head local-lis/les=113/114 n=559 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:ecbe4ab7:::obj125:head v 111'125 2026-02-11T23:31:27.832+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eb29d295:::obj554:head local-lis/les=113/114 n=559 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.832+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.832+0000 7f414f993640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:ecbe4ab7:::obj125:head, version: 111'125, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ecbe4ab7:::obj125:head@111'125, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:27.832+0000 7f414f993640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eb29d295:::obj554:head local-lis/les=113/114 n=559 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.832+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.832+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.832+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:eb806c57:::obj373:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) pg 0x4ef0000 2026-02-11T23:31:27.832+0000 7f414b98b640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:eb806c57:::obj373:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 1.0 e 117/117 lb 1:eb806c57:::obj373:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:eb29d295:::obj554:head local-lis/les=113/114 n=559 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.832+0000 7f414b98b640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.832+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eb29d295:::obj554:head local-lis/les=113/114 n=559 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 117/117 lb 1:eb806c57:::obj373:head) v3 2026-02-11T23:31:27.832+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eb29d295:::obj554:head local-lis/les=113/114 n=559 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 117/117 lb 1:eb806c57:::obj373:head) 2026-02-11T23:31:27.832+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.832+0000 7f414b98b640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:eb806c57:::obj373:head) v3 finish latency 0.000012 2026-02-11T23:31:27.832+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.833+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.833+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eb806c57:::obj373:head local-lis/les=113/114 n=560 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.833+0000 7f414f993640 20 osd.0 117 get_con_osd_cluster to osd.1 from_epoch 117 2026-02-11T23:31:27.833+0000 7f414f993640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:27.833+0000 7f414f993640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:27.833+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e117/117) -- 0x600b500 con 0x4acb000 2026-02-11T23:31:27.833+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 117/113 [PushReplyOp(1:ecbe4ab7:::obj125:head)]) -- 0x5daa900 con 0x4acb000 2026-02-11T23:31:27.833+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.833+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1158 ==== MOSDPGPush(1.0 117/113 [PushOp(1:ed234181:::obj472:head, version: 111'472, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ed234181:::obj472:head@111'472, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x38156c0 con 0x4acb000 2026-02-11T23:31:27.833+0000 7f416abfa640 15 osd.0 117 enqueue_op MOSDPGPush(1.0 117/113 [PushOp(1:ed234181:::obj472:head, version: 111'472, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ed234181:::obj472:head@111'472, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 117 2026-02-11T23:31:27.833+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:ed234181:::obj472:head, version: 111'472, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ed234181:::obj472:head@111'472, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e117) 2026-02-11T23:31:27.833+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.833+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:ed234181:::obj472:head, version: 111'472, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ed234181:::obj472:head@111'472, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) queued 2026-02-11T23:31:27.833+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1159 ==== pg_backfill(progress 1.0 e 117/117 lb 1:ecbe4ab7:::obj125:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e100 con 0x4acb000 2026-02-11T23:31:27.833+0000 7f416abfa640 15 osd.0 117 enqueue_op pg_backfill(progress 1.0 e 117/117 lb 1:ecbe4ab7:::obj125:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 117 2026-02-11T23:31:27.833+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:ecbe4ab7:::obj125:head) v3) class_id 0 prio 15 cost 0 e117) 2026-02-11T23:31:27.833+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.833+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:ed234181:::obj472:head, version: 111'472, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ed234181:::obj472:head@111'472, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) pg 0x4ef0000 2026-02-11T23:31:27.833+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.833+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:ed234181:::obj472:head, version: 111'472, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ed234181:::obj472:head@111'472, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000042 MOSDPGPush(1.0 117/113 [PushOp(1:ed234181:::obj472:head, version: 111'472, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ed234181:::obj472:head@111'472, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:eb806c57:::obj373:head local-lis/les=113/114 n=560 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.833+0000 7f414b98b640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.833+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eb806c57:::obj373:head local-lis/les=113/114 n=560 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 117/113 [PushOp(1:ed234181:::obj472:head, version: 111'472, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ed234181:::obj472:head@111'472, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.833+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eb806c57:::obj373:head local-lis/les=113/114 n=560 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:ed234181:::obj472:head@111'472, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.833+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.833+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:ecbe4ab7:::obj125:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) queued 2026-02-11T23:31:27.833+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eb806c57:::obj373:head local-lis/les=113/114 n=560 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:ed234181:::obj472:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.833+0000 7f414b98b640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eb806c57:::obj373:head local-lis/les=113/114 n=560 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:ed234181:::obj472:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.833+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eb806c57:::obj373:head local-lis/les=113/114 n=560 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:ed234181:::obj472:head 2026-02-11T23:31:27.833+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eb806c57:::obj373:head local-lis/les=113/114 n=560 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:ed234181:::obj472:head v 111'472 2026-02-11T23:31:27.833+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eb806c57:::obj373:head local-lis/les=113/114 n=560 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.833+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.833+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:ed234181:::obj472:head, version: 111'472, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ed234181:::obj472:head@111'472, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.833+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.833+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.833+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:ecbe4ab7:::obj125:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) pg 0x4ef0000 2026-02-11T23:31:27.833+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:ecbe4ab7:::obj125:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 1.0 e 117/117 lb 1:ecbe4ab7:::obj125:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:eb806c57:::obj373:head local-lis/les=113/114 n=560 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.833+0000 7f414f993640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.833+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eb806c57:::obj373:head local-lis/les=113/114 n=560 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 117/117 lb 1:ecbe4ab7:::obj125:head) v3 2026-02-11T23:31:27.833+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eb806c57:::obj373:head local-lis/les=113/114 n=560 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 117/117 lb 1:ecbe4ab7:::obj125:head) 2026-02-11T23:31:27.833+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.833+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:ecbe4ab7:::obj125:head) v3 finish latency 0.000011 2026-02-11T23:31:27.833+0000 7f414f993640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ecbe4ab7:::obj125:head local-lis/les=113/114 n=561 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.833+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.833+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.833+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ecbe4ab7:::obj125:head local-lis/les=113/114 n=561 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.833+0000 7f414f993640 20 osd.0 117 get_con_osd_cluster to osd.1 from_epoch 117 2026-02-11T23:31:27.833+0000 7f414f993640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:27.833+0000 7f414f993640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:27.833+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e117/117) -- 0x600a900 con 0x4acb000 2026-02-11T23:31:27.833+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 117/113 [PushReplyOp(1:ed234181:::obj472:head)]) -- 0x5fd2780 con 0x4acb000 2026-02-11T23:31:27.833+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.833+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1160 ==== MOSDPGPush(1.0 117/113 [PushOp(1:ee44f25c:::obj334:head, version: 111'334, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ee44f25c:::obj334:head@111'334, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c9c0 con 0x4acb000 2026-02-11T23:31:27.833+0000 7f416abfa640 15 osd.0 117 enqueue_op MOSDPGPush(1.0 117/113 [PushOp(1:ee44f25c:::obj334:head, version: 111'334, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ee44f25c:::obj334:head@111'334, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 117 2026-02-11T23:31:27.833+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:ee44f25c:::obj334:head, version: 111'334, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ee44f25c:::obj334:head@111'334, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e117) 2026-02-11T23:31:27.833+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.833+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1161 ==== pg_backfill(progress 1.0 e 117/117 lb 1:ed234181:::obj472:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e680 con 0x4acb000 2026-02-11T23:31:27.833+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:ee44f25c:::obj334:head, version: 111'334, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ee44f25c:::obj334:head@111'334, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) queued 2026-02-11T23:31:27.833+0000 7f416abfa640 15 osd.0 117 enqueue_op pg_backfill(progress 1.0 e 117/117 lb 1:ed234181:::obj472:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 117 2026-02-11T23:31:27.833+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.833+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:ed234181:::obj472:head) v3) class_id 0 prio 15 cost 0 e117) 2026-02-11T23:31:27.833+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:ee44f25c:::obj334:head, version: 111'334, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ee44f25c:::obj334:head@111'334, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) pg 0x4ef0000 2026-02-11T23:31:27.834+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.834+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:ee44f25c:::obj334:head, version: 111'334, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ee44f25c:::obj334:head@111'334, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 117/113 [PushOp(1:ee44f25c:::obj334:head, version: 111'334, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ee44f25c:::obj334:head@111'334, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:ecbe4ab7:::obj125:head local-lis/les=113/114 n=561 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.834+0000 7f414b98b640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.834+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ecbe4ab7:::obj125:head local-lis/les=113/114 n=561 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 117/113 [PushOp(1:ee44f25c:::obj334:head, version: 111'334, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ee44f25c:::obj334:head@111'334, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.834+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ecbe4ab7:::obj125:head local-lis/les=113/114 n=561 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:ee44f25c:::obj334:head@111'334, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.834+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.834+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:ed234181:::obj472:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) queued 2026-02-11T23:31:27.834+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ecbe4ab7:::obj125:head local-lis/les=113/114 n=561 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:ee44f25c:::obj334:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.834+0000 7f414b98b640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ecbe4ab7:::obj125:head local-lis/les=113/114 n=561 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:ee44f25c:::obj334:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.834+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ecbe4ab7:::obj125:head local-lis/les=113/114 n=561 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:ee44f25c:::obj334:head 2026-02-11T23:31:27.834+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ecbe4ab7:::obj125:head local-lis/les=113/114 n=561 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:ee44f25c:::obj334:head v 111'334 2026-02-11T23:31:27.834+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ecbe4ab7:::obj125:head local-lis/les=113/114 n=561 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.834+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.834+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:ee44f25c:::obj334:head, version: 111'334, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ee44f25c:::obj334:head@111'334, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:27.834+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.834+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.834+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:ed234181:::obj472:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) pg 0x4ef0000 2026-02-11T23:31:27.834+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:ed234181:::obj472:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 1.0 e 117/117 lb 1:ed234181:::obj472:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:ecbe4ab7:::obj125:head local-lis/les=113/114 n=561 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.834+0000 7f414f993640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.834+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ecbe4ab7:::obj125:head local-lis/les=113/114 n=561 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 117/117 lb 1:ed234181:::obj472:head) v3 2026-02-11T23:31:27.834+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ecbe4ab7:::obj125:head local-lis/les=113/114 n=561 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 117/117 lb 1:ed234181:::obj472:head) 2026-02-11T23:31:27.834+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.834+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:ed234181:::obj472:head) v3 finish latency 0.000009 2026-02-11T23:31:27.834+0000 7f414f993640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ed234181:::obj472:head local-lis/les=113/114 n=562 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.834+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.834+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.834+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ed234181:::obj472:head local-lis/les=113/114 n=562 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.834+0000 7f414f993640 20 osd.0 117 get_con_osd_cluster to osd.1 from_epoch 117 2026-02-11T23:31:27.834+0000 7f414f993640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:27.834+0000 7f414f993640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:27.834+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e117/117) -- 0x5d1d380 con 0x4acb000 2026-02-11T23:31:27.834+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 117/113 [PushReplyOp(1:ee44f25c:::obj334:head)]) -- 0x5fd2480 con 0x4acb000 2026-02-11T23:31:27.834+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.834+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1162 ==== MOSDPGPush(1.0 117/113 [PushOp(1:eeae2a94:::obj11:head, version: 111'11, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eeae2a94:::obj11:head@111'11, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x3814ea0 con 0x4acb000 2026-02-11T23:31:27.834+0000 7f416abfa640 15 osd.0 117 enqueue_op MOSDPGPush(1.0 117/113 [PushOp(1:eeae2a94:::obj11:head, version: 111'11, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eeae2a94:::obj11:head@111'11, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 117 2026-02-11T23:31:27.834+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:eeae2a94:::obj11:head, version: 111'11, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eeae2a94:::obj11:head@111'11, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e117) 2026-02-11T23:31:27.834+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.834+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1163 ==== pg_backfill(progress 1.0 e 117/117 lb 1:ee44f25c:::obj334:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2d080 con 0x4acb000 2026-02-11T23:31:27.834+0000 7f416abfa640 15 osd.0 117 enqueue_op pg_backfill(progress 1.0 e 117/117 lb 1:ee44f25c:::obj334:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 117 2026-02-11T23:31:27.834+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:ee44f25c:::obj334:head) v3) class_id 0 prio 15 cost 0 e117) 2026-02-11T23:31:27.834+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:eeae2a94:::obj11:head, version: 111'11, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eeae2a94:::obj11:head@111'11, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) queued 2026-02-11T23:31:27.834+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.834+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:eeae2a94:::obj11:head, version: 111'11, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eeae2a94:::obj11:head@111'11, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) pg 0x4ef0000 2026-02-11T23:31:27.834+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:eeae2a94:::obj11:head, version: 111'11, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eeae2a94:::obj11:head@111'11, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000041 MOSDPGPush(1.0 117/113 [PushOp(1:eeae2a94:::obj11:head, version: 111'11, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eeae2a94:::obj11:head@111'11, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:ed234181:::obj472:head local-lis/les=113/114 n=562 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.834+0000 7f414b98b640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.834+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.834+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ed234181:::obj472:head local-lis/les=113/114 n=562 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 117/113 [PushOp(1:eeae2a94:::obj11:head, version: 111'11, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eeae2a94:::obj11:head@111'11, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.834+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:ee44f25c:::obj334:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) queued 2026-02-11T23:31:27.834+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ed234181:::obj472:head local-lis/les=113/114 n=562 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:eeae2a94:::obj11:head@111'11, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.834+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ed234181:::obj472:head local-lis/les=113/114 n=562 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:eeae2a94:::obj11:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.834+0000 7f414b98b640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ed234181:::obj472:head local-lis/les=113/114 n=562 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:eeae2a94:::obj11:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.834+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ed234181:::obj472:head local-lis/les=113/114 n=562 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:eeae2a94:::obj11:head 2026-02-11T23:31:27.834+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ed234181:::obj472:head local-lis/les=113/114 n=562 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:eeae2a94:::obj11:head v 111'11 2026-02-11T23:31:27.834+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ed234181:::obj472:head local-lis/les=113/114 n=562 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.834+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.834+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:eeae2a94:::obj11:head, version: 111'11, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eeae2a94:::obj11:head@111'11, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:27.834+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.834+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.834+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:ee44f25c:::obj334:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) pg 0x4ef0000 2026-02-11T23:31:27.834+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:ee44f25c:::obj334:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 1.0 e 117/117 lb 1:ee44f25c:::obj334:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:ed234181:::obj472:head local-lis/les=113/114 n=562 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.834+0000 7f414f993640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.834+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ed234181:::obj472:head local-lis/les=113/114 n=562 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 117/117 lb 1:ee44f25c:::obj334:head) v3 2026-02-11T23:31:27.834+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ed234181:::obj472:head local-lis/les=113/114 n=562 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 117/117 lb 1:ee44f25c:::obj334:head) 2026-02-11T23:31:27.834+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.834+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:ee44f25c:::obj334:head) v3 finish latency 0.000011 2026-02-11T23:31:27.834+0000 7f414f993640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ee44f25c:::obj334:head local-lis/les=113/114 n=563 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.834+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.834+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.834+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ee44f25c:::obj334:head local-lis/les=113/114 n=563 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.834+0000 7f414f993640 20 osd.0 117 get_con_osd_cluster to osd.1 from_epoch 117 2026-02-11T23:31:27.834+0000 7f414f993640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:27.834+0000 7f414f993640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:27.834+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e117/117) -- 0x5f2c300 con 0x4acb000 2026-02-11T23:31:27.834+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 117/113 [PushReplyOp(1:eeae2a94:::obj11:head)]) -- 0x5f1d200 con 0x4acb000 2026-02-11T23:31:27.834+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.835+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1164 ==== MOSDPGPush(1.0 117/113 [PushOp(1:eff0a739:::obj127:head, version: 111'127, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eff0a739:::obj127:head@111'127, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3815520 con 0x4acb000 2026-02-11T23:31:27.835+0000 7f416abfa640 15 osd.0 117 enqueue_op MOSDPGPush(1.0 117/113 [PushOp(1:eff0a739:::obj127:head, version: 111'127, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eff0a739:::obj127:head@111'127, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 117 2026-02-11T23:31:27.835+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:eff0a739:::obj127:head, version: 111'127, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eff0a739:::obj127:head@111'127, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e117) 2026-02-11T23:31:27.835+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.835+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1165 ==== pg_backfill(progress 1.0 e 117/117 lb 1:eeae2a94:::obj11:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d2f700 con 0x4acb000 2026-02-11T23:31:27.835+0000 7f416abfa640 15 osd.0 117 enqueue_op pg_backfill(progress 1.0 e 117/117 lb 1:eeae2a94:::obj11:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 117 2026-02-11T23:31:27.835+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:eff0a739:::obj127:head, version: 111'127, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eff0a739:::obj127:head@111'127, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) queued 2026-02-11T23:31:27.835+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.835+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:eff0a739:::obj127:head, version: 111'127, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eff0a739:::obj127:head@111'127, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) pg 0x4ef0000 2026-02-11T23:31:27.835+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:eeae2a94:::obj11:head) v3) class_id 0 prio 15 cost 0 e117) 2026-02-11T23:31:27.835+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.835+0000 7f414f993640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:eff0a739:::obj127:head, version: 111'127, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eff0a739:::obj127:head@111'127, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 117/113 [PushOp(1:eff0a739:::obj127:head, version: 111'127, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eff0a739:::obj127:head@111'127, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:ee44f25c:::obj334:head local-lis/les=113/114 n=563 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.835+0000 7f414f993640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.835+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ee44f25c:::obj334:head local-lis/les=113/114 n=563 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 117/113 [PushOp(1:eff0a739:::obj127:head, version: 111'127, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eff0a739:::obj127:head@111'127, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.835+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.835+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ee44f25c:::obj334:head local-lis/les=113/114 n=563 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:eff0a739:::obj127:head@111'127, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.835+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:eeae2a94:::obj11:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) queued 2026-02-11T23:31:27.835+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ee44f25c:::obj334:head local-lis/les=113/114 n=563 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:eff0a739:::obj127:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.835+0000 7f414f993640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ee44f25c:::obj334:head local-lis/les=113/114 n=563 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:eff0a739:::obj127:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.835+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ee44f25c:::obj334:head local-lis/les=113/114 n=563 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:eff0a739:::obj127:head 2026-02-11T23:31:27.835+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ee44f25c:::obj334:head local-lis/les=113/114 n=563 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:eff0a739:::obj127:head v 111'127 2026-02-11T23:31:27.835+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ee44f25c:::obj334:head local-lis/les=113/114 n=563 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.835+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.835+0000 7f414f993640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:eff0a739:::obj127:head, version: 111'127, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eff0a739:::obj127:head@111'127, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:27.835+0000 7f414f993640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ee44f25c:::obj334:head local-lis/les=113/114 n=563 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.835+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.835+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.835+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:eeae2a94:::obj11:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) pg 0x4ef0000 2026-02-11T23:31:27.835+0000 7f414b98b640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:eeae2a94:::obj11:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 1.0 e 117/117 lb 1:eeae2a94:::obj11:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:ee44f25c:::obj334:head local-lis/les=113/114 n=563 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.835+0000 7f414b98b640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.835+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ee44f25c:::obj334:head local-lis/les=113/114 n=563 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 117/117 lb 1:eeae2a94:::obj11:head) v3 2026-02-11T23:31:27.835+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ee44f25c:::obj334:head local-lis/les=113/114 n=563 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 117/117 lb 1:eeae2a94:::obj11:head) 2026-02-11T23:31:27.835+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.835+0000 7f414b98b640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:eeae2a94:::obj11:head) v3 finish latency 0.000011 2026-02-11T23:31:27.835+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.835+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.835+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eeae2a94:::obj11:head local-lis/les=113/114 n=564 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.835+0000 7f414f993640 20 osd.0 117 get_con_osd_cluster to osd.1 from_epoch 117 2026-02-11T23:31:27.835+0000 7f414f993640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:27.835+0000 7f414f993640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:27.835+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e117/117) -- 0x6171680 con 0x4acb000 2026-02-11T23:31:27.835+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 117/113 [PushReplyOp(1:eff0a739:::obj127:head)]) -- 0x6000180 con 0x4acb000 2026-02-11T23:31:27.835+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.835+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1166 ==== MOSDPGPush(1.0 117/113 [PushOp(1:f0a12295:::obj487:head, version: 111'487, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f0a12295:::obj487:head@111'487, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5646b60 con 0x4acb000 2026-02-11T23:31:27.835+0000 7f416abfa640 15 osd.0 117 enqueue_op MOSDPGPush(1.0 117/113 [PushOp(1:f0a12295:::obj487:head, version: 111'487, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f0a12295:::obj487:head@111'487, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 117 2026-02-11T23:31:27.835+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f0a12295:::obj487:head, version: 111'487, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f0a12295:::obj487:head@111'487, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e117) 2026-02-11T23:31:27.835+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.835+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f0a12295:::obj487:head, version: 111'487, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f0a12295:::obj487:head@111'487, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) queued 2026-02-11T23:31:27.835+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.835+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f0a12295:::obj487:head, version: 111'487, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f0a12295:::obj487:head@111'487, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) pg 0x4ef0000 2026-02-11T23:31:27.835+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.835+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:f0a12295:::obj487:head, version: 111'487, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f0a12295:::obj487:head@111'487, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 117/113 [PushOp(1:f0a12295:::obj487:head, version: 111'487, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f0a12295:::obj487:head@111'487, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:eeae2a94:::obj11:head local-lis/les=113/114 n=564 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.835+0000 7f414b98b640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.835+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eeae2a94:::obj11:head local-lis/les=113/114 n=564 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 117/113 [PushOp(1:f0a12295:::obj487:head, version: 111'487, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f0a12295:::obj487:head@111'487, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.835+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eeae2a94:::obj11:head local-lis/les=113/114 n=564 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:f0a12295:::obj487:head@111'487, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.835+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eeae2a94:::obj11:head local-lis/les=113/114 n=564 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:f0a12295:::obj487:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.835+0000 7f414b98b640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eeae2a94:::obj11:head local-lis/les=113/114 n=564 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:f0a12295:::obj487:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.835+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eeae2a94:::obj11:head local-lis/les=113/114 n=564 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:f0a12295:::obj487:head 2026-02-11T23:31:27.835+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eeae2a94:::obj11:head local-lis/les=113/114 n=564 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:f0a12295:::obj487:head v 111'487 2026-02-11T23:31:27.835+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eeae2a94:::obj11:head local-lis/les=113/114 n=564 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.835+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.835+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:f0a12295:::obj487:head, version: 111'487, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f0a12295:::obj487:head@111'487, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.835+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.835+0000 7f414f993640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eeae2a94:::obj11:head local-lis/les=113/114 n=564 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.835+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.835+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1167 ==== pg_backfill(progress 1.0 e 117/117 lb 1:eff0a739:::obj127:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f11600 con 0x4acb000 2026-02-11T23:31:27.835+0000 7f416abfa640 15 osd.0 117 enqueue_op pg_backfill(progress 1.0 e 117/117 lb 1:eff0a739:::obj127:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 117 2026-02-11T23:31:27.835+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:eff0a739:::obj127:head) v3) class_id 0 prio 15 cost 0 e117) 2026-02-11T23:31:27.835+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.835+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:eff0a739:::obj127:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) queued 2026-02-11T23:31:27.835+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.835+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.835+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:eff0a739:::obj127:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) pg 0x4ef0000 2026-02-11T23:31:27.835+0000 7f414b98b640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:eff0a739:::obj127:head) v3 prio 15 cost 0 latency 0.000024 pg_backfill(progress 1.0 e 117/117 lb 1:eff0a739:::obj127:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:eeae2a94:::obj11:head local-lis/les=113/114 n=564 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.835+0000 7f414b98b640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.835+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eeae2a94:::obj11:head local-lis/les=113/114 n=564 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 117/117 lb 1:eff0a739:::obj127:head) v3 2026-02-11T23:31:27.835+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eeae2a94:::obj11:head local-lis/les=113/114 n=564 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 117/117 lb 1:eff0a739:::obj127:head) 2026-02-11T23:31:27.835+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.835+0000 7f414b98b640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:eff0a739:::obj127:head) v3 finish latency 0.000011 2026-02-11T23:31:27.835+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.835+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.835+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eff0a739:::obj127:head local-lis/les=113/114 n=565 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.835+0000 7f414f993640 20 osd.0 117 get_con_osd_cluster to osd.1 from_epoch 117 2026-02-11T23:31:27.835+0000 7f414f993640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:27.835+0000 7f414f993640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:27.835+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e117/117) -- 0x5f2c600 con 0x4acb000 2026-02-11T23:31:27.835+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 117/113 [PushReplyOp(1:f0a12295:::obj487:head)]) -- 0x5f1d500 con 0x4acb000 2026-02-11T23:31:27.835+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.836+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.836+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.836+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1168 ==== MOSDPGPush(1.0 117/113 [PushOp(1:f1182822:::obj193:head, version: 111'193, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f1182822:::obj193:head@111'193, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56ca820 con 0x4acb000 2026-02-11T23:31:27.836+0000 7f416abfa640 15 osd.0 117 enqueue_op MOSDPGPush(1.0 117/113 [PushOp(1:f1182822:::obj193:head, version: 111'193, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f1182822:::obj193:head@111'193, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 117 2026-02-11T23:31:27.836+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f1182822:::obj193:head, version: 111'193, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f1182822:::obj193:head@111'193, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e117) 2026-02-11T23:31:27.836+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.836+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1169 ==== pg_backfill(progress 1.0 e 117/117 lb 1:f0a12295:::obj487:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f11080 con 0x4acb000 2026-02-11T23:31:27.836+0000 7f416abfa640 15 osd.0 117 enqueue_op pg_backfill(progress 1.0 e 117/117 lb 1:f0a12295:::obj487:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 117 2026-02-11T23:31:27.836+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f1182822:::obj193:head, version: 111'193, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f1182822:::obj193:head@111'193, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) queued 2026-02-11T23:31:27.836+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f0a12295:::obj487:head) v3) class_id 0 prio 15 cost 0 e117) 2026-02-11T23:31:27.836+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.836+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f0a12295:::obj487:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) queued 2026-02-11T23:31:27.836+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.836+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f1182822:::obj193:head, version: 111'193, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f1182822:::obj193:head@111'193, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) pg 0x4ef0000 2026-02-11T23:31:27.836+0000 7f414f993640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:f1182822:::obj193:head, version: 111'193, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f1182822:::obj193:head@111'193, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000040 MOSDPGPush(1.0 117/113 [PushOp(1:f1182822:::obj193:head, version: 111'193, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f1182822:::obj193:head@111'193, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:eff0a739:::obj127:head local-lis/les=113/114 n=565 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.836+0000 7f414f993640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.836+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eff0a739:::obj127:head local-lis/les=113/114 n=565 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 117/113 [PushOp(1:f1182822:::obj193:head, version: 111'193, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f1182822:::obj193:head@111'193, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.836+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eff0a739:::obj127:head local-lis/les=113/114 n=565 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:f1182822:::obj193:head@111'193, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.836+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eff0a739:::obj127:head local-lis/les=113/114 n=565 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:f1182822:::obj193:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.836+0000 7f414f993640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eff0a739:::obj127:head local-lis/les=113/114 n=565 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:f1182822:::obj193:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.836+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eff0a739:::obj127:head local-lis/les=113/114 n=565 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:f1182822:::obj193:head 2026-02-11T23:31:27.836+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eff0a739:::obj127:head local-lis/les=113/114 n=565 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:f1182822:::obj193:head v 111'193 2026-02-11T23:31:27.836+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eff0a739:::obj127:head local-lis/les=113/114 n=565 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.836+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.836+0000 7f414f993640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:f1182822:::obj193:head, version: 111'193, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f1182822:::obj193:head@111'193, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.836+0000 7f414f993640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eff0a739:::obj127:head local-lis/les=113/114 n=565 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.836+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.836+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.836+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f0a12295:::obj487:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) pg 0x4ef0000 2026-02-11T23:31:27.836+0000 7f414b98b640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:f0a12295:::obj487:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 1.0 e 117/117 lb 1:f0a12295:::obj487:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:eff0a739:::obj127:head local-lis/les=113/114 n=565 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.836+0000 7f414b98b640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.836+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eff0a739:::obj127:head local-lis/les=113/114 n=565 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 117/117 lb 1:f0a12295:::obj487:head) v3 2026-02-11T23:31:27.836+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:eff0a739:::obj127:head local-lis/les=113/114 n=565 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 117/117 lb 1:f0a12295:::obj487:head) 2026-02-11T23:31:27.836+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.836+0000 7f414b98b640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:f0a12295:::obj487:head) v3 finish latency 0.000009 2026-02-11T23:31:27.836+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.836+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.836+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f0a12295:::obj487:head local-lis/les=113/114 n=566 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.836+0000 7f414f993640 20 osd.0 117 get_con_osd_cluster to osd.1 from_epoch 117 2026-02-11T23:31:27.836+0000 7f414f993640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:27.836+0000 7f414f993640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:27.836+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e117/117) -- 0x5d1cd80 con 0x4acb000 2026-02-11T23:31:27.836+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 117/113 [PushReplyOp(1:f1182822:::obj193:head)]) -- 0x5fcfe00 con 0x4acb000 2026-02-11T23:31:27.836+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.836+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1170 ==== MOSDPGPush(1.0 117/113 [PushOp(1:f11aef48:::obj58:head, version: 111'58, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f11aef48:::obj58:head@111'58, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x56cad00 con 0x4acb000 2026-02-11T23:31:27.836+0000 7f416abfa640 15 osd.0 117 enqueue_op MOSDPGPush(1.0 117/113 [PushOp(1:f11aef48:::obj58:head, version: 111'58, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f11aef48:::obj58:head@111'58, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 117 2026-02-11T23:31:27.836+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f11aef48:::obj58:head, version: 111'58, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f11aef48:::obj58:head@111'58, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e117) 2026-02-11T23:31:27.836+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.836+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1171 ==== pg_backfill(progress 1.0 e 117/117 lb 1:f1182822:::obj193:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f10b00 con 0x4acb000 2026-02-11T23:31:27.836+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f11aef48:::obj58:head, version: 111'58, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f11aef48:::obj58:head@111'58, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) queued 2026-02-11T23:31:27.836+0000 7f416abfa640 15 osd.0 117 enqueue_op pg_backfill(progress 1.0 e 117/117 lb 1:f1182822:::obj193:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 117 2026-02-11T23:31:27.836+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f1182822:::obj193:head) v3) class_id 0 prio 15 cost 0 e117) 2026-02-11T23:31:27.836+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.836+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f11aef48:::obj58:head, version: 111'58, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f11aef48:::obj58:head@111'58, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) pg 0x4ef0000 2026-02-11T23:31:27.836+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.836+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:f11aef48:::obj58:head, version: 111'58, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f11aef48:::obj58:head@111'58, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 117/113 [PushOp(1:f11aef48:::obj58:head, version: 111'58, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f11aef48:::obj58:head@111'58, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:f0a12295:::obj487:head local-lis/les=113/114 n=566 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.836+0000 7f414b98b640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.836+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f0a12295:::obj487:head local-lis/les=113/114 n=566 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 117/113 [PushOp(1:f11aef48:::obj58:head, version: 111'58, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f11aef48:::obj58:head@111'58, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.836+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f0a12295:::obj487:head local-lis/les=113/114 n=566 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:f11aef48:::obj58:head@111'58, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.836+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.836+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f1182822:::obj193:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) queued 2026-02-11T23:31:27.836+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f0a12295:::obj487:head local-lis/les=113/114 n=566 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:f11aef48:::obj58:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.836+0000 7f414b98b640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f0a12295:::obj487:head local-lis/les=113/114 n=566 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:f11aef48:::obj58:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.836+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f0a12295:::obj487:head local-lis/les=113/114 n=566 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:f11aef48:::obj58:head 2026-02-11T23:31:27.836+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f0a12295:::obj487:head local-lis/les=113/114 n=566 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:f11aef48:::obj58:head v 111'58 2026-02-11T23:31:27.836+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f0a12295:::obj487:head local-lis/les=113/114 n=566 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.836+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.836+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:f11aef48:::obj58:head, version: 111'58, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f11aef48:::obj58:head@111'58, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.836+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.836+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.836+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f1182822:::obj193:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) pg 0x4ef0000 2026-02-11T23:31:27.836+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:f1182822:::obj193:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 1.0 e 117/117 lb 1:f1182822:::obj193:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:f0a12295:::obj487:head local-lis/les=113/114 n=566 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.836+0000 7f414f993640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.836+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f0a12295:::obj487:head local-lis/les=113/114 n=566 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 117/117 lb 1:f1182822:::obj193:head) v3 2026-02-11T23:31:27.836+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f0a12295:::obj487:head local-lis/les=113/114 n=566 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 117/117 lb 1:f1182822:::obj193:head) 2026-02-11T23:31:27.836+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.836+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:f1182822:::obj193:head) v3 finish latency 0.000011 2026-02-11T23:31:27.836+0000 7f414f993640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f1182822:::obj193:head local-lis/les=113/114 n=567 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.836+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.837+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.837+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f1182822:::obj193:head local-lis/les=113/114 n=567 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.837+0000 7f414f993640 20 osd.0 117 get_con_osd_cluster to osd.1 from_epoch 117 2026-02-11T23:31:27.837+0000 7f414f993640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:27.837+0000 7f414f993640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:27.837+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e117/117) -- 0x5f2dc80 con 0x4acb000 2026-02-11T23:31:27.837+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 117/113 [PushReplyOp(1:f11aef48:::obj58:head)]) -- 0x5f1d080 con 0x4acb000 2026-02-11T23:31:27.837+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.837+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1172 ==== MOSDPGPush(1.0 117/113 [PushOp(1:f13c731e:::obj449:head, version: 111'449, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f13c731e:::obj449:head@111'449, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56171e0 con 0x4acb000 2026-02-11T23:31:27.837+0000 7f416abfa640 15 osd.0 117 enqueue_op MOSDPGPush(1.0 117/113 [PushOp(1:f13c731e:::obj449:head, version: 111'449, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f13c731e:::obj449:head@111'449, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 117 2026-02-11T23:31:27.837+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f13c731e:::obj449:head, version: 111'449, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f13c731e:::obj449:head@111'449, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e117) 2026-02-11T23:31:27.837+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.837+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1173 ==== pg_backfill(progress 1.0 e 117/117 lb 1:f11aef48:::obj58:head) ==== 1061+0+0 (unknown 0 0 0) 0x5f10580 con 0x4acb000 2026-02-11T23:31:27.837+0000 7f416abfa640 15 osd.0 117 enqueue_op pg_backfill(progress 1.0 e 117/117 lb 1:f11aef48:::obj58:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 117 2026-02-11T23:31:27.837+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f13c731e:::obj449:head, version: 111'449, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f13c731e:::obj449:head@111'449, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) queued 2026-02-11T23:31:27.837+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f11aef48:::obj58:head) v3) class_id 0 prio 15 cost 0 e117) 2026-02-11T23:31:27.837+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.837+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f11aef48:::obj58:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) queued 2026-02-11T23:31:27.837+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.837+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f13c731e:::obj449:head, version: 111'449, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f13c731e:::obj449:head@111'449, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) pg 0x4ef0000 2026-02-11T23:31:27.837+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:f13c731e:::obj449:head, version: 111'449, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f13c731e:::obj449:head@111'449, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000041 MOSDPGPush(1.0 117/113 [PushOp(1:f13c731e:::obj449:head, version: 111'449, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f13c731e:::obj449:head@111'449, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:f1182822:::obj193:head local-lis/les=113/114 n=567 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.837+0000 7f414b98b640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.837+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f1182822:::obj193:head local-lis/les=113/114 n=567 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 117/113 [PushOp(1:f13c731e:::obj449:head, version: 111'449, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f13c731e:::obj449:head@111'449, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.837+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f1182822:::obj193:head local-lis/les=113/114 n=567 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:f13c731e:::obj449:head@111'449, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.837+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f1182822:::obj193:head local-lis/les=113/114 n=567 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:f13c731e:::obj449:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.837+0000 7f414b98b640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f1182822:::obj193:head local-lis/les=113/114 n=567 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:f13c731e:::obj449:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.837+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f1182822:::obj193:head local-lis/les=113/114 n=567 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:f13c731e:::obj449:head 2026-02-11T23:31:27.837+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f1182822:::obj193:head local-lis/les=113/114 n=567 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:f13c731e:::obj449:head v 111'449 2026-02-11T23:31:27.837+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f1182822:::obj193:head local-lis/les=113/114 n=567 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.837+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.837+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:f13c731e:::obj449:head, version: 111'449, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f13c731e:::obj449:head@111'449, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:27.837+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.837+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.837+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f11aef48:::obj58:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) pg 0x4ef0000 2026-02-11T23:31:27.837+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:f11aef48:::obj58:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 1.0 e 117/117 lb 1:f11aef48:::obj58:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:f1182822:::obj193:head local-lis/les=113/114 n=567 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.837+0000 7f414f993640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.837+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f1182822:::obj193:head local-lis/les=113/114 n=567 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 117/117 lb 1:f11aef48:::obj58:head) v3 2026-02-11T23:31:27.837+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f1182822:::obj193:head local-lis/les=113/114 n=567 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 117/117 lb 1:f11aef48:::obj58:head) 2026-02-11T23:31:27.837+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.837+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:f11aef48:::obj58:head) v3 finish latency 0.000009 2026-02-11T23:31:27.837+0000 7f414f993640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f11aef48:::obj58:head local-lis/les=113/114 n=568 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.837+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.837+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.837+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f11aef48:::obj58:head local-lis/les=113/114 n=568 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.837+0000 7f414f993640 20 osd.0 117 get_con_osd_cluster to osd.1 from_epoch 117 2026-02-11T23:31:27.837+0000 7f414f993640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:27.837+0000 7f414f993640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:27.837+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e117/117) -- 0x5e7ed80 con 0x4acb000 2026-02-11T23:31:27.837+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 117/113 [PushReplyOp(1:f13c731e:::obj449:head)]) -- 0x5fd2180 con 0x4acb000 2026-02-11T23:31:27.837+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.837+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1174 ==== MOSDPGPush(1.0 117/113 [PushOp(1:f22dcb20:::obj569:head, version: 111'569, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f22dcb20:::obj569:head@111'569, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5617520 con 0x4acb000 2026-02-11T23:31:27.837+0000 7f416abfa640 15 osd.0 117 enqueue_op MOSDPGPush(1.0 117/113 [PushOp(1:f22dcb20:::obj569:head, version: 111'569, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f22dcb20:::obj569:head@111'569, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 117 2026-02-11T23:31:27.837+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f22dcb20:::obj569:head, version: 111'569, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f22dcb20:::obj569:head@111'569, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e117) 2026-02-11T23:31:27.837+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.837+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1175 ==== pg_backfill(progress 1.0 e 117/117 lb 1:f13c731e:::obj449:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f10000 con 0x4acb000 2026-02-11T23:31:27.837+0000 7f416abfa640 15 osd.0 117 enqueue_op pg_backfill(progress 1.0 e 117/117 lb 1:f13c731e:::obj449:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 117 2026-02-11T23:31:27.837+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f22dcb20:::obj569:head, version: 111'569, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f22dcb20:::obj569:head@111'569, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) queued 2026-02-11T23:31:27.837+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f13c731e:::obj449:head) v3) class_id 0 prio 15 cost 0 e117) 2026-02-11T23:31:27.837+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.837+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f13c731e:::obj449:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) queued 2026-02-11T23:31:27.837+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.837+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f22dcb20:::obj569:head, version: 111'569, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f22dcb20:::obj569:head@111'569, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) pg 0x4ef0000 2026-02-11T23:31:27.837+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:f22dcb20:::obj569:head, version: 111'569, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f22dcb20:::obj569:head@111'569, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000041 MOSDPGPush(1.0 117/113 [PushOp(1:f22dcb20:::obj569:head, version: 111'569, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f22dcb20:::obj569:head@111'569, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:f11aef48:::obj58:head local-lis/les=113/114 n=568 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.837+0000 7f414b98b640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.837+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f11aef48:::obj58:head local-lis/les=113/114 n=568 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 117/113 [PushOp(1:f22dcb20:::obj569:head, version: 111'569, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f22dcb20:::obj569:head@111'569, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.837+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f11aef48:::obj58:head local-lis/les=113/114 n=568 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:f22dcb20:::obj569:head@111'569, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.837+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f11aef48:::obj58:head local-lis/les=113/114 n=568 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:f22dcb20:::obj569:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.837+0000 7f414b98b640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f11aef48:::obj58:head local-lis/les=113/114 n=568 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:f22dcb20:::obj569:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.837+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f11aef48:::obj58:head local-lis/les=113/114 n=568 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:f22dcb20:::obj569:head 2026-02-11T23:31:27.837+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f11aef48:::obj58:head local-lis/les=113/114 n=568 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:f22dcb20:::obj569:head v 111'569 2026-02-11T23:31:27.837+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f11aef48:::obj58:head local-lis/les=113/114 n=568 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.837+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.837+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:f22dcb20:::obj569:head, version: 111'569, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f22dcb20:::obj569:head@111'569, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.837+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.837+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.837+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f13c731e:::obj449:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) pg 0x4ef0000 2026-02-11T23:31:27.837+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:f13c731e:::obj449:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 1.0 e 117/117 lb 1:f13c731e:::obj449:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:f11aef48:::obj58:head local-lis/les=113/114 n=568 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.837+0000 7f414f993640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.837+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f11aef48:::obj58:head local-lis/les=113/114 n=568 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 117/117 lb 1:f13c731e:::obj449:head) v3 2026-02-11T23:31:27.837+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f11aef48:::obj58:head local-lis/les=113/114 n=568 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 117/117 lb 1:f13c731e:::obj449:head) 2026-02-11T23:31:27.837+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.837+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:f13c731e:::obj449:head) v3 finish latency 0.000009 2026-02-11T23:31:27.837+0000 7f414f993640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f13c731e:::obj449:head local-lis/les=113/114 n=569 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.837+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.838+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.838+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f13c731e:::obj449:head local-lis/les=113/114 n=569 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.838+0000 7f414f993640 20 osd.0 117 get_con_osd_cluster to osd.1 from_epoch 117 2026-02-11T23:31:27.838+0000 7f414f993640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:27.838+0000 7f414f993640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:27.838+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e117/117) -- 0x6171800 con 0x4acb000 2026-02-11T23:31:27.838+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 117/113 [PushReplyOp(1:f22dcb20:::obj569:head)]) -- 0x5d1ca80 con 0x4acb000 2026-02-11T23:31:27.838+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.838+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1176 ==== MOSDPGPush(1.0 117/113 [PushOp(1:f2a1ba99:::obj364:head, version: 111'364, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2a1ba99:::obj364:head@111'364, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a3ba0 con 0x4acb000 2026-02-11T23:31:27.838+0000 7f416abfa640 15 osd.0 117 enqueue_op MOSDPGPush(1.0 117/113 [PushOp(1:f2a1ba99:::obj364:head, version: 111'364, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2a1ba99:::obj364:head@111'364, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 117 2026-02-11T23:31:27.838+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f2a1ba99:::obj364:head, version: 111'364, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2a1ba99:::obj364:head@111'364, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e117) 2026-02-11T23:31:27.838+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.838+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1177 ==== pg_backfill(progress 1.0 e 117/117 lb 1:f22dcb20:::obj569:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f13700 con 0x4acb000 2026-02-11T23:31:27.838+0000 7f416abfa640 15 osd.0 117 enqueue_op pg_backfill(progress 1.0 e 117/117 lb 1:f22dcb20:::obj569:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 117 2026-02-11T23:31:27.838+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f2a1ba99:::obj364:head, version: 111'364, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2a1ba99:::obj364:head@111'364, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) queued 2026-02-11T23:31:27.838+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f22dcb20:::obj569:head) v3) class_id 0 prio 15 cost 0 e117) 2026-02-11T23:31:27.838+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.838+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f2a1ba99:::obj364:head, version: 111'364, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2a1ba99:::obj364:head@111'364, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) pg 0x4ef0000 2026-02-11T23:31:27.838+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:f2a1ba99:::obj364:head, version: 111'364, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2a1ba99:::obj364:head@111'364, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 117/113 [PushOp(1:f2a1ba99:::obj364:head, version: 111'364, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2a1ba99:::obj364:head@111'364, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:f13c731e:::obj449:head local-lis/les=113/114 n=569 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.838+0000 7f414b98b640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.838+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.838+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f13c731e:::obj449:head local-lis/les=113/114 n=569 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 117/113 [PushOp(1:f2a1ba99:::obj364:head, version: 111'364, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2a1ba99:::obj364:head@111'364, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.838+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f22dcb20:::obj569:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) queued 2026-02-11T23:31:27.838+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f13c731e:::obj449:head local-lis/les=113/114 n=569 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:f2a1ba99:::obj364:head@111'364, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.838+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f13c731e:::obj449:head local-lis/les=113/114 n=569 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:f2a1ba99:::obj364:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.838+0000 7f414b98b640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f13c731e:::obj449:head local-lis/les=113/114 n=569 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:f2a1ba99:::obj364:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.838+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f13c731e:::obj449:head local-lis/les=113/114 n=569 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:f2a1ba99:::obj364:head 2026-02-11T23:31:27.838+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f13c731e:::obj449:head local-lis/les=113/114 n=569 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:f2a1ba99:::obj364:head v 111'364 2026-02-11T23:31:27.838+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f13c731e:::obj449:head local-lis/les=113/114 n=569 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.838+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.838+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:f2a1ba99:::obj364:head, version: 111'364, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2a1ba99:::obj364:head@111'364, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.838+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.838+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.838+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f22dcb20:::obj569:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) pg 0x4ef0000 2026-02-11T23:31:27.838+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:f22dcb20:::obj569:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 117/117 lb 1:f22dcb20:::obj569:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:f13c731e:::obj449:head local-lis/les=113/114 n=569 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.838+0000 7f414f993640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.838+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f13c731e:::obj449:head local-lis/les=113/114 n=569 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 117/117 lb 1:f22dcb20:::obj569:head) v3 2026-02-11T23:31:27.838+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f13c731e:::obj449:head local-lis/les=113/114 n=569 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 117/117 lb 1:f22dcb20:::obj569:head) 2026-02-11T23:31:27.838+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.838+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:f22dcb20:::obj569:head) v3 finish latency 0.000010 2026-02-11T23:31:27.838+0000 7f414f993640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f22dcb20:::obj569:head local-lis/les=113/114 n=570 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.838+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.838+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.838+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f22dcb20:::obj569:head local-lis/les=113/114 n=570 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.838+0000 7f414f993640 20 osd.0 117 get_con_osd_cluster to osd.1 from_epoch 117 2026-02-11T23:31:27.838+0000 7f414f993640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:27.838+0000 7f414f993640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:27.838+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e117/117) -- 0x600b800 con 0x4acb000 2026-02-11T23:31:27.838+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 117/113 [PushReplyOp(1:f2a1ba99:::obj364:head)]) -- 0x5ca1680 con 0x4acb000 2026-02-11T23:31:27.838+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.838+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1178 ==== MOSDPGPush(1.0 117/113 [PushOp(1:f2bc7f0e:::obj94:head, version: 111'94, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2bc7f0e:::obj94:head@111'94, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x56a2b60 con 0x4acb000 2026-02-11T23:31:27.838+0000 7f416abfa640 15 osd.0 117 enqueue_op MOSDPGPush(1.0 117/113 [PushOp(1:f2bc7f0e:::obj94:head, version: 111'94, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2bc7f0e:::obj94:head@111'94, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 117 2026-02-11T23:31:27.838+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f2bc7f0e:::obj94:head, version: 111'94, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2bc7f0e:::obj94:head@111'94, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e117) 2026-02-11T23:31:27.839+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.839+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f2bc7f0e:::obj94:head, version: 111'94, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2bc7f0e:::obj94:head@111'94, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) queued 2026-02-11T23:31:27.839+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.839+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1179 ==== pg_backfill(progress 1.0 e 117/117 lb 1:f2a1ba99:::obj364:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f13180 con 0x4acb000 2026-02-11T23:31:27.839+0000 7f416abfa640 15 osd.0 117 enqueue_op pg_backfill(progress 1.0 e 117/117 lb 1:f2a1ba99:::obj364:head) v3 prio 15 type 95 cost 0 latency 0.000015 epoch 117 2026-02-11T23:31:27.839+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f2bc7f0e:::obj94:head, version: 111'94, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2bc7f0e:::obj94:head@111'94, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) pg 0x4ef0000 2026-02-11T23:31:27.839+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f2a1ba99:::obj364:head) v3) class_id 0 prio 15 cost 0 e117) 2026-02-11T23:31:27.839+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:f2bc7f0e:::obj94:head, version: 111'94, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2bc7f0e:::obj94:head@111'94, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000040 MOSDPGPush(1.0 117/113 [PushOp(1:f2bc7f0e:::obj94:head, version: 111'94, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2bc7f0e:::obj94:head@111'94, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:f22dcb20:::obj569:head local-lis/les=113/114 n=570 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.839+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.839+0000 7f414b98b640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.839+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f2a1ba99:::obj364:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) queued 2026-02-11T23:31:27.839+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f22dcb20:::obj569:head local-lis/les=113/114 n=570 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 117/113 [PushOp(1:f2bc7f0e:::obj94:head, version: 111'94, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2bc7f0e:::obj94:head@111'94, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.839+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f22dcb20:::obj569:head local-lis/les=113/114 n=570 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:f2bc7f0e:::obj94:head@111'94, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.839+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f22dcb20:::obj569:head local-lis/les=113/114 n=570 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:f2bc7f0e:::obj94:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.839+0000 7f414b98b640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f22dcb20:::obj569:head local-lis/les=113/114 n=570 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:f2bc7f0e:::obj94:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.839+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f22dcb20:::obj569:head local-lis/les=113/114 n=570 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:f2bc7f0e:::obj94:head 2026-02-11T23:31:27.839+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f22dcb20:::obj569:head local-lis/les=113/114 n=570 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:f2bc7f0e:::obj94:head v 111'94 2026-02-11T23:31:27.839+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f22dcb20:::obj569:head local-lis/les=113/114 n=570 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.839+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.839+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:f2bc7f0e:::obj94:head, version: 111'94, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2bc7f0e:::obj94:head@111'94, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:27.839+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.839+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.839+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f2a1ba99:::obj364:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) pg 0x4ef0000 2026-02-11T23:31:27.839+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:f2a1ba99:::obj364:head) v3 prio 15 cost 0 latency 0.000091 pg_backfill(progress 1.0 e 117/117 lb 1:f2a1ba99:::obj364:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:f22dcb20:::obj569:head local-lis/les=113/114 n=570 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.839+0000 7f414f993640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.839+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f22dcb20:::obj569:head local-lis/les=113/114 n=570 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 117/117 lb 1:f2a1ba99:::obj364:head) v3 2026-02-11T23:31:27.839+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f22dcb20:::obj569:head local-lis/les=113/114 n=570 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 117/117 lb 1:f2a1ba99:::obj364:head) 2026-02-11T23:31:27.839+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.839+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:f2a1ba99:::obj364:head) v3 finish latency 0.000009 2026-02-11T23:31:27.839+0000 7f414f993640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f2a1ba99:::obj364:head local-lis/les=113/114 n=571 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.839+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.839+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.839+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f2a1ba99:::obj364:head local-lis/les=113/114 n=571 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.839+0000 7f414f993640 20 osd.0 117 get_con_osd_cluster to osd.1 from_epoch 117 2026-02-11T23:31:27.839+0000 7f414f993640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:27.839+0000 7f414f993640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:27.839+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e117/117) -- 0x5f2d980 con 0x4acb000 2026-02-11T23:31:27.839+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 117/113 [PushReplyOp(1:f2bc7f0e:::obj94:head)]) -- 0x5ca1680 con 0x4acb000 2026-02-11T23:31:27.839+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.839+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1180 ==== MOSDPGPush(1.0 117/113 [PushOp(1:f2f9dd5a:::obj121:head, version: 111'121, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2f9dd5a:::obj121:head@111'121, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56ca340 con 0x4acb000 2026-02-11T23:31:27.839+0000 7f416abfa640 15 osd.0 117 enqueue_op MOSDPGPush(1.0 117/113 [PushOp(1:f2f9dd5a:::obj121:head, version: 111'121, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2f9dd5a:::obj121:head@111'121, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 117 2026-02-11T23:31:27.839+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f2f9dd5a:::obj121:head, version: 111'121, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2f9dd5a:::obj121:head@111'121, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e117) 2026-02-11T23:31:27.839+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.839+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1181 ==== pg_backfill(progress 1.0 e 117/117 lb 1:f2bc7f0e:::obj94:head) ==== 1061+0+0 (unknown 0 0 0) 0x5f12c00 con 0x4acb000 2026-02-11T23:31:27.839+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f2f9dd5a:::obj121:head, version: 111'121, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2f9dd5a:::obj121:head@111'121, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) queued 2026-02-11T23:31:27.839+0000 7f416abfa640 15 osd.0 117 enqueue_op pg_backfill(progress 1.0 e 117/117 lb 1:f2bc7f0e:::obj94:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 117 2026-02-11T23:31:27.839+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.839+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f2bc7f0e:::obj94:head) v3) class_id 0 prio 15 cost 0 e117) 2026-02-11T23:31:27.839+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f2f9dd5a:::obj121:head, version: 111'121, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2f9dd5a:::obj121:head@111'121, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) pg 0x4ef0000 2026-02-11T23:31:27.839+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.839+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:f2f9dd5a:::obj121:head, version: 111'121, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2f9dd5a:::obj121:head@111'121, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 117/113 [PushOp(1:f2f9dd5a:::obj121:head, version: 111'121, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2f9dd5a:::obj121:head@111'121, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:f2a1ba99:::obj364:head local-lis/les=113/114 n=571 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.839+0000 7f414b98b640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.839+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f2a1ba99:::obj364:head local-lis/les=113/114 n=571 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 117/113 [PushOp(1:f2f9dd5a:::obj121:head, version: 111'121, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2f9dd5a:::obj121:head@111'121, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.839+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f2a1ba99:::obj364:head local-lis/les=113/114 n=571 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:f2f9dd5a:::obj121:head@111'121, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.839+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.839+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f2bc7f0e:::obj94:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) queued 2026-02-11T23:31:27.839+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f2a1ba99:::obj364:head local-lis/les=113/114 n=571 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:f2f9dd5a:::obj121:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.839+0000 7f414b98b640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f2a1ba99:::obj364:head local-lis/les=113/114 n=571 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:f2f9dd5a:::obj121:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.839+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f2a1ba99:::obj364:head local-lis/les=113/114 n=571 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:f2f9dd5a:::obj121:head 2026-02-11T23:31:27.839+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f2a1ba99:::obj364:head local-lis/les=113/114 n=571 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:f2f9dd5a:::obj121:head v 111'121 2026-02-11T23:31:27.839+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f2a1ba99:::obj364:head local-lis/les=113/114 n=571 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.839+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.839+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:f2f9dd5a:::obj121:head, version: 111'121, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2f9dd5a:::obj121:head@111'121, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:27.839+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.839+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.839+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f2bc7f0e:::obj94:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) pg 0x4ef0000 2026-02-11T23:31:27.839+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:f2bc7f0e:::obj94:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 1.0 e 117/117 lb 1:f2bc7f0e:::obj94:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:f2a1ba99:::obj364:head local-lis/les=113/114 n=571 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.839+0000 7f414f993640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.839+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f2a1ba99:::obj364:head local-lis/les=113/114 n=571 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 117/117 lb 1:f2bc7f0e:::obj94:head) v3 2026-02-11T23:31:27.839+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f2a1ba99:::obj364:head local-lis/les=113/114 n=571 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 117/117 lb 1:f2bc7f0e:::obj94:head) 2026-02-11T23:31:27.839+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.839+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:f2bc7f0e:::obj94:head) v3 finish latency 0.000013 2026-02-11T23:31:27.839+0000 7f414f993640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f2bc7f0e:::obj94:head local-lis/les=113/114 n=572 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.839+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.839+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.839+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f2bc7f0e:::obj94:head local-lis/les=113/114 n=572 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.839+0000 7f414f993640 20 osd.0 117 get_con_osd_cluster to osd.1 from_epoch 117 2026-02-11T23:31:27.839+0000 7f414f993640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:27.839+0000 7f414f993640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:27.839+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e117/117) -- 0x5f1c000 con 0x4acb000 2026-02-11T23:31:27.839+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 117/113 [PushReplyOp(1:f2f9dd5a:::obj121:head)]) -- 0x600b800 con 0x4acb000 2026-02-11T23:31:27.839+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.840+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1182 ==== MOSDPGPush(1.0 117/113 [PushOp(1:f2fc2104:::obj429:head, version: 111'429, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2fc2104:::obj429:head@111'429, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2000 con 0x4acb000 2026-02-11T23:31:27.840+0000 7f416abfa640 15 osd.0 117 enqueue_op MOSDPGPush(1.0 117/113 [PushOp(1:f2fc2104:::obj429:head, version: 111'429, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2fc2104:::obj429:head@111'429, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 117 2026-02-11T23:31:27.840+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f2fc2104:::obj429:head, version: 111'429, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2fc2104:::obj429:head@111'429, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e117) 2026-02-11T23:31:27.840+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.840+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1183 ==== pg_backfill(progress 1.0 e 117/117 lb 1:f2f9dd5a:::obj121:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f12680 con 0x4acb000 2026-02-11T23:31:27.840+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f2fc2104:::obj429:head, version: 111'429, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2fc2104:::obj429:head@111'429, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) queued 2026-02-11T23:31:27.840+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.840+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f2fc2104:::obj429:head, version: 111'429, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2fc2104:::obj429:head@111'429, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) pg 0x4ef0000 2026-02-11T23:31:27.840+0000 7f416abfa640 15 osd.0 117 enqueue_op pg_backfill(progress 1.0 e 117/117 lb 1:f2f9dd5a:::obj121:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 117 2026-02-11T23:31:27.840+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f2f9dd5a:::obj121:head) v3) class_id 0 prio 15 cost 0 e117) 2026-02-11T23:31:27.840+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:f2fc2104:::obj429:head, version: 111'429, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2fc2104:::obj429:head@111'429, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 117/113 [PushOp(1:f2fc2104:::obj429:head, version: 111'429, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2fc2104:::obj429:head@111'429, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:f2bc7f0e:::obj94:head local-lis/les=113/114 n=572 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.840+0000 7f414b98b640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.840+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.840+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f2bc7f0e:::obj94:head local-lis/les=113/114 n=572 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 117/113 [PushOp(1:f2fc2104:::obj429:head, version: 111'429, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2fc2104:::obj429:head@111'429, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.840+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f2bc7f0e:::obj94:head local-lis/les=113/114 n=572 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:f2fc2104:::obj429:head@111'429, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.840+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f2bc7f0e:::obj94:head local-lis/les=113/114 n=572 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:f2fc2104:::obj429:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.840+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.840+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f2f9dd5a:::obj121:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) queued 2026-02-11T23:31:27.840+0000 7f414b98b640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f2bc7f0e:::obj94:head local-lis/les=113/114 n=572 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:f2fc2104:::obj429:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.840+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f2bc7f0e:::obj94:head local-lis/les=113/114 n=572 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:f2fc2104:::obj429:head 2026-02-11T23:31:27.840+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f2bc7f0e:::obj94:head local-lis/les=113/114 n=572 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:f2fc2104:::obj429:head v 111'429 2026-02-11T23:31:27.840+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f2bc7f0e:::obj94:head local-lis/les=113/114 n=572 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.840+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.840+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:f2fc2104:::obj429:head, version: 111'429, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2fc2104:::obj429:head@111'429, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:27.840+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.840+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.840+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f2f9dd5a:::obj121:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) pg 0x4ef0000 2026-02-11T23:31:27.840+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:f2f9dd5a:::obj121:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 1.0 e 117/117 lb 1:f2f9dd5a:::obj121:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:f2bc7f0e:::obj94:head local-lis/les=113/114 n=572 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.840+0000 7f414f993640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.840+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f2bc7f0e:::obj94:head local-lis/les=113/114 n=572 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 117/117 lb 1:f2f9dd5a:::obj121:head) v3 2026-02-11T23:31:27.840+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f2bc7f0e:::obj94:head local-lis/les=113/114 n=572 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 117/117 lb 1:f2f9dd5a:::obj121:head) 2026-02-11T23:31:27.840+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.840+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:f2f9dd5a:::obj121:head) v3 finish latency 0.000009 2026-02-11T23:31:27.840+0000 7f414f993640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f2f9dd5a:::obj121:head local-lis/les=113/114 n=573 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.840+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.840+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.840+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f2f9dd5a:::obj121:head local-lis/les=113/114 n=573 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.840+0000 7f414f993640 20 osd.0 117 get_con_osd_cluster to osd.1 from_epoch 117 2026-02-11T23:31:27.840+0000 7f414f993640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:27.840+0000 7f414f993640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:27.840+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e117/117) -- 0x5f2d800 con 0x4acb000 2026-02-11T23:31:27.840+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 117/113 [PushReplyOp(1:f2fc2104:::obj429:head)]) -- 0x6171800 con 0x4acb000 2026-02-11T23:31:27.840+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.840+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1184 ==== MOSDPGPush(1.0 117/113 [PushOp(1:f31cde5e:::obj446:head, version: 111'446, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f31cde5e:::obj446:head@111'446, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a24e0 con 0x4acb000 2026-02-11T23:31:27.840+0000 7f416abfa640 15 osd.0 117 enqueue_op MOSDPGPush(1.0 117/113 [PushOp(1:f31cde5e:::obj446:head, version: 111'446, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f31cde5e:::obj446:head@111'446, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 117 2026-02-11T23:31:27.840+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f31cde5e:::obj446:head, version: 111'446, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f31cde5e:::obj446:head@111'446, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e117) 2026-02-11T23:31:27.840+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.840+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1185 ==== pg_backfill(progress 1.0 e 117/117 lb 1:f2fc2104:::obj429:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f12100 con 0x4acb000 2026-02-11T23:31:27.840+0000 7f416abfa640 15 osd.0 117 enqueue_op pg_backfill(progress 1.0 e 117/117 lb 1:f2fc2104:::obj429:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 117 2026-02-11T23:31:27.840+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f31cde5e:::obj446:head, version: 111'446, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f31cde5e:::obj446:head@111'446, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) queued 2026-02-11T23:31:27.840+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f2fc2104:::obj429:head) v3) class_id 0 prio 15 cost 0 e117) 2026-02-11T23:31:27.840+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.840+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f31cde5e:::obj446:head, version: 111'446, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f31cde5e:::obj446:head@111'446, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) pg 0x4ef0000 2026-02-11T23:31:27.840+0000 7f414f993640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:f31cde5e:::obj446:head, version: 111'446, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f31cde5e:::obj446:head@111'446, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 117/113 [PushOp(1:f31cde5e:::obj446:head, version: 111'446, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f31cde5e:::obj446:head@111'446, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:f2f9dd5a:::obj121:head local-lis/les=113/114 n=573 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.840+0000 7f414f993640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.840+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f2f9dd5a:::obj121:head local-lis/les=113/114 n=573 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 117/113 [PushOp(1:f31cde5e:::obj446:head, version: 111'446, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f31cde5e:::obj446:head@111'446, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.840+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.840+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f2fc2104:::obj429:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) queued 2026-02-11T23:31:27.840+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f2f9dd5a:::obj121:head local-lis/les=113/114 n=573 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:f31cde5e:::obj446:head@111'446, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.840+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f2f9dd5a:::obj121:head local-lis/les=113/114 n=573 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:f31cde5e:::obj446:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.840+0000 7f414f993640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f2f9dd5a:::obj121:head local-lis/les=113/114 n=573 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:f31cde5e:::obj446:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.840+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f2f9dd5a:::obj121:head local-lis/les=113/114 n=573 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:f31cde5e:::obj446:head 2026-02-11T23:31:27.840+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f2f9dd5a:::obj121:head local-lis/les=113/114 n=573 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:f31cde5e:::obj446:head v 111'446 2026-02-11T23:31:27.840+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f2f9dd5a:::obj121:head local-lis/les=113/114 n=573 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.840+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.840+0000 7f414f993640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:f31cde5e:::obj446:head, version: 111'446, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f31cde5e:::obj446:head@111'446, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:27.840+0000 7f414f993640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f2f9dd5a:::obj121:head local-lis/les=113/114 n=573 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.840+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.840+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.840+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f2fc2104:::obj429:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) pg 0x4ef0000 2026-02-11T23:31:27.840+0000 7f414b98b640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:f2fc2104:::obj429:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 1.0 e 117/117 lb 1:f2fc2104:::obj429:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:f2f9dd5a:::obj121:head local-lis/les=113/114 n=573 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.840+0000 7f414b98b640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.840+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f2f9dd5a:::obj121:head local-lis/les=113/114 n=573 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 117/117 lb 1:f2fc2104:::obj429:head) v3 2026-02-11T23:31:27.840+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f2f9dd5a:::obj121:head local-lis/les=113/114 n=573 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 117/117 lb 1:f2fc2104:::obj429:head) 2026-02-11T23:31:27.840+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.840+0000 7f414b98b640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:f2fc2104:::obj429:head) v3 finish latency 0.000012 2026-02-11T23:31:27.840+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.840+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.840+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f2fc2104:::obj429:head local-lis/les=113/114 n=574 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.840+0000 7f414f993640 20 osd.0 117 get_con_osd_cluster to osd.1 from_epoch 117 2026-02-11T23:31:27.840+0000 7f414f993640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:27.840+0000 7f414f993640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:27.840+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e117/117) -- 0x5f1c180 con 0x4acb000 2026-02-11T23:31:27.840+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 117/113 [PushReplyOp(1:f31cde5e:::obj446:head)]) -- 0x612ed80 con 0x4acb000 2026-02-11T23:31:27.840+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.841+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1186 ==== MOSDPGPush(1.0 117/113 [PushOp(1:f33597d0:::obj489:head, version: 111'489, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f33597d0:::obj489:head@111'489, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x554e680 con 0x4acb000 2026-02-11T23:31:27.841+0000 7f416abfa640 15 osd.0 117 enqueue_op MOSDPGPush(1.0 117/113 [PushOp(1:f33597d0:::obj489:head, version: 111'489, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f33597d0:::obj489:head@111'489, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 117 2026-02-11T23:31:27.841+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f33597d0:::obj489:head, version: 111'489, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f33597d0:::obj489:head@111'489, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e117) 2026-02-11T23:31:27.841+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.841+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1187 ==== pg_backfill(progress 1.0 e 117/117 lb 1:f31cde5e:::obj446:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f11b80 con 0x4acb000 2026-02-11T23:31:27.841+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f33597d0:::obj489:head, version: 111'489, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f33597d0:::obj489:head@111'489, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) queued 2026-02-11T23:31:27.841+0000 7f416abfa640 15 osd.0 117 enqueue_op pg_backfill(progress 1.0 e 117/117 lb 1:f31cde5e:::obj446:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 117 2026-02-11T23:31:27.841+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f31cde5e:::obj446:head) v3) class_id 0 prio 15 cost 0 e117) 2026-02-11T23:31:27.841+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.841+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f33597d0:::obj489:head, version: 111'489, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f33597d0:::obj489:head@111'489, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) pg 0x4ef0000 2026-02-11T23:31:27.841+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:f33597d0:::obj489:head, version: 111'489, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f33597d0:::obj489:head@111'489, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 117/113 [PushOp(1:f33597d0:::obj489:head, version: 111'489, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f33597d0:::obj489:head@111'489, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:f2fc2104:::obj429:head local-lis/les=113/114 n=574 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.841+0000 7f414b98b640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.841+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f2fc2104:::obj429:head local-lis/les=113/114 n=574 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 117/113 [PushOp(1:f33597d0:::obj489:head, version: 111'489, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f33597d0:::obj489:head@111'489, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.841+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.841+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f31cde5e:::obj446:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) queued 2026-02-11T23:31:27.841+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f2fc2104:::obj429:head local-lis/les=113/114 n=574 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:f33597d0:::obj489:head@111'489, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.841+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f2fc2104:::obj429:head local-lis/les=113/114 n=574 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:f33597d0:::obj489:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.841+0000 7f414b98b640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f2fc2104:::obj429:head local-lis/les=113/114 n=574 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:f33597d0:::obj489:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.841+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f2fc2104:::obj429:head local-lis/les=113/114 n=574 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:f33597d0:::obj489:head 2026-02-11T23:31:27.841+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f2fc2104:::obj429:head local-lis/les=113/114 n=574 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:f33597d0:::obj489:head v 111'489 2026-02-11T23:31:27.841+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f2fc2104:::obj429:head local-lis/les=113/114 n=574 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.841+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.841+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:f33597d0:::obj489:head, version: 111'489, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f33597d0:::obj489:head@111'489, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.841+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.841+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.841+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f31cde5e:::obj446:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) pg 0x4ef0000 2026-02-11T23:31:27.841+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:f31cde5e:::obj446:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 1.0 e 117/117 lb 1:f31cde5e:::obj446:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:f2fc2104:::obj429:head local-lis/les=113/114 n=574 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.841+0000 7f414f993640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.841+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f2fc2104:::obj429:head local-lis/les=113/114 n=574 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 117/117 lb 1:f31cde5e:::obj446:head) v3 2026-02-11T23:31:27.841+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f2fc2104:::obj429:head local-lis/les=113/114 n=574 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 117/117 lb 1:f31cde5e:::obj446:head) 2026-02-11T23:31:27.841+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.841+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:f31cde5e:::obj446:head) v3 finish latency 0.000012 2026-02-11T23:31:27.841+0000 7f414f993640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f31cde5e:::obj446:head local-lis/les=113/114 n=575 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.841+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.841+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.841+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f31cde5e:::obj446:head local-lis/les=113/114 n=575 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.841+0000 7f414f993640 20 osd.0 117 get_con_osd_cluster to osd.1 from_epoch 117 2026-02-11T23:31:27.841+0000 7f414f993640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:27.841+0000 7f414f993640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:27.841+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e117/117) -- 0x5f2d200 con 0x4acb000 2026-02-11T23:31:27.841+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 117/113 [PushReplyOp(1:f33597d0:::obj489:head)]) -- 0x5e7ed80 con 0x4acb000 2026-02-11T23:31:27.841+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.841+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1188 ==== MOSDPGPush(1.0 117/113 [PushOp(1:f3768836:::obj189:head, version: 111'189, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f3768836:::obj189:head@111'189, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5617520 con 0x4acb000 2026-02-11T23:31:27.841+0000 7f416abfa640 15 osd.0 117 enqueue_op MOSDPGPush(1.0 117/113 [PushOp(1:f3768836:::obj189:head, version: 111'189, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f3768836:::obj189:head@111'189, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 117 2026-02-11T23:31:27.841+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f3768836:::obj189:head, version: 111'189, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f3768836:::obj189:head@111'189, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e117) 2026-02-11T23:31:27.841+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.841+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1189 ==== pg_backfill(progress 1.0 e 117/117 lb 1:f33597d0:::obj489:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f13180 con 0x4acb000 2026-02-11T23:31:27.841+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f3768836:::obj189:head, version: 111'189, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f3768836:::obj189:head@111'189, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) queued 2026-02-11T23:31:27.841+0000 7f416abfa640 15 osd.0 117 enqueue_op pg_backfill(progress 1.0 e 117/117 lb 1:f33597d0:::obj489:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 117 2026-02-11T23:31:27.841+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f33597d0:::obj489:head) v3) class_id 0 prio 15 cost 0 e117) 2026-02-11T23:31:27.841+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.841+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f3768836:::obj189:head, version: 111'189, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f3768836:::obj189:head@111'189, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) pg 0x4ef0000 2026-02-11T23:31:27.841+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:f3768836:::obj189:head, version: 111'189, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f3768836:::obj189:head@111'189, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 117/113 [PushOp(1:f3768836:::obj189:head, version: 111'189, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f3768836:::obj189:head@111'189, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:f31cde5e:::obj446:head local-lis/les=113/114 n=575 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.841+0000 7f414b98b640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.841+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.841+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f31cde5e:::obj446:head local-lis/les=113/114 n=575 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 117/113 [PushOp(1:f3768836:::obj189:head, version: 111'189, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f3768836:::obj189:head@111'189, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.841+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f33597d0:::obj489:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) queued 2026-02-11T23:31:27.841+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f31cde5e:::obj446:head local-lis/les=113/114 n=575 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:f3768836:::obj189:head@111'189, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.841+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f31cde5e:::obj446:head local-lis/les=113/114 n=575 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:f3768836:::obj189:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.841+0000 7f414b98b640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f31cde5e:::obj446:head local-lis/les=113/114 n=575 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:f3768836:::obj189:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.841+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f31cde5e:::obj446:head local-lis/les=113/114 n=575 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:f3768836:::obj189:head 2026-02-11T23:31:27.841+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f31cde5e:::obj446:head local-lis/les=113/114 n=575 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:f3768836:::obj189:head v 111'189 2026-02-11T23:31:27.841+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f31cde5e:::obj446:head local-lis/les=113/114 n=575 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.841+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.841+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:f3768836:::obj189:head, version: 111'189, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f3768836:::obj189:head@111'189, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:27.841+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.841+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.841+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f33597d0:::obj489:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) pg 0x4ef0000 2026-02-11T23:31:27.841+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:f33597d0:::obj489:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 117/117 lb 1:f33597d0:::obj489:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:f31cde5e:::obj446:head local-lis/les=113/114 n=575 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.841+0000 7f414f993640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.841+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f31cde5e:::obj446:head local-lis/les=113/114 n=575 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 117/117 lb 1:f33597d0:::obj489:head) v3 2026-02-11T23:31:27.841+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f31cde5e:::obj446:head local-lis/les=113/114 n=575 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 117/117 lb 1:f33597d0:::obj489:head) 2026-02-11T23:31:27.841+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.841+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:f33597d0:::obj489:head) v3 finish latency 0.000009 2026-02-11T23:31:27.841+0000 7f414f993640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f33597d0:::obj489:head local-lis/les=113/114 n=576 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.841+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.841+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.841+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f33597d0:::obj489:head local-lis/les=113/114 n=576 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.841+0000 7f414f993640 20 osd.0 117 get_con_osd_cluster to osd.1 from_epoch 117 2026-02-11T23:31:27.841+0000 7f414f993640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:27.841+0000 7f414f993640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:27.841+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e117/117) -- 0x6170300 con 0x4acb000 2026-02-11T23:31:27.841+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 117/113 [PushReplyOp(1:f3768836:::obj189:head)]) -- 0x5f2dc80 con 0x4acb000 2026-02-11T23:31:27.841+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.842+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1190 ==== MOSDPGPush(1.0 117/113 [PushOp(1:f3c2593a:::obj26:head, version: 111'26, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f3c2593a:::obj26:head@111'26, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x56171e0 con 0x4acb000 2026-02-11T23:31:27.842+0000 7f416abfa640 15 osd.0 117 enqueue_op MOSDPGPush(1.0 117/113 [PushOp(1:f3c2593a:::obj26:head, version: 111'26, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f3c2593a:::obj26:head@111'26, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 117 2026-02-11T23:31:27.842+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f3c2593a:::obj26:head, version: 111'26, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f3c2593a:::obj26:head@111'26, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e117) 2026-02-11T23:31:27.842+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.842+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1191 ==== pg_backfill(progress 1.0 e 117/117 lb 1:f3768836:::obj189:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f13700 con 0x4acb000 2026-02-11T23:31:27.842+0000 7f416abfa640 15 osd.0 117 enqueue_op pg_backfill(progress 1.0 e 117/117 lb 1:f3768836:::obj189:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 117 2026-02-11T23:31:27.842+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f3c2593a:::obj26:head, version: 111'26, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f3c2593a:::obj26:head@111'26, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) queued 2026-02-11T23:31:27.842+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f3768836:::obj189:head) v3) class_id 0 prio 15 cost 0 e117) 2026-02-11T23:31:27.842+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.842+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f3c2593a:::obj26:head, version: 111'26, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f3c2593a:::obj26:head@111'26, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) pg 0x4ef0000 2026-02-11T23:31:27.842+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:f3c2593a:::obj26:head, version: 111'26, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f3c2593a:::obj26:head@111'26, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 117/113 [PushOp(1:f3c2593a:::obj26:head, version: 111'26, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f3c2593a:::obj26:head@111'26, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:f33597d0:::obj489:head local-lis/les=113/114 n=576 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.842+0000 7f414b98b640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.842+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.842+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f33597d0:::obj489:head local-lis/les=113/114 n=576 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 117/113 [PushOp(1:f3c2593a:::obj26:head, version: 111'26, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f3c2593a:::obj26:head@111'26, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.842+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f3768836:::obj189:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) queued 2026-02-11T23:31:27.842+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f33597d0:::obj489:head local-lis/les=113/114 n=576 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:f3c2593a:::obj26:head@111'26, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.842+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f33597d0:::obj489:head local-lis/les=113/114 n=576 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:f3c2593a:::obj26:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.842+0000 7f414b98b640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f33597d0:::obj489:head local-lis/les=113/114 n=576 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:f3c2593a:::obj26:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.842+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f33597d0:::obj489:head local-lis/les=113/114 n=576 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:f3c2593a:::obj26:head 2026-02-11T23:31:27.842+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f33597d0:::obj489:head local-lis/les=113/114 n=576 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:f3c2593a:::obj26:head v 111'26 2026-02-11T23:31:27.842+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f33597d0:::obj489:head local-lis/les=113/114 n=576 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.842+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.842+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:f3c2593a:::obj26:head, version: 111'26, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f3c2593a:::obj26:head@111'26, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.842+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.842+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.842+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f3768836:::obj189:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) pg 0x4ef0000 2026-02-11T23:31:27.842+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:f3768836:::obj189:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 117/117 lb 1:f3768836:::obj189:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:f33597d0:::obj489:head local-lis/les=113/114 n=576 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.842+0000 7f414f993640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.842+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f33597d0:::obj489:head local-lis/les=113/114 n=576 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 117/117 lb 1:f3768836:::obj189:head) v3 2026-02-11T23:31:27.842+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f33597d0:::obj489:head local-lis/les=113/114 n=576 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 117/117 lb 1:f3768836:::obj189:head) 2026-02-11T23:31:27.842+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.842+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:f3768836:::obj189:head) v3 finish latency 0.000009 2026-02-11T23:31:27.842+0000 7f414f993640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f3768836:::obj189:head local-lis/les=113/114 n=577 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.842+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.842+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.842+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f3768836:::obj189:head local-lis/les=113/114 n=577 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.842+0000 7f414f993640 20 osd.0 117 get_con_osd_cluster to osd.1 from_epoch 117 2026-02-11T23:31:27.842+0000 7f414f993640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:27.842+0000 7f414f993640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:27.842+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e117/117) -- 0x5f2cf00 con 0x4acb000 2026-02-11T23:31:27.842+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 117/113 [PushReplyOp(1:f3c2593a:::obj26:head)]) -- 0x5d1cd80 con 0x4acb000 2026-02-11T23:31:27.842+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.842+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1192 ==== MOSDPGPush(1.0 117/113 [PushOp(1:f42cbfe2:::obj212:head, version: 111'212, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f42cbfe2:::obj212:head@111'212, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56cad00 con 0x4acb000 2026-02-11T23:31:27.842+0000 7f416abfa640 15 osd.0 117 enqueue_op MOSDPGPush(1.0 117/113 [PushOp(1:f42cbfe2:::obj212:head, version: 111'212, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f42cbfe2:::obj212:head@111'212, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 117 2026-02-11T23:31:27.842+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f42cbfe2:::obj212:head, version: 111'212, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f42cbfe2:::obj212:head@111'212, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e117) 2026-02-11T23:31:27.842+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.842+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1193 ==== pg_backfill(progress 1.0 e 117/117 lb 1:f3c2593a:::obj26:head) ==== 1061+0+0 (unknown 0 0 0) 0x5f10000 con 0x4acb000 2026-02-11T23:31:27.842+0000 7f416abfa640 15 osd.0 117 enqueue_op pg_backfill(progress 1.0 e 117/117 lb 1:f3c2593a:::obj26:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 117 2026-02-11T23:31:27.842+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f42cbfe2:::obj212:head, version: 111'212, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f42cbfe2:::obj212:head@111'212, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) queued 2026-02-11T23:31:27.842+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f3c2593a:::obj26:head) v3) class_id 0 prio 15 cost 0 e117) 2026-02-11T23:31:27.842+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.842+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f42cbfe2:::obj212:head, version: 111'212, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f42cbfe2:::obj212:head@111'212, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) pg 0x4ef0000 2026-02-11T23:31:27.842+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:f42cbfe2:::obj212:head, version: 111'212, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f42cbfe2:::obj212:head@111'212, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 117/113 [PushOp(1:f42cbfe2:::obj212:head, version: 111'212, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f42cbfe2:::obj212:head@111'212, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:f3768836:::obj189:head local-lis/les=113/114 n=577 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.842+0000 7f414b98b640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.842+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.842+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f3768836:::obj189:head local-lis/les=113/114 n=577 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 117/113 [PushOp(1:f42cbfe2:::obj212:head, version: 111'212, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f42cbfe2:::obj212:head@111'212, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.842+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f3c2593a:::obj26:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) queued 2026-02-11T23:31:27.842+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f3768836:::obj189:head local-lis/les=113/114 n=577 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:f42cbfe2:::obj212:head@111'212, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.842+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f3768836:::obj189:head local-lis/les=113/114 n=577 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:f42cbfe2:::obj212:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.842+0000 7f414b98b640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f3768836:::obj189:head local-lis/les=113/114 n=577 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:f42cbfe2:::obj212:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.842+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f3768836:::obj189:head local-lis/les=113/114 n=577 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:f42cbfe2:::obj212:head 2026-02-11T23:31:27.842+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f3768836:::obj189:head local-lis/les=113/114 n=577 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:f42cbfe2:::obj212:head v 111'212 2026-02-11T23:31:27.842+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f3768836:::obj189:head local-lis/les=113/114 n=577 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.842+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.842+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:f42cbfe2:::obj212:head, version: 111'212, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f42cbfe2:::obj212:head@111'212, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.842+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.842+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.842+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f3c2593a:::obj26:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) pg 0x4ef0000 2026-02-11T23:31:27.842+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:f3c2593a:::obj26:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 1.0 e 117/117 lb 1:f3c2593a:::obj26:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:f3768836:::obj189:head local-lis/les=113/114 n=577 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.842+0000 7f414f993640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.842+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f3768836:::obj189:head local-lis/les=113/114 n=577 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 117/117 lb 1:f3c2593a:::obj26:head) v3 2026-02-11T23:31:27.842+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f3768836:::obj189:head local-lis/les=113/114 n=577 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 117/117 lb 1:f3c2593a:::obj26:head) 2026-02-11T23:31:27.842+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.842+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:f3c2593a:::obj26:head) v3 finish latency 0.000011 2026-02-11T23:31:27.842+0000 7f414f993640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f3c2593a:::obj26:head local-lis/les=113/114 n=578 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.842+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.843+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.843+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f3c2593a:::obj26:head local-lis/les=113/114 n=578 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.843+0000 7f414f993640 20 osd.0 117 get_con_osd_cluster to osd.1 from_epoch 117 2026-02-11T23:31:27.843+0000 7f414f993640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:27.843+0000 7f414f993640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:27.843+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e117/117) -- 0x5f1c300 con 0x4acb000 2026-02-11T23:31:27.843+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 117/113 [PushReplyOp(1:f42cbfe2:::obj212:head)]) -- 0x5f1d500 con 0x4acb000 2026-02-11T23:31:27.843+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.843+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1194 ==== MOSDPGPush(1.0 117/113 [PushOp(1:f4af07f1:::obj515:head, version: 111'515, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f4af07f1:::obj515:head@111'515, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56ca820 con 0x4acb000 2026-02-11T23:31:27.843+0000 7f416abfa640 15 osd.0 117 enqueue_op MOSDPGPush(1.0 117/113 [PushOp(1:f4af07f1:::obj515:head, version: 111'515, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f4af07f1:::obj515:head@111'515, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 117 2026-02-11T23:31:27.843+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f4af07f1:::obj515:head, version: 111'515, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f4af07f1:::obj515:head@111'515, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e117) 2026-02-11T23:31:27.843+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.843+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1195 ==== pg_backfill(progress 1.0 e 117/117 lb 1:f42cbfe2:::obj212:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f10580 con 0x4acb000 2026-02-11T23:31:27.843+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f4af07f1:::obj515:head, version: 111'515, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f4af07f1:::obj515:head@111'515, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) queued 2026-02-11T23:31:27.843+0000 7f416abfa640 15 osd.0 117 enqueue_op pg_backfill(progress 1.0 e 117/117 lb 1:f42cbfe2:::obj212:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 117 2026-02-11T23:31:27.843+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f42cbfe2:::obj212:head) v3) class_id 0 prio 15 cost 0 e117) 2026-02-11T23:31:27.843+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.843+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f42cbfe2:::obj212:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) queued 2026-02-11T23:31:27.843+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.843+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f4af07f1:::obj515:head, version: 111'515, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f4af07f1:::obj515:head@111'515, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) pg 0x4ef0000 2026-02-11T23:31:27.843+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:f4af07f1:::obj515:head, version: 111'515, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f4af07f1:::obj515:head@111'515, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000042 MOSDPGPush(1.0 117/113 [PushOp(1:f4af07f1:::obj515:head, version: 111'515, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f4af07f1:::obj515:head@111'515, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:f3c2593a:::obj26:head local-lis/les=113/114 n=578 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.843+0000 7f414b98b640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.843+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f3c2593a:::obj26:head local-lis/les=113/114 n=578 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 117/113 [PushOp(1:f4af07f1:::obj515:head, version: 111'515, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f4af07f1:::obj515:head@111'515, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.843+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f3c2593a:::obj26:head local-lis/les=113/114 n=578 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:f4af07f1:::obj515:head@111'515, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.843+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f3c2593a:::obj26:head local-lis/les=113/114 n=578 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:f4af07f1:::obj515:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.843+0000 7f414b98b640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f3c2593a:::obj26:head local-lis/les=113/114 n=578 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:f4af07f1:::obj515:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.843+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f3c2593a:::obj26:head local-lis/les=113/114 n=578 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:f4af07f1:::obj515:head 2026-02-11T23:31:27.843+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f3c2593a:::obj26:head local-lis/les=113/114 n=578 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:f4af07f1:::obj515:head v 111'515 2026-02-11T23:31:27.843+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f3c2593a:::obj26:head local-lis/les=113/114 n=578 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.843+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.843+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:f4af07f1:::obj515:head, version: 111'515, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f4af07f1:::obj515:head@111'515, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.843+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.843+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.843+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f42cbfe2:::obj212:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) pg 0x4ef0000 2026-02-11T23:31:27.843+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:f42cbfe2:::obj212:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 1.0 e 117/117 lb 1:f42cbfe2:::obj212:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:f3c2593a:::obj26:head local-lis/les=113/114 n=578 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.843+0000 7f414f993640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.843+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f3c2593a:::obj26:head local-lis/les=113/114 n=578 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 117/117 lb 1:f42cbfe2:::obj212:head) v3 2026-02-11T23:31:27.843+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f3c2593a:::obj26:head local-lis/les=113/114 n=578 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 117/117 lb 1:f42cbfe2:::obj212:head) 2026-02-11T23:31:27.843+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.843+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:f42cbfe2:::obj212:head) v3 finish latency 0.000010 2026-02-11T23:31:27.843+0000 7f414f993640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f42cbfe2:::obj212:head local-lis/les=113/114 n=579 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.843+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.843+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.843+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f42cbfe2:::obj212:head local-lis/les=113/114 n=579 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.843+0000 7f414f993640 20 osd.0 117 get_con_osd_cluster to osd.1 from_epoch 117 2026-02-11T23:31:27.843+0000 7f414f993640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:27.843+0000 7f414f993640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:27.843+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e117/117) -- 0x5f2cc00 con 0x4acb000 2026-02-11T23:31:27.843+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 117/113 [PushReplyOp(1:f4af07f1:::obj515:head)]) -- 0x6000180 con 0x4acb000 2026-02-11T23:31:27.843+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.843+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1196 ==== MOSDPGPush(1.0 117/113 [PushOp(1:f4b32ae1:::obj130:head, version: 111'130, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f4b32ae1:::obj130:head@111'130, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5646b60 con 0x4acb000 2026-02-11T23:31:27.843+0000 7f416abfa640 15 osd.0 117 enqueue_op MOSDPGPush(1.0 117/113 [PushOp(1:f4b32ae1:::obj130:head, version: 111'130, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f4b32ae1:::obj130:head@111'130, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 117 2026-02-11T23:31:27.843+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f4b32ae1:::obj130:head, version: 111'130, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f4b32ae1:::obj130:head@111'130, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e117) 2026-02-11T23:31:27.843+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.843+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1197 ==== pg_backfill(progress 1.0 e 117/117 lb 1:f4af07f1:::obj515:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f10b00 con 0x4acb000 2026-02-11T23:31:27.843+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f4b32ae1:::obj130:head, version: 111'130, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f4b32ae1:::obj130:head@111'130, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) queued 2026-02-11T23:31:27.843+0000 7f416abfa640 15 osd.0 117 enqueue_op pg_backfill(progress 1.0 e 117/117 lb 1:f4af07f1:::obj515:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 117 2026-02-11T23:31:27.843+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f4af07f1:::obj515:head) v3) class_id 0 prio 15 cost 0 e117) 2026-02-11T23:31:27.843+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.843+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f4b32ae1:::obj130:head, version: 111'130, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f4b32ae1:::obj130:head@111'130, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) pg 0x4ef0000 2026-02-11T23:31:27.843+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.843+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:f4b32ae1:::obj130:head, version: 111'130, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f4b32ae1:::obj130:head@111'130, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 117/113 [PushOp(1:f4b32ae1:::obj130:head, version: 111'130, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f4b32ae1:::obj130:head@111'130, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:f42cbfe2:::obj212:head local-lis/les=113/114 n=579 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.843+0000 7f414b98b640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.843+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f42cbfe2:::obj212:head local-lis/les=113/114 n=579 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 117/113 [PushOp(1:f4b32ae1:::obj130:head, version: 111'130, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f4b32ae1:::obj130:head@111'130, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.843+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f42cbfe2:::obj212:head local-lis/les=113/114 n=579 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:f4b32ae1:::obj130:head@111'130, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.843+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.843+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f4af07f1:::obj515:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) queued 2026-02-11T23:31:27.843+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f42cbfe2:::obj212:head local-lis/les=113/114 n=579 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:f4b32ae1:::obj130:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.843+0000 7f414b98b640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f42cbfe2:::obj212:head local-lis/les=113/114 n=579 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:f4b32ae1:::obj130:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.843+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f42cbfe2:::obj212:head local-lis/les=113/114 n=579 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:f4b32ae1:::obj130:head 2026-02-11T23:31:27.843+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f42cbfe2:::obj212:head local-lis/les=113/114 n=579 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:f4b32ae1:::obj130:head v 111'130 2026-02-11T23:31:27.843+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f42cbfe2:::obj212:head local-lis/les=113/114 n=579 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.843+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.843+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:f4b32ae1:::obj130:head, version: 111'130, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f4b32ae1:::obj130:head@111'130, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.843+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.843+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.843+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f4af07f1:::obj515:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) pg 0x4ef0000 2026-02-11T23:31:27.843+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:f4af07f1:::obj515:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 117/117 lb 1:f4af07f1:::obj515:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:f42cbfe2:::obj212:head local-lis/les=113/114 n=579 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.843+0000 7f414f993640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.843+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f42cbfe2:::obj212:head local-lis/les=113/114 n=579 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 117/117 lb 1:f4af07f1:::obj515:head) v3 2026-02-11T23:31:27.843+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f42cbfe2:::obj212:head local-lis/les=113/114 n=579 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 117/117 lb 1:f4af07f1:::obj515:head) 2026-02-11T23:31:27.843+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.843+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:f4af07f1:::obj515:head) v3 finish latency 0.000009 2026-02-11T23:31:27.843+0000 7f414f993640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f4af07f1:::obj515:head local-lis/les=113/114 n=580 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.843+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.844+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.844+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f4af07f1:::obj515:head local-lis/les=113/114 n=580 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.844+0000 7f414f993640 20 osd.0 117 get_con_osd_cluster to osd.1 from_epoch 117 2026-02-11T23:31:27.844+0000 7f414f993640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:27.844+0000 7f414f993640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:27.844+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e117/117) -- 0x600a480 con 0x4acb000 2026-02-11T23:31:27.844+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 117/113 [PushReplyOp(1:f4b32ae1:::obj130:head)]) -- 0x5f1d200 con 0x4acb000 2026-02-11T23:31:27.844+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.844+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1198 ==== MOSDPGPush(1.0 117/113 [PushOp(1:f677cfaa:::obj144:head, version: 111'144, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f677cfaa:::obj144:head@111'144, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3815520 con 0x4acb000 2026-02-11T23:31:27.844+0000 7f416abfa640 15 osd.0 117 enqueue_op MOSDPGPush(1.0 117/113 [PushOp(1:f677cfaa:::obj144:head, version: 111'144, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f677cfaa:::obj144:head@111'144, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 117 2026-02-11T23:31:27.844+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f677cfaa:::obj144:head, version: 111'144, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f677cfaa:::obj144:head@111'144, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e117) 2026-02-11T23:31:27.844+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.844+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1199 ==== pg_backfill(progress 1.0 e 117/117 lb 1:f4b32ae1:::obj130:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f11080 con 0x4acb000 2026-02-11T23:31:27.844+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f677cfaa:::obj144:head, version: 111'144, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f677cfaa:::obj144:head@111'144, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) queued 2026-02-11T23:31:27.844+0000 7f416abfa640 15 osd.0 117 enqueue_op pg_backfill(progress 1.0 e 117/117 lb 1:f4b32ae1:::obj130:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 117 2026-02-11T23:31:27.844+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f4b32ae1:::obj130:head) v3) class_id 0 prio 15 cost 0 e117) 2026-02-11T23:31:27.844+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.844+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f677cfaa:::obj144:head, version: 111'144, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f677cfaa:::obj144:head@111'144, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) pg 0x4ef0000 2026-02-11T23:31:27.844+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.844+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:f677cfaa:::obj144:head, version: 111'144, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f677cfaa:::obj144:head@111'144, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 117/113 [PushOp(1:f677cfaa:::obj144:head, version: 111'144, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f677cfaa:::obj144:head@111'144, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:f4af07f1:::obj515:head local-lis/les=113/114 n=580 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.844+0000 7f414b98b640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.844+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f4af07f1:::obj515:head local-lis/les=113/114 n=580 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 117/113 [PushOp(1:f677cfaa:::obj144:head, version: 111'144, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f677cfaa:::obj144:head@111'144, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.844+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f4af07f1:::obj515:head local-lis/les=113/114 n=580 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:f677cfaa:::obj144:head@111'144, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.844+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.844+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f4b32ae1:::obj130:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) queued 2026-02-11T23:31:27.844+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f4af07f1:::obj515:head local-lis/les=113/114 n=580 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:f677cfaa:::obj144:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.844+0000 7f414b98b640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f4af07f1:::obj515:head local-lis/les=113/114 n=580 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:f677cfaa:::obj144:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.844+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f4af07f1:::obj515:head local-lis/les=113/114 n=580 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:f677cfaa:::obj144:head 2026-02-11T23:31:27.844+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f4af07f1:::obj515:head local-lis/les=113/114 n=580 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:f677cfaa:::obj144:head v 111'144 2026-02-11T23:31:27.844+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f4af07f1:::obj515:head local-lis/les=113/114 n=580 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.844+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.844+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:f677cfaa:::obj144:head, version: 111'144, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f677cfaa:::obj144:head@111'144, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.844+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.844+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.844+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f4b32ae1:::obj130:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) pg 0x4ef0000 2026-02-11T23:31:27.844+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:f4b32ae1:::obj130:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 1.0 e 117/117 lb 1:f4b32ae1:::obj130:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:f4af07f1:::obj515:head local-lis/les=113/114 n=580 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.844+0000 7f414f993640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.844+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f4af07f1:::obj515:head local-lis/les=113/114 n=580 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 117/117 lb 1:f4b32ae1:::obj130:head) v3 2026-02-11T23:31:27.844+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f4af07f1:::obj515:head local-lis/les=113/114 n=580 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 117/117 lb 1:f4b32ae1:::obj130:head) 2026-02-11T23:31:27.844+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.844+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:f4b32ae1:::obj130:head) v3 finish latency 0.000010 2026-02-11T23:31:27.844+0000 7f414f993640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f4b32ae1:::obj130:head local-lis/les=113/114 n=581 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.844+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.844+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.844+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f4b32ae1:::obj130:head local-lis/les=113/114 n=581 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.844+0000 7f414f993640 20 osd.0 117 get_con_osd_cluster to osd.1 from_epoch 117 2026-02-11T23:31:27.844+0000 7f414f993640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:27.844+0000 7f414f993640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:27.844+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e117/117) -- 0x5f2c900 con 0x4acb000 2026-02-11T23:31:27.844+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 117/113 [PushReplyOp(1:f677cfaa:::obj144:head)]) -- 0x5fd2480 con 0x4acb000 2026-02-11T23:31:27.844+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.844+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1200 ==== MOSDPGPush(1.0 117/113 [PushOp(1:f76d37c8:::obj371:head, version: 111'371, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f76d37c8:::obj371:head@111'371, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3814ea0 con 0x4acb000 2026-02-11T23:31:27.844+0000 7f416abfa640 15 osd.0 117 enqueue_op MOSDPGPush(1.0 117/113 [PushOp(1:f76d37c8:::obj371:head, version: 111'371, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f76d37c8:::obj371:head@111'371, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 117 2026-02-11T23:31:27.844+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f76d37c8:::obj371:head, version: 111'371, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f76d37c8:::obj371:head@111'371, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e117) 2026-02-11T23:31:27.844+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.844+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1201 ==== pg_backfill(progress 1.0 e 117/117 lb 1:f677cfaa:::obj144:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f11600 con 0x4acb000 2026-02-11T23:31:27.844+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f76d37c8:::obj371:head, version: 111'371, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f76d37c8:::obj371:head@111'371, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) queued 2026-02-11T23:31:27.844+0000 7f416abfa640 15 osd.0 117 enqueue_op pg_backfill(progress 1.0 e 117/117 lb 1:f677cfaa:::obj144:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 117 2026-02-11T23:31:27.844+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.844+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f677cfaa:::obj144:head) v3) class_id 0 prio 15 cost 0 e117) 2026-02-11T23:31:27.844+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f76d37c8:::obj371:head, version: 111'371, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f76d37c8:::obj371:head@111'371, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) pg 0x4ef0000 2026-02-11T23:31:27.844+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.844+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:f76d37c8:::obj371:head, version: 111'371, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f76d37c8:::obj371:head@111'371, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 117/113 [PushOp(1:f76d37c8:::obj371:head, version: 111'371, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f76d37c8:::obj371:head@111'371, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:f4b32ae1:::obj130:head local-lis/les=113/114 n=581 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.845+0000 7f414b98b640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.845+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f4b32ae1:::obj130:head local-lis/les=113/114 n=581 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 117/113 [PushOp(1:f76d37c8:::obj371:head, version: 111'371, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f76d37c8:::obj371:head@111'371, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.845+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.845+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f677cfaa:::obj144:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) queued 2026-02-11T23:31:27.845+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f4b32ae1:::obj130:head local-lis/les=113/114 n=581 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:f76d37c8:::obj371:head@111'371, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.845+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f4b32ae1:::obj130:head local-lis/les=113/114 n=581 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:f76d37c8:::obj371:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.845+0000 7f414b98b640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f4b32ae1:::obj130:head local-lis/les=113/114 n=581 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:f76d37c8:::obj371:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.845+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f4b32ae1:::obj130:head local-lis/les=113/114 n=581 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:f76d37c8:::obj371:head 2026-02-11T23:31:27.845+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f4b32ae1:::obj130:head local-lis/les=113/114 n=581 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:f76d37c8:::obj371:head v 111'371 2026-02-11T23:31:27.845+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f4b32ae1:::obj130:head local-lis/les=113/114 n=581 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.845+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.845+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:f76d37c8:::obj371:head, version: 111'371, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f76d37c8:::obj371:head@111'371, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:27.845+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.845+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.845+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f677cfaa:::obj144:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) pg 0x4ef0000 2026-02-11T23:31:27.845+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:f677cfaa:::obj144:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 1.0 e 117/117 lb 1:f677cfaa:::obj144:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:f4b32ae1:::obj130:head local-lis/les=113/114 n=581 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.845+0000 7f414f993640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.845+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f4b32ae1:::obj130:head local-lis/les=113/114 n=581 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 117/117 lb 1:f677cfaa:::obj144:head) v3 2026-02-11T23:31:27.845+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f4b32ae1:::obj130:head local-lis/les=113/114 n=581 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 117/117 lb 1:f677cfaa:::obj144:head) 2026-02-11T23:31:27.845+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.845+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:f677cfaa:::obj144:head) v3 finish latency 0.000009 2026-02-11T23:31:27.845+0000 7f414f993640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f677cfaa:::obj144:head local-lis/les=113/114 n=582 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.845+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.845+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.845+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f677cfaa:::obj144:head local-lis/les=113/114 n=582 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.845+0000 7f414f993640 20 osd.0 117 get_con_osd_cluster to osd.1 from_epoch 117 2026-02-11T23:31:27.845+0000 7f414f993640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:27.845+0000 7f414f993640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:27.845+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e117/117) -- 0x5f1c480 con 0x4acb000 2026-02-11T23:31:27.845+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 117/113 [PushReplyOp(1:f76d37c8:::obj371:head)]) -- 0x5fd2780 con 0x4acb000 2026-02-11T23:31:27.845+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.845+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1202 ==== MOSDPGPush(1.0 117/113 [PushOp(1:f77ac86f:::obj198:head, version: 111'198, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f77ac86f:::obj198:head@111'198, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c9c0 con 0x4acb000 2026-02-11T23:31:27.845+0000 7f416abfa640 15 osd.0 117 enqueue_op MOSDPGPush(1.0 117/113 [PushOp(1:f77ac86f:::obj198:head, version: 111'198, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f77ac86f:::obj198:head@111'198, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 117 2026-02-11T23:31:27.845+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f77ac86f:::obj198:head, version: 111'198, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f77ac86f:::obj198:head@111'198, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e117) 2026-02-11T23:31:27.845+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.845+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1203 ==== pg_backfill(progress 1.0 e 117/117 lb 1:f76d37c8:::obj371:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2f700 con 0x4acb000 2026-02-11T23:31:27.845+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f77ac86f:::obj198:head, version: 111'198, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f77ac86f:::obj198:head@111'198, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) queued 2026-02-11T23:31:27.845+0000 7f416abfa640 15 osd.0 117 enqueue_op pg_backfill(progress 1.0 e 117/117 lb 1:f76d37c8:::obj371:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 117 2026-02-11T23:31:27.845+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f76d37c8:::obj371:head) v3) class_id 0 prio 15 cost 0 e117) 2026-02-11T23:31:27.845+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.845+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f77ac86f:::obj198:head, version: 111'198, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f77ac86f:::obj198:head@111'198, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) pg 0x4ef0000 2026-02-11T23:31:27.845+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.845+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:f77ac86f:::obj198:head, version: 111'198, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f77ac86f:::obj198:head@111'198, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 117/113 [PushOp(1:f77ac86f:::obj198:head, version: 111'198, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f77ac86f:::obj198:head@111'198, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:f677cfaa:::obj144:head local-lis/les=113/114 n=582 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.845+0000 7f414b98b640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.845+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f677cfaa:::obj144:head local-lis/les=113/114 n=582 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 117/113 [PushOp(1:f77ac86f:::obj198:head, version: 111'198, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f77ac86f:::obj198:head@111'198, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.845+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f677cfaa:::obj144:head local-lis/les=113/114 n=582 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:f77ac86f:::obj198:head@111'198, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.845+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.845+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f76d37c8:::obj371:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) queued 2026-02-11T23:31:27.845+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f677cfaa:::obj144:head local-lis/les=113/114 n=582 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:f77ac86f:::obj198:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.845+0000 7f414b98b640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f677cfaa:::obj144:head local-lis/les=113/114 n=582 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:f77ac86f:::obj198:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.845+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f677cfaa:::obj144:head local-lis/les=113/114 n=582 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:f77ac86f:::obj198:head 2026-02-11T23:31:27.845+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f677cfaa:::obj144:head local-lis/les=113/114 n=582 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:f77ac86f:::obj198:head v 111'198 2026-02-11T23:31:27.845+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f677cfaa:::obj144:head local-lis/les=113/114 n=582 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.845+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.845+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:f77ac86f:::obj198:head, version: 111'198, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f77ac86f:::obj198:head@111'198, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.845+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.845+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.845+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f76d37c8:::obj371:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) pg 0x4ef0000 2026-02-11T23:31:27.845+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:f76d37c8:::obj371:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 1.0 e 117/117 lb 1:f76d37c8:::obj371:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:f677cfaa:::obj144:head local-lis/les=113/114 n=582 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.845+0000 7f414f993640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.845+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f677cfaa:::obj144:head local-lis/les=113/114 n=582 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 117/117 lb 1:f76d37c8:::obj371:head) v3 2026-02-11T23:31:27.845+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f677cfaa:::obj144:head local-lis/les=113/114 n=582 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 117/117 lb 1:f76d37c8:::obj371:head) 2026-02-11T23:31:27.845+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.845+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:f76d37c8:::obj371:head) v3 finish latency 0.000008 2026-02-11T23:31:27.845+0000 7f414f993640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f76d37c8:::obj371:head local-lis/les=113/114 n=583 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.845+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.845+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.845+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f76d37c8:::obj371:head local-lis/les=113/114 n=583 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.845+0000 7f414f993640 20 osd.0 117 get_con_osd_cluster to osd.1 from_epoch 117 2026-02-11T23:31:27.845+0000 7f414f993640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:27.845+0000 7f414f993640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:27.845+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e117/117) -- 0x5f74000 con 0x4acb000 2026-02-11T23:31:27.845+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 117/113 [PushReplyOp(1:f77ac86f:::obj198:head)]) -- 0x5daa900 con 0x4acb000 2026-02-11T23:31:27.845+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.846+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1204 ==== MOSDPGPush(1.0 117/113 [PushOp(1:f7c5b032:::obj82:head, version: 111'82, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f7c5b032:::obj82:head@111'82, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x38156c0 con 0x4acb000 2026-02-11T23:31:27.846+0000 7f416abfa640 15 osd.0 117 enqueue_op MOSDPGPush(1.0 117/113 [PushOp(1:f7c5b032:::obj82:head, version: 111'82, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f7c5b032:::obj82:head@111'82, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 117 2026-02-11T23:31:27.846+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f7c5b032:::obj82:head, version: 111'82, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f7c5b032:::obj82:head@111'82, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e117) 2026-02-11T23:31:27.846+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.846+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1205 ==== pg_backfill(progress 1.0 e 117/117 lb 1:f77ac86f:::obj198:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2d080 con 0x4acb000 2026-02-11T23:31:27.846+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f7c5b032:::obj82:head, version: 111'82, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f7c5b032:::obj82:head@111'82, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) queued 2026-02-11T23:31:27.846+0000 7f416abfa640 15 osd.0 117 enqueue_op pg_backfill(progress 1.0 e 117/117 lb 1:f77ac86f:::obj198:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 117 2026-02-11T23:31:27.846+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.846+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f77ac86f:::obj198:head) v3) class_id 0 prio 15 cost 0 e117) 2026-02-11T23:31:27.846+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f7c5b032:::obj82:head, version: 111'82, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f7c5b032:::obj82:head@111'82, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) pg 0x4ef0000 2026-02-11T23:31:27.846+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.846+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:f7c5b032:::obj82:head, version: 111'82, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f7c5b032:::obj82:head@111'82, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 117/113 [PushOp(1:f7c5b032:::obj82:head, version: 111'82, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f7c5b032:::obj82:head@111'82, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:f76d37c8:::obj371:head local-lis/les=113/114 n=583 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.846+0000 7f414b98b640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.846+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f76d37c8:::obj371:head local-lis/les=113/114 n=583 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 117/113 [PushOp(1:f7c5b032:::obj82:head, version: 111'82, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f7c5b032:::obj82:head@111'82, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.846+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f76d37c8:::obj371:head local-lis/les=113/114 n=583 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:f7c5b032:::obj82:head@111'82, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.846+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.846+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f77ac86f:::obj198:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) queued 2026-02-11T23:31:27.846+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f76d37c8:::obj371:head local-lis/les=113/114 n=583 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:f7c5b032:::obj82:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.846+0000 7f414b98b640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f76d37c8:::obj371:head local-lis/les=113/114 n=583 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:f7c5b032:::obj82:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.846+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f76d37c8:::obj371:head local-lis/les=113/114 n=583 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:f7c5b032:::obj82:head 2026-02-11T23:31:27.846+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f76d37c8:::obj371:head local-lis/les=113/114 n=583 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:f7c5b032:::obj82:head v 111'82 2026-02-11T23:31:27.846+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f76d37c8:::obj371:head local-lis/les=113/114 n=583 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.846+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.846+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:f7c5b032:::obj82:head, version: 111'82, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f7c5b032:::obj82:head@111'82, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:27.846+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.846+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.846+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f77ac86f:::obj198:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) pg 0x4ef0000 2026-02-11T23:31:27.846+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:f77ac86f:::obj198:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 1.0 e 117/117 lb 1:f77ac86f:::obj198:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:f76d37c8:::obj371:head local-lis/les=113/114 n=583 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.846+0000 7f414f993640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.846+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f76d37c8:::obj371:head local-lis/les=113/114 n=583 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 117/117 lb 1:f77ac86f:::obj198:head) v3 2026-02-11T23:31:27.846+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f76d37c8:::obj371:head local-lis/les=113/114 n=583 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 117/117 lb 1:f77ac86f:::obj198:head) 2026-02-11T23:31:27.846+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.846+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:f77ac86f:::obj198:head) v3 finish latency 0.000009 2026-02-11T23:31:27.846+0000 7f414f993640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f77ac86f:::obj198:head local-lis/les=113/114 n=584 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.846+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.846+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.846+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f77ac86f:::obj198:head local-lis/les=113/114 n=584 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.846+0000 7f414f993640 20 osd.0 117 get_con_osd_cluster to osd.1 from_epoch 117 2026-02-11T23:31:27.846+0000 7f414f993640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:27.846+0000 7f414f993640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:27.846+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e117/117) -- 0x5f74180 con 0x4acb000 2026-02-11T23:31:27.846+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 117/113 [PushReplyOp(1:f7c5b032:::obj82:head)]) -- 0x5d78d80 con 0x4acb000 2026-02-11T23:31:27.846+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.846+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1206 ==== MOSDPGPush(1.0 117/113 [PushOp(1:f7eaa9b8:::obj104:head, version: 111'104, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f7eaa9b8:::obj104:head@111'104, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f1520 con 0x4acb000 2026-02-11T23:31:27.846+0000 7f416abfa640 15 osd.0 117 enqueue_op MOSDPGPush(1.0 117/113 [PushOp(1:f7eaa9b8:::obj104:head, version: 111'104, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f7eaa9b8:::obj104:head@111'104, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 117 2026-02-11T23:31:27.846+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f7eaa9b8:::obj104:head, version: 111'104, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f7eaa9b8:::obj104:head@111'104, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e117) 2026-02-11T23:31:27.846+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.846+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1207 ==== pg_backfill(progress 1.0 e 117/117 lb 1:f7c5b032:::obj82:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d2e680 con 0x4acb000 2026-02-11T23:31:27.846+0000 7f416abfa640 15 osd.0 117 enqueue_op pg_backfill(progress 1.0 e 117/117 lb 1:f7c5b032:::obj82:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 117 2026-02-11T23:31:27.846+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f7eaa9b8:::obj104:head, version: 111'104, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f7eaa9b8:::obj104:head@111'104, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) queued 2026-02-11T23:31:27.846+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f7c5b032:::obj82:head) v3) class_id 0 prio 15 cost 0 e117) 2026-02-11T23:31:27.846+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.846+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f7eaa9b8:::obj104:head, version: 111'104, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f7eaa9b8:::obj104:head@111'104, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) pg 0x4ef0000 2026-02-11T23:31:27.846+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:f7eaa9b8:::obj104:head, version: 111'104, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f7eaa9b8:::obj104:head@111'104, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 117/113 [PushOp(1:f7eaa9b8:::obj104:head, version: 111'104, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f7eaa9b8:::obj104:head@111'104, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:f77ac86f:::obj198:head local-lis/les=113/114 n=584 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.846+0000 7f414b98b640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.846+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.846+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f77ac86f:::obj198:head local-lis/les=113/114 n=584 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 117/113 [PushOp(1:f7eaa9b8:::obj104:head, version: 111'104, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f7eaa9b8:::obj104:head@111'104, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.846+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f7c5b032:::obj82:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) queued 2026-02-11T23:31:27.846+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f77ac86f:::obj198:head local-lis/les=113/114 n=584 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:f7eaa9b8:::obj104:head@111'104, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.846+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f77ac86f:::obj198:head local-lis/les=113/114 n=584 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:f7eaa9b8:::obj104:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.846+0000 7f414b98b640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f77ac86f:::obj198:head local-lis/les=113/114 n=584 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:f7eaa9b8:::obj104:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.846+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f77ac86f:::obj198:head local-lis/les=113/114 n=584 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:f7eaa9b8:::obj104:head 2026-02-11T23:31:27.846+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f77ac86f:::obj198:head local-lis/les=113/114 n=584 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:f7eaa9b8:::obj104:head v 111'104 2026-02-11T23:31:27.846+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f77ac86f:::obj198:head local-lis/les=113/114 n=584 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.846+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.846+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:f7eaa9b8:::obj104:head, version: 111'104, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f7eaa9b8:::obj104:head@111'104, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.846+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.846+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.846+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f7c5b032:::obj82:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) pg 0x4ef0000 2026-02-11T23:31:27.846+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:f7c5b032:::obj82:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 117/117 lb 1:f7c5b032:::obj82:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:f77ac86f:::obj198:head local-lis/les=113/114 n=584 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.846+0000 7f414f993640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.846+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f77ac86f:::obj198:head local-lis/les=113/114 n=584 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 117/117 lb 1:f7c5b032:::obj82:head) v3 2026-02-11T23:31:27.846+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f77ac86f:::obj198:head local-lis/les=113/114 n=584 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 117/117 lb 1:f7c5b032:::obj82:head) 2026-02-11T23:31:27.846+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.846+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:f7c5b032:::obj82:head) v3 finish latency 0.000008 2026-02-11T23:31:27.846+0000 7f414f993640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f7c5b032:::obj82:head local-lis/les=113/114 n=585 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.846+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.846+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.846+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f7c5b032:::obj82:head local-lis/les=113/114 n=585 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.846+0000 7f414f993640 20 osd.0 117 get_con_osd_cluster to osd.1 from_epoch 117 2026-02-11T23:31:27.846+0000 7f414f993640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:27.846+0000 7f414f993640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:27.846+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e117/117) -- 0x5f74300 con 0x4acb000 2026-02-11T23:31:27.846+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 117/113 [PushReplyOp(1:f7eaa9b8:::obj104:head)]) -- 0x5fd2c00 con 0x4acb000 2026-02-11T23:31:27.846+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.847+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1208 ==== MOSDPGPush(1.0 117/113 [PushOp(1:f948044a:::obj145:head, version: 111'145, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f948044a:::obj145:head@111'145, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3815860 con 0x4acb000 2026-02-11T23:31:27.847+0000 7f416abfa640 15 osd.0 117 enqueue_op MOSDPGPush(1.0 117/113 [PushOp(1:f948044a:::obj145:head, version: 111'145, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f948044a:::obj145:head@111'145, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 117 2026-02-11T23:31:27.847+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f948044a:::obj145:head, version: 111'145, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f948044a:::obj145:head@111'145, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e117) 2026-02-11T23:31:27.847+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.847+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1209 ==== pg_backfill(progress 1.0 e 117/117 lb 1:f7eaa9b8:::obj104:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e100 con 0x4acb000 2026-02-11T23:31:27.847+0000 7f416abfa640 15 osd.0 117 enqueue_op pg_backfill(progress 1.0 e 117/117 lb 1:f7eaa9b8:::obj104:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 117 2026-02-11T23:31:27.847+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f948044a:::obj145:head, version: 111'145, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f948044a:::obj145:head@111'145, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) queued 2026-02-11T23:31:27.847+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f7eaa9b8:::obj104:head) v3) class_id 0 prio 15 cost 0 e117) 2026-02-11T23:31:27.847+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.847+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:f948044a:::obj145:head, version: 111'145, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f948044a:::obj145:head@111'145, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) pg 0x4ef0000 2026-02-11T23:31:27.847+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:f948044a:::obj145:head, version: 111'145, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f948044a:::obj145:head@111'145, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 117/113 [PushOp(1:f948044a:::obj145:head, version: 111'145, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f948044a:::obj145:head@111'145, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:f7c5b032:::obj82:head local-lis/les=113/114 n=585 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.847+0000 7f414b98b640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.847+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.847+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f7eaa9b8:::obj104:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) queued 2026-02-11T23:31:27.847+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f7c5b032:::obj82:head local-lis/les=113/114 n=585 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 117/113 [PushOp(1:f948044a:::obj145:head, version: 111'145, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f948044a:::obj145:head@111'145, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.847+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f7c5b032:::obj82:head local-lis/les=113/114 n=585 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:f948044a:::obj145:head@111'145, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.847+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f7c5b032:::obj82:head local-lis/les=113/114 n=585 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:f948044a:::obj145:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.847+0000 7f414b98b640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f7c5b032:::obj82:head local-lis/les=113/114 n=585 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:f948044a:::obj145:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.847+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f7c5b032:::obj82:head local-lis/les=113/114 n=585 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:f948044a:::obj145:head 2026-02-11T23:31:27.847+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f7c5b032:::obj82:head local-lis/les=113/114 n=585 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:f948044a:::obj145:head v 111'145 2026-02-11T23:31:27.847+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f7c5b032:::obj82:head local-lis/les=113/114 n=585 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.847+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.847+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:f948044a:::obj145:head, version: 111'145, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f948044a:::obj145:head@111'145, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.847+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.847+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.847+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f7eaa9b8:::obj104:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) pg 0x4ef0000 2026-02-11T23:31:27.847+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:f7eaa9b8:::obj104:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 117/117 lb 1:f7eaa9b8:::obj104:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:f7c5b032:::obj82:head local-lis/les=113/114 n=585 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.847+0000 7f414f993640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.847+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f7c5b032:::obj82:head local-lis/les=113/114 n=585 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 117/117 lb 1:f7eaa9b8:::obj104:head) v3 2026-02-11T23:31:27.847+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f7c5b032:::obj82:head local-lis/les=113/114 n=585 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 117/117 lb 1:f7eaa9b8:::obj104:head) 2026-02-11T23:31:27.847+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.847+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:f7eaa9b8:::obj104:head) v3 finish latency 0.000009 2026-02-11T23:31:27.847+0000 7f414f993640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f7eaa9b8:::obj104:head local-lis/les=113/114 n=586 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.847+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.847+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.847+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f7eaa9b8:::obj104:head local-lis/les=113/114 n=586 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.847+0000 7f414f993640 20 osd.0 117 get_con_osd_cluster to osd.1 from_epoch 117 2026-02-11T23:31:27.847+0000 7f414f993640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:27.847+0000 7f414f993640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:27.847+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e117/117) -- 0x5f74480 con 0x4acb000 2026-02-11T23:31:27.847+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 117/113 [PushReplyOp(1:f948044a:::obj145:head)]) -- 0x5daa300 con 0x4acb000 2026-02-11T23:31:27.847+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.847+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1210 ==== MOSDPGPush(1.0 117/113 [PushOp(1:fa02d04b:::obj266:head, version: 111'266, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fa02d04b:::obj266:head@111'266, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5646ea0 con 0x4acb000 2026-02-11T23:31:27.847+0000 7f416abfa640 15 osd.0 117 enqueue_op MOSDPGPush(1.0 117/113 [PushOp(1:fa02d04b:::obj266:head, version: 111'266, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fa02d04b:::obj266:head@111'266, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 117 2026-02-11T23:31:27.847+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:fa02d04b:::obj266:head, version: 111'266, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fa02d04b:::obj266:head@111'266, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e117) 2026-02-11T23:31:27.847+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.847+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1211 ==== pg_backfill(progress 1.0 e 117/117 lb 1:f948044a:::obj145:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2c000 con 0x4acb000 2026-02-11T23:31:27.847+0000 7f416abfa640 15 osd.0 117 enqueue_op pg_backfill(progress 1.0 e 117/117 lb 1:f948044a:::obj145:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 117 2026-02-11T23:31:27.847+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:fa02d04b:::obj266:head, version: 111'266, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fa02d04b:::obj266:head@111'266, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) queued 2026-02-11T23:31:27.847+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.847+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f948044a:::obj145:head) v3) class_id 0 prio 15 cost 0 e117) 2026-02-11T23:31:27.847+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:fa02d04b:::obj266:head, version: 111'266, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fa02d04b:::obj266:head@111'266, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) pg 0x4ef0000 2026-02-11T23:31:27.847+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.847+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:fa02d04b:::obj266:head, version: 111'266, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fa02d04b:::obj266:head@111'266, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 117/113 [PushOp(1:fa02d04b:::obj266:head, version: 111'266, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fa02d04b:::obj266:head@111'266, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:f7eaa9b8:::obj104:head local-lis/les=113/114 n=586 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.847+0000 7f414b98b640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.847+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f7eaa9b8:::obj104:head local-lis/les=113/114 n=586 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 117/113 [PushOp(1:fa02d04b:::obj266:head, version: 111'266, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fa02d04b:::obj266:head@111'266, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.847+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f7eaa9b8:::obj104:head local-lis/les=113/114 n=586 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:fa02d04b:::obj266:head@111'266, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.847+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.847+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f948044a:::obj145:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) queued 2026-02-11T23:31:27.847+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f7eaa9b8:::obj104:head local-lis/les=113/114 n=586 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:fa02d04b:::obj266:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.847+0000 7f414b98b640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f7eaa9b8:::obj104:head local-lis/les=113/114 n=586 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:fa02d04b:::obj266:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.847+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f7eaa9b8:::obj104:head local-lis/les=113/114 n=586 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:fa02d04b:::obj266:head 2026-02-11T23:31:27.847+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f7eaa9b8:::obj104:head local-lis/les=113/114 n=586 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:fa02d04b:::obj266:head v 111'266 2026-02-11T23:31:27.847+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f7eaa9b8:::obj104:head local-lis/les=113/114 n=586 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.847+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.847+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:fa02d04b:::obj266:head, version: 111'266, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fa02d04b:::obj266:head@111'266, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000044 2026-02-11T23:31:27.847+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.847+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.847+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:f948044a:::obj145:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) pg 0x4ef0000 2026-02-11T23:31:27.847+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:f948044a:::obj145:head) v3 prio 15 cost 0 latency 0.000060 pg_backfill(progress 1.0 e 117/117 lb 1:f948044a:::obj145:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:f7eaa9b8:::obj104:head local-lis/les=113/114 n=586 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.847+0000 7f414f993640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.847+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f7eaa9b8:::obj104:head local-lis/les=113/114 n=586 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 117/117 lb 1:f948044a:::obj145:head) v3 2026-02-11T23:31:27.847+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f7eaa9b8:::obj104:head local-lis/les=113/114 n=586 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 117/117 lb 1:f948044a:::obj145:head) 2026-02-11T23:31:27.847+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.847+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:f948044a:::obj145:head) v3 finish latency 0.000009 2026-02-11T23:31:27.847+0000 7f414f993640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f948044a:::obj145:head local-lis/les=113/114 n=587 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.847+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.847+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.847+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f948044a:::obj145:head local-lis/les=113/114 n=587 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.847+0000 7f414f993640 20 osd.0 117 get_con_osd_cluster to osd.1 from_epoch 117 2026-02-11T23:31:27.847+0000 7f414f993640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:27.847+0000 7f414f993640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:27.847+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e117/117) -- 0x5f74600 con 0x4acb000 2026-02-11T23:31:27.847+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 117/113 [PushReplyOp(1:fa02d04b:::obj266:head)]) -- 0x600b680 con 0x4acb000 2026-02-11T23:31:27.847+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.848+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1212 ==== MOSDPGPush(1.0 117/113 [PushOp(1:fa6c2b5c:::obj438:head, version: 111'438, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fa6c2b5c:::obj438:head@111'438, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56cab60 con 0x4acb000 2026-02-11T23:31:27.848+0000 7f416abfa640 15 osd.0 117 enqueue_op MOSDPGPush(1.0 117/113 [PushOp(1:fa6c2b5c:::obj438:head, version: 111'438, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fa6c2b5c:::obj438:head@111'438, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 117 2026-02-11T23:31:27.848+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:fa6c2b5c:::obj438:head, version: 111'438, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fa6c2b5c:::obj438:head@111'438, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e117) 2026-02-11T23:31:27.848+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.848+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1213 ==== pg_backfill(progress 1.0 e 117/117 lb 1:fa02d04b:::obj266:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2cb00 con 0x4acb000 2026-02-11T23:31:27.848+0000 7f416abfa640 15 osd.0 117 enqueue_op pg_backfill(progress 1.0 e 117/117 lb 1:fa02d04b:::obj266:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 117 2026-02-11T23:31:27.848+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:fa6c2b5c:::obj438:head, version: 111'438, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fa6c2b5c:::obj438:head@111'438, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) queued 2026-02-11T23:31:27.848+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:fa02d04b:::obj266:head) v3) class_id 0 prio 15 cost 0 e117) 2026-02-11T23:31:27.848+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.848+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:fa6c2b5c:::obj438:head, version: 111'438, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fa6c2b5c:::obj438:head@111'438, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) pg 0x4ef0000 2026-02-11T23:31:27.848+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:fa6c2b5c:::obj438:head, version: 111'438, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fa6c2b5c:::obj438:head@111'438, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 117/113 [PushOp(1:fa6c2b5c:::obj438:head, version: 111'438, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fa6c2b5c:::obj438:head@111'438, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:f948044a:::obj145:head local-lis/les=113/114 n=587 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.848+0000 7f414b98b640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.848+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.848+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f948044a:::obj145:head local-lis/les=113/114 n=587 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 117/113 [PushOp(1:fa6c2b5c:::obj438:head, version: 111'438, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fa6c2b5c:::obj438:head@111'438, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.848+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:fa02d04b:::obj266:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) queued 2026-02-11T23:31:27.848+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f948044a:::obj145:head local-lis/les=113/114 n=587 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:fa6c2b5c:::obj438:head@111'438, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.848+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f948044a:::obj145:head local-lis/les=113/114 n=587 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:fa6c2b5c:::obj438:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.848+0000 7f414b98b640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f948044a:::obj145:head local-lis/les=113/114 n=587 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:fa6c2b5c:::obj438:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.848+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f948044a:::obj145:head local-lis/les=113/114 n=587 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:fa6c2b5c:::obj438:head 2026-02-11T23:31:27.848+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f948044a:::obj145:head local-lis/les=113/114 n=587 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:fa6c2b5c:::obj438:head v 111'438 2026-02-11T23:31:27.848+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f948044a:::obj145:head local-lis/les=113/114 n=587 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.848+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.848+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:fa6c2b5c:::obj438:head, version: 111'438, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fa6c2b5c:::obj438:head@111'438, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:27.848+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.848+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.848+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:fa02d04b:::obj266:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) pg 0x4ef0000 2026-02-11T23:31:27.848+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:fa02d04b:::obj266:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 117/117 lb 1:fa02d04b:::obj266:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:f948044a:::obj145:head local-lis/les=113/114 n=587 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.848+0000 7f414f993640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.848+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f948044a:::obj145:head local-lis/les=113/114 n=587 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 117/117 lb 1:fa02d04b:::obj266:head) v3 2026-02-11T23:31:27.848+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:f948044a:::obj145:head local-lis/les=113/114 n=587 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 117/117 lb 1:fa02d04b:::obj266:head) 2026-02-11T23:31:27.848+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.848+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:fa02d04b:::obj266:head) v3 finish latency 0.000010 2026-02-11T23:31:27.848+0000 7f414f993640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fa02d04b:::obj266:head local-lis/les=113/114 n=588 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.848+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.848+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.848+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fa02d04b:::obj266:head local-lis/les=113/114 n=588 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.848+0000 7f414f993640 20 osd.0 117 get_con_osd_cluster to osd.1 from_epoch 117 2026-02-11T23:31:27.848+0000 7f414f993640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:27.848+0000 7f414f993640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:27.848+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e117/117) -- 0x5f74780 con 0x4acb000 2026-02-11T23:31:27.848+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 117/113 [PushReplyOp(1:fa6c2b5c:::obj438:head)]) -- 0x5fcf800 con 0x4acb000 2026-02-11T23:31:27.848+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.848+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1214 ==== MOSDPGPush(1.0 117/113 [PushOp(1:fb333d49:::obj180:head, version: 111'180, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb333d49:::obj180:head@111'180, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2680 con 0x4acb000 2026-02-11T23:31:27.848+0000 7f416abfa640 15 osd.0 117 enqueue_op MOSDPGPush(1.0 117/113 [PushOp(1:fb333d49:::obj180:head, version: 111'180, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb333d49:::obj180:head@111'180, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 117 2026-02-11T23:31:27.848+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:fb333d49:::obj180:head, version: 111'180, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb333d49:::obj180:head@111'180, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e117) 2026-02-11T23:31:27.848+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.848+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1215 ==== pg_backfill(progress 1.0 e 117/117 lb 1:fa6c2b5c:::obj438:head) ==== 1062+0+0 (unknown 0 0 0) 0x6179700 con 0x4acb000 2026-02-11T23:31:27.848+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:fb333d49:::obj180:head, version: 111'180, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb333d49:::obj180:head@111'180, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) queued 2026-02-11T23:31:27.848+0000 7f416abfa640 15 osd.0 117 enqueue_op pg_backfill(progress 1.0 e 117/117 lb 1:fa6c2b5c:::obj438:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 117 2026-02-11T23:31:27.848+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.848+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:fb333d49:::obj180:head, version: 111'180, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb333d49:::obj180:head@111'180, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) pg 0x4ef0000 2026-02-11T23:31:27.848+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:fa6c2b5c:::obj438:head) v3) class_id 0 prio 15 cost 0 e117) 2026-02-11T23:31:27.848+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.848+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:fa6c2b5c:::obj438:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) queued 2026-02-11T23:31:27.848+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:fb333d49:::obj180:head, version: 111'180, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb333d49:::obj180:head@111'180, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(1.0 117/113 [PushOp(1:fb333d49:::obj180:head, version: 111'180, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb333d49:::obj180:head@111'180, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:fa02d04b:::obj266:head local-lis/les=113/114 n=588 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.848+0000 7f414b98b640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.848+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fa02d04b:::obj266:head local-lis/les=113/114 n=588 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 117/113 [PushOp(1:fb333d49:::obj180:head, version: 111'180, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb333d49:::obj180:head@111'180, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.848+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fa02d04b:::obj266:head local-lis/les=113/114 n=588 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:fb333d49:::obj180:head@111'180, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.848+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fa02d04b:::obj266:head local-lis/les=113/114 n=588 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:fb333d49:::obj180:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.848+0000 7f414b98b640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fa02d04b:::obj266:head local-lis/les=113/114 n=588 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:fb333d49:::obj180:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.848+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fa02d04b:::obj266:head local-lis/les=113/114 n=588 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:fb333d49:::obj180:head 2026-02-11T23:31:27.848+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fa02d04b:::obj266:head local-lis/les=113/114 n=588 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:fb333d49:::obj180:head v 111'180 2026-02-11T23:31:27.848+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fa02d04b:::obj266:head local-lis/les=113/114 n=588 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.848+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.848+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:fb333d49:::obj180:head, version: 111'180, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb333d49:::obj180:head@111'180, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.848+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.848+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.848+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:fa6c2b5c:::obj438:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) pg 0x4ef0000 2026-02-11T23:31:27.848+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:fa6c2b5c:::obj438:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 1.0 e 117/117 lb 1:fa6c2b5c:::obj438:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:fa02d04b:::obj266:head local-lis/les=113/114 n=588 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.848+0000 7f414f993640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.848+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fa02d04b:::obj266:head local-lis/les=113/114 n=588 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 117/117 lb 1:fa6c2b5c:::obj438:head) v3 2026-02-11T23:31:27.848+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fa02d04b:::obj266:head local-lis/les=113/114 n=588 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 117/117 lb 1:fa6c2b5c:::obj438:head) 2026-02-11T23:31:27.848+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.848+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:fa6c2b5c:::obj438:head) v3 finish latency 0.000010 2026-02-11T23:31:27.848+0000 7f414f993640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fa6c2b5c:::obj438:head local-lis/les=113/114 n=589 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.848+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.848+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.848+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fa6c2b5c:::obj438:head local-lis/les=113/114 n=589 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.848+0000 7f414f993640 20 osd.0 117 get_con_osd_cluster to osd.1 from_epoch 117 2026-02-11T23:31:27.848+0000 7f414f993640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:27.848+0000 7f414f993640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:27.848+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e117/117) -- 0x5f74900 con 0x4acb000 2026-02-11T23:31:27.848+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 117/113 [PushReplyOp(1:fb333d49:::obj180:head)]) -- 0x5fed380 con 0x4acb000 2026-02-11T23:31:27.848+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.849+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1216 ==== MOSDPGPush(1.0 117/113 [PushOp(1:fb44f156:::obj279:head, version: 111'279, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb44f156:::obj279:head@111'279, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56aa000 con 0x4acb000 2026-02-11T23:31:27.849+0000 7f416abfa640 15 osd.0 117 enqueue_op MOSDPGPush(1.0 117/113 [PushOp(1:fb44f156:::obj279:head, version: 111'279, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb44f156:::obj279:head@111'279, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 117 2026-02-11T23:31:27.849+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:fb44f156:::obj279:head, version: 111'279, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb44f156:::obj279:head@111'279, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e117) 2026-02-11T23:31:27.849+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.849+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1217 ==== pg_backfill(progress 1.0 e 117/117 lb 1:fb333d49:::obj180:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176000 con 0x4acb000 2026-02-11T23:31:27.849+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:fb44f156:::obj279:head, version: 111'279, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb44f156:::obj279:head@111'279, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) queued 2026-02-11T23:31:27.849+0000 7f416abfa640 15 osd.0 117 enqueue_op pg_backfill(progress 1.0 e 117/117 lb 1:fb333d49:::obj180:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 117 2026-02-11T23:31:27.849+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:fb333d49:::obj180:head) v3) class_id 0 prio 15 cost 0 e117) 2026-02-11T23:31:27.849+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.849+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:fb44f156:::obj279:head, version: 111'279, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb44f156:::obj279:head@111'279, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) pg 0x4ef0000 2026-02-11T23:31:27.849+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.849+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:fb44f156:::obj279:head, version: 111'279, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb44f156:::obj279:head@111'279, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000044 MOSDPGPush(1.0 117/113 [PushOp(1:fb44f156:::obj279:head, version: 111'279, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb44f156:::obj279:head@111'279, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:fa6c2b5c:::obj438:head local-lis/les=113/114 n=589 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.849+0000 7f414b98b640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.849+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fa6c2b5c:::obj438:head local-lis/les=113/114 n=589 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 117/113 [PushOp(1:fb44f156:::obj279:head, version: 111'279, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb44f156:::obj279:head@111'279, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.849+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fa6c2b5c:::obj438:head local-lis/les=113/114 n=589 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:fb44f156:::obj279:head@111'279, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.849+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.849+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:fb333d49:::obj180:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) queued 2026-02-11T23:31:27.849+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fa6c2b5c:::obj438:head local-lis/les=113/114 n=589 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:fb44f156:::obj279:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.849+0000 7f414b98b640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fa6c2b5c:::obj438:head local-lis/les=113/114 n=589 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:fb44f156:::obj279:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.849+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fa6c2b5c:::obj438:head local-lis/les=113/114 n=589 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:fb44f156:::obj279:head 2026-02-11T23:31:27.849+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fa6c2b5c:::obj438:head local-lis/les=113/114 n=589 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:fb44f156:::obj279:head v 111'279 2026-02-11T23:31:27.849+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fa6c2b5c:::obj438:head local-lis/les=113/114 n=589 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.849+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.849+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:fb44f156:::obj279:head, version: 111'279, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb44f156:::obj279:head@111'279, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:27.849+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.849+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.849+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:fb333d49:::obj180:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) pg 0x4ef0000 2026-02-11T23:31:27.849+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:fb333d49:::obj180:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 1.0 e 117/117 lb 1:fb333d49:::obj180:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:fa6c2b5c:::obj438:head local-lis/les=113/114 n=589 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.849+0000 7f414f993640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.849+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fa6c2b5c:::obj438:head local-lis/les=113/114 n=589 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 117/117 lb 1:fb333d49:::obj180:head) v3 2026-02-11T23:31:27.849+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fa6c2b5c:::obj438:head local-lis/les=113/114 n=589 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 117/117 lb 1:fb333d49:::obj180:head) 2026-02-11T23:31:27.849+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.849+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:fb333d49:::obj180:head) v3 finish latency 0.000011 2026-02-11T23:31:27.849+0000 7f414f993640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fb333d49:::obj180:head local-lis/les=113/114 n=590 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.849+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.849+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.849+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fb333d49:::obj180:head local-lis/les=113/114 n=590 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.849+0000 7f414f993640 20 osd.0 117 get_con_osd_cluster to osd.1 from_epoch 117 2026-02-11T23:31:27.849+0000 7f414f993640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:27.849+0000 7f414f993640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:27.849+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e117/117) -- 0x5f74a80 con 0x4acb000 2026-02-11T23:31:27.849+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 117/113 [PushReplyOp(1:fb44f156:::obj279:head)]) -- 0x5f74480 con 0x4acb000 2026-02-11T23:31:27.849+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.849+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1218 ==== MOSDPGPush(1.0 117/113 [PushOp(1:fb8f5c57:::obj248:head, version: 111'248, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb8f5c57:::obj248:head@111'248, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a3040 con 0x4acb000 2026-02-11T23:31:27.849+0000 7f416abfa640 15 osd.0 117 enqueue_op MOSDPGPush(1.0 117/113 [PushOp(1:fb8f5c57:::obj248:head, version: 111'248, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb8f5c57:::obj248:head@111'248, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000018 epoch 117 2026-02-11T23:31:27.849+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:fb8f5c57:::obj248:head, version: 111'248, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb8f5c57:::obj248:head@111'248, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e117) 2026-02-11T23:31:27.849+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.849+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:fb8f5c57:::obj248:head, version: 111'248, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb8f5c57:::obj248:head@111'248, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) queued 2026-02-11T23:31:27.849+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1219 ==== pg_backfill(progress 1.0 e 117/117 lb 1:fb44f156:::obj279:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176b00 con 0x4acb000 2026-02-11T23:31:27.849+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.849+0000 7f416abfa640 15 osd.0 117 enqueue_op pg_backfill(progress 1.0 e 117/117 lb 1:fb44f156:::obj279:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 117 2026-02-11T23:31:27.849+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:fb8f5c57:::obj248:head, version: 111'248, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb8f5c57:::obj248:head@111'248, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) pg 0x4ef0000 2026-02-11T23:31:27.849+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:fb44f156:::obj279:head) v3) class_id 0 prio 15 cost 0 e117) 2026-02-11T23:31:27.849+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.849+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:fb8f5c57:::obj248:head, version: 111'248, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb8f5c57:::obj248:head@111'248, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000046 MOSDPGPush(1.0 117/113 [PushOp(1:fb8f5c57:::obj248:head, version: 111'248, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb8f5c57:::obj248:head@111'248, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:fb333d49:::obj180:head local-lis/les=113/114 n=590 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.849+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:fb44f156:::obj279:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) queued 2026-02-11T23:31:27.849+0000 7f414b98b640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.849+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fb333d49:::obj180:head local-lis/les=113/114 n=590 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 117/113 [PushOp(1:fb8f5c57:::obj248:head, version: 111'248, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb8f5c57:::obj248:head@111'248, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.849+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fb333d49:::obj180:head local-lis/les=113/114 n=590 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:fb8f5c57:::obj248:head@111'248, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.849+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fb333d49:::obj180:head local-lis/les=113/114 n=590 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:fb8f5c57:::obj248:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.849+0000 7f414b98b640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fb333d49:::obj180:head local-lis/les=113/114 n=590 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:fb8f5c57:::obj248:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.849+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fb333d49:::obj180:head local-lis/les=113/114 n=590 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:fb8f5c57:::obj248:head 2026-02-11T23:31:27.849+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fb333d49:::obj180:head local-lis/les=113/114 n=590 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:fb8f5c57:::obj248:head v 111'248 2026-02-11T23:31:27.849+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fb333d49:::obj180:head local-lis/les=113/114 n=590 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.849+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.849+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:fb8f5c57:::obj248:head, version: 111'248, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb8f5c57:::obj248:head@111'248, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:27.849+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.849+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.849+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:fb44f156:::obj279:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) pg 0x4ef0000 2026-02-11T23:31:27.849+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:fb44f156:::obj279:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 117/117 lb 1:fb44f156:::obj279:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:fb333d49:::obj180:head local-lis/les=113/114 n=590 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.849+0000 7f414f993640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.849+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fb333d49:::obj180:head local-lis/les=113/114 n=590 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 117/117 lb 1:fb44f156:::obj279:head) v3 2026-02-11T23:31:27.849+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fb333d49:::obj180:head local-lis/les=113/114 n=590 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 117/117 lb 1:fb44f156:::obj279:head) 2026-02-11T23:31:27.849+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.849+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:fb44f156:::obj279:head) v3 finish latency 0.000010 2026-02-11T23:31:27.849+0000 7f414f993640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fb44f156:::obj279:head local-lis/les=113/114 n=591 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.849+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.850+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.850+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fb44f156:::obj279:head local-lis/les=113/114 n=591 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.850+0000 7f414f993640 20 osd.0 117 get_con_osd_cluster to osd.1 from_epoch 117 2026-02-11T23:31:27.850+0000 7f414f993640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:27.850+0000 7f414f993640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:27.850+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e117/117) -- 0x5f74c00 con 0x4acb000 2026-02-11T23:31:27.850+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 117/113 [PushReplyOp(1:fb8f5c57:::obj248:head)]) -- 0x5f74300 con 0x4acb000 2026-02-11T23:31:27.850+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.850+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1220 ==== MOSDPGPush(1.0 117/113 [PushOp(1:fcc34b3b:::obj191:head, version: 111'191, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fcc34b3b:::obj191:head@111'191, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x554e340 con 0x4acb000 2026-02-11T23:31:27.850+0000 7f416abfa640 15 osd.0 117 enqueue_op MOSDPGPush(1.0 117/113 [PushOp(1:fcc34b3b:::obj191:head, version: 111'191, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fcc34b3b:::obj191:head@111'191, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 117 2026-02-11T23:31:27.850+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:fcc34b3b:::obj191:head, version: 111'191, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fcc34b3b:::obj191:head@111'191, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e117) 2026-02-11T23:31:27.850+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.850+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1221 ==== pg_backfill(progress 1.0 e 117/117 lb 1:fb8f5c57:::obj248:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177080 con 0x4acb000 2026-02-11T23:31:27.850+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:fcc34b3b:::obj191:head, version: 111'191, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fcc34b3b:::obj191:head@111'191, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) queued 2026-02-11T23:31:27.850+0000 7f416abfa640 15 osd.0 117 enqueue_op pg_backfill(progress 1.0 e 117/117 lb 1:fb8f5c57:::obj248:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 117 2026-02-11T23:31:27.850+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.850+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:fb8f5c57:::obj248:head) v3) class_id 0 prio 15 cost 0 e117) 2026-02-11T23:31:27.850+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:fcc34b3b:::obj191:head, version: 111'191, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fcc34b3b:::obj191:head@111'191, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) pg 0x4ef0000 2026-02-11T23:31:27.850+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.850+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:fcc34b3b:::obj191:head, version: 111'191, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fcc34b3b:::obj191:head@111'191, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 117/113 [PushOp(1:fcc34b3b:::obj191:head, version: 111'191, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fcc34b3b:::obj191:head@111'191, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:fb44f156:::obj279:head local-lis/les=113/114 n=591 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.850+0000 7f414b98b640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.850+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fb44f156:::obj279:head local-lis/les=113/114 n=591 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 117/113 [PushOp(1:fcc34b3b:::obj191:head, version: 111'191, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fcc34b3b:::obj191:head@111'191, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.850+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fb44f156:::obj279:head local-lis/les=113/114 n=591 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:fcc34b3b:::obj191:head@111'191, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.850+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.850+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:fb8f5c57:::obj248:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) queued 2026-02-11T23:31:27.850+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fb44f156:::obj279:head local-lis/les=113/114 n=591 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:fcc34b3b:::obj191:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.850+0000 7f414b98b640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fb44f156:::obj279:head local-lis/les=113/114 n=591 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:fcc34b3b:::obj191:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.850+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fb44f156:::obj279:head local-lis/les=113/114 n=591 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:fcc34b3b:::obj191:head 2026-02-11T23:31:27.850+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fb44f156:::obj279:head local-lis/les=113/114 n=591 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:fcc34b3b:::obj191:head v 111'191 2026-02-11T23:31:27.850+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fb44f156:::obj279:head local-lis/les=113/114 n=591 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.850+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.850+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:fcc34b3b:::obj191:head, version: 111'191, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fcc34b3b:::obj191:head@111'191, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:27.850+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.850+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.850+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:fb8f5c57:::obj248:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) pg 0x4ef0000 2026-02-11T23:31:27.850+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:fb8f5c57:::obj248:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 117/117 lb 1:fb8f5c57:::obj248:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:fb44f156:::obj279:head local-lis/les=113/114 n=591 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.850+0000 7f414f993640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.850+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fb44f156:::obj279:head local-lis/les=113/114 n=591 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 117/117 lb 1:fb8f5c57:::obj248:head) v3 2026-02-11T23:31:27.850+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fb44f156:::obj279:head local-lis/les=113/114 n=591 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 117/117 lb 1:fb8f5c57:::obj248:head) 2026-02-11T23:31:27.850+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.850+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:fb8f5c57:::obj248:head) v3 finish latency 0.000010 2026-02-11T23:31:27.850+0000 7f414f993640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fb8f5c57:::obj248:head local-lis/les=113/114 n=592 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.850+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.850+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.850+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fb8f5c57:::obj248:head local-lis/les=113/114 n=592 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.850+0000 7f414f993640 20 osd.0 117 get_con_osd_cluster to osd.1 from_epoch 117 2026-02-11T23:31:27.850+0000 7f414f993640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:27.850+0000 7f414f993640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:27.850+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e117/117) -- 0x5f74d80 con 0x4acb000 2026-02-11T23:31:27.850+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 117/113 [PushReplyOp(1:fcc34b3b:::obj191:head)]) -- 0x5f74180 con 0x4acb000 2026-02-11T23:31:27.850+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.850+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1222 ==== MOSDPGPush(1.0 117/113 [PushOp(1:fcdeb4b1:::obj379:head, version: 111'379, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fcdeb4b1:::obj379:head@111'379, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a21a0 con 0x4acb000 2026-02-11T23:31:27.850+0000 7f416abfa640 15 osd.0 117 enqueue_op MOSDPGPush(1.0 117/113 [PushOp(1:fcdeb4b1:::obj379:head, version: 111'379, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fcdeb4b1:::obj379:head@111'379, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 117 2026-02-11T23:31:27.850+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:fcdeb4b1:::obj379:head, version: 111'379, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fcdeb4b1:::obj379:head@111'379, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e117) 2026-02-11T23:31:27.850+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.850+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1223 ==== pg_backfill(progress 1.0 e 117/117 lb 1:fcc34b3b:::obj191:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177600 con 0x4acb000 2026-02-11T23:31:27.850+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:fcdeb4b1:::obj379:head, version: 111'379, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fcdeb4b1:::obj379:head@111'379, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) queued 2026-02-11T23:31:27.850+0000 7f416abfa640 15 osd.0 117 enqueue_op pg_backfill(progress 1.0 e 117/117 lb 1:fcc34b3b:::obj191:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 117 2026-02-11T23:31:27.850+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.850+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:fcc34b3b:::obj191:head) v3) class_id 0 prio 15 cost 0 e117) 2026-02-11T23:31:27.850+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:fcdeb4b1:::obj379:head, version: 111'379, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fcdeb4b1:::obj379:head@111'379, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) pg 0x4ef0000 2026-02-11T23:31:27.850+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.850+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:fcdeb4b1:::obj379:head, version: 111'379, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fcdeb4b1:::obj379:head@111'379, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 117/113 [PushOp(1:fcdeb4b1:::obj379:head, version: 111'379, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fcdeb4b1:::obj379:head@111'379, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:fb8f5c57:::obj248:head local-lis/les=113/114 n=592 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.850+0000 7f414b98b640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.850+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fb8f5c57:::obj248:head local-lis/les=113/114 n=592 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 117/113 [PushOp(1:fcdeb4b1:::obj379:head, version: 111'379, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fcdeb4b1:::obj379:head@111'379, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.850+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fb8f5c57:::obj248:head local-lis/les=113/114 n=592 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:fcdeb4b1:::obj379:head@111'379, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.850+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.850+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:fcc34b3b:::obj191:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) queued 2026-02-11T23:31:27.850+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fb8f5c57:::obj248:head local-lis/les=113/114 n=592 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:fcdeb4b1:::obj379:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.850+0000 7f414b98b640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fb8f5c57:::obj248:head local-lis/les=113/114 n=592 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:fcdeb4b1:::obj379:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.850+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fb8f5c57:::obj248:head local-lis/les=113/114 n=592 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:fcdeb4b1:::obj379:head 2026-02-11T23:31:27.850+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fb8f5c57:::obj248:head local-lis/les=113/114 n=592 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:fcdeb4b1:::obj379:head v 111'379 2026-02-11T23:31:27.850+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fb8f5c57:::obj248:head local-lis/les=113/114 n=592 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.850+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.850+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:fcdeb4b1:::obj379:head, version: 111'379, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fcdeb4b1:::obj379:head@111'379, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.850+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.850+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.850+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:fcc34b3b:::obj191:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) pg 0x4ef0000 2026-02-11T23:31:27.850+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:fcc34b3b:::obj191:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 117/117 lb 1:fcc34b3b:::obj191:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:fb8f5c57:::obj248:head local-lis/les=113/114 n=592 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.850+0000 7f414f993640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.850+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fb8f5c57:::obj248:head local-lis/les=113/114 n=592 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 117/117 lb 1:fcc34b3b:::obj191:head) v3 2026-02-11T23:31:27.850+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fb8f5c57:::obj248:head local-lis/les=113/114 n=592 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 117/117 lb 1:fcc34b3b:::obj191:head) 2026-02-11T23:31:27.850+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.850+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:fcc34b3b:::obj191:head) v3 finish latency 0.000010 2026-02-11T23:31:27.850+0000 7f414f993640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fcc34b3b:::obj191:head local-lis/les=113/114 n=593 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.850+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.851+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.851+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fcc34b3b:::obj191:head local-lis/les=113/114 n=593 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.851+0000 7f414f993640 20 osd.0 117 get_con_osd_cluster to osd.1 from_epoch 117 2026-02-11T23:31:27.851+0000 7f414f993640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:27.851+0000 7f414f993640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:27.851+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e117/117) -- 0x5f74f00 con 0x4acb000 2026-02-11T23:31:27.851+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 117/113 [PushReplyOp(1:fcdeb4b1:::obj379:head)]) -- 0x5f74000 con 0x4acb000 2026-02-11T23:31:27.851+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.851+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1224 ==== MOSDPGPush(1.0 117/113 [PushOp(1:fd91bcb6:::obj528:head, version: 111'528, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fd91bcb6:::obj528:head@111'528, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56ca4e0 con 0x4acb000 2026-02-11T23:31:27.851+0000 7f416abfa640 15 osd.0 117 enqueue_op MOSDPGPush(1.0 117/113 [PushOp(1:fd91bcb6:::obj528:head, version: 111'528, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fd91bcb6:::obj528:head@111'528, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 117 2026-02-11T23:31:27.851+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:fd91bcb6:::obj528:head, version: 111'528, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fd91bcb6:::obj528:head@111'528, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e117) 2026-02-11T23:31:27.851+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.851+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1225 ==== pg_backfill(progress 1.0 e 117/117 lb 1:fcdeb4b1:::obj379:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177b80 con 0x4acb000 2026-02-11T23:31:27.851+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:fd91bcb6:::obj528:head, version: 111'528, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fd91bcb6:::obj528:head@111'528, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) queued 2026-02-11T23:31:27.851+0000 7f416abfa640 15 osd.0 117 enqueue_op pg_backfill(progress 1.0 e 117/117 lb 1:fcdeb4b1:::obj379:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 117 2026-02-11T23:31:27.851+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.851+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:fcdeb4b1:::obj379:head) v3) class_id 0 prio 15 cost 0 e117) 2026-02-11T23:31:27.851+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:fd91bcb6:::obj528:head, version: 111'528, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fd91bcb6:::obj528:head@111'528, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) pg 0x4ef0000 2026-02-11T23:31:27.851+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.851+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:fd91bcb6:::obj528:head, version: 111'528, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fd91bcb6:::obj528:head@111'528, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 117/113 [PushOp(1:fd91bcb6:::obj528:head, version: 111'528, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fd91bcb6:::obj528:head@111'528, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:fcc34b3b:::obj191:head local-lis/les=113/114 n=593 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.851+0000 7f414b98b640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.851+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fcc34b3b:::obj191:head local-lis/les=113/114 n=593 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 117/113 [PushOp(1:fd91bcb6:::obj528:head, version: 111'528, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fd91bcb6:::obj528:head@111'528, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.851+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fcc34b3b:::obj191:head local-lis/les=113/114 n=593 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:fd91bcb6:::obj528:head@111'528, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.851+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.851+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:fcdeb4b1:::obj379:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) queued 2026-02-11T23:31:27.851+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fcc34b3b:::obj191:head local-lis/les=113/114 n=593 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:fd91bcb6:::obj528:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.851+0000 7f414b98b640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fcc34b3b:::obj191:head local-lis/les=113/114 n=593 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:fd91bcb6:::obj528:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.851+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fcc34b3b:::obj191:head local-lis/les=113/114 n=593 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:fd91bcb6:::obj528:head 2026-02-11T23:31:27.851+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fcc34b3b:::obj191:head local-lis/les=113/114 n=593 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:fd91bcb6:::obj528:head v 111'528 2026-02-11T23:31:27.851+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fcc34b3b:::obj191:head local-lis/les=113/114 n=593 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.851+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.851+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:fd91bcb6:::obj528:head, version: 111'528, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fd91bcb6:::obj528:head@111'528, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.851+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.851+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.851+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:fcdeb4b1:::obj379:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) pg 0x4ef0000 2026-02-11T23:31:27.851+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:fcdeb4b1:::obj379:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 117/117 lb 1:fcdeb4b1:::obj379:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:fcc34b3b:::obj191:head local-lis/les=113/114 n=593 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.851+0000 7f414f993640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.851+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fcc34b3b:::obj191:head local-lis/les=113/114 n=593 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 117/117 lb 1:fcdeb4b1:::obj379:head) v3 2026-02-11T23:31:27.851+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fcc34b3b:::obj191:head local-lis/les=113/114 n=593 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 117/117 lb 1:fcdeb4b1:::obj379:head) 2026-02-11T23:31:27.851+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.851+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:fcdeb4b1:::obj379:head) v3 finish latency 0.000009 2026-02-11T23:31:27.851+0000 7f414f993640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fcdeb4b1:::obj379:head local-lis/les=113/114 n=594 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.851+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.851+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.851+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fcdeb4b1:::obj379:head local-lis/les=113/114 n=594 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.851+0000 7f414f993640 20 osd.0 117 get_con_osd_cluster to osd.1 from_epoch 117 2026-02-11T23:31:27.851+0000 7f414f993640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:27.851+0000 7f414f993640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:27.851+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e117/117) -- 0x5f75080 con 0x4acb000 2026-02-11T23:31:27.851+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 117/113 [PushReplyOp(1:fd91bcb6:::obj528:head)]) -- 0x5f1c480 con 0x4acb000 2026-02-11T23:31:27.851+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.851+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1226 ==== MOSDPGPush(1.0 117/113 [PushOp(1:fe351e27:::obj9:head, version: 111'9, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fe351e27:::obj9:head@111'9, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 4999+0+0 (unknown 0 0 0) 0x56a2d00 con 0x4acb000 2026-02-11T23:31:27.851+0000 7f416abfa640 15 osd.0 117 enqueue_op MOSDPGPush(1.0 117/113 [PushOp(1:fe351e27:::obj9:head, version: 111'9, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fe351e27:::obj9:head@111'9, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 117 2026-02-11T23:31:27.851+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:fe351e27:::obj9:head, version: 111'9, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fe351e27:::obj9:head@111'9, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e117) 2026-02-11T23:31:27.851+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.851+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1227 ==== pg_backfill(progress 1.0 e 117/117 lb 1:fd91bcb6:::obj528:head) ==== 1062+0+0 (unknown 0 0 0) 0x6178100 con 0x4acb000 2026-02-11T23:31:27.851+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:fe351e27:::obj9:head, version: 111'9, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fe351e27:::obj9:head@111'9, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) queued 2026-02-11T23:31:27.851+0000 7f416abfa640 15 osd.0 117 enqueue_op pg_backfill(progress 1.0 e 117/117 lb 1:fd91bcb6:::obj528:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 117 2026-02-11T23:31:27.851+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.851+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:fd91bcb6:::obj528:head) v3) class_id 0 prio 15 cost 0 e117) 2026-02-11T23:31:27.851+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:fe351e27:::obj9:head, version: 111'9, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fe351e27:::obj9:head@111'9, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) pg 0x4ef0000 2026-02-11T23:31:27.851+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.851+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:fe351e27:::obj9:head, version: 111'9, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fe351e27:::obj9:head@111'9, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 117/113 [PushOp(1:fe351e27:::obj9:head, version: 111'9, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fe351e27:::obj9:head@111'9, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:fcdeb4b1:::obj379:head local-lis/les=113/114 n=594 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.851+0000 7f414b98b640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.851+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fcdeb4b1:::obj379:head local-lis/les=113/114 n=594 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 117/113 [PushOp(1:fe351e27:::obj9:head, version: 111'9, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fe351e27:::obj9:head@111'9, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.851+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fcdeb4b1:::obj379:head local-lis/les=113/114 n=594 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:fe351e27:::obj9:head@111'9, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.851+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.851+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:fd91bcb6:::obj528:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) queued 2026-02-11T23:31:27.851+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fcdeb4b1:::obj379:head local-lis/les=113/114 n=594 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:fe351e27:::obj9:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.851+0000 7f414b98b640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fcdeb4b1:::obj379:head local-lis/les=113/114 n=594 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:fe351e27:::obj9:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.851+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fcdeb4b1:::obj379:head local-lis/les=113/114 n=594 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:fe351e27:::obj9:head 2026-02-11T23:31:27.851+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fcdeb4b1:::obj379:head local-lis/les=113/114 n=594 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:fe351e27:::obj9:head v 111'9 2026-02-11T23:31:27.851+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fcdeb4b1:::obj379:head local-lis/les=113/114 n=594 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.851+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.851+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:fe351e27:::obj9:head, version: 111'9, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fe351e27:::obj9:head@111'9, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.851+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.851+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.851+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:fd91bcb6:::obj528:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) pg 0x4ef0000 2026-02-11T23:31:27.851+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:fd91bcb6:::obj528:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 1.0 e 117/117 lb 1:fd91bcb6:::obj528:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:fcdeb4b1:::obj379:head local-lis/les=113/114 n=594 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.851+0000 7f414f993640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.851+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fcdeb4b1:::obj379:head local-lis/les=113/114 n=594 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 117/117 lb 1:fd91bcb6:::obj528:head) v3 2026-02-11T23:31:27.851+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fcdeb4b1:::obj379:head local-lis/les=113/114 n=594 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 117/117 lb 1:fd91bcb6:::obj528:head) 2026-02-11T23:31:27.851+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.851+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:fd91bcb6:::obj528:head) v3 finish latency 0.000010 2026-02-11T23:31:27.851+0000 7f414f993640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fd91bcb6:::obj528:head local-lis/les=113/114 n=595 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.851+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.852+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.852+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fd91bcb6:::obj528:head local-lis/les=113/114 n=595 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.852+0000 7f414f993640 20 osd.0 117 get_con_osd_cluster to osd.1 from_epoch 117 2026-02-11T23:31:27.852+0000 7f414f993640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:27.852+0000 7f414f993640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:27.852+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e117/117) -- 0x5f75200 con 0x4acb000 2026-02-11T23:31:27.852+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 117/113 [PushReplyOp(1:fe351e27:::obj9:head)]) -- 0x5f2c900 con 0x4acb000 2026-02-11T23:31:27.852+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.852+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1228 ==== MOSDPGPush(1.0 117/113 [PushOp(1:fedf1386:::obj565:head, version: 111'565, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fedf1386:::obj565:head@111'565, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2820 con 0x4acb000 2026-02-11T23:31:27.852+0000 7f416abfa640 15 osd.0 117 enqueue_op MOSDPGPush(1.0 117/113 [PushOp(1:fedf1386:::obj565:head, version: 111'565, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fedf1386:::obj565:head@111'565, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 117 2026-02-11T23:31:27.852+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:fedf1386:::obj565:head, version: 111'565, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fedf1386:::obj565:head@111'565, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e117) 2026-02-11T23:31:27.852+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.852+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1229 ==== pg_backfill(progress 1.0 e 117/117 lb 1:fe351e27:::obj9:head) ==== 1060+0+0 (unknown 0 0 0) 0x6179180 con 0x4acb000 2026-02-11T23:31:27.852+0000 7f416abfa640 15 osd.0 117 enqueue_op pg_backfill(progress 1.0 e 117/117 lb 1:fe351e27:::obj9:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 117 2026-02-11T23:31:27.852+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:fedf1386:::obj565:head, version: 111'565, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fedf1386:::obj565:head@111'565, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) queued 2026-02-11T23:31:27.852+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:fe351e27:::obj9:head) v3) class_id 0 prio 15 cost 0 e117) 2026-02-11T23:31:27.852+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.852+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:fedf1386:::obj565:head, version: 111'565, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fedf1386:::obj565:head@111'565, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) pg 0x4ef0000 2026-02-11T23:31:27.852+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:fedf1386:::obj565:head, version: 111'565, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fedf1386:::obj565:head@111'565, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 117/113 [PushOp(1:fedf1386:::obj565:head, version: 111'565, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fedf1386:::obj565:head@111'565, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:fd91bcb6:::obj528:head local-lis/les=113/114 n=595 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.852+0000 7f414b98b640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.852+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.852+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fd91bcb6:::obj528:head local-lis/les=113/114 n=595 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 117/113 [PushOp(1:fedf1386:::obj565:head, version: 111'565, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fedf1386:::obj565:head@111'565, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.852+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:fe351e27:::obj9:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) queued 2026-02-11T23:31:27.852+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fd91bcb6:::obj528:head local-lis/les=113/114 n=595 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:fedf1386:::obj565:head@111'565, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.852+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fd91bcb6:::obj528:head local-lis/les=113/114 n=595 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:fedf1386:::obj565:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.852+0000 7f414b98b640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fd91bcb6:::obj528:head local-lis/les=113/114 n=595 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:fedf1386:::obj565:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.852+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fd91bcb6:::obj528:head local-lis/les=113/114 n=595 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:fedf1386:::obj565:head 2026-02-11T23:31:27.852+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fd91bcb6:::obj528:head local-lis/les=113/114 n=595 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:fedf1386:::obj565:head v 111'565 2026-02-11T23:31:27.852+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fd91bcb6:::obj528:head local-lis/les=113/114 n=595 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.852+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.852+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:fedf1386:::obj565:head, version: 111'565, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fedf1386:::obj565:head@111'565, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.852+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.852+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.852+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:fe351e27:::obj9:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) pg 0x4ef0000 2026-02-11T23:31:27.852+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:fe351e27:::obj9:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 117/117 lb 1:fe351e27:::obj9:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:fd91bcb6:::obj528:head local-lis/les=113/114 n=595 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.852+0000 7f414f993640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.852+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fd91bcb6:::obj528:head local-lis/les=113/114 n=595 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 117/117 lb 1:fe351e27:::obj9:head) v3 2026-02-11T23:31:27.852+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fd91bcb6:::obj528:head local-lis/les=113/114 n=595 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 117/117 lb 1:fe351e27:::obj9:head) 2026-02-11T23:31:27.852+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.852+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:fe351e27:::obj9:head) v3 finish latency 0.000010 2026-02-11T23:31:27.852+0000 7f414f993640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fe351e27:::obj9:head local-lis/les=113/114 n=596 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.852+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.852+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.852+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fe351e27:::obj9:head local-lis/les=113/114 n=596 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.852+0000 7f414f993640 20 osd.0 117 get_con_osd_cluster to osd.1 from_epoch 117 2026-02-11T23:31:27.852+0000 7f414f993640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:27.852+0000 7f414f993640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:27.852+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e117/117) -- 0x5f75380 con 0x4acb000 2026-02-11T23:31:27.852+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 117/113 [PushReplyOp(1:fedf1386:::obj565:head)]) -- 0x5f1d200 con 0x4acb000 2026-02-11T23:31:27.852+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.852+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1230 ==== MOSDPGPush(1.0 117/113 [PushOp(1:ff38ecdd:::obj155:head, version: 111'155, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ff38ecdd:::obj155:head@111'155, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a3860 con 0x4acb000 2026-02-11T23:31:27.852+0000 7f416abfa640 15 osd.0 117 enqueue_op MOSDPGPush(1.0 117/113 [PushOp(1:ff38ecdd:::obj155:head, version: 111'155, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ff38ecdd:::obj155:head@111'155, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 117 2026-02-11T23:31:27.852+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:ff38ecdd:::obj155:head, version: 111'155, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ff38ecdd:::obj155:head@111'155, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e117) 2026-02-11T23:31:27.852+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.852+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1231 ==== pg_backfill(progress 1.0 e 117/117 lb 1:fedf1386:::obj565:head) ==== 1062+0+0 (unknown 0 0 0) 0x6178c00 con 0x4acb000 2026-02-11T23:31:27.852+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:ff38ecdd:::obj155:head, version: 111'155, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ff38ecdd:::obj155:head@111'155, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) queued 2026-02-11T23:31:27.852+0000 7f416abfa640 15 osd.0 117 enqueue_op pg_backfill(progress 1.0 e 117/117 lb 1:fedf1386:::obj565:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 117 2026-02-11T23:31:27.852+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:fedf1386:::obj565:head) v3) class_id 0 prio 15 cost 0 e117) 2026-02-11T23:31:27.852+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.852+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:ff38ecdd:::obj155:head, version: 111'155, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ff38ecdd:::obj155:head@111'155, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) pg 0x4ef0000 2026-02-11T23:31:27.852+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:ff38ecdd:::obj155:head, version: 111'155, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ff38ecdd:::obj155:head@111'155, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 117/113 [PushOp(1:ff38ecdd:::obj155:head, version: 111'155, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ff38ecdd:::obj155:head@111'155, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:fe351e27:::obj9:head local-lis/les=113/114 n=596 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.852+0000 7f414b98b640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.852+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.852+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fe351e27:::obj9:head local-lis/les=113/114 n=596 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 117/113 [PushOp(1:ff38ecdd:::obj155:head, version: 111'155, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ff38ecdd:::obj155:head@111'155, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.852+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:fedf1386:::obj565:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) queued 2026-02-11T23:31:27.852+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fe351e27:::obj9:head local-lis/les=113/114 n=596 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:ff38ecdd:::obj155:head@111'155, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.852+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fe351e27:::obj9:head local-lis/les=113/114 n=596 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:ff38ecdd:::obj155:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.852+0000 7f414b98b640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fe351e27:::obj9:head local-lis/les=113/114 n=596 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:ff38ecdd:::obj155:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.852+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fe351e27:::obj9:head local-lis/les=113/114 n=596 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:ff38ecdd:::obj155:head 2026-02-11T23:31:27.852+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fe351e27:::obj9:head local-lis/les=113/114 n=596 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:ff38ecdd:::obj155:head v 111'155 2026-02-11T23:31:27.852+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fe351e27:::obj9:head local-lis/les=113/114 n=596 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.852+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.852+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:ff38ecdd:::obj155:head, version: 111'155, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ff38ecdd:::obj155:head@111'155, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.852+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.852+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.852+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:fedf1386:::obj565:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) pg 0x4ef0000 2026-02-11T23:31:27.852+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:fedf1386:::obj565:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 117/117 lb 1:fedf1386:::obj565:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:fe351e27:::obj9:head local-lis/les=113/114 n=596 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.852+0000 7f414f993640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.852+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fe351e27:::obj9:head local-lis/les=113/114 n=596 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 117/117 lb 1:fedf1386:::obj565:head) v3 2026-02-11T23:31:27.852+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fe351e27:::obj9:head local-lis/les=113/114 n=596 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 117/117 lb 1:fedf1386:::obj565:head) 2026-02-11T23:31:27.853+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.853+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:fedf1386:::obj565:head) v3 finish latency 0.000012 2026-02-11T23:31:27.853+0000 7f414f993640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fedf1386:::obj565:head local-lis/les=113/114 n=597 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.853+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.853+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.853+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fedf1386:::obj565:head local-lis/les=113/114 n=597 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.853+0000 7f414f993640 20 osd.0 117 get_con_osd_cluster to osd.1 from_epoch 117 2026-02-11T23:31:27.853+0000 7f414f993640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:27.853+0000 7f414f993640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:27.853+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e117/117) -- 0x5f75500 con 0x4acb000 2026-02-11T23:31:27.853+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 117/113 [PushReplyOp(1:ff38ecdd:::obj155:head)]) -- 0x6000180 con 0x4acb000 2026-02-11T23:31:27.853+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.853+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1232 ==== MOSDPGPush(1.0 117/113 [PushOp(1:ff7b1f36:::obj1:head, version: 111'1, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ff7b1f36:::obj1:head@111'1, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 4999+0+0 (unknown 0 0 0) 0x5617380 con 0x4acb000 2026-02-11T23:31:27.853+0000 7f416abfa640 15 osd.0 117 enqueue_op MOSDPGPush(1.0 117/113 [PushOp(1:ff7b1f36:::obj1:head, version: 111'1, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ff7b1f36:::obj1:head@111'1, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 117 2026-02-11T23:31:27.853+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:ff7b1f36:::obj1:head, version: 111'1, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ff7b1f36:::obj1:head@111'1, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e117) 2026-02-11T23:31:27.853+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.853+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1233 ==== pg_backfill(progress 1.0 e 117/117 lb 1:ff38ecdd:::obj155:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9b180 con 0x4acb000 2026-02-11T23:31:27.853+0000 7f416abfa640 15 osd.0 117 enqueue_op pg_backfill(progress 1.0 e 117/117 lb 1:ff38ecdd:::obj155:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 117 2026-02-11T23:31:27.853+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:ff7b1f36:::obj1:head, version: 111'1, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ff7b1f36:::obj1:head@111'1, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) queued 2026-02-11T23:31:27.853+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:ff38ecdd:::obj155:head) v3) class_id 0 prio 15 cost 0 e117) 2026-02-11T23:31:27.853+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.853+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:ff7b1f36:::obj1:head, version: 111'1, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ff7b1f36:::obj1:head@111'1, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) pg 0x4ef0000 2026-02-11T23:31:27.853+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:ff7b1f36:::obj1:head, version: 111'1, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ff7b1f36:::obj1:head@111'1, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 117/113 [PushOp(1:ff7b1f36:::obj1:head, version: 111'1, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ff7b1f36:::obj1:head@111'1, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:fedf1386:::obj565:head local-lis/les=113/114 n=597 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.853+0000 7f414b98b640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.853+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.853+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fedf1386:::obj565:head local-lis/les=113/114 n=597 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 117/113 [PushOp(1:ff7b1f36:::obj1:head, version: 111'1, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ff7b1f36:::obj1:head@111'1, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.853+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:ff38ecdd:::obj155:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) queued 2026-02-11T23:31:27.853+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fedf1386:::obj565:head local-lis/les=113/114 n=597 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:ff7b1f36:::obj1:head@111'1, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.853+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fedf1386:::obj565:head local-lis/les=113/114 n=597 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:ff7b1f36:::obj1:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.853+0000 7f414b98b640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fedf1386:::obj565:head local-lis/les=113/114 n=597 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:ff7b1f36:::obj1:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.853+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fedf1386:::obj565:head local-lis/les=113/114 n=597 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:ff7b1f36:::obj1:head 2026-02-11T23:31:27.853+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fedf1386:::obj565:head local-lis/les=113/114 n=597 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:ff7b1f36:::obj1:head v 111'1 2026-02-11T23:31:27.853+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fedf1386:::obj565:head local-lis/les=113/114 n=597 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.853+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.853+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:ff7b1f36:::obj1:head, version: 111'1, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ff7b1f36:::obj1:head@111'1, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:27.853+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.853+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.853+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:ff38ecdd:::obj155:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) pg 0x4ef0000 2026-02-11T23:31:27.853+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:ff38ecdd:::obj155:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 117/117 lb 1:ff38ecdd:::obj155:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:fedf1386:::obj565:head local-lis/les=113/114 n=597 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.853+0000 7f414f993640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.853+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fedf1386:::obj565:head local-lis/les=113/114 n=597 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 117/117 lb 1:ff38ecdd:::obj155:head) v3 2026-02-11T23:31:27.853+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:fedf1386:::obj565:head local-lis/les=113/114 n=597 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 117/117 lb 1:ff38ecdd:::obj155:head) 2026-02-11T23:31:27.853+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.853+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:ff38ecdd:::obj155:head) v3 finish latency 0.000009 2026-02-11T23:31:27.853+0000 7f414f993640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ff38ecdd:::obj155:head local-lis/les=113/114 n=598 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.853+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.853+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.853+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ff38ecdd:::obj155:head local-lis/les=113/114 n=598 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.853+0000 7f414f993640 20 osd.0 117 get_con_osd_cluster to osd.1 from_epoch 117 2026-02-11T23:31:27.853+0000 7f414f993640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:27.853+0000 7f414f993640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:27.853+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e117/117) -- 0x5f75680 con 0x4acb000 2026-02-11T23:31:27.853+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 117/113 [PushReplyOp(1:ff7b1f36:::obj1:head)]) -- 0x5f1d500 con 0x4acb000 2026-02-11T23:31:27.853+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.853+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1234 ==== MOSDPGPush(1.0 117/113 [PushOp(1:ffd8ed3d:::obj316:head, version: 111'316, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ffd8ed3d:::obj316:head@111'316, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56caea0 con 0x4acb000 2026-02-11T23:31:27.853+0000 7f416abfa640 15 osd.0 117 enqueue_op MOSDPGPush(1.0 117/113 [PushOp(1:ffd8ed3d:::obj316:head, version: 111'316, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ffd8ed3d:::obj316:head@111'316, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 117 2026-02-11T23:31:27.853+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:ffd8ed3d:::obj316:head, version: 111'316, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ffd8ed3d:::obj316:head@111'316, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e117) 2026-02-11T23:31:27.853+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.853+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1235 ==== pg_backfill(progress 1.0 e 117/117 lb 1:ff7b1f36:::obj1:head) ==== 1060+0+0 (unknown 0 0 0) 0x5d98000 con 0x4acb000 2026-02-11T23:31:27.853+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:ffd8ed3d:::obj316:head, version: 111'316, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ffd8ed3d:::obj316:head@111'316, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) queued 2026-02-11T23:31:27.853+0000 7f416abfa640 15 osd.0 117 enqueue_op pg_backfill(progress 1.0 e 117/117 lb 1:ff7b1f36:::obj1:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 117 2026-02-11T23:31:27.853+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:ff7b1f36:::obj1:head) v3) class_id 0 prio 15 cost 0 e117) 2026-02-11T23:31:27.853+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.853+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 117/113 [PushOp(1:ffd8ed3d:::obj316:head, version: 111'316, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ffd8ed3d:::obj316:head@111'316, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e117) pg 0x4ef0000 2026-02-11T23:31:27.853+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.853+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:ffd8ed3d:::obj316:head, version: 111'316, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ffd8ed3d:::obj316:head@111'316, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 117/113 [PushOp(1:ffd8ed3d:::obj316:head, version: 111'316, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ffd8ed3d:::obj316:head@111'316, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:ff38ecdd:::obj155:head local-lis/les=113/114 n=598 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.853+0000 7f414b98b640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.853+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ff38ecdd:::obj155:head local-lis/les=113/114 n=598 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(1.0 117/113 [PushOp(1:ffd8ed3d:::obj316:head, version: 111'316, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ffd8ed3d:::obj316:head@111'316, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.853+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ff38ecdd:::obj155:head local-lis/les=113/114 n=598 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(1:ffd8ed3d:::obj316:head@111'316, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.853+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.853+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:ff7b1f36:::obj1:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) queued 2026-02-11T23:31:27.853+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ff38ecdd:::obj155:head local-lis/les=113/114 n=598 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 1:ffd8ed3d:::obj316:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.853+0000 7f414b98b640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ff38ecdd:::obj155:head local-lis/les=113/114 n=598 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 1:ffd8ed3d:::obj316:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.853+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ff38ecdd:::obj155:head local-lis/les=113/114 n=598 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 1:ffd8ed3d:::obj316:head 2026-02-11T23:31:27.853+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ff38ecdd:::obj155:head local-lis/les=113/114 n=598 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 1:ffd8ed3d:::obj316:head v 111'316 2026-02-11T23:31:27.853+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ff38ecdd:::obj155:head local-lis/les=113/114 n=598 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.853+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.854+0000 7f414b98b640 10 osd.0 117 dequeue_op MOSDPGPush(1.0 117/113 [PushOp(1:ffd8ed3d:::obj316:head, version: 111'316, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ffd8ed3d:::obj316:head@111'316, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:27.854+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.854+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.854+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 117/117 lb 1:ff7b1f36:::obj1:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) pg 0x4ef0000 2026-02-11T23:31:27.854+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:ff7b1f36:::obj1:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 1.0 e 117/117 lb 1:ff7b1f36:::obj1:head) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:ff38ecdd:::obj155:head local-lis/les=113/114 n=598 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.854+0000 7f414f993640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.854+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ff38ecdd:::obj155:head local-lis/les=113/114 n=598 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 117/117 lb 1:ff7b1f36:::obj1:head) v3 2026-02-11T23:31:27.854+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ff38ecdd:::obj155:head local-lis/les=113/114 n=598 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 117/117 lb 1:ff7b1f36:::obj1:head) 2026-02-11T23:31:27.854+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.854+0000 7f414f993640 10 osd.0 117 dequeue_op pg_backfill(progress 1.0 e 117/117 lb 1:ff7b1f36:::obj1:head) v3 finish latency 0.000008 2026-02-11T23:31:27.854+0000 7f414f993640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ff7b1f36:::obj1:head local-lis/les=113/114 n=599 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.854+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.854+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.854+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ff7b1f36:::obj1:head local-lis/les=113/114 n=599 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.854+0000 7f414f993640 20 osd.0 117 get_con_osd_cluster to osd.1 from_epoch 117 2026-02-11T23:31:27.854+0000 7f414f993640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:27.854+0000 7f414f993640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:27.854+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_trim(1.0 111'600 e117/117) -- 0x5f75800 con 0x4acb000 2026-02-11T23:31:27.854+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- MOSDPGPushReply(1.0 117/113 [PushReplyOp(1:ffd8ed3d:::obj316:head)]) -- 0x5f1c300 con 0x4acb000 2026-02-11T23:31:27.854+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.854+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1236 ==== pg_backfill(finish 1.0 e 117/117 lb MAX) ==== 1098+0+0 (unknown 0 0 0) 0x5d98580 con 0x4acb000 2026-02-11T23:31:27.854+0000 7f416abfa640 15 osd.0 117 enqueue_op pg_backfill(finish 1.0 e 117/117 lb MAX) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 117 2026-02-11T23:31:27.854+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(finish 1.0 e 117/117 lb MAX) v3) class_id 0 prio 15 cost 0 e117) 2026-02-11T23:31:27.854+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.854+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(finish 1.0 e 117/117 lb MAX) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) queued 2026-02-11T23:31:27.854+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.854+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(finish 1.0 e 117/117 lb MAX) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e117) pg 0x4ef0000 2026-02-11T23:31:27.854+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.854+0000 7f414b98b640 10 osd.0 117 dequeue_op pg_backfill(finish 1.0 e 117/117 lb MAX) v3 prio 15 cost 0 latency 0.000020 pg_backfill(finish 1.0 e 117/117 lb MAX) v3 pg pg[1.0( v 111'600 (111'590,111'600] lb 1:ff7b1f36:::obj1:head local-lis/les=113/114 n=599 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.854+0000 7f414b98b640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:27.854+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ff7b1f36:::obj1:head local-lis/les=113/114 n=599 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(finish 1.0 e 117/117 lb MAX) v3 2026-02-11T23:31:27.854+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] lb 1:ff7b1f36:::obj1:head local-lis/les=113/114 n=599 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(finish 1.0 e 117/117 lb MAX) 2026-02-11T23:31:27.854+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_backfill(finish_ack 1.0 e 117/117 lb MIN) -- 0x5f84000 con 0x4acb000 2026-02-11T23:31:27.854+0000 7f414b98b640 15 osd.0 117 enqueue_peering_evt 1.0 epoch_sent: 117 epoch_requested: 117 RecoveryDone 2026-02-11T23:31:27.854+0000 7f414b98b640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 117 epoch_requested: 117 RecoveryDone) class_id 2 prio 255 cost 10 e117) 2026-02-11T23:31:27.854+0000 7f414b98b640 20 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=599 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::update_backfill_progress final 2457600 replaces local 2457600 2026-02-11T23:31:27.854+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.854+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 117 epoch_requested: 117 RecoveryDone) class_id 2immediate prio 255 cost 10 e117) queued 2026-02-11T23:31:27.854+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.854+0000 7f414b98b640 10 osd.0 117 dequeue_op pg_backfill(finish 1.0 e 117/117 lb MAX) v3 finish latency 0.000029 2026-02-11T23:31:27.854+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.854+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.854+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 117 epoch_requested: 117 RecoveryDone) class_id 2immediate prio 255 cost 10 e117) pg 0x4ef0000 2026-02-11T23:31:27.854+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_peering_event: epoch_sent: 117 epoch_requested: 117 RecoveryDone 2026-02-11T23:31:27.854+0000 7f414f993640 5 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] exit Started/ReplicaActive/RepRecovering 0.375336 7 0.000021 2026-02-11T23:31:27.854+0000 7f414f993640 5 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] enter Started/ReplicaActive/RepNotRecovering 2026-02-11T23:31:27.854+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.854+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.854+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.854+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.854+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1237 ==== MBackfillReserve(1.0 RELEASE e117/117) ==== 46+0+0 (unknown 0 0 0) 0x5f1c300 con 0x4acb000 2026-02-11T23:31:27.854+0000 7f416abfa640 15 osd.0 117 enqueue_peering_evt 1.0 epoch_sent: 117 epoch_requested: 117 RemoteReservationCanceled 2026-02-11T23:31:27.854+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 117 epoch_requested: 117 RemoteReservationCanceled) class_id 2 prio 255 cost 10 e117) 2026-02-11T23:31:27.854+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.854+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 117 epoch_requested: 117 RemoteReservationCanceled) class_id 2immediate prio 255 cost 10 e117) queued 2026-02-11T23:31:27.854+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.854+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 117 epoch_requested: 117 RemoteReservationCanceled) class_id 2immediate prio 255 cost 10 e117) pg 0x4ef0000 2026-02-11T23:31:27.854+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_peering_event: epoch_sent: 117 epoch_requested: 117 RemoteReservationCanceled 2026-02-11T23:31:27.854+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.854+0000 7f414b98b640 5 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] exit Started/ReplicaActive/RepNotRecovering 0.000102 1 0.000006 2026-02-11T23:31:27.854+0000 7f414b98b640 5 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] enter Started/ReplicaActive/RepNotRecovering 2026-02-11T23:31:27.854+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.854+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.854+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1238 ==== pg_info2(1.0 1.0 1.0( v 111'600 (111'500,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) e117/117) ==== 1122+0+0 (unknown 0 0 0) 0x5f39100 con 0x4acb000 2026-02-11T23:31:27.854+0000 7f416abfa640 15 osd.0 117 enqueue_peering_evt 1.0 epoch_sent: 117 epoch_requested: 117 MInfoRec from 1 info: 1.0( v 111'600 (111'500,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) pg_lease(ru 84.143806458s ub 84.143806458s int 16.000000000s) 2026-02-11T23:31:27.854+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 117 epoch_requested: 117 MInfoRec from 1 info: 1.0( v 111'600 (111'500,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) pg_lease(ru 84.143806458s ub 84.143806458s int 16.000000000s)) class_id 2 prio 255 cost 10 e117) 2026-02-11T23:31:27.854+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.854+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 117 epoch_requested: 117 MInfoRec from 1 info: 1.0( v 111'600 (111'500,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) pg_lease(ru 84.143806458s ub 84.143806458s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e117) queued 2026-02-11T23:31:27.854+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.854+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 117 epoch_requested: 117 MInfoRec from 1 info: 1.0( v 111'600 (111'500,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) pg_lease(ru 84.143806458s ub 84.143806458s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e117) pg 0x4ef0000 2026-02-11T23:31:27.854+0000 7f414f993640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] do_peering_event: epoch_sent: 117 epoch_requested: 117 MInfoRec from 1 info: 1.0( v 111'600 (111'500,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) pg_lease(ru 84.143806458s ub 84.143806458s int 16.000000000s) 2026-02-11T23:31:27.854+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.854+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.854+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.930+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 54 ==== osd_map(117..118 src has 1..118) ==== 610+0+0 (crc 0 0 0) 0x59556c0 con 0x4acbc00 2026-02-11T23:31:27.930+0000 7f415cf02640 20 osd.0 117 OSD::ms_dispatch: osd_map(117..118 src has 1..118) 2026-02-11T23:31:27.930+0000 7f415cf02640 20 osd.0 117 _dispatch 0x59556c0 osd_map(117..118 src has 1..118) 2026-02-11T23:31:27.930+0000 7f415cf02640 3 osd.0 117 handle_osd_map epochs [117,118], i have 117, src has [1,118] 2026-02-11T23:31:27.930+0000 7f415cf02640 10 osd.0 117 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:31:27.930+0000 7f415cf02640 10 osd.0 117 handle_osd_map got inc map for epoch 118 2026-02-11T23:31:27.930+0000 7f415cf02640 20 osd.0 117 got_full_map 118, nothing requested 2026-02-11T23:31:27.930+0000 7f415cf02640 20 osd.0 117 trim_maps: min=1 oldest_map=1 2026-02-11T23:31:27.930+0000 7f415cf02640 20 osd.0 117 handle_osd_map pg_num_history pg_num_history(e118 pg_nums {1={27=1},2={34=1},3={42=1},4={49=1},5={56=1},6={64=1},7={70=1},8={77=1},9={85=1},10={92=1}} deleted_pools 99,2,100,3,101,5,102,6,103,7,104,8,105,9,106,10) 2026-02-11T23:31:27.930+0000 7f415cf02640 10 snap_mapper record_purged_snaps purged_snaps {118={}} 2026-02-11T23:31:27.930+0000 7f415cf02640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:31:27.930+0000 7f415cf02640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e118 maps [1~118] lci=[5,118] tlb=1) 2026-02-11T23:31:27.930+0000 7f41591a6640 10 osd.0 117 _committed_osd_maps 118..118 2026-02-11T23:31:27.930+0000 7f41591a6640 10 osd.0 117 advance to epoch 118 (<= last 118 <= newest_map 118) 2026-02-11T23:31:27.930+0000 7f41591a6640 20 osd.0 118 consume_map version 118 2026-02-11T23:31:27.930+0000 7f41591a6640 20 osd.0 117 await_reserved_maps epoch:118 2026-02-11T23:31:27.930+0000 7f41591a6640 20 osd.0 117 await_reserved_maps done 0 2026-02-11T23:31:27.930+0000 7f41591a6640 20 osd.0 118 consume_map 118 -- publish done 2026-02-11T23:31:27.930+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:31:27.930+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x5efd340 2026-02-11T23:31:27.930+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot 4.0 has no pg and waiting_for_split 2026-02-11T23:31:27.930+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3701340 2026-02-11T23:31:27.930+0000 7f41591a6640 20 osd.0 118 identify_splits_and_merges 1.0 e117 to e118 2026-02-11T23:31:27.930+0000 7f41591a6640 20 osd.0 118 identify_splits_and_merges 1.0 e117 to e118 pg_nums {27=1} 2026-02-11T23:31:27.930+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:27.930+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:27.930+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:27.930+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:27.930+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:27.930+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:27.930+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:27.930+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:27.930+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:27.930+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:27.930+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:27.930+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:27.930+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:27.930+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:27.930+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:27.930+0000 7f41591a6640 20 osd.0 118 prune_pg_created 2026-02-11T23:31:27.930+0000 7f41591a6640 10 osd.0:0.consume_map 118 (was 117) 2026-02-11T23:31:27.930+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:31:27.930+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 empty, pruning 2026-02-11T23:31:27.930+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:31:27.930+0000 7f41591a6640 10 osd.0:1.consume_map 118 (was 117) 2026-02-11T23:31:27.930+0000 7f41591a6640 10 osd.0:2.consume_map 118 (was 117) 2026-02-11T23:31:27.930+0000 7f41591a6640 10 osd.0:3.consume_map 118 (was 117) 2026-02-11T23:31:27.930+0000 7f41591a6640 10 osd.0:4.consume_map 118 (was 117) 2026-02-11T23:31:27.930+0000 7f41591a6640 10 osd.0:5.consume_map 118 (was 117) 2026-02-11T23:31:27.930+0000 7f41591a6640 10 osd.0:6.consume_map 118 (was 117) 2026-02-11T23:31:27.930+0000 7f41591a6640 10 osd.0:7.consume_map 118 (was 117) 2026-02-11T23:31:27.930+0000 7f41591a6640 10 osd.0 118 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:31:27.930+0000 7f41591a6640 15 osd.0 118 enqueue_peering_evt 1.0 epoch_sent: 118 epoch_requested: 118 NullEvt 2026-02-11T23:31:27.930+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 118 epoch_requested: 118 NullEvt) class_id 2 prio 255 cost 10 e118) 2026-02-11T23:31:27.930+0000 7f41591a6640 7 osd.0 118 activate_map version 118 2026-02-11T23:31:27.930+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.930+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 118 epoch_requested: 118 NullEvt) class_id 2immediate prio 255 cost 10 e118) queued 2026-02-11T23:31:27.930+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.930+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 118 epoch_requested: 118 NullEvt) class_id 2immediate prio 255 cost 10 e118) pg 0x4ef0000 2026-02-11T23:31:27.930+0000 7f4150194640 20 osd.0 118 agent_entry empty queue 2026-02-11T23:31:27.930+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.930+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] handle_advance_map: 118 2026-02-11T23:31:27.930+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 1.0 117 -> 118 2026-02-11T23:31:27.930+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:31:27.930+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] state: Started advmap 2026-02-11T23:31:27.930+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::should_restart_peering new interval newup [1,0] newacting [1,0] 2026-02-11T23:31:27.930+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] state: should_restart_peering, transitioning to Reset 2026-02-11T23:31:27.930+0000 7f414b98b640 5 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] exit Started/ReplicaActive/RepNotRecovering 0.076420 2 0.000004 2026-02-11T23:31:27.930+0000 7f414b98b640 5 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] exit Started/ReplicaActive 0.454756 0 0.000000 2026-02-11T23:31:27.930+0000 7f414b98b640 5 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] exit Started 0.455082 0 0.000000 2026-02-11T23:31:27.930+0000 7f414b98b640 5 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] enter Reset 2026-02-11T23:31:27.930+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=113 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::set_last_peering_reset set_last_peering_reset 118 2026-02-11T23:31:27.930+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=118 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::set_last_peering_reset Clearing blocked outgoing recovery messages 2026-02-11T23:31:27.930+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=118 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::set_last_peering_reset Not blocking outgoing recovery messages 2026-02-11T23:31:27.930+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=118 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] state: Reset advmap 2026-02-11T23:31:27.930+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=118 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::should_restart_peering new interval newup [1,0] newacting [1,0] 2026-02-11T23:31:27.930+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=118 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] state: should restart peering, calling start_peering_interval again 2026-02-11T23:31:27.930+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0]/[1] r=-1 lpr=118 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::set_last_peering_reset set_last_peering_reset 118 2026-02-11T23:31:27.930+0000 7f414b98b640 15 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0] r=-1 lpr=118 pi=[107,113)/1 pct=0'0 crt=111'600 active mbc={}] clear_stats 2026-02-11T23:31:27.930+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0] r=1 lpr=118 pi=[107,118)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::start_peering_interval : check_new_interval output: check_new_interval interval(113-117 up [1, 0](1) acting [1](1)) up_thru 113 up_from 10 last_epoch_clean 109 interval(113-117 up [1, 0](1) acting [1](1) maybe_went_rw) : primary up 10-113 includes interval 2026-02-11T23:31:27.930+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=113) [1,0] r=1 lpr=118 pi=[107,118)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::start_peering_interval noting past ([107,117] all_participants={0, 1} intervals=[([113,117] acting={1})]) 2026-02-11T23:31:27.930+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::on_new_interval 2026-02-11T23:31:27.930+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::on_new_interval upacting_features 0x3f0ffffffffdffff from [1,0]+[1,0] 2026-02-11T23:31:27.930+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::on_new_interval checking missing set deletes flag. missing = missing(0 may_include_deletes = 1) 2026-02-11T23:31:27.930+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::init_hb_stamps now [0x56849a0] 2026-02-11T23:31:27.930+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::on_new_interval prior_readable_until_ub 0.000000000s (mnow 70.607711792s + 0.000000000s) 2026-02-11T23:31:27.931+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 pct=0'0 crt=111'600 active mbc={}] cancel_recovery 2026-02-11T23:31:27.931+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 pct=0'0 crt=111'600 active mbc={}] clear_recovery_state 2026-02-11T23:31:27.931+0000 7f414b98b640 15 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 pct=0'0 crt=111'600 active mbc={}] _clear_recovery_state 2026-02-11T23:31:27.931+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 pct=0'0 crt=111'600 active mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:31:27.931+0000 7f414b98b640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:31:27.931+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 pct=0'0 crt=111'600 active mbc={}] scrubber: FSM: ReplicaActive::exit() 2026-02-11T23:31:27.931+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 pct=0'0 crt=111'600 active mbc={}] scrubber: FSM: ReplicaActive::clear_remote_reservation(): pending_reservation_nonce 0, reservation_granted false 2026-02-11T23:31:27.931+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 pct=0'0 crt=111'600 active mbc={}] scrubber: FSM: -- state -->> NotActive 2026-02-11T23:31:27.931+0000 7f414b98b640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:31:27.931+0000 7f414b98b640 1 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::start_peering_interval up [1,0] -> [1,0], acting [1] -> [1,0], acting_primary 1 -> 1, up_primary 1 -> 1, role -1 -> 1, features acting 4544132024016699391 upacting 4544132024016699391 2026-02-11T23:31:27.931+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 unknown mbc={}] PeeringState::clear_primary_state clear_primary_state 2026-02-11T23:31:27.931+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 unknown mbc={}] clear_primary_state 2026-02-11T23:31:27.931+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 unknown mbc={}] release_backoffs [1:00000000::::0,MAX) 2026-02-11T23:31:27.931+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 unknown mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:31:27.931+0000 7f414b98b640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:31:27.931+0000 7f414b98b640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:31:27.931+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 unknown mbc={}] agent_stop 2026-02-11T23:31:27.931+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 unknown mbc={}] on_change 2026-02-11T23:31:27.931+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 unknown mbc={}] cancel_copy_ops 2026-02-11T23:31:27.931+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 unknown mbc={}] cancel_flush_ops 2026-02-11T23:31:27.931+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 unknown mbc={}] cancel_proxy_ops 2026-02-11T23:31:27.931+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 unknown mbc={}] cancel_manifest_ops 2026-02-11T23:31:27.931+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 unknown mbc={}] cancel_cls_gather_ops 2026-02-11T23:31:27.931+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 unknown mbc={}] on_change_cleanup 2026-02-11T23:31:27.931+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 unknown mbc={}] on_change 2026-02-11T23:31:27.931+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 unknown mbc={}] exit NotTrimming 2026-02-11T23:31:27.931+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 unknown mbc={}] enter NotTrimming 2026-02-11T23:31:27.931+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 unknown NOTIFY mbc={}] plpg_on_role_change 2026-02-11T23:31:27.931+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 unknown NOTIFY mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:31:27.931+0000 7f414b98b640 20 osd.0 118 advance_pg new pool opts old pool opts 2026-02-11T23:31:27.931+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 unknown NOTIFY mbc={}] handle_activate_map: epoch range: 118..118 2026-02-11T23:31:27.931+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 unknown NOTIFY mbc={}] PeeringState::activate_map 2026-02-11T23:31:27.931+0000 7f414b98b640 5 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 unknown NOTIFY mbc={}] exit Reset 0.000110 1 0.000115 2026-02-11T23:31:27.931+0000 7f414b98b640 5 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 unknown NOTIFY mbc={}] enter Started 2026-02-11T23:31:27.931+0000 7f414b98b640 5 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 unknown NOTIFY mbc={}] enter Start 2026-02-11T23:31:27.931+0000 7f414b98b640 1 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 unknown NOTIFY mbc={}] state: transitioning to Stray 2026-02-11T23:31:27.931+0000 7f414b98b640 5 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 unknown NOTIFY mbc={}] exit Start 0.000002 0 0.000000 2026-02-11T23:31:27.931+0000 7f414b98b640 5 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 unknown NOTIFY mbc={}] enter Started/Stray 2026-02-11T23:31:27.931+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 unknown NOTIFY mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 117 while current is 118 2026-02-11T23:31:27.931+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.931+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 unknown NOTIFY mbc={}] do_peering_event: epoch_sent: 118 epoch_requested: 118 NullEvt 2026-02-11T23:31:27.931+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.931+0000 7f414b98b640 20 osd.0 118 get_con_osd_cluster to osd.1 from_epoch 118 2026-02-11T23:31:27.931+0000 7f414b98b640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.931+0000 7f414b98b640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.931+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6812/3043114013 map epoch 117 -> 118 (shared) 2026-02-11T23:31:27.931+0000 7f414b98b640 10 osd.0 118 send_incremental_map epoch range: (118, 118] to v2:127.0.0.1:6812/3043114013 2026-02-11T23:31:27.931+0000 7f414b98b640 10 osd.0 118 add_map_inc_bl 118 281 bytes 2026-02-11T23:31:27.931+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- osd_map(118..118 src has 1..118) -- 0x5955c00 con 0x4acb000 2026-02-11T23:31:27.931+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_notify2(1.0 1.0 (query:118 sent:118 1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) ([107,117] all_participants={0, 1} intervals=[([113,117] acting={1})])) e118/118) -- 0x5f79100 con 0x4acb000 2026-02-11T23:31:27.931+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.931+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.931+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.931+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.931+0000 7f415aefe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1239 ==== osd_map(118..118 src has 1..118) ==== 325+0+0 (unknown 0 0 0) 0x5955c00 con 0x4acb000 2026-02-11T23:31:27.931+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1240 ==== pg_query2(1.0 1.0 query(info 0'0 epoch_sent 118) e118/118) ==== 158+0+0 (unknown 0 0 0) 0x5f614a0 con 0x4acb000 2026-02-11T23:31:27.931+0000 7f415aefe640 20 osd.0 118 OSD::ms_dispatch: osd_map(118..118 src has 1..118) 2026-02-11T23:31:27.931+0000 7f415aefe640 20 osd.0 118 _dispatch 0x5955c00 osd_map(118..118 src has 1..118) 2026-02-11T23:31:27.931+0000 7f415aefe640 3 osd.0 118 handle_osd_map epochs [118,118], i have 118, src has [1,118] 2026-02-11T23:31:27.931+0000 7f415aefe640 10 osd.0 118 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:31:27.931+0000 7f415aefe640 10 osd.0 118 no new maps here, dropping 2026-02-11T23:31:27.931+0000 7f416abfa640 15 osd.0 118 enqueue_peering_evt 1.0 epoch_sent: 118 epoch_requested: 118 MQuery 1.0 from 1 query_epoch 118 query: query(info 0'0 epoch_sent 118) 2026-02-11T23:31:27.931+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 118 epoch_requested: 118 MQuery 1.0 from 1 query_epoch 118 query: query(info 0'0 epoch_sent 118)) class_id 2 prio 255 cost 10 e118) 2026-02-11T23:31:27.931+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.931+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 118 epoch_requested: 118 MQuery 1.0 from 1 query_epoch 118 query: query(info 0'0 epoch_sent 118)) class_id 2immediate prio 255 cost 10 e118) queued 2026-02-11T23:31:27.931+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.931+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 118 epoch_requested: 118 MQuery 1.0 from 1 query_epoch 118 query: query(info 0'0 epoch_sent 118)) class_id 2immediate prio 255 cost 10 e118) pg 0x4ef0000 2026-02-11T23:31:27.931+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.931+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 unknown NOTIFY mbc={}] do_peering_event: epoch_sent: 118 epoch_requested: 118 MQuery 1.0 from 1 query_epoch 118 query: query(info 0'0 epoch_sent 118) 2026-02-11T23:31:27.931+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 unknown NOTIFY mbc={}] PeeringState::fulfill_info sending info 2026-02-11T23:31:27.931+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.931+0000 7f414b98b640 20 osd.0 118 get_con_osd_cluster to osd.1 from_epoch 118 2026-02-11T23:31:27.931+0000 7f414b98b640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.931+0000 7f414b98b640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.931+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.931+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_notify2(1.0 1.0 (query:118 sent:118 1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) ([107,117] all_participants={0, 1} intervals=[([113,117] acting={1})])) e118/118) -- 0x5f79800 con 0x4acb000 2026-02-11T23:31:27.931+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.932+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 34 ==== PGlog(4.0 log log((111'590,111'600], crt=111'600) pi ([109,116] all_participants={0, 4} intervals=[([109,114] acting={4})]) pg_lease(ru 76.771820068s ub 76.771820068s int 16.000000000s) e118/117) ==== 39009+0+0 (unknown 0 0 0) 0x4a1c000 con 0x507ac00 2026-02-11T23:31:27.932+0000 7f416abfa640 15 osd.0 118 enqueue_peering_evt 4.0 epoch_sent: 118 epoch_requested: 117 MLogRec from 4 log log((111'590,111'600], crt=111'600) pi ([109,116] all_participants={0, 4} intervals=[([109,114] acting={4})]) pg_lease(ru 76.771820068s ub 76.771820068s int 16.000000000s) +create_info 2026-02-11T23:31:27.932+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 118 epoch_requested: 117 MLogRec from 4 log log((111'590,111'600], crt=111'600) pi ([109,116] all_participants={0, 4} intervals=[([109,114] acting={4})]) pg_lease(ru 76.771820068s ub 76.771820068s int 16.000000000s) +create_info) class_id 2 prio 255 cost 10 e118) 2026-02-11T23:31:27.932+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 (new) to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.932+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 118 epoch_requested: 117 MLogRec from 4 log log((111'590,111'600], crt=111'600) pi ([109,116] all_participants={0, 4} intervals=[([109,114] acting={4})]) pg_lease(ru 76.771820068s ub 76.771820068s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e118) queued 2026-02-11T23:31:27.932+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.932+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 118 epoch_requested: 117 MLogRec from 4 log log((111'590,111'600], crt=111'600) pi ([109,116] all_participants={0, 4} intervals=[([109,114] acting={4})]) pg_lease(ru 76.771820068s ub 76.771820068s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e118) pg 0 2026-02-11T23:31:27.932+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 no pg, should create on OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 118 epoch_requested: 117 MLogRec from 4 log log((111'590,111'600], crt=111'600) pi ([109,116] all_participants={0, 4} intervals=[([109,114] acting={4})]) pg_lease(ru 76.771820068s ub 76.771820068s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e118) 2026-02-11T23:31:27.932+0000 7f414b98b640 10 osd.0 118 _make_pg 4.0 2026-02-11T23:31:27.932+0000 7f414b98b640 10 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:4, shard:-1, shard_prefix: , prefixes: {}] SnapMapper 2026-02-11T23:31:27.932+0000 7f414b98b640 20 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:4, shard:-1, shard_prefix: , prefixes: {}] update_bits new_bits: 0 2026-02-11T23:31:27.932+0000 7f414b98b640 20 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:4, shard:-1, shard_prefix: , prefixes: {"0000000000000004."}] update_bits prefix updated 2026-02-11T23:31:27.932+0000 7f414b98b640 10 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:4, shard:-1, shard_prefix: , prefixes: {"0000000000000004."}] reset_prefix_itr::from <0> to ::update_bits 2026-02-11T23:31:27.932+0000 7f414b98b640 5 osd.0 pg_epoch: 117 pg[4.0(unlocked)] enter Initial 2026-02-11T23:31:27.932+0000 7f414b98b640 20 osd.0 pg_epoch: 117 pg[4.0(unlocked)] enter NotTrimming 2026-02-11T23:31:27.932+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[4.0(unlocked)] scrubber: FSM: -- state -->> NotActive 2026-02-11T23:31:27.932+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[4.0( DNE empty local-lis/les=0/0 n=0 ec=0/0 lis/c=0/0 les/c/f=0/0/0 sis=0) [] r=-1 lpr=0 crt=0'0 unknown mbc={}] PeeringState::init init role -1 up [4,0] acting [4] history ec=49/49 lis/c=109/109 les/c/f=111/111/0 sis=117 past_intervals ([109,116] all_participants={0, 4} intervals=[([109,114] acting={4})]) 2026-02-11T23:31:27.932+0000 7f414b98b640 20 osd.0 pg_epoch: 117 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=109/109 les/c/f=111/111/0 sis=117) [4,0]/[4] r=-1 lpr=0 pi=[109,117)/1 crt=0'0 unknown mbc={}] PeeringState::on_new_interval 2026-02-11T23:31:27.932+0000 7f414b98b640 20 osd.0 pg_epoch: 117 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=109/109 les/c/f=111/111/0 sis=117) [4,0]/[4] r=-1 lpr=0 pi=[109,117)/1 crt=0'0 unknown mbc={}] PeeringState::on_new_interval upacting_features 0x3f0ffffffffdffff from [4]+[4,0] 2026-02-11T23:31:27.932+0000 7f414b98b640 20 osd.0 pg_epoch: 117 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=109/109 les/c/f=111/111/0 sis=117) [4,0]/[4] r=-1 lpr=0 pi=[109,117)/1 crt=0'0 unknown mbc={}] PeeringState::on_new_interval checking missing set deletes flag. missing = missing(0 may_include_deletes = 1) 2026-02-11T23:31:27.932+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=109/109 les/c/f=111/111/0 sis=117) [4,0]/[4] r=-1 lpr=0 pi=[109,117)/1 crt=0'0 unknown mbc={}] PeeringState::init_hb_stamps now [] 2026-02-11T23:31:27.932+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=109/109 les/c/f=111/111/0 sis=117) [4,0]/[4] r=-1 lpr=0 pi=[109,117)/1 crt=0'0 unknown mbc={}] PeeringState::on_new_interval prior_readable_until_ub 0.000000000s (mnow 70.608985901s + 0.000000000s) 2026-02-11T23:31:27.932+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=109/109 les/c/f=111/111/0 sis=117) [4,0]/[4] r=-1 lpr=0 pi=[109,117)/1 crt=0'0 unknown mbc={}] cancel_recovery 2026-02-11T23:31:27.932+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=109/109 les/c/f=111/111/0 sis=117) [4,0]/[4] r=-1 lpr=0 pi=[109,117)/1 crt=0'0 unknown mbc={}] clear_recovery_state 2026-02-11T23:31:27.932+0000 7f414b98b640 15 osd.0 pg_epoch: 117 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=109/109 les/c/f=111/111/0 sis=117) [4,0]/[4] r=-1 lpr=0 pi=[109,117)/1 crt=0'0 unknown mbc={}] _clear_recovery_state 2026-02-11T23:31:27.932+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=109/109 les/c/f=111/111/0 sis=117) [4,0]/[4] r=-1 lpr=0 pi=[109,117)/1 crt=0'0 unknown mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:31:27.932+0000 7f414b98b640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:31:27.932+0000 7f414b98b640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:31:27.932+0000 7f414b98b640 6 write_log_and_missing with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, writeout_from: 4294967295'18446744073709551615, trimmed: , trimmed_dups: , clear_divergent_priors: 0 2026-02-11T23:31:27.932+0000 7f414b98b640 10 _write_log_and_missing clearing up to 0'0 dirty_to_dups=0'0 dirty_from_dups=4294967295'18446744073709551615 write_from_dups=4294967295'18446744073709551615 trimmed_dups.size()=0 2026-02-11T23:31:27.932+0000 7f414b98b640 10 _write_log_and_missing going to encode log.dups.size()=0 2026-02-11T23:31:27.932+0000 7f414b98b640 10 _write_log_and_missing 1st round encoded log.dups.size()=0 2026-02-11T23:31:27.932+0000 7f414b98b640 10 _write_log_and_missing 2st round encoded log.dups.size()=0 2026-02-11T23:31:27.932+0000 7f414b98b640 10 end of _write_log_and_missing 2026-02-11T23:31:27.932+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=109/109 les/c/f=111/111/0 sis=117) [4,0]/[4] r=-1 lpr=0 pi=[109,117)/1 crt=0'0 unknown mbc={}] handle_initialize 2026-02-11T23:31:27.932+0000 7f414b98b640 5 osd.0 pg_epoch: 117 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=109/109 les/c/f=111/111/0 sis=117) [4,0]/[4] r=-1 lpr=0 pi=[109,117)/1 crt=0'0 unknown mbc={}] exit Initial 0.000047 0 0.000000 2026-02-11T23:31:27.932+0000 7f414b98b640 5 osd.0 pg_epoch: 117 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=109/109 les/c/f=111/111/0 sis=117) [4,0]/[4] r=-1 lpr=0 pi=[109,117)/1 crt=0'0 unknown mbc={}] enter Reset 2026-02-11T23:31:27.932+0000 7f414b98b640 20 osd.0 pg_epoch: 117 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=109/109 les/c/f=111/111/0 sis=117) [4,0]/[4] r=-1 lpr=0 pi=[109,117)/1 crt=0'0 unknown mbc={}] PeeringState::set_last_peering_reset set_last_peering_reset 117 2026-02-11T23:31:27.932+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=109/109 les/c/f=111/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 crt=0'0 unknown mbc={}] PeeringState::set_last_peering_reset Clearing blocked outgoing recovery messages 2026-02-11T23:31:27.932+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=109/109 les/c/f=111/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 crt=0'0 unknown mbc={}] PeeringState::set_last_peering_reset Not blocking outgoing recovery messages 2026-02-11T23:31:27.932+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=109/109 les/c/f=111/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 crt=0'0 unknown mbc={}] handle_activate_map: epoch range: 117..117 2026-02-11T23:31:27.932+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=109/109 les/c/f=111/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 crt=0'0 unknown mbc={}] PeeringState::activate_map 2026-02-11T23:31:27.932+0000 7f414b98b640 5 osd.0 pg_epoch: 117 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=109/109 les/c/f=111/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 crt=0'0 unknown mbc={}] exit Reset 0.000008 1 0.000008 2026-02-11T23:31:27.932+0000 7f414b98b640 5 osd.0 pg_epoch: 117 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=109/109 les/c/f=111/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 crt=0'0 unknown mbc={}] enter Started 2026-02-11T23:31:27.932+0000 7f414b98b640 5 osd.0 pg_epoch: 117 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=109/109 les/c/f=111/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 crt=0'0 unknown mbc={}] enter Start 2026-02-11T23:31:27.932+0000 7f414b98b640 1 osd.0 pg_epoch: 117 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=109/109 les/c/f=111/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 crt=0'0 unknown mbc={}] state: transitioning to Stray 2026-02-11T23:31:27.932+0000 7f414b98b640 5 osd.0 pg_epoch: 117 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=109/109 les/c/f=111/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 crt=0'0 unknown mbc={}] exit Start 0.000001 0 0.000000 2026-02-11T23:31:27.932+0000 7f414b98b640 5 osd.0 pg_epoch: 117 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=109/109 les/c/f=111/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 crt=0'0 unknown mbc={}] enter Started/Stray 2026-02-11T23:31:27.932+0000 7f414b98b640 20 osd.0 pg_epoch: 117 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=109/109 les/c/f=111/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 crt=0'0 unknown mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 117 while current is 117 2026-02-11T23:31:27.932+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.932+0000 7f414b98b640 10 osd.0 118 handle_pg_create_info new pg pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=109/109 les/c/f=111/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 crt=0'0 unknown mbc={}] 2026-02-11T23:31:27.932+0000 7f414b98b640 10 osd.0:0._attach_pg 4.0 0x5978000 2026-02-11T23:31:27.932+0000 7f414b98b640 20 osd.0:0._wake_pg_slot _wake_pg_slot 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.932+0000 7f414b98b640 20 osd.0 118 identify_splits_and_merges 4.0 e117 to e118 2026-02-11T23:31:27.932+0000 7f414b98b640 20 osd.0 118 identify_splits_and_merges 4.0 e117 to e118 pg_nums {49=1} 2026-02-11T23:31:27.932+0000 7f414b98b640 10 osd.0:0._prime_splits 2026-02-11T23:31:27.932+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=109/109 les/c/f=111/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 crt=0'0 unknown mbc={}] handle_advance_map: 118 2026-02-11T23:31:27.932+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.932+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 4.0 117 -> 118 2026-02-11T23:31:27.932+0000 7f414b98b640 10 osd.0 pg_epoch: 117 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=109/109 les/c/f=111/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 crt=0'0 unknown mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4] -- 4/4 2026-02-11T23:31:27.932+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=109/109 les/c/f=111/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 crt=0'0 unknown mbc={}] state: Started advmap 2026-02-11T23:31:27.932+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=109/109 les/c/f=111/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 crt=0'0 unknown mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:31:27.932+0000 7f414b98b640 20 osd.0 118 advance_pg new pool opts old pool opts 2026-02-11T23:31:27.932+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=109/109 les/c/f=111/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 crt=0'0 unknown mbc={}] handle_activate_map: epoch range: 118..118 2026-02-11T23:31:27.932+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=109/109 les/c/f=111/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 crt=0'0 unknown mbc={}] PeeringState::activate_map 2026-02-11T23:31:27.932+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=109/109 les/c/f=111/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 crt=0'0 unknown mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 117 while current is 118 2026-02-11T23:31:27.932+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.932+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=109/109 les/c/f=111/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 crt=0'0 unknown mbc={}] do_peering_event: epoch_sent: 118 epoch_requested: 117 MLogRec from 4 log log((111'590,111'600], crt=111'600) pi ([109,116] all_participants={0, 4} intervals=[([109,114] acting={4})]) pg_lease(ru 76.771820068s ub 76.771820068s int 16.000000000s) +create_info 2026-02-11T23:31:27.932+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( empty local-lis/les=0/0 n=0 ec=49/49 lis/c=109/109 les/c/f=111/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 crt=0'0 unknown mbc={}] state: got info+log from osd.4 4.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=117/118 n=0 ec=49/49 lis/c=109/109 les/c/f=111/111/0 sis=117) log((111'590,111'600], crt=111'600) 2026-02-11T23:31:27.932+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=117/118 n=0 ec=49/49 lis/c=109/109 les/c/f=111/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 crt=111'600 lcod 0'0 unknown mbc={}] PeeringState::proc_lease no-op, !nonprimary 2026-02-11T23:31:27.932+0000 7f414b98b640 5 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=117/118 n=0 ec=49/49 lis/c=109/109 les/c/f=111/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 crt=111'600 lcod 0'0 unknown mbc={}] exit Started/Stray 0.000137 3 0.000009 2026-02-11T23:31:27.932+0000 7f414b98b640 5 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=117/118 n=0 ec=49/49 lis/c=109/109 les/c/f=111/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 crt=111'600 lcod 0'0 unknown mbc={}] enter Started/ReplicaActive 2026-02-11T23:31:27.932+0000 7f414b98b640 5 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=117/118 n=0 ec=49/49 lis/c=109/109 les/c/f=111/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 crt=111'600 lcod 0'0 unknown mbc={}] enter Started/ReplicaActive/RepNotRecovering 2026-02-11T23:31:27.932+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=117/118 n=0 ec=49/49 lis/c=109/109 les/c/f=111/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 crt=111'600 lcod 0'0 unknown mbc={}] state: In ReplicaActive, about to call activate 2026-02-11T23:31:27.932+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=117/118 n=0 ec=49/49 lis/c=109/109 les/c/f=111/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 crt=111'600 lcod 0'0 unknown mbc={}] PeeringState::activate activate - no missing, moving last_complete 111'600 -> 111'600 2026-02-11T23:31:27.932+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=117/118 n=0 ec=49/49 lis/c=109/109 les/c/f=111/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 crt=111'600 lcod 0'0 unknown mbc={}] scrubber: on_replica_activate 2026-02-11T23:31:27.932+0000 7f414b98b640 20 scrubberFSM event: --vvvv---- ReplicaActivate 2026-02-11T23:31:27.932+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=117/118 n=0 ec=49/49 lis/c=109/109 les/c/f=111/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 crt=111'600 lcod 0'0 unknown mbc={}] scrubber: FSM: -- state -->> ReplicaActive 2026-02-11T23:31:27.932+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=117/118 n=0 ec=49/49 lis/c=109/109 les/c/f=111/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 crt=111'600 lcod 0'0 unknown mbc={}] scrubber: FSM: -- state -->> ReplicaActive/ReplicaIdle 2026-02-11T23:31:27.932+0000 7f414b98b640 20 scrubberFSM event: --^^^^---- ReplicaActivate 2026-02-11T23:31:27.932+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=117/118 n=0 ec=49/49 lis/c=109/109 les/c/f=111/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 crt=111'600 lcod 0'0 unknown mbc={}] state: Activate Finished 2026-02-11T23:31:27.932+0000 7f414b98b640 6 write_log_and_missing with: dirty_to: 4294967295'18446744073709551615, dirty_from: 4294967295'18446744073709551615, writeout_from: 4294967295'18446744073709551615, trimmed: , trimmed_dups: , clear_divergent_priors: 0 2026-02-11T23:31:27.932+0000 7f414b98b640 10 _write_log_and_missing clearing up to 4294967295'18446744073709551615 dirty_to_dups=4294967295'18446744073709551615 dirty_from_dups=4294967295'18446744073709551615 write_from_dups=4294967295'18446744073709551615 trimmed_dups.size()=0 2026-02-11T23:31:27.932+0000 7f414b98b640 10 _write_log_and_missing remove dups min=dup_0000000000.00000000000000000000 to dirty_to_dup=dup_4294967295.18446744073709551615 2026-02-11T23:31:27.932+0000 7f414b98b640 10 _write_log_and_missing going to encode log.dups.size()=590 2026-02-11T23:31:27.932+0000 7f414b98b640 10 _write_log_and_missing 1st round encoded log.dups.size()=590 2026-02-11T23:31:27.932+0000 7f414b98b640 10 _write_log_and_missing 2st round encoded log.dups.size()=590 2026-02-11T23:31:27.932+0000 7f414b98b640 10 end of _write_log_and_missing 2026-02-11T23:31:27.933+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.933+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.933+0000 7f414f993640 15 osd.0 118 enqueue_peering_evt 4.0 epoch_sent: 118 epoch_requested: 118 ActivateCommitted from 118 processed at 118 2026-02-11T23:31:27.933+0000 7f414f993640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 118 epoch_requested: 118 ActivateCommitted from 118 processed at 118) class_id 2 prio 255 cost 10 e118) 2026-02-11T23:31:27.933+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.933+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.933+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 118 epoch_requested: 118 ActivateCommitted from 118 processed at 118) class_id 2immediate prio 255 cost 10 e118) queued 2026-02-11T23:31:27.933+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.933+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 118 epoch_requested: 118 ActivateCommitted from 118 processed at 118) class_id 2immediate prio 255 cost 10 e118) pg 0x5978000 2026-02-11T23:31:27.933+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.933+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=117/118 n=0 ec=49/49 lis/c=109/109 les/c/f=111/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 crt=111'600 lcod 0'0 unknown mbc={}] do_peering_event: epoch_sent: 118 epoch_requested: 118 ActivateCommitted from 118 processed at 118 2026-02-11T23:31:27.933+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=117/118 n=0 ec=49/49 lis/c=109/109 les/c/f=111/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 crt=111'600 lcod 0'0 unknown mbc={}] state: react 118 telling primary 2026-02-11T23:31:27.933+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.933+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.933+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.933+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.933+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 map epoch 117 -> 118 (shared) 2026-02-11T23:31:27.933+0000 7f414f993640 10 osd.0 118 send_incremental_map epoch range: (118, 118] to v2:127.0.0.1:6836/3331151400 2026-02-11T23:31:27.933+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- osd_map(118..118 src has 1..118) -- 0x59541c0 con 0x507ac00 2026-02-11T23:31:27.933+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_info2(4.0 4.0 4.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=117/118 n=0 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) e118/118) -- 0x63a8000 con 0x507ac00 2026-02-11T23:31:27.933+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.933+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 35 ==== pg_info2(4.0 4.0 4.0( v 111'600 (111'500,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) e118/118) ==== 1111+0+0 (unknown 0 0 0) 0x63a8000 con 0x507ac00 2026-02-11T23:31:27.933+0000 7f416abfa640 15 osd.0 118 enqueue_peering_evt 4.0 epoch_sent: 118 epoch_requested: 118 MInfoRec from 4 info: 4.0( v 111'600 (111'500,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) pg_lease(ru 76.773750305s ub 76.773750305s int 16.000000000s) 2026-02-11T23:31:27.933+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 118 epoch_requested: 118 MInfoRec from 4 info: 4.0( v 111'600 (111'500,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) pg_lease(ru 76.773750305s ub 76.773750305s int 16.000000000s)) class_id 2 prio 255 cost 10 e118) 2026-02-11T23:31:27.933+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.933+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 118 epoch_requested: 118 MInfoRec from 4 info: 4.0( v 111'600 (111'500,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) pg_lease(ru 76.773750305s ub 76.773750305s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e118) queued 2026-02-11T23:31:27.933+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.933+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 118 epoch_requested: 118 MInfoRec from 4 info: 4.0( v 111'600 (111'500,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) pg_lease(ru 76.773750305s ub 76.773750305s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e118) pg 0x5978000 2026-02-11T23:31:27.933+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.933+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=117/118 n=0 ec=49/49 lis/c=109/109 les/c/f=111/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 lcod 0'0 active mbc={}] do_peering_event: epoch_sent: 118 epoch_requested: 118 MInfoRec from 4 info: 4.0( v 111'600 (111'500,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) pg_lease(ru 76.773750305s ub 76.773750305s int 16.000000000s) 2026-02-11T23:31:27.933+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=117/118 n=0 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 lcod 0'0 active mbc={}] PeeringState::update_history advanced history from ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117 2026-02-11T23:31:27.933+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.933+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.933+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.933+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.933+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 36 ==== MBackfillReserve(4.0 REQUEST prio: 151 e118/118) ==== 46+0+0 (unknown 0 0 0) 0x5f1c300 con 0x507ac00 2026-02-11T23:31:27.933+0000 7f416abfa640 15 osd.0 118 enqueue_peering_evt 4.0 epoch_sent: 118 epoch_requested: 118 RequestBackfillPrio: priority 151 primary bytes 2457600 local bytes 0 2026-02-11T23:31:27.933+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 118 epoch_requested: 118 RequestBackfillPrio: priority 151 primary bytes 2457600 local bytes 0) class_id 2 prio 255 cost 10 e118) 2026-02-11T23:31:27.933+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.933+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 118 epoch_requested: 118 RequestBackfillPrio: priority 151 primary bytes 2457600 local bytes 0) class_id 2immediate prio 255 cost 10 e118) queued 2026-02-11T23:31:27.933+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.933+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 118 epoch_requested: 118 RequestBackfillPrio: priority 151 primary bytes 2457600 local bytes 0) class_id 2immediate prio 255 cost 10 e118) pg 0x5978000 2026-02-11T23:31:27.933+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=117/118 n=0 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 lcod 0'0 active mbc={}] do_peering_event: epoch_sent: 118 epoch_requested: 118 RequestBackfillPrio: priority 151 primary bytes 2457600 local bytes 0 2026-02-11T23:31:27.933+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.933+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=117/118 n=0 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 lcod 0'0 active mbc={}] try_reserve_recovery_space primary_bytes 2400KiB local 0KiB pending_adjustments 2400KiB 2026-02-11T23:31:27.933+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=117/118 n=0 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 lcod 0'0 active mbc={}] _tentative_full type backfillfull adjust_used 2400KiB 2026-02-11T23:31:27.933+0000 7f414f993640 20 osd.0 118 compute_adjusted_ratio Before kb_used() 0 2026-02-11T23:31:27.933+0000 7f414f993640 20 osd.0 118 compute_adjusted_ratio After kb_used() 2400 2026-02-11T23:31:27.933+0000 7f414f993640 5 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=117/118 n=0 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 lcod 0'0 active mbc={}] exit Started/ReplicaActive/RepNotRecovering 0.001459 3 0.000122 2026-02-11T23:31:27.933+0000 7f414f993640 5 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=117/118 n=0 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 lcod 0'0 active mbc={}] enter Started/ReplicaActive/RepWaitBackfillReserved 2026-02-11T23:31:27.933+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.933+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.933+0000 7f4152999640 15 osd.0 118 enqueue_peering_evt 4.0 epoch_sent: 118 epoch_requested: 118 RemoteBackfillReserved 2026-02-11T23:31:27.933+0000 7f4152999640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 118 epoch_requested: 118 RemoteBackfillReserved) class_id 2 prio 255 cost 10 e118) 2026-02-11T23:31:27.933+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.933+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 118 epoch_requested: 118 RemoteBackfillReserved) class_id 2immediate prio 255 cost 10 e118) queued 2026-02-11T23:31:27.933+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.933+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 118 epoch_requested: 118 RemoteBackfillReserved) class_id 2immediate prio 255 cost 10 e118) pg 0x5978000 2026-02-11T23:31:27.933+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=117/118 n=0 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 lcod 0'0 active mbc={}] do_peering_event: epoch_sent: 118 epoch_requested: 118 RemoteBackfillReserved 2026-02-11T23:31:27.933+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.933+0000 7f414b98b640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.933+0000 7f414b98b640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.933+0000 7f414b98b640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.933+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MBackfillReserve(4.0 GRANT e118/118) -- 0x5f2d980 con 0x507ac00 2026-02-11T23:31:27.933+0000 7f414b98b640 5 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=117/118 n=0 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 lcod 0'0 active mbc={}] exit Started/ReplicaActive/RepWaitBackfillReserved 0.000027 1 0.000014 2026-02-11T23:31:27.933+0000 7f414b98b640 5 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=117/118 n=0 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 lcod 0'0 active mbc={}] enter Started/ReplicaActive/RepRecovering 2026-02-11T23:31:27.933+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.933+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.936+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 38 ==== pg_scan(get_digest 4.0 MIN-MIN e 118/117) ==== 119+0+0 (unknown 0 0 0) 0x4a7af00 con 0x507ac00 2026-02-11T23:31:27.936+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_scan(get_digest 4.0 MIN-MIN e 118/117) v2 prio 15 type 94 cost 0 latency 0.000006 epoch 118 2026-02-11T23:31:27.936+0000 7f415aefe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 37 ==== osd_map(118..118 src has 1..118) ==== 325+0+0 (unknown 0 0 0) 0x59541c0 con 0x507ac00 2026-02-11T23:31:27.936+0000 7f415aefe640 20 osd.0 118 OSD::ms_dispatch: osd_map(118..118 src has 1..118) 2026-02-11T23:31:27.936+0000 7f415aefe640 20 osd.0 118 _dispatch 0x59541c0 osd_map(118..118 src has 1..118) 2026-02-11T23:31:27.936+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_scan(get_digest 4.0 MIN-MIN e 118/117) v2) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.936+0000 7f415aefe640 3 osd.0 118 handle_osd_map epochs [118,118], i have 118, src has [1,118] 2026-02-11T23:31:27.936+0000 7f415aefe640 10 osd.0 118 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:31:27.936+0000 7f415aefe640 10 osd.0 118 no new maps here, dropping 2026-02-11T23:31:27.936+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.936+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_scan(get_digest 4.0 MIN-MIN e 118/117) v2) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.936+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.936+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_scan(get_digest 4.0 MIN-MIN e 118/117) v2) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.936+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.936+0000 7f414f993640 10 osd.0 118 dequeue_op pg_scan(get_digest 4.0 MIN-MIN e 118/117) v2 prio 15 cost 0 latency 0.000025 pg_scan(get_digest 4.0 MIN-MIN e 118/117) v2 pg pg[4.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=117/118 n=0 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 lcod 0'0 active mbc={}] 2026-02-11T23:31:27.936+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.936+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=117/118 n=0 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 lcod 0'0 active mbc={}] _handle_message: pg_scan(get_digest 4.0 MIN-MIN e 118/117) v2 2026-02-11T23:31:27.936+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=117/118 n=0 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 lcod 0'0 active mbc={}] do_scan pg_scan(get_digest 4.0 MIN-MIN e 118/117) 2026-02-11T23:31:27.936+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=117/118 n=0 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 lcod 0'0 active mbc={}] scan_range_replica from MIN 2026-02-11T23:31:27.936+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=117/118 n=0 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 lcod 0'0 active mbc={}] got 0 items, next MAX 2026-02-11T23:31:27.936+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=117/118 n=0 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 lcod 0'0 active mbc={}] [] 2026-02-11T23:31:27.936+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_scan(digest 4.0 MIN-MAX e 118/117) -- 0x4a7ac80 con 0x507ac00 2026-02-11T23:31:27.936+0000 7f414f993640 10 osd.0 118 dequeue_op pg_scan(get_digest 4.0 MIN-MIN e 118/117) v2 finish latency 0.000054 2026-02-11T23:31:27.936+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.936+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 39 ==== MOSDPGPush(4.0 118/117 [PushOp(4:006bea6a:::obj172:head, version: 111'172, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:006bea6a:::obj172:head@111'172, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5646b60 con 0x507ac00 2026-02-11T23:31:27.936+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:006bea6a:::obj172:head, version: 111'172, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:006bea6a:::obj172:head@111'172, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000015 epoch 118 2026-02-11T23:31:27.936+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:006bea6a:::obj172:head, version: 111'172, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:006bea6a:::obj172:head@111'172, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.936+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.936+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:006bea6a:::obj172:head, version: 111'172, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:006bea6a:::obj172:head@111'172, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.936+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.936+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:006bea6a:::obj172:head, version: 111'172, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:006bea6a:::obj172:head@111'172, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.936+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.936+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:006bea6a:::obj172:head, version: 111'172, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:006bea6a:::obj172:head@111'172, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000050 MOSDPGPush(4.0 118/117 [PushOp(4:006bea6a:::obj172:head, version: 111'172, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:006bea6a:::obj172:head@111'172, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=117/118 n=0 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 lcod 0'0 active mbc={}] 2026-02-11T23:31:27.936+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.936+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=117/118 n=0 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 lcod 0'0 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:006bea6a:::obj172:head, version: 111'172, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:006bea6a:::obj172:head@111'172, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.936+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=117/118 n=0 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 lcod 0'0 active mbc={}] handle_push ObjectRecoveryInfo(4:006bea6a:::obj172:head@111'172, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.936+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=117/118 n=0 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 lcod 0'0 active mbc={}] submit_push_data 4:006bea6a:::obj172:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.936+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=117/118 n=0 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 lcod 0'0 active mbc={}] submit_push_data recovering object 4:006bea6a:::obj172:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.936+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=117/118 n=0 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 lcod 0'0 active mbc={}] on_local_recover: 4:006bea6a:::obj172:head 2026-02-11T23:31:27.936+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=117/118 n=0 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 lcod 0'0 active mbc={}] PeeringState::recover_got got missing 4:006bea6a:::obj172:head v 111'172 2026-02-11T23:31:27.936+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=117/118 n=0 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 lcod 0'0 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.936+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.936+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:006bea6a:::obj172:head, version: 111'172, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:006bea6a:::obj172:head@111'172, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000096 2026-02-11T23:31:27.936+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.936+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=117/118 n=0 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 lcod 0'0 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.936+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.936+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.936+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=117/118 n=0 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 lcod 0'0 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.936+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.936+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.936+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.936+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x5f75980 con 0x507ac00 2026-02-11T23:31:27.936+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:006bea6a:::obj172:head)]) -- 0x5f2d800 con 0x507ac00 2026-02-11T23:31:27.936+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.937+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 40 ==== MOSDPGPush(4.0 118/117 [PushOp(4:00833431:::obj76:head, version: 111'76, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:00833431:::obj76:head@111'76, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x3815a00 con 0x507ac00 2026-02-11T23:31:27.937+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:00833431:::obj76:head, version: 111'76, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:00833431:::obj76:head@111'76, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 118 2026-02-11T23:31:27.937+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:00833431:::obj76:head, version: 111'76, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:00833431:::obj76:head@111'76, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.937+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.937+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 41 ==== pg_backfill(progress 4.0 e 118/118 lb 4:006bea6a:::obj172:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f84000 con 0x507ac00 2026-02-11T23:31:27.937+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:00833431:::obj76:head, version: 111'76, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:00833431:::obj76:head@111'76, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.937+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:006bea6a:::obj172:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 118 2026-02-11T23:31:27.937+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.937+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:006bea6a:::obj172:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.937+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:00833431:::obj76:head, version: 111'76, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:00833431:::obj76:head@111'76, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.937+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.937+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:00833431:::obj76:head, version: 111'76, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:00833431:::obj76:head@111'76, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 118/117 [PushOp(4:00833431:::obj76:head, version: 111'76, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:00833431:::obj76:head@111'76, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=117/118 n=0 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.937+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.937+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=117/118 n=0 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:00833431:::obj76:head, version: 111'76, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:00833431:::obj76:head@111'76, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.937+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=117/118 n=0 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:00833431:::obj76:head@111'76, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.937+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.937+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:006bea6a:::obj172:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.937+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=117/118 n=0 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:00833431:::obj76:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.937+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=117/118 n=0 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:00833431:::obj76:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.937+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=117/118 n=0 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:00833431:::obj76:head 2026-02-11T23:31:27.937+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=117/118 n=0 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:00833431:::obj76:head v 111'76 2026-02-11T23:31:27.937+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=117/118 n=0 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.937+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.937+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:00833431:::obj76:head, version: 111'76, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:00833431:::obj76:head@111'76, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:27.937+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.937+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.937+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:006bea6a:::obj172:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.937+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:006bea6a:::obj172:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 4.0 e 118/118 lb 4:006bea6a:::obj172:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=117/118 n=0 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.937+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.937+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=117/118 n=0 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:006bea6a:::obj172:head) v3 2026-02-11T23:31:27.937+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb MIN local-lis/les=117/118 n=0 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:006bea6a:::obj172:head) 2026-02-11T23:31:27.937+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.937+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:006bea6a:::obj172:head) v3 finish latency 0.000015 2026-02-11T23:31:27.937+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:006bea6a:::obj172:head local-lis/les=117/118 n=1 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.937+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.937+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.937+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:006bea6a:::obj172:head local-lis/les=117/118 n=1 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.937+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.937+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.937+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.937+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x5f75b00 con 0x507ac00 2026-02-11T23:31:27.937+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:00833431:::obj76:head)]) -- 0x5f2dc80 con 0x507ac00 2026-02-11T23:31:27.937+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.938+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 42 ==== MOSDPGPush(4.0 118/117 [PushOp(4:00b31b6c:::obj19:head, version: 111'19, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:00b31b6c:::obj19:head@111'19, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5647380 con 0x507ac00 2026-02-11T23:31:27.938+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:00b31b6c:::obj19:head, version: 111'19, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:00b31b6c:::obj19:head@111'19, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 118 2026-02-11T23:31:27.938+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:00b31b6c:::obj19:head, version: 111'19, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:00b31b6c:::obj19:head@111'19, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.938+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.938+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 43 ==== pg_backfill(progress 4.0 e 118/118 lb 4:00833431:::obj76:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d99080 con 0x507ac00 2026-02-11T23:31:27.938+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:00b31b6c:::obj19:head, version: 111'19, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:00b31b6c:::obj19:head@111'19, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.938+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:00833431:::obj76:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 118 2026-02-11T23:31:27.938+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.938+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:00833431:::obj76:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.938+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:00b31b6c:::obj19:head, version: 111'19, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:00b31b6c:::obj19:head@111'19, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.938+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.938+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:00b31b6c:::obj19:head, version: 111'19, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:00b31b6c:::obj19:head@111'19, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(4.0 118/117 [PushOp(4:00b31b6c:::obj19:head, version: 111'19, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:00b31b6c:::obj19:head@111'19, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:006bea6a:::obj172:head local-lis/les=117/118 n=1 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.938+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.938+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:006bea6a:::obj172:head local-lis/les=117/118 n=1 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:00b31b6c:::obj19:head, version: 111'19, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:00b31b6c:::obj19:head@111'19, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.938+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:006bea6a:::obj172:head local-lis/les=117/118 n=1 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:00b31b6c:::obj19:head@111'19, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.938+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.938+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:00833431:::obj76:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.938+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:006bea6a:::obj172:head local-lis/les=117/118 n=1 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:00b31b6c:::obj19:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.938+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:006bea6a:::obj172:head local-lis/les=117/118 n=1 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:00b31b6c:::obj19:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.938+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:006bea6a:::obj172:head local-lis/les=117/118 n=1 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:00b31b6c:::obj19:head 2026-02-11T23:31:27.938+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:006bea6a:::obj172:head local-lis/les=117/118 n=1 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:00b31b6c:::obj19:head v 111'19 2026-02-11T23:31:27.938+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:006bea6a:::obj172:head local-lis/les=117/118 n=1 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.938+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.938+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:00b31b6c:::obj19:head, version: 111'19, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:00b31b6c:::obj19:head@111'19, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.938+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.938+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.938+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:00833431:::obj76:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.938+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:00833431:::obj76:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 4.0 e 118/118 lb 4:00833431:::obj76:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:006bea6a:::obj172:head local-lis/les=117/118 n=1 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.938+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.938+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:006bea6a:::obj172:head local-lis/les=117/118 n=1 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:00833431:::obj76:head) v3 2026-02-11T23:31:27.938+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:006bea6a:::obj172:head local-lis/les=117/118 n=1 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:00833431:::obj76:head) 2026-02-11T23:31:27.938+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.938+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:00833431:::obj76:head) v3 finish latency 0.000011 2026-02-11T23:31:27.938+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:00833431:::obj76:head local-lis/les=117/118 n=2 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.938+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.938+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.938+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:00833431:::obj76:head local-lis/les=117/118 n=2 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.938+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.938+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.938+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.938+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x5f75c80 con 0x507ac00 2026-02-11T23:31:27.938+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:00b31b6c:::obj19:head)]) -- 0x6171800 con 0x507ac00 2026-02-11T23:31:27.938+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.938+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 44 ==== MOSDPGPush(4.0 118/117 [PushOp(4:010e1e72:::obj351:head, version: 111'351, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:010e1e72:::obj351:head@111'351, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e26680 con 0x507ac00 2026-02-11T23:31:27.938+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:010e1e72:::obj351:head, version: 111'351, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:010e1e72:::obj351:head@111'351, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 118 2026-02-11T23:31:27.938+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:010e1e72:::obj351:head, version: 111'351, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:010e1e72:::obj351:head@111'351, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.938+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.938+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 45 ==== pg_backfill(progress 4.0 e 118/118 lb 4:00b31b6c:::obj19:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d99600 con 0x507ac00 2026-02-11T23:31:27.938+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:010e1e72:::obj351:head, version: 111'351, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:010e1e72:::obj351:head@111'351, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.938+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:00b31b6c:::obj19:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:27.938+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:00b31b6c:::obj19:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.938+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.938+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:010e1e72:::obj351:head, version: 111'351, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:010e1e72:::obj351:head@111'351, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.938+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:010e1e72:::obj351:head, version: 111'351, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:010e1e72:::obj351:head@111'351, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 118/117 [PushOp(4:010e1e72:::obj351:head, version: 111'351, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:010e1e72:::obj351:head@111'351, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:00833431:::obj76:head local-lis/les=117/118 n=2 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.938+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.938+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.938+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:00b31b6c:::obj19:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.938+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:00833431:::obj76:head local-lis/les=117/118 n=2 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:010e1e72:::obj351:head, version: 111'351, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:010e1e72:::obj351:head@111'351, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.938+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:00833431:::obj76:head local-lis/les=117/118 n=2 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:010e1e72:::obj351:head@111'351, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.938+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:00833431:::obj76:head local-lis/les=117/118 n=2 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:010e1e72:::obj351:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.938+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:00833431:::obj76:head local-lis/les=117/118 n=2 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:010e1e72:::obj351:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.938+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:00833431:::obj76:head local-lis/les=117/118 n=2 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:010e1e72:::obj351:head 2026-02-11T23:31:27.938+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:00833431:::obj76:head local-lis/les=117/118 n=2 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:010e1e72:::obj351:head v 111'351 2026-02-11T23:31:27.938+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:00833431:::obj76:head local-lis/les=117/118 n=2 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.938+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.938+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:010e1e72:::obj351:head, version: 111'351, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:010e1e72:::obj351:head@111'351, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.938+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.938+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.938+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:00b31b6c:::obj19:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.938+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:00b31b6c:::obj19:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 4.0 e 118/118 lb 4:00b31b6c:::obj19:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:00833431:::obj76:head local-lis/les=117/118 n=2 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.938+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.938+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:00833431:::obj76:head local-lis/les=117/118 n=2 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:00b31b6c:::obj19:head) v3 2026-02-11T23:31:27.938+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:00833431:::obj76:head local-lis/les=117/118 n=2 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:00b31b6c:::obj19:head) 2026-02-11T23:31:27.938+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.938+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:00b31b6c:::obj19:head) v3 finish latency 0.000010 2026-02-11T23:31:27.938+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:00b31b6c:::obj19:head local-lis/les=117/118 n=3 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.938+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.939+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.939+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:00b31b6c:::obj19:head local-lis/les=117/118 n=3 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.939+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.939+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.939+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.939+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x5f75e00 con 0x507ac00 2026-02-11T23:31:27.939+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:010e1e72:::obj351:head)]) -- 0x6170300 con 0x507ac00 2026-02-11T23:31:27.939+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.939+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 46 ==== MOSDPGPush(4.0 118/117 [PushOp(4:015e5cd2:::obj354:head, version: 111'354, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:015e5cd2:::obj354:head@111'354, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559cb60 con 0x507ac00 2026-02-11T23:31:27.939+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:015e5cd2:::obj354:head, version: 111'354, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:015e5cd2:::obj354:head@111'354, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 118 2026-02-11T23:31:27.939+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:015e5cd2:::obj354:head, version: 111'354, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:015e5cd2:::obj354:head@111'354, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.939+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.939+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 47 ==== pg_backfill(progress 4.0 e 118/118 lb 4:010e1e72:::obj351:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99b80 con 0x507ac00 2026-02-11T23:31:27.939+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:015e5cd2:::obj354:head, version: 111'354, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:015e5cd2:::obj354:head@111'354, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.939+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:010e1e72:::obj351:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:27.939+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.939+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:010e1e72:::obj351:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.939+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:015e5cd2:::obj354:head, version: 111'354, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:015e5cd2:::obj354:head@111'354, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.939+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.939+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:015e5cd2:::obj354:head, version: 111'354, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:015e5cd2:::obj354:head@111'354, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 118/117 [PushOp(4:015e5cd2:::obj354:head, version: 111'354, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:015e5cd2:::obj354:head@111'354, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:00b31b6c:::obj19:head local-lis/les=117/118 n=3 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.939+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.939+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:00b31b6c:::obj19:head local-lis/les=117/118 n=3 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:015e5cd2:::obj354:head, version: 111'354, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:015e5cd2:::obj354:head@111'354, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.939+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:00b31b6c:::obj19:head local-lis/les=117/118 n=3 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:015e5cd2:::obj354:head@111'354, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.939+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.939+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:010e1e72:::obj351:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.939+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:00b31b6c:::obj19:head local-lis/les=117/118 n=3 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:015e5cd2:::obj354:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.939+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:00b31b6c:::obj19:head local-lis/les=117/118 n=3 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:015e5cd2:::obj354:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.939+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:00b31b6c:::obj19:head local-lis/les=117/118 n=3 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:015e5cd2:::obj354:head 2026-02-11T23:31:27.939+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:00b31b6c:::obj19:head local-lis/les=117/118 n=3 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:015e5cd2:::obj354:head v 111'354 2026-02-11T23:31:27.939+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:00b31b6c:::obj19:head local-lis/les=117/118 n=3 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.939+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.939+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:015e5cd2:::obj354:head, version: 111'354, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:015e5cd2:::obj354:head@111'354, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000044 2026-02-11T23:31:27.939+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.939+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.939+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:010e1e72:::obj351:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.939+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:010e1e72:::obj351:head) v3 prio 15 cost 0 latency 0.000060 pg_backfill(progress 4.0 e 118/118 lb 4:010e1e72:::obj351:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:00b31b6c:::obj19:head local-lis/les=117/118 n=3 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.939+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.939+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:00b31b6c:::obj19:head local-lis/les=117/118 n=3 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:010e1e72:::obj351:head) v3 2026-02-11T23:31:27.939+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:00b31b6c:::obj19:head local-lis/les=117/118 n=3 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:010e1e72:::obj351:head) 2026-02-11T23:31:27.939+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.939+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:010e1e72:::obj351:head) v3 finish latency 0.000010 2026-02-11T23:31:27.939+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:010e1e72:::obj351:head local-lis/les=117/118 n=4 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.939+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.939+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.939+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:010e1e72:::obj351:head local-lis/les=117/118 n=4 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.939+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.939+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.939+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.939+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x5f75c80 con 0x507ac00 2026-02-11T23:31:27.939+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:015e5cd2:::obj354:head)]) -- 0x612ed80 con 0x507ac00 2026-02-11T23:31:27.939+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.940+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 48 ==== MOSDPGPush(4.0 118/117 [PushOp(4:022ea2cc:::obj392:head, version: 111'392, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:022ea2cc:::obj392:head@111'392, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f0b60 con 0x507ac00 2026-02-11T23:31:27.940+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:022ea2cc:::obj392:head, version: 111'392, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:022ea2cc:::obj392:head@111'392, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 118 2026-02-11T23:31:27.940+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:022ea2cc:::obj392:head, version: 111'392, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:022ea2cc:::obj392:head@111'392, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.940+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.940+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 49 ==== pg_backfill(progress 4.0 e 118/118 lb 4:015e5cd2:::obj354:head) ==== 1062+0+0 (unknown 0 0 0) 0x5982b00 con 0x507ac00 2026-02-11T23:31:27.940+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:022ea2cc:::obj392:head, version: 111'392, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:022ea2cc:::obj392:head@111'392, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.940+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:015e5cd2:::obj354:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:27.940+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:015e5cd2:::obj354:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.940+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.940+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:022ea2cc:::obj392:head, version: 111'392, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:022ea2cc:::obj392:head@111'392, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.940+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.940+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:022ea2cc:::obj392:head, version: 111'392, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:022ea2cc:::obj392:head@111'392, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(4.0 118/117 [PushOp(4:022ea2cc:::obj392:head, version: 111'392, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:022ea2cc:::obj392:head@111'392, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:010e1e72:::obj351:head local-lis/les=117/118 n=4 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.940+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.940+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:010e1e72:::obj351:head local-lis/les=117/118 n=4 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:022ea2cc:::obj392:head, version: 111'392, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:022ea2cc:::obj392:head@111'392, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.940+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:010e1e72:::obj351:head local-lis/les=117/118 n=4 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:022ea2cc:::obj392:head@111'392, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.940+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.940+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:015e5cd2:::obj354:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.940+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:010e1e72:::obj351:head local-lis/les=117/118 n=4 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:022ea2cc:::obj392:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.940+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:010e1e72:::obj351:head local-lis/les=117/118 n=4 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:022ea2cc:::obj392:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.940+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:010e1e72:::obj351:head local-lis/les=117/118 n=4 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:022ea2cc:::obj392:head 2026-02-11T23:31:27.940+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:010e1e72:::obj351:head local-lis/les=117/118 n=4 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:022ea2cc:::obj392:head v 111'392 2026-02-11T23:31:27.940+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:010e1e72:::obj351:head local-lis/les=117/118 n=4 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.940+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.940+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:022ea2cc:::obj392:head, version: 111'392, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:022ea2cc:::obj392:head@111'392, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.940+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.940+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.940+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:015e5cd2:::obj354:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.940+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:015e5cd2:::obj354:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 4.0 e 118/118 lb 4:015e5cd2:::obj354:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:010e1e72:::obj351:head local-lis/les=117/118 n=4 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.940+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.940+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:010e1e72:::obj351:head local-lis/les=117/118 n=4 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:015e5cd2:::obj354:head) v3 2026-02-11T23:31:27.940+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:010e1e72:::obj351:head local-lis/les=117/118 n=4 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:015e5cd2:::obj354:head) 2026-02-11T23:31:27.940+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.940+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:015e5cd2:::obj354:head) v3 finish latency 0.000011 2026-02-11T23:31:27.940+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:015e5cd2:::obj354:head local-lis/les=117/118 n=5 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.940+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.940+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.940+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:015e5cd2:::obj354:head local-lis/les=117/118 n=5 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.940+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.940+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.940+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.940+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x5f2dc80 con 0x507ac00 2026-02-11T23:31:27.940+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:022ea2cc:::obj392:head)]) -- 0x5f1c180 con 0x507ac00 2026-02-11T23:31:27.940+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.940+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 50 ==== MOSDPGPush(4.0 118/117 [PushOp(4:02455f6c:::obj462:head, version: 111'462, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:02455f6c:::obj462:head@111'462, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f1ba0 con 0x507ac00 2026-02-11T23:31:27.940+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:02455f6c:::obj462:head, version: 111'462, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:02455f6c:::obj462:head@111'462, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 118 2026-02-11T23:31:27.940+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:02455f6c:::obj462:head, version: 111'462, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:02455f6c:::obj462:head@111'462, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.940+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.940+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 51 ==== pg_backfill(progress 4.0 e 118/118 lb 4:022ea2cc:::obj392:head) ==== 1062+0+0 (unknown 0 0 0) 0x4a2c000 con 0x507ac00 2026-02-11T23:31:27.940+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:02455f6c:::obj462:head, version: 111'462, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:02455f6c:::obj462:head@111'462, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.940+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:022ea2cc:::obj392:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:27.940+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:022ea2cc:::obj392:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.940+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.940+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:02455f6c:::obj462:head, version: 111'462, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:02455f6c:::obj462:head@111'462, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.940+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.940+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:02455f6c:::obj462:head, version: 111'462, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:02455f6c:::obj462:head@111'462, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(4.0 118/117 [PushOp(4:02455f6c:::obj462:head, version: 111'462, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:02455f6c:::obj462:head@111'462, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:015e5cd2:::obj354:head local-lis/les=117/118 n=5 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.940+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.940+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:015e5cd2:::obj354:head local-lis/les=117/118 n=5 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:02455f6c:::obj462:head, version: 111'462, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:02455f6c:::obj462:head@111'462, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.940+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:015e5cd2:::obj354:head local-lis/les=117/118 n=5 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:02455f6c:::obj462:head@111'462, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.940+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.940+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:022ea2cc:::obj392:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.940+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:015e5cd2:::obj354:head local-lis/les=117/118 n=5 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:02455f6c:::obj462:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.940+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:015e5cd2:::obj354:head local-lis/les=117/118 n=5 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:02455f6c:::obj462:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.940+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:015e5cd2:::obj354:head local-lis/les=117/118 n=5 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:02455f6c:::obj462:head 2026-02-11T23:31:27.940+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:015e5cd2:::obj354:head local-lis/les=117/118 n=5 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:02455f6c:::obj462:head v 111'462 2026-02-11T23:31:27.940+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:015e5cd2:::obj354:head local-lis/les=117/118 n=5 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.940+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.940+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:02455f6c:::obj462:head, version: 111'462, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:02455f6c:::obj462:head@111'462, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.940+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.940+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.940+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:022ea2cc:::obj392:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.940+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:022ea2cc:::obj392:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 4.0 e 118/118 lb 4:022ea2cc:::obj392:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:015e5cd2:::obj354:head local-lis/les=117/118 n=5 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.940+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.940+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:015e5cd2:::obj354:head local-lis/les=117/118 n=5 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:022ea2cc:::obj392:head) v3 2026-02-11T23:31:27.940+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:015e5cd2:::obj354:head local-lis/les=117/118 n=5 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:022ea2cc:::obj392:head) 2026-02-11T23:31:27.940+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.940+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:022ea2cc:::obj392:head) v3 finish latency 0.000016 2026-02-11T23:31:27.940+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:022ea2cc:::obj392:head local-lis/les=117/118 n=6 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.940+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.940+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.940+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:022ea2cc:::obj392:head local-lis/les=117/118 n=6 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.940+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.940+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.941+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.941+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x5f75b00 con 0x507ac00 2026-02-11T23:31:27.941+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:02455f6c:::obj462:head)]) -- 0x5ca1680 con 0x507ac00 2026-02-11T23:31:27.941+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.941+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 52 ==== MOSDPGPush(4.0 118/117 [PushOp(4:024707f1:::obj490:head, version: 111'490, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:024707f1:::obj490:head@111'490, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f0d00 con 0x507ac00 2026-02-11T23:31:27.941+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:024707f1:::obj490:head, version: 111'490, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:024707f1:::obj490:head@111'490, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 118 2026-02-11T23:31:27.941+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:024707f1:::obj490:head, version: 111'490, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:024707f1:::obj490:head@111'490, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.941+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.941+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 53 ==== pg_backfill(progress 4.0 e 118/118 lb 4:02455f6c:::obj462:head) ==== 1062+0+0 (unknown 0 0 0) 0x4a2cb00 con 0x507ac00 2026-02-11T23:31:27.941+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:02455f6c:::obj462:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:27.941+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:024707f1:::obj490:head, version: 111'490, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:024707f1:::obj490:head@111'490, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.941+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:02455f6c:::obj462:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.941+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.941+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:024707f1:::obj490:head, version: 111'490, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:024707f1:::obj490:head@111'490, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.941+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:024707f1:::obj490:head, version: 111'490, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:024707f1:::obj490:head@111'490, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 118/117 [PushOp(4:024707f1:::obj490:head, version: 111'490, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:024707f1:::obj490:head@111'490, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:022ea2cc:::obj392:head local-lis/les=117/118 n=6 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.941+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.941+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.941+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:022ea2cc:::obj392:head local-lis/les=117/118 n=6 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:024707f1:::obj490:head, version: 111'490, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:024707f1:::obj490:head@111'490, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.941+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:02455f6c:::obj462:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.941+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:022ea2cc:::obj392:head local-lis/les=117/118 n=6 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:024707f1:::obj490:head@111'490, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.941+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:022ea2cc:::obj392:head local-lis/les=117/118 n=6 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:024707f1:::obj490:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.941+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:022ea2cc:::obj392:head local-lis/les=117/118 n=6 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:024707f1:::obj490:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.941+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:022ea2cc:::obj392:head local-lis/les=117/118 n=6 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:024707f1:::obj490:head 2026-02-11T23:31:27.941+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:022ea2cc:::obj392:head local-lis/les=117/118 n=6 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:024707f1:::obj490:head v 111'490 2026-02-11T23:31:27.941+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:022ea2cc:::obj392:head local-lis/les=117/118 n=6 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.941+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.941+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:024707f1:::obj490:head, version: 111'490, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:024707f1:::obj490:head@111'490, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:27.941+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.941+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.941+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:02455f6c:::obj462:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.941+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:02455f6c:::obj462:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 4.0 e 118/118 lb 4:02455f6c:::obj462:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:022ea2cc:::obj392:head local-lis/les=117/118 n=6 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.941+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.941+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:022ea2cc:::obj392:head local-lis/les=117/118 n=6 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:02455f6c:::obj462:head) v3 2026-02-11T23:31:27.941+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:022ea2cc:::obj392:head local-lis/les=117/118 n=6 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:02455f6c:::obj462:head) 2026-02-11T23:31:27.941+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.941+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:02455f6c:::obj462:head) v3 finish latency 0.000010 2026-02-11T23:31:27.941+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:02455f6c:::obj462:head local-lis/les=117/118 n=7 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.941+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.941+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.941+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:02455f6c:::obj462:head local-lis/les=117/118 n=7 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.941+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.941+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.941+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.941+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x5f2d800 con 0x507ac00 2026-02-11T23:31:27.941+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:024707f1:::obj490:head)]) -- 0x5d34180 con 0x507ac00 2026-02-11T23:31:27.941+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.942+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 54 ==== MOSDPGPush(4.0 118/117 [PushOp(4:02817247:::obj101:head, version: 111'101, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:02817247:::obj101:head@111'101, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56ca9c0 con 0x507ac00 2026-02-11T23:31:27.942+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:02817247:::obj101:head, version: 111'101, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:02817247:::obj101:head@111'101, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 118 2026-02-11T23:31:27.942+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:02817247:::obj101:head, version: 111'101, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:02817247:::obj101:head@111'101, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.942+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.942+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 55 ==== pg_backfill(progress 4.0 e 118/118 lb 4:024707f1:::obj490:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9ac00 con 0x507ac00 2026-02-11T23:31:27.942+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:024707f1:::obj490:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:27.942+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:024707f1:::obj490:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.942+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:02817247:::obj101:head, version: 111'101, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:02817247:::obj101:head@111'101, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.942+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.942+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:02817247:::obj101:head, version: 111'101, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:02817247:::obj101:head@111'101, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.942+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.942+0000 7f414f993640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:02817247:::obj101:head, version: 111'101, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:02817247:::obj101:head@111'101, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(4.0 118/117 [PushOp(4:02817247:::obj101:head, version: 111'101, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:02817247:::obj101:head@111'101, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:02455f6c:::obj462:head local-lis/les=117/118 n=7 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.942+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.942+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:024707f1:::obj490:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.942+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:02455f6c:::obj462:head local-lis/les=117/118 n=7 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:02817247:::obj101:head, version: 111'101, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:02817247:::obj101:head@111'101, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.942+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:02455f6c:::obj462:head local-lis/les=117/118 n=7 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:02817247:::obj101:head@111'101, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.942+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:02455f6c:::obj462:head local-lis/les=117/118 n=7 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:02817247:::obj101:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.942+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:02455f6c:::obj462:head local-lis/les=117/118 n=7 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:02817247:::obj101:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.942+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:02455f6c:::obj462:head local-lis/les=117/118 n=7 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:02817247:::obj101:head 2026-02-11T23:31:27.942+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:02455f6c:::obj462:head local-lis/les=117/118 n=7 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:02817247:::obj101:head v 111'101 2026-02-11T23:31:27.942+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:02455f6c:::obj462:head local-lis/les=117/118 n=7 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.942+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.942+0000 7f414f993640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:02817247:::obj101:head, version: 111'101, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:02817247:::obj101:head@111'101, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:27.942+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:02455f6c:::obj462:head local-lis/les=117/118 n=7 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.942+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.942+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.942+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:024707f1:::obj490:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.942+0000 7f414b98b640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:024707f1:::obj490:head) v3 prio 15 cost 0 latency 0.000082 pg_backfill(progress 4.0 e 118/118 lb 4:024707f1:::obj490:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:02455f6c:::obj462:head local-lis/les=117/118 n=7 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.942+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.942+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:02455f6c:::obj462:head local-lis/les=117/118 n=7 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:024707f1:::obj490:head) v3 2026-02-11T23:31:27.942+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:02455f6c:::obj462:head local-lis/les=117/118 n=7 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:024707f1:::obj490:head) 2026-02-11T23:31:27.942+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.942+0000 7f414b98b640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:024707f1:::obj490:head) v3 finish latency 0.000012 2026-02-11T23:31:27.942+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.942+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.942+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:024707f1:::obj490:head local-lis/les=117/118 n=8 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.942+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.942+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.942+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.942+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x5f1c300 con 0x507ac00 2026-02-11T23:31:27.942+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:02817247:::obj101:head)]) -- 0x5f75980 con 0x507ac00 2026-02-11T23:31:27.942+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.942+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 56 ==== MOSDPGPush(4.0 118/117 [PushOp(4:03553543:::obj227:head, version: 111'227, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:03553543:::obj227:head@111'227, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56caea0 con 0x507ac00 2026-02-11T23:31:27.942+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:03553543:::obj227:head, version: 111'227, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:03553543:::obj227:head@111'227, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 118 2026-02-11T23:31:27.942+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:03553543:::obj227:head, version: 111'227, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:03553543:::obj227:head@111'227, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.942+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.942+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 57 ==== pg_backfill(progress 4.0 e 118/118 lb 4:02817247:::obj101:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99600 con 0x507ac00 2026-02-11T23:31:27.942+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:03553543:::obj227:head, version: 111'227, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:03553543:::obj227:head@111'227, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.942+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:02817247:::obj101:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:27.942+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.942+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:02817247:::obj101:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.942+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:03553543:::obj227:head, version: 111'227, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:03553543:::obj227:head@111'227, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.942+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.942+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:03553543:::obj227:head, version: 111'227, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:03553543:::obj227:head@111'227, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 118/117 [PushOp(4:03553543:::obj227:head, version: 111'227, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:03553543:::obj227:head@111'227, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:024707f1:::obj490:head local-lis/les=117/118 n=8 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.942+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.942+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:024707f1:::obj490:head local-lis/les=117/118 n=8 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:03553543:::obj227:head, version: 111'227, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:03553543:::obj227:head@111'227, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.942+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:024707f1:::obj490:head local-lis/les=117/118 n=8 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:03553543:::obj227:head@111'227, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.942+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.942+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:02817247:::obj101:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.942+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:024707f1:::obj490:head local-lis/les=117/118 n=8 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:03553543:::obj227:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.942+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:024707f1:::obj490:head local-lis/les=117/118 n=8 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:03553543:::obj227:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.942+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:024707f1:::obj490:head local-lis/les=117/118 n=8 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:03553543:::obj227:head 2026-02-11T23:31:27.942+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:024707f1:::obj490:head local-lis/les=117/118 n=8 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:03553543:::obj227:head v 111'227 2026-02-11T23:31:27.942+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:024707f1:::obj490:head local-lis/les=117/118 n=8 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.942+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.942+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:03553543:::obj227:head, version: 111'227, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:03553543:::obj227:head@111'227, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000044 2026-02-11T23:31:27.942+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.942+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.942+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:02817247:::obj101:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.942+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:02817247:::obj101:head) v3 prio 15 cost 0 latency 0.000058 pg_backfill(progress 4.0 e 118/118 lb 4:02817247:::obj101:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:024707f1:::obj490:head local-lis/les=117/118 n=8 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.942+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.942+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:024707f1:::obj490:head local-lis/les=117/118 n=8 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:02817247:::obj101:head) v3 2026-02-11T23:31:27.942+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:024707f1:::obj490:head local-lis/les=117/118 n=8 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:02817247:::obj101:head) 2026-02-11T23:31:27.942+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.942+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:02817247:::obj101:head) v3 finish latency 0.000012 2026-02-11T23:31:27.942+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:02817247:::obj101:head local-lis/les=117/118 n=9 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.942+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.942+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.942+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:02817247:::obj101:head local-lis/les=117/118 n=9 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.942+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.942+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.942+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.942+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x5f75800 con 0x507ac00 2026-02-11T23:31:27.942+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:03553543:::obj227:head)]) -- 0x5d34480 con 0x507ac00 2026-02-11T23:31:27.942+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.943+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 58 ==== MOSDPGPush(4.0 118/117 [PushOp(4:0366f813:::obj557:head, version: 111'557, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0366f813:::obj557:head@111'557, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5617380 con 0x507ac00 2026-02-11T23:31:27.943+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:0366f813:::obj557:head, version: 111'557, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0366f813:::obj557:head@111'557, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 118 2026-02-11T23:31:27.943+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0366f813:::obj557:head, version: 111'557, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0366f813:::obj557:head@111'557, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.943+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.943+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 59 ==== pg_backfill(progress 4.0 e 118/118 lb 4:03553543:::obj227:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99080 con 0x507ac00 2026-02-11T23:31:27.943+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:03553543:::obj227:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:27.943+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0366f813:::obj557:head, version: 111'557, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0366f813:::obj557:head@111'557, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.943+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:03553543:::obj227:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.943+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.943+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0366f813:::obj557:head, version: 111'557, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0366f813:::obj557:head@111'557, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.943+0000 7f414f993640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:0366f813:::obj557:head, version: 111'557, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0366f813:::obj557:head@111'557, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 118/117 [PushOp(4:0366f813:::obj557:head, version: 111'557, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0366f813:::obj557:head@111'557, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:02817247:::obj101:head local-lis/les=117/118 n=9 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.943+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.943+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:02817247:::obj101:head local-lis/les=117/118 n=9 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:0366f813:::obj557:head, version: 111'557, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0366f813:::obj557:head@111'557, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.943+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.943+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:03553543:::obj227:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.943+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:02817247:::obj101:head local-lis/les=117/118 n=9 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:0366f813:::obj557:head@111'557, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.943+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:02817247:::obj101:head local-lis/les=117/118 n=9 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:0366f813:::obj557:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.943+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:02817247:::obj101:head local-lis/les=117/118 n=9 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:0366f813:::obj557:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.943+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:02817247:::obj101:head local-lis/les=117/118 n=9 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:0366f813:::obj557:head 2026-02-11T23:31:27.943+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:02817247:::obj101:head local-lis/les=117/118 n=9 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:0366f813:::obj557:head v 111'557 2026-02-11T23:31:27.943+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:02817247:::obj101:head local-lis/les=117/118 n=9 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.943+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.943+0000 7f414f993640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:0366f813:::obj557:head, version: 111'557, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0366f813:::obj557:head@111'557, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:27.943+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:02817247:::obj101:head local-lis/les=117/118 n=9 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.943+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.943+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.943+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:03553543:::obj227:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.943+0000 7f414b98b640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:03553543:::obj227:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 118/118 lb 4:03553543:::obj227:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:02817247:::obj101:head local-lis/les=117/118 n=9 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.943+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.943+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:02817247:::obj101:head local-lis/les=117/118 n=9 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:03553543:::obj227:head) v3 2026-02-11T23:31:27.943+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:02817247:::obj101:head local-lis/les=117/118 n=9 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:03553543:::obj227:head) 2026-02-11T23:31:27.943+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.943+0000 7f414b98b640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:03553543:::obj227:head) v3 finish latency 0.000010 2026-02-11T23:31:27.943+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.943+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.943+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:03553543:::obj227:head local-lis/les=117/118 n=10 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.943+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.943+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.943+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.943+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x6000180 con 0x507ac00 2026-02-11T23:31:27.943+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:0366f813:::obj557:head)]) -- 0x5f1d500 con 0x507ac00 2026-02-11T23:31:27.943+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.943+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 60 ==== MOSDPGPush(4.0 118/117 [PushOp(4:03b9100b:::obj283:head, version: 111'283, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:03b9100b:::obj283:head@111'283, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a3860 con 0x507ac00 2026-02-11T23:31:27.943+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:03b9100b:::obj283:head, version: 111'283, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:03b9100b:::obj283:head@111'283, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 118 2026-02-11T23:31:27.943+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:03b9100b:::obj283:head, version: 111'283, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:03b9100b:::obj283:head@111'283, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.944+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.944+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 61 ==== pg_backfill(progress 4.0 e 118/118 lb 4:0366f813:::obj557:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f84000 con 0x507ac00 2026-02-11T23:31:27.944+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:0366f813:::obj557:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 118 2026-02-11T23:31:27.944+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:03b9100b:::obj283:head, version: 111'283, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:03b9100b:::obj283:head@111'283, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.944+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.944+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:03b9100b:::obj283:head, version: 111'283, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:03b9100b:::obj283:head@111'283, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.944+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.944+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:03b9100b:::obj283:head, version: 111'283, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:03b9100b:::obj283:head@111'283, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(4.0 118/117 [PushOp(4:03b9100b:::obj283:head, version: 111'283, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:03b9100b:::obj283:head@111'283, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:03553543:::obj227:head local-lis/les=117/118 n=10 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.944+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.944+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0366f813:::obj557:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.944+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:03553543:::obj227:head local-lis/les=117/118 n=10 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:03b9100b:::obj283:head, version: 111'283, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:03b9100b:::obj283:head@111'283, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.944+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:03553543:::obj227:head local-lis/les=117/118 n=10 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:03b9100b:::obj283:head@111'283, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.944+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.944+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:03553543:::obj227:head local-lis/les=117/118 n=10 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:03b9100b:::obj283:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.944+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0366f813:::obj557:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.944+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:03553543:::obj227:head local-lis/les=117/118 n=10 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:03b9100b:::obj283:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.944+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:03553543:::obj227:head local-lis/les=117/118 n=10 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:03b9100b:::obj283:head 2026-02-11T23:31:27.944+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:03553543:::obj227:head local-lis/les=117/118 n=10 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:03b9100b:::obj283:head v 111'283 2026-02-11T23:31:27.944+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:03553543:::obj227:head local-lis/les=117/118 n=10 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.944+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.944+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:03b9100b:::obj283:head, version: 111'283, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:03b9100b:::obj283:head@111'283, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.944+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.944+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.944+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0366f813:::obj557:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.944+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:0366f813:::obj557:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 4.0 e 118/118 lb 4:0366f813:::obj557:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:03553543:::obj227:head local-lis/les=117/118 n=10 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.944+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.944+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:03553543:::obj227:head local-lis/les=117/118 n=10 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:0366f813:::obj557:head) v3 2026-02-11T23:31:27.944+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:03553543:::obj227:head local-lis/les=117/118 n=10 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:0366f813:::obj557:head) 2026-02-11T23:31:27.944+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.944+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:0366f813:::obj557:head) v3 finish latency 0.000015 2026-02-11T23:31:27.944+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0366f813:::obj557:head local-lis/les=117/118 n=11 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.944+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.944+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.944+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0366f813:::obj557:head local-lis/les=117/118 n=11 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.944+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.944+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.944+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.944+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x5f75500 con 0x507ac00 2026-02-11T23:31:27.944+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:03b9100b:::obj283:head)]) -- 0x5d34780 con 0x507ac00 2026-02-11T23:31:27.944+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.944+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 62 ==== MOSDPGPush(4.0 118/117 [PushOp(4:040c73f9:::obj237:head, version: 111'237, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:040c73f9:::obj237:head@111'237, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2820 con 0x507ac00 2026-02-11T23:31:27.944+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:040c73f9:::obj237:head, version: 111'237, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:040c73f9:::obj237:head@111'237, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 118 2026-02-11T23:31:27.944+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:040c73f9:::obj237:head, version: 111'237, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:040c73f9:::obj237:head@111'237, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.944+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.944+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 63 ==== pg_backfill(progress 4.0 e 118/118 lb 4:03b9100b:::obj283:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98580 con 0x507ac00 2026-02-11T23:31:27.944+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:03b9100b:::obj283:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 118 2026-02-11T23:31:27.944+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:040c73f9:::obj237:head, version: 111'237, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:040c73f9:::obj237:head@111'237, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.944+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:03b9100b:::obj283:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.944+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.944+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:040c73f9:::obj237:head, version: 111'237, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:040c73f9:::obj237:head@111'237, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.944+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:040c73f9:::obj237:head, version: 111'237, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:040c73f9:::obj237:head@111'237, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(4.0 118/117 [PushOp(4:040c73f9:::obj237:head, version: 111'237, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:040c73f9:::obj237:head@111'237, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:0366f813:::obj557:head local-lis/les=117/118 n=11 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.944+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.944+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.944+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0366f813:::obj557:head local-lis/les=117/118 n=11 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:040c73f9:::obj237:head, version: 111'237, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:040c73f9:::obj237:head@111'237, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.944+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:03b9100b:::obj283:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.944+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0366f813:::obj557:head local-lis/les=117/118 n=11 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:040c73f9:::obj237:head@111'237, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.944+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0366f813:::obj557:head local-lis/les=117/118 n=11 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:040c73f9:::obj237:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.944+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0366f813:::obj557:head local-lis/les=117/118 n=11 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:040c73f9:::obj237:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.944+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0366f813:::obj557:head local-lis/les=117/118 n=11 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:040c73f9:::obj237:head 2026-02-11T23:31:27.944+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0366f813:::obj557:head local-lis/les=117/118 n=11 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:040c73f9:::obj237:head v 111'237 2026-02-11T23:31:27.944+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0366f813:::obj557:head local-lis/les=117/118 n=11 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.944+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.944+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:040c73f9:::obj237:head, version: 111'237, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:040c73f9:::obj237:head@111'237, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:27.944+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.944+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.944+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:03b9100b:::obj283:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.944+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:03b9100b:::obj283:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 4.0 e 118/118 lb 4:03b9100b:::obj283:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:0366f813:::obj557:head local-lis/les=117/118 n=11 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.944+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.944+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0366f813:::obj557:head local-lis/les=117/118 n=11 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:03b9100b:::obj283:head) v3 2026-02-11T23:31:27.944+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0366f813:::obj557:head local-lis/les=117/118 n=11 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:03b9100b:::obj283:head) 2026-02-11T23:31:27.944+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.944+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:03b9100b:::obj283:head) v3 finish latency 0.000009 2026-02-11T23:31:27.944+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:03b9100b:::obj283:head local-lis/les=117/118 n=12 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.944+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.944+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.944+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:03b9100b:::obj283:head local-lis/les=117/118 n=12 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.944+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.944+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.944+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.944+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x5f1d200 con 0x507ac00 2026-02-11T23:31:27.944+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:040c73f9:::obj237:head)]) -- 0x5d34a80 con 0x507ac00 2026-02-11T23:31:27.944+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.945+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 64 ==== MOSDPGPush(4.0 118/117 [PushOp(4:04510f17:::obj73:head, version: 111'73, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04510f17:::obj73:head@111'73, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x56a2d00 con 0x507ac00 2026-02-11T23:31:27.945+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:04510f17:::obj73:head, version: 111'73, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04510f17:::obj73:head@111'73, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 118 2026-02-11T23:31:27.945+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:04510f17:::obj73:head, version: 111'73, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04510f17:::obj73:head@111'73, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.945+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.945+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 65 ==== pg_backfill(progress 4.0 e 118/118 lb 4:040c73f9:::obj237:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98000 con 0x507ac00 2026-02-11T23:31:27.945+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:040c73f9:::obj237:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 118 2026-02-11T23:31:27.945+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:04510f17:::obj73:head, version: 111'73, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04510f17:::obj73:head@111'73, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.945+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:040c73f9:::obj237:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.945+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.945+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:04510f17:::obj73:head, version: 111'73, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04510f17:::obj73:head@111'73, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.945+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:04510f17:::obj73:head, version: 111'73, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04510f17:::obj73:head@111'73, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 118/117 [PushOp(4:04510f17:::obj73:head, version: 111'73, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04510f17:::obj73:head@111'73, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:03b9100b:::obj283:head local-lis/les=117/118 n=12 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.945+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.945+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.945+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:03b9100b:::obj283:head local-lis/les=117/118 n=12 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:04510f17:::obj73:head, version: 111'73, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04510f17:::obj73:head@111'73, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.945+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:040c73f9:::obj237:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.945+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:03b9100b:::obj283:head local-lis/les=117/118 n=12 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:04510f17:::obj73:head@111'73, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.945+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:03b9100b:::obj283:head local-lis/les=117/118 n=12 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:04510f17:::obj73:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.945+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:03b9100b:::obj283:head local-lis/les=117/118 n=12 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:04510f17:::obj73:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.945+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:03b9100b:::obj283:head local-lis/les=117/118 n=12 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:04510f17:::obj73:head 2026-02-11T23:31:27.945+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:03b9100b:::obj283:head local-lis/les=117/118 n=12 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:04510f17:::obj73:head v 111'73 2026-02-11T23:31:27.945+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:03b9100b:::obj283:head local-lis/les=117/118 n=12 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.945+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.945+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:04510f17:::obj73:head, version: 111'73, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04510f17:::obj73:head@111'73, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.945+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.945+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.945+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:040c73f9:::obj237:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.945+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:040c73f9:::obj237:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 4.0 e 118/118 lb 4:040c73f9:::obj237:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:03b9100b:::obj283:head local-lis/les=117/118 n=12 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.945+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.945+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:03b9100b:::obj283:head local-lis/les=117/118 n=12 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:040c73f9:::obj237:head) v3 2026-02-11T23:31:27.945+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:03b9100b:::obj283:head local-lis/les=117/118 n=12 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:040c73f9:::obj237:head) 2026-02-11T23:31:27.945+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.945+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:040c73f9:::obj237:head) v3 finish latency 0.000010 2026-02-11T23:31:27.945+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:040c73f9:::obj237:head local-lis/les=117/118 n=13 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.945+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.945+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.945+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:040c73f9:::obj237:head local-lis/les=117/118 n=13 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.945+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.945+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.945+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.945+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x5f75380 con 0x507ac00 2026-02-11T23:31:27.945+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:04510f17:::obj73:head)]) -- 0x5d34d80 con 0x507ac00 2026-02-11T23:31:27.945+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.945+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 66 ==== MOSDPGPush(4.0 118/117 [PushOp(4:0485ab12:::obj244:head, version: 111'244, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0485ab12:::obj244:head@111'244, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56ca4e0 con 0x507ac00 2026-02-11T23:31:27.945+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:0485ab12:::obj244:head, version: 111'244, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0485ab12:::obj244:head@111'244, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 118 2026-02-11T23:31:27.945+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0485ab12:::obj244:head, version: 111'244, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0485ab12:::obj244:head@111'244, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.945+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.945+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 67 ==== pg_backfill(progress 4.0 e 118/118 lb 4:04510f17:::obj73:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d9b180 con 0x507ac00 2026-02-11T23:31:27.945+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0485ab12:::obj244:head, version: 111'244, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0485ab12:::obj244:head@111'244, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.945+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:04510f17:::obj73:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:27.945+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:04510f17:::obj73:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.945+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.945+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0485ab12:::obj244:head, version: 111'244, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0485ab12:::obj244:head@111'244, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.945+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:0485ab12:::obj244:head, version: 111'244, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0485ab12:::obj244:head@111'244, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 118/117 [PushOp(4:0485ab12:::obj244:head, version: 111'244, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0485ab12:::obj244:head@111'244, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:040c73f9:::obj237:head local-lis/les=117/118 n=13 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.945+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.945+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.945+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:04510f17:::obj73:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.945+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:040c73f9:::obj237:head local-lis/les=117/118 n=13 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:0485ab12:::obj244:head, version: 111'244, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0485ab12:::obj244:head@111'244, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.945+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:040c73f9:::obj237:head local-lis/les=117/118 n=13 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:0485ab12:::obj244:head@111'244, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.945+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:040c73f9:::obj237:head local-lis/les=117/118 n=13 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:0485ab12:::obj244:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.945+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:040c73f9:::obj237:head local-lis/les=117/118 n=13 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:0485ab12:::obj244:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.945+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:040c73f9:::obj237:head local-lis/les=117/118 n=13 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:0485ab12:::obj244:head 2026-02-11T23:31:27.945+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:040c73f9:::obj237:head local-lis/les=117/118 n=13 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:0485ab12:::obj244:head v 111'244 2026-02-11T23:31:27.945+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:040c73f9:::obj237:head local-lis/les=117/118 n=13 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.945+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.946+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:0485ab12:::obj244:head, version: 111'244, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0485ab12:::obj244:head@111'244, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:27.946+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.946+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.946+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:04510f17:::obj73:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.946+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:04510f17:::obj73:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 4.0 e 118/118 lb 4:04510f17:::obj73:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:040c73f9:::obj237:head local-lis/les=117/118 n=13 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.946+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.946+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:040c73f9:::obj237:head local-lis/les=117/118 n=13 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:04510f17:::obj73:head) v3 2026-02-11T23:31:27.946+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:040c73f9:::obj237:head local-lis/les=117/118 n=13 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:04510f17:::obj73:head) 2026-02-11T23:31:27.946+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.946+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:04510f17:::obj73:head) v3 finish latency 0.000010 2026-02-11T23:31:27.946+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:04510f17:::obj73:head local-lis/les=117/118 n=14 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.946+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.946+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.946+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:04510f17:::obj73:head local-lis/les=117/118 n=14 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.946+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.946+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.946+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.946+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x5f2c900 con 0x507ac00 2026-02-11T23:31:27.946+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:0485ab12:::obj244:head)]) -- 0x5d34f00 con 0x507ac00 2026-02-11T23:31:27.946+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.946+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 68 ==== MOSDPGPush(4.0 118/117 [PushOp(4:04865490:::obj600:head, version: 111'600, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04865490:::obj600:head@111'600, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a21a0 con 0x507ac00 2026-02-11T23:31:27.946+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:04865490:::obj600:head, version: 111'600, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04865490:::obj600:head@111'600, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 118 2026-02-11T23:31:27.946+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:04865490:::obj600:head, version: 111'600, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04865490:::obj600:head@111'600, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.946+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.946+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 69 ==== pg_backfill(progress 4.0 e 118/118 lb 4:0485ab12:::obj244:head) ==== 1062+0+0 (unknown 0 0 0) 0x6178c00 con 0x507ac00 2026-02-11T23:31:27.946+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:0485ab12:::obj244:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:27.946+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:04865490:::obj600:head, version: 111'600, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04865490:::obj600:head@111'600, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.946+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0485ab12:::obj244:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.946+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.946+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:04865490:::obj600:head, version: 111'600, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04865490:::obj600:head@111'600, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.946+0000 7f414f993640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:04865490:::obj600:head, version: 111'600, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04865490:::obj600:head@111'600, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 118/117 [PushOp(4:04865490:::obj600:head, version: 111'600, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04865490:::obj600:head@111'600, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:04510f17:::obj73:head local-lis/les=117/118 n=14 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.946+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.946+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:04510f17:::obj73:head local-lis/les=117/118 n=14 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:04865490:::obj600:head, version: 111'600, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04865490:::obj600:head@111'600, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.946+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.946+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0485ab12:::obj244:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.946+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:04510f17:::obj73:head local-lis/les=117/118 n=14 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:04865490:::obj600:head@111'600, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.946+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:04510f17:::obj73:head local-lis/les=117/118 n=14 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:04865490:::obj600:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.946+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:04510f17:::obj73:head local-lis/les=117/118 n=14 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:04865490:::obj600:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.946+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:04510f17:::obj73:head local-lis/les=117/118 n=14 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:04865490:::obj600:head 2026-02-11T23:31:27.946+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:04510f17:::obj73:head local-lis/les=117/118 n=14 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:04865490:::obj600:head v 111'600 2026-02-11T23:31:27.946+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:04510f17:::obj73:head local-lis/les=117/118 n=14 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.946+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.946+0000 7f414f993640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:04865490:::obj600:head, version: 111'600, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04865490:::obj600:head@111'600, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.946+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:04510f17:::obj73:head local-lis/les=117/118 n=14 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.946+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.946+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.946+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0485ab12:::obj244:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.946+0000 7f414b98b640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:0485ab12:::obj244:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 118/118 lb 4:0485ab12:::obj244:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:04510f17:::obj73:head local-lis/les=117/118 n=14 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.946+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.946+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:04510f17:::obj73:head local-lis/les=117/118 n=14 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:0485ab12:::obj244:head) v3 2026-02-11T23:31:27.946+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:04510f17:::obj73:head local-lis/les=117/118 n=14 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:0485ab12:::obj244:head) 2026-02-11T23:31:27.946+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.946+0000 7f414b98b640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:0485ab12:::obj244:head) v3 finish latency 0.000012 2026-02-11T23:31:27.946+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.946+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.946+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0485ab12:::obj244:head local-lis/les=117/118 n=15 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.946+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.946+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.946+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.946+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x5f75080 con 0x507ac00 2026-02-11T23:31:27.946+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:04865490:::obj600:head)]) -- 0x5f75200 con 0x507ac00 2026-02-11T23:31:27.946+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.947+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 70 ==== MOSDPGPush(4.0 118/117 [PushOp(4:048f8fd3:::obj518:head, version: 111'518, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:048f8fd3:::obj518:head@111'518, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x554e340 con 0x507ac00 2026-02-11T23:31:27.947+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:048f8fd3:::obj518:head, version: 111'518, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:048f8fd3:::obj518:head@111'518, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 118 2026-02-11T23:31:27.947+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:048f8fd3:::obj518:head, version: 111'518, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:048f8fd3:::obj518:head@111'518, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.947+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.947+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 71 ==== pg_backfill(progress 4.0 e 118/118 lb 4:04865490:::obj600:head) ==== 1062+0+0 (unknown 0 0 0) 0x6179180 con 0x507ac00 2026-02-11T23:31:27.947+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:04865490:::obj600:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:27.947+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:048f8fd3:::obj518:head, version: 111'518, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:048f8fd3:::obj518:head@111'518, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.947+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:04865490:::obj600:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.947+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.947+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:048f8fd3:::obj518:head, version: 111'518, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:048f8fd3:::obj518:head@111'518, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.947+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:048f8fd3:::obj518:head, version: 111'518, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:048f8fd3:::obj518:head@111'518, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(4.0 118/117 [PushOp(4:048f8fd3:::obj518:head, version: 111'518, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:048f8fd3:::obj518:head@111'518, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:0485ab12:::obj244:head local-lis/les=117/118 n=15 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.947+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.947+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.947+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0485ab12:::obj244:head local-lis/les=117/118 n=15 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:048f8fd3:::obj518:head, version: 111'518, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:048f8fd3:::obj518:head@111'518, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.947+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:04865490:::obj600:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.947+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0485ab12:::obj244:head local-lis/les=117/118 n=15 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:048f8fd3:::obj518:head@111'518, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.947+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0485ab12:::obj244:head local-lis/les=117/118 n=15 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:048f8fd3:::obj518:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.947+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0485ab12:::obj244:head local-lis/les=117/118 n=15 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:048f8fd3:::obj518:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.947+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0485ab12:::obj244:head local-lis/les=117/118 n=15 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:048f8fd3:::obj518:head 2026-02-11T23:31:27.947+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0485ab12:::obj244:head local-lis/les=117/118 n=15 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:048f8fd3:::obj518:head v 111'518 2026-02-11T23:31:27.947+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0485ab12:::obj244:head local-lis/les=117/118 n=15 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.947+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.947+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:048f8fd3:::obj518:head, version: 111'518, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:048f8fd3:::obj518:head@111'518, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.947+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.947+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.947+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:04865490:::obj600:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.947+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:04865490:::obj600:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 4.0 e 118/118 lb 4:04865490:::obj600:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:0485ab12:::obj244:head local-lis/les=117/118 n=15 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.947+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.947+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0485ab12:::obj244:head local-lis/les=117/118 n=15 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:04865490:::obj600:head) v3 2026-02-11T23:31:27.947+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0485ab12:::obj244:head local-lis/les=117/118 n=15 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:04865490:::obj600:head) 2026-02-11T23:31:27.947+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.947+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:04865490:::obj600:head) v3 finish latency 0.000011 2026-02-11T23:31:27.947+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:04865490:::obj600:head local-lis/les=117/118 n=16 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.947+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.947+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.947+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:04865490:::obj600:head local-lis/les=117/118 n=16 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.947+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.947+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.947+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.947+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x5f74000 con 0x507ac00 2026-02-11T23:31:27.947+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:048f8fd3:::obj518:head)]) -- 0x5d35200 con 0x507ac00 2026-02-11T23:31:27.947+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.948+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 72 ==== MOSDPGPush(4.0 118/117 [PushOp(4:04b71422:::obj226:head, version: 111'226, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04b71422:::obj226:head@111'226, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a3040 con 0x507ac00 2026-02-11T23:31:27.948+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:04b71422:::obj226:head, version: 111'226, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04b71422:::obj226:head@111'226, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 118 2026-02-11T23:31:27.948+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:04b71422:::obj226:head, version: 111'226, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04b71422:::obj226:head@111'226, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.948+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.948+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 73 ==== pg_backfill(progress 4.0 e 118/118 lb 4:048f8fd3:::obj518:head) ==== 1062+0+0 (unknown 0 0 0) 0x6178100 con 0x507ac00 2026-02-11T23:31:27.948+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:04b71422:::obj226:head, version: 111'226, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04b71422:::obj226:head@111'226, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.948+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:048f8fd3:::obj518:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:27.948+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.948+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:048f8fd3:::obj518:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.948+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:04b71422:::obj226:head, version: 111'226, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04b71422:::obj226:head@111'226, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.948+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.948+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:04b71422:::obj226:head, version: 111'226, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04b71422:::obj226:head@111'226, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 118/117 [PushOp(4:04b71422:::obj226:head, version: 111'226, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04b71422:::obj226:head@111'226, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:04865490:::obj600:head local-lis/les=117/118 n=16 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.948+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.948+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:04865490:::obj600:head local-lis/les=117/118 n=16 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:04b71422:::obj226:head, version: 111'226, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04b71422:::obj226:head@111'226, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.948+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:04865490:::obj600:head local-lis/les=117/118 n=16 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:04b71422:::obj226:head@111'226, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.948+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.948+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:048f8fd3:::obj518:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.948+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:04865490:::obj600:head local-lis/les=117/118 n=16 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:04b71422:::obj226:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.948+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:04865490:::obj600:head local-lis/les=117/118 n=16 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:04b71422:::obj226:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.948+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:04865490:::obj600:head local-lis/les=117/118 n=16 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:04b71422:::obj226:head 2026-02-11T23:31:27.948+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:04865490:::obj600:head local-lis/les=117/118 n=16 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:04b71422:::obj226:head v 111'226 2026-02-11T23:31:27.948+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:04865490:::obj600:head local-lis/les=117/118 n=16 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.948+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.948+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:04b71422:::obj226:head, version: 111'226, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04b71422:::obj226:head@111'226, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.948+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.948+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.948+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:048f8fd3:::obj518:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.948+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:048f8fd3:::obj518:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 4.0 e 118/118 lb 4:048f8fd3:::obj518:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:04865490:::obj600:head local-lis/les=117/118 n=16 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.948+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.948+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:04865490:::obj600:head local-lis/les=117/118 n=16 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:048f8fd3:::obj518:head) v3 2026-02-11T23:31:27.948+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:04865490:::obj600:head local-lis/les=117/118 n=16 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:048f8fd3:::obj518:head) 2026-02-11T23:31:27.948+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.948+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:048f8fd3:::obj518:head) v3 finish latency 0.000011 2026-02-11T23:31:27.948+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:048f8fd3:::obj518:head local-lis/les=117/118 n=17 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.948+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.948+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.948+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:048f8fd3:::obj518:head local-lis/les=117/118 n=17 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.948+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.948+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.948+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.948+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x5f74f00 con 0x507ac00 2026-02-11T23:31:27.948+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:04b71422:::obj226:head)]) -- 0x5d35500 con 0x507ac00 2026-02-11T23:31:27.948+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.948+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 74 ==== MOSDPGPush(4.0 118/117 [PushOp(4:05651d4b:::obj461:head, version: 111'461, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05651d4b:::obj461:head@111'461, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a3860 con 0x507ac00 2026-02-11T23:31:27.948+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:05651d4b:::obj461:head, version: 111'461, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05651d4b:::obj461:head@111'461, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 118 2026-02-11T23:31:27.948+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:05651d4b:::obj461:head, version: 111'461, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05651d4b:::obj461:head@111'461, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.948+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.948+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 75 ==== pg_backfill(progress 4.0 e 118/118 lb 4:04b71422:::obj226:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177b80 con 0x507ac00 2026-02-11T23:31:27.948+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:04b71422:::obj226:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 118 2026-02-11T23:31:27.948+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:05651d4b:::obj461:head, version: 111'461, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05651d4b:::obj461:head@111'461, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.948+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:04b71422:::obj226:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.948+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.948+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:05651d4b:::obj461:head, version: 111'461, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05651d4b:::obj461:head@111'461, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.948+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:05651d4b:::obj461:head, version: 111'461, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05651d4b:::obj461:head@111'461, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 118/117 [PushOp(4:05651d4b:::obj461:head, version: 111'461, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05651d4b:::obj461:head@111'461, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:048f8fd3:::obj518:head local-lis/les=117/118 n=17 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.948+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.948+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.948+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:048f8fd3:::obj518:head local-lis/les=117/118 n=17 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:05651d4b:::obj461:head, version: 111'461, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05651d4b:::obj461:head@111'461, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.948+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:04b71422:::obj226:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.948+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:048f8fd3:::obj518:head local-lis/les=117/118 n=17 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:05651d4b:::obj461:head@111'461, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.948+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:048f8fd3:::obj518:head local-lis/les=117/118 n=17 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:05651d4b:::obj461:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.948+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:048f8fd3:::obj518:head local-lis/les=117/118 n=17 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:05651d4b:::obj461:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.948+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:048f8fd3:::obj518:head local-lis/les=117/118 n=17 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:05651d4b:::obj461:head 2026-02-11T23:31:27.948+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:048f8fd3:::obj518:head local-lis/les=117/118 n=17 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:05651d4b:::obj461:head v 111'461 2026-02-11T23:31:27.948+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:048f8fd3:::obj518:head local-lis/les=117/118 n=17 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.948+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.948+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:05651d4b:::obj461:head, version: 111'461, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05651d4b:::obj461:head@111'461, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000043 2026-02-11T23:31:27.948+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.948+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.948+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:04b71422:::obj226:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.948+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:04b71422:::obj226:head) v3 prio 15 cost 0 latency 0.000059 pg_backfill(progress 4.0 e 118/118 lb 4:04b71422:::obj226:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:048f8fd3:::obj518:head local-lis/les=117/118 n=17 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.948+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.948+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:048f8fd3:::obj518:head local-lis/les=117/118 n=17 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:04b71422:::obj226:head) v3 2026-02-11T23:31:27.948+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:048f8fd3:::obj518:head local-lis/les=117/118 n=17 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:04b71422:::obj226:head) 2026-02-11T23:31:27.948+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.948+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:04b71422:::obj226:head) v3 finish latency 0.000011 2026-02-11T23:31:27.948+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:04b71422:::obj226:head local-lis/les=117/118 n=18 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.948+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.948+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.948+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:04b71422:::obj226:head local-lis/les=117/118 n=18 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.948+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.948+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.948+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.948+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x5f74180 con 0x507ac00 2026-02-11T23:31:27.948+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:05651d4b:::obj461:head)]) -- 0x5d35800 con 0x507ac00 2026-02-11T23:31:27.948+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.949+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 76 ==== MOSDPGPush(4.0 118/117 [PushOp(4:05793669:::obj415:head, version: 111'415, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05793669:::obj415:head@111'415, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5617380 con 0x507ac00 2026-02-11T23:31:27.949+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:05793669:::obj415:head, version: 111'415, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05793669:::obj415:head@111'415, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 118 2026-02-11T23:31:27.949+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:05793669:::obj415:head, version: 111'415, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05793669:::obj415:head@111'415, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.949+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.949+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 77 ==== pg_backfill(progress 4.0 e 118/118 lb 4:05651d4b:::obj461:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177600 con 0x507ac00 2026-02-11T23:31:27.949+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:05793669:::obj415:head, version: 111'415, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05793669:::obj415:head@111'415, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.949+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:05651d4b:::obj461:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 118 2026-02-11T23:31:27.949+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.949+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:05651d4b:::obj461:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.949+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:05793669:::obj415:head, version: 111'415, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05793669:::obj415:head@111'415, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.949+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.949+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:05793669:::obj415:head, version: 111'415, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05793669:::obj415:head@111'415, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 118/117 [PushOp(4:05793669:::obj415:head, version: 111'415, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05793669:::obj415:head@111'415, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:04b71422:::obj226:head local-lis/les=117/118 n=18 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.949+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.949+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:04b71422:::obj226:head local-lis/les=117/118 n=18 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:05793669:::obj415:head, version: 111'415, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05793669:::obj415:head@111'415, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.949+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:04b71422:::obj226:head local-lis/les=117/118 n=18 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:05793669:::obj415:head@111'415, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.949+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.949+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:05651d4b:::obj461:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.949+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:04b71422:::obj226:head local-lis/les=117/118 n=18 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:05793669:::obj415:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.949+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:04b71422:::obj226:head local-lis/les=117/118 n=18 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:05793669:::obj415:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.949+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:04b71422:::obj226:head local-lis/les=117/118 n=18 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:05793669:::obj415:head 2026-02-11T23:31:27.949+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:04b71422:::obj226:head local-lis/les=117/118 n=18 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:05793669:::obj415:head v 111'415 2026-02-11T23:31:27.949+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:04b71422:::obj226:head local-lis/les=117/118 n=18 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.949+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.949+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:05793669:::obj415:head, version: 111'415, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05793669:::obj415:head@111'415, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.949+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.949+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.949+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:05651d4b:::obj461:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.949+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:05651d4b:::obj461:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 4.0 e 118/118 lb 4:05651d4b:::obj461:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:04b71422:::obj226:head local-lis/les=117/118 n=18 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.949+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.949+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:04b71422:::obj226:head local-lis/les=117/118 n=18 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:05651d4b:::obj461:head) v3 2026-02-11T23:31:27.949+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:04b71422:::obj226:head local-lis/les=117/118 n=18 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:05651d4b:::obj461:head) 2026-02-11T23:31:27.949+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.949+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:05651d4b:::obj461:head) v3 finish latency 0.000010 2026-02-11T23:31:27.949+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:05651d4b:::obj461:head local-lis/les=117/118 n=19 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.949+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.949+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.949+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:05651d4b:::obj461:head local-lis/les=117/118 n=19 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.949+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.949+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.949+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.949+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x5f74d80 con 0x507ac00 2026-02-11T23:31:27.949+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:05793669:::obj415:head)]) -- 0x5d35b00 con 0x507ac00 2026-02-11T23:31:27.949+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.949+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 78 ==== MOSDPGPush(4.0 118/117 [PushOp(4:05e4a07c:::obj265:head, version: 111'265, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05e4a07c:::obj265:head@111'265, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56caea0 con 0x507ac00 2026-02-11T23:31:27.949+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:05e4a07c:::obj265:head, version: 111'265, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05e4a07c:::obj265:head@111'265, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 118 2026-02-11T23:31:27.949+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:05e4a07c:::obj265:head, version: 111'265, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05e4a07c:::obj265:head@111'265, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.949+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.949+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 79 ==== pg_backfill(progress 4.0 e 118/118 lb 4:05793669:::obj415:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177080 con 0x507ac00 2026-02-11T23:31:27.949+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:05793669:::obj415:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:27.949+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:05e4a07c:::obj265:head, version: 111'265, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05e4a07c:::obj265:head@111'265, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.949+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:05793669:::obj415:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.949+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.949+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:05e4a07c:::obj265:head, version: 111'265, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05e4a07c:::obj265:head@111'265, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.949+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:05e4a07c:::obj265:head, version: 111'265, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05e4a07c:::obj265:head@111'265, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 118/117 [PushOp(4:05e4a07c:::obj265:head, version: 111'265, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05e4a07c:::obj265:head@111'265, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:05651d4b:::obj461:head local-lis/les=117/118 n=19 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.949+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.949+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.949+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:05651d4b:::obj461:head local-lis/les=117/118 n=19 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:05e4a07c:::obj265:head, version: 111'265, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05e4a07c:::obj265:head@111'265, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.949+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:05793669:::obj415:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.949+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:05651d4b:::obj461:head local-lis/les=117/118 n=19 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:05e4a07c:::obj265:head@111'265, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.949+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:05651d4b:::obj461:head local-lis/les=117/118 n=19 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:05e4a07c:::obj265:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.949+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:05651d4b:::obj461:head local-lis/les=117/118 n=19 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:05e4a07c:::obj265:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.949+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:05651d4b:::obj461:head local-lis/les=117/118 n=19 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:05e4a07c:::obj265:head 2026-02-11T23:31:27.949+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:05651d4b:::obj461:head local-lis/les=117/118 n=19 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:05e4a07c:::obj265:head v 111'265 2026-02-11T23:31:27.949+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:05651d4b:::obj461:head local-lis/les=117/118 n=19 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.949+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.950+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:05e4a07c:::obj265:head, version: 111'265, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05e4a07c:::obj265:head@111'265, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:27.950+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.950+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.950+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:05793669:::obj415:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.950+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:05793669:::obj415:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 4.0 e 118/118 lb 4:05793669:::obj415:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:05651d4b:::obj461:head local-lis/les=117/118 n=19 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.950+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.950+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:05651d4b:::obj461:head local-lis/les=117/118 n=19 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:05793669:::obj415:head) v3 2026-02-11T23:31:27.950+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:05651d4b:::obj461:head local-lis/les=117/118 n=19 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:05793669:::obj415:head) 2026-02-11T23:31:27.950+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.950+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:05793669:::obj415:head) v3 finish latency 0.000010 2026-02-11T23:31:27.950+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:05793669:::obj415:head local-lis/les=117/118 n=20 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.950+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.950+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.950+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:05793669:::obj415:head local-lis/les=117/118 n=20 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.950+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.950+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.950+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.950+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x5f74300 con 0x507ac00 2026-02-11T23:31:27.950+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:05e4a07c:::obj265:head)]) -- 0x5d35e00 con 0x507ac00 2026-02-11T23:31:27.950+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.950+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 80 ==== MOSDPGPush(4.0 118/117 [PushOp(4:05eb3c5d:::obj514:head, version: 111'514, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05eb3c5d:::obj514:head@111'514, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56ca9c0 con 0x507ac00 2026-02-11T23:31:27.950+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:05eb3c5d:::obj514:head, version: 111'514, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05eb3c5d:::obj514:head@111'514, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 118 2026-02-11T23:31:27.950+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:05eb3c5d:::obj514:head, version: 111'514, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05eb3c5d:::obj514:head@111'514, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.950+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.950+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 81 ==== pg_backfill(progress 4.0 e 118/118 lb 4:05e4a07c:::obj265:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176b00 con 0x507ac00 2026-02-11T23:31:27.950+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:05e4a07c:::obj265:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 118 2026-02-11T23:31:27.950+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:05eb3c5d:::obj514:head, version: 111'514, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05eb3c5d:::obj514:head@111'514, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.950+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:05e4a07c:::obj265:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.950+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.950+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:05e4a07c:::obj265:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.950+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.950+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:05eb3c5d:::obj514:head, version: 111'514, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05eb3c5d:::obj514:head@111'514, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.950+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:05eb3c5d:::obj514:head, version: 111'514, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05eb3c5d:::obj514:head@111'514, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000043 MOSDPGPush(4.0 118/117 [PushOp(4:05eb3c5d:::obj514:head, version: 111'514, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05eb3c5d:::obj514:head@111'514, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:05793669:::obj415:head local-lis/les=117/118 n=20 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.950+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.950+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:05793669:::obj415:head local-lis/les=117/118 n=20 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:05eb3c5d:::obj514:head, version: 111'514, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05eb3c5d:::obj514:head@111'514, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.950+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:05793669:::obj415:head local-lis/les=117/118 n=20 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:05eb3c5d:::obj514:head@111'514, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.950+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:05793669:::obj415:head local-lis/les=117/118 n=20 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:05eb3c5d:::obj514:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.950+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:05793669:::obj415:head local-lis/les=117/118 n=20 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:05eb3c5d:::obj514:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.950+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:05793669:::obj415:head local-lis/les=117/118 n=20 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:05eb3c5d:::obj514:head 2026-02-11T23:31:27.950+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:05793669:::obj415:head local-lis/les=117/118 n=20 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:05eb3c5d:::obj514:head v 111'514 2026-02-11T23:31:27.950+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:05793669:::obj415:head local-lis/les=117/118 n=20 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.950+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.950+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:05eb3c5d:::obj514:head, version: 111'514, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05eb3c5d:::obj514:head@111'514, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:27.950+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.950+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.950+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:05e4a07c:::obj265:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.950+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:05e4a07c:::obj265:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 4.0 e 118/118 lb 4:05e4a07c:::obj265:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:05793669:::obj415:head local-lis/les=117/118 n=20 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.950+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.950+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:05793669:::obj415:head local-lis/les=117/118 n=20 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:05e4a07c:::obj265:head) v3 2026-02-11T23:31:27.950+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:05793669:::obj415:head local-lis/les=117/118 n=20 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:05e4a07c:::obj265:head) 2026-02-11T23:31:27.950+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.950+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:05e4a07c:::obj265:head) v3 finish latency 0.000014 2026-02-11T23:31:27.950+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:05e4a07c:::obj265:head local-lis/les=117/118 n=21 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.950+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.950+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.950+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:05e4a07c:::obj265:head local-lis/les=117/118 n=21 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.950+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.950+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.950+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.950+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x5f74c00 con 0x507ac00 2026-02-11T23:31:27.950+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:05eb3c5d:::obj514:head)]) -- 0x5f74180 con 0x507ac00 2026-02-11T23:31:27.950+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.951+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 82 ==== MOSDPGPush(4.0 118/117 [PushOp(4:0698d46e:::obj173:head, version: 111'173, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0698d46e:::obj173:head@111'173, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f0d00 con 0x507ac00 2026-02-11T23:31:27.951+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:0698d46e:::obj173:head, version: 111'173, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0698d46e:::obj173:head@111'173, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 118 2026-02-11T23:31:27.951+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0698d46e:::obj173:head, version: 111'173, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0698d46e:::obj173:head@111'173, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.951+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.951+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 83 ==== pg_backfill(progress 4.0 e 118/118 lb 4:05eb3c5d:::obj514:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176000 con 0x507ac00 2026-02-11T23:31:27.951+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:05eb3c5d:::obj514:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 118 2026-02-11T23:31:27.951+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0698d46e:::obj173:head, version: 111'173, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0698d46e:::obj173:head@111'173, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.951+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:05eb3c5d:::obj514:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.951+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.951+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0698d46e:::obj173:head, version: 111'173, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0698d46e:::obj173:head@111'173, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.951+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:0698d46e:::obj173:head, version: 111'173, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0698d46e:::obj173:head@111'173, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 118/117 [PushOp(4:0698d46e:::obj173:head, version: 111'173, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0698d46e:::obj173:head@111'173, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:05e4a07c:::obj265:head local-lis/les=117/118 n=21 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.951+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.951+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.951+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:05eb3c5d:::obj514:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.951+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:05e4a07c:::obj265:head local-lis/les=117/118 n=21 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:0698d46e:::obj173:head, version: 111'173, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0698d46e:::obj173:head@111'173, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.951+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:05e4a07c:::obj265:head local-lis/les=117/118 n=21 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:0698d46e:::obj173:head@111'173, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.951+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:05e4a07c:::obj265:head local-lis/les=117/118 n=21 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:0698d46e:::obj173:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.951+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:05e4a07c:::obj265:head local-lis/les=117/118 n=21 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:0698d46e:::obj173:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.951+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:05e4a07c:::obj265:head local-lis/les=117/118 n=21 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:0698d46e:::obj173:head 2026-02-11T23:31:27.951+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:05e4a07c:::obj265:head local-lis/les=117/118 n=21 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:0698d46e:::obj173:head v 111'173 2026-02-11T23:31:27.951+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:05e4a07c:::obj265:head local-lis/les=117/118 n=21 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.951+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.951+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:0698d46e:::obj173:head, version: 111'173, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0698d46e:::obj173:head@111'173, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:27.951+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.951+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.951+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:05eb3c5d:::obj514:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.951+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:05eb3c5d:::obj514:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 4.0 e 118/118 lb 4:05eb3c5d:::obj514:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:05e4a07c:::obj265:head local-lis/les=117/118 n=21 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.951+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.951+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:05e4a07c:::obj265:head local-lis/les=117/118 n=21 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:05eb3c5d:::obj514:head) v3 2026-02-11T23:31:27.951+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:05e4a07c:::obj265:head local-lis/les=117/118 n=21 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:05eb3c5d:::obj514:head) 2026-02-11T23:31:27.951+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.951+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:05eb3c5d:::obj514:head) v3 finish latency 0.000012 2026-02-11T23:31:27.951+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:05eb3c5d:::obj514:head local-lis/les=117/118 n=22 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.951+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.951+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.951+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:05eb3c5d:::obj514:head local-lis/les=117/118 n=22 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.951+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.951+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.951+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.951+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x5f74480 con 0x507ac00 2026-02-11T23:31:27.951+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:0698d46e:::obj173:head)]) -- 0x5f74f00 con 0x507ac00 2026-02-11T23:31:27.951+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.951+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 84 ==== MOSDPGPush(4.0 118/117 [PushOp(4:06c5d68c:::obj504:head, version: 111'504, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:06c5d68c:::obj504:head@111'504, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f1ba0 con 0x507ac00 2026-02-11T23:31:27.951+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:06c5d68c:::obj504:head, version: 111'504, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:06c5d68c:::obj504:head@111'504, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 118 2026-02-11T23:31:27.951+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:06c5d68c:::obj504:head, version: 111'504, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:06c5d68c:::obj504:head@111'504, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.951+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.951+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 85 ==== pg_backfill(progress 4.0 e 118/118 lb 4:0698d46e:::obj173:head) ==== 1062+0+0 (unknown 0 0 0) 0x6179700 con 0x507ac00 2026-02-11T23:31:27.951+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:06c5d68c:::obj504:head, version: 111'504, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:06c5d68c:::obj504:head@111'504, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.951+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:0698d46e:::obj173:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 118 2026-02-11T23:31:27.951+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.951+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0698d46e:::obj173:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.951+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:06c5d68c:::obj504:head, version: 111'504, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:06c5d68c:::obj504:head@111'504, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.951+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.951+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:06c5d68c:::obj504:head, version: 111'504, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:06c5d68c:::obj504:head@111'504, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(4.0 118/117 [PushOp(4:06c5d68c:::obj504:head, version: 111'504, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:06c5d68c:::obj504:head@111'504, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:05eb3c5d:::obj514:head local-lis/les=117/118 n=22 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.951+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.951+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:05eb3c5d:::obj514:head local-lis/les=117/118 n=22 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:06c5d68c:::obj504:head, version: 111'504, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:06c5d68c:::obj504:head@111'504, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.951+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:05eb3c5d:::obj514:head local-lis/les=117/118 n=22 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:06c5d68c:::obj504:head@111'504, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.951+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.951+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0698d46e:::obj173:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.951+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:05eb3c5d:::obj514:head local-lis/les=117/118 n=22 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:06c5d68c:::obj504:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.951+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:05eb3c5d:::obj514:head local-lis/les=117/118 n=22 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:06c5d68c:::obj504:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.951+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:05eb3c5d:::obj514:head local-lis/les=117/118 n=22 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:06c5d68c:::obj504:head 2026-02-11T23:31:27.951+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:05eb3c5d:::obj514:head local-lis/les=117/118 n=22 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:06c5d68c:::obj504:head v 111'504 2026-02-11T23:31:27.951+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:05eb3c5d:::obj514:head local-lis/les=117/118 n=22 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.951+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.951+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:06c5d68c:::obj504:head, version: 111'504, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:06c5d68c:::obj504:head@111'504, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000044 2026-02-11T23:31:27.951+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.951+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.952+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0698d46e:::obj173:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.952+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:0698d46e:::obj173:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 4.0 e 118/118 lb 4:0698d46e:::obj173:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:05eb3c5d:::obj514:head local-lis/les=117/118 n=22 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.952+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.952+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:05eb3c5d:::obj514:head local-lis/les=117/118 n=22 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:0698d46e:::obj173:head) v3 2026-02-11T23:31:27.952+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:05eb3c5d:::obj514:head local-lis/les=117/118 n=22 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:0698d46e:::obj173:head) 2026-02-11T23:31:27.952+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.952+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:0698d46e:::obj173:head) v3 finish latency 0.000012 2026-02-11T23:31:27.952+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0698d46e:::obj173:head local-lis/les=117/118 n=23 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.952+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.952+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.952+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0698d46e:::obj173:head local-lis/les=117/118 n=23 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.952+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.952+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.952+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.952+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x5f74a80 con 0x507ac00 2026-02-11T23:31:27.952+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:06c5d68c:::obj504:head)]) -- 0x5f74000 con 0x507ac00 2026-02-11T23:31:27.952+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.952+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 86 ==== MOSDPGPush(4.0 118/117 [PushOp(4:06d56074:::obj162:head, version: 111'162, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:06d56074:::obj162:head@111'162, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f0b60 con 0x507ac00 2026-02-11T23:31:27.952+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:06d56074:::obj162:head, version: 111'162, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:06d56074:::obj162:head@111'162, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 118 2026-02-11T23:31:27.952+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:06d56074:::obj162:head, version: 111'162, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:06d56074:::obj162:head@111'162, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.952+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.952+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 87 ==== pg_backfill(progress 4.0 e 118/118 lb 4:06c5d68c:::obj504:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2cb00 con 0x507ac00 2026-02-11T23:31:27.952+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:06c5d68c:::obj504:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 118 2026-02-11T23:31:27.952+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:06d56074:::obj162:head, version: 111'162, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:06d56074:::obj162:head@111'162, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.952+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:06c5d68c:::obj504:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.952+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.952+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:06d56074:::obj162:head, version: 111'162, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:06d56074:::obj162:head@111'162, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.952+0000 7f414f993640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:06d56074:::obj162:head, version: 111'162, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:06d56074:::obj162:head@111'162, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 118/117 [PushOp(4:06d56074:::obj162:head, version: 111'162, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:06d56074:::obj162:head@111'162, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:0698d46e:::obj173:head local-lis/les=117/118 n=23 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.952+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.952+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.952+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0698d46e:::obj173:head local-lis/les=117/118 n=23 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:06d56074:::obj162:head, version: 111'162, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:06d56074:::obj162:head@111'162, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.952+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:06c5d68c:::obj504:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.952+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0698d46e:::obj173:head local-lis/les=117/118 n=23 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:06d56074:::obj162:head@111'162, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.952+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0698d46e:::obj173:head local-lis/les=117/118 n=23 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:06d56074:::obj162:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.952+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0698d46e:::obj173:head local-lis/les=117/118 n=23 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:06d56074:::obj162:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.952+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0698d46e:::obj173:head local-lis/les=117/118 n=23 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:06d56074:::obj162:head 2026-02-11T23:31:27.952+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0698d46e:::obj173:head local-lis/les=117/118 n=23 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:06d56074:::obj162:head v 111'162 2026-02-11T23:31:27.952+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0698d46e:::obj173:head local-lis/les=117/118 n=23 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.952+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.952+0000 7f414f993640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:06d56074:::obj162:head, version: 111'162, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:06d56074:::obj162:head@111'162, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:27.952+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0698d46e:::obj173:head local-lis/les=117/118 n=23 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.952+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.952+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.952+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:06c5d68c:::obj504:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.952+0000 7f414b98b640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:06c5d68c:::obj504:head) v3 prio 15 cost 0 latency 0.000084 pg_backfill(progress 4.0 e 118/118 lb 4:06c5d68c:::obj504:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:0698d46e:::obj173:head local-lis/les=117/118 n=23 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.952+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.952+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0698d46e:::obj173:head local-lis/les=117/118 n=23 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:06c5d68c:::obj504:head) v3 2026-02-11T23:31:27.952+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0698d46e:::obj173:head local-lis/les=117/118 n=23 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:06c5d68c:::obj504:head) 2026-02-11T23:31:27.952+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.952+0000 7f414b98b640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:06c5d68c:::obj504:head) v3 finish latency 0.000013 2026-02-11T23:31:27.952+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.952+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.952+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:06c5d68c:::obj504:head local-lis/les=117/118 n=24 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.952+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.952+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.952+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.952+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x5fcf800 con 0x507ac00 2026-02-11T23:31:27.952+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:06d56074:::obj162:head)]) -- 0x5fed380 con 0x507ac00 2026-02-11T23:31:27.952+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.953+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 88 ==== MOSDPGPush(4.0 118/117 [PushOp(4:07c44a4d:::obj181:head, version: 111'181, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:07c44a4d:::obj181:head@111'181, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559cb60 con 0x507ac00 2026-02-11T23:31:27.953+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:07c44a4d:::obj181:head, version: 111'181, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:07c44a4d:::obj181:head@111'181, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 118 2026-02-11T23:31:27.953+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:07c44a4d:::obj181:head, version: 111'181, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:07c44a4d:::obj181:head@111'181, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.953+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.953+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 89 ==== pg_backfill(progress 4.0 e 118/118 lb 4:06d56074:::obj162:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2c000 con 0x507ac00 2026-02-11T23:31:27.953+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:06d56074:::obj162:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:27.953+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:07c44a4d:::obj181:head, version: 111'181, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:07c44a4d:::obj181:head@111'181, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.953+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:06d56074:::obj162:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.953+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.953+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:07c44a4d:::obj181:head, version: 111'181, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:07c44a4d:::obj181:head@111'181, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.953+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:07c44a4d:::obj181:head, version: 111'181, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:07c44a4d:::obj181:head@111'181, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 118/117 [PushOp(4:07c44a4d:::obj181:head, version: 111'181, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:07c44a4d:::obj181:head@111'181, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:06c5d68c:::obj504:head local-lis/les=117/118 n=24 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.953+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.953+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:06c5d68c:::obj504:head local-lis/les=117/118 n=24 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:07c44a4d:::obj181:head, version: 111'181, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:07c44a4d:::obj181:head@111'181, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.953+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.953+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:06d56074:::obj162:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.953+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:06c5d68c:::obj504:head local-lis/les=117/118 n=24 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:07c44a4d:::obj181:head@111'181, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.953+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:06c5d68c:::obj504:head local-lis/les=117/118 n=24 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:07c44a4d:::obj181:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.953+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:06c5d68c:::obj504:head local-lis/les=117/118 n=24 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:07c44a4d:::obj181:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.953+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:06c5d68c:::obj504:head local-lis/les=117/118 n=24 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:07c44a4d:::obj181:head 2026-02-11T23:31:27.953+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:06c5d68c:::obj504:head local-lis/les=117/118 n=24 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:07c44a4d:::obj181:head v 111'181 2026-02-11T23:31:27.953+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:06c5d68c:::obj504:head local-lis/les=117/118 n=24 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.953+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.953+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:07c44a4d:::obj181:head, version: 111'181, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:07c44a4d:::obj181:head@111'181, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:27.953+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.953+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.953+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:06d56074:::obj162:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.953+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:06d56074:::obj162:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 4.0 e 118/118 lb 4:06d56074:::obj162:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:06c5d68c:::obj504:head local-lis/les=117/118 n=24 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.953+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.953+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:06c5d68c:::obj504:head local-lis/les=117/118 n=24 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:06d56074:::obj162:head) v3 2026-02-11T23:31:27.953+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:06c5d68c:::obj504:head local-lis/les=117/118 n=24 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:06d56074:::obj162:head) 2026-02-11T23:31:27.953+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.953+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:06d56074:::obj162:head) v3 finish latency 0.000013 2026-02-11T23:31:27.953+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:06d56074:::obj162:head local-lis/les=117/118 n=25 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.953+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.953+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.953+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:06d56074:::obj162:head local-lis/les=117/118 n=25 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.953+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.953+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.953+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.953+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x5f74780 con 0x507ac00 2026-02-11T23:31:27.953+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:07c44a4d:::obj181:head)]) -- 0x5f75080 con 0x507ac00 2026-02-11T23:31:27.953+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.953+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 90 ==== MOSDPGPush(4.0 118/117 [PushOp(4:080cd0cf:::obj146:head, version: 111'146, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:080cd0cf:::obj146:head@111'146, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e26680 con 0x507ac00 2026-02-11T23:31:27.953+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:080cd0cf:::obj146:head, version: 111'146, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:080cd0cf:::obj146:head@111'146, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 118 2026-02-11T23:31:27.953+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:080cd0cf:::obj146:head, version: 111'146, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:080cd0cf:::obj146:head@111'146, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.953+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.953+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 91 ==== pg_backfill(progress 4.0 e 118/118 lb 4:07c44a4d:::obj181:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e100 con 0x507ac00 2026-02-11T23:31:27.953+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:07c44a4d:::obj181:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 118 2026-02-11T23:31:27.953+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:080cd0cf:::obj146:head, version: 111'146, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:080cd0cf:::obj146:head@111'146, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.953+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:07c44a4d:::obj181:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.953+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.953+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:080cd0cf:::obj146:head, version: 111'146, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:080cd0cf:::obj146:head@111'146, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.953+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:080cd0cf:::obj146:head, version: 111'146, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:080cd0cf:::obj146:head@111'146, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 118/117 [PushOp(4:080cd0cf:::obj146:head, version: 111'146, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:080cd0cf:::obj146:head@111'146, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:06d56074:::obj162:head local-lis/les=117/118 n=25 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.953+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.953+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.953+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:06d56074:::obj162:head local-lis/les=117/118 n=25 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:080cd0cf:::obj146:head, version: 111'146, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:080cd0cf:::obj146:head@111'146, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.953+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:07c44a4d:::obj181:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.953+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:06d56074:::obj162:head local-lis/les=117/118 n=25 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:080cd0cf:::obj146:head@111'146, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.953+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:06d56074:::obj162:head local-lis/les=117/118 n=25 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:080cd0cf:::obj146:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.953+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:06d56074:::obj162:head local-lis/les=117/118 n=25 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:080cd0cf:::obj146:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.953+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:06d56074:::obj162:head local-lis/les=117/118 n=25 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:080cd0cf:::obj146:head 2026-02-11T23:31:27.953+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:06d56074:::obj162:head local-lis/les=117/118 n=25 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:080cd0cf:::obj146:head v 111'146 2026-02-11T23:31:27.953+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:06d56074:::obj162:head local-lis/les=117/118 n=25 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.953+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.953+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:080cd0cf:::obj146:head, version: 111'146, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:080cd0cf:::obj146:head@111'146, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:27.953+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.953+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.953+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:07c44a4d:::obj181:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.953+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:07c44a4d:::obj181:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 4.0 e 118/118 lb 4:07c44a4d:::obj181:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:06d56074:::obj162:head local-lis/les=117/118 n=25 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.953+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.953+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:06d56074:::obj162:head local-lis/les=117/118 n=25 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:07c44a4d:::obj181:head) v3 2026-02-11T23:31:27.953+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:06d56074:::obj162:head local-lis/les=117/118 n=25 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:07c44a4d:::obj181:head) 2026-02-11T23:31:27.953+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.953+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:07c44a4d:::obj181:head) v3 finish latency 0.000011 2026-02-11T23:31:27.953+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:07c44a4d:::obj181:head local-lis/les=117/118 n=26 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.953+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.954+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.954+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:07c44a4d:::obj181:head local-lis/les=117/118 n=26 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.954+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.954+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.954+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.954+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x600b680 con 0x507ac00 2026-02-11T23:31:27.954+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:080cd0cf:::obj146:head)]) -- 0x5f2c900 con 0x507ac00 2026-02-11T23:31:27.954+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.954+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 92 ==== MOSDPGPush(4.0 118/117 [PushOp(4:0877fdce:::obj530:head, version: 111'530, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0877fdce:::obj530:head@111'530, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5647380 con 0x507ac00 2026-02-11T23:31:27.954+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:0877fdce:::obj530:head, version: 111'530, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0877fdce:::obj530:head@111'530, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 118 2026-02-11T23:31:27.954+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0877fdce:::obj530:head, version: 111'530, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0877fdce:::obj530:head@111'530, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.954+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.954+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 93 ==== pg_backfill(progress 4.0 e 118/118 lb 4:080cd0cf:::obj146:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e680 con 0x507ac00 2026-02-11T23:31:27.954+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0877fdce:::obj530:head, version: 111'530, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0877fdce:::obj530:head@111'530, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.954+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:080cd0cf:::obj146:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:27.954+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:080cd0cf:::obj146:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.954+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.954+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0877fdce:::obj530:head, version: 111'530, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0877fdce:::obj530:head@111'530, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.954+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:0877fdce:::obj530:head, version: 111'530, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0877fdce:::obj530:head@111'530, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 118/117 [PushOp(4:0877fdce:::obj530:head, version: 111'530, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0877fdce:::obj530:head@111'530, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:07c44a4d:::obj181:head local-lis/les=117/118 n=26 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.954+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.954+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.954+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:07c44a4d:::obj181:head local-lis/les=117/118 n=26 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:0877fdce:::obj530:head, version: 111'530, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0877fdce:::obj530:head@111'530, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.954+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:080cd0cf:::obj146:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.954+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:07c44a4d:::obj181:head local-lis/les=117/118 n=26 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:0877fdce:::obj530:head@111'530, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.954+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:07c44a4d:::obj181:head local-lis/les=117/118 n=26 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:0877fdce:::obj530:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.954+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:07c44a4d:::obj181:head local-lis/les=117/118 n=26 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:0877fdce:::obj530:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.954+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:07c44a4d:::obj181:head local-lis/les=117/118 n=26 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:0877fdce:::obj530:head 2026-02-11T23:31:27.954+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:07c44a4d:::obj181:head local-lis/les=117/118 n=26 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:0877fdce:::obj530:head v 111'530 2026-02-11T23:31:27.954+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:07c44a4d:::obj181:head local-lis/les=117/118 n=26 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.954+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.954+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:0877fdce:::obj530:head, version: 111'530, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0877fdce:::obj530:head@111'530, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000044 2026-02-11T23:31:27.954+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.954+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.954+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:080cd0cf:::obj146:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.954+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:080cd0cf:::obj146:head) v3 prio 15 cost 0 latency 0.000059 pg_backfill(progress 4.0 e 118/118 lb 4:080cd0cf:::obj146:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:07c44a4d:::obj181:head local-lis/les=117/118 n=26 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.954+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.954+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:07c44a4d:::obj181:head local-lis/les=117/118 n=26 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:080cd0cf:::obj146:head) v3 2026-02-11T23:31:27.954+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:07c44a4d:::obj181:head local-lis/les=117/118 n=26 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:080cd0cf:::obj146:head) 2026-02-11T23:31:27.954+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.954+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:080cd0cf:::obj146:head) v3 finish latency 0.000009 2026-02-11T23:31:27.954+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:080cd0cf:::obj146:head local-lis/les=117/118 n=27 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.954+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.954+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.954+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:080cd0cf:::obj146:head local-lis/les=117/118 n=27 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.954+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.954+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.954+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.954+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x5f74600 con 0x507ac00 2026-02-11T23:31:27.954+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:0877fdce:::obj530:head)]) -- 0x5f75380 con 0x507ac00 2026-02-11T23:31:27.954+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.955+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 94 ==== MOSDPGPush(4.0 118/117 [PushOp(4:0881dee6:::obj412:head, version: 111'412, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0881dee6:::obj412:head@111'412, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3815a00 con 0x507ac00 2026-02-11T23:31:27.955+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:0881dee6:::obj412:head, version: 111'412, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0881dee6:::obj412:head@111'412, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 118 2026-02-11T23:31:27.955+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0881dee6:::obj412:head, version: 111'412, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0881dee6:::obj412:head@111'412, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.955+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.955+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 95 ==== pg_backfill(progress 4.0 e 118/118 lb 4:0877fdce:::obj530:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2d080 con 0x507ac00 2026-02-11T23:31:27.955+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0881dee6:::obj412:head, version: 111'412, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0881dee6:::obj412:head@111'412, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.955+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:0877fdce:::obj530:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:27.955+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0877fdce:::obj530:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.955+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.955+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0881dee6:::obj412:head, version: 111'412, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0881dee6:::obj412:head@111'412, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.955+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.955+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:0881dee6:::obj412:head, version: 111'412, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0881dee6:::obj412:head@111'412, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 118/117 [PushOp(4:0881dee6:::obj412:head, version: 111'412, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0881dee6:::obj412:head@111'412, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:080cd0cf:::obj146:head local-lis/les=117/118 n=27 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.955+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.955+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:080cd0cf:::obj146:head local-lis/les=117/118 n=27 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:0881dee6:::obj412:head, version: 111'412, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0881dee6:::obj412:head@111'412, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.955+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:080cd0cf:::obj146:head local-lis/les=117/118 n=27 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:0881dee6:::obj412:head@111'412, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.955+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.955+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0877fdce:::obj530:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.955+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:080cd0cf:::obj146:head local-lis/les=117/118 n=27 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:0881dee6:::obj412:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.955+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:080cd0cf:::obj146:head local-lis/les=117/118 n=27 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:0881dee6:::obj412:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.955+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:080cd0cf:::obj146:head local-lis/les=117/118 n=27 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:0881dee6:::obj412:head 2026-02-11T23:31:27.955+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:080cd0cf:::obj146:head local-lis/les=117/118 n=27 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:0881dee6:::obj412:head v 111'412 2026-02-11T23:31:27.955+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:080cd0cf:::obj146:head local-lis/les=117/118 n=27 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.955+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.955+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:0881dee6:::obj412:head, version: 111'412, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0881dee6:::obj412:head@111'412, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:27.955+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.955+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.955+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0877fdce:::obj530:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.955+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:0877fdce:::obj530:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 4.0 e 118/118 lb 4:0877fdce:::obj530:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:080cd0cf:::obj146:head local-lis/les=117/118 n=27 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.955+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.955+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:080cd0cf:::obj146:head local-lis/les=117/118 n=27 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:0877fdce:::obj530:head) v3 2026-02-11T23:31:27.955+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:080cd0cf:::obj146:head local-lis/les=117/118 n=27 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:0877fdce:::obj530:head) 2026-02-11T23:31:27.955+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.955+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:0877fdce:::obj530:head) v3 finish latency 0.000010 2026-02-11T23:31:27.955+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0877fdce:::obj530:head local-lis/les=117/118 n=28 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.955+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.955+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.955+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0877fdce:::obj530:head local-lis/les=117/118 n=28 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.955+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.955+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.955+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.955+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x5f75980 con 0x507ac00 2026-02-11T23:31:27.955+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:0881dee6:::obj412:head)]) -- 0x5f1d200 con 0x507ac00 2026-02-11T23:31:27.955+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.955+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 96 ==== MOSDPGPush(4.0 118/117 [PushOp(4:08a5234b:::obj62:head, version: 111'62, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:08a5234b:::obj62:head@111'62, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5646d00 con 0x507ac00 2026-02-11T23:31:27.955+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:08a5234b:::obj62:head, version: 111'62, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:08a5234b:::obj62:head@111'62, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 118 2026-02-11T23:31:27.955+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:08a5234b:::obj62:head, version: 111'62, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:08a5234b:::obj62:head@111'62, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.955+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.955+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 97 ==== pg_backfill(progress 4.0 e 118/118 lb 4:0881dee6:::obj412:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2f700 con 0x507ac00 2026-02-11T23:31:27.955+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:0881dee6:::obj412:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:27.955+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:08a5234b:::obj62:head, version: 111'62, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:08a5234b:::obj62:head@111'62, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.955+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0881dee6:::obj412:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.955+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.955+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:08a5234b:::obj62:head, version: 111'62, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:08a5234b:::obj62:head@111'62, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.955+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:08a5234b:::obj62:head, version: 111'62, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:08a5234b:::obj62:head@111'62, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 118/117 [PushOp(4:08a5234b:::obj62:head, version: 111'62, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:08a5234b:::obj62:head@111'62, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:0877fdce:::obj530:head local-lis/les=117/118 n=28 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.955+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.955+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.955+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0881dee6:::obj412:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.955+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0877fdce:::obj530:head local-lis/les=117/118 n=28 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:08a5234b:::obj62:head, version: 111'62, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:08a5234b:::obj62:head@111'62, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.955+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0877fdce:::obj530:head local-lis/les=117/118 n=28 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:08a5234b:::obj62:head@111'62, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.955+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0877fdce:::obj530:head local-lis/les=117/118 n=28 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:08a5234b:::obj62:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.955+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0877fdce:::obj530:head local-lis/les=117/118 n=28 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:08a5234b:::obj62:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.955+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0877fdce:::obj530:head local-lis/les=117/118 n=28 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:08a5234b:::obj62:head 2026-02-11T23:31:27.955+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0877fdce:::obj530:head local-lis/les=117/118 n=28 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:08a5234b:::obj62:head v 111'62 2026-02-11T23:31:27.955+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0877fdce:::obj530:head local-lis/les=117/118 n=28 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.955+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.955+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:08a5234b:::obj62:head, version: 111'62, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:08a5234b:::obj62:head@111'62, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:27.955+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.955+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.955+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0881dee6:::obj412:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.955+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:0881dee6:::obj412:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 4.0 e 118/118 lb 4:0881dee6:::obj412:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:0877fdce:::obj530:head local-lis/les=117/118 n=28 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.955+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.955+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0877fdce:::obj530:head local-lis/les=117/118 n=28 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:0881dee6:::obj412:head) v3 2026-02-11T23:31:27.955+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0877fdce:::obj530:head local-lis/les=117/118 n=28 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:0881dee6:::obj412:head) 2026-02-11T23:31:27.955+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.955+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:0881dee6:::obj412:head) v3 finish latency 0.000011 2026-02-11T23:31:27.955+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0881dee6:::obj412:head local-lis/les=117/118 n=29 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.955+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.956+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.956+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0881dee6:::obj412:head local-lis/les=117/118 n=29 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.956+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.956+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.956+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.956+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x649e000 con 0x507ac00 2026-02-11T23:31:27.956+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:08a5234b:::obj62:head)]) -- 0x5f75500 con 0x507ac00 2026-02-11T23:31:27.956+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.956+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 98 ==== MOSDPGPush(4.0 118/117 [PushOp(4:08fb0692:::obj136:head, version: 111'136, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:08fb0692:::obj136:head@111'136, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56cad00 con 0x507ac00 2026-02-11T23:31:27.956+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:08fb0692:::obj136:head, version: 111'136, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:08fb0692:::obj136:head@111'136, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 118 2026-02-11T23:31:27.956+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:08fb0692:::obj136:head, version: 111'136, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:08fb0692:::obj136:head@111'136, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.956+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.956+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 99 ==== pg_backfill(progress 4.0 e 118/118 lb 4:08a5234b:::obj62:head) ==== 1061+0+0 (unknown 0 0 0) 0x5f11600 con 0x507ac00 2026-02-11T23:31:27.956+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:08a5234b:::obj62:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:27.956+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:08fb0692:::obj136:head, version: 111'136, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:08fb0692:::obj136:head@111'136, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.956+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:08a5234b:::obj62:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.956+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.956+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:08fb0692:::obj136:head, version: 111'136, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:08fb0692:::obj136:head@111'136, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.956+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.956+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:08a5234b:::obj62:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.956+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:08fb0692:::obj136:head, version: 111'136, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:08fb0692:::obj136:head@111'136, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 118/117 [PushOp(4:08fb0692:::obj136:head, version: 111'136, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:08fb0692:::obj136:head@111'136, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:0881dee6:::obj412:head local-lis/les=117/118 n=29 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.956+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.956+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0881dee6:::obj412:head local-lis/les=117/118 n=29 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:08fb0692:::obj136:head, version: 111'136, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:08fb0692:::obj136:head@111'136, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.956+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0881dee6:::obj412:head local-lis/les=117/118 n=29 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:08fb0692:::obj136:head@111'136, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.956+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0881dee6:::obj412:head local-lis/les=117/118 n=29 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:08fb0692:::obj136:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.956+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0881dee6:::obj412:head local-lis/les=117/118 n=29 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:08fb0692:::obj136:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.956+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0881dee6:::obj412:head local-lis/les=117/118 n=29 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:08fb0692:::obj136:head 2026-02-11T23:31:27.956+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0881dee6:::obj412:head local-lis/les=117/118 n=29 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:08fb0692:::obj136:head v 111'136 2026-02-11T23:31:27.956+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0881dee6:::obj412:head local-lis/les=117/118 n=29 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.956+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.956+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:08fb0692:::obj136:head, version: 111'136, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:08fb0692:::obj136:head@111'136, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:27.956+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.956+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.956+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:08a5234b:::obj62:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.956+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:08a5234b:::obj62:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 4.0 e 118/118 lb 4:08a5234b:::obj62:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:0881dee6:::obj412:head local-lis/les=117/118 n=29 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.956+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.956+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0881dee6:::obj412:head local-lis/les=117/118 n=29 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:08a5234b:::obj62:head) v3 2026-02-11T23:31:27.956+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0881dee6:::obj412:head local-lis/les=117/118 n=29 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:08a5234b:::obj62:head) 2026-02-11T23:31:27.956+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.956+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:08a5234b:::obj62:head) v3 finish latency 0.000011 2026-02-11T23:31:27.956+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:08a5234b:::obj62:head local-lis/les=117/118 n=30 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.956+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.956+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.956+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:08a5234b:::obj62:head local-lis/les=117/118 n=30 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.956+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.956+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.956+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.956+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x649e180 con 0x507ac00 2026-02-11T23:31:27.956+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:08fb0692:::obj136:head)]) -- 0x5f1d500 con 0x507ac00 2026-02-11T23:31:27.956+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.957+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 100 ==== MOSDPGPush(4.0 118/117 [PushOp(4:095c6b7a:::obj79:head, version: 111'79, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:095c6b7a:::obj79:head@111'79, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x56171e0 con 0x507ac00 2026-02-11T23:31:27.957+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:095c6b7a:::obj79:head, version: 111'79, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:095c6b7a:::obj79:head@111'79, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 118 2026-02-11T23:31:27.957+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:095c6b7a:::obj79:head, version: 111'79, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:095c6b7a:::obj79:head@111'79, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.957+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.957+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:095c6b7a:::obj79:head, version: 111'79, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:095c6b7a:::obj79:head@111'79, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.957+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 101 ==== pg_backfill(progress 4.0 e 118/118 lb 4:08fb0692:::obj136:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f11080 con 0x507ac00 2026-02-11T23:31:27.957+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:08fb0692:::obj136:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 118 2026-02-11T23:31:27.957+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:08fb0692:::obj136:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.957+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.957+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:095c6b7a:::obj79:head, version: 111'79, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:095c6b7a:::obj79:head@111'79, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.957+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.957+0000 7f414f993640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:095c6b7a:::obj79:head, version: 111'79, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:095c6b7a:::obj79:head@111'79, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(4.0 118/117 [PushOp(4:095c6b7a:::obj79:head, version: 111'79, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:095c6b7a:::obj79:head@111'79, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:08a5234b:::obj62:head local-lis/les=117/118 n=30 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.957+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.957+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:08a5234b:::obj62:head local-lis/les=117/118 n=30 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:095c6b7a:::obj79:head, version: 111'79, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:095c6b7a:::obj79:head@111'79, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.957+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:08a5234b:::obj62:head local-lis/les=117/118 n=30 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:095c6b7a:::obj79:head@111'79, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.957+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.957+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:08fb0692:::obj136:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.957+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:08a5234b:::obj62:head local-lis/les=117/118 n=30 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:095c6b7a:::obj79:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.957+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:08a5234b:::obj62:head local-lis/les=117/118 n=30 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:095c6b7a:::obj79:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.957+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:08a5234b:::obj62:head local-lis/les=117/118 n=30 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:095c6b7a:::obj79:head 2026-02-11T23:31:27.957+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:08a5234b:::obj62:head local-lis/les=117/118 n=30 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:095c6b7a:::obj79:head v 111'79 2026-02-11T23:31:27.957+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:08a5234b:::obj62:head local-lis/les=117/118 n=30 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.957+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.957+0000 7f414f993640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:095c6b7a:::obj79:head, version: 111'79, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:095c6b7a:::obj79:head@111'79, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:27.957+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:08a5234b:::obj62:head local-lis/les=117/118 n=30 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.957+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.957+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.957+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:08fb0692:::obj136:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.957+0000 7f414b98b640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:08fb0692:::obj136:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 4.0 e 118/118 lb 4:08fb0692:::obj136:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:08a5234b:::obj62:head local-lis/les=117/118 n=30 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.957+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.957+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:08a5234b:::obj62:head local-lis/les=117/118 n=30 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:08fb0692:::obj136:head) v3 2026-02-11T23:31:27.957+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:08a5234b:::obj62:head local-lis/les=117/118 n=30 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:08fb0692:::obj136:head) 2026-02-11T23:31:27.957+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.957+0000 7f414b98b640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:08fb0692:::obj136:head) v3 finish latency 0.000012 2026-02-11T23:31:27.957+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.957+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.957+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:08fb0692:::obj136:head local-lis/les=117/118 n=31 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.957+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.957+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.957+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.957+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x649e480 con 0x507ac00 2026-02-11T23:31:27.957+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:095c6b7a:::obj79:head)]) -- 0x649e300 con 0x507ac00 2026-02-11T23:31:27.957+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.957+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 102 ==== MOSDPGPush(4.0 118/117 [PushOp(4:097a0595:::obj169:head, version: 111'169, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:097a0595:::obj169:head@111'169, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5617520 con 0x507ac00 2026-02-11T23:31:27.957+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:097a0595:::obj169:head, version: 111'169, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:097a0595:::obj169:head@111'169, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 118 2026-02-11T23:31:27.957+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:097a0595:::obj169:head, version: 111'169, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:097a0595:::obj169:head@111'169, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.957+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.957+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 103 ==== pg_backfill(progress 4.0 e 118/118 lb 4:095c6b7a:::obj79:head) ==== 1061+0+0 (unknown 0 0 0) 0x5f10b00 con 0x507ac00 2026-02-11T23:31:27.957+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:097a0595:::obj169:head, version: 111'169, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:097a0595:::obj169:head@111'169, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.957+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:095c6b7a:::obj79:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:27.957+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:095c6b7a:::obj79:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.957+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.957+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:097a0595:::obj169:head, version: 111'169, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:097a0595:::obj169:head@111'169, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.957+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:097a0595:::obj169:head, version: 111'169, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:097a0595:::obj169:head@111'169, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(4.0 118/117 [PushOp(4:097a0595:::obj169:head, version: 111'169, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:097a0595:::obj169:head@111'169, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:08fb0692:::obj136:head local-lis/les=117/118 n=31 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.957+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.957+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:08fb0692:::obj136:head local-lis/les=117/118 n=31 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:097a0595:::obj169:head, version: 111'169, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:097a0595:::obj169:head@111'169, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.957+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.957+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:08fb0692:::obj136:head local-lis/les=117/118 n=31 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:097a0595:::obj169:head@111'169, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.957+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:095c6b7a:::obj79:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.957+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:08fb0692:::obj136:head local-lis/les=117/118 n=31 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:097a0595:::obj169:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.957+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:08fb0692:::obj136:head local-lis/les=117/118 n=31 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:097a0595:::obj169:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.957+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:08fb0692:::obj136:head local-lis/les=117/118 n=31 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:097a0595:::obj169:head 2026-02-11T23:31:27.957+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:08fb0692:::obj136:head local-lis/les=117/118 n=31 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:097a0595:::obj169:head v 111'169 2026-02-11T23:31:27.957+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:08fb0692:::obj136:head local-lis/les=117/118 n=31 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.957+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.958+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:097a0595:::obj169:head, version: 111'169, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:097a0595:::obj169:head@111'169, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:27.958+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.958+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.958+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:095c6b7a:::obj79:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.958+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:095c6b7a:::obj79:head) v3 prio 15 cost 0 latency 0.000082 pg_backfill(progress 4.0 e 118/118 lb 4:095c6b7a:::obj79:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:08fb0692:::obj136:head local-lis/les=117/118 n=31 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.958+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.958+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:08fb0692:::obj136:head local-lis/les=117/118 n=31 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:095c6b7a:::obj79:head) v3 2026-02-11T23:31:27.958+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:08fb0692:::obj136:head local-lis/les=117/118 n=31 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:095c6b7a:::obj79:head) 2026-02-11T23:31:27.958+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.958+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:095c6b7a:::obj79:head) v3 finish latency 0.000020 2026-02-11T23:31:27.958+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:095c6b7a:::obj79:head local-lis/les=117/118 n=32 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.958+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.958+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.958+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:095c6b7a:::obj79:head local-lis/les=117/118 n=32 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.958+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.958+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.958+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.958+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x649e600 con 0x507ac00 2026-02-11T23:31:27.958+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:097a0595:::obj169:head)]) -- 0x5d34480 con 0x507ac00 2026-02-11T23:31:27.958+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.958+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 104 ==== MOSDPGPush(4.0 118/117 [PushOp(4:09d0cce6:::obj481:head, version: 111'481, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:09d0cce6:::obj481:head@111'481, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a3ba0 con 0x507ac00 2026-02-11T23:31:27.958+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:09d0cce6:::obj481:head, version: 111'481, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:09d0cce6:::obj481:head@111'481, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 118 2026-02-11T23:31:27.958+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:09d0cce6:::obj481:head, version: 111'481, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:09d0cce6:::obj481:head@111'481, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.958+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.958+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 105 ==== pg_backfill(progress 4.0 e 118/118 lb 4:097a0595:::obj169:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f10580 con 0x507ac00 2026-02-11T23:31:27.958+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:097a0595:::obj169:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 118 2026-02-11T23:31:27.958+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:09d0cce6:::obj481:head, version: 111'481, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:09d0cce6:::obj481:head@111'481, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.958+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:097a0595:::obj169:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.958+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.958+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:097a0595:::obj169:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.958+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.958+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:09d0cce6:::obj481:head, version: 111'481, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:09d0cce6:::obj481:head@111'481, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.958+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:09d0cce6:::obj481:head, version: 111'481, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:09d0cce6:::obj481:head@111'481, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000044 MOSDPGPush(4.0 118/117 [PushOp(4:09d0cce6:::obj481:head, version: 111'481, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:09d0cce6:::obj481:head@111'481, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:095c6b7a:::obj79:head local-lis/les=117/118 n=32 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.958+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.958+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:095c6b7a:::obj79:head local-lis/les=117/118 n=32 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:09d0cce6:::obj481:head, version: 111'481, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:09d0cce6:::obj481:head@111'481, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.958+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:095c6b7a:::obj79:head local-lis/les=117/118 n=32 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:09d0cce6:::obj481:head@111'481, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.958+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:095c6b7a:::obj79:head local-lis/les=117/118 n=32 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:09d0cce6:::obj481:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.958+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:095c6b7a:::obj79:head local-lis/les=117/118 n=32 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:09d0cce6:::obj481:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.958+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:095c6b7a:::obj79:head local-lis/les=117/118 n=32 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:09d0cce6:::obj481:head 2026-02-11T23:31:27.958+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:095c6b7a:::obj79:head local-lis/les=117/118 n=32 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:09d0cce6:::obj481:head v 111'481 2026-02-11T23:31:27.958+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:095c6b7a:::obj79:head local-lis/les=117/118 n=32 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.958+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.958+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:09d0cce6:::obj481:head, version: 111'481, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:09d0cce6:::obj481:head@111'481, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:27.958+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.958+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.958+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:097a0595:::obj169:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.958+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:097a0595:::obj169:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 4.0 e 118/118 lb 4:097a0595:::obj169:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:095c6b7a:::obj79:head local-lis/les=117/118 n=32 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.958+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.958+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:095c6b7a:::obj79:head local-lis/les=117/118 n=32 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:097a0595:::obj169:head) v3 2026-02-11T23:31:27.958+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:095c6b7a:::obj79:head local-lis/les=117/118 n=32 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:097a0595:::obj169:head) 2026-02-11T23:31:27.958+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.958+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:097a0595:::obj169:head) v3 finish latency 0.000017 2026-02-11T23:31:27.958+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:097a0595:::obj169:head local-lis/les=117/118 n=33 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.958+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.958+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.958+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:097a0595:::obj169:head local-lis/les=117/118 n=33 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.958+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.958+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.958+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.958+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x649e780 con 0x507ac00 2026-02-11T23:31:27.958+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:09d0cce6:::obj481:head)]) -- 0x5f1c300 con 0x507ac00 2026-02-11T23:31:27.958+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.959+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 106 ==== MOSDPGPush(4.0 118/117 [PushOp(4:09eb3833:::obj488:head, version: 111'488, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:09eb3833:::obj488:head@111'488, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x554e4e0 con 0x507ac00 2026-02-11T23:31:27.959+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:09eb3833:::obj488:head, version: 111'488, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:09eb3833:::obj488:head@111'488, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 118 2026-02-11T23:31:27.959+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:09eb3833:::obj488:head, version: 111'488, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:09eb3833:::obj488:head@111'488, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.959+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.959+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 107 ==== pg_backfill(progress 4.0 e 118/118 lb 4:09d0cce6:::obj481:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f10000 con 0x507ac00 2026-02-11T23:31:27.959+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:09eb3833:::obj488:head, version: 111'488, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:09eb3833:::obj488:head@111'488, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.959+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:09d0cce6:::obj481:head) v3 prio 15 type 95 cost 0 latency 0.000010 epoch 118 2026-02-11T23:31:27.959+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:09d0cce6:::obj481:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.959+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.959+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:09eb3833:::obj488:head, version: 111'488, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:09eb3833:::obj488:head@111'488, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.959+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.959+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:09eb3833:::obj488:head, version: 111'488, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:09eb3833:::obj488:head@111'488, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 118/117 [PushOp(4:09eb3833:::obj488:head, version: 111'488, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:09eb3833:::obj488:head@111'488, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:097a0595:::obj169:head local-lis/les=117/118 n=33 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.959+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.959+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:097a0595:::obj169:head local-lis/les=117/118 n=33 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:09eb3833:::obj488:head, version: 111'488, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:09eb3833:::obj488:head@111'488, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.959+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.959+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:09d0cce6:::obj481:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.959+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:097a0595:::obj169:head local-lis/les=117/118 n=33 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:09eb3833:::obj488:head@111'488, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.959+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:097a0595:::obj169:head local-lis/les=117/118 n=33 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:09eb3833:::obj488:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.959+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:097a0595:::obj169:head local-lis/les=117/118 n=33 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:09eb3833:::obj488:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.959+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:097a0595:::obj169:head local-lis/les=117/118 n=33 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:09eb3833:::obj488:head 2026-02-11T23:31:27.959+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:097a0595:::obj169:head local-lis/les=117/118 n=33 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:09eb3833:::obj488:head v 111'488 2026-02-11T23:31:27.959+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:097a0595:::obj169:head local-lis/les=117/118 n=33 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.959+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.959+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:09eb3833:::obj488:head, version: 111'488, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:09eb3833:::obj488:head@111'488, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:27.959+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.959+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.959+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:09d0cce6:::obj481:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.959+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:09d0cce6:::obj481:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 4.0 e 118/118 lb 4:09d0cce6:::obj481:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:097a0595:::obj169:head local-lis/les=117/118 n=33 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.959+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.959+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:097a0595:::obj169:head local-lis/les=117/118 n=33 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:09d0cce6:::obj481:head) v3 2026-02-11T23:31:27.959+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:097a0595:::obj169:head local-lis/les=117/118 n=33 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:09d0cce6:::obj481:head) 2026-02-11T23:31:27.959+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.959+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:09d0cce6:::obj481:head) v3 finish latency 0.000013 2026-02-11T23:31:27.959+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:09d0cce6:::obj481:head local-lis/les=117/118 n=34 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.959+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.959+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.959+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:09d0cce6:::obj481:head local-lis/les=117/118 n=34 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.959+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.959+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.959+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.959+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x649e900 con 0x507ac00 2026-02-11T23:31:27.959+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:09eb3833:::obj488:head)]) -- 0x5f2d800 con 0x507ac00 2026-02-11T23:31:27.959+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.960+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 108 ==== MOSDPGPush(4.0 118/117 [PushOp(4:0a4fcbe7:::obj507:head, version: 111'507, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0a4fcbe7:::obj507:head@111'507, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2340 con 0x507ac00 2026-02-11T23:31:27.960+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:0a4fcbe7:::obj507:head, version: 111'507, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0a4fcbe7:::obj507:head@111'507, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 118 2026-02-11T23:31:27.960+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0a4fcbe7:::obj507:head, version: 111'507, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0a4fcbe7:::obj507:head@111'507, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.960+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.960+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 109 ==== pg_backfill(progress 4.0 e 118/118 lb 4:09eb3833:::obj488:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f13700 con 0x507ac00 2026-02-11T23:31:27.960+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:09eb3833:::obj488:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:27.960+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0a4fcbe7:::obj507:head, version: 111'507, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0a4fcbe7:::obj507:head@111'507, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.960+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:09eb3833:::obj488:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.960+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.960+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0a4fcbe7:::obj507:head, version: 111'507, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0a4fcbe7:::obj507:head@111'507, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.960+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:0a4fcbe7:::obj507:head, version: 111'507, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0a4fcbe7:::obj507:head@111'507, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 118/117 [PushOp(4:0a4fcbe7:::obj507:head, version: 111'507, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0a4fcbe7:::obj507:head@111'507, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:09d0cce6:::obj481:head local-lis/les=117/118 n=34 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.960+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.960+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.960+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:09eb3833:::obj488:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.960+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:09d0cce6:::obj481:head local-lis/les=117/118 n=34 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:0a4fcbe7:::obj507:head, version: 111'507, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0a4fcbe7:::obj507:head@111'507, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.960+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:09d0cce6:::obj481:head local-lis/les=117/118 n=34 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:0a4fcbe7:::obj507:head@111'507, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.960+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:09d0cce6:::obj481:head local-lis/les=117/118 n=34 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:0a4fcbe7:::obj507:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.960+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:09d0cce6:::obj481:head local-lis/les=117/118 n=34 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:0a4fcbe7:::obj507:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.960+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:09d0cce6:::obj481:head local-lis/les=117/118 n=34 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:0a4fcbe7:::obj507:head 2026-02-11T23:31:27.960+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:09d0cce6:::obj481:head local-lis/les=117/118 n=34 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:0a4fcbe7:::obj507:head v 111'507 2026-02-11T23:31:27.960+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:09d0cce6:::obj481:head local-lis/les=117/118 n=34 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.960+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.960+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:0a4fcbe7:::obj507:head, version: 111'507, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0a4fcbe7:::obj507:head@111'507, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:27.960+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.960+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.960+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:09eb3833:::obj488:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.960+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:09eb3833:::obj488:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 4.0 e 118/118 lb 4:09eb3833:::obj488:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:09d0cce6:::obj481:head local-lis/les=117/118 n=34 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.960+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.960+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:09d0cce6:::obj481:head local-lis/les=117/118 n=34 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:09eb3833:::obj488:head) v3 2026-02-11T23:31:27.960+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:09d0cce6:::obj481:head local-lis/les=117/118 n=34 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:09eb3833:::obj488:head) 2026-02-11T23:31:27.960+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.960+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:09eb3833:::obj488:head) v3 finish latency 0.000017 2026-02-11T23:31:27.960+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:09eb3833:::obj488:head local-lis/les=117/118 n=35 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.960+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.960+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.960+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:09eb3833:::obj488:head local-lis/les=117/118 n=35 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.960+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.960+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.960+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.960+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x649ea80 con 0x507ac00 2026-02-11T23:31:27.960+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:0a4fcbe7:::obj507:head)]) -- 0x5f75b00 con 0x507ac00 2026-02-11T23:31:27.960+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.960+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 110 ==== MOSDPGPush(4.0 118/117 [PushOp(4:0b3f05c8:::obj589:head, version: 111'589, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0b3f05c8:::obj589:head@111'589, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56ca680 con 0x507ac00 2026-02-11T23:31:27.960+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:0b3f05c8:::obj589:head, version: 111'589, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0b3f05c8:::obj589:head@111'589, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 118 2026-02-11T23:31:27.960+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0b3f05c8:::obj589:head, version: 111'589, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0b3f05c8:::obj589:head@111'589, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.960+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.960+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 111 ==== pg_backfill(progress 4.0 e 118/118 lb 4:0a4fcbe7:::obj507:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f13180 con 0x507ac00 2026-02-11T23:31:27.960+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0b3f05c8:::obj589:head, version: 111'589, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0b3f05c8:::obj589:head@111'589, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.960+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:0a4fcbe7:::obj507:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:27.960+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0a4fcbe7:::obj507:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.960+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.960+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0b3f05c8:::obj589:head, version: 111'589, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0b3f05c8:::obj589:head@111'589, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.960+0000 7f414f993640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:0b3f05c8:::obj589:head, version: 111'589, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0b3f05c8:::obj589:head@111'589, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(4.0 118/117 [PushOp(4:0b3f05c8:::obj589:head, version: 111'589, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0b3f05c8:::obj589:head@111'589, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:09eb3833:::obj488:head local-lis/les=117/118 n=35 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.960+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.960+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:09eb3833:::obj488:head local-lis/les=117/118 n=35 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:0b3f05c8:::obj589:head, version: 111'589, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0b3f05c8:::obj589:head@111'589, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.960+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.960+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:09eb3833:::obj488:head local-lis/les=117/118 n=35 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:0b3f05c8:::obj589:head@111'589, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.960+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0a4fcbe7:::obj507:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.960+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:09eb3833:::obj488:head local-lis/les=117/118 n=35 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:0b3f05c8:::obj589:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.960+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:09eb3833:::obj488:head local-lis/les=117/118 n=35 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:0b3f05c8:::obj589:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.960+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:09eb3833:::obj488:head local-lis/les=117/118 n=35 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:0b3f05c8:::obj589:head 2026-02-11T23:31:27.960+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:09eb3833:::obj488:head local-lis/les=117/118 n=35 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:0b3f05c8:::obj589:head v 111'589 2026-02-11T23:31:27.960+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:09eb3833:::obj488:head local-lis/les=117/118 n=35 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.960+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.960+0000 7f414f993640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:0b3f05c8:::obj589:head, version: 111'589, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0b3f05c8:::obj589:head@111'589, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:27.960+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:09eb3833:::obj488:head local-lis/les=117/118 n=35 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.960+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.960+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.960+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0a4fcbe7:::obj507:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.960+0000 7f414b98b640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:0a4fcbe7:::obj507:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 4.0 e 118/118 lb 4:0a4fcbe7:::obj507:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:09eb3833:::obj488:head local-lis/les=117/118 n=35 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.960+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.960+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:09eb3833:::obj488:head local-lis/les=117/118 n=35 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:0a4fcbe7:::obj507:head) v3 2026-02-11T23:31:27.960+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:09eb3833:::obj488:head local-lis/les=117/118 n=35 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:0a4fcbe7:::obj507:head) 2026-02-11T23:31:27.960+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.960+0000 7f414b98b640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:0a4fcbe7:::obj507:head) v3 finish latency 0.000013 2026-02-11T23:31:27.960+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.960+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.960+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0a4fcbe7:::obj507:head local-lis/les=117/118 n=36 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.960+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.960+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.960+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.960+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x649ef00 con 0x507ac00 2026-02-11T23:31:27.960+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.960+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:0b3f05c8:::obj589:head)]) -- 0x649ec00 con 0x507ac00 2026-02-11T23:31:27.960+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.961+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 112 ==== MOSDPGPush(4.0 118/117 [PushOp(4:0bdade2e:::obj246:head, version: 111'246, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0bdade2e:::obj246:head@111'246, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2ea0 con 0x507ac00 2026-02-11T23:31:27.961+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:0bdade2e:::obj246:head, version: 111'246, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0bdade2e:::obj246:head@111'246, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 118 2026-02-11T23:31:27.961+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0bdade2e:::obj246:head, version: 111'246, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0bdade2e:::obj246:head@111'246, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.961+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 113 ==== pg_backfill(progress 4.0 e 118/118 lb 4:0b3f05c8:::obj589:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f11b80 con 0x507ac00 2026-02-11T23:31:27.961+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:0b3f05c8:::obj589:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 118 2026-02-11T23:31:27.961+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.961+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0b3f05c8:::obj589:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.961+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0bdade2e:::obj246:head, version: 111'246, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0bdade2e:::obj246:head@111'246, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.961+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.961+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0bdade2e:::obj246:head, version: 111'246, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0bdade2e:::obj246:head@111'246, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.961+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:0bdade2e:::obj246:head, version: 111'246, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0bdade2e:::obj246:head@111'246, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(4.0 118/117 [PushOp(4:0bdade2e:::obj246:head, version: 111'246, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0bdade2e:::obj246:head@111'246, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:0a4fcbe7:::obj507:head local-lis/les=117/118 n=36 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.961+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.961+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.961+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0a4fcbe7:::obj507:head local-lis/les=117/118 n=36 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:0bdade2e:::obj246:head, version: 111'246, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0bdade2e:::obj246:head@111'246, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.961+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0b3f05c8:::obj589:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.961+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0a4fcbe7:::obj507:head local-lis/les=117/118 n=36 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:0bdade2e:::obj246:head@111'246, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.961+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0a4fcbe7:::obj507:head local-lis/les=117/118 n=36 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:0bdade2e:::obj246:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.961+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0a4fcbe7:::obj507:head local-lis/les=117/118 n=36 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:0bdade2e:::obj246:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.961+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0a4fcbe7:::obj507:head local-lis/les=117/118 n=36 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:0bdade2e:::obj246:head 2026-02-11T23:31:27.961+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0a4fcbe7:::obj507:head local-lis/les=117/118 n=36 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:0bdade2e:::obj246:head v 111'246 2026-02-11T23:31:27.961+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0a4fcbe7:::obj507:head local-lis/les=117/118 n=36 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.961+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.961+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:0bdade2e:::obj246:head, version: 111'246, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0bdade2e:::obj246:head@111'246, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:27.961+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.961+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.961+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0b3f05c8:::obj589:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.961+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:0b3f05c8:::obj589:head) v3 prio 15 cost 0 latency 0.000079 pg_backfill(progress 4.0 e 118/118 lb 4:0b3f05c8:::obj589:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:0a4fcbe7:::obj507:head local-lis/les=117/118 n=36 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.961+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.961+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0a4fcbe7:::obj507:head local-lis/les=117/118 n=36 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:0b3f05c8:::obj589:head) v3 2026-02-11T23:31:27.961+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0a4fcbe7:::obj507:head local-lis/les=117/118 n=36 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:0b3f05c8:::obj589:head) 2026-02-11T23:31:27.961+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.961+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:0b3f05c8:::obj589:head) v3 finish latency 0.000016 2026-02-11T23:31:27.961+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0b3f05c8:::obj589:head local-lis/les=117/118 n=37 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.961+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.961+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.961+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0b3f05c8:::obj589:head local-lis/les=117/118 n=37 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.961+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.961+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.961+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.961+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x649f080 con 0x507ac00 2026-02-11T23:31:27.961+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:0bdade2e:::obj246:head)]) -- 0x5f2dc80 con 0x507ac00 2026-02-11T23:31:27.961+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.962+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 114 ==== MOSDPGPush(4.0 118/117 [PushOp(4:0bde2796:::obj413:head, version: 111'413, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0bde2796:::obj413:head@111'413, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a29c0 con 0x507ac00 2026-02-11T23:31:27.962+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:0bde2796:::obj413:head, version: 111'413, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0bde2796:::obj413:head@111'413, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 118 2026-02-11T23:31:27.962+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0bde2796:::obj413:head, version: 111'413, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0bde2796:::obj413:head@111'413, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.962+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 115 ==== pg_backfill(progress 4.0 e 118/118 lb 4:0bdade2e:::obj246:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f12100 con 0x507ac00 2026-02-11T23:31:27.962+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:0bdade2e:::obj246:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 118 2026-02-11T23:31:27.962+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.962+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0bdade2e:::obj246:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.962+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0bde2796:::obj413:head, version: 111'413, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0bde2796:::obj413:head@111'413, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.962+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.962+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0bde2796:::obj413:head, version: 111'413, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0bde2796:::obj413:head@111'413, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.962+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:0bde2796:::obj413:head, version: 111'413, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0bde2796:::obj413:head@111'413, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(4.0 118/117 [PushOp(4:0bde2796:::obj413:head, version: 111'413, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0bde2796:::obj413:head@111'413, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:0b3f05c8:::obj589:head local-lis/les=117/118 n=37 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.962+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.962+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.962+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0bdade2e:::obj246:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.962+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0b3f05c8:::obj589:head local-lis/les=117/118 n=37 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:0bde2796:::obj413:head, version: 111'413, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0bde2796:::obj413:head@111'413, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.962+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0b3f05c8:::obj589:head local-lis/les=117/118 n=37 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:0bde2796:::obj413:head@111'413, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.962+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0b3f05c8:::obj589:head local-lis/les=117/118 n=37 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:0bde2796:::obj413:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.962+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0b3f05c8:::obj589:head local-lis/les=117/118 n=37 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:0bde2796:::obj413:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.962+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0b3f05c8:::obj589:head local-lis/les=117/118 n=37 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:0bde2796:::obj413:head 2026-02-11T23:31:27.962+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0b3f05c8:::obj589:head local-lis/les=117/118 n=37 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:0bde2796:::obj413:head v 111'413 2026-02-11T23:31:27.962+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0b3f05c8:::obj589:head local-lis/les=117/118 n=37 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.962+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.962+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:0bde2796:::obj413:head, version: 111'413, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0bde2796:::obj413:head@111'413, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:27.962+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.962+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.962+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0bdade2e:::obj246:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.962+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:0bdade2e:::obj246:head) v3 prio 15 cost 0 latency 0.000081 pg_backfill(progress 4.0 e 118/118 lb 4:0bdade2e:::obj246:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:0b3f05c8:::obj589:head local-lis/les=117/118 n=37 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.962+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.962+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0b3f05c8:::obj589:head local-lis/les=117/118 n=37 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:0bdade2e:::obj246:head) v3 2026-02-11T23:31:27.962+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0b3f05c8:::obj589:head local-lis/les=117/118 n=37 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:0bdade2e:::obj246:head) 2026-02-11T23:31:27.962+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.962+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:0bdade2e:::obj246:head) v3 finish latency 0.000014 2026-02-11T23:31:27.962+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0bdade2e:::obj246:head local-lis/les=117/118 n=38 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.962+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.962+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.962+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0bdade2e:::obj246:head local-lis/les=117/118 n=38 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.962+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.962+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.962+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.962+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x649f200 con 0x507ac00 2026-02-11T23:31:27.962+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:0bde2796:::obj413:head)]) -- 0x5f75c80 con 0x507ac00 2026-02-11T23:31:27.962+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.962+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 116 ==== MOSDPGPush(4.0 118/117 [PushOp(4:0c03281d:::obj596:head, version: 111'596, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0c03281d:::obj596:head@111'596, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a3a00 con 0x507ac00 2026-02-11T23:31:27.962+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:0c03281d:::obj596:head, version: 111'596, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0c03281d:::obj596:head@111'596, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 118 2026-02-11T23:31:27.962+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0c03281d:::obj596:head, version: 111'596, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0c03281d:::obj596:head@111'596, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.962+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.962+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 117 ==== pg_backfill(progress 4.0 e 118/118 lb 4:0bde2796:::obj413:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f12680 con 0x507ac00 2026-02-11T23:31:27.962+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0c03281d:::obj596:head, version: 111'596, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0c03281d:::obj596:head@111'596, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.962+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:0bde2796:::obj413:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 118 2026-02-11T23:31:27.962+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.962+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0bde2796:::obj413:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.962+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0c03281d:::obj596:head, version: 111'596, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0c03281d:::obj596:head@111'596, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.962+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.962+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:0c03281d:::obj596:head, version: 111'596, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0c03281d:::obj596:head@111'596, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 118/117 [PushOp(4:0c03281d:::obj596:head, version: 111'596, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0c03281d:::obj596:head@111'596, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:0bdade2e:::obj246:head local-lis/les=117/118 n=38 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.962+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.962+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0bdade2e:::obj246:head local-lis/les=117/118 n=38 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:0c03281d:::obj596:head, version: 111'596, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0c03281d:::obj596:head@111'596, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.962+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0bdade2e:::obj246:head local-lis/les=117/118 n=38 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:0c03281d:::obj596:head@111'596, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.962+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.962+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0bde2796:::obj413:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.962+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0bdade2e:::obj246:head local-lis/les=117/118 n=38 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:0c03281d:::obj596:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.962+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0bdade2e:::obj246:head local-lis/les=117/118 n=38 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:0c03281d:::obj596:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.962+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0bdade2e:::obj246:head local-lis/les=117/118 n=38 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:0c03281d:::obj596:head 2026-02-11T23:31:27.962+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0bdade2e:::obj246:head local-lis/les=117/118 n=38 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:0c03281d:::obj596:head v 111'596 2026-02-11T23:31:27.962+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0bdade2e:::obj246:head local-lis/les=117/118 n=38 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.962+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.962+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:0c03281d:::obj596:head, version: 111'596, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0c03281d:::obj596:head@111'596, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:27.962+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.962+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.962+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0bde2796:::obj413:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.962+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:0bde2796:::obj413:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 118/118 lb 4:0bde2796:::obj413:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:0bdade2e:::obj246:head local-lis/les=117/118 n=38 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.962+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.962+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0bdade2e:::obj246:head local-lis/les=117/118 n=38 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:0bde2796:::obj413:head) v3 2026-02-11T23:31:27.962+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0bdade2e:::obj246:head local-lis/les=117/118 n=38 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:0bde2796:::obj413:head) 2026-02-11T23:31:27.962+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.962+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:0bde2796:::obj413:head) v3 finish latency 0.000013 2026-02-11T23:31:27.962+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0bde2796:::obj413:head local-lis/les=117/118 n=39 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.962+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.963+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.963+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0bde2796:::obj413:head local-lis/les=117/118 n=39 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.963+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.963+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.963+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.963+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x649f380 con 0x507ac00 2026-02-11T23:31:27.963+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:0c03281d:::obj596:head)]) -- 0x5f75e00 con 0x507ac00 2026-02-11T23:31:27.963+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.963+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 118 ==== MOSDPGPush(4.0 118/117 [PushOp(4:0cfeb9e4:::obj357:head, version: 111'357, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0cfeb9e4:::obj357:head@111'357, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56aa000 con 0x507ac00 2026-02-11T23:31:27.963+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:0cfeb9e4:::obj357:head, version: 111'357, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0cfeb9e4:::obj357:head@111'357, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 118 2026-02-11T23:31:27.963+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0cfeb9e4:::obj357:head, version: 111'357, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0cfeb9e4:::obj357:head@111'357, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.963+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.963+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 119 ==== pg_backfill(progress 4.0 e 118/118 lb 4:0c03281d:::obj596:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f12c00 con 0x507ac00 2026-02-11T23:31:27.963+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0cfeb9e4:::obj357:head, version: 111'357, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0cfeb9e4:::obj357:head@111'357, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.963+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:0c03281d:::obj596:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 118 2026-02-11T23:31:27.963+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0c03281d:::obj596:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.963+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.963+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0cfeb9e4:::obj357:head, version: 111'357, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0cfeb9e4:::obj357:head@111'357, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.963+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:0cfeb9e4:::obj357:head, version: 111'357, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0cfeb9e4:::obj357:head@111'357, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 118/117 [PushOp(4:0cfeb9e4:::obj357:head, version: 111'357, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0cfeb9e4:::obj357:head@111'357, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:0bde2796:::obj413:head local-lis/les=117/118 n=39 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.963+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.963+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.963+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0c03281d:::obj596:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.963+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0bde2796:::obj413:head local-lis/les=117/118 n=39 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:0cfeb9e4:::obj357:head, version: 111'357, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0cfeb9e4:::obj357:head@111'357, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.963+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0bde2796:::obj413:head local-lis/les=117/118 n=39 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:0cfeb9e4:::obj357:head@111'357, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.963+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0bde2796:::obj413:head local-lis/les=117/118 n=39 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:0cfeb9e4:::obj357:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.963+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0bde2796:::obj413:head local-lis/les=117/118 n=39 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:0cfeb9e4:::obj357:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.963+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0bde2796:::obj413:head local-lis/les=117/118 n=39 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:0cfeb9e4:::obj357:head 2026-02-11T23:31:27.963+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0bde2796:::obj413:head local-lis/les=117/118 n=39 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:0cfeb9e4:::obj357:head v 111'357 2026-02-11T23:31:27.963+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0bde2796:::obj413:head local-lis/les=117/118 n=39 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.963+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.963+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:0cfeb9e4:::obj357:head, version: 111'357, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0cfeb9e4:::obj357:head@111'357, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:27.963+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.963+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.963+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0c03281d:::obj596:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.963+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:0c03281d:::obj596:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 4.0 e 118/118 lb 4:0c03281d:::obj596:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:0bde2796:::obj413:head local-lis/les=117/118 n=39 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.963+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.963+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0bde2796:::obj413:head local-lis/les=117/118 n=39 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:0c03281d:::obj596:head) v3 2026-02-11T23:31:27.963+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0bde2796:::obj413:head local-lis/les=117/118 n=39 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:0c03281d:::obj596:head) 2026-02-11T23:31:27.963+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.963+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:0c03281d:::obj596:head) v3 finish latency 0.000011 2026-02-11T23:31:27.963+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0c03281d:::obj596:head local-lis/les=117/118 n=40 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.963+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.963+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.963+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0c03281d:::obj596:head local-lis/les=117/118 n=40 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.963+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.963+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.963+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.963+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x649f500 con 0x507ac00 2026-02-11T23:31:27.963+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:0cfeb9e4:::obj357:head)]) -- 0x5f2d800 con 0x507ac00 2026-02-11T23:31:27.963+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.964+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 120 ==== MOSDPGPush(4.0 118/117 [PushOp(4:0dc4a75f:::obj282:head, version: 111'282, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0dc4a75f:::obj282:head@111'282, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2680 con 0x507ac00 2026-02-11T23:31:27.964+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:0dc4a75f:::obj282:head, version: 111'282, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0dc4a75f:::obj282:head@111'282, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 118 2026-02-11T23:31:27.964+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0dc4a75f:::obj282:head, version: 111'282, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0dc4a75f:::obj282:head@111'282, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.964+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.964+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 121 ==== pg_backfill(progress 4.0 e 118/118 lb 4:0cfeb9e4:::obj357:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f13700 con 0x507ac00 2026-02-11T23:31:27.964+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0dc4a75f:::obj282:head, version: 111'282, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0dc4a75f:::obj282:head@111'282, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.964+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:0cfeb9e4:::obj357:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 118 2026-02-11T23:31:27.964+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0cfeb9e4:::obj357:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.964+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.964+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0dc4a75f:::obj282:head, version: 111'282, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0dc4a75f:::obj282:head@111'282, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.964+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.964+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:0dc4a75f:::obj282:head, version: 111'282, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0dc4a75f:::obj282:head@111'282, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(4.0 118/117 [PushOp(4:0dc4a75f:::obj282:head, version: 111'282, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0dc4a75f:::obj282:head@111'282, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:0c03281d:::obj596:head local-lis/les=117/118 n=40 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.964+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.964+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0c03281d:::obj596:head local-lis/les=117/118 n=40 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:0dc4a75f:::obj282:head, version: 111'282, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0dc4a75f:::obj282:head@111'282, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.964+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0c03281d:::obj596:head local-lis/les=117/118 n=40 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:0dc4a75f:::obj282:head@111'282, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.964+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.964+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0cfeb9e4:::obj357:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.964+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0c03281d:::obj596:head local-lis/les=117/118 n=40 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:0dc4a75f:::obj282:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.964+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0c03281d:::obj596:head local-lis/les=117/118 n=40 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:0dc4a75f:::obj282:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.964+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0c03281d:::obj596:head local-lis/les=117/118 n=40 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:0dc4a75f:::obj282:head 2026-02-11T23:31:27.964+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0c03281d:::obj596:head local-lis/les=117/118 n=40 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:0dc4a75f:::obj282:head v 111'282 2026-02-11T23:31:27.964+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0c03281d:::obj596:head local-lis/les=117/118 n=40 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.964+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.964+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:0dc4a75f:::obj282:head, version: 111'282, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0dc4a75f:::obj282:head@111'282, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:27.964+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.964+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.964+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0cfeb9e4:::obj357:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.964+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:0cfeb9e4:::obj357:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 4.0 e 118/118 lb 4:0cfeb9e4:::obj357:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:0c03281d:::obj596:head local-lis/les=117/118 n=40 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.964+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.964+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0c03281d:::obj596:head local-lis/les=117/118 n=40 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:0cfeb9e4:::obj357:head) v3 2026-02-11T23:31:27.964+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0c03281d:::obj596:head local-lis/les=117/118 n=40 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:0cfeb9e4:::obj357:head) 2026-02-11T23:31:27.964+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.964+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:0cfeb9e4:::obj357:head) v3 finish latency 0.000012 2026-02-11T23:31:27.964+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0cfeb9e4:::obj357:head local-lis/les=117/118 n=41 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.964+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.964+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.964+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0cfeb9e4:::obj357:head local-lis/les=117/118 n=41 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.964+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.964+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.964+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.964+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x649f680 con 0x507ac00 2026-02-11T23:31:27.964+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:0dc4a75f:::obj282:head)]) -- 0x5f1c300 con 0x507ac00 2026-02-11T23:31:27.964+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.964+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 122 ==== MOSDPGPush(4.0 118/117 [PushOp(4:0e39285d:::obj525:head, version: 111'525, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e39285d:::obj525:head@111'525, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56cab60 con 0x507ac00 2026-02-11T23:31:27.964+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:0e39285d:::obj525:head, version: 111'525, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e39285d:::obj525:head@111'525, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 118 2026-02-11T23:31:27.964+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0e39285d:::obj525:head, version: 111'525, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e39285d:::obj525:head@111'525, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.964+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.964+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 123 ==== pg_backfill(progress 4.0 e 118/118 lb 4:0dc4a75f:::obj282:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f10000 con 0x507ac00 2026-02-11T23:31:27.964+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0e39285d:::obj525:head, version: 111'525, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e39285d:::obj525:head@111'525, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.964+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:0dc4a75f:::obj282:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 118 2026-02-11T23:31:27.964+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0dc4a75f:::obj282:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.964+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.964+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0e39285d:::obj525:head, version: 111'525, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e39285d:::obj525:head@111'525, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.964+0000 7f414f993640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:0e39285d:::obj525:head, version: 111'525, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e39285d:::obj525:head@111'525, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 118/117 [PushOp(4:0e39285d:::obj525:head, version: 111'525, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e39285d:::obj525:head@111'525, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:0cfeb9e4:::obj357:head local-lis/les=117/118 n=41 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.964+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.964+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.964+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0dc4a75f:::obj282:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.964+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0cfeb9e4:::obj357:head local-lis/les=117/118 n=41 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:0e39285d:::obj525:head, version: 111'525, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e39285d:::obj525:head@111'525, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.964+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0cfeb9e4:::obj357:head local-lis/les=117/118 n=41 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:0e39285d:::obj525:head@111'525, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.964+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0cfeb9e4:::obj357:head local-lis/les=117/118 n=41 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:0e39285d:::obj525:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.964+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0cfeb9e4:::obj357:head local-lis/les=117/118 n=41 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:0e39285d:::obj525:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.964+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0cfeb9e4:::obj357:head local-lis/les=117/118 n=41 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:0e39285d:::obj525:head 2026-02-11T23:31:27.964+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0cfeb9e4:::obj357:head local-lis/les=117/118 n=41 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:0e39285d:::obj525:head v 111'525 2026-02-11T23:31:27.964+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0cfeb9e4:::obj357:head local-lis/les=117/118 n=41 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.964+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.964+0000 7f414f993640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:0e39285d:::obj525:head, version: 111'525, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e39285d:::obj525:head@111'525, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:27.964+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0cfeb9e4:::obj357:head local-lis/les=117/118 n=41 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.964+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.964+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.964+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0dc4a75f:::obj282:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.964+0000 7f414b98b640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:0dc4a75f:::obj282:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 4.0 e 118/118 lb 4:0dc4a75f:::obj282:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:0cfeb9e4:::obj357:head local-lis/les=117/118 n=41 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.964+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.964+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0cfeb9e4:::obj357:head local-lis/les=117/118 n=41 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:0dc4a75f:::obj282:head) v3 2026-02-11T23:31:27.964+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0cfeb9e4:::obj357:head local-lis/les=117/118 n=41 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:0dc4a75f:::obj282:head) 2026-02-11T23:31:27.964+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.964+0000 7f414b98b640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:0dc4a75f:::obj282:head) v3 finish latency 0.000013 2026-02-11T23:31:27.964+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.965+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.965+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0dc4a75f:::obj282:head local-lis/les=117/118 n=42 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.965+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.965+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.965+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.965+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x649fb00 con 0x507ac00 2026-02-11T23:31:27.965+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:0e39285d:::obj525:head)]) -- 0x649f800 con 0x507ac00 2026-02-11T23:31:27.965+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.965+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 124 ==== MOSDPGPush(4.0 118/117 [PushOp(4:0e3dc695:::obj78:head, version: 111'78, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e3dc695:::obj78:head@111'78, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x555cea0 con 0x507ac00 2026-02-11T23:31:27.965+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:0e3dc695:::obj78:head, version: 111'78, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e3dc695:::obj78:head@111'78, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 118 2026-02-11T23:31:27.965+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0e3dc695:::obj78:head, version: 111'78, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e3dc695:::obj78:head@111'78, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.965+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.965+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 125 ==== pg_backfill(progress 4.0 e 118/118 lb 4:0e39285d:::obj525:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f10580 con 0x507ac00 2026-02-11T23:31:27.965+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0e3dc695:::obj78:head, version: 111'78, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e3dc695:::obj78:head@111'78, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.965+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:0e39285d:::obj525:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 118 2026-02-11T23:31:27.965+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0e39285d:::obj525:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.965+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.965+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0e3dc695:::obj78:head, version: 111'78, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e3dc695:::obj78:head@111'78, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.965+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.965+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:0e3dc695:::obj78:head, version: 111'78, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e3dc695:::obj78:head@111'78, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 118/117 [PushOp(4:0e3dc695:::obj78:head, version: 111'78, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e3dc695:::obj78:head@111'78, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:0dc4a75f:::obj282:head local-lis/les=117/118 n=42 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.965+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.965+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0dc4a75f:::obj282:head local-lis/les=117/118 n=42 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:0e3dc695:::obj78:head, version: 111'78, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e3dc695:::obj78:head@111'78, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.965+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0dc4a75f:::obj282:head local-lis/les=117/118 n=42 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:0e3dc695:::obj78:head@111'78, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.965+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.965+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0e39285d:::obj525:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.965+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0dc4a75f:::obj282:head local-lis/les=117/118 n=42 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:0e3dc695:::obj78:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.965+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0dc4a75f:::obj282:head local-lis/les=117/118 n=42 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:0e3dc695:::obj78:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.965+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0dc4a75f:::obj282:head local-lis/les=117/118 n=42 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:0e3dc695:::obj78:head 2026-02-11T23:31:27.965+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0dc4a75f:::obj282:head local-lis/les=117/118 n=42 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:0e3dc695:::obj78:head v 111'78 2026-02-11T23:31:27.965+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0dc4a75f:::obj282:head local-lis/les=117/118 n=42 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.965+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.965+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:0e3dc695:::obj78:head, version: 111'78, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e3dc695:::obj78:head@111'78, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:27.965+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.965+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.965+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0e39285d:::obj525:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.965+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:0e39285d:::obj525:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 4.0 e 118/118 lb 4:0e39285d:::obj525:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:0dc4a75f:::obj282:head local-lis/les=117/118 n=42 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.965+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.965+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0dc4a75f:::obj282:head local-lis/les=117/118 n=42 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:0e39285d:::obj525:head) v3 2026-02-11T23:31:27.965+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0dc4a75f:::obj282:head local-lis/les=117/118 n=42 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:0e39285d:::obj525:head) 2026-02-11T23:31:27.965+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.965+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:0e39285d:::obj525:head) v3 finish latency 0.000012 2026-02-11T23:31:27.965+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0e39285d:::obj525:head local-lis/les=117/118 n=43 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.965+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.965+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.965+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0e39285d:::obj525:head local-lis/les=117/118 n=43 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.965+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.965+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.965+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.965+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x649fc80 con 0x507ac00 2026-02-11T23:31:27.965+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:0e3dc695:::obj78:head)]) -- 0x5d34480 con 0x507ac00 2026-02-11T23:31:27.965+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.966+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 126 ==== MOSDPGPush(4.0 118/117 [PushOp(4:0e5b0695:::obj117:head, version: 111'117, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e5b0695:::obj117:head@111'117, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c4e0 con 0x507ac00 2026-02-11T23:31:27.966+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:0e5b0695:::obj117:head, version: 111'117, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e5b0695:::obj117:head@111'117, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 118 2026-02-11T23:31:27.966+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0e5b0695:::obj117:head, version: 111'117, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e5b0695:::obj117:head@111'117, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.966+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 127 ==== pg_backfill(progress 4.0 e 118/118 lb 4:0e3dc695:::obj78:head) ==== 1061+0+0 (unknown 0 0 0) 0x5f10b00 con 0x507ac00 2026-02-11T23:31:27.966+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:0e3dc695:::obj78:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 118 2026-02-11T23:31:27.966+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.966+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0e3dc695:::obj78:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.966+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0e5b0695:::obj117:head, version: 111'117, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e5b0695:::obj117:head@111'117, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.966+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.966+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0e5b0695:::obj117:head, version: 111'117, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e5b0695:::obj117:head@111'117, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.966+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:0e5b0695:::obj117:head, version: 111'117, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e5b0695:::obj117:head@111'117, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(4.0 118/117 [PushOp(4:0e5b0695:::obj117:head, version: 111'117, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e5b0695:::obj117:head@111'117, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:0e39285d:::obj525:head local-lis/les=117/118 n=43 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.966+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.966+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.966+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0e3dc695:::obj78:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.966+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0e39285d:::obj525:head local-lis/les=117/118 n=43 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:0e5b0695:::obj117:head, version: 111'117, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e5b0695:::obj117:head@111'117, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.966+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0e39285d:::obj525:head local-lis/les=117/118 n=43 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:0e5b0695:::obj117:head@111'117, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.966+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0e39285d:::obj525:head local-lis/les=117/118 n=43 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:0e5b0695:::obj117:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.966+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0e39285d:::obj525:head local-lis/les=117/118 n=43 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:0e5b0695:::obj117:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.966+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0e39285d:::obj525:head local-lis/les=117/118 n=43 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:0e5b0695:::obj117:head 2026-02-11T23:31:27.966+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0e39285d:::obj525:head local-lis/les=117/118 n=43 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:0e5b0695:::obj117:head v 111'117 2026-02-11T23:31:27.966+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0e39285d:::obj525:head local-lis/les=117/118 n=43 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.966+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.966+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:0e5b0695:::obj117:head, version: 111'117, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e5b0695:::obj117:head@111'117, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:27.966+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.966+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.966+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0e3dc695:::obj78:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.966+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:0e3dc695:::obj78:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 4.0 e 118/118 lb 4:0e3dc695:::obj78:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:0e39285d:::obj525:head local-lis/les=117/118 n=43 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.966+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.966+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0e39285d:::obj525:head local-lis/les=117/118 n=43 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:0e3dc695:::obj78:head) v3 2026-02-11T23:31:27.966+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0e39285d:::obj525:head local-lis/les=117/118 n=43 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:0e3dc695:::obj78:head) 2026-02-11T23:31:27.966+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.966+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:0e3dc695:::obj78:head) v3 finish latency 0.000012 2026-02-11T23:31:27.966+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0e3dc695:::obj78:head local-lis/les=117/118 n=44 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.966+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.966+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.966+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0e3dc695:::obj78:head local-lis/les=117/118 n=44 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.966+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.966+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.966+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.966+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x649fe00 con 0x507ac00 2026-02-11T23:31:27.966+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:0e5b0695:::obj117:head)]) -- 0x649e600 con 0x507ac00 2026-02-11T23:31:27.966+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.966+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 128 ==== MOSDPGPush(4.0 118/117 [PushOp(4:0e5e5c87:::obj521:head, version: 111'521, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e5e5c87:::obj521:head@111'521, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f0ea0 con 0x507ac00 2026-02-11T23:31:27.966+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:0e5e5c87:::obj521:head, version: 111'521, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e5e5c87:::obj521:head@111'521, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 118 2026-02-11T23:31:27.966+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0e5e5c87:::obj521:head, version: 111'521, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e5e5c87:::obj521:head@111'521, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.966+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 129 ==== pg_backfill(progress 4.0 e 118/118 lb 4:0e5b0695:::obj117:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f11080 con 0x507ac00 2026-02-11T23:31:27.966+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.966+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:0e5b0695:::obj117:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:27.966+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0e5b0695:::obj117:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.966+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0e5e5c87:::obj521:head, version: 111'521, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e5e5c87:::obj521:head@111'521, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.966+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.966+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0e5e5c87:::obj521:head, version: 111'521, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e5e5c87:::obj521:head@111'521, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.966+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:0e5e5c87:::obj521:head, version: 111'521, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e5e5c87:::obj521:head@111'521, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(4.0 118/117 [PushOp(4:0e5e5c87:::obj521:head, version: 111'521, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e5e5c87:::obj521:head@111'521, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:0e3dc695:::obj78:head local-lis/les=117/118 n=44 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.966+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.966+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.966+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0e5b0695:::obj117:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.966+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0e3dc695:::obj78:head local-lis/les=117/118 n=44 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:0e5e5c87:::obj521:head, version: 111'521, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e5e5c87:::obj521:head@111'521, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.966+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0e3dc695:::obj78:head local-lis/les=117/118 n=44 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:0e5e5c87:::obj521:head@111'521, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.966+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0e3dc695:::obj78:head local-lis/les=117/118 n=44 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:0e5e5c87:::obj521:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.966+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0e3dc695:::obj78:head local-lis/les=117/118 n=44 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:0e5e5c87:::obj521:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.966+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0e3dc695:::obj78:head local-lis/les=117/118 n=44 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:0e5e5c87:::obj521:head 2026-02-11T23:31:27.966+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0e3dc695:::obj78:head local-lis/les=117/118 n=44 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:0e5e5c87:::obj521:head v 111'521 2026-02-11T23:31:27.966+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0e3dc695:::obj78:head local-lis/les=117/118 n=44 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.966+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.966+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:0e5e5c87:::obj521:head, version: 111'521, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e5e5c87:::obj521:head@111'521, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:27.966+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.966+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.966+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0e5b0695:::obj117:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.966+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:0e5b0695:::obj117:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 4.0 e 118/118 lb 4:0e5b0695:::obj117:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:0e3dc695:::obj78:head local-lis/les=117/118 n=44 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.966+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.966+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0e3dc695:::obj78:head local-lis/les=117/118 n=44 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:0e5b0695:::obj117:head) v3 2026-02-11T23:31:27.966+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0e3dc695:::obj78:head local-lis/les=117/118 n=44 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:0e5b0695:::obj117:head) 2026-02-11T23:31:27.966+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.966+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:0e5b0695:::obj117:head) v3 finish latency 0.000013 2026-02-11T23:31:27.966+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0e5b0695:::obj117:head local-lis/les=117/118 n=45 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.966+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.966+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.966+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0e5b0695:::obj117:head local-lis/les=117/118 n=45 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.966+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.966+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.966+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.966+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63b8000 con 0x507ac00 2026-02-11T23:31:27.966+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:0e5e5c87:::obj521:head)]) -- 0x649e480 con 0x507ac00 2026-02-11T23:31:27.966+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.967+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 130 ==== MOSDPGPush(4.0 118/117 [PushOp(4:0ed26813:::obj311:head, version: 111'311, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ed26813:::obj311:head@111'311, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e261a0 con 0x507ac00 2026-02-11T23:31:27.967+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:0ed26813:::obj311:head, version: 111'311, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ed26813:::obj311:head@111'311, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 118 2026-02-11T23:31:27.967+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0ed26813:::obj311:head, version: 111'311, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ed26813:::obj311:head@111'311, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.967+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 131 ==== pg_backfill(progress 4.0 e 118/118 lb 4:0e5e5c87:::obj521:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f11600 con 0x507ac00 2026-02-11T23:31:27.967+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.967+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:0e5e5c87:::obj521:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:27.967+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0e5e5c87:::obj521:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.967+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0ed26813:::obj311:head, version: 111'311, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ed26813:::obj311:head@111'311, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.967+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.967+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0ed26813:::obj311:head, version: 111'311, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ed26813:::obj311:head@111'311, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.967+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.967+0000 7f414f993640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:0ed26813:::obj311:head, version: 111'311, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ed26813:::obj311:head@111'311, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 118/117 [PushOp(4:0ed26813:::obj311:head, version: 111'311, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ed26813:::obj311:head@111'311, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:0e5b0695:::obj117:head local-lis/les=117/118 n=45 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.967+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0e5e5c87:::obj521:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.967+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.967+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0e5b0695:::obj117:head local-lis/les=117/118 n=45 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:0ed26813:::obj311:head, version: 111'311, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ed26813:::obj311:head@111'311, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.967+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0e5b0695:::obj117:head local-lis/les=117/118 n=45 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:0ed26813:::obj311:head@111'311, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.967+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0e5b0695:::obj117:head local-lis/les=117/118 n=45 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:0ed26813:::obj311:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.967+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0e5b0695:::obj117:head local-lis/les=117/118 n=45 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:0ed26813:::obj311:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.967+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0e5b0695:::obj117:head local-lis/les=117/118 n=45 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:0ed26813:::obj311:head 2026-02-11T23:31:27.967+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0e5b0695:::obj117:head local-lis/les=117/118 n=45 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:0ed26813:::obj311:head v 111'311 2026-02-11T23:31:27.967+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0e5b0695:::obj117:head local-lis/les=117/118 n=45 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.967+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.967+0000 7f414f993640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:0ed26813:::obj311:head, version: 111'311, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ed26813:::obj311:head@111'311, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:27.967+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0e5b0695:::obj117:head local-lis/les=117/118 n=45 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.967+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.967+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.967+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0e5e5c87:::obj521:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.967+0000 7f414b98b640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:0e5e5c87:::obj521:head) v3 prio 15 cost 0 latency 0.000082 pg_backfill(progress 4.0 e 118/118 lb 4:0e5e5c87:::obj521:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:0e5b0695:::obj117:head local-lis/les=117/118 n=45 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.967+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.967+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0e5b0695:::obj117:head local-lis/les=117/118 n=45 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:0e5e5c87:::obj521:head) v3 2026-02-11T23:31:27.967+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0e5b0695:::obj117:head local-lis/les=117/118 n=45 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:0e5e5c87:::obj521:head) 2026-02-11T23:31:27.967+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.967+0000 7f414b98b640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:0e5e5c87:::obj521:head) v3 finish latency 0.000016 2026-02-11T23:31:27.967+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.967+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.967+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0e5e5c87:::obj521:head local-lis/les=117/118 n=46 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.967+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.967+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.967+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.967+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63b8480 con 0x507ac00 2026-02-11T23:31:27.967+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:0ed26813:::obj311:head)]) -- 0x63b8180 con 0x507ac00 2026-02-11T23:31:27.967+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.968+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 132 ==== MOSDPGPush(4.0 118/117 [PushOp(4:0ed3c684:::obj270:head, version: 111'270, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ed3c684:::obj270:head@111'270, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3814d00 con 0x507ac00 2026-02-11T23:31:27.968+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:0ed3c684:::obj270:head, version: 111'270, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ed3c684:::obj270:head@111'270, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 118 2026-02-11T23:31:27.968+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0ed3c684:::obj270:head, version: 111'270, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ed3c684:::obj270:head@111'270, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.968+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 133 ==== pg_backfill(progress 4.0 e 118/118 lb 4:0ed26813:::obj311:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2f700 con 0x507ac00 2026-02-11T23:31:27.968+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:0ed26813:::obj311:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 118 2026-02-11T23:31:27.968+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.968+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0ed26813:::obj311:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.968+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0ed3c684:::obj270:head, version: 111'270, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ed3c684:::obj270:head@111'270, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.968+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.968+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0ed3c684:::obj270:head, version: 111'270, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ed3c684:::obj270:head@111'270, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.968+0000 7f414f993640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:0ed3c684:::obj270:head, version: 111'270, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ed3c684:::obj270:head@111'270, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(4.0 118/117 [PushOp(4:0ed3c684:::obj270:head, version: 111'270, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ed3c684:::obj270:head@111'270, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:0e5e5c87:::obj521:head local-lis/les=117/118 n=46 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.968+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.968+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.968+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0e5e5c87:::obj521:head local-lis/les=117/118 n=46 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:0ed3c684:::obj270:head, version: 111'270, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ed3c684:::obj270:head@111'270, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.968+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0ed26813:::obj311:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.968+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0e5e5c87:::obj521:head local-lis/les=117/118 n=46 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:0ed3c684:::obj270:head@111'270, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.968+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0e5e5c87:::obj521:head local-lis/les=117/118 n=46 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:0ed3c684:::obj270:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.968+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0e5e5c87:::obj521:head local-lis/les=117/118 n=46 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:0ed3c684:::obj270:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.968+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0e5e5c87:::obj521:head local-lis/les=117/118 n=46 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:0ed3c684:::obj270:head 2026-02-11T23:31:27.968+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0e5e5c87:::obj521:head local-lis/les=117/118 n=46 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:0ed3c684:::obj270:head v 111'270 2026-02-11T23:31:27.968+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0e5e5c87:::obj521:head local-lis/les=117/118 n=46 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.968+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.968+0000 7f414f993640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:0ed3c684:::obj270:head, version: 111'270, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ed3c684:::obj270:head@111'270, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:27.968+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0e5e5c87:::obj521:head local-lis/les=117/118 n=46 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.968+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.968+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.968+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0ed26813:::obj311:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.968+0000 7f414b98b640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:0ed26813:::obj311:head) v3 prio 15 cost 0 latency 0.000080 pg_backfill(progress 4.0 e 118/118 lb 4:0ed26813:::obj311:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:0e5e5c87:::obj521:head local-lis/les=117/118 n=46 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.968+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.968+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0e5e5c87:::obj521:head local-lis/les=117/118 n=46 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:0ed26813:::obj311:head) v3 2026-02-11T23:31:27.968+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0e5e5c87:::obj521:head local-lis/les=117/118 n=46 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:0ed26813:::obj311:head) 2026-02-11T23:31:27.968+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.968+0000 7f414b98b640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:0ed26813:::obj311:head) v3 finish latency 0.000009 2026-02-11T23:31:27.968+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.968+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.968+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0ed26813:::obj311:head local-lis/les=117/118 n=47 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.968+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.968+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.968+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.968+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63b8900 con 0x507ac00 2026-02-11T23:31:27.968+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:0ed3c684:::obj270:head)]) -- 0x63b8600 con 0x507ac00 2026-02-11T23:31:27.968+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.968+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 134 ==== MOSDPGPush(4.0 118/117 [PushOp(4:0ee9ae15:::obj4:head, version: 111'4, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ee9ae15:::obj4:head@111'4, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 4999+0+0 (unknown 0 0 0) 0x3815380 con 0x507ac00 2026-02-11T23:31:27.968+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:0ee9ae15:::obj4:head, version: 111'4, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ee9ae15:::obj4:head@111'4, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 118 2026-02-11T23:31:27.968+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0ee9ae15:::obj4:head, version: 111'4, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ee9ae15:::obj4:head@111'4, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.968+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.968+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 135 ==== pg_backfill(progress 4.0 e 118/118 lb 4:0ed3c684:::obj270:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2d080 con 0x507ac00 2026-02-11T23:31:27.968+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:0ed3c684:::obj270:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 118 2026-02-11T23:31:27.968+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0ee9ae15:::obj4:head, version: 111'4, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ee9ae15:::obj4:head@111'4, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.968+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0ed3c684:::obj270:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.968+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.968+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0ee9ae15:::obj4:head, version: 111'4, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ee9ae15:::obj4:head@111'4, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.968+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:0ee9ae15:::obj4:head, version: 111'4, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ee9ae15:::obj4:head@111'4, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(4.0 118/117 [PushOp(4:0ee9ae15:::obj4:head, version: 111'4, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ee9ae15:::obj4:head@111'4, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:0ed26813:::obj311:head local-lis/les=117/118 n=47 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.968+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.968+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.968+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0ed26813:::obj311:head local-lis/les=117/118 n=47 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:0ee9ae15:::obj4:head, version: 111'4, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ee9ae15:::obj4:head@111'4, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.968+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0ed3c684:::obj270:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.968+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0ed26813:::obj311:head local-lis/les=117/118 n=47 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:0ee9ae15:::obj4:head@111'4, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.968+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0ed26813:::obj311:head local-lis/les=117/118 n=47 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:0ee9ae15:::obj4:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.968+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0ed26813:::obj311:head local-lis/les=117/118 n=47 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:0ee9ae15:::obj4:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.968+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0ed26813:::obj311:head local-lis/les=117/118 n=47 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:0ee9ae15:::obj4:head 2026-02-11T23:31:27.968+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0ed26813:::obj311:head local-lis/les=117/118 n=47 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:0ee9ae15:::obj4:head v 111'4 2026-02-11T23:31:27.968+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0ed26813:::obj311:head local-lis/les=117/118 n=47 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.968+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.968+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:0ee9ae15:::obj4:head, version: 111'4, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ee9ae15:::obj4:head@111'4, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:27.968+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.968+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.968+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0ed3c684:::obj270:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.968+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:0ed3c684:::obj270:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 4.0 e 118/118 lb 4:0ed3c684:::obj270:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:0ed26813:::obj311:head local-lis/les=117/118 n=47 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.968+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.968+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0ed26813:::obj311:head local-lis/les=117/118 n=47 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:0ed3c684:::obj270:head) v3 2026-02-11T23:31:27.968+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0ed26813:::obj311:head local-lis/les=117/118 n=47 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:0ed3c684:::obj270:head) 2026-02-11T23:31:27.968+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.968+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:0ed3c684:::obj270:head) v3 finish latency 0.000016 2026-02-11T23:31:27.968+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0ed3c684:::obj270:head local-lis/les=117/118 n=48 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.968+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.968+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.968+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0ed3c684:::obj270:head local-lis/les=117/118 n=48 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.968+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.968+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.968+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.968+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63b8a80 con 0x507ac00 2026-02-11T23:31:27.968+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:0ee9ae15:::obj4:head)]) -- 0x649e180 con 0x507ac00 2026-02-11T23:31:27.968+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.969+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 136 ==== MOSDPGPush(4.0 118/117 [PushOp(4:0efd4012:::obj445:head, version: 111'445, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0efd4012:::obj445:head@111'445, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c340 con 0x507ac00 2026-02-11T23:31:27.969+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:0efd4012:::obj445:head, version: 111'445, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0efd4012:::obj445:head@111'445, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 118 2026-02-11T23:31:27.969+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0efd4012:::obj445:head, version: 111'445, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0efd4012:::obj445:head@111'445, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.969+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.969+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 137 ==== pg_backfill(progress 4.0 e 118/118 lb 4:0ee9ae15:::obj4:head) ==== 1060+0+0 (unknown 0 0 0) 0x5d2e680 con 0x507ac00 2026-02-11T23:31:27.969+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:0ee9ae15:::obj4:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:27.969+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0efd4012:::obj445:head, version: 111'445, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0efd4012:::obj445:head@111'445, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.969+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0ee9ae15:::obj4:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.969+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.969+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0efd4012:::obj445:head, version: 111'445, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0efd4012:::obj445:head@111'445, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.969+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:0efd4012:::obj445:head, version: 111'445, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0efd4012:::obj445:head@111'445, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 118/117 [PushOp(4:0efd4012:::obj445:head, version: 111'445, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0efd4012:::obj445:head@111'445, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:0ed3c684:::obj270:head local-lis/les=117/118 n=48 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.969+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.969+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.969+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0ee9ae15:::obj4:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.969+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0ed3c684:::obj270:head local-lis/les=117/118 n=48 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:0efd4012:::obj445:head, version: 111'445, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0efd4012:::obj445:head@111'445, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.969+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0ed3c684:::obj270:head local-lis/les=117/118 n=48 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:0efd4012:::obj445:head@111'445, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.969+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0ed3c684:::obj270:head local-lis/les=117/118 n=48 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:0efd4012:::obj445:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.969+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0ed3c684:::obj270:head local-lis/les=117/118 n=48 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:0efd4012:::obj445:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.969+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0ed3c684:::obj270:head local-lis/les=117/118 n=48 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:0efd4012:::obj445:head 2026-02-11T23:31:27.969+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0ed3c684:::obj270:head local-lis/les=117/118 n=48 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:0efd4012:::obj445:head v 111'445 2026-02-11T23:31:27.969+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0ed3c684:::obj270:head local-lis/les=117/118 n=48 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.969+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.969+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:0efd4012:::obj445:head, version: 111'445, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0efd4012:::obj445:head@111'445, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:27.969+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.969+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.969+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0ee9ae15:::obj4:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.969+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:0ee9ae15:::obj4:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 4.0 e 118/118 lb 4:0ee9ae15:::obj4:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:0ed3c684:::obj270:head local-lis/les=117/118 n=48 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.969+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.969+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0ed3c684:::obj270:head local-lis/les=117/118 n=48 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:0ee9ae15:::obj4:head) v3 2026-02-11T23:31:27.969+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0ed3c684:::obj270:head local-lis/les=117/118 n=48 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:0ee9ae15:::obj4:head) 2026-02-11T23:31:27.969+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.969+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:0ee9ae15:::obj4:head) v3 finish latency 0.000013 2026-02-11T23:31:27.969+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0ee9ae15:::obj4:head local-lis/les=117/118 n=49 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.969+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.969+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.969+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0ee9ae15:::obj4:head local-lis/les=117/118 n=49 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.969+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.969+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.969+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.969+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63b8c00 con 0x507ac00 2026-02-11T23:31:27.969+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:0efd4012:::obj445:head)]) -- 0x5f75500 con 0x507ac00 2026-02-11T23:31:27.969+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.969+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 138 ==== MOSDPGPush(4.0 118/117 [PushOp(4:0f33f807:::obj273:head, version: 111'273, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0f33f807:::obj273:head@111'273, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x555cea0 con 0x507ac00 2026-02-11T23:31:27.969+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:0f33f807:::obj273:head, version: 111'273, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0f33f807:::obj273:head@111'273, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 118 2026-02-11T23:31:27.969+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0f33f807:::obj273:head, version: 111'273, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0f33f807:::obj273:head@111'273, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.969+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.969+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 139 ==== pg_backfill(progress 4.0 e 118/118 lb 4:0efd4012:::obj445:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e100 con 0x507ac00 2026-02-11T23:31:27.969+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:0efd4012:::obj445:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:27.969+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0f33f807:::obj273:head, version: 111'273, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0f33f807:::obj273:head@111'273, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.969+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0efd4012:::obj445:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.969+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.969+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0f33f807:::obj273:head, version: 111'273, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0f33f807:::obj273:head@111'273, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.969+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.969+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0efd4012:::obj445:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.969+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:0f33f807:::obj273:head, version: 111'273, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0f33f807:::obj273:head@111'273, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 118/117 [PushOp(4:0f33f807:::obj273:head, version: 111'273, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0f33f807:::obj273:head@111'273, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:0ee9ae15:::obj4:head local-lis/les=117/118 n=49 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.969+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.969+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0ee9ae15:::obj4:head local-lis/les=117/118 n=49 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:0f33f807:::obj273:head, version: 111'273, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0f33f807:::obj273:head@111'273, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.969+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0ee9ae15:::obj4:head local-lis/les=117/118 n=49 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:0f33f807:::obj273:head@111'273, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.969+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0ee9ae15:::obj4:head local-lis/les=117/118 n=49 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:0f33f807:::obj273:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.969+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0ee9ae15:::obj4:head local-lis/les=117/118 n=49 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:0f33f807:::obj273:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.969+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0ee9ae15:::obj4:head local-lis/les=117/118 n=49 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:0f33f807:::obj273:head 2026-02-11T23:31:27.969+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0ee9ae15:::obj4:head local-lis/les=117/118 n=49 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:0f33f807:::obj273:head v 111'273 2026-02-11T23:31:27.969+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0ee9ae15:::obj4:head local-lis/les=117/118 n=49 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.969+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.970+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:0f33f807:::obj273:head, version: 111'273, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0f33f807:::obj273:head@111'273, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:27.970+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.970+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.970+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0efd4012:::obj445:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.970+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:0efd4012:::obj445:head) v3 prio 15 cost 0 latency 0.000085 pg_backfill(progress 4.0 e 118/118 lb 4:0efd4012:::obj445:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:0ee9ae15:::obj4:head local-lis/les=117/118 n=49 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.970+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.970+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0ee9ae15:::obj4:head local-lis/les=117/118 n=49 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:0efd4012:::obj445:head) v3 2026-02-11T23:31:27.970+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0ee9ae15:::obj4:head local-lis/les=117/118 n=49 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:0efd4012:::obj445:head) 2026-02-11T23:31:27.970+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.970+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:0efd4012:::obj445:head) v3 finish latency 0.000019 2026-02-11T23:31:27.970+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0efd4012:::obj445:head local-lis/les=117/118 n=50 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.970+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.970+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.970+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0efd4012:::obj445:head local-lis/les=117/118 n=50 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.970+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.970+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.970+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.970+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63b8d80 con 0x507ac00 2026-02-11T23:31:27.970+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:0f33f807:::obj273:head)]) -- 0x5f1d200 con 0x507ac00 2026-02-11T23:31:27.970+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.970+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 140 ==== MOSDPGPush(4.0 118/117 [PushOp(4:0f45d4cb:::obj171:head, version: 111'171, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0f45d4cb:::obj171:head@111'171, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56cab60 con 0x507ac00 2026-02-11T23:31:27.970+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:0f45d4cb:::obj171:head, version: 111'171, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0f45d4cb:::obj171:head@111'171, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 118 2026-02-11T23:31:27.970+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0f45d4cb:::obj171:head, version: 111'171, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0f45d4cb:::obj171:head@111'171, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.970+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.970+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 141 ==== pg_backfill(progress 4.0 e 118/118 lb 4:0f33f807:::obj273:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2c000 con 0x507ac00 2026-02-11T23:31:27.970+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0f45d4cb:::obj171:head, version: 111'171, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0f45d4cb:::obj171:head@111'171, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.970+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:0f33f807:::obj273:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 118 2026-02-11T23:31:27.970+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.970+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0f33f807:::obj273:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.970+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:0f45d4cb:::obj171:head, version: 111'171, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0f45d4cb:::obj171:head@111'171, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.970+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.970+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:0f45d4cb:::obj171:head, version: 111'171, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0f45d4cb:::obj171:head@111'171, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(4.0 118/117 [PushOp(4:0f45d4cb:::obj171:head, version: 111'171, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0f45d4cb:::obj171:head@111'171, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:0efd4012:::obj445:head local-lis/les=117/118 n=50 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.970+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.970+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0efd4012:::obj445:head local-lis/les=117/118 n=50 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:0f45d4cb:::obj171:head, version: 111'171, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0f45d4cb:::obj171:head@111'171, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.970+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0efd4012:::obj445:head local-lis/les=117/118 n=50 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:0f45d4cb:::obj171:head@111'171, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.970+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.970+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0f33f807:::obj273:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.970+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0efd4012:::obj445:head local-lis/les=117/118 n=50 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:0f45d4cb:::obj171:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.970+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0efd4012:::obj445:head local-lis/les=117/118 n=50 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:0f45d4cb:::obj171:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.970+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0efd4012:::obj445:head local-lis/les=117/118 n=50 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:0f45d4cb:::obj171:head 2026-02-11T23:31:27.970+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0efd4012:::obj445:head local-lis/les=117/118 n=50 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:0f45d4cb:::obj171:head v 111'171 2026-02-11T23:31:27.970+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0efd4012:::obj445:head local-lis/les=117/118 n=50 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.970+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.970+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:0f45d4cb:::obj171:head, version: 111'171, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0f45d4cb:::obj171:head@111'171, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:27.970+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.970+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.970+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0f33f807:::obj273:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.970+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:0f33f807:::obj273:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 4.0 e 118/118 lb 4:0f33f807:::obj273:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:0efd4012:::obj445:head local-lis/les=117/118 n=50 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.970+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.970+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0efd4012:::obj445:head local-lis/les=117/118 n=50 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:0f33f807:::obj273:head) v3 2026-02-11T23:31:27.970+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0efd4012:::obj445:head local-lis/les=117/118 n=50 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:0f33f807:::obj273:head) 2026-02-11T23:31:27.970+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.970+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:0f33f807:::obj273:head) v3 finish latency 0.000019 2026-02-11T23:31:27.970+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0f33f807:::obj273:head local-lis/les=117/118 n=51 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.970+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.970+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.970+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0f33f807:::obj273:head local-lis/les=117/118 n=51 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.970+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.970+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.970+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.970+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63b8f00 con 0x507ac00 2026-02-11T23:31:27.970+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:0f45d4cb:::obj171:head)]) -- 0x5f75380 con 0x507ac00 2026-02-11T23:31:27.970+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.971+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 142 ==== MOSDPGPush(4.0 118/117 [PushOp(4:104778fc:::obj2:head, version: 111'2, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:104778fc:::obj2:head@111'2, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 4999+0+0 (unknown 0 0 0) 0x56a2680 con 0x507ac00 2026-02-11T23:31:27.971+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:104778fc:::obj2:head, version: 111'2, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:104778fc:::obj2:head@111'2, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 118 2026-02-11T23:31:27.971+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:104778fc:::obj2:head, version: 111'2, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:104778fc:::obj2:head@111'2, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.971+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.971+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 143 ==== pg_backfill(progress 4.0 e 118/118 lb 4:0f45d4cb:::obj171:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2cb00 con 0x507ac00 2026-02-11T23:31:27.971+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:104778fc:::obj2:head, version: 111'2, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:104778fc:::obj2:head@111'2, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.971+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:0f45d4cb:::obj171:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 118 2026-02-11T23:31:27.971+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0f45d4cb:::obj171:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.971+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.971+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:104778fc:::obj2:head, version: 111'2, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:104778fc:::obj2:head@111'2, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.971+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:104778fc:::obj2:head, version: 111'2, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:104778fc:::obj2:head@111'2, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 118/117 [PushOp(4:104778fc:::obj2:head, version: 111'2, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:104778fc:::obj2:head@111'2, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:0f33f807:::obj273:head local-lis/les=117/118 n=51 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.971+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.971+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.971+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0f45d4cb:::obj171:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.971+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0f33f807:::obj273:head local-lis/les=117/118 n=51 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:104778fc:::obj2:head, version: 111'2, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:104778fc:::obj2:head@111'2, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.971+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0f33f807:::obj273:head local-lis/les=117/118 n=51 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:104778fc:::obj2:head@111'2, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.971+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0f33f807:::obj273:head local-lis/les=117/118 n=51 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:104778fc:::obj2:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.971+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0f33f807:::obj273:head local-lis/les=117/118 n=51 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:104778fc:::obj2:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.971+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0f33f807:::obj273:head local-lis/les=117/118 n=51 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:104778fc:::obj2:head 2026-02-11T23:31:27.971+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0f33f807:::obj273:head local-lis/les=117/118 n=51 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:104778fc:::obj2:head v 111'2 2026-02-11T23:31:27.971+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0f33f807:::obj273:head local-lis/les=117/118 n=51 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.971+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.971+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:104778fc:::obj2:head, version: 111'2, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:104778fc:::obj2:head@111'2, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:27.971+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.971+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.971+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:0f45d4cb:::obj171:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.971+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:0f45d4cb:::obj171:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 4.0 e 118/118 lb 4:0f45d4cb:::obj171:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:0f33f807:::obj273:head local-lis/les=117/118 n=51 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.971+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.971+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0f33f807:::obj273:head local-lis/les=117/118 n=51 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:0f45d4cb:::obj171:head) v3 2026-02-11T23:31:27.971+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0f33f807:::obj273:head local-lis/les=117/118 n=51 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:0f45d4cb:::obj171:head) 2026-02-11T23:31:27.971+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.971+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:0f45d4cb:::obj171:head) v3 finish latency 0.000012 2026-02-11T23:31:27.971+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0f45d4cb:::obj171:head local-lis/les=117/118 n=52 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.971+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.971+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.971+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0f45d4cb:::obj171:head local-lis/les=117/118 n=52 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.971+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.971+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.971+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.971+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63b9080 con 0x507ac00 2026-02-11T23:31:27.971+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:104778fc:::obj2:head)]) -- 0x5f2c900 con 0x507ac00 2026-02-11T23:31:27.971+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.971+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 144 ==== MOSDPGPush(4.0 118/117 [PushOp(4:105983a6:::obj88:head, version: 111'88, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:105983a6:::obj88:head@111'88, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x56aa000 con 0x507ac00 2026-02-11T23:31:27.971+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:105983a6:::obj88:head, version: 111'88, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:105983a6:::obj88:head@111'88, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 118 2026-02-11T23:31:27.971+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:105983a6:::obj88:head, version: 111'88, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:105983a6:::obj88:head@111'88, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.971+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.972+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:105983a6:::obj88:head, version: 111'88, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:105983a6:::obj88:head@111'88, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.971+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 145 ==== pg_backfill(progress 4.0 e 118/118 lb 4:104778fc:::obj2:head) ==== 1060+0+0 (unknown 0 0 0) 0x6179700 con 0x507ac00 2026-02-11T23:31:27.972+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:104778fc:::obj2:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 118 2026-02-11T23:31:27.972+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.972+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:104778fc:::obj2:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.972+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:105983a6:::obj88:head, version: 111'88, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:105983a6:::obj88:head@111'88, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.972+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.972+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:105983a6:::obj88:head, version: 111'88, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:105983a6:::obj88:head@111'88, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(4.0 118/117 [PushOp(4:105983a6:::obj88:head, version: 111'88, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:105983a6:::obj88:head@111'88, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:0f45d4cb:::obj171:head local-lis/les=117/118 n=52 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.972+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.972+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0f45d4cb:::obj171:head local-lis/les=117/118 n=52 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:105983a6:::obj88:head, version: 111'88, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:105983a6:::obj88:head@111'88, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.972+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.972+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0f45d4cb:::obj171:head local-lis/les=117/118 n=52 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:105983a6:::obj88:head@111'88, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.972+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:104778fc:::obj2:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.972+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0f45d4cb:::obj171:head local-lis/les=117/118 n=52 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:105983a6:::obj88:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.972+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0f45d4cb:::obj171:head local-lis/les=117/118 n=52 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:105983a6:::obj88:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.972+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0f45d4cb:::obj171:head local-lis/les=117/118 n=52 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:105983a6:::obj88:head 2026-02-11T23:31:27.972+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0f45d4cb:::obj171:head local-lis/les=117/118 n=52 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:105983a6:::obj88:head v 111'88 2026-02-11T23:31:27.972+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0f45d4cb:::obj171:head local-lis/les=117/118 n=52 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.972+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.972+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:105983a6:::obj88:head, version: 111'88, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:105983a6:::obj88:head@111'88, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:27.972+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.972+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.972+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:104778fc:::obj2:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.972+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:104778fc:::obj2:head) v3 prio 15 cost 0 latency 0.000082 pg_backfill(progress 4.0 e 118/118 lb 4:104778fc:::obj2:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:0f45d4cb:::obj171:head local-lis/les=117/118 n=52 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.972+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.972+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0f45d4cb:::obj171:head local-lis/les=117/118 n=52 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:104778fc:::obj2:head) v3 2026-02-11T23:31:27.972+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:0f45d4cb:::obj171:head local-lis/les=117/118 n=52 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:104778fc:::obj2:head) 2026-02-11T23:31:27.972+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.972+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:104778fc:::obj2:head) v3 finish latency 0.000013 2026-02-11T23:31:27.972+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:104778fc:::obj2:head local-lis/les=117/118 n=53 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.972+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.972+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.972+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:104778fc:::obj2:head local-lis/les=117/118 n=53 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.972+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.972+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.972+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.972+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63b9200 con 0x507ac00 2026-02-11T23:31:27.972+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:105983a6:::obj88:head)]) -- 0x600b680 con 0x507ac00 2026-02-11T23:31:27.972+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.972+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 146 ==== MOSDPGPush(4.0 118/117 [PushOp(4:10754df1:::obj159:head, version: 111'159, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10754df1:::obj159:head@111'159, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a3a00 con 0x507ac00 2026-02-11T23:31:27.972+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:10754df1:::obj159:head, version: 111'159, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10754df1:::obj159:head@111'159, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 118 2026-02-11T23:31:27.972+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:10754df1:::obj159:head, version: 111'159, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10754df1:::obj159:head@111'159, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.972+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 147 ==== pg_backfill(progress 4.0 e 118/118 lb 4:105983a6:::obj88:head) ==== 1061+0+0 (unknown 0 0 0) 0x6176000 con 0x507ac00 2026-02-11T23:31:27.972+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.972+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:105983a6:::obj88:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 118 2026-02-11T23:31:27.972+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:105983a6:::obj88:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.972+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:10754df1:::obj159:head, version: 111'159, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10754df1:::obj159:head@111'159, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.972+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.972+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:10754df1:::obj159:head, version: 111'159, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10754df1:::obj159:head@111'159, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.972+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:10754df1:::obj159:head, version: 111'159, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10754df1:::obj159:head@111'159, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 118/117 [PushOp(4:10754df1:::obj159:head, version: 111'159, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10754df1:::obj159:head@111'159, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:104778fc:::obj2:head local-lis/les=117/118 n=53 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.972+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.972+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.972+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:105983a6:::obj88:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.972+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:104778fc:::obj2:head local-lis/les=117/118 n=53 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:10754df1:::obj159:head, version: 111'159, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10754df1:::obj159:head@111'159, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.972+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:104778fc:::obj2:head local-lis/les=117/118 n=53 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:10754df1:::obj159:head@111'159, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.972+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:104778fc:::obj2:head local-lis/les=117/118 n=53 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:10754df1:::obj159:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.972+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:104778fc:::obj2:head local-lis/les=117/118 n=53 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:10754df1:::obj159:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.972+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:104778fc:::obj2:head local-lis/les=117/118 n=53 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:10754df1:::obj159:head 2026-02-11T23:31:27.972+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:104778fc:::obj2:head local-lis/les=117/118 n=53 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:10754df1:::obj159:head v 111'159 2026-02-11T23:31:27.972+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:104778fc:::obj2:head local-lis/les=117/118 n=53 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.972+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.972+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:10754df1:::obj159:head, version: 111'159, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10754df1:::obj159:head@111'159, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:27.972+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.972+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.972+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:105983a6:::obj88:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.972+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:105983a6:::obj88:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 4.0 e 118/118 lb 4:105983a6:::obj88:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:104778fc:::obj2:head local-lis/les=117/118 n=53 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.972+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.972+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:104778fc:::obj2:head local-lis/les=117/118 n=53 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:105983a6:::obj88:head) v3 2026-02-11T23:31:27.972+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:104778fc:::obj2:head local-lis/les=117/118 n=53 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:105983a6:::obj88:head) 2026-02-11T23:31:27.972+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.972+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:105983a6:::obj88:head) v3 finish latency 0.000016 2026-02-11T23:31:27.972+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:105983a6:::obj88:head local-lis/les=117/118 n=54 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.972+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.972+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.972+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:105983a6:::obj88:head local-lis/les=117/118 n=54 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.973+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.973+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.973+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.973+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.973+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63b9380 con 0x507ac00 2026-02-11T23:31:27.973+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:10754df1:::obj159:head)]) -- 0x5f75080 con 0x507ac00 2026-02-11T23:31:27.973+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.973+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 148 ==== MOSDPGPush(4.0 118/117 [PushOp(4:10b422cc:::obj197:head, version: 111'197, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10b422cc:::obj197:head@111'197, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a29c0 con 0x507ac00 2026-02-11T23:31:27.973+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:10b422cc:::obj197:head, version: 111'197, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10b422cc:::obj197:head@111'197, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 118 2026-02-11T23:31:27.973+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:10b422cc:::obj197:head, version: 111'197, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10b422cc:::obj197:head@111'197, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.973+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 149 ==== pg_backfill(progress 4.0 e 118/118 lb 4:10754df1:::obj159:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176b00 con 0x507ac00 2026-02-11T23:31:27.973+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:10754df1:::obj159:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:27.973+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:10754df1:::obj159:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.973+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.973+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:10b422cc:::obj197:head, version: 111'197, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10b422cc:::obj197:head@111'197, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.973+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.973+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:10b422cc:::obj197:head, version: 111'197, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10b422cc:::obj197:head@111'197, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.973+0000 7f414f993640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:10b422cc:::obj197:head, version: 111'197, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10b422cc:::obj197:head@111'197, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(4.0 118/117 [PushOp(4:10b422cc:::obj197:head, version: 111'197, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10b422cc:::obj197:head@111'197, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:105983a6:::obj88:head local-lis/les=117/118 n=54 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.973+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.973+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.973+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:105983a6:::obj88:head local-lis/les=117/118 n=54 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:10b422cc:::obj197:head, version: 111'197, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10b422cc:::obj197:head@111'197, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.973+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:10754df1:::obj159:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.973+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:105983a6:::obj88:head local-lis/les=117/118 n=54 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:10b422cc:::obj197:head@111'197, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.973+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:105983a6:::obj88:head local-lis/les=117/118 n=54 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:10b422cc:::obj197:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.973+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:105983a6:::obj88:head local-lis/les=117/118 n=54 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:10b422cc:::obj197:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.973+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:105983a6:::obj88:head local-lis/les=117/118 n=54 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:10b422cc:::obj197:head 2026-02-11T23:31:27.973+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:105983a6:::obj88:head local-lis/les=117/118 n=54 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:10b422cc:::obj197:head v 111'197 2026-02-11T23:31:27.973+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:105983a6:::obj88:head local-lis/les=117/118 n=54 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.973+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.973+0000 7f414f993640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:10b422cc:::obj197:head, version: 111'197, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10b422cc:::obj197:head@111'197, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000072 2026-02-11T23:31:27.973+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:105983a6:::obj88:head local-lis/les=117/118 n=54 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.973+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.973+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.973+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:10754df1:::obj159:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.973+0000 7f414b98b640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:10754df1:::obj159:head) v3 prio 15 cost 0 latency 0.000103 pg_backfill(progress 4.0 e 118/118 lb 4:10754df1:::obj159:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:105983a6:::obj88:head local-lis/les=117/118 n=54 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.973+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.973+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:105983a6:::obj88:head local-lis/les=117/118 n=54 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:10754df1:::obj159:head) v3 2026-02-11T23:31:27.973+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:105983a6:::obj88:head local-lis/les=117/118 n=54 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:10754df1:::obj159:head) 2026-02-11T23:31:27.973+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.973+0000 7f414b98b640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:10754df1:::obj159:head) v3 finish latency 0.000014 2026-02-11T23:31:27.973+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.973+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.973+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:10754df1:::obj159:head local-lis/les=117/118 n=55 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.973+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.973+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.973+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.973+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63b9800 con 0x507ac00 2026-02-11T23:31:27.973+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:10b422cc:::obj197:head)]) -- 0x63b9500 con 0x507ac00 2026-02-11T23:31:27.973+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.974+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 150 ==== MOSDPGPush(4.0 118/117 [PushOp(4:10c2d3a1:::obj448:head, version: 111'448, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10c2d3a1:::obj448:head@111'448, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2ea0 con 0x507ac00 2026-02-11T23:31:27.974+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:10c2d3a1:::obj448:head, version: 111'448, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10c2d3a1:::obj448:head@111'448, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 118 2026-02-11T23:31:27.974+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:10c2d3a1:::obj448:head, version: 111'448, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10c2d3a1:::obj448:head@111'448, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.974+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.974+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 151 ==== pg_backfill(progress 4.0 e 118/118 lb 4:10b422cc:::obj197:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177080 con 0x507ac00 2026-02-11T23:31:27.974+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:10b422cc:::obj197:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 118 2026-02-11T23:31:27.974+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:10b422cc:::obj197:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.974+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:10c2d3a1:::obj448:head, version: 111'448, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10c2d3a1:::obj448:head@111'448, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.974+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.974+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:10c2d3a1:::obj448:head, version: 111'448, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10c2d3a1:::obj448:head@111'448, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.974+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:10c2d3a1:::obj448:head, version: 111'448, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10c2d3a1:::obj448:head@111'448, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000051 MOSDPGPush(4.0 118/117 [PushOp(4:10c2d3a1:::obj448:head, version: 111'448, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10c2d3a1:::obj448:head@111'448, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:10754df1:::obj159:head local-lis/les=117/118 n=55 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.974+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.974+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.974+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:10b422cc:::obj197:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.974+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:10754df1:::obj159:head local-lis/les=117/118 n=55 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:10c2d3a1:::obj448:head, version: 111'448, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10c2d3a1:::obj448:head@111'448, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.974+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:10754df1:::obj159:head local-lis/les=117/118 n=55 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:10c2d3a1:::obj448:head@111'448, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.974+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:10754df1:::obj159:head local-lis/les=117/118 n=55 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:10c2d3a1:::obj448:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.974+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:10754df1:::obj159:head local-lis/les=117/118 n=55 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:10c2d3a1:::obj448:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.974+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:10754df1:::obj159:head local-lis/les=117/118 n=55 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:10c2d3a1:::obj448:head 2026-02-11T23:31:27.974+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:10754df1:::obj159:head local-lis/les=117/118 n=55 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:10c2d3a1:::obj448:head v 111'448 2026-02-11T23:31:27.974+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:10754df1:::obj159:head local-lis/les=117/118 n=55 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.974+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.974+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:10c2d3a1:::obj448:head, version: 111'448, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10c2d3a1:::obj448:head@111'448, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000110 2026-02-11T23:31:27.974+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.974+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.974+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:10b422cc:::obj197:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.974+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:10b422cc:::obj197:head) v3 prio 15 cost 0 latency 0.000152 pg_backfill(progress 4.0 e 118/118 lb 4:10b422cc:::obj197:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:10754df1:::obj159:head local-lis/les=117/118 n=55 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.974+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.974+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:10754df1:::obj159:head local-lis/les=117/118 n=55 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:10b422cc:::obj197:head) v3 2026-02-11T23:31:27.974+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:10754df1:::obj159:head local-lis/les=117/118 n=55 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:10b422cc:::obj197:head) 2026-02-11T23:31:27.974+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.974+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:10b422cc:::obj197:head) v3 finish latency 0.000018 2026-02-11T23:31:27.974+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:10b422cc:::obj197:head local-lis/les=117/118 n=56 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.974+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.974+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.974+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:10b422cc:::obj197:head local-lis/les=117/118 n=56 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.974+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.974+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.974+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.974+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63b9980 con 0x507ac00 2026-02-11T23:31:27.974+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:10c2d3a1:::obj448:head)]) -- 0x5fed380 con 0x507ac00 2026-02-11T23:31:27.974+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.974+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 152 ==== MOSDPGPush(4.0 118/117 [PushOp(4:10f4e680:::obj204:head, version: 111'204, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10f4e680:::obj204:head@111'204, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56ca680 con 0x507ac00 2026-02-11T23:31:27.974+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:10f4e680:::obj204:head, version: 111'204, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10f4e680:::obj204:head@111'204, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 118 2026-02-11T23:31:27.974+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:10f4e680:::obj204:head, version: 111'204, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10f4e680:::obj204:head@111'204, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.974+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.974+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 153 ==== pg_backfill(progress 4.0 e 118/118 lb 4:10c2d3a1:::obj448:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177600 con 0x507ac00 2026-02-11T23:31:27.974+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:10c2d3a1:::obj448:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 118 2026-02-11T23:31:27.974+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:10f4e680:::obj204:head, version: 111'204, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10f4e680:::obj204:head@111'204, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.974+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:10c2d3a1:::obj448:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.974+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.974+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:10f4e680:::obj204:head, version: 111'204, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10f4e680:::obj204:head@111'204, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.974+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.974+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:10c2d3a1:::obj448:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.974+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:10f4e680:::obj204:head, version: 111'204, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10f4e680:::obj204:head@111'204, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000044 MOSDPGPush(4.0 118/117 [PushOp(4:10f4e680:::obj204:head, version: 111'204, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10f4e680:::obj204:head@111'204, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:10b422cc:::obj197:head local-lis/les=117/118 n=56 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.974+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.974+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:10b422cc:::obj197:head local-lis/les=117/118 n=56 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:10f4e680:::obj204:head, version: 111'204, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10f4e680:::obj204:head@111'204, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.974+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:10b422cc:::obj197:head local-lis/les=117/118 n=56 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:10f4e680:::obj204:head@111'204, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.974+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:10b422cc:::obj197:head local-lis/les=117/118 n=56 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:10f4e680:::obj204:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.974+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:10b422cc:::obj197:head local-lis/les=117/118 n=56 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:10f4e680:::obj204:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.974+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:10b422cc:::obj197:head local-lis/les=117/118 n=56 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:10f4e680:::obj204:head 2026-02-11T23:31:27.974+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:10b422cc:::obj197:head local-lis/les=117/118 n=56 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:10f4e680:::obj204:head v 111'204 2026-02-11T23:31:27.974+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:10b422cc:::obj197:head local-lis/les=117/118 n=56 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.974+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.974+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:10f4e680:::obj204:head, version: 111'204, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10f4e680:::obj204:head@111'204, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000071 2026-02-11T23:31:27.974+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.974+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.974+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:10c2d3a1:::obj448:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.974+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:10c2d3a1:::obj448:head) v3 prio 15 cost 0 latency 0.000097 pg_backfill(progress 4.0 e 118/118 lb 4:10c2d3a1:::obj448:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:10b422cc:::obj197:head local-lis/les=117/118 n=56 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.974+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.974+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:10b422cc:::obj197:head local-lis/les=117/118 n=56 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:10c2d3a1:::obj448:head) v3 2026-02-11T23:31:27.974+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:10b422cc:::obj197:head local-lis/les=117/118 n=56 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:10c2d3a1:::obj448:head) 2026-02-11T23:31:27.974+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.974+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:10c2d3a1:::obj448:head) v3 finish latency 0.000018 2026-02-11T23:31:27.974+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:10c2d3a1:::obj448:head local-lis/les=117/118 n=57 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.974+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.975+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.975+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:10c2d3a1:::obj448:head local-lis/les=117/118 n=57 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.975+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.975+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.975+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.975+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63b9b00 con 0x507ac00 2026-02-11T23:31:27.975+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:10f4e680:::obj204:head)]) -- 0x5f74000 con 0x507ac00 2026-02-11T23:31:27.975+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.975+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 154 ==== MOSDPGPush(4.0 118/117 [PushOp(4:11426d25:::obj314:head, version: 111'314, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11426d25:::obj314:head@111'314, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2340 con 0x507ac00 2026-02-11T23:31:27.975+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:11426d25:::obj314:head, version: 111'314, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11426d25:::obj314:head@111'314, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 118 2026-02-11T23:31:27.975+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:11426d25:::obj314:head, version: 111'314, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11426d25:::obj314:head@111'314, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.975+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.975+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:11426d25:::obj314:head, version: 111'314, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11426d25:::obj314:head@111'314, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.975+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 155 ==== pg_backfill(progress 4.0 e 118/118 lb 4:10f4e680:::obj204:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177b80 con 0x507ac00 2026-02-11T23:31:27.975+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:10f4e680:::obj204:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 118 2026-02-11T23:31:27.975+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.975+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:10f4e680:::obj204:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.975+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:11426d25:::obj314:head, version: 111'314, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11426d25:::obj314:head@111'314, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.975+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.975+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:11426d25:::obj314:head, version: 111'314, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11426d25:::obj314:head@111'314, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(4.0 118/117 [PushOp(4:11426d25:::obj314:head, version: 111'314, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11426d25:::obj314:head@111'314, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:10c2d3a1:::obj448:head local-lis/les=117/118 n=57 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.975+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.975+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:10c2d3a1:::obj448:head local-lis/les=117/118 n=57 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:11426d25:::obj314:head, version: 111'314, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11426d25:::obj314:head@111'314, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.975+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.975+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:10f4e680:::obj204:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.975+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:10c2d3a1:::obj448:head local-lis/les=117/118 n=57 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:11426d25:::obj314:head@111'314, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.975+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:10c2d3a1:::obj448:head local-lis/les=117/118 n=57 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:11426d25:::obj314:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.975+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:10c2d3a1:::obj448:head local-lis/les=117/118 n=57 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:11426d25:::obj314:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.975+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:10c2d3a1:::obj448:head local-lis/les=117/118 n=57 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:11426d25:::obj314:head 2026-02-11T23:31:27.975+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:10c2d3a1:::obj448:head local-lis/les=117/118 n=57 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:11426d25:::obj314:head v 111'314 2026-02-11T23:31:27.975+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:10c2d3a1:::obj448:head local-lis/les=117/118 n=57 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.975+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.975+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:11426d25:::obj314:head, version: 111'314, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11426d25:::obj314:head@111'314, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000076 2026-02-11T23:31:27.975+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.975+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.975+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:10f4e680:::obj204:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.975+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:10f4e680:::obj204:head) v3 prio 15 cost 0 latency 0.000099 pg_backfill(progress 4.0 e 118/118 lb 4:10f4e680:::obj204:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:10c2d3a1:::obj448:head local-lis/les=117/118 n=57 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.975+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.975+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:10c2d3a1:::obj448:head local-lis/les=117/118 n=57 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:10f4e680:::obj204:head) v3 2026-02-11T23:31:27.975+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:10c2d3a1:::obj448:head local-lis/les=117/118 n=57 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:10f4e680:::obj204:head) 2026-02-11T23:31:27.975+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.975+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:10f4e680:::obj204:head) v3 finish latency 0.000017 2026-02-11T23:31:27.975+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:10f4e680:::obj204:head local-lis/les=117/118 n=58 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.975+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.975+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.975+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:10f4e680:::obj204:head local-lis/les=117/118 n=58 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.975+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.975+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.975+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.975+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63b9c80 con 0x507ac00 2026-02-11T23:31:27.975+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:11426d25:::obj314:head)]) -- 0x5f74f00 con 0x507ac00 2026-02-11T23:31:27.975+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.976+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 156 ==== MOSDPGPush(4.0 118/117 [PushOp(4:11a85aa0:::obj43:head, version: 111'43, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11a85aa0:::obj43:head@111'43, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x554e4e0 con 0x507ac00 2026-02-11T23:31:27.976+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:11a85aa0:::obj43:head, version: 111'43, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11a85aa0:::obj43:head@111'43, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 118 2026-02-11T23:31:27.976+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:11a85aa0:::obj43:head, version: 111'43, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11a85aa0:::obj43:head@111'43, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.976+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.976+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 157 ==== pg_backfill(progress 4.0 e 118/118 lb 4:11426d25:::obj314:head) ==== 1062+0+0 (unknown 0 0 0) 0x6178100 con 0x507ac00 2026-02-11T23:31:27.976+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:11a85aa0:::obj43:head, version: 111'43, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11a85aa0:::obj43:head@111'43, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.976+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:11426d25:::obj314:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 118 2026-02-11T23:31:27.976+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:11426d25:::obj314:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.976+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.976+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:11a85aa0:::obj43:head, version: 111'43, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11a85aa0:::obj43:head@111'43, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.976+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:11a85aa0:::obj43:head, version: 111'43, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11a85aa0:::obj43:head@111'43, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(4.0 118/117 [PushOp(4:11a85aa0:::obj43:head, version: 111'43, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11a85aa0:::obj43:head@111'43, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:10f4e680:::obj204:head local-lis/les=117/118 n=58 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.976+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.976+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.976+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:10f4e680:::obj204:head local-lis/les=117/118 n=58 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:11a85aa0:::obj43:head, version: 111'43, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11a85aa0:::obj43:head@111'43, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.976+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:11426d25:::obj314:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.976+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:10f4e680:::obj204:head local-lis/les=117/118 n=58 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:11a85aa0:::obj43:head@111'43, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.976+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:10f4e680:::obj204:head local-lis/les=117/118 n=58 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:11a85aa0:::obj43:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.976+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:10f4e680:::obj204:head local-lis/les=117/118 n=58 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:11a85aa0:::obj43:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.976+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:10f4e680:::obj204:head local-lis/les=117/118 n=58 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:11a85aa0:::obj43:head 2026-02-11T23:31:27.976+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:10f4e680:::obj204:head local-lis/les=117/118 n=58 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:11a85aa0:::obj43:head v 111'43 2026-02-11T23:31:27.976+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:10f4e680:::obj204:head local-lis/les=117/118 n=58 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.976+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.976+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:11a85aa0:::obj43:head, version: 111'43, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11a85aa0:::obj43:head@111'43, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000065 2026-02-11T23:31:27.976+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.976+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.976+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:11426d25:::obj314:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.976+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:11426d25:::obj314:head) v3 prio 15 cost 0 latency 0.000087 pg_backfill(progress 4.0 e 118/118 lb 4:11426d25:::obj314:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:10f4e680:::obj204:head local-lis/les=117/118 n=58 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.976+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.976+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:10f4e680:::obj204:head local-lis/les=117/118 n=58 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:11426d25:::obj314:head) v3 2026-02-11T23:31:27.976+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:10f4e680:::obj204:head local-lis/les=117/118 n=58 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:11426d25:::obj314:head) 2026-02-11T23:31:27.976+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.976+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:11426d25:::obj314:head) v3 finish latency 0.000018 2026-02-11T23:31:27.976+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:11426d25:::obj314:head local-lis/les=117/118 n=59 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.976+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.976+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.976+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:11426d25:::obj314:head local-lis/les=117/118 n=59 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.976+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.976+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.976+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.976+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63b9e00 con 0x507ac00 2026-02-11T23:31:27.976+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:11a85aa0:::obj43:head)]) -- 0x5f74180 con 0x507ac00 2026-02-11T23:31:27.976+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.976+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 158 ==== MOSDPGPush(4.0 118/117 [PushOp(4:11bee109:::obj319:head, version: 111'319, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11bee109:::obj319:head@111'319, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a3ba0 con 0x507ac00 2026-02-11T23:31:27.976+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:11bee109:::obj319:head, version: 111'319, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11bee109:::obj319:head@111'319, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 118 2026-02-11T23:31:27.976+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:11bee109:::obj319:head, version: 111'319, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11bee109:::obj319:head@111'319, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.976+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.976+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 159 ==== pg_backfill(progress 4.0 e 118/118 lb 4:11a85aa0:::obj43:head) ==== 1061+0+0 (unknown 0 0 0) 0x6179180 con 0x507ac00 2026-02-11T23:31:27.976+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:11bee109:::obj319:head, version: 111'319, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11bee109:::obj319:head@111'319, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.976+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:11a85aa0:::obj43:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 118 2026-02-11T23:31:27.976+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:11a85aa0:::obj43:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.976+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.977+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:11bee109:::obj319:head, version: 111'319, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11bee109:::obj319:head@111'319, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.977+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.977+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:11bee109:::obj319:head, version: 111'319, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11bee109:::obj319:head@111'319, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000040 MOSDPGPush(4.0 118/117 [PushOp(4:11bee109:::obj319:head, version: 111'319, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11bee109:::obj319:head@111'319, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:11426d25:::obj314:head local-lis/les=117/118 n=59 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.977+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.977+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:11426d25:::obj314:head local-lis/les=117/118 n=59 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:11bee109:::obj319:head, version: 111'319, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11bee109:::obj319:head@111'319, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.977+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.977+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:11a85aa0:::obj43:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.977+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:11426d25:::obj314:head local-lis/les=117/118 n=59 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:11bee109:::obj319:head@111'319, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.977+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:11426d25:::obj314:head local-lis/les=117/118 n=59 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:11bee109:::obj319:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.977+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:11426d25:::obj314:head local-lis/les=117/118 n=59 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:11bee109:::obj319:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.977+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:11426d25:::obj314:head local-lis/les=117/118 n=59 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:11bee109:::obj319:head 2026-02-11T23:31:27.977+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:11426d25:::obj314:head local-lis/les=117/118 n=59 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:11bee109:::obj319:head v 111'319 2026-02-11T23:31:27.977+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:11426d25:::obj314:head local-lis/les=117/118 n=59 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.977+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.977+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:11bee109:::obj319:head, version: 111'319, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11bee109:::obj319:head@111'319, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000067 2026-02-11T23:31:27.977+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.977+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.977+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:11a85aa0:::obj43:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.977+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:11a85aa0:::obj43:head) v3 prio 15 cost 0 latency 0.000092 pg_backfill(progress 4.0 e 118/118 lb 4:11a85aa0:::obj43:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:11426d25:::obj314:head local-lis/les=117/118 n=59 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.977+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.977+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:11426d25:::obj314:head local-lis/les=117/118 n=59 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:11a85aa0:::obj43:head) v3 2026-02-11T23:31:27.977+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:11426d25:::obj314:head local-lis/les=117/118 n=59 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:11a85aa0:::obj43:head) 2026-02-11T23:31:27.977+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.977+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:11a85aa0:::obj43:head) v3 finish latency 0.000016 2026-02-11T23:31:27.977+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:11a85aa0:::obj43:head local-lis/les=117/118 n=60 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.977+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.977+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.977+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:11a85aa0:::obj43:head local-lis/les=117/118 n=60 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.977+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.977+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.977+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.977+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x5f1d200 con 0x507ac00 2026-02-11T23:31:27.977+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:11bee109:::obj319:head)]) -- 0x5f74c00 con 0x507ac00 2026-02-11T23:31:27.977+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.977+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 160 ==== MOSDPGPush(4.0 118/117 [PushOp(4:121cd428:::obj443:head, version: 111'443, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:121cd428:::obj443:head@111'443, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5617520 con 0x507ac00 2026-02-11T23:31:27.977+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:121cd428:::obj443:head, version: 111'443, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:121cd428:::obj443:head@111'443, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 118 2026-02-11T23:31:27.977+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:121cd428:::obj443:head, version: 111'443, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:121cd428:::obj443:head@111'443, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.977+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.977+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 161 ==== pg_backfill(progress 4.0 e 118/118 lb 4:11bee109:::obj319:head) ==== 1062+0+0 (unknown 0 0 0) 0x6178c00 con 0x507ac00 2026-02-11T23:31:27.977+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:121cd428:::obj443:head, version: 111'443, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:121cd428:::obj443:head@111'443, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.977+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:11bee109:::obj319:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 118 2026-02-11T23:31:27.977+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:11bee109:::obj319:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.977+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.977+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:121cd428:::obj443:head, version: 111'443, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:121cd428:::obj443:head@111'443, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.977+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.977+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:11bee109:::obj319:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.977+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:121cd428:::obj443:head, version: 111'443, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:121cd428:::obj443:head@111'443, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000041 MOSDPGPush(4.0 118/117 [PushOp(4:121cd428:::obj443:head, version: 111'443, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:121cd428:::obj443:head@111'443, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:11a85aa0:::obj43:head local-lis/les=117/118 n=60 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.977+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.977+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:11a85aa0:::obj43:head local-lis/les=117/118 n=60 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:121cd428:::obj443:head, version: 111'443, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:121cd428:::obj443:head@111'443, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.977+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:11a85aa0:::obj43:head local-lis/les=117/118 n=60 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:121cd428:::obj443:head@111'443, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.977+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:11a85aa0:::obj43:head local-lis/les=117/118 n=60 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:121cd428:::obj443:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.977+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:11a85aa0:::obj43:head local-lis/les=117/118 n=60 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:121cd428:::obj443:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.977+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:11a85aa0:::obj43:head local-lis/les=117/118 n=60 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:121cd428:::obj443:head 2026-02-11T23:31:27.977+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:11a85aa0:::obj43:head local-lis/les=117/118 n=60 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:121cd428:::obj443:head v 111'443 2026-02-11T23:31:27.977+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:11a85aa0:::obj43:head local-lis/les=117/118 n=60 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.977+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.977+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:121cd428:::obj443:head, version: 111'443, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:121cd428:::obj443:head@111'443, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000077 2026-02-11T23:31:27.977+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.977+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.977+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:11bee109:::obj319:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.977+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:11bee109:::obj319:head) v3 prio 15 cost 0 latency 0.000101 pg_backfill(progress 4.0 e 118/118 lb 4:11bee109:::obj319:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:11a85aa0:::obj43:head local-lis/les=117/118 n=60 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.977+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.977+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:11a85aa0:::obj43:head local-lis/les=117/118 n=60 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:11bee109:::obj319:head) v3 2026-02-11T23:31:27.977+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:11a85aa0:::obj43:head local-lis/les=117/118 n=60 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:11bee109:::obj319:head) 2026-02-11T23:31:27.977+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.977+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:11bee109:::obj319:head) v3 finish latency 0.000016 2026-02-11T23:31:27.977+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:11bee109:::obj319:head local-lis/les=117/118 n=61 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.977+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.977+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.977+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:11bee109:::obj319:head local-lis/les=117/118 n=61 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.977+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.977+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.977+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.977+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63b8d80 con 0x507ac00 2026-02-11T23:31:27.977+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:121cd428:::obj443:head)]) -- 0x5f74300 con 0x507ac00 2026-02-11T23:31:27.977+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.978+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 162 ==== MOSDPGPush(4.0 118/117 [PushOp(4:1243fe1a:::obj21:head, version: 111'21, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1243fe1a:::obj21:head@111'21, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x56171e0 con 0x507ac00 2026-02-11T23:31:27.978+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:1243fe1a:::obj21:head, version: 111'21, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1243fe1a:::obj21:head@111'21, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 118 2026-02-11T23:31:27.978+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1243fe1a:::obj21:head, version: 111'21, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1243fe1a:::obj21:head@111'21, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.978+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.978+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 163 ==== pg_backfill(progress 4.0 e 118/118 lb 4:121cd428:::obj443:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9b180 con 0x507ac00 2026-02-11T23:31:27.978+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:121cd428:::obj443:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 118 2026-02-11T23:31:27.978+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1243fe1a:::obj21:head, version: 111'21, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1243fe1a:::obj21:head@111'21, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.978+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:121cd428:::obj443:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.978+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.978+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1243fe1a:::obj21:head, version: 111'21, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1243fe1a:::obj21:head@111'21, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.978+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.978+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:121cd428:::obj443:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.978+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:1243fe1a:::obj21:head, version: 111'21, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1243fe1a:::obj21:head@111'21, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000044 MOSDPGPush(4.0 118/117 [PushOp(4:1243fe1a:::obj21:head, version: 111'21, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1243fe1a:::obj21:head@111'21, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:11bee109:::obj319:head local-lis/les=117/118 n=61 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.978+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.978+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:11bee109:::obj319:head local-lis/les=117/118 n=61 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:1243fe1a:::obj21:head, version: 111'21, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1243fe1a:::obj21:head@111'21, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.978+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:11bee109:::obj319:head local-lis/les=117/118 n=61 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:1243fe1a:::obj21:head@111'21, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.978+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:11bee109:::obj319:head local-lis/les=117/118 n=61 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:1243fe1a:::obj21:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.978+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:11bee109:::obj319:head local-lis/les=117/118 n=61 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:1243fe1a:::obj21:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.978+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:11bee109:::obj319:head local-lis/les=117/118 n=61 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:1243fe1a:::obj21:head 2026-02-11T23:31:27.978+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:11bee109:::obj319:head local-lis/les=117/118 n=61 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:1243fe1a:::obj21:head v 111'21 2026-02-11T23:31:27.978+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:11bee109:::obj319:head local-lis/les=117/118 n=61 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.978+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.978+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:1243fe1a:::obj21:head, version: 111'21, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1243fe1a:::obj21:head@111'21, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000069 2026-02-11T23:31:27.978+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.978+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.978+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:121cd428:::obj443:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.978+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:121cd428:::obj443:head) v3 prio 15 cost 0 latency 0.000095 pg_backfill(progress 4.0 e 118/118 lb 4:121cd428:::obj443:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:11bee109:::obj319:head local-lis/les=117/118 n=61 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.978+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.978+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:11bee109:::obj319:head local-lis/les=117/118 n=61 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:121cd428:::obj443:head) v3 2026-02-11T23:31:27.978+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:11bee109:::obj319:head local-lis/les=117/118 n=61 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:121cd428:::obj443:head) 2026-02-11T23:31:27.978+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.978+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:121cd428:::obj443:head) v3 finish latency 0.000015 2026-02-11T23:31:27.978+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:121cd428:::obj443:head local-lis/les=117/118 n=62 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.978+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.978+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.978+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:121cd428:::obj443:head local-lis/les=117/118 n=62 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.978+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.978+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.978+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.978+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x5f75500 con 0x507ac00 2026-02-11T23:31:27.978+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:1243fe1a:::obj21:head)]) -- 0x5f74d80 con 0x507ac00 2026-02-11T23:31:27.978+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.979+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 164 ==== MOSDPGPush(4.0 118/117 [PushOp(4:124a99b2:::obj215:head, version: 111'215, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:124a99b2:::obj215:head@111'215, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56cad00 con 0x507ac00 2026-02-11T23:31:27.979+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:124a99b2:::obj215:head, version: 111'215, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:124a99b2:::obj215:head@111'215, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 118 2026-02-11T23:31:27.979+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:124a99b2:::obj215:head, version: 111'215, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:124a99b2:::obj215:head@111'215, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.979+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.979+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 165 ==== pg_backfill(progress 4.0 e 118/118 lb 4:1243fe1a:::obj21:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d98000 con 0x507ac00 2026-02-11T23:31:27.979+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:124a99b2:::obj215:head, version: 111'215, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:124a99b2:::obj215:head@111'215, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.979+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:1243fe1a:::obj21:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 118 2026-02-11T23:31:27.979+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1243fe1a:::obj21:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.979+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.979+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:124a99b2:::obj215:head, version: 111'215, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:124a99b2:::obj215:head@111'215, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.979+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.979+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1243fe1a:::obj21:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.979+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:124a99b2:::obj215:head, version: 111'215, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:124a99b2:::obj215:head@111'215, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000040 MOSDPGPush(4.0 118/117 [PushOp(4:124a99b2:::obj215:head, version: 111'215, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:124a99b2:::obj215:head@111'215, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:121cd428:::obj443:head local-lis/les=117/118 n=62 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.979+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.979+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:121cd428:::obj443:head local-lis/les=117/118 n=62 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:124a99b2:::obj215:head, version: 111'215, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:124a99b2:::obj215:head@111'215, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.979+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:121cd428:::obj443:head local-lis/les=117/118 n=62 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:124a99b2:::obj215:head@111'215, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.979+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:121cd428:::obj443:head local-lis/les=117/118 n=62 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:124a99b2:::obj215:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.979+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:121cd428:::obj443:head local-lis/les=117/118 n=62 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:124a99b2:::obj215:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.979+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:121cd428:::obj443:head local-lis/les=117/118 n=62 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:124a99b2:::obj215:head 2026-02-11T23:31:27.979+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:121cd428:::obj443:head local-lis/les=117/118 n=62 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:124a99b2:::obj215:head v 111'215 2026-02-11T23:31:27.979+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:121cd428:::obj443:head local-lis/les=117/118 n=62 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.979+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.979+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:124a99b2:::obj215:head, version: 111'215, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:124a99b2:::obj215:head@111'215, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000076 2026-02-11T23:31:27.979+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.979+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.979+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1243fe1a:::obj21:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.979+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:1243fe1a:::obj21:head) v3 prio 15 cost 0 latency 0.000100 pg_backfill(progress 4.0 e 118/118 lb 4:1243fe1a:::obj21:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:121cd428:::obj443:head local-lis/les=117/118 n=62 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.979+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.979+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:121cd428:::obj443:head local-lis/les=117/118 n=62 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:1243fe1a:::obj21:head) v3 2026-02-11T23:31:27.979+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:121cd428:::obj443:head local-lis/les=117/118 n=62 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:1243fe1a:::obj21:head) 2026-02-11T23:31:27.979+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.979+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:1243fe1a:::obj21:head) v3 finish latency 0.000016 2026-02-11T23:31:27.979+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1243fe1a:::obj21:head local-lis/les=117/118 n=63 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.979+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.979+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.979+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1243fe1a:::obj21:head local-lis/les=117/118 n=63 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.979+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.979+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.979+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.979+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63b8c00 con 0x507ac00 2026-02-11T23:31:27.979+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:124a99b2:::obj215:head)]) -- 0x6446000 con 0x507ac00 2026-02-11T23:31:27.979+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.979+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 166 ==== MOSDPGPush(4.0 118/117 [PushOp(4:1405054b:::obj129:head, version: 111'129, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1405054b:::obj129:head@111'129, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5646d00 con 0x507ac00 2026-02-11T23:31:27.979+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:1405054b:::obj129:head, version: 111'129, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1405054b:::obj129:head@111'129, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 118 2026-02-11T23:31:27.979+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1405054b:::obj129:head, version: 111'129, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1405054b:::obj129:head@111'129, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.979+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.979+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 167 ==== pg_backfill(progress 4.0 e 118/118 lb 4:124a99b2:::obj215:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98580 con 0x507ac00 2026-02-11T23:31:27.979+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1405054b:::obj129:head, version: 111'129, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1405054b:::obj129:head@111'129, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.979+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:124a99b2:::obj215:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 118 2026-02-11T23:31:27.979+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:124a99b2:::obj215:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.979+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.979+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1405054b:::obj129:head, version: 111'129, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1405054b:::obj129:head@111'129, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.979+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.979+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:124a99b2:::obj215:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.979+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:1405054b:::obj129:head, version: 111'129, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1405054b:::obj129:head@111'129, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000042 MOSDPGPush(4.0 118/117 [PushOp(4:1405054b:::obj129:head, version: 111'129, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1405054b:::obj129:head@111'129, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:1243fe1a:::obj21:head local-lis/les=117/118 n=63 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.979+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.979+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1243fe1a:::obj21:head local-lis/les=117/118 n=63 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:1405054b:::obj129:head, version: 111'129, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1405054b:::obj129:head@111'129, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.979+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1243fe1a:::obj21:head local-lis/les=117/118 n=63 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:1405054b:::obj129:head@111'129, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.979+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1243fe1a:::obj21:head local-lis/les=117/118 n=63 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:1405054b:::obj129:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.979+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1243fe1a:::obj21:head local-lis/les=117/118 n=63 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:1405054b:::obj129:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.979+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1243fe1a:::obj21:head local-lis/les=117/118 n=63 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:1405054b:::obj129:head 2026-02-11T23:31:27.979+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1243fe1a:::obj21:head local-lis/les=117/118 n=63 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:1405054b:::obj129:head v 111'129 2026-02-11T23:31:27.979+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1243fe1a:::obj21:head local-lis/les=117/118 n=63 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.979+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.979+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:1405054b:::obj129:head, version: 111'129, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1405054b:::obj129:head@111'129, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000073 2026-02-11T23:31:27.979+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.979+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.979+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:124a99b2:::obj215:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.979+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:124a99b2:::obj215:head) v3 prio 15 cost 0 latency 0.000099 pg_backfill(progress 4.0 e 118/118 lb 4:124a99b2:::obj215:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:1243fe1a:::obj21:head local-lis/les=117/118 n=63 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.979+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.979+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1243fe1a:::obj21:head local-lis/les=117/118 n=63 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:124a99b2:::obj215:head) v3 2026-02-11T23:31:27.979+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1243fe1a:::obj21:head local-lis/les=117/118 n=63 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:124a99b2:::obj215:head) 2026-02-11T23:31:27.979+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.979+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:124a99b2:::obj215:head) v3 finish latency 0.000013 2026-02-11T23:31:27.979+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:124a99b2:::obj215:head local-lis/les=117/118 n=64 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.979+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.980+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:124a99b2:::obj215:head local-lis/les=117/118 n=64 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.980+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.980+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.980+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.980+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.980+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x649e180 con 0x507ac00 2026-02-11T23:31:27.980+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:1405054b:::obj129:head)]) -- 0x6446300 con 0x507ac00 2026-02-11T23:31:27.980+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.980+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 168 ==== MOSDPGPush(4.0 118/117 [PushOp(4:147354e0:::obj436:head, version: 111'436, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:147354e0:::obj436:head@111'436, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3815a00 con 0x507ac00 2026-02-11T23:31:27.980+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:147354e0:::obj436:head, version: 111'436, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:147354e0:::obj436:head@111'436, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 118 2026-02-11T23:31:27.980+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:147354e0:::obj436:head, version: 111'436, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:147354e0:::obj436:head@111'436, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.980+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.980+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 169 ==== pg_backfill(progress 4.0 e 118/118 lb 4:1405054b:::obj129:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f84000 con 0x507ac00 2026-02-11T23:31:27.980+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:147354e0:::obj436:head, version: 111'436, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:147354e0:::obj436:head@111'436, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.980+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:1405054b:::obj129:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 118 2026-02-11T23:31:27.980+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1405054b:::obj129:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.980+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.980+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:147354e0:::obj436:head, version: 111'436, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:147354e0:::obj436:head@111'436, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.980+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.980+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:147354e0:::obj436:head, version: 111'436, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:147354e0:::obj436:head@111'436, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000044 MOSDPGPush(4.0 118/117 [PushOp(4:147354e0:::obj436:head, version: 111'436, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:147354e0:::obj436:head@111'436, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:124a99b2:::obj215:head local-lis/les=117/118 n=64 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.980+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.980+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.980+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:124a99b2:::obj215:head local-lis/les=117/118 n=64 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:147354e0:::obj436:head, version: 111'436, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:147354e0:::obj436:head@111'436, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.980+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1405054b:::obj129:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.980+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:124a99b2:::obj215:head local-lis/les=117/118 n=64 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:147354e0:::obj436:head@111'436, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.980+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:124a99b2:::obj215:head local-lis/les=117/118 n=64 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:147354e0:::obj436:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.980+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:124a99b2:::obj215:head local-lis/les=117/118 n=64 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:147354e0:::obj436:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.980+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:124a99b2:::obj215:head local-lis/les=117/118 n=64 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:147354e0:::obj436:head 2026-02-11T23:31:27.980+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:124a99b2:::obj215:head local-lis/les=117/118 n=64 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:147354e0:::obj436:head v 111'436 2026-02-11T23:31:27.980+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:124a99b2:::obj215:head local-lis/les=117/118 n=64 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.980+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.980+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:147354e0:::obj436:head, version: 111'436, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:147354e0:::obj436:head@111'436, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000074 2026-02-11T23:31:27.980+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.980+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.980+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1405054b:::obj129:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.980+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:1405054b:::obj129:head) v3 prio 15 cost 0 latency 0.000098 pg_backfill(progress 4.0 e 118/118 lb 4:1405054b:::obj129:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:124a99b2:::obj215:head local-lis/les=117/118 n=64 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.980+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.980+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:124a99b2:::obj215:head local-lis/les=117/118 n=64 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:1405054b:::obj129:head) v3 2026-02-11T23:31:27.980+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:124a99b2:::obj215:head local-lis/les=117/118 n=64 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:1405054b:::obj129:head) 2026-02-11T23:31:27.980+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.980+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:1405054b:::obj129:head) v3 finish latency 0.000015 2026-02-11T23:31:27.980+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1405054b:::obj129:head local-lis/les=117/118 n=65 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.980+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.980+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1405054b:::obj129:head local-lis/les=117/118 n=65 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.980+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.980+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.980+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.980+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.980+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63b8a80 con 0x507ac00 2026-02-11T23:31:27.980+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:147354e0:::obj436:head)]) -- 0x6446600 con 0x507ac00 2026-02-11T23:31:27.980+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.981+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 170 ==== MOSDPGPush(4.0 118/117 [PushOp(4:147b5010:::obj178:head, version: 111'178, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:147b5010:::obj178:head@111'178, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3815520 con 0x507ac00 2026-02-11T23:31:27.981+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:147b5010:::obj178:head, version: 111'178, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:147b5010:::obj178:head@111'178, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 118 2026-02-11T23:31:27.981+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:147b5010:::obj178:head, version: 111'178, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:147b5010:::obj178:head@111'178, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.981+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.981+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 171 ==== pg_backfill(progress 4.0 e 118/118 lb 4:147354e0:::obj436:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99080 con 0x507ac00 2026-02-11T23:31:27.981+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:147b5010:::obj178:head, version: 111'178, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:147b5010:::obj178:head@111'178, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.981+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:147354e0:::obj436:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 118 2026-02-11T23:31:27.981+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:147354e0:::obj436:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.981+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.981+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:147b5010:::obj178:head, version: 111'178, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:147b5010:::obj178:head@111'178, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.981+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.981+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:147b5010:::obj178:head, version: 111'178, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:147b5010:::obj178:head@111'178, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000046 MOSDPGPush(4.0 118/117 [PushOp(4:147b5010:::obj178:head, version: 111'178, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:147b5010:::obj178:head@111'178, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:1405054b:::obj129:head local-lis/les=117/118 n=65 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.981+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.981+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.981+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:147354e0:::obj436:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.981+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1405054b:::obj129:head local-lis/les=117/118 n=65 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:147b5010:::obj178:head, version: 111'178, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:147b5010:::obj178:head@111'178, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.981+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1405054b:::obj129:head local-lis/les=117/118 n=65 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:147b5010:::obj178:head@111'178, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.981+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1405054b:::obj129:head local-lis/les=117/118 n=65 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:147b5010:::obj178:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.981+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1405054b:::obj129:head local-lis/les=117/118 n=65 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:147b5010:::obj178:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.981+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1405054b:::obj129:head local-lis/les=117/118 n=65 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:147b5010:::obj178:head 2026-02-11T23:31:27.981+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1405054b:::obj129:head local-lis/les=117/118 n=65 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:147b5010:::obj178:head v 111'178 2026-02-11T23:31:27.981+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1405054b:::obj129:head local-lis/les=117/118 n=65 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.981+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.981+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:147b5010:::obj178:head, version: 111'178, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:147b5010:::obj178:head@111'178, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000076 2026-02-11T23:31:27.981+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.981+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.981+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:147354e0:::obj436:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.981+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:147354e0:::obj436:head) v3 prio 15 cost 0 latency 0.000103 pg_backfill(progress 4.0 e 118/118 lb 4:147354e0:::obj436:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:1405054b:::obj129:head local-lis/les=117/118 n=65 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.981+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.981+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1405054b:::obj129:head local-lis/les=117/118 n=65 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:147354e0:::obj436:head) v3 2026-02-11T23:31:27.981+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1405054b:::obj129:head local-lis/les=117/118 n=65 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:147354e0:::obj436:head) 2026-02-11T23:31:27.981+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.981+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:147354e0:::obj436:head) v3 finish latency 0.000014 2026-02-11T23:31:27.981+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:147354e0:::obj436:head local-lis/les=117/118 n=66 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.981+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.981+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.981+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:147354e0:::obj436:head local-lis/les=117/118 n=66 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.981+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.981+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.981+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.981+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63b8600 con 0x507ac00 2026-02-11T23:31:27.981+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:147b5010:::obj178:head)]) -- 0x6446900 con 0x507ac00 2026-02-11T23:31:27.981+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.982+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 172 ==== MOSDPGPush(4.0 118/117 [PushOp(4:1503467e:::obj118:head, version: 111'118, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1503467e:::obj118:head@111'118, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3814ea0 con 0x507ac00 2026-02-11T23:31:27.982+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:1503467e:::obj118:head, version: 111'118, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1503467e:::obj118:head@111'118, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000016 epoch 118 2026-02-11T23:31:27.982+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1503467e:::obj118:head, version: 111'118, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1503467e:::obj118:head@111'118, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.982+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.982+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 173 ==== pg_backfill(progress 4.0 e 118/118 lb 4:147b5010:::obj178:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99600 con 0x507ac00 2026-02-11T23:31:27.982+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:147b5010:::obj178:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 118 2026-02-11T23:31:27.982+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1503467e:::obj118:head, version: 111'118, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1503467e:::obj118:head@111'118, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.982+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:147b5010:::obj178:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.982+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.982+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1503467e:::obj118:head, version: 111'118, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1503467e:::obj118:head@111'118, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.982+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:1503467e:::obj118:head, version: 111'118, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1503467e:::obj118:head@111'118, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000046 MOSDPGPush(4.0 118/117 [PushOp(4:1503467e:::obj118:head, version: 111'118, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1503467e:::obj118:head@111'118, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:147354e0:::obj436:head local-lis/les=117/118 n=66 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.982+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.982+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.982+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:147b5010:::obj178:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.982+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:147354e0:::obj436:head local-lis/les=117/118 n=66 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:1503467e:::obj118:head, version: 111'118, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1503467e:::obj118:head@111'118, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.982+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:147354e0:::obj436:head local-lis/les=117/118 n=66 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:1503467e:::obj118:head@111'118, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.982+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:147354e0:::obj436:head local-lis/les=117/118 n=66 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:1503467e:::obj118:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.982+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:147354e0:::obj436:head local-lis/les=117/118 n=66 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:1503467e:::obj118:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.982+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:147354e0:::obj436:head local-lis/les=117/118 n=66 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:1503467e:::obj118:head 2026-02-11T23:31:27.982+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:147354e0:::obj436:head local-lis/les=117/118 n=66 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:1503467e:::obj118:head v 111'118 2026-02-11T23:31:27.982+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:147354e0:::obj436:head local-lis/les=117/118 n=66 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.982+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.982+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:1503467e:::obj118:head, version: 111'118, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1503467e:::obj118:head@111'118, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000068 2026-02-11T23:31:27.982+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.982+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.982+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:147b5010:::obj178:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.982+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:147b5010:::obj178:head) v3 prio 15 cost 0 latency 0.000090 pg_backfill(progress 4.0 e 118/118 lb 4:147b5010:::obj178:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:147354e0:::obj436:head local-lis/les=117/118 n=66 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.982+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.982+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:147354e0:::obj436:head local-lis/les=117/118 n=66 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:147b5010:::obj178:head) v3 2026-02-11T23:31:27.982+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:147354e0:::obj436:head local-lis/les=117/118 n=66 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:147b5010:::obj178:head) 2026-02-11T23:31:27.982+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.982+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:147b5010:::obj178:head) v3 finish latency 0.000016 2026-02-11T23:31:27.982+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:147b5010:::obj178:head local-lis/les=117/118 n=67 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.982+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.982+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.982+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:147b5010:::obj178:head local-lis/les=117/118 n=67 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.982+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.982+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.982+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.982+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63b8900 con 0x507ac00 2026-02-11T23:31:27.982+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:1503467e:::obj118:head)]) -- 0x6446c00 con 0x507ac00 2026-02-11T23:31:27.982+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.982+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 174 ==== MOSDPGPush(4.0 118/117 [PushOp(4:1547e7aa:::obj466:head, version: 111'466, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1547e7aa:::obj466:head@111'466, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c9c0 con 0x507ac00 2026-02-11T23:31:27.982+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:1547e7aa:::obj466:head, version: 111'466, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1547e7aa:::obj466:head@111'466, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 118 2026-02-11T23:31:27.982+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1547e7aa:::obj466:head, version: 111'466, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1547e7aa:::obj466:head@111'466, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.982+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.982+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 175 ==== pg_backfill(progress 4.0 e 118/118 lb 4:1503467e:::obj118:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9ac00 con 0x507ac00 2026-02-11T23:31:27.982+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:1503467e:::obj118:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 118 2026-02-11T23:31:27.982+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1547e7aa:::obj466:head, version: 111'466, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1547e7aa:::obj466:head@111'466, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.982+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1503467e:::obj118:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.982+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.982+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1547e7aa:::obj466:head, version: 111'466, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1547e7aa:::obj466:head@111'466, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.982+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.982+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:1547e7aa:::obj466:head, version: 111'466, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1547e7aa:::obj466:head@111'466, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000041 MOSDPGPush(4.0 118/117 [PushOp(4:1547e7aa:::obj466:head, version: 111'466, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1547e7aa:::obj466:head@111'466, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:147b5010:::obj178:head local-lis/les=117/118 n=67 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.982+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1503467e:::obj118:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.982+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.982+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:147b5010:::obj178:head local-lis/les=117/118 n=67 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:1547e7aa:::obj466:head, version: 111'466, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1547e7aa:::obj466:head@111'466, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.982+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:147b5010:::obj178:head local-lis/les=117/118 n=67 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:1547e7aa:::obj466:head@111'466, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.982+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:147b5010:::obj178:head local-lis/les=117/118 n=67 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:1547e7aa:::obj466:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.982+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:147b5010:::obj178:head local-lis/les=117/118 n=67 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:1547e7aa:::obj466:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.982+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:147b5010:::obj178:head local-lis/les=117/118 n=67 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:1547e7aa:::obj466:head 2026-02-11T23:31:27.982+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:147b5010:::obj178:head local-lis/les=117/118 n=67 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:1547e7aa:::obj466:head v 111'466 2026-02-11T23:31:27.982+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:147b5010:::obj178:head local-lis/les=117/118 n=67 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.982+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.982+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:1547e7aa:::obj466:head, version: 111'466, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1547e7aa:::obj466:head@111'466, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:27.982+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.982+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.982+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1503467e:::obj118:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.982+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:1503467e:::obj118:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 4.0 e 118/118 lb 4:1503467e:::obj118:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:147b5010:::obj178:head local-lis/les=117/118 n=67 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.982+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.982+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:147b5010:::obj178:head local-lis/les=117/118 n=67 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:1503467e:::obj118:head) v3 2026-02-11T23:31:27.982+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:147b5010:::obj178:head local-lis/les=117/118 n=67 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:1503467e:::obj118:head) 2026-02-11T23:31:27.983+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.983+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:1503467e:::obj118:head) v3 finish latency 0.000027 2026-02-11T23:31:27.983+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1503467e:::obj118:head local-lis/les=117/118 n=68 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.983+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.983+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1503467e:::obj118:head local-lis/les=117/118 n=68 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.983+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.983+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.983+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.983+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.983+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63b8180 con 0x507ac00 2026-02-11T23:31:27.983+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:1547e7aa:::obj466:head)]) -- 0x6446f00 con 0x507ac00 2026-02-11T23:31:27.983+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.983+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 176 ==== MOSDPGPush(4.0 118/117 [PushOp(4:170a838d:::obj199:head, version: 111'199, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:170a838d:::obj199:head@111'199, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x38156c0 con 0x507ac00 2026-02-11T23:31:27.983+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:170a838d:::obj199:head, version: 111'199, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:170a838d:::obj199:head@111'199, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 118 2026-02-11T23:31:27.983+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:170a838d:::obj199:head, version: 111'199, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:170a838d:::obj199:head@111'199, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.983+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.983+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 177 ==== pg_backfill(progress 4.0 e 118/118 lb 4:1547e7aa:::obj466:head) ==== 1062+0+0 (unknown 0 0 0) 0x4a2cb00 con 0x507ac00 2026-02-11T23:31:27.983+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:1547e7aa:::obj466:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:27.983+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:170a838d:::obj199:head, version: 111'199, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:170a838d:::obj199:head@111'199, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.983+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1547e7aa:::obj466:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.983+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.983+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:170a838d:::obj199:head, version: 111'199, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:170a838d:::obj199:head@111'199, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.983+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:170a838d:::obj199:head, version: 111'199, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:170a838d:::obj199:head@111'199, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(4.0 118/117 [PushOp(4:170a838d:::obj199:head, version: 111'199, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:170a838d:::obj199:head@111'199, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:1503467e:::obj118:head local-lis/les=117/118 n=68 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.983+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.983+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.983+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1547e7aa:::obj466:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.983+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1503467e:::obj118:head local-lis/les=117/118 n=68 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:170a838d:::obj199:head, version: 111'199, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:170a838d:::obj199:head@111'199, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.983+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1503467e:::obj118:head local-lis/les=117/118 n=68 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:170a838d:::obj199:head@111'199, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.983+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1503467e:::obj118:head local-lis/les=117/118 n=68 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:170a838d:::obj199:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.983+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1503467e:::obj118:head local-lis/les=117/118 n=68 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:170a838d:::obj199:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.983+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1503467e:::obj118:head local-lis/les=117/118 n=68 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:170a838d:::obj199:head 2026-02-11T23:31:27.983+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1503467e:::obj118:head local-lis/les=117/118 n=68 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:170a838d:::obj199:head v 111'199 2026-02-11T23:31:27.983+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1503467e:::obj118:head local-lis/les=117/118 n=68 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.983+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.983+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:170a838d:::obj199:head, version: 111'199, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:170a838d:::obj199:head@111'199, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:27.983+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.983+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.983+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1547e7aa:::obj466:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.983+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:1547e7aa:::obj466:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 4.0 e 118/118 lb 4:1547e7aa:::obj466:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:1503467e:::obj118:head local-lis/les=117/118 n=68 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.983+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.983+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1503467e:::obj118:head local-lis/les=117/118 n=68 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:1547e7aa:::obj466:head) v3 2026-02-11T23:31:27.983+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1503467e:::obj118:head local-lis/les=117/118 n=68 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:1547e7aa:::obj466:head) 2026-02-11T23:31:27.983+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.983+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:1547e7aa:::obj466:head) v3 finish latency 0.000013 2026-02-11T23:31:27.983+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1547e7aa:::obj466:head local-lis/les=117/118 n=69 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.983+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.983+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.983+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1547e7aa:::obj466:head local-lis/les=117/118 n=69 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.983+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.983+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.983+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.983+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63b8480 con 0x507ac00 2026-02-11T23:31:27.983+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:170a838d:::obj199:head)]) -- 0x6447200 con 0x507ac00 2026-02-11T23:31:27.983+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.984+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 178 ==== MOSDPGPush(4.0 118/117 [PushOp(4:171b715e:::obj570:head, version: 111'570, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:171b715e:::obj570:head@111'570, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f1520 con 0x507ac00 2026-02-11T23:31:27.984+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:171b715e:::obj570:head, version: 111'570, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:171b715e:::obj570:head@111'570, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 118 2026-02-11T23:31:27.984+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:171b715e:::obj570:head, version: 111'570, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:171b715e:::obj570:head@111'570, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.984+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.984+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 179 ==== pg_backfill(progress 4.0 e 118/118 lb 4:170a838d:::obj199:head) ==== 1062+0+0 (unknown 0 0 0) 0x4a2c000 con 0x507ac00 2026-02-11T23:31:27.984+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:171b715e:::obj570:head, version: 111'570, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:171b715e:::obj570:head@111'570, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.984+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:170a838d:::obj199:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:27.984+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:170a838d:::obj199:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.984+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.984+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:171b715e:::obj570:head, version: 111'570, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:171b715e:::obj570:head@111'570, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.984+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:171b715e:::obj570:head, version: 111'570, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:171b715e:::obj570:head@111'570, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(4.0 118/117 [PushOp(4:171b715e:::obj570:head, version: 111'570, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:171b715e:::obj570:head@111'570, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:1547e7aa:::obj466:head local-lis/les=117/118 n=69 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.984+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.984+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.984+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:170a838d:::obj199:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.984+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1547e7aa:::obj466:head local-lis/les=117/118 n=69 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:171b715e:::obj570:head, version: 111'570, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:171b715e:::obj570:head@111'570, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.984+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1547e7aa:::obj466:head local-lis/les=117/118 n=69 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:171b715e:::obj570:head@111'570, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.984+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1547e7aa:::obj466:head local-lis/les=117/118 n=69 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:171b715e:::obj570:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.984+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1547e7aa:::obj466:head local-lis/les=117/118 n=69 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:171b715e:::obj570:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.984+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1547e7aa:::obj466:head local-lis/les=117/118 n=69 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:171b715e:::obj570:head 2026-02-11T23:31:27.984+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1547e7aa:::obj466:head local-lis/les=117/118 n=69 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:171b715e:::obj570:head v 111'570 2026-02-11T23:31:27.984+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1547e7aa:::obj466:head local-lis/les=117/118 n=69 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.984+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.984+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:171b715e:::obj570:head, version: 111'570, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:171b715e:::obj570:head@111'570, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:27.984+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.984+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.984+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:170a838d:::obj199:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.984+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:170a838d:::obj199:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 4.0 e 118/118 lb 4:170a838d:::obj199:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:1547e7aa:::obj466:head local-lis/les=117/118 n=69 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.984+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.984+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1547e7aa:::obj466:head local-lis/les=117/118 n=69 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:170a838d:::obj199:head) v3 2026-02-11T23:31:27.984+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1547e7aa:::obj466:head local-lis/les=117/118 n=69 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:170a838d:::obj199:head) 2026-02-11T23:31:27.984+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.984+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:170a838d:::obj199:head) v3 finish latency 0.000015 2026-02-11T23:31:27.984+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:170a838d:::obj199:head local-lis/les=117/118 n=70 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.984+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.984+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.984+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:170a838d:::obj199:head local-lis/les=117/118 n=70 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.984+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.984+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.984+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.984+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x649e480 con 0x507ac00 2026-02-11T23:31:27.984+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:171b715e:::obj570:head)]) -- 0x6447500 con 0x507ac00 2026-02-11T23:31:27.984+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.985+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 180 ==== MOSDPGPush(4.0 118/117 [PushOp(4:17371e94:::obj362:head, version: 111'362, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:17371e94:::obj362:head@111'362, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2d00 con 0x507ac00 2026-02-11T23:31:27.985+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:17371e94:::obj362:head, version: 111'362, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:17371e94:::obj362:head@111'362, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000016 epoch 118 2026-02-11T23:31:27.985+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:17371e94:::obj362:head, version: 111'362, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:17371e94:::obj362:head@111'362, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.985+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.985+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 181 ==== pg_backfill(progress 4.0 e 118/118 lb 4:171b715e:::obj570:head) ==== 1062+0+0 (unknown 0 0 0) 0x5982b00 con 0x507ac00 2026-02-11T23:31:27.985+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:17371e94:::obj362:head, version: 111'362, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:17371e94:::obj362:head@111'362, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.985+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:171b715e:::obj570:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:27.985+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:171b715e:::obj570:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.985+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.985+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:17371e94:::obj362:head, version: 111'362, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:17371e94:::obj362:head@111'362, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.985+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:17371e94:::obj362:head, version: 111'362, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:17371e94:::obj362:head@111'362, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(4.0 118/117 [PushOp(4:17371e94:::obj362:head, version: 111'362, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:17371e94:::obj362:head@111'362, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:170a838d:::obj199:head local-lis/les=117/118 n=70 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.985+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.985+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.985+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:171b715e:::obj570:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.985+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:170a838d:::obj199:head local-lis/les=117/118 n=70 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:17371e94:::obj362:head, version: 111'362, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:17371e94:::obj362:head@111'362, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.985+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:170a838d:::obj199:head local-lis/les=117/118 n=70 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:17371e94:::obj362:head@111'362, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.985+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:170a838d:::obj199:head local-lis/les=117/118 n=70 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:17371e94:::obj362:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.985+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:170a838d:::obj199:head local-lis/les=117/118 n=70 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:17371e94:::obj362:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.985+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:170a838d:::obj199:head local-lis/les=117/118 n=70 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:17371e94:::obj362:head 2026-02-11T23:31:27.985+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:170a838d:::obj199:head local-lis/les=117/118 n=70 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:17371e94:::obj362:head v 111'362 2026-02-11T23:31:27.985+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:170a838d:::obj199:head local-lis/les=117/118 n=70 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.985+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.985+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:17371e94:::obj362:head, version: 111'362, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:17371e94:::obj362:head@111'362, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:27.985+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.985+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.985+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:171b715e:::obj570:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.985+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:171b715e:::obj570:head) v3 prio 15 cost 0 latency 0.000080 pg_backfill(progress 4.0 e 118/118 lb 4:171b715e:::obj570:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:170a838d:::obj199:head local-lis/les=117/118 n=70 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.985+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.985+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:170a838d:::obj199:head local-lis/les=117/118 n=70 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:171b715e:::obj570:head) v3 2026-02-11T23:31:27.985+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:170a838d:::obj199:head local-lis/les=117/118 n=70 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:171b715e:::obj570:head) 2026-02-11T23:31:27.985+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.985+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:171b715e:::obj570:head) v3 finish latency 0.000016 2026-02-11T23:31:27.985+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:171b715e:::obj570:head local-lis/les=117/118 n=71 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.985+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.985+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.985+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:171b715e:::obj570:head local-lis/les=117/118 n=71 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.985+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.985+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.985+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.985+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63b8000 con 0x507ac00 2026-02-11T23:31:27.985+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:17371e94:::obj362:head)]) -- 0x6447800 con 0x507ac00 2026-02-11T23:31:27.985+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.985+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 182 ==== MOSDPGPush(4.0 118/117 [PushOp(4:18100969:::obj255:head, version: 111'255, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18100969:::obj255:head@111'255, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5646d00 con 0x507ac00 2026-02-11T23:31:27.985+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:18100969:::obj255:head, version: 111'255, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18100969:::obj255:head@111'255, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 118 2026-02-11T23:31:27.985+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:18100969:::obj255:head, version: 111'255, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18100969:::obj255:head@111'255, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.985+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.985+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 183 ==== pg_backfill(progress 4.0 e 118/118 lb 4:17371e94:::obj362:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99b80 con 0x507ac00 2026-02-11T23:31:27.985+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:17371e94:::obj362:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:27.985+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:18100969:::obj255:head, version: 111'255, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18100969:::obj255:head@111'255, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.985+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:17371e94:::obj362:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.985+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.985+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:18100969:::obj255:head, version: 111'255, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18100969:::obj255:head@111'255, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.985+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:18100969:::obj255:head, version: 111'255, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18100969:::obj255:head@111'255, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(4.0 118/117 [PushOp(4:18100969:::obj255:head, version: 111'255, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18100969:::obj255:head@111'255, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:171b715e:::obj570:head local-lis/les=117/118 n=71 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.985+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.985+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.985+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:17371e94:::obj362:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.985+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:171b715e:::obj570:head local-lis/les=117/118 n=71 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:18100969:::obj255:head, version: 111'255, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18100969:::obj255:head@111'255, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.985+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:171b715e:::obj570:head local-lis/les=117/118 n=71 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:18100969:::obj255:head@111'255, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.985+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:171b715e:::obj570:head local-lis/les=117/118 n=71 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:18100969:::obj255:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.985+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:171b715e:::obj570:head local-lis/les=117/118 n=71 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:18100969:::obj255:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.985+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:171b715e:::obj570:head local-lis/les=117/118 n=71 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:18100969:::obj255:head 2026-02-11T23:31:27.985+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:171b715e:::obj570:head local-lis/les=117/118 n=71 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:18100969:::obj255:head v 111'255 2026-02-11T23:31:27.985+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:171b715e:::obj570:head local-lis/les=117/118 n=71 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.985+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.985+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:18100969:::obj255:head, version: 111'255, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18100969:::obj255:head@111'255, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:27.985+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.985+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.985+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:17371e94:::obj362:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.985+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:17371e94:::obj362:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 4.0 e 118/118 lb 4:17371e94:::obj362:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:171b715e:::obj570:head local-lis/les=117/118 n=71 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.985+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.985+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:171b715e:::obj570:head local-lis/les=117/118 n=71 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:17371e94:::obj362:head) v3 2026-02-11T23:31:27.985+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:171b715e:::obj570:head local-lis/les=117/118 n=71 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:17371e94:::obj362:head) 2026-02-11T23:31:27.985+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.985+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:17371e94:::obj362:head) v3 finish latency 0.000014 2026-02-11T23:31:27.985+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:17371e94:::obj362:head local-lis/les=117/118 n=72 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.985+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.985+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.985+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:17371e94:::obj362:head local-lis/les=117/118 n=72 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.985+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.985+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.985+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.985+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x649e600 con 0x507ac00 2026-02-11T23:31:27.985+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:18100969:::obj255:head)]) -- 0x6447b00 con 0x507ac00 2026-02-11T23:31:27.985+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.986+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 184 ==== MOSDPGPush(4.0 118/117 [PushOp(4:1842fa44:::obj284:head, version: 111'284, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1842fa44:::obj284:head@111'284, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56cad00 con 0x507ac00 2026-02-11T23:31:27.986+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:1842fa44:::obj284:head, version: 111'284, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1842fa44:::obj284:head@111'284, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 118 2026-02-11T23:31:27.986+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1842fa44:::obj284:head, version: 111'284, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1842fa44:::obj284:head@111'284, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.986+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.986+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1842fa44:::obj284:head, version: 111'284, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1842fa44:::obj284:head@111'284, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.986+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 185 ==== pg_backfill(progress 4.0 e 118/118 lb 4:18100969:::obj255:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f84000 con 0x507ac00 2026-02-11T23:31:27.986+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:18100969:::obj255:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:27.986+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.986+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:18100969:::obj255:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.986+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1842fa44:::obj284:head, version: 111'284, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1842fa44:::obj284:head@111'284, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.986+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.986+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:1842fa44:::obj284:head, version: 111'284, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1842fa44:::obj284:head@111'284, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(4.0 118/117 [PushOp(4:1842fa44:::obj284:head, version: 111'284, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1842fa44:::obj284:head@111'284, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:17371e94:::obj362:head local-lis/les=117/118 n=72 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.986+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.986+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:17371e94:::obj362:head local-lis/les=117/118 n=72 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:1842fa44:::obj284:head, version: 111'284, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1842fa44:::obj284:head@111'284, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.986+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:17371e94:::obj362:head local-lis/les=117/118 n=72 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:1842fa44:::obj284:head@111'284, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.986+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.986+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:18100969:::obj255:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.986+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:17371e94:::obj362:head local-lis/les=117/118 n=72 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:1842fa44:::obj284:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.986+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:17371e94:::obj362:head local-lis/les=117/118 n=72 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:1842fa44:::obj284:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.986+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:17371e94:::obj362:head local-lis/les=117/118 n=72 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:1842fa44:::obj284:head 2026-02-11T23:31:27.986+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:17371e94:::obj362:head local-lis/les=117/118 n=72 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:1842fa44:::obj284:head v 111'284 2026-02-11T23:31:27.986+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:17371e94:::obj362:head local-lis/les=117/118 n=72 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.986+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.986+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:1842fa44:::obj284:head, version: 111'284, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1842fa44:::obj284:head@111'284, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:27.986+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.986+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.986+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:18100969:::obj255:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.986+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:18100969:::obj255:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 4.0 e 118/118 lb 4:18100969:::obj255:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:17371e94:::obj362:head local-lis/les=117/118 n=72 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.986+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.986+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:17371e94:::obj362:head local-lis/les=117/118 n=72 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:18100969:::obj255:head) v3 2026-02-11T23:31:27.986+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:17371e94:::obj362:head local-lis/les=117/118 n=72 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:18100969:::obj255:head) 2026-02-11T23:31:27.986+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.986+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:18100969:::obj255:head) v3 finish latency 0.000014 2026-02-11T23:31:27.986+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:18100969:::obj255:head local-lis/les=117/118 n=73 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.986+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.986+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.986+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:18100969:::obj255:head local-lis/les=117/118 n=73 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.986+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.986+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.986+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.986+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x649fe00 con 0x507ac00 2026-02-11T23:31:27.986+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.986+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:1842fa44:::obj284:head)]) -- 0x6447e00 con 0x507ac00 2026-02-11T23:31:27.986+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.987+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 186 ==== MOSDPGPush(4.0 118/117 [PushOp(4:18b572c6:::obj440:head, version: 111'440, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18b572c6:::obj440:head@111'440, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56171e0 con 0x507ac00 2026-02-11T23:31:27.987+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:18b572c6:::obj440:head, version: 111'440, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18b572c6:::obj440:head@111'440, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 118 2026-02-11T23:31:27.987+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:18b572c6:::obj440:head, version: 111'440, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18b572c6:::obj440:head@111'440, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.987+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.987+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 187 ==== pg_backfill(progress 4.0 e 118/118 lb 4:1842fa44:::obj284:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176b00 con 0x507ac00 2026-02-11T23:31:27.987+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:1842fa44:::obj284:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:27.987+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:18b572c6:::obj440:head, version: 111'440, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18b572c6:::obj440:head@111'440, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.987+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1842fa44:::obj284:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.987+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.987+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:18b572c6:::obj440:head, version: 111'440, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18b572c6:::obj440:head@111'440, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.987+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:18b572c6:::obj440:head, version: 111'440, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18b572c6:::obj440:head@111'440, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(4.0 118/117 [PushOp(4:18b572c6:::obj440:head, version: 111'440, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18b572c6:::obj440:head@111'440, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:18100969:::obj255:head local-lis/les=117/118 n=73 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.987+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.987+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.987+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1842fa44:::obj284:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.987+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:18100969:::obj255:head local-lis/les=117/118 n=73 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:18b572c6:::obj440:head, version: 111'440, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18b572c6:::obj440:head@111'440, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.987+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:18100969:::obj255:head local-lis/les=117/118 n=73 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:18b572c6:::obj440:head@111'440, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.987+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:18100969:::obj255:head local-lis/les=117/118 n=73 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:18b572c6:::obj440:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.987+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:18100969:::obj255:head local-lis/les=117/118 n=73 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:18b572c6:::obj440:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.987+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:18100969:::obj255:head local-lis/les=117/118 n=73 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:18b572c6:::obj440:head 2026-02-11T23:31:27.987+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:18100969:::obj255:head local-lis/les=117/118 n=73 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:18b572c6:::obj440:head v 111'440 2026-02-11T23:31:27.987+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:18100969:::obj255:head local-lis/les=117/118 n=73 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.987+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.987+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:18b572c6:::obj440:head, version: 111'440, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18b572c6:::obj440:head@111'440, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:27.987+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.987+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.987+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1842fa44:::obj284:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.987+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:1842fa44:::obj284:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 4.0 e 118/118 lb 4:1842fa44:::obj284:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:18100969:::obj255:head local-lis/les=117/118 n=73 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.987+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.987+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:18100969:::obj255:head local-lis/les=117/118 n=73 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:1842fa44:::obj284:head) v3 2026-02-11T23:31:27.987+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:18100969:::obj255:head local-lis/les=117/118 n=73 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:1842fa44:::obj284:head) 2026-02-11T23:31:27.987+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.987+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:1842fa44:::obj284:head) v3 finish latency 0.000013 2026-02-11T23:31:27.987+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1842fa44:::obj284:head local-lis/les=117/118 n=74 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.987+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.987+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.987+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1842fa44:::obj284:head local-lis/les=117/118 n=74 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.987+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.987+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.987+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.987+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x5d34480 con 0x507ac00 2026-02-11T23:31:27.987+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:18b572c6:::obj440:head)]) -- 0x63b8600 con 0x507ac00 2026-02-11T23:31:27.987+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.987+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 188 ==== MOSDPGPush(4.0 118/117 [PushOp(4:18c50d58:::obj63:head, version: 111'63, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18c50d58:::obj63:head@111'63, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x56a3860 con 0x507ac00 2026-02-11T23:31:27.987+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:18c50d58:::obj63:head, version: 111'63, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18c50d58:::obj63:head@111'63, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 118 2026-02-11T23:31:27.987+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:18c50d58:::obj63:head, version: 111'63, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18c50d58:::obj63:head@111'63, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.987+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.987+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 189 ==== pg_backfill(progress 4.0 e 118/118 lb 4:18b572c6:::obj440:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98580 con 0x507ac00 2026-02-11T23:31:27.987+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:18c50d58:::obj63:head, version: 111'63, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18c50d58:::obj63:head@111'63, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.987+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:18b572c6:::obj440:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:27.987+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.987+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:18b572c6:::obj440:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.987+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:18c50d58:::obj63:head, version: 111'63, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18c50d58:::obj63:head@111'63, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.987+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.987+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:18c50d58:::obj63:head, version: 111'63, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18c50d58:::obj63:head@111'63, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 118/117 [PushOp(4:18c50d58:::obj63:head, version: 111'63, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18c50d58:::obj63:head@111'63, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:1842fa44:::obj284:head local-lis/les=117/118 n=74 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.987+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.987+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1842fa44:::obj284:head local-lis/les=117/118 n=74 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:18c50d58:::obj63:head, version: 111'63, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18c50d58:::obj63:head@111'63, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.987+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1842fa44:::obj284:head local-lis/les=117/118 n=74 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:18c50d58:::obj63:head@111'63, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.987+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.987+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:18b572c6:::obj440:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.987+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1842fa44:::obj284:head local-lis/les=117/118 n=74 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:18c50d58:::obj63:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.987+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1842fa44:::obj284:head local-lis/les=117/118 n=74 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:18c50d58:::obj63:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.987+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1842fa44:::obj284:head local-lis/les=117/118 n=74 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:18c50d58:::obj63:head 2026-02-11T23:31:27.987+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1842fa44:::obj284:head local-lis/les=117/118 n=74 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:18c50d58:::obj63:head v 111'63 2026-02-11T23:31:27.987+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1842fa44:::obj284:head local-lis/les=117/118 n=74 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.987+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.987+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:18c50d58:::obj63:head, version: 111'63, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18c50d58:::obj63:head@111'63, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:27.987+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.987+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.987+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:18b572c6:::obj440:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.987+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:18b572c6:::obj440:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 4.0 e 118/118 lb 4:18b572c6:::obj440:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:1842fa44:::obj284:head local-lis/les=117/118 n=74 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.987+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.987+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1842fa44:::obj284:head local-lis/les=117/118 n=74 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:18b572c6:::obj440:head) v3 2026-02-11T23:31:27.987+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1842fa44:::obj284:head local-lis/les=117/118 n=74 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:18b572c6:::obj440:head) 2026-02-11T23:31:27.987+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.987+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:18b572c6:::obj440:head) v3 finish latency 0.000015 2026-02-11T23:31:27.987+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:18b572c6:::obj440:head local-lis/les=117/118 n=75 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.987+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.988+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.988+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:18b572c6:::obj440:head local-lis/les=117/118 n=75 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.988+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.988+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.988+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.988+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x649fc80 con 0x507ac00 2026-02-11T23:31:27.988+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:18c50d58:::obj63:head)]) -- 0x63b8a80 con 0x507ac00 2026-02-11T23:31:27.988+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.988+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 190 ==== MOSDPGPush(4.0 118/117 [PushOp(4:19299046:::obj541:head, version: 111'541, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:19299046:::obj541:head@111'541, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a3040 con 0x507ac00 2026-02-11T23:31:27.988+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:19299046:::obj541:head, version: 111'541, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:19299046:::obj541:head@111'541, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 118 2026-02-11T23:31:27.988+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:19299046:::obj541:head, version: 111'541, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:19299046:::obj541:head@111'541, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.988+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.988+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:19299046:::obj541:head, version: 111'541, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:19299046:::obj541:head@111'541, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.988+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 191 ==== pg_backfill(progress 4.0 e 118/118 lb 4:18c50d58:::obj63:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d98000 con 0x507ac00 2026-02-11T23:31:27.988+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:18c50d58:::obj63:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:27.988+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.988+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:18c50d58:::obj63:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.988+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:19299046:::obj541:head, version: 111'541, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:19299046:::obj541:head@111'541, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.988+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:19299046:::obj541:head, version: 111'541, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:19299046:::obj541:head@111'541, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(4.0 118/117 [PushOp(4:19299046:::obj541:head, version: 111'541, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:19299046:::obj541:head@111'541, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:18b572c6:::obj440:head local-lis/les=117/118 n=75 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.988+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.988+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.988+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:18c50d58:::obj63:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.988+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:18b572c6:::obj440:head local-lis/les=117/118 n=75 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:19299046:::obj541:head, version: 111'541, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:19299046:::obj541:head@111'541, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.988+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:18b572c6:::obj440:head local-lis/les=117/118 n=75 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:19299046:::obj541:head@111'541, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.988+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:18b572c6:::obj440:head local-lis/les=117/118 n=75 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:19299046:::obj541:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.988+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:18b572c6:::obj440:head local-lis/les=117/118 n=75 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:19299046:::obj541:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.988+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:18b572c6:::obj440:head local-lis/les=117/118 n=75 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:19299046:::obj541:head 2026-02-11T23:31:27.988+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:18b572c6:::obj440:head local-lis/les=117/118 n=75 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:19299046:::obj541:head v 111'541 2026-02-11T23:31:27.988+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:18b572c6:::obj440:head local-lis/les=117/118 n=75 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.988+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.988+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:19299046:::obj541:head, version: 111'541, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:19299046:::obj541:head@111'541, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:27.988+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.988+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.988+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:18c50d58:::obj63:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.988+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:18c50d58:::obj63:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 4.0 e 118/118 lb 4:18c50d58:::obj63:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:18b572c6:::obj440:head local-lis/les=117/118 n=75 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.988+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.988+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:18b572c6:::obj440:head local-lis/les=117/118 n=75 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:18c50d58:::obj63:head) v3 2026-02-11T23:31:27.988+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:18b572c6:::obj440:head local-lis/les=117/118 n=75 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:18c50d58:::obj63:head) 2026-02-11T23:31:27.988+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.988+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:18c50d58:::obj63:head) v3 finish latency 0.000018 2026-02-11T23:31:27.988+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:18c50d58:::obj63:head local-lis/les=117/118 n=76 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.988+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.988+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.988+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:18c50d58:::obj63:head local-lis/les=117/118 n=76 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.988+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.988+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.988+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.988+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x649f800 con 0x507ac00 2026-02-11T23:31:27.988+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:19299046:::obj541:head)]) -- 0x6446300 con 0x507ac00 2026-02-11T23:31:27.988+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.989+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 192 ==== MOSDPGPush(4.0 118/117 [PushOp(4:1a1150ce:::obj60:head, version: 111'60, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a1150ce:::obj60:head@111'60, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x554e340 con 0x507ac00 2026-02-11T23:31:27.989+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:1a1150ce:::obj60:head, version: 111'60, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a1150ce:::obj60:head@111'60, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 118 2026-02-11T23:31:27.989+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1a1150ce:::obj60:head, version: 111'60, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a1150ce:::obj60:head@111'60, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.989+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.989+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 193 ==== pg_backfill(progress 4.0 e 118/118 lb 4:19299046:::obj541:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9b180 con 0x507ac00 2026-02-11T23:31:27.989+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1a1150ce:::obj60:head, version: 111'60, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a1150ce:::obj60:head@111'60, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.989+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:19299046:::obj541:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:27.989+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.989+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:19299046:::obj541:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.989+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1a1150ce:::obj60:head, version: 111'60, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a1150ce:::obj60:head@111'60, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.989+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.989+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:1a1150ce:::obj60:head, version: 111'60, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a1150ce:::obj60:head@111'60, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(4.0 118/117 [PushOp(4:1a1150ce:::obj60:head, version: 111'60, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a1150ce:::obj60:head@111'60, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:18c50d58:::obj63:head local-lis/les=117/118 n=76 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.989+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.989+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:18c50d58:::obj63:head local-lis/les=117/118 n=76 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:1a1150ce:::obj60:head, version: 111'60, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a1150ce:::obj60:head@111'60, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.989+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:18c50d58:::obj63:head local-lis/les=117/118 n=76 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:1a1150ce:::obj60:head@111'60, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.989+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.989+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:19299046:::obj541:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.989+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:18c50d58:::obj63:head local-lis/les=117/118 n=76 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:1a1150ce:::obj60:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.989+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:18c50d58:::obj63:head local-lis/les=117/118 n=76 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:1a1150ce:::obj60:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.989+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:18c50d58:::obj63:head local-lis/les=117/118 n=76 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:1a1150ce:::obj60:head 2026-02-11T23:31:27.989+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:18c50d58:::obj63:head local-lis/les=117/118 n=76 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:1a1150ce:::obj60:head v 111'60 2026-02-11T23:31:27.989+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:18c50d58:::obj63:head local-lis/les=117/118 n=76 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.989+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.989+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:1a1150ce:::obj60:head, version: 111'60, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a1150ce:::obj60:head@111'60, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:27.989+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.989+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.989+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:19299046:::obj541:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.989+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:19299046:::obj541:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 4.0 e 118/118 lb 4:19299046:::obj541:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:18c50d58:::obj63:head local-lis/les=117/118 n=76 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.989+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.989+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:18c50d58:::obj63:head local-lis/les=117/118 n=76 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:19299046:::obj541:head) v3 2026-02-11T23:31:27.989+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:18c50d58:::obj63:head local-lis/les=117/118 n=76 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:19299046:::obj541:head) 2026-02-11T23:31:27.989+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.989+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:19299046:::obj541:head) v3 finish latency 0.000016 2026-02-11T23:31:27.989+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:19299046:::obj541:head local-lis/les=117/118 n=77 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.989+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.989+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.989+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:19299046:::obj541:head local-lis/les=117/118 n=77 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.989+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.989+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.989+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.989+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x649fb00 con 0x507ac00 2026-02-11T23:31:27.989+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:1a1150ce:::obj60:head)]) -- 0x6446000 con 0x507ac00 2026-02-11T23:31:27.989+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.989+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 194 ==== MOSDPGPush(4.0 118/117 [PushOp(4:1a20990d:::obj103:head, version: 111'103, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a20990d:::obj103:head@111'103, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a21a0 con 0x507ac00 2026-02-11T23:31:27.989+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:1a20990d:::obj103:head, version: 111'103, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a20990d:::obj103:head@111'103, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 118 2026-02-11T23:31:27.989+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1a20990d:::obj103:head, version: 111'103, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a20990d:::obj103:head@111'103, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.989+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.989+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1a20990d:::obj103:head, version: 111'103, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a20990d:::obj103:head@111'103, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.989+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 195 ==== pg_backfill(progress 4.0 e 118/118 lb 4:1a1150ce:::obj60:head) ==== 1061+0+0 (unknown 0 0 0) 0x6178c00 con 0x507ac00 2026-02-11T23:31:27.989+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:1a1150ce:::obj60:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:27.989+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.989+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1a1150ce:::obj60:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.989+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1a20990d:::obj103:head, version: 111'103, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a20990d:::obj103:head@111'103, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.989+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.989+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:1a20990d:::obj103:head, version: 111'103, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a20990d:::obj103:head@111'103, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 118/117 [PushOp(4:1a20990d:::obj103:head, version: 111'103, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a20990d:::obj103:head@111'103, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:19299046:::obj541:head local-lis/les=117/118 n=77 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.989+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.989+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:19299046:::obj541:head local-lis/les=117/118 n=77 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:1a20990d:::obj103:head, version: 111'103, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a20990d:::obj103:head@111'103, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.989+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:19299046:::obj541:head local-lis/les=117/118 n=77 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:1a20990d:::obj103:head@111'103, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.989+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.989+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1a1150ce:::obj60:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.989+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:19299046:::obj541:head local-lis/les=117/118 n=77 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:1a20990d:::obj103:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.989+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:19299046:::obj541:head local-lis/les=117/118 n=77 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:1a20990d:::obj103:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.989+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:19299046:::obj541:head local-lis/les=117/118 n=77 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:1a20990d:::obj103:head 2026-02-11T23:31:27.989+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:19299046:::obj541:head local-lis/les=117/118 n=77 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:1a20990d:::obj103:head v 111'103 2026-02-11T23:31:27.989+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:19299046:::obj541:head local-lis/les=117/118 n=77 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.989+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.989+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:1a20990d:::obj103:head, version: 111'103, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a20990d:::obj103:head@111'103, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:27.989+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.989+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.989+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1a1150ce:::obj60:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.989+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:1a1150ce:::obj60:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 4.0 e 118/118 lb 4:1a1150ce:::obj60:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:19299046:::obj541:head local-lis/les=117/118 n=77 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.989+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.989+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:19299046:::obj541:head local-lis/les=117/118 n=77 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:1a1150ce:::obj60:head) v3 2026-02-11T23:31:27.989+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:19299046:::obj541:head local-lis/les=117/118 n=77 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:1a1150ce:::obj60:head) 2026-02-11T23:31:27.989+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.989+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:1a1150ce:::obj60:head) v3 finish latency 0.000017 2026-02-11T23:31:27.989+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1a1150ce:::obj60:head local-lis/les=117/118 n=78 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.989+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.990+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.990+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1a1150ce:::obj60:head local-lis/les=117/118 n=78 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.990+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.990+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.990+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.990+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x5f1c300 con 0x507ac00 2026-02-11T23:31:27.990+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:1a20990d:::obj103:head)]) -- 0x63b8c00 con 0x507ac00 2026-02-11T23:31:27.990+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.990+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 196 ==== MOSDPGPush(4.0 118/117 [PushOp(4:1a2eef70:::obj126:head, version: 111'126, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a2eef70:::obj126:head@111'126, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3815ba0 con 0x507ac00 2026-02-11T23:31:27.990+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:1a2eef70:::obj126:head, version: 111'126, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a2eef70:::obj126:head@111'126, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 118 2026-02-11T23:31:27.990+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1a2eef70:::obj126:head, version: 111'126, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a2eef70:::obj126:head@111'126, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.990+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.990+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1a2eef70:::obj126:head, version: 111'126, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a2eef70:::obj126:head@111'126, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.990+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 197 ==== pg_backfill(progress 4.0 e 118/118 lb 4:1a20990d:::obj103:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2db80 con 0x507ac00 2026-02-11T23:31:27.990+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:1a20990d:::obj103:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:27.990+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.990+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1a20990d:::obj103:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.990+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1a2eef70:::obj126:head, version: 111'126, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a2eef70:::obj126:head@111'126, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.990+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.990+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:1a2eef70:::obj126:head, version: 111'126, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a2eef70:::obj126:head@111'126, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(4.0 118/117 [PushOp(4:1a2eef70:::obj126:head, version: 111'126, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a2eef70:::obj126:head@111'126, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:1a1150ce:::obj60:head local-lis/les=117/118 n=78 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.990+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.990+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1a1150ce:::obj60:head local-lis/les=117/118 n=78 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:1a2eef70:::obj126:head, version: 111'126, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a2eef70:::obj126:head@111'126, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.990+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.990+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1a1150ce:::obj60:head local-lis/les=117/118 n=78 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:1a2eef70:::obj126:head@111'126, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.990+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1a20990d:::obj103:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.990+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1a1150ce:::obj60:head local-lis/les=117/118 n=78 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:1a2eef70:::obj126:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.990+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1a1150ce:::obj60:head local-lis/les=117/118 n=78 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:1a2eef70:::obj126:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.990+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1a1150ce:::obj60:head local-lis/les=117/118 n=78 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:1a2eef70:::obj126:head 2026-02-11T23:31:27.990+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1a1150ce:::obj60:head local-lis/les=117/118 n=78 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:1a2eef70:::obj126:head v 111'126 2026-02-11T23:31:27.990+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1a1150ce:::obj60:head local-lis/les=117/118 n=78 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.990+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.990+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:1a2eef70:::obj126:head, version: 111'126, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a2eef70:::obj126:head@111'126, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:27.990+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.990+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.990+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1a20990d:::obj103:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.990+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:1a20990d:::obj103:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 4.0 e 118/118 lb 4:1a20990d:::obj103:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:1a1150ce:::obj60:head local-lis/les=117/118 n=78 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.990+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.990+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1a1150ce:::obj60:head local-lis/les=117/118 n=78 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:1a20990d:::obj103:head) v3 2026-02-11T23:31:27.990+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1a1150ce:::obj60:head local-lis/les=117/118 n=78 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:1a20990d:::obj103:head) 2026-02-11T23:31:27.990+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.990+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:1a20990d:::obj103:head) v3 finish latency 0.000016 2026-02-11T23:31:27.990+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1a20990d:::obj103:head local-lis/les=117/118 n=79 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.990+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.990+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.990+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1a20990d:::obj103:head local-lis/les=117/118 n=79 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.990+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.990+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.990+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.990+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x649f680 con 0x507ac00 2026-02-11T23:31:27.990+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:1a2eef70:::obj126:head)]) -- 0x5f75500 con 0x507ac00 2026-02-11T23:31:27.990+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.990+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.991+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 198 ==== MOSDPGPush(4.0 118/117 [PushOp(4:1a783fd9:::obj328:head, version: 111'328, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a783fd9:::obj328:head@111'328, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x555d040 con 0x507ac00 2026-02-11T23:31:27.991+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:1a783fd9:::obj328:head, version: 111'328, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a783fd9:::obj328:head@111'328, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 118 2026-02-11T23:31:27.991+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1a783fd9:::obj328:head, version: 111'328, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a783fd9:::obj328:head@111'328, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.991+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.991+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 199 ==== pg_backfill(progress 4.0 e 118/118 lb 4:1a2eef70:::obj126:head) ==== 1062+0+0 (unknown 0 0 0) 0x6179180 con 0x507ac00 2026-02-11T23:31:27.991+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:1a2eef70:::obj126:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:27.991+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1a783fd9:::obj328:head, version: 111'328, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a783fd9:::obj328:head@111'328, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.991+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1a2eef70:::obj126:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.991+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.991+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1a783fd9:::obj328:head, version: 111'328, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a783fd9:::obj328:head@111'328, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.991+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.991+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:1a783fd9:::obj328:head, version: 111'328, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a783fd9:::obj328:head@111'328, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(4.0 118/117 [PushOp(4:1a783fd9:::obj328:head, version: 111'328, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a783fd9:::obj328:head@111'328, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:1a20990d:::obj103:head local-lis/les=117/118 n=79 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.991+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1a2eef70:::obj126:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.991+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.991+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1a20990d:::obj103:head local-lis/les=117/118 n=79 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:1a783fd9:::obj328:head, version: 111'328, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a783fd9:::obj328:head@111'328, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.991+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1a20990d:::obj103:head local-lis/les=117/118 n=79 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:1a783fd9:::obj328:head@111'328, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.991+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1a20990d:::obj103:head local-lis/les=117/118 n=79 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:1a783fd9:::obj328:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.991+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1a20990d:::obj103:head local-lis/les=117/118 n=79 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:1a783fd9:::obj328:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.991+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1a20990d:::obj103:head local-lis/les=117/118 n=79 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:1a783fd9:::obj328:head 2026-02-11T23:31:27.991+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1a20990d:::obj103:head local-lis/les=117/118 n=79 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:1a783fd9:::obj328:head v 111'328 2026-02-11T23:31:27.991+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1a20990d:::obj103:head local-lis/les=117/118 n=79 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.991+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.991+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:1a783fd9:::obj328:head, version: 111'328, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a783fd9:::obj328:head@111'328, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:27.991+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.991+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.991+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1a2eef70:::obj126:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.991+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:1a2eef70:::obj126:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 4.0 e 118/118 lb 4:1a2eef70:::obj126:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:1a20990d:::obj103:head local-lis/les=117/118 n=79 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.991+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.991+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1a20990d:::obj103:head local-lis/les=117/118 n=79 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:1a2eef70:::obj126:head) v3 2026-02-11T23:31:27.991+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1a20990d:::obj103:head local-lis/les=117/118 n=79 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:1a2eef70:::obj126:head) 2026-02-11T23:31:27.991+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.991+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:1a2eef70:::obj126:head) v3 finish latency 0.000015 2026-02-11T23:31:27.991+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1a2eef70:::obj126:head local-lis/les=117/118 n=80 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.991+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.991+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.991+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1a2eef70:::obj126:head local-lis/les=117/118 n=80 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.991+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.991+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.991+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.991+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x5f2d800 con 0x507ac00 2026-02-11T23:31:27.991+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:1a783fd9:::obj328:head)]) -- 0x63b8d80 con 0x507ac00 2026-02-11T23:31:27.991+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.991+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 200 ==== MOSDPGPush(4.0 118/117 [PushOp(4:1abd4411:::obj66:head, version: 111'66, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1abd4411:::obj66:head@111'66, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x56a2820 con 0x507ac00 2026-02-11T23:31:27.991+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:1abd4411:::obj66:head, version: 111'66, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1abd4411:::obj66:head@111'66, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 118 2026-02-11T23:31:27.991+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1abd4411:::obj66:head, version: 111'66, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1abd4411:::obj66:head@111'66, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.991+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.991+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 201 ==== pg_backfill(progress 4.0 e 118/118 lb 4:1a783fd9:::obj328:head) ==== 1062+0+0 (unknown 0 0 0) 0x6178100 con 0x507ac00 2026-02-11T23:31:27.991+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1abd4411:::obj66:head, version: 111'66, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1abd4411:::obj66:head@111'66, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.991+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:1a783fd9:::obj328:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:27.991+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.991+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1a783fd9:::obj328:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.991+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1abd4411:::obj66:head, version: 111'66, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1abd4411:::obj66:head@111'66, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.991+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.991+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:1abd4411:::obj66:head, version: 111'66, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1abd4411:::obj66:head@111'66, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(4.0 118/117 [PushOp(4:1abd4411:::obj66:head, version: 111'66, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1abd4411:::obj66:head@111'66, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:1a2eef70:::obj126:head local-lis/les=117/118 n=80 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.991+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.991+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1a2eef70:::obj126:head local-lis/les=117/118 n=80 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:1abd4411:::obj66:head, version: 111'66, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1abd4411:::obj66:head@111'66, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.991+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1a2eef70:::obj126:head local-lis/les=117/118 n=80 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:1abd4411:::obj66:head@111'66, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.991+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.991+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1a783fd9:::obj328:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.991+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1a2eef70:::obj126:head local-lis/les=117/118 n=80 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:1abd4411:::obj66:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.991+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1a2eef70:::obj126:head local-lis/les=117/118 n=80 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:1abd4411:::obj66:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.991+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1a2eef70:::obj126:head local-lis/les=117/118 n=80 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:1abd4411:::obj66:head 2026-02-11T23:31:27.991+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1a2eef70:::obj126:head local-lis/les=117/118 n=80 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:1abd4411:::obj66:head v 111'66 2026-02-11T23:31:27.991+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1a2eef70:::obj126:head local-lis/les=117/118 n=80 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.991+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.991+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:1abd4411:::obj66:head, version: 111'66, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1abd4411:::obj66:head@111'66, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:27.991+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.991+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.991+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1a783fd9:::obj328:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.991+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:1a783fd9:::obj328:head) v3 prio 15 cost 0 latency 0.000079 pg_backfill(progress 4.0 e 118/118 lb 4:1a783fd9:::obj328:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:1a2eef70:::obj126:head local-lis/les=117/118 n=80 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.991+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.991+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1a2eef70:::obj126:head local-lis/les=117/118 n=80 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:1a783fd9:::obj328:head) v3 2026-02-11T23:31:27.991+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1a2eef70:::obj126:head local-lis/les=117/118 n=80 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:1a783fd9:::obj328:head) 2026-02-11T23:31:27.991+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.991+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:1a783fd9:::obj328:head) v3 finish latency 0.000016 2026-02-11T23:31:27.991+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1a783fd9:::obj328:head local-lis/les=117/118 n=81 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.991+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.992+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1a783fd9:::obj328:head local-lis/les=117/118 n=81 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.992+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.992+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.992+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.992+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.992+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x649f500 con 0x507ac00 2026-02-11T23:31:27.992+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:1abd4411:::obj66:head)]) -- 0x5f1d200 con 0x507ac00 2026-02-11T23:31:27.992+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.992+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 202 ==== MOSDPGPush(4.0 118/117 [PushOp(4:1b644371:::obj48:head, version: 111'48, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b644371:::obj48:head@111'48, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x56a3ba0 con 0x507ac00 2026-02-11T23:31:27.992+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:1b644371:::obj48:head, version: 111'48, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b644371:::obj48:head@111'48, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 118 2026-02-11T23:31:27.992+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1b644371:::obj48:head, version: 111'48, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b644371:::obj48:head@111'48, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.992+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.992+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 203 ==== pg_backfill(progress 4.0 e 118/118 lb 4:1abd4411:::obj66:head) ==== 1061+0+0 (unknown 0 0 0) 0x6177b80 con 0x507ac00 2026-02-11T23:31:27.992+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1b644371:::obj48:head, version: 111'48, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b644371:::obj48:head@111'48, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.992+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:1abd4411:::obj66:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:27.992+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1abd4411:::obj66:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.992+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.992+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1b644371:::obj48:head, version: 111'48, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b644371:::obj48:head@111'48, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.992+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.992+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1abd4411:::obj66:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.992+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:1b644371:::obj48:head, version: 111'48, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b644371:::obj48:head@111'48, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(4.0 118/117 [PushOp(4:1b644371:::obj48:head, version: 111'48, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b644371:::obj48:head@111'48, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:1a783fd9:::obj328:head local-lis/les=117/118 n=81 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.992+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.992+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1a783fd9:::obj328:head local-lis/les=117/118 n=81 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:1b644371:::obj48:head, version: 111'48, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b644371:::obj48:head@111'48, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.992+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1a783fd9:::obj328:head local-lis/les=117/118 n=81 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:1b644371:::obj48:head@111'48, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.992+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1a783fd9:::obj328:head local-lis/les=117/118 n=81 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:1b644371:::obj48:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.992+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1a783fd9:::obj328:head local-lis/les=117/118 n=81 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:1b644371:::obj48:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.992+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1a783fd9:::obj328:head local-lis/les=117/118 n=81 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:1b644371:::obj48:head 2026-02-11T23:31:27.992+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1a783fd9:::obj328:head local-lis/les=117/118 n=81 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:1b644371:::obj48:head v 111'48 2026-02-11T23:31:27.992+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1a783fd9:::obj328:head local-lis/les=117/118 n=81 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.992+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.992+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:1b644371:::obj48:head, version: 111'48, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b644371:::obj48:head@111'48, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:27.992+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.992+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.992+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1abd4411:::obj66:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.992+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:1abd4411:::obj66:head) v3 prio 15 cost 0 latency 0.000079 pg_backfill(progress 4.0 e 118/118 lb 4:1abd4411:::obj66:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:1a783fd9:::obj328:head local-lis/les=117/118 n=81 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.992+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.992+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1a783fd9:::obj328:head local-lis/les=117/118 n=81 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:1abd4411:::obj66:head) v3 2026-02-11T23:31:27.992+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1a783fd9:::obj328:head local-lis/les=117/118 n=81 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:1abd4411:::obj66:head) 2026-02-11T23:31:27.992+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.992+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:1abd4411:::obj66:head) v3 finish latency 0.000015 2026-02-11T23:31:27.992+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1abd4411:::obj66:head local-lis/les=117/118 n=82 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.992+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.992+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.992+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1abd4411:::obj66:head local-lis/les=117/118 n=82 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.992+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.992+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.992+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.992+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x5f75e00 con 0x507ac00 2026-02-11T23:31:27.992+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:1b644371:::obj48:head)]) -- 0x5f74180 con 0x507ac00 2026-02-11T23:31:27.992+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.993+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 204 ==== MOSDPGPush(4.0 118/117 [PushOp(4:1b6bffcc:::obj61:head, version: 111'61, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b6bffcc:::obj61:head@111'61, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5db0b60 con 0x507ac00 2026-02-11T23:31:27.993+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:1b6bffcc:::obj61:head, version: 111'61, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b6bffcc:::obj61:head@111'61, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 118 2026-02-11T23:31:27.993+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1b6bffcc:::obj61:head, version: 111'61, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b6bffcc:::obj61:head@111'61, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.993+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.993+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1b6bffcc:::obj61:head, version: 111'61, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b6bffcc:::obj61:head@111'61, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.993+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 205 ==== pg_backfill(progress 4.0 e 118/118 lb 4:1b644371:::obj48:head) ==== 1061+0+0 (unknown 0 0 0) 0x6177600 con 0x507ac00 2026-02-11T23:31:27.993+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:1b644371:::obj48:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:27.993+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.993+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1b644371:::obj48:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.993+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1b6bffcc:::obj61:head, version: 111'61, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b6bffcc:::obj61:head@111'61, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.993+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:1b6bffcc:::obj61:head, version: 111'61, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b6bffcc:::obj61:head@111'61, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 118/117 [PushOp(4:1b6bffcc:::obj61:head, version: 111'61, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b6bffcc:::obj61:head@111'61, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:1abd4411:::obj66:head local-lis/les=117/118 n=82 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.993+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.993+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.993+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1abd4411:::obj66:head local-lis/les=117/118 n=82 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:1b6bffcc:::obj61:head, version: 111'61, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b6bffcc:::obj61:head@111'61, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.993+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1b644371:::obj48:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.993+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1abd4411:::obj66:head local-lis/les=117/118 n=82 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:1b6bffcc:::obj61:head@111'61, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.993+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1abd4411:::obj66:head local-lis/les=117/118 n=82 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:1b6bffcc:::obj61:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.993+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1abd4411:::obj66:head local-lis/les=117/118 n=82 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:1b6bffcc:::obj61:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.993+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1abd4411:::obj66:head local-lis/les=117/118 n=82 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:1b6bffcc:::obj61:head 2026-02-11T23:31:27.993+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1abd4411:::obj66:head local-lis/les=117/118 n=82 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:1b6bffcc:::obj61:head v 111'61 2026-02-11T23:31:27.993+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1abd4411:::obj66:head local-lis/les=117/118 n=82 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.993+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.993+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:1b6bffcc:::obj61:head, version: 111'61, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b6bffcc:::obj61:head@111'61, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:27.993+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.993+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.993+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1b644371:::obj48:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.993+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:1b644371:::obj48:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 4.0 e 118/118 lb 4:1b644371:::obj48:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:1abd4411:::obj66:head local-lis/les=117/118 n=82 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.993+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.993+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1abd4411:::obj66:head local-lis/les=117/118 n=82 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:1b644371:::obj48:head) v3 2026-02-11T23:31:27.993+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1abd4411:::obj66:head local-lis/les=117/118 n=82 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:1b644371:::obj48:head) 2026-02-11T23:31:27.993+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.993+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:1b644371:::obj48:head) v3 finish latency 0.000014 2026-02-11T23:31:27.993+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1b644371:::obj48:head local-lis/les=117/118 n=83 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.993+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.993+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.993+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1b644371:::obj48:head local-lis/les=117/118 n=83 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.993+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.993+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.993+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.993+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x649f380 con 0x507ac00 2026-02-11T23:31:27.993+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:1b6bffcc:::obj61:head)]) -- 0x63b9e00 con 0x507ac00 2026-02-11T23:31:27.993+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.993+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 206 ==== MOSDPGPush(4.0 118/117 [PushOp(4:1b6ee6bf:::obj501:head, version: 111'501, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b6ee6bf:::obj501:head@111'501, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2680 con 0x507ac00 2026-02-11T23:31:27.993+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:1b6ee6bf:::obj501:head, version: 111'501, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b6ee6bf:::obj501:head@111'501, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 118 2026-02-11T23:31:27.993+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1b6ee6bf:::obj501:head, version: 111'501, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b6ee6bf:::obj501:head@111'501, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.993+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.993+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 207 ==== pg_backfill(progress 4.0 e 118/118 lb 4:1b6bffcc:::obj61:head) ==== 1061+0+0 (unknown 0 0 0) 0x5985700 con 0x507ac00 2026-02-11T23:31:27.993+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1b6ee6bf:::obj501:head, version: 111'501, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b6ee6bf:::obj501:head@111'501, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.993+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:1b6bffcc:::obj61:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:27.993+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1b6bffcc:::obj61:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.993+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.993+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1b6ee6bf:::obj501:head, version: 111'501, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b6ee6bf:::obj501:head@111'501, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.993+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:1b6ee6bf:::obj501:head, version: 111'501, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b6ee6bf:::obj501:head@111'501, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(4.0 118/117 [PushOp(4:1b6ee6bf:::obj501:head, version: 111'501, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b6ee6bf:::obj501:head@111'501, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:1b644371:::obj48:head local-lis/les=117/118 n=83 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.993+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.993+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.993+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1b644371:::obj48:head local-lis/les=117/118 n=83 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:1b6ee6bf:::obj501:head, version: 111'501, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b6ee6bf:::obj501:head@111'501, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.993+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1b6bffcc:::obj61:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.993+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1b644371:::obj48:head local-lis/les=117/118 n=83 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:1b6ee6bf:::obj501:head@111'501, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.993+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1b644371:::obj48:head local-lis/les=117/118 n=83 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:1b6ee6bf:::obj501:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.993+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1b644371:::obj48:head local-lis/les=117/118 n=83 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:1b6ee6bf:::obj501:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.993+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1b644371:::obj48:head local-lis/les=117/118 n=83 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:1b6ee6bf:::obj501:head 2026-02-11T23:31:27.993+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1b644371:::obj48:head local-lis/les=117/118 n=83 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:1b6ee6bf:::obj501:head v 111'501 2026-02-11T23:31:27.993+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1b644371:::obj48:head local-lis/les=117/118 n=83 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.993+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.993+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:1b6ee6bf:::obj501:head, version: 111'501, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b6ee6bf:::obj501:head@111'501, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:27.993+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.993+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.993+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1b6bffcc:::obj61:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.993+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:1b6bffcc:::obj61:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 4.0 e 118/118 lb 4:1b6bffcc:::obj61:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:1b644371:::obj48:head local-lis/les=117/118 n=83 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.993+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.993+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1b644371:::obj48:head local-lis/les=117/118 n=83 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:1b6bffcc:::obj61:head) v3 2026-02-11T23:31:27.993+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1b644371:::obj48:head local-lis/les=117/118 n=83 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:1b6bffcc:::obj61:head) 2026-02-11T23:31:27.993+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.993+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:1b6bffcc:::obj61:head) v3 finish latency 0.000016 2026-02-11T23:31:27.993+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1b6bffcc:::obj61:head local-lis/les=117/118 n=84 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.993+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.994+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.994+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1b6bffcc:::obj61:head local-lis/les=117/118 n=84 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.994+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.994+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.994+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.994+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x5f75c80 con 0x507ac00 2026-02-11T23:31:27.994+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:1b6ee6bf:::obj501:head)]) -- 0x5f74f00 con 0x507ac00 2026-02-11T23:31:27.994+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.994+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 208 ==== MOSDPGPush(4.0 118/117 [PushOp(4:1bad8e21:::obj122:head, version: 111'122, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1bad8e21:::obj122:head@111'122, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56cab60 con 0x507ac00 2026-02-11T23:31:27.994+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:1bad8e21:::obj122:head, version: 111'122, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1bad8e21:::obj122:head@111'122, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 118 2026-02-11T23:31:27.994+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1bad8e21:::obj122:head, version: 111'122, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1bad8e21:::obj122:head@111'122, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.994+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.994+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 209 ==== pg_backfill(progress 4.0 e 118/118 lb 4:1b6ee6bf:::obj501:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9a680 con 0x507ac00 2026-02-11T23:31:27.994+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:1b6ee6bf:::obj501:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:27.994+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1bad8e21:::obj122:head, version: 111'122, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1bad8e21:::obj122:head@111'122, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.994+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1b6ee6bf:::obj501:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.994+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.994+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1b6ee6bf:::obj501:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.994+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.994+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1bad8e21:::obj122:head, version: 111'122, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1bad8e21:::obj122:head@111'122, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.994+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:1bad8e21:::obj122:head, version: 111'122, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1bad8e21:::obj122:head@111'122, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000051 MOSDPGPush(4.0 118/117 [PushOp(4:1bad8e21:::obj122:head, version: 111'122, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1bad8e21:::obj122:head@111'122, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:1b6bffcc:::obj61:head local-lis/les=117/118 n=84 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.994+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.994+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1b6bffcc:::obj61:head local-lis/les=117/118 n=84 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:1bad8e21:::obj122:head, version: 111'122, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1bad8e21:::obj122:head@111'122, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.994+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1b6bffcc:::obj61:head local-lis/les=117/118 n=84 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:1bad8e21:::obj122:head@111'122, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.994+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1b6bffcc:::obj61:head local-lis/les=117/118 n=84 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:1bad8e21:::obj122:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.994+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1b6bffcc:::obj61:head local-lis/les=117/118 n=84 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:1bad8e21:::obj122:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.994+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1b6bffcc:::obj61:head local-lis/les=117/118 n=84 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:1bad8e21:::obj122:head 2026-02-11T23:31:27.994+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1b6bffcc:::obj61:head local-lis/les=117/118 n=84 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:1bad8e21:::obj122:head v 111'122 2026-02-11T23:31:27.994+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1b6bffcc:::obj61:head local-lis/les=117/118 n=84 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.994+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.994+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:1bad8e21:::obj122:head, version: 111'122, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1bad8e21:::obj122:head@111'122, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:27.994+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.994+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.994+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1b6ee6bf:::obj501:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.994+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:1b6ee6bf:::obj501:head) v3 prio 15 cost 0 latency 0.000089 pg_backfill(progress 4.0 e 118/118 lb 4:1b6ee6bf:::obj501:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:1b6bffcc:::obj61:head local-lis/les=117/118 n=84 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.994+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.994+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1b6bffcc:::obj61:head local-lis/les=117/118 n=84 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:1b6ee6bf:::obj501:head) v3 2026-02-11T23:31:27.994+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1b6bffcc:::obj61:head local-lis/les=117/118 n=84 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:1b6ee6bf:::obj501:head) 2026-02-11T23:31:27.994+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.994+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:1b6ee6bf:::obj501:head) v3 finish latency 0.000013 2026-02-11T23:31:27.994+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1b6ee6bf:::obj501:head local-lis/les=117/118 n=85 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.994+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.994+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.994+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1b6ee6bf:::obj501:head local-lis/les=117/118 n=85 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.994+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.994+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.994+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.994+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x649f200 con 0x507ac00 2026-02-11T23:31:27.994+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:1bad8e21:::obj122:head)]) -- 0x5f74000 con 0x507ac00 2026-02-11T23:31:27.994+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.995+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 210 ==== MOSDPGPush(4.0 118/117 [PushOp(4:1bb1e69e:::obj390:head, version: 111'390, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1bb1e69e:::obj390:head@111'390, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x555cea0 con 0x507ac00 2026-02-11T23:31:27.995+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:1bb1e69e:::obj390:head, version: 111'390, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1bb1e69e:::obj390:head@111'390, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 118 2026-02-11T23:31:27.995+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1bb1e69e:::obj390:head, version: 111'390, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1bb1e69e:::obj390:head@111'390, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.995+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.995+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 211 ==== pg_backfill(progress 4.0 e 118/118 lb 4:1bad8e21:::obj122:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176000 con 0x507ac00 2026-02-11T23:31:27.995+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1bb1e69e:::obj390:head, version: 111'390, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1bb1e69e:::obj390:head@111'390, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.995+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:1bad8e21:::obj122:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 118 2026-02-11T23:31:27.995+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1bad8e21:::obj122:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.995+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.995+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1bb1e69e:::obj390:head, version: 111'390, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1bb1e69e:::obj390:head@111'390, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.995+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:1bb1e69e:::obj390:head, version: 111'390, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1bb1e69e:::obj390:head@111'390, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 118/117 [PushOp(4:1bb1e69e:::obj390:head, version: 111'390, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1bb1e69e:::obj390:head@111'390, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:1b6ee6bf:::obj501:head local-lis/les=117/118 n=85 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.995+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.995+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.995+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1bad8e21:::obj122:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.995+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1b6ee6bf:::obj501:head local-lis/les=117/118 n=85 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:1bb1e69e:::obj390:head, version: 111'390, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1bb1e69e:::obj390:head@111'390, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.995+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1b6ee6bf:::obj501:head local-lis/les=117/118 n=85 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:1bb1e69e:::obj390:head@111'390, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.995+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1b6ee6bf:::obj501:head local-lis/les=117/118 n=85 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:1bb1e69e:::obj390:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.995+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1b6ee6bf:::obj501:head local-lis/les=117/118 n=85 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:1bb1e69e:::obj390:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.995+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1b6ee6bf:::obj501:head local-lis/les=117/118 n=85 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:1bb1e69e:::obj390:head 2026-02-11T23:31:27.995+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1b6ee6bf:::obj501:head local-lis/les=117/118 n=85 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:1bb1e69e:::obj390:head v 111'390 2026-02-11T23:31:27.995+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1b6ee6bf:::obj501:head local-lis/les=117/118 n=85 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.995+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.995+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:1bb1e69e:::obj390:head, version: 111'390, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1bb1e69e:::obj390:head@111'390, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:27.995+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.995+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.995+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1bad8e21:::obj122:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.995+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:1bad8e21:::obj122:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 4.0 e 118/118 lb 4:1bad8e21:::obj122:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:1b6ee6bf:::obj501:head local-lis/les=117/118 n=85 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.995+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.995+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1b6ee6bf:::obj501:head local-lis/les=117/118 n=85 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:1bad8e21:::obj122:head) v3 2026-02-11T23:31:27.995+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1b6ee6bf:::obj501:head local-lis/les=117/118 n=85 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:1bad8e21:::obj122:head) 2026-02-11T23:31:27.995+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.995+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:1bad8e21:::obj122:head) v3 finish latency 0.000014 2026-02-11T23:31:27.995+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1bad8e21:::obj122:head local-lis/les=117/118 n=86 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.995+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.995+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1bad8e21:::obj122:head local-lis/les=117/118 n=86 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.995+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.995+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.995+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.995+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.995+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x5f2dc80 con 0x507ac00 2026-02-11T23:31:27.995+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:1bb1e69e:::obj390:head)]) -- 0x5fed380 con 0x507ac00 2026-02-11T23:31:27.995+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.995+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 212 ==== MOSDPGPush(4.0 118/117 [PushOp(4:1c47bf8e:::obj558:head, version: 111'558, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1c47bf8e:::obj558:head@111'558, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f1ba0 con 0x507ac00 2026-02-11T23:31:27.995+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:1c47bf8e:::obj558:head, version: 111'558, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1c47bf8e:::obj558:head@111'558, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 118 2026-02-11T23:31:27.995+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1c47bf8e:::obj558:head, version: 111'558, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1c47bf8e:::obj558:head@111'558, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.995+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.995+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1c47bf8e:::obj558:head, version: 111'558, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1c47bf8e:::obj558:head@111'558, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.995+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 213 ==== pg_backfill(progress 4.0 e 118/118 lb 4:1bb1e69e:::obj390:head) ==== 1062+0+0 (unknown 0 0 0) 0x6179700 con 0x507ac00 2026-02-11T23:31:27.995+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.995+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:1bb1e69e:::obj390:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 118 2026-02-11T23:31:27.995+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1c47bf8e:::obj558:head, version: 111'558, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1c47bf8e:::obj558:head@111'558, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.995+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1bb1e69e:::obj390:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.995+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.995+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:1c47bf8e:::obj558:head, version: 111'558, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1c47bf8e:::obj558:head@111'558, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 118/117 [PushOp(4:1c47bf8e:::obj558:head, version: 111'558, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1c47bf8e:::obj558:head@111'558, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:1bad8e21:::obj122:head local-lis/les=117/118 n=86 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.995+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1bb1e69e:::obj390:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.995+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.995+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1bad8e21:::obj122:head local-lis/les=117/118 n=86 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:1c47bf8e:::obj558:head, version: 111'558, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1c47bf8e:::obj558:head@111'558, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.995+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1bad8e21:::obj122:head local-lis/les=117/118 n=86 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:1c47bf8e:::obj558:head@111'558, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.995+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1bad8e21:::obj122:head local-lis/les=117/118 n=86 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:1c47bf8e:::obj558:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.995+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1bad8e21:::obj122:head local-lis/les=117/118 n=86 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:1c47bf8e:::obj558:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.995+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1bad8e21:::obj122:head local-lis/les=117/118 n=86 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:1c47bf8e:::obj558:head 2026-02-11T23:31:27.995+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1bad8e21:::obj122:head local-lis/les=117/118 n=86 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:1c47bf8e:::obj558:head v 111'558 2026-02-11T23:31:27.995+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1bad8e21:::obj122:head local-lis/les=117/118 n=86 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.995+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.995+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:1c47bf8e:::obj558:head, version: 111'558, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1c47bf8e:::obj558:head@111'558, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:27.995+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.995+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.995+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1bb1e69e:::obj390:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.995+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:1bb1e69e:::obj390:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 118/118 lb 4:1bb1e69e:::obj390:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:1bad8e21:::obj122:head local-lis/les=117/118 n=86 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.995+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.995+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1bad8e21:::obj122:head local-lis/les=117/118 n=86 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:1bb1e69e:::obj390:head) v3 2026-02-11T23:31:27.995+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1bad8e21:::obj122:head local-lis/les=117/118 n=86 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:1bb1e69e:::obj390:head) 2026-02-11T23:31:27.995+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.995+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:1bb1e69e:::obj390:head) v3 finish latency 0.000015 2026-02-11T23:31:27.995+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1bb1e69e:::obj390:head local-lis/les=117/118 n=87 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.995+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.996+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.996+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1bb1e69e:::obj390:head local-lis/les=117/118 n=87 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.996+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.996+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.996+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.996+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x649f080 con 0x507ac00 2026-02-11T23:31:27.996+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:1c47bf8e:::obj558:head)]) -- 0x63b9500 con 0x507ac00 2026-02-11T23:31:27.996+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.996+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 214 ==== MOSDPGPush(4.0 118/117 [PushOp(4:1cb33d95:::obj190:head, version: 111'190, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cb33d95:::obj190:head@111'190, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5647380 con 0x507ac00 2026-02-11T23:31:27.996+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:1cb33d95:::obj190:head, version: 111'190, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cb33d95:::obj190:head@111'190, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 118 2026-02-11T23:31:27.996+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1cb33d95:::obj190:head, version: 111'190, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cb33d95:::obj190:head@111'190, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.996+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.996+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 215 ==== pg_backfill(progress 4.0 e 118/118 lb 4:1c47bf8e:::obj558:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2cb00 con 0x507ac00 2026-02-11T23:31:27.996+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1cb33d95:::obj190:head, version: 111'190, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cb33d95:::obj190:head@111'190, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.996+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:1c47bf8e:::obj558:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:27.996+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.996+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1c47bf8e:::obj558:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.996+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1cb33d95:::obj190:head, version: 111'190, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cb33d95:::obj190:head@111'190, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.996+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.996+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:1cb33d95:::obj190:head, version: 111'190, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cb33d95:::obj190:head@111'190, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(4.0 118/117 [PushOp(4:1cb33d95:::obj190:head, version: 111'190, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cb33d95:::obj190:head@111'190, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:1bb1e69e:::obj390:head local-lis/les=117/118 n=87 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.996+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.996+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1bb1e69e:::obj390:head local-lis/les=117/118 n=87 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:1cb33d95:::obj190:head, version: 111'190, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cb33d95:::obj190:head@111'190, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.996+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1bb1e69e:::obj390:head local-lis/les=117/118 n=87 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:1cb33d95:::obj190:head@111'190, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.996+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.996+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1c47bf8e:::obj558:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.996+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1bb1e69e:::obj390:head local-lis/les=117/118 n=87 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:1cb33d95:::obj190:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.996+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1bb1e69e:::obj390:head local-lis/les=117/118 n=87 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:1cb33d95:::obj190:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.996+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1bb1e69e:::obj390:head local-lis/les=117/118 n=87 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:1cb33d95:::obj190:head 2026-02-11T23:31:27.996+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1bb1e69e:::obj390:head local-lis/les=117/118 n=87 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:1cb33d95:::obj190:head v 111'190 2026-02-11T23:31:27.996+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1bb1e69e:::obj390:head local-lis/les=117/118 n=87 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.996+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.996+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:1cb33d95:::obj190:head, version: 111'190, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cb33d95:::obj190:head@111'190, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:27.996+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.996+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.996+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1c47bf8e:::obj558:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.996+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:1c47bf8e:::obj558:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 4.0 e 118/118 lb 4:1c47bf8e:::obj558:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:1bb1e69e:::obj390:head local-lis/les=117/118 n=87 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.996+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.996+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1bb1e69e:::obj390:head local-lis/les=117/118 n=87 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:1c47bf8e:::obj558:head) v3 2026-02-11T23:31:27.996+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1bb1e69e:::obj390:head local-lis/les=117/118 n=87 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:1c47bf8e:::obj558:head) 2026-02-11T23:31:27.996+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.996+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:1c47bf8e:::obj558:head) v3 finish latency 0.000013 2026-02-11T23:31:27.996+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1c47bf8e:::obj558:head local-lis/les=117/118 n=88 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.996+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.996+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1c47bf8e:::obj558:head local-lis/les=117/118 n=88 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.996+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.996+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.996+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.996+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.996+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x649ec00 con 0x507ac00 2026-02-11T23:31:27.996+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:1cb33d95:::obj190:head)]) -- 0x5f75080 con 0x507ac00 2026-02-11T23:31:27.996+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.997+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 216 ==== MOSDPGPush(4.0 118/117 [PushOp(4:1cd4c3d5:::obj142:head, version: 111'142, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cd4c3d5:::obj142:head@111'142, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e26680 con 0x507ac00 2026-02-11T23:31:27.997+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:1cd4c3d5:::obj142:head, version: 111'142, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cd4c3d5:::obj142:head@111'142, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 118 2026-02-11T23:31:27.997+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1cd4c3d5:::obj142:head, version: 111'142, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cd4c3d5:::obj142:head@111'142, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.997+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.997+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 217 ==== pg_backfill(progress 4.0 e 118/118 lb 4:1cb33d95:::obj190:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2c000 con 0x507ac00 2026-02-11T23:31:27.997+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1cd4c3d5:::obj142:head, version: 111'142, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cd4c3d5:::obj142:head@111'142, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.997+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:1cb33d95:::obj190:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:27.997+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1cb33d95:::obj190:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.997+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.997+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1cd4c3d5:::obj142:head, version: 111'142, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cd4c3d5:::obj142:head@111'142, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.997+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:1cd4c3d5:::obj142:head, version: 111'142, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cd4c3d5:::obj142:head@111'142, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 118/117 [PushOp(4:1cd4c3d5:::obj142:head, version: 111'142, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cd4c3d5:::obj142:head@111'142, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:1c47bf8e:::obj558:head local-lis/les=117/118 n=88 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.997+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.997+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.997+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1cb33d95:::obj190:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.997+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1c47bf8e:::obj558:head local-lis/les=117/118 n=88 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:1cd4c3d5:::obj142:head, version: 111'142, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cd4c3d5:::obj142:head@111'142, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.997+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1c47bf8e:::obj558:head local-lis/les=117/118 n=88 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:1cd4c3d5:::obj142:head@111'142, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.997+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1c47bf8e:::obj558:head local-lis/les=117/118 n=88 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:1cd4c3d5:::obj142:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.997+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1c47bf8e:::obj558:head local-lis/les=117/118 n=88 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:1cd4c3d5:::obj142:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.997+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1c47bf8e:::obj558:head local-lis/les=117/118 n=88 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:1cd4c3d5:::obj142:head 2026-02-11T23:31:27.997+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1c47bf8e:::obj558:head local-lis/les=117/118 n=88 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:1cd4c3d5:::obj142:head v 111'142 2026-02-11T23:31:27.997+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1c47bf8e:::obj558:head local-lis/les=117/118 n=88 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.997+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.997+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:1cd4c3d5:::obj142:head, version: 111'142, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cd4c3d5:::obj142:head@111'142, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:27.997+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.997+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.997+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1cb33d95:::obj190:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.997+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:1cb33d95:::obj190:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 4.0 e 118/118 lb 4:1cb33d95:::obj190:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:1c47bf8e:::obj558:head local-lis/les=117/118 n=88 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.997+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.997+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1c47bf8e:::obj558:head local-lis/les=117/118 n=88 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:1cb33d95:::obj190:head) v3 2026-02-11T23:31:27.997+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1c47bf8e:::obj558:head local-lis/les=117/118 n=88 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:1cb33d95:::obj190:head) 2026-02-11T23:31:27.997+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.997+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:1cb33d95:::obj190:head) v3 finish latency 0.000015 2026-02-11T23:31:27.997+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1cb33d95:::obj190:head local-lis/les=117/118 n=89 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.997+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.997+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.997+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1cb33d95:::obj190:head local-lis/les=117/118 n=89 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.997+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.997+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.997+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.997+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x649ef00 con 0x507ac00 2026-02-11T23:31:27.997+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:1cd4c3d5:::obj142:head)]) -- 0x600b680 con 0x507ac00 2026-02-11T23:31:27.997+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.997+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 218 ==== MOSDPGPush(4.0 118/117 [PushOp(4:1cf1ad59:::obj156:head, version: 111'156, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cf1ad59:::obj156:head@111'156, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559cb60 con 0x507ac00 2026-02-11T23:31:27.997+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:1cf1ad59:::obj156:head, version: 111'156, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cf1ad59:::obj156:head@111'156, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 118 2026-02-11T23:31:27.997+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1cf1ad59:::obj156:head, version: 111'156, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cf1ad59:::obj156:head@111'156, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.997+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.997+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1cf1ad59:::obj156:head, version: 111'156, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cf1ad59:::obj156:head@111'156, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.997+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 219 ==== pg_backfill(progress 4.0 e 118/118 lb 4:1cd4c3d5:::obj142:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e100 con 0x507ac00 2026-02-11T23:31:27.997+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:1cd4c3d5:::obj142:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:27.997+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.997+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1cd4c3d5:::obj142:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.997+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1cf1ad59:::obj156:head, version: 111'156, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cf1ad59:::obj156:head@111'156, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.997+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.997+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:1cf1ad59:::obj156:head, version: 111'156, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cf1ad59:::obj156:head@111'156, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 118/117 [PushOp(4:1cf1ad59:::obj156:head, version: 111'156, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cf1ad59:::obj156:head@111'156, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:1cb33d95:::obj190:head local-lis/les=117/118 n=89 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.997+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.997+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1cb33d95:::obj190:head local-lis/les=117/118 n=89 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:1cf1ad59:::obj156:head, version: 111'156, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cf1ad59:::obj156:head@111'156, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.997+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1cb33d95:::obj190:head local-lis/les=117/118 n=89 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:1cf1ad59:::obj156:head@111'156, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.997+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.997+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1cd4c3d5:::obj142:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.997+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1cb33d95:::obj190:head local-lis/les=117/118 n=89 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:1cf1ad59:::obj156:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.997+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1cb33d95:::obj190:head local-lis/les=117/118 n=89 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:1cf1ad59:::obj156:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.997+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1cb33d95:::obj190:head local-lis/les=117/118 n=89 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:1cf1ad59:::obj156:head 2026-02-11T23:31:27.997+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1cb33d95:::obj190:head local-lis/les=117/118 n=89 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:1cf1ad59:::obj156:head v 111'156 2026-02-11T23:31:27.997+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1cb33d95:::obj190:head local-lis/les=117/118 n=89 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.997+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.997+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:1cf1ad59:::obj156:head, version: 111'156, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cf1ad59:::obj156:head@111'156, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:27.997+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.997+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.997+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1cd4c3d5:::obj142:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.997+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:1cd4c3d5:::obj142:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 118/118 lb 4:1cd4c3d5:::obj142:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:1cb33d95:::obj190:head local-lis/les=117/118 n=89 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.997+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.997+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1cb33d95:::obj190:head local-lis/les=117/118 n=89 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:1cd4c3d5:::obj142:head) v3 2026-02-11T23:31:27.997+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1cb33d95:::obj190:head local-lis/les=117/118 n=89 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:1cd4c3d5:::obj142:head) 2026-02-11T23:31:27.997+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.997+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:1cd4c3d5:::obj142:head) v3 finish latency 0.000015 2026-02-11T23:31:27.997+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1cd4c3d5:::obj142:head local-lis/les=117/118 n=90 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.997+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.997+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.997+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1cd4c3d5:::obj142:head local-lis/les=117/118 n=90 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.998+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.998+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.998+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.998+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x5f75b00 con 0x507ac00 2026-02-11T23:31:27.998+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:1cf1ad59:::obj156:head)]) -- 0x5f2c900 con 0x507ac00 2026-02-11T23:31:27.998+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.998+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 220 ==== MOSDPGPush(4.0 118/117 [PushOp(4:1cff0951:::obj258:head, version: 111'258, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cff0951:::obj258:head@111'258, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f0b60 con 0x507ac00 2026-02-11T23:31:27.998+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:1cff0951:::obj258:head, version: 111'258, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cff0951:::obj258:head@111'258, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000016 epoch 118 2026-02-11T23:31:27.998+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1cff0951:::obj258:head, version: 111'258, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cff0951:::obj258:head@111'258, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.998+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.998+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 221 ==== pg_backfill(progress 4.0 e 118/118 lb 4:1cf1ad59:::obj156:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e680 con 0x507ac00 2026-02-11T23:31:27.998+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1cff0951:::obj258:head, version: 111'258, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cff0951:::obj258:head@111'258, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.998+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:1cf1ad59:::obj156:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 118 2026-02-11T23:31:27.998+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.998+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1cf1ad59:::obj156:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.998+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1cff0951:::obj258:head, version: 111'258, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cff0951:::obj258:head@111'258, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.998+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.998+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:1cff0951:::obj258:head, version: 111'258, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cff0951:::obj258:head@111'258, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000040 MOSDPGPush(4.0 118/117 [PushOp(4:1cff0951:::obj258:head, version: 111'258, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cff0951:::obj258:head@111'258, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:1cd4c3d5:::obj142:head local-lis/les=117/118 n=90 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.998+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1cf1ad59:::obj156:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.998+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.998+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1cd4c3d5:::obj142:head local-lis/les=117/118 n=90 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:1cff0951:::obj258:head, version: 111'258, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cff0951:::obj258:head@111'258, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.998+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1cd4c3d5:::obj142:head local-lis/les=117/118 n=90 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:1cff0951:::obj258:head@111'258, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.998+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1cd4c3d5:::obj142:head local-lis/les=117/118 n=90 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:1cff0951:::obj258:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.998+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1cd4c3d5:::obj142:head local-lis/les=117/118 n=90 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:1cff0951:::obj258:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.998+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1cd4c3d5:::obj142:head local-lis/les=117/118 n=90 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:1cff0951:::obj258:head 2026-02-11T23:31:27.998+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1cd4c3d5:::obj142:head local-lis/les=117/118 n=90 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:1cff0951:::obj258:head v 111'258 2026-02-11T23:31:27.998+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1cd4c3d5:::obj142:head local-lis/les=117/118 n=90 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.998+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.998+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:1cff0951:::obj258:head, version: 111'258, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cff0951:::obj258:head@111'258, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:27.998+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.998+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.998+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1cf1ad59:::obj156:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.998+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:1cf1ad59:::obj156:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 4.0 e 118/118 lb 4:1cf1ad59:::obj156:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:1cd4c3d5:::obj142:head local-lis/les=117/118 n=90 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.998+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.998+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1cd4c3d5:::obj142:head local-lis/les=117/118 n=90 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:1cf1ad59:::obj156:head) v3 2026-02-11T23:31:27.998+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1cd4c3d5:::obj142:head local-lis/les=117/118 n=90 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:1cf1ad59:::obj156:head) 2026-02-11T23:31:27.998+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.998+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:1cf1ad59:::obj156:head) v3 finish latency 0.000018 2026-02-11T23:31:27.998+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1cf1ad59:::obj156:head local-lis/les=117/118 n=91 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.998+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.998+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.998+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1cf1ad59:::obj156:head local-lis/les=117/118 n=91 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.998+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.998+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.998+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.998+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x649ea80 con 0x507ac00 2026-02-11T23:31:27.998+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:1cff0951:::obj258:head)]) -- 0x5f75380 con 0x507ac00 2026-02-11T23:31:27.998+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.999+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 222 ==== MOSDPGPush(4.0 118/117 [PushOp(4:1d2e941a:::obj186:head, version: 111'186, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1d2e941a:::obj186:head@111'186, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f1ba0 con 0x507ac00 2026-02-11T23:31:27.999+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:1d2e941a:::obj186:head, version: 111'186, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1d2e941a:::obj186:head@111'186, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 118 2026-02-11T23:31:27.999+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1d2e941a:::obj186:head, version: 111'186, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1d2e941a:::obj186:head@111'186, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.999+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.999+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 223 ==== pg_backfill(progress 4.0 e 118/118 lb 4:1cff0951:::obj258:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2d080 con 0x507ac00 2026-02-11T23:31:27.999+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1d2e941a:::obj186:head, version: 111'186, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1d2e941a:::obj186:head@111'186, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.999+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:1cff0951:::obj258:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:27.999+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.999+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1cff0951:::obj258:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.999+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1d2e941a:::obj186:head, version: 111'186, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1d2e941a:::obj186:head@111'186, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.999+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.999+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:1d2e941a:::obj186:head, version: 111'186, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1d2e941a:::obj186:head@111'186, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(4.0 118/117 [PushOp(4:1d2e941a:::obj186:head, version: 111'186, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1d2e941a:::obj186:head@111'186, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:1cf1ad59:::obj156:head local-lis/les=117/118 n=91 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.999+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.999+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1cf1ad59:::obj156:head local-lis/les=117/118 n=91 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:1d2e941a:::obj186:head, version: 111'186, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1d2e941a:::obj186:head@111'186, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.999+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1cf1ad59:::obj156:head local-lis/les=117/118 n=91 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:1d2e941a:::obj186:head@111'186, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.999+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.999+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1cff0951:::obj258:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.999+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1cf1ad59:::obj156:head local-lis/les=117/118 n=91 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:1d2e941a:::obj186:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.999+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1cf1ad59:::obj156:head local-lis/les=117/118 n=91 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:1d2e941a:::obj186:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.999+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1cf1ad59:::obj156:head local-lis/les=117/118 n=91 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:1d2e941a:::obj186:head 2026-02-11T23:31:27.999+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1cf1ad59:::obj156:head local-lis/les=117/118 n=91 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:1d2e941a:::obj186:head v 111'186 2026-02-11T23:31:27.999+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1cf1ad59:::obj156:head local-lis/les=117/118 n=91 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.999+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.999+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:1d2e941a:::obj186:head, version: 111'186, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1d2e941a:::obj186:head@111'186, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000062 2026-02-11T23:31:27.999+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.999+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.999+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1cff0951:::obj258:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.999+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:1cff0951:::obj258:head) v3 prio 15 cost 0 latency 0.000080 pg_backfill(progress 4.0 e 118/118 lb 4:1cff0951:::obj258:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:1cf1ad59:::obj156:head local-lis/les=117/118 n=91 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.999+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.999+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1cf1ad59:::obj156:head local-lis/les=117/118 n=91 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:1cff0951:::obj258:head) v3 2026-02-11T23:31:27.999+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1cf1ad59:::obj156:head local-lis/les=117/118 n=91 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:1cff0951:::obj258:head) 2026-02-11T23:31:27.999+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.999+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:1cff0951:::obj258:head) v3 finish latency 0.000016 2026-02-11T23:31:27.999+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1cff0951:::obj258:head local-lis/les=117/118 n=92 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.999+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.999+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.999+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1cff0951:::obj258:head local-lis/les=117/118 n=92 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:27.999+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.999+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.999+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.999+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63c8000 con 0x507ac00 2026-02-11T23:31:27.999+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:1d2e941a:::obj186:head)]) -- 0x5f74180 con 0x507ac00 2026-02-11T23:31:27.999+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.999+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 224 ==== MOSDPGPush(4.0 118/117 [PushOp(4:1dafe4f0:::obj47:head, version: 111'47, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1dafe4f0:::obj47:head@111'47, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x555cea0 con 0x507ac00 2026-02-11T23:31:27.999+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:1dafe4f0:::obj47:head, version: 111'47, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1dafe4f0:::obj47:head@111'47, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 118 2026-02-11T23:31:27.999+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1dafe4f0:::obj47:head, version: 111'47, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1dafe4f0:::obj47:head@111'47, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:27.999+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.999+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 225 ==== pg_backfill(progress 4.0 e 118/118 lb 4:1d2e941a:::obj186:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2f700 con 0x507ac00 2026-02-11T23:31:27.999+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:1d2e941a:::obj186:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:27.999+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1dafe4f0:::obj47:head, version: 111'47, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1dafe4f0:::obj47:head@111'47, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:27.999+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1d2e941a:::obj186:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:27.999+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.999+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1dafe4f0:::obj47:head, version: 111'47, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1dafe4f0:::obj47:head@111'47, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:27.999+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:1dafe4f0:::obj47:head, version: 111'47, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1dafe4f0:::obj47:head@111'47, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(4.0 118/117 [PushOp(4:1dafe4f0:::obj47:head, version: 111'47, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1dafe4f0:::obj47:head@111'47, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:1cff0951:::obj258:head local-lis/les=117/118 n=92 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.999+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.999+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:27.999+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1d2e941a:::obj186:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:27.999+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1cff0951:::obj258:head local-lis/les=117/118 n=92 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:1dafe4f0:::obj47:head, version: 111'47, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1dafe4f0:::obj47:head@111'47, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:27.999+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1cff0951:::obj258:head local-lis/les=117/118 n=92 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:1dafe4f0:::obj47:head@111'47, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:27.999+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1cff0951:::obj258:head local-lis/les=117/118 n=92 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:1dafe4f0:::obj47:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:27.999+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1cff0951:::obj258:head local-lis/les=117/118 n=92 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:1dafe4f0:::obj47:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:27.999+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1cff0951:::obj258:head local-lis/les=117/118 n=92 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:1dafe4f0:::obj47:head 2026-02-11T23:31:27.999+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1cff0951:::obj258:head local-lis/les=117/118 n=92 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:1dafe4f0:::obj47:head v 111'47 2026-02-11T23:31:27.999+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1cff0951:::obj258:head local-lis/les=117/118 n=92 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:27.999+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:27.999+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:1dafe4f0:::obj47:head, version: 111'47, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1dafe4f0:::obj47:head@111'47, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:27.999+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:27.999+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:27.999+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1d2e941a:::obj186:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:27.999+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:1d2e941a:::obj186:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 4.0 e 118/118 lb 4:1d2e941a:::obj186:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:1cff0951:::obj258:head local-lis/les=117/118 n=92 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:27.999+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.999+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1cff0951:::obj258:head local-lis/les=117/118 n=92 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:1d2e941a:::obj186:head) v3 2026-02-11T23:31:27.999+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1cff0951:::obj258:head local-lis/les=117/118 n=92 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:1d2e941a:::obj186:head) 2026-02-11T23:31:27.999+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:27.999+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:1d2e941a:::obj186:head) v3 finish latency 0.000020 2026-02-11T23:31:27.999+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1d2e941a:::obj186:head local-lis/les=117/118 n=93 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:27.999+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.000+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1d2e941a:::obj186:head local-lis/les=117/118 n=93 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.000+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.000+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:28.000+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:28.000+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:28.000+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63c8180 con 0x507ac00 2026-02-11T23:31:28.000+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:1dafe4f0:::obj47:head)]) -- 0x5f1d200 con 0x507ac00 2026-02-11T23:31:28.000+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.000+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 226 ==== MOSDPGPush(4.0 118/117 [PushOp(4:1e53d0d0:::obj151:head, version: 111'151, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1e53d0d0:::obj151:head@111'151, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56cab60 con 0x507ac00 2026-02-11T23:31:28.000+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:1e53d0d0:::obj151:head, version: 111'151, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1e53d0d0:::obj151:head@111'151, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 118 2026-02-11T23:31:28.000+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1e53d0d0:::obj151:head, version: 111'151, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1e53d0d0:::obj151:head@111'151, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:28.000+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.000+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 227 ==== pg_backfill(progress 4.0 e 118/118 lb 4:1dafe4f0:::obj47:head) ==== 1061+0+0 (unknown 0 0 0) 0x5f11600 con 0x507ac00 2026-02-11T23:31:28.000+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1e53d0d0:::obj151:head, version: 111'151, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1e53d0d0:::obj151:head@111'151, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:28.000+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:1dafe4f0:::obj47:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 118 2026-02-11T23:31:28.000+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1dafe4f0:::obj47:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:28.000+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.000+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1e53d0d0:::obj151:head, version: 111'151, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1e53d0d0:::obj151:head@111'151, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:28.000+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:1e53d0d0:::obj151:head, version: 111'151, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1e53d0d0:::obj151:head@111'151, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(4.0 118/117 [PushOp(4:1e53d0d0:::obj151:head, version: 111'151, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1e53d0d0:::obj151:head@111'151, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:1d2e941a:::obj186:head local-lis/les=117/118 n=93 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.000+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.000+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.000+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1dafe4f0:::obj47:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:28.000+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1d2e941a:::obj186:head local-lis/les=117/118 n=93 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:1e53d0d0:::obj151:head, version: 111'151, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1e53d0d0:::obj151:head@111'151, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.000+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1d2e941a:::obj186:head local-lis/les=117/118 n=93 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:1e53d0d0:::obj151:head@111'151, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.000+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1d2e941a:::obj186:head local-lis/les=117/118 n=93 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:1e53d0d0:::obj151:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.000+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1d2e941a:::obj186:head local-lis/les=117/118 n=93 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:1e53d0d0:::obj151:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.000+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1d2e941a:::obj186:head local-lis/les=117/118 n=93 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:1e53d0d0:::obj151:head 2026-02-11T23:31:28.000+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1d2e941a:::obj186:head local-lis/les=117/118 n=93 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:1e53d0d0:::obj151:head v 111'151 2026-02-11T23:31:28.000+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1d2e941a:::obj186:head local-lis/les=117/118 n=93 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.000+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.000+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:1e53d0d0:::obj151:head, version: 111'151, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1e53d0d0:::obj151:head@111'151, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:28.000+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.000+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.000+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1dafe4f0:::obj47:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:28.000+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:1dafe4f0:::obj47:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 4.0 e 118/118 lb 4:1dafe4f0:::obj47:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:1d2e941a:::obj186:head local-lis/les=117/118 n=93 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.000+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.000+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1d2e941a:::obj186:head local-lis/les=117/118 n=93 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:1dafe4f0:::obj47:head) v3 2026-02-11T23:31:28.000+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1d2e941a:::obj186:head local-lis/les=117/118 n=93 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:1dafe4f0:::obj47:head) 2026-02-11T23:31:28.000+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.000+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:1dafe4f0:::obj47:head) v3 finish latency 0.000015 2026-02-11T23:31:28.000+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1dafe4f0:::obj47:head local-lis/les=117/118 n=94 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.000+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.000+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.000+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1dafe4f0:::obj47:head local-lis/les=117/118 n=94 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.000+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:28.000+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:28.000+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:28.000+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63c8300 con 0x507ac00 2026-02-11T23:31:28.000+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:1e53d0d0:::obj151:head)]) -- 0x649f500 con 0x507ac00 2026-02-11T23:31:28.000+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.001+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 228 ==== MOSDPGPush(4.0 118/117 [PushOp(4:1efaad2b:::obj120:head, version: 111'120, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1efaad2b:::obj120:head@111'120, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2680 con 0x507ac00 2026-02-11T23:31:28.001+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:1efaad2b:::obj120:head, version: 111'120, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1efaad2b:::obj120:head@111'120, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 118 2026-02-11T23:31:28.001+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1efaad2b:::obj120:head, version: 111'120, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1efaad2b:::obj120:head@111'120, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:28.001+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.001+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 229 ==== pg_backfill(progress 4.0 e 118/118 lb 4:1e53d0d0:::obj151:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f11080 con 0x507ac00 2026-02-11T23:31:28.001+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:1e53d0d0:::obj151:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:28.001+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1efaad2b:::obj120:head, version: 111'120, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1efaad2b:::obj120:head@111'120, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:28.001+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1e53d0d0:::obj151:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:28.001+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.001+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:1efaad2b:::obj120:head, version: 111'120, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1efaad2b:::obj120:head@111'120, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:28.001+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:1efaad2b:::obj120:head, version: 111'120, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1efaad2b:::obj120:head@111'120, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 118/117 [PushOp(4:1efaad2b:::obj120:head, version: 111'120, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1efaad2b:::obj120:head@111'120, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:1dafe4f0:::obj47:head local-lis/les=117/118 n=94 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.001+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.001+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.001+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1e53d0d0:::obj151:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:28.001+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1dafe4f0:::obj47:head local-lis/les=117/118 n=94 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:1efaad2b:::obj120:head, version: 111'120, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1efaad2b:::obj120:head@111'120, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.001+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1dafe4f0:::obj47:head local-lis/les=117/118 n=94 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:1efaad2b:::obj120:head@111'120, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.001+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1dafe4f0:::obj47:head local-lis/les=117/118 n=94 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:1efaad2b:::obj120:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.001+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1dafe4f0:::obj47:head local-lis/les=117/118 n=94 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:1efaad2b:::obj120:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.001+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1dafe4f0:::obj47:head local-lis/les=117/118 n=94 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:1efaad2b:::obj120:head 2026-02-11T23:31:28.001+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1dafe4f0:::obj47:head local-lis/les=117/118 n=94 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:1efaad2b:::obj120:head v 111'120 2026-02-11T23:31:28.001+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1dafe4f0:::obj47:head local-lis/les=117/118 n=94 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.001+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.001+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:1efaad2b:::obj120:head, version: 111'120, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1efaad2b:::obj120:head@111'120, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:28.001+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.001+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.001+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1e53d0d0:::obj151:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:28.001+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:1e53d0d0:::obj151:head) v3 prio 15 cost 0 latency 0.000079 pg_backfill(progress 4.0 e 118/118 lb 4:1e53d0d0:::obj151:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:1dafe4f0:::obj47:head local-lis/les=117/118 n=94 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.001+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.001+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1dafe4f0:::obj47:head local-lis/les=117/118 n=94 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:1e53d0d0:::obj151:head) v3 2026-02-11T23:31:28.001+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1dafe4f0:::obj47:head local-lis/les=117/118 n=94 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:1e53d0d0:::obj151:head) 2026-02-11T23:31:28.001+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.001+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:1e53d0d0:::obj151:head) v3 finish latency 0.000016 2026-02-11T23:31:28.001+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1e53d0d0:::obj151:head local-lis/les=117/118 n=95 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.001+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.001+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.001+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1e53d0d0:::obj151:head local-lis/les=117/118 n=95 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.001+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:28.001+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:28.001+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:28.001+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63c8480 con 0x507ac00 2026-02-11T23:31:28.001+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:1efaad2b:::obj120:head)]) -- 0x5f2d800 con 0x507ac00 2026-02-11T23:31:28.001+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.001+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 230 ==== MOSDPGPush(4.0 118/117 [PushOp(4:20b87b8b:::obj369:head, version: 111'369, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:20b87b8b:::obj369:head@111'369, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db0b60 con 0x507ac00 2026-02-11T23:31:28.001+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:20b87b8b:::obj369:head, version: 111'369, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:20b87b8b:::obj369:head@111'369, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 118 2026-02-11T23:31:28.001+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:20b87b8b:::obj369:head, version: 111'369, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:20b87b8b:::obj369:head@111'369, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:28.001+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.001+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 231 ==== pg_backfill(progress 4.0 e 118/118 lb 4:1efaad2b:::obj120:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f10b00 con 0x507ac00 2026-02-11T23:31:28.001+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:1efaad2b:::obj120:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:28.001+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:20b87b8b:::obj369:head, version: 111'369, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:20b87b8b:::obj369:head@111'369, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:28.001+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1efaad2b:::obj120:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:28.001+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.001+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1efaad2b:::obj120:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:28.001+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.001+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:20b87b8b:::obj369:head, version: 111'369, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:20b87b8b:::obj369:head@111'369, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:28.001+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:20b87b8b:::obj369:head, version: 111'369, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:20b87b8b:::obj369:head@111'369, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000049 MOSDPGPush(4.0 118/117 [PushOp(4:20b87b8b:::obj369:head, version: 111'369, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:20b87b8b:::obj369:head@111'369, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:1e53d0d0:::obj151:head local-lis/les=117/118 n=95 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.001+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.001+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1e53d0d0:::obj151:head local-lis/les=117/118 n=95 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:20b87b8b:::obj369:head, version: 111'369, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:20b87b8b:::obj369:head@111'369, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.001+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1e53d0d0:::obj151:head local-lis/les=117/118 n=95 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:20b87b8b:::obj369:head@111'369, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.001+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1e53d0d0:::obj151:head local-lis/les=117/118 n=95 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:20b87b8b:::obj369:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.001+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1e53d0d0:::obj151:head local-lis/les=117/118 n=95 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:20b87b8b:::obj369:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.001+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1e53d0d0:::obj151:head local-lis/les=117/118 n=95 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:20b87b8b:::obj369:head 2026-02-11T23:31:28.001+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1e53d0d0:::obj151:head local-lis/les=117/118 n=95 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:20b87b8b:::obj369:head v 111'369 2026-02-11T23:31:28.001+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1e53d0d0:::obj151:head local-lis/les=117/118 n=95 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.001+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.001+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:20b87b8b:::obj369:head, version: 111'369, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:20b87b8b:::obj369:head@111'369, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:28.001+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.001+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.001+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:1efaad2b:::obj120:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:28.001+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:1efaad2b:::obj120:head) v3 prio 15 cost 0 latency 0.000088 pg_backfill(progress 4.0 e 118/118 lb 4:1efaad2b:::obj120:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:1e53d0d0:::obj151:head local-lis/les=117/118 n=95 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.001+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.001+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1e53d0d0:::obj151:head local-lis/les=117/118 n=95 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:1efaad2b:::obj120:head) v3 2026-02-11T23:31:28.001+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1e53d0d0:::obj151:head local-lis/les=117/118 n=95 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:1efaad2b:::obj120:head) 2026-02-11T23:31:28.001+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.001+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:1efaad2b:::obj120:head) v3 finish latency 0.000016 2026-02-11T23:31:28.001+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1efaad2b:::obj120:head local-lis/les=117/118 n=96 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.001+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.002+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.002+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1efaad2b:::obj120:head local-lis/les=117/118 n=96 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.002+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:28.002+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:28.002+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:28.002+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63c8600 con 0x507ac00 2026-02-11T23:31:28.002+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:20b87b8b:::obj369:head)]) -- 0x649f680 con 0x507ac00 2026-02-11T23:31:28.002+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.002+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 232 ==== MOSDPGPush(4.0 118/117 [PushOp(4:212e6483:::obj209:head, version: 111'209, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:212e6483:::obj209:head@111'209, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a3ba0 con 0x507ac00 2026-02-11T23:31:28.002+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:212e6483:::obj209:head, version: 111'209, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:212e6483:::obj209:head@111'209, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 118 2026-02-11T23:31:28.002+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:212e6483:::obj209:head, version: 111'209, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:212e6483:::obj209:head@111'209, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:28.002+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.002+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 233 ==== pg_backfill(progress 4.0 e 118/118 lb 4:20b87b8b:::obj369:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f10580 con 0x507ac00 2026-02-11T23:31:28.002+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:212e6483:::obj209:head, version: 111'209, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:212e6483:::obj209:head@111'209, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:28.002+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:20b87b8b:::obj369:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:28.002+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:20b87b8b:::obj369:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:28.002+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.002+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:20b87b8b:::obj369:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:28.002+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.002+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:212e6483:::obj209:head, version: 111'209, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:212e6483:::obj209:head@111'209, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:28.002+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:212e6483:::obj209:head, version: 111'209, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:212e6483:::obj209:head@111'209, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000047 MOSDPGPush(4.0 118/117 [PushOp(4:212e6483:::obj209:head, version: 111'209, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:212e6483:::obj209:head@111'209, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:1efaad2b:::obj120:head local-lis/les=117/118 n=96 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.002+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.002+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1efaad2b:::obj120:head local-lis/les=117/118 n=96 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:212e6483:::obj209:head, version: 111'209, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:212e6483:::obj209:head@111'209, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.002+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1efaad2b:::obj120:head local-lis/les=117/118 n=96 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:212e6483:::obj209:head@111'209, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.002+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1efaad2b:::obj120:head local-lis/les=117/118 n=96 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:212e6483:::obj209:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.002+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1efaad2b:::obj120:head local-lis/les=117/118 n=96 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:212e6483:::obj209:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.002+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1efaad2b:::obj120:head local-lis/les=117/118 n=96 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:212e6483:::obj209:head 2026-02-11T23:31:28.002+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1efaad2b:::obj120:head local-lis/les=117/118 n=96 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:212e6483:::obj209:head v 111'209 2026-02-11T23:31:28.002+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1efaad2b:::obj120:head local-lis/les=117/118 n=96 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.002+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.002+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:212e6483:::obj209:head, version: 111'209, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:212e6483:::obj209:head@111'209, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:28.002+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.002+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.002+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:20b87b8b:::obj369:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:28.002+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:20b87b8b:::obj369:head) v3 prio 15 cost 0 latency 0.000091 pg_backfill(progress 4.0 e 118/118 lb 4:20b87b8b:::obj369:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:1efaad2b:::obj120:head local-lis/les=117/118 n=96 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.002+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.002+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1efaad2b:::obj120:head local-lis/les=117/118 n=96 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:20b87b8b:::obj369:head) v3 2026-02-11T23:31:28.002+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:1efaad2b:::obj120:head local-lis/les=117/118 n=96 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:20b87b8b:::obj369:head) 2026-02-11T23:31:28.002+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.002+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:20b87b8b:::obj369:head) v3 finish latency 0.000016 2026-02-11T23:31:28.002+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:20b87b8b:::obj369:head local-lis/les=117/118 n=97 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.002+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.002+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:20b87b8b:::obj369:head local-lis/les=117/118 n=97 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.002+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:28.002+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:28.002+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:28.002+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.002+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63c8780 con 0x507ac00 2026-02-11T23:31:28.002+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:212e6483:::obj209:head)]) -- 0x5f1c300 con 0x507ac00 2026-02-11T23:31:28.002+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.003+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 234 ==== MOSDPGPush(4.0 118/117 [PushOp(4:2222704f:::obj161:head, version: 111'161, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2222704f:::obj161:head@111'161, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2820 con 0x507ac00 2026-02-11T23:31:28.003+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:2222704f:::obj161:head, version: 111'161, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2222704f:::obj161:head@111'161, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 118 2026-02-11T23:31:28.003+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:2222704f:::obj161:head, version: 111'161, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2222704f:::obj161:head@111'161, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:28.003+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.003+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 235 ==== pg_backfill(progress 4.0 e 118/118 lb 4:212e6483:::obj209:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f10000 con 0x507ac00 2026-02-11T23:31:28.003+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:212e6483:::obj209:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 118 2026-02-11T23:31:28.003+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:2222704f:::obj161:head, version: 111'161, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2222704f:::obj161:head@111'161, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:28.003+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:212e6483:::obj209:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:28.003+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.003+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:2222704f:::obj161:head, version: 111'161, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2222704f:::obj161:head@111'161, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:28.003+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:2222704f:::obj161:head, version: 111'161, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2222704f:::obj161:head@111'161, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(4.0 118/117 [PushOp(4:2222704f:::obj161:head, version: 111'161, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2222704f:::obj161:head@111'161, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:20b87b8b:::obj369:head local-lis/les=117/118 n=97 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.003+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.003+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.003+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:212e6483:::obj209:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:28.003+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:20b87b8b:::obj369:head local-lis/les=117/118 n=97 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:2222704f:::obj161:head, version: 111'161, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2222704f:::obj161:head@111'161, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.003+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:20b87b8b:::obj369:head local-lis/les=117/118 n=97 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:2222704f:::obj161:head@111'161, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.003+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:20b87b8b:::obj369:head local-lis/les=117/118 n=97 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:2222704f:::obj161:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.003+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:20b87b8b:::obj369:head local-lis/les=117/118 n=97 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:2222704f:::obj161:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.003+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:20b87b8b:::obj369:head local-lis/les=117/118 n=97 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:2222704f:::obj161:head 2026-02-11T23:31:28.003+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:20b87b8b:::obj369:head local-lis/les=117/118 n=97 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:2222704f:::obj161:head v 111'161 2026-02-11T23:31:28.003+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:20b87b8b:::obj369:head local-lis/les=117/118 n=97 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.003+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.003+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:2222704f:::obj161:head, version: 111'161, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2222704f:::obj161:head@111'161, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:28.003+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.003+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.003+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:212e6483:::obj209:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:28.003+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:212e6483:::obj209:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 4.0 e 118/118 lb 4:212e6483:::obj209:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:20b87b8b:::obj369:head local-lis/les=117/118 n=97 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.003+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.003+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:20b87b8b:::obj369:head local-lis/les=117/118 n=97 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:212e6483:::obj209:head) v3 2026-02-11T23:31:28.003+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:20b87b8b:::obj369:head local-lis/les=117/118 n=97 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:212e6483:::obj209:head) 2026-02-11T23:31:28.003+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.003+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:212e6483:::obj209:head) v3 finish latency 0.000016 2026-02-11T23:31:28.003+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:212e6483:::obj209:head local-lis/les=117/118 n=98 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.003+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.003+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.003+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:212e6483:::obj209:head local-lis/les=117/118 n=98 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.003+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:28.003+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:28.003+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:28.003+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63c8900 con 0x507ac00 2026-02-11T23:31:28.003+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:2222704f:::obj161:head)]) -- 0x649fb00 con 0x507ac00 2026-02-11T23:31:28.003+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.003+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 236 ==== MOSDPGPush(4.0 118/117 [PushOp(4:2260a46a:::obj167:head, version: 111'167, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2260a46a:::obj167:head@111'167, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x555d040 con 0x507ac00 2026-02-11T23:31:28.003+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:2260a46a:::obj167:head, version: 111'167, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2260a46a:::obj167:head@111'167, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 118 2026-02-11T23:31:28.003+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:2260a46a:::obj167:head, version: 111'167, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2260a46a:::obj167:head@111'167, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:28.003+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.003+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:2260a46a:::obj167:head, version: 111'167, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2260a46a:::obj167:head@111'167, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:28.003+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 237 ==== pg_backfill(progress 4.0 e 118/118 lb 4:2222704f:::obj161:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f13700 con 0x507ac00 2026-02-11T23:31:28.003+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:2222704f:::obj161:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:28.003+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.003+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:2222704f:::obj161:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:28.003+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:2260a46a:::obj167:head, version: 111'167, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2260a46a:::obj167:head@111'167, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:28.003+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.003+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:2260a46a:::obj167:head, version: 111'167, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2260a46a:::obj167:head@111'167, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 118/117 [PushOp(4:2260a46a:::obj167:head, version: 111'167, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2260a46a:::obj167:head@111'167, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:212e6483:::obj209:head local-lis/les=117/118 n=98 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.003+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.003+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:212e6483:::obj209:head local-lis/les=117/118 n=98 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:2260a46a:::obj167:head, version: 111'167, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2260a46a:::obj167:head@111'167, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.003+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.003+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:212e6483:::obj209:head local-lis/les=117/118 n=98 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:2260a46a:::obj167:head@111'167, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.003+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:2222704f:::obj161:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:28.003+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:212e6483:::obj209:head local-lis/les=117/118 n=98 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:2260a46a:::obj167:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.003+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:212e6483:::obj209:head local-lis/les=117/118 n=98 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:2260a46a:::obj167:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.003+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:212e6483:::obj209:head local-lis/les=117/118 n=98 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:2260a46a:::obj167:head 2026-02-11T23:31:28.003+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:212e6483:::obj209:head local-lis/les=117/118 n=98 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:2260a46a:::obj167:head v 111'167 2026-02-11T23:31:28.003+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:212e6483:::obj209:head local-lis/les=117/118 n=98 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.003+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.003+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:2260a46a:::obj167:head, version: 111'167, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2260a46a:::obj167:head@111'167, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:28.003+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.003+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.003+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:2222704f:::obj161:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:28.003+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:2222704f:::obj161:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 4.0 e 118/118 lb 4:2222704f:::obj161:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:212e6483:::obj209:head local-lis/les=117/118 n=98 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.003+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.003+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:212e6483:::obj209:head local-lis/les=117/118 n=98 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:2222704f:::obj161:head) v3 2026-02-11T23:31:28.003+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:212e6483:::obj209:head local-lis/les=117/118 n=98 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:2222704f:::obj161:head) 2026-02-11T23:31:28.003+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.003+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:2222704f:::obj161:head) v3 finish latency 0.000014 2026-02-11T23:31:28.003+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2222704f:::obj161:head local-lis/les=117/118 n=99 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.003+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.004+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.004+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2222704f:::obj161:head local-lis/les=117/118 n=99 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.004+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:28.004+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:28.004+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:28.004+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63c8a80 con 0x507ac00 2026-02-11T23:31:28.004+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:2260a46a:::obj167:head)]) -- 0x649f800 con 0x507ac00 2026-02-11T23:31:28.004+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.004+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 238 ==== MOSDPGPush(4.0 118/117 [PushOp(4:233a42c1:::obj6:head, version: 111'6, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:233a42c1:::obj6:head@111'6, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 4999+0+0 (unknown 0 0 0) 0x3815ba0 con 0x507ac00 2026-02-11T23:31:28.004+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:233a42c1:::obj6:head, version: 111'6, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:233a42c1:::obj6:head@111'6, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 118 2026-02-11T23:31:28.004+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:233a42c1:::obj6:head, version: 111'6, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:233a42c1:::obj6:head@111'6, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:28.004+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.004+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 239 ==== pg_backfill(progress 4.0 e 118/118 lb 4:2260a46a:::obj167:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f12c00 con 0x507ac00 2026-02-11T23:31:28.004+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:2260a46a:::obj167:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:28.004+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:233a42c1:::obj6:head, version: 111'6, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:233a42c1:::obj6:head@111'6, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:28.004+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:2260a46a:::obj167:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:28.004+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.004+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:233a42c1:::obj6:head, version: 111'6, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:233a42c1:::obj6:head@111'6, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:28.004+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:233a42c1:::obj6:head, version: 111'6, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:233a42c1:::obj6:head@111'6, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 118/117 [PushOp(4:233a42c1:::obj6:head, version: 111'6, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:233a42c1:::obj6:head@111'6, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:2222704f:::obj161:head local-lis/les=117/118 n=99 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.004+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.004+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.004+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:2260a46a:::obj167:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:28.004+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2222704f:::obj161:head local-lis/les=117/118 n=99 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:233a42c1:::obj6:head, version: 111'6, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:233a42c1:::obj6:head@111'6, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.004+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2222704f:::obj161:head local-lis/les=117/118 n=99 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:233a42c1:::obj6:head@111'6, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.004+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2222704f:::obj161:head local-lis/les=117/118 n=99 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:233a42c1:::obj6:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.004+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2222704f:::obj161:head local-lis/les=117/118 n=99 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:233a42c1:::obj6:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.004+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2222704f:::obj161:head local-lis/les=117/118 n=99 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:233a42c1:::obj6:head 2026-02-11T23:31:28.004+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2222704f:::obj161:head local-lis/les=117/118 n=99 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:233a42c1:::obj6:head v 111'6 2026-02-11T23:31:28.004+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2222704f:::obj161:head local-lis/les=117/118 n=99 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.004+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.004+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:233a42c1:::obj6:head, version: 111'6, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:233a42c1:::obj6:head@111'6, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:28.004+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.004+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.004+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:2260a46a:::obj167:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:28.004+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:2260a46a:::obj167:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 4.0 e 118/118 lb 4:2260a46a:::obj167:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:2222704f:::obj161:head local-lis/les=117/118 n=99 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.004+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.004+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2222704f:::obj161:head local-lis/les=117/118 n=99 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:2260a46a:::obj167:head) v3 2026-02-11T23:31:28.004+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2222704f:::obj161:head local-lis/les=117/118 n=99 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:2260a46a:::obj167:head) 2026-02-11T23:31:28.004+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.004+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:2260a46a:::obj167:head) v3 finish latency 0.000015 2026-02-11T23:31:28.004+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2260a46a:::obj167:head local-lis/les=117/118 n=100 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.004+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.004+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.004+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2260a46a:::obj167:head local-lis/les=117/118 n=100 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.004+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:28.004+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:28.004+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:28.004+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63c8c00 con 0x507ac00 2026-02-11T23:31:28.004+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:233a42c1:::obj6:head)]) -- 0x649fc80 con 0x507ac00 2026-02-11T23:31:28.004+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.005+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 240 ==== MOSDPGPush(4.0 118/117 [PushOp(4:239f544e:::obj128:head, version: 111'128, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:239f544e:::obj128:head@111'128, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a21a0 con 0x507ac00 2026-02-11T23:31:28.005+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:239f544e:::obj128:head, version: 111'128, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:239f544e:::obj128:head@111'128, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 118 2026-02-11T23:31:28.005+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:239f544e:::obj128:head, version: 111'128, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:239f544e:::obj128:head@111'128, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:28.005+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.005+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 241 ==== pg_backfill(progress 4.0 e 118/118 lb 4:233a42c1:::obj6:head) ==== 1060+0+0 (unknown 0 0 0) 0x5f12680 con 0x507ac00 2026-02-11T23:31:28.005+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:233a42c1:::obj6:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:28.005+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:239f544e:::obj128:head, version: 111'128, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:239f544e:::obj128:head@111'128, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:28.005+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:233a42c1:::obj6:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:28.005+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.005+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:239f544e:::obj128:head, version: 111'128, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:239f544e:::obj128:head@111'128, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:28.005+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:239f544e:::obj128:head, version: 111'128, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:239f544e:::obj128:head@111'128, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 118/117 [PushOp(4:239f544e:::obj128:head, version: 111'128, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:239f544e:::obj128:head@111'128, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:2260a46a:::obj167:head local-lis/les=117/118 n=100 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.005+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.005+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.005+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2260a46a:::obj167:head local-lis/les=117/118 n=100 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:239f544e:::obj128:head, version: 111'128, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:239f544e:::obj128:head@111'128, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.005+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:233a42c1:::obj6:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:28.005+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2260a46a:::obj167:head local-lis/les=117/118 n=100 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:239f544e:::obj128:head@111'128, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.005+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2260a46a:::obj167:head local-lis/les=117/118 n=100 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:239f544e:::obj128:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.005+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2260a46a:::obj167:head local-lis/les=117/118 n=100 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:239f544e:::obj128:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.005+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2260a46a:::obj167:head local-lis/les=117/118 n=100 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:239f544e:::obj128:head 2026-02-11T23:31:28.005+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2260a46a:::obj167:head local-lis/les=117/118 n=100 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:239f544e:::obj128:head v 111'128 2026-02-11T23:31:28.005+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2260a46a:::obj167:head local-lis/les=117/118 n=100 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.005+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.005+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:239f544e:::obj128:head, version: 111'128, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:239f544e:::obj128:head@111'128, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:28.005+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.005+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.005+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:233a42c1:::obj6:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:28.005+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:233a42c1:::obj6:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 4.0 e 118/118 lb 4:233a42c1:::obj6:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:2260a46a:::obj167:head local-lis/les=117/118 n=100 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.005+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.005+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2260a46a:::obj167:head local-lis/les=117/118 n=100 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:233a42c1:::obj6:head) v3 2026-02-11T23:31:28.005+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2260a46a:::obj167:head local-lis/les=117/118 n=100 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:233a42c1:::obj6:head) 2026-02-11T23:31:28.005+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.005+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:233a42c1:::obj6:head) v3 finish latency 0.000015 2026-02-11T23:31:28.005+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:233a42c1:::obj6:head local-lis/les=117/118 n=101 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.005+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.005+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.005+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:233a42c1:::obj6:head local-lis/les=117/118 n=101 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.005+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:28.005+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:28.005+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:28.005+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63c8d80 con 0x507ac00 2026-02-11T23:31:28.005+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:239f544e:::obj128:head)]) -- 0x63b8600 con 0x507ac00 2026-02-11T23:31:28.005+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.005+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 242 ==== MOSDPGPush(4.0 118/117 [PushOp(4:242c40e0:::obj23:head, version: 111'23, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:242c40e0:::obj23:head@111'23, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x554e340 con 0x507ac00 2026-02-11T23:31:28.005+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:242c40e0:::obj23:head, version: 111'23, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:242c40e0:::obj23:head@111'23, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 118 2026-02-11T23:31:28.005+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:242c40e0:::obj23:head, version: 111'23, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:242c40e0:::obj23:head@111'23, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:28.005+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.005+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 243 ==== pg_backfill(progress 4.0 e 118/118 lb 4:239f544e:::obj128:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f12100 con 0x507ac00 2026-02-11T23:31:28.005+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:239f544e:::obj128:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:28.005+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:242c40e0:::obj23:head, version: 111'23, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:242c40e0:::obj23:head@111'23, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:28.005+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:239f544e:::obj128:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:28.005+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.005+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:239f544e:::obj128:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:28.005+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.005+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:242c40e0:::obj23:head, version: 111'23, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:242c40e0:::obj23:head@111'23, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:28.005+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:242c40e0:::obj23:head, version: 111'23, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:242c40e0:::obj23:head@111'23, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000047 MOSDPGPush(4.0 118/117 [PushOp(4:242c40e0:::obj23:head, version: 111'23, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:242c40e0:::obj23:head@111'23, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:233a42c1:::obj6:head local-lis/les=117/118 n=101 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.005+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.005+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:233a42c1:::obj6:head local-lis/les=117/118 n=101 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:242c40e0:::obj23:head, version: 111'23, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:242c40e0:::obj23:head@111'23, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.005+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:233a42c1:::obj6:head local-lis/les=117/118 n=101 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:242c40e0:::obj23:head@111'23, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.005+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:233a42c1:::obj6:head local-lis/les=117/118 n=101 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:242c40e0:::obj23:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.005+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:233a42c1:::obj6:head local-lis/les=117/118 n=101 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:242c40e0:::obj23:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.005+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:233a42c1:::obj6:head local-lis/les=117/118 n=101 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:242c40e0:::obj23:head 2026-02-11T23:31:28.005+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:233a42c1:::obj6:head local-lis/les=117/118 n=101 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:242c40e0:::obj23:head v 111'23 2026-02-11T23:31:28.005+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:233a42c1:::obj6:head local-lis/les=117/118 n=101 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.005+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.005+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:242c40e0:::obj23:head, version: 111'23, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:242c40e0:::obj23:head@111'23, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:28.005+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.005+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.005+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:239f544e:::obj128:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:28.005+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:239f544e:::obj128:head) v3 prio 15 cost 0 latency 0.000086 pg_backfill(progress 4.0 e 118/118 lb 4:239f544e:::obj128:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:233a42c1:::obj6:head local-lis/les=117/118 n=101 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.005+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.005+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:233a42c1:::obj6:head local-lis/les=117/118 n=101 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:239f544e:::obj128:head) v3 2026-02-11T23:31:28.005+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:233a42c1:::obj6:head local-lis/les=117/118 n=101 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:239f544e:::obj128:head) 2026-02-11T23:31:28.005+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.005+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:239f544e:::obj128:head) v3 finish latency 0.000015 2026-02-11T23:31:28.005+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:239f544e:::obj128:head local-lis/les=117/118 n=102 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.005+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.006+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.006+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:239f544e:::obj128:head local-lis/les=117/118 n=102 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.006+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:28.006+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:28.006+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:28.006+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63c8f00 con 0x507ac00 2026-02-11T23:31:28.006+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:242c40e0:::obj23:head)]) -- 0x6447e00 con 0x507ac00 2026-02-11T23:31:28.006+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.006+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 244 ==== MOSDPGPush(4.0 118/117 [PushOp(4:249a23dc:::obj422:head, version: 111'422, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:249a23dc:::obj422:head@111'422, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a3040 con 0x507ac00 2026-02-11T23:31:28.006+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:249a23dc:::obj422:head, version: 111'422, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:249a23dc:::obj422:head@111'422, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 118 2026-02-11T23:31:28.006+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:249a23dc:::obj422:head, version: 111'422, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:249a23dc:::obj422:head@111'422, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:28.006+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.006+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 245 ==== pg_backfill(progress 4.0 e 118/118 lb 4:242c40e0:::obj23:head) ==== 1061+0+0 (unknown 0 0 0) 0x5f11b80 con 0x507ac00 2026-02-11T23:31:28.006+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:249a23dc:::obj422:head, version: 111'422, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:249a23dc:::obj422:head@111'422, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:28.006+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:242c40e0:::obj23:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 118 2026-02-11T23:31:28.006+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:242c40e0:::obj23:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:28.006+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.006+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:249a23dc:::obj422:head, version: 111'422, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:249a23dc:::obj422:head@111'422, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:28.006+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:249a23dc:::obj422:head, version: 111'422, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:249a23dc:::obj422:head@111'422, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(4.0 118/117 [PushOp(4:249a23dc:::obj422:head, version: 111'422, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:249a23dc:::obj422:head@111'422, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:239f544e:::obj128:head local-lis/les=117/118 n=102 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.006+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.006+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.006+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:239f544e:::obj128:head local-lis/les=117/118 n=102 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:249a23dc:::obj422:head, version: 111'422, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:249a23dc:::obj422:head@111'422, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.006+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:242c40e0:::obj23:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:28.006+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:239f544e:::obj128:head local-lis/les=117/118 n=102 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:249a23dc:::obj422:head@111'422, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.006+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:239f544e:::obj128:head local-lis/les=117/118 n=102 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:249a23dc:::obj422:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.006+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:239f544e:::obj128:head local-lis/les=117/118 n=102 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:249a23dc:::obj422:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.006+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:239f544e:::obj128:head local-lis/les=117/118 n=102 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:249a23dc:::obj422:head 2026-02-11T23:31:28.006+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:239f544e:::obj128:head local-lis/les=117/118 n=102 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:249a23dc:::obj422:head v 111'422 2026-02-11T23:31:28.006+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:239f544e:::obj128:head local-lis/les=117/118 n=102 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.006+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.006+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:249a23dc:::obj422:head, version: 111'422, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:249a23dc:::obj422:head@111'422, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:28.006+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.006+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.006+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:242c40e0:::obj23:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:28.006+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:242c40e0:::obj23:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 4.0 e 118/118 lb 4:242c40e0:::obj23:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:239f544e:::obj128:head local-lis/les=117/118 n=102 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.006+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.006+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:239f544e:::obj128:head local-lis/les=117/118 n=102 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:242c40e0:::obj23:head) v3 2026-02-11T23:31:28.006+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:239f544e:::obj128:head local-lis/les=117/118 n=102 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:242c40e0:::obj23:head) 2026-02-11T23:31:28.006+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.006+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:242c40e0:::obj23:head) v3 finish latency 0.000019 2026-02-11T23:31:28.006+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:242c40e0:::obj23:head local-lis/les=117/118 n=103 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.006+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.006+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.006+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:242c40e0:::obj23:head local-lis/les=117/118 n=103 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.006+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:28.006+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:28.006+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:28.006+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63c9080 con 0x507ac00 2026-02-11T23:31:28.006+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:249a23dc:::obj422:head)]) -- 0x649fe00 con 0x507ac00 2026-02-11T23:31:28.006+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.007+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 246 ==== MOSDPGPush(4.0 118/117 [PushOp(4:24b5611b:::obj20:head, version: 111'20, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:24b5611b:::obj20:head@111'20, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x56a3860 con 0x507ac00 2026-02-11T23:31:28.007+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:24b5611b:::obj20:head, version: 111'20, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:24b5611b:::obj20:head@111'20, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 118 2026-02-11T23:31:28.007+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:24b5611b:::obj20:head, version: 111'20, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:24b5611b:::obj20:head@111'20, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:28.007+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.007+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 247 ==== pg_backfill(progress 4.0 e 118/118 lb 4:249a23dc:::obj422:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f13180 con 0x507ac00 2026-02-11T23:31:28.007+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:249a23dc:::obj422:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:28.007+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:24b5611b:::obj20:head, version: 111'20, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:24b5611b:::obj20:head@111'20, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:28.007+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:249a23dc:::obj422:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:28.007+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.007+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:24b5611b:::obj20:head, version: 111'20, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:24b5611b:::obj20:head@111'20, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:28.007+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.007+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:24b5611b:::obj20:head, version: 111'20, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:24b5611b:::obj20:head@111'20, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 118/117 [PushOp(4:24b5611b:::obj20:head, version: 111'20, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:24b5611b:::obj20:head@111'20, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:242c40e0:::obj23:head local-lis/les=117/118 n=103 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.007+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:249a23dc:::obj422:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:28.007+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.007+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:242c40e0:::obj23:head local-lis/les=117/118 n=103 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:24b5611b:::obj20:head, version: 111'20, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:24b5611b:::obj20:head@111'20, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.007+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:242c40e0:::obj23:head local-lis/les=117/118 n=103 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:24b5611b:::obj20:head@111'20, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.007+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:242c40e0:::obj23:head local-lis/les=117/118 n=103 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:24b5611b:::obj20:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.007+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:242c40e0:::obj23:head local-lis/les=117/118 n=103 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:24b5611b:::obj20:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.007+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:242c40e0:::obj23:head local-lis/les=117/118 n=103 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:24b5611b:::obj20:head 2026-02-11T23:31:28.007+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:242c40e0:::obj23:head local-lis/les=117/118 n=103 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:24b5611b:::obj20:head v 111'20 2026-02-11T23:31:28.007+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:242c40e0:::obj23:head local-lis/les=117/118 n=103 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.007+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.007+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:24b5611b:::obj20:head, version: 111'20, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:24b5611b:::obj20:head@111'20, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:28.007+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.007+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.007+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:249a23dc:::obj422:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:28.007+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:249a23dc:::obj422:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 4.0 e 118/118 lb 4:249a23dc:::obj422:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:242c40e0:::obj23:head local-lis/les=117/118 n=103 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.007+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.007+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:242c40e0:::obj23:head local-lis/les=117/118 n=103 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:249a23dc:::obj422:head) v3 2026-02-11T23:31:28.007+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:242c40e0:::obj23:head local-lis/les=117/118 n=103 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:249a23dc:::obj422:head) 2026-02-11T23:31:28.007+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.007+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:249a23dc:::obj422:head) v3 finish latency 0.000013 2026-02-11T23:31:28.007+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:249a23dc:::obj422:head local-lis/les=117/118 n=104 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.007+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.007+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.007+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:249a23dc:::obj422:head local-lis/les=117/118 n=104 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.007+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:28.007+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:28.007+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:28.007+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63c9200 con 0x507ac00 2026-02-11T23:31:28.007+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:24b5611b:::obj20:head)]) -- 0x649e600 con 0x507ac00 2026-02-11T23:31:28.007+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.007+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 248 ==== MOSDPGPush(4.0 118/117 [PushOp(4:24fc6e92:::obj17:head, version: 111'17, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:24fc6e92:::obj17:head@111'17, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x56171e0 con 0x507ac00 2026-02-11T23:31:28.007+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:24fc6e92:::obj17:head, version: 111'17, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:24fc6e92:::obj17:head@111'17, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 118 2026-02-11T23:31:28.007+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:24fc6e92:::obj17:head, version: 111'17, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:24fc6e92:::obj17:head@111'17, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:28.007+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.007+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 249 ==== pg_backfill(progress 4.0 e 118/118 lb 4:24b5611b:::obj20:head) ==== 1061+0+0 (unknown 0 0 0) 0x5f10580 con 0x507ac00 2026-02-11T23:31:28.007+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:24fc6e92:::obj17:head, version: 111'17, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:24fc6e92:::obj17:head@111'17, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:28.007+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:24b5611b:::obj20:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:28.007+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:24b5611b:::obj20:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:28.007+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.007+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:24fc6e92:::obj17:head, version: 111'17, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:24fc6e92:::obj17:head@111'17, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:28.007+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:24fc6e92:::obj17:head, version: 111'17, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:24fc6e92:::obj17:head@111'17, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 118/117 [PushOp(4:24fc6e92:::obj17:head, version: 111'17, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:24fc6e92:::obj17:head@111'17, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:249a23dc:::obj422:head local-lis/les=117/118 n=104 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.007+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.007+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.007+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:24b5611b:::obj20:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:28.007+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:249a23dc:::obj422:head local-lis/les=117/118 n=104 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:24fc6e92:::obj17:head, version: 111'17, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:24fc6e92:::obj17:head@111'17, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.007+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:249a23dc:::obj422:head local-lis/les=117/118 n=104 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:24fc6e92:::obj17:head@111'17, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.007+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:249a23dc:::obj422:head local-lis/les=117/118 n=104 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:24fc6e92:::obj17:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.007+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:249a23dc:::obj422:head local-lis/les=117/118 n=104 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:24fc6e92:::obj17:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.007+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:249a23dc:::obj422:head local-lis/les=117/118 n=104 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:24fc6e92:::obj17:head 2026-02-11T23:31:28.007+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:249a23dc:::obj422:head local-lis/les=117/118 n=104 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:24fc6e92:::obj17:head v 111'17 2026-02-11T23:31:28.007+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:249a23dc:::obj422:head local-lis/les=117/118 n=104 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.007+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.007+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:24fc6e92:::obj17:head, version: 111'17, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:24fc6e92:::obj17:head@111'17, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:28.007+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.007+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.007+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:24b5611b:::obj20:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:28.007+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:24b5611b:::obj20:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 4.0 e 118/118 lb 4:24b5611b:::obj20:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:249a23dc:::obj422:head local-lis/les=117/118 n=104 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.007+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.007+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:249a23dc:::obj422:head local-lis/les=117/118 n=104 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:24b5611b:::obj20:head) v3 2026-02-11T23:31:28.007+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:249a23dc:::obj422:head local-lis/les=117/118 n=104 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:24b5611b:::obj20:head) 2026-02-11T23:31:28.007+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.007+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:24b5611b:::obj20:head) v3 finish latency 0.000015 2026-02-11T23:31:28.007+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:24b5611b:::obj20:head local-lis/les=117/118 n=105 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.007+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.008+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.008+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:24b5611b:::obj20:head local-lis/les=117/118 n=105 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.008+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:28.008+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:28.008+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:28.008+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63c9380 con 0x507ac00 2026-02-11T23:31:28.008+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:24fc6e92:::obj17:head)]) -- 0x6447800 con 0x507ac00 2026-02-11T23:31:28.008+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.008+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 250 ==== MOSDPGPush(4.0 118/117 [PushOp(4:2540f982:::obj403:head, version: 111'403, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2540f982:::obj403:head@111'403, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56cad00 con 0x507ac00 2026-02-11T23:31:28.008+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:2540f982:::obj403:head, version: 111'403, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2540f982:::obj403:head@111'403, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 118 2026-02-11T23:31:28.008+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:2540f982:::obj403:head, version: 111'403, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2540f982:::obj403:head@111'403, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:28.008+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.008+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 251 ==== pg_backfill(progress 4.0 e 118/118 lb 4:24fc6e92:::obj17:head) ==== 1061+0+0 (unknown 0 0 0) 0x5f10b00 con 0x507ac00 2026-02-11T23:31:28.008+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:24fc6e92:::obj17:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:28.008+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:2540f982:::obj403:head, version: 111'403, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2540f982:::obj403:head@111'403, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:28.008+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:24fc6e92:::obj17:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:28.008+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.008+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:2540f982:::obj403:head, version: 111'403, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2540f982:::obj403:head@111'403, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:28.008+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:2540f982:::obj403:head, version: 111'403, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2540f982:::obj403:head@111'403, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 118/117 [PushOp(4:2540f982:::obj403:head, version: 111'403, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2540f982:::obj403:head@111'403, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:24b5611b:::obj20:head local-lis/les=117/118 n=105 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.008+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.008+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.008+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:24fc6e92:::obj17:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:28.008+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:24b5611b:::obj20:head local-lis/les=117/118 n=105 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:2540f982:::obj403:head, version: 111'403, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2540f982:::obj403:head@111'403, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.008+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:24b5611b:::obj20:head local-lis/les=117/118 n=105 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:2540f982:::obj403:head@111'403, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.008+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:24b5611b:::obj20:head local-lis/les=117/118 n=105 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:2540f982:::obj403:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.008+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:24b5611b:::obj20:head local-lis/les=117/118 n=105 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:2540f982:::obj403:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.008+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:24b5611b:::obj20:head local-lis/les=117/118 n=105 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:2540f982:::obj403:head 2026-02-11T23:31:28.008+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:24b5611b:::obj20:head local-lis/les=117/118 n=105 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:2540f982:::obj403:head v 111'403 2026-02-11T23:31:28.008+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:24b5611b:::obj20:head local-lis/les=117/118 n=105 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.008+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.008+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:2540f982:::obj403:head, version: 111'403, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2540f982:::obj403:head@111'403, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:28.008+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.008+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.008+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:24fc6e92:::obj17:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:28.008+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:24fc6e92:::obj17:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 4.0 e 118/118 lb 4:24fc6e92:::obj17:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:24b5611b:::obj20:head local-lis/les=117/118 n=105 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.008+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.008+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:24b5611b:::obj20:head local-lis/les=117/118 n=105 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:24fc6e92:::obj17:head) v3 2026-02-11T23:31:28.008+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:24b5611b:::obj20:head local-lis/les=117/118 n=105 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:24fc6e92:::obj17:head) 2026-02-11T23:31:28.008+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.008+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:24fc6e92:::obj17:head) v3 finish latency 0.000016 2026-02-11T23:31:28.008+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:24fc6e92:::obj17:head local-lis/les=117/118 n=106 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.008+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.008+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.008+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:24fc6e92:::obj17:head local-lis/les=117/118 n=106 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.008+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:28.008+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:28.008+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:28.008+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63c9500 con 0x507ac00 2026-02-11T23:31:28.008+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:2540f982:::obj403:head)]) -- 0x63b8000 con 0x507ac00 2026-02-11T23:31:28.008+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.009+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 252 ==== MOSDPGPush(4.0 118/117 [PushOp(4:2551b5e8:::obj231:head, version: 111'231, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2551b5e8:::obj231:head@111'231, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5646d00 con 0x507ac00 2026-02-11T23:31:28.009+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:2551b5e8:::obj231:head, version: 111'231, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2551b5e8:::obj231:head@111'231, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 118 2026-02-11T23:31:28.009+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:2551b5e8:::obj231:head, version: 111'231, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2551b5e8:::obj231:head@111'231, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:28.009+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.009+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 253 ==== pg_backfill(progress 4.0 e 118/118 lb 4:2540f982:::obj403:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f11080 con 0x507ac00 2026-02-11T23:31:28.009+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:2551b5e8:::obj231:head, version: 111'231, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2551b5e8:::obj231:head@111'231, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:28.009+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:2540f982:::obj403:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:28.009+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:2540f982:::obj403:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:28.009+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.009+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:2551b5e8:::obj231:head, version: 111'231, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2551b5e8:::obj231:head@111'231, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:28.009+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:2551b5e8:::obj231:head, version: 111'231, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2551b5e8:::obj231:head@111'231, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 118/117 [PushOp(4:2551b5e8:::obj231:head, version: 111'231, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2551b5e8:::obj231:head@111'231, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:24fc6e92:::obj17:head local-lis/les=117/118 n=106 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.009+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.009+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.009+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:24fc6e92:::obj17:head local-lis/les=117/118 n=106 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:2551b5e8:::obj231:head, version: 111'231, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2551b5e8:::obj231:head@111'231, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.009+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:2540f982:::obj403:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:28.009+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:24fc6e92:::obj17:head local-lis/les=117/118 n=106 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:2551b5e8:::obj231:head@111'231, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.009+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:24fc6e92:::obj17:head local-lis/les=117/118 n=106 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:2551b5e8:::obj231:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.009+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:24fc6e92:::obj17:head local-lis/les=117/118 n=106 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:2551b5e8:::obj231:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.009+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:24fc6e92:::obj17:head local-lis/les=117/118 n=106 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:2551b5e8:::obj231:head 2026-02-11T23:31:28.009+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:24fc6e92:::obj17:head local-lis/les=117/118 n=106 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:2551b5e8:::obj231:head v 111'231 2026-02-11T23:31:28.009+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:24fc6e92:::obj17:head local-lis/les=117/118 n=106 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.009+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.009+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:2551b5e8:::obj231:head, version: 111'231, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2551b5e8:::obj231:head@111'231, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:28.009+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.009+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.009+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:2540f982:::obj403:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:28.009+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:2540f982:::obj403:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 4.0 e 118/118 lb 4:2540f982:::obj403:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:24fc6e92:::obj17:head local-lis/les=117/118 n=106 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.009+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.009+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:24fc6e92:::obj17:head local-lis/les=117/118 n=106 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:2540f982:::obj403:head) v3 2026-02-11T23:31:28.009+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:24fc6e92:::obj17:head local-lis/les=117/118 n=106 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:2540f982:::obj403:head) 2026-02-11T23:31:28.009+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.009+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:2540f982:::obj403:head) v3 finish latency 0.000016 2026-02-11T23:31:28.009+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2540f982:::obj403:head local-lis/les=117/118 n=107 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.009+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.009+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.009+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2540f982:::obj403:head local-lis/les=117/118 n=107 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.009+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:28.009+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:28.009+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:28.009+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63c9680 con 0x507ac00 2026-02-11T23:31:28.009+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:2551b5e8:::obj231:head)]) -- 0x649e480 con 0x507ac00 2026-02-11T23:31:28.009+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.009+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 254 ==== MOSDPGPush(4.0 118/117 [PushOp(4:26b5008c:::obj336:head, version: 111'336, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:26b5008c:::obj336:head@111'336, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2d00 con 0x507ac00 2026-02-11T23:31:28.009+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:26b5008c:::obj336:head, version: 111'336, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:26b5008c:::obj336:head@111'336, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 118 2026-02-11T23:31:28.009+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:26b5008c:::obj336:head, version: 111'336, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:26b5008c:::obj336:head@111'336, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:28.009+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.009+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 255 ==== pg_backfill(progress 4.0 e 118/118 lb 4:2551b5e8:::obj231:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f11600 con 0x507ac00 2026-02-11T23:31:28.009+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:26b5008c:::obj336:head, version: 111'336, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:26b5008c:::obj336:head@111'336, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:28.009+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:2551b5e8:::obj231:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:28.009+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:2551b5e8:::obj231:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:28.009+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.009+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:26b5008c:::obj336:head, version: 111'336, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:26b5008c:::obj336:head@111'336, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:28.009+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:26b5008c:::obj336:head, version: 111'336, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:26b5008c:::obj336:head@111'336, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 118/117 [PushOp(4:26b5008c:::obj336:head, version: 111'336, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:26b5008c:::obj336:head@111'336, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:2540f982:::obj403:head local-lis/les=117/118 n=107 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.009+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.009+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.009+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2540f982:::obj403:head local-lis/les=117/118 n=107 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:26b5008c:::obj336:head, version: 111'336, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:26b5008c:::obj336:head@111'336, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.009+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2540f982:::obj403:head local-lis/les=117/118 n=107 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:26b5008c:::obj336:head@111'336, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.009+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:2551b5e8:::obj231:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:28.009+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2540f982:::obj403:head local-lis/les=117/118 n=107 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:26b5008c:::obj336:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.009+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2540f982:::obj403:head local-lis/les=117/118 n=107 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:26b5008c:::obj336:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.009+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2540f982:::obj403:head local-lis/les=117/118 n=107 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:26b5008c:::obj336:head 2026-02-11T23:31:28.009+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2540f982:::obj403:head local-lis/les=117/118 n=107 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:26b5008c:::obj336:head v 111'336 2026-02-11T23:31:28.009+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2540f982:::obj403:head local-lis/les=117/118 n=107 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.009+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.009+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:26b5008c:::obj336:head, version: 111'336, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:26b5008c:::obj336:head@111'336, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:28.009+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.009+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.009+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:2551b5e8:::obj231:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:28.009+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:2551b5e8:::obj231:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 4.0 e 118/118 lb 4:2551b5e8:::obj231:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:2540f982:::obj403:head local-lis/les=117/118 n=107 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.009+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.009+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2540f982:::obj403:head local-lis/les=117/118 n=107 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:2551b5e8:::obj231:head) v3 2026-02-11T23:31:28.009+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2540f982:::obj403:head local-lis/les=117/118 n=107 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:2551b5e8:::obj231:head) 2026-02-11T23:31:28.009+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.009+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:2551b5e8:::obj231:head) v3 finish latency 0.000014 2026-02-11T23:31:28.009+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2551b5e8:::obj231:head local-lis/les=117/118 n=108 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.009+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.010+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.010+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2551b5e8:::obj231:head local-lis/les=117/118 n=108 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.010+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:28.010+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:28.010+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:28.010+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63c9800 con 0x507ac00 2026-02-11T23:31:28.010+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:26b5008c:::obj336:head)]) -- 0x63b8480 con 0x507ac00 2026-02-11T23:31:28.010+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.010+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 256 ==== MOSDPGPush(4.0 118/117 [PushOp(4:26cf9b2d:::obj546:head, version: 111'546, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:26cf9b2d:::obj546:head@111'546, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c340 con 0x507ac00 2026-02-11T23:31:28.010+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:26cf9b2d:::obj546:head, version: 111'546, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:26cf9b2d:::obj546:head@111'546, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 118 2026-02-11T23:31:28.010+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:26cf9b2d:::obj546:head, version: 111'546, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:26cf9b2d:::obj546:head@111'546, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:28.010+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.010+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 257 ==== pg_backfill(progress 4.0 e 118/118 lb 4:26b5008c:::obj336:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2f700 con 0x507ac00 2026-02-11T23:31:28.010+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:26b5008c:::obj336:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:28.010+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:26cf9b2d:::obj546:head, version: 111'546, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:26cf9b2d:::obj546:head@111'546, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:28.010+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:26b5008c:::obj336:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:28.010+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.010+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:26cf9b2d:::obj546:head, version: 111'546, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:26cf9b2d:::obj546:head@111'546, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:28.010+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:26cf9b2d:::obj546:head, version: 111'546, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:26cf9b2d:::obj546:head@111'546, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 118/117 [PushOp(4:26cf9b2d:::obj546:head, version: 111'546, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:26cf9b2d:::obj546:head@111'546, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:2551b5e8:::obj231:head local-lis/les=117/118 n=108 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.010+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.010+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.010+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:26b5008c:::obj336:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:28.010+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2551b5e8:::obj231:head local-lis/les=117/118 n=108 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:26cf9b2d:::obj546:head, version: 111'546, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:26cf9b2d:::obj546:head@111'546, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.010+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2551b5e8:::obj231:head local-lis/les=117/118 n=108 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:26cf9b2d:::obj546:head@111'546, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.010+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2551b5e8:::obj231:head local-lis/les=117/118 n=108 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:26cf9b2d:::obj546:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.010+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2551b5e8:::obj231:head local-lis/les=117/118 n=108 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:26cf9b2d:::obj546:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.010+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2551b5e8:::obj231:head local-lis/les=117/118 n=108 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:26cf9b2d:::obj546:head 2026-02-11T23:31:28.010+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2551b5e8:::obj231:head local-lis/les=117/118 n=108 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:26cf9b2d:::obj546:head v 111'546 2026-02-11T23:31:28.010+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2551b5e8:::obj231:head local-lis/les=117/118 n=108 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.010+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.010+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:26cf9b2d:::obj546:head, version: 111'546, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:26cf9b2d:::obj546:head@111'546, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:28.010+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.010+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.010+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:26b5008c:::obj336:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:28.010+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:26b5008c:::obj336:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 4.0 e 118/118 lb 4:26b5008c:::obj336:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:2551b5e8:::obj231:head local-lis/les=117/118 n=108 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.010+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.010+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2551b5e8:::obj231:head local-lis/les=117/118 n=108 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:26b5008c:::obj336:head) v3 2026-02-11T23:31:28.010+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2551b5e8:::obj231:head local-lis/les=117/118 n=108 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:26b5008c:::obj336:head) 2026-02-11T23:31:28.010+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.010+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:26b5008c:::obj336:head) v3 finish latency 0.000016 2026-02-11T23:31:28.010+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:26b5008c:::obj336:head local-lis/les=117/118 n=109 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.010+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.010+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.010+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:26b5008c:::obj336:head local-lis/les=117/118 n=109 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.010+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:28.010+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:28.010+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:28.010+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63c9980 con 0x507ac00 2026-02-11T23:31:28.010+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:26cf9b2d:::obj546:head)]) -- 0x63b8180 con 0x507ac00 2026-02-11T23:31:28.010+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.011+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 258 ==== MOSDPGPush(4.0 118/117 [PushOp(4:27794e2d:::obj115:head, version: 111'115, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:27794e2d:::obj115:head@111'115, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3814d00 con 0x507ac00 2026-02-11T23:31:28.011+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:27794e2d:::obj115:head, version: 111'115, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:27794e2d:::obj115:head@111'115, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 118 2026-02-11T23:31:28.011+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:27794e2d:::obj115:head, version: 111'115, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:27794e2d:::obj115:head@111'115, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:28.011+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.011+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 259 ==== pg_backfill(progress 4.0 e 118/118 lb 4:26cf9b2d:::obj546:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2d080 con 0x507ac00 2026-02-11T23:31:28.011+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:27794e2d:::obj115:head, version: 111'115, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:27794e2d:::obj115:head@111'115, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:28.011+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:26cf9b2d:::obj546:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:28.011+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:26cf9b2d:::obj546:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:28.011+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.011+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:27794e2d:::obj115:head, version: 111'115, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:27794e2d:::obj115:head@111'115, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:28.011+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:27794e2d:::obj115:head, version: 111'115, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:27794e2d:::obj115:head@111'115, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 118/117 [PushOp(4:27794e2d:::obj115:head, version: 111'115, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:27794e2d:::obj115:head@111'115, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:26b5008c:::obj336:head local-lis/les=117/118 n=109 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.011+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.011+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.011+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:26cf9b2d:::obj546:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:28.011+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:26b5008c:::obj336:head local-lis/les=117/118 n=109 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:27794e2d:::obj115:head, version: 111'115, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:27794e2d:::obj115:head@111'115, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.011+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:26b5008c:::obj336:head local-lis/les=117/118 n=109 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:27794e2d:::obj115:head@111'115, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.011+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:26b5008c:::obj336:head local-lis/les=117/118 n=109 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:27794e2d:::obj115:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.011+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:26b5008c:::obj336:head local-lis/les=117/118 n=109 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:27794e2d:::obj115:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.011+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:26b5008c:::obj336:head local-lis/les=117/118 n=109 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:27794e2d:::obj115:head 2026-02-11T23:31:28.011+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:26b5008c:::obj336:head local-lis/les=117/118 n=109 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:27794e2d:::obj115:head v 111'115 2026-02-11T23:31:28.011+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:26b5008c:::obj336:head local-lis/les=117/118 n=109 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.011+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.011+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:27794e2d:::obj115:head, version: 111'115, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:27794e2d:::obj115:head@111'115, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:28.011+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.011+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.011+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:26cf9b2d:::obj546:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:28.011+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:26cf9b2d:::obj546:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 4.0 e 118/118 lb 4:26cf9b2d:::obj546:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:26b5008c:::obj336:head local-lis/les=117/118 n=109 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.011+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.011+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:26b5008c:::obj336:head local-lis/les=117/118 n=109 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:26cf9b2d:::obj546:head) v3 2026-02-11T23:31:28.011+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:26b5008c:::obj336:head local-lis/les=117/118 n=109 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:26cf9b2d:::obj546:head) 2026-02-11T23:31:28.011+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.011+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:26cf9b2d:::obj546:head) v3 finish latency 0.000016 2026-02-11T23:31:28.011+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:26cf9b2d:::obj546:head local-lis/les=117/118 n=110 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.011+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.011+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.011+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:26cf9b2d:::obj546:head local-lis/les=117/118 n=110 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.011+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:28.011+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:28.011+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:28.011+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63c9b00 con 0x507ac00 2026-02-11T23:31:28.011+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:27794e2d:::obj115:head)]) -- 0x63b8900 con 0x507ac00 2026-02-11T23:31:28.011+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.011+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 260 ==== MOSDPGPush(4.0 118/117 [PushOp(4:27952899:::obj201:head, version: 111'201, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:27952899:::obj201:head@111'201, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e261a0 con 0x507ac00 2026-02-11T23:31:28.011+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:27952899:::obj201:head, version: 111'201, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:27952899:::obj201:head@111'201, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 118 2026-02-11T23:31:28.011+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:27952899:::obj201:head, version: 111'201, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:27952899:::obj201:head@111'201, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:28.011+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.011+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 261 ==== pg_backfill(progress 4.0 e 118/118 lb 4:27794e2d:::obj115:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e680 con 0x507ac00 2026-02-11T23:31:28.011+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:27794e2d:::obj115:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:28.011+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:27952899:::obj201:head, version: 111'201, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:27952899:::obj201:head@111'201, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:28.011+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:27794e2d:::obj115:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:28.011+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.011+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:27952899:::obj201:head, version: 111'201, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:27952899:::obj201:head@111'201, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:28.011+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.011+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:27794e2d:::obj115:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:28.011+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:27952899:::obj201:head, version: 111'201, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:27952899:::obj201:head@111'201, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(4.0 118/117 [PushOp(4:27952899:::obj201:head, version: 111'201, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:27952899:::obj201:head@111'201, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:26cf9b2d:::obj546:head local-lis/les=117/118 n=110 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.011+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.011+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:26cf9b2d:::obj546:head local-lis/les=117/118 n=110 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:27952899:::obj201:head, version: 111'201, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:27952899:::obj201:head@111'201, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.011+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:26cf9b2d:::obj546:head local-lis/les=117/118 n=110 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:27952899:::obj201:head@111'201, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.011+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:26cf9b2d:::obj546:head local-lis/les=117/118 n=110 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:27952899:::obj201:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.011+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:26cf9b2d:::obj546:head local-lis/les=117/118 n=110 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:27952899:::obj201:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.011+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:26cf9b2d:::obj546:head local-lis/les=117/118 n=110 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:27952899:::obj201:head 2026-02-11T23:31:28.011+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:26cf9b2d:::obj546:head local-lis/les=117/118 n=110 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:27952899:::obj201:head v 111'201 2026-02-11T23:31:28.011+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:26cf9b2d:::obj546:head local-lis/les=117/118 n=110 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.011+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.011+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:27952899:::obj201:head, version: 111'201, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:27952899:::obj201:head@111'201, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:28.011+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.011+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.011+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:27794e2d:::obj115:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:28.011+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:27794e2d:::obj115:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 4.0 e 118/118 lb 4:27794e2d:::obj115:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:26cf9b2d:::obj546:head local-lis/les=117/118 n=110 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.011+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.011+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:26cf9b2d:::obj546:head local-lis/les=117/118 n=110 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:27794e2d:::obj115:head) v3 2026-02-11T23:31:28.011+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:26cf9b2d:::obj546:head local-lis/les=117/118 n=110 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:27794e2d:::obj115:head) 2026-02-11T23:31:28.011+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.011+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:27794e2d:::obj115:head) v3 finish latency 0.000015 2026-02-11T23:31:28.011+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:27794e2d:::obj115:head local-lis/les=117/118 n=111 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.011+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.011+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:27794e2d:::obj115:head local-lis/les=117/118 n=111 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.011+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:28.011+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:28.011+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:28.011+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63c9c80 con 0x507ac00 2026-02-11T23:31:28.011+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.012+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:27952899:::obj201:head)]) -- 0x63c8900 con 0x507ac00 2026-02-11T23:31:28.012+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.012+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 262 ==== MOSDPGPush(4.0 118/117 [PushOp(4:29cd40a5:::obj549:head, version: 111'549, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:29cd40a5:::obj549:head@111'549, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f0ea0 con 0x507ac00 2026-02-11T23:31:28.012+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:29cd40a5:::obj549:head, version: 111'549, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:29cd40a5:::obj549:head@111'549, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 118 2026-02-11T23:31:28.012+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:29cd40a5:::obj549:head, version: 111'549, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:29cd40a5:::obj549:head@111'549, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:28.012+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.012+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 263 ==== pg_backfill(progress 4.0 e 118/118 lb 4:27952899:::obj201:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e100 con 0x507ac00 2026-02-11T23:31:28.012+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:29cd40a5:::obj549:head, version: 111'549, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:29cd40a5:::obj549:head@111'549, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:28.012+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:27952899:::obj201:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:28.012+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:27952899:::obj201:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:28.012+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.012+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:29cd40a5:::obj549:head, version: 111'549, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:29cd40a5:::obj549:head@111'549, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:28.012+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:29cd40a5:::obj549:head, version: 111'549, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:29cd40a5:::obj549:head@111'549, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 118/117 [PushOp(4:29cd40a5:::obj549:head, version: 111'549, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:29cd40a5:::obj549:head@111'549, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:27794e2d:::obj115:head local-lis/les=117/118 n=111 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.012+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.012+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.012+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:27952899:::obj201:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:28.012+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:27794e2d:::obj115:head local-lis/les=117/118 n=111 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:29cd40a5:::obj549:head, version: 111'549, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:29cd40a5:::obj549:head@111'549, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.012+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:27794e2d:::obj115:head local-lis/les=117/118 n=111 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:29cd40a5:::obj549:head@111'549, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.012+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:27794e2d:::obj115:head local-lis/les=117/118 n=111 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:29cd40a5:::obj549:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.012+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:27794e2d:::obj115:head local-lis/les=117/118 n=111 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:29cd40a5:::obj549:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.012+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:27794e2d:::obj115:head local-lis/les=117/118 n=111 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:29cd40a5:::obj549:head 2026-02-11T23:31:28.012+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:27794e2d:::obj115:head local-lis/les=117/118 n=111 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:29cd40a5:::obj549:head v 111'549 2026-02-11T23:31:28.012+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:27794e2d:::obj115:head local-lis/les=117/118 n=111 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.012+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.012+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:29cd40a5:::obj549:head, version: 111'549, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:29cd40a5:::obj549:head@111'549, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:28.012+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.012+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.012+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:27952899:::obj201:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:28.012+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:27952899:::obj201:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 4.0 e 118/118 lb 4:27952899:::obj201:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:27794e2d:::obj115:head local-lis/les=117/118 n=111 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.012+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.012+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:27794e2d:::obj115:head local-lis/les=117/118 n=111 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:27952899:::obj201:head) v3 2026-02-11T23:31:28.012+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:27794e2d:::obj115:head local-lis/les=117/118 n=111 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:27952899:::obj201:head) 2026-02-11T23:31:28.012+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.012+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:27952899:::obj201:head) v3 finish latency 0.000013 2026-02-11T23:31:28.012+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:27952899:::obj201:head local-lis/les=117/118 n=112 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.012+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.012+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.012+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:27952899:::obj201:head local-lis/les=117/118 n=112 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.012+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:28.012+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:28.012+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:28.012+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63c9e00 con 0x507ac00 2026-02-11T23:31:28.012+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:29cd40a5:::obj549:head)]) -- 0x63c8780 con 0x507ac00 2026-02-11T23:31:28.012+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.012+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 264 ==== MOSDPGPush(4.0 118/117 [PushOp(4:2a627378:::obj100:head, version: 111'100, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2a627378:::obj100:head@111'100, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c4e0 con 0x507ac00 2026-02-11T23:31:28.012+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:2a627378:::obj100:head, version: 111'100, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2a627378:::obj100:head@111'100, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 118 2026-02-11T23:31:28.012+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:2a627378:::obj100:head, version: 111'100, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2a627378:::obj100:head@111'100, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:28.012+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.012+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:2a627378:::obj100:head, version: 111'100, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2a627378:::obj100:head@111'100, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:28.012+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 265 ==== pg_backfill(progress 4.0 e 118/118 lb 4:29cd40a5:::obj549:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2c000 con 0x507ac00 2026-02-11T23:31:28.013+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:29cd40a5:::obj549:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 118 2026-02-11T23:31:28.013+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:29cd40a5:::obj549:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:28.013+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.013+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:2a627378:::obj100:head, version: 111'100, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2a627378:::obj100:head@111'100, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:28.013+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.013+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:2a627378:::obj100:head, version: 111'100, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2a627378:::obj100:head@111'100, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(4.0 118/117 [PushOp(4:2a627378:::obj100:head, version: 111'100, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2a627378:::obj100:head@111'100, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:27952899:::obj201:head local-lis/les=117/118 n=112 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.013+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.013+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:27952899:::obj201:head local-lis/les=117/118 n=112 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:2a627378:::obj100:head, version: 111'100, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2a627378:::obj100:head@111'100, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.013+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:27952899:::obj201:head local-lis/les=117/118 n=112 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:2a627378:::obj100:head@111'100, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.013+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.013+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:29cd40a5:::obj549:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:28.013+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:27952899:::obj201:head local-lis/les=117/118 n=112 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:2a627378:::obj100:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.013+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:27952899:::obj201:head local-lis/les=117/118 n=112 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:2a627378:::obj100:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.013+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:27952899:::obj201:head local-lis/les=117/118 n=112 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:2a627378:::obj100:head 2026-02-11T23:31:28.013+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:27952899:::obj201:head local-lis/les=117/118 n=112 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:2a627378:::obj100:head v 111'100 2026-02-11T23:31:28.013+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:27952899:::obj201:head local-lis/les=117/118 n=112 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.013+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.013+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:2a627378:::obj100:head, version: 111'100, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2a627378:::obj100:head@111'100, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:28.013+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.013+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.013+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:29cd40a5:::obj549:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:28.013+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:29cd40a5:::obj549:head) v3 prio 15 cost 0 latency 0.000082 pg_backfill(progress 4.0 e 118/118 lb 4:29cd40a5:::obj549:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:27952899:::obj201:head local-lis/les=117/118 n=112 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.013+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.013+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:27952899:::obj201:head local-lis/les=117/118 n=112 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:29cd40a5:::obj549:head) v3 2026-02-11T23:31:28.013+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:27952899:::obj201:head local-lis/les=117/118 n=112 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:29cd40a5:::obj549:head) 2026-02-11T23:31:28.013+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.013+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:29cd40a5:::obj549:head) v3 finish latency 0.000014 2026-02-11T23:31:28.013+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:29cd40a5:::obj549:head local-lis/les=117/118 n=113 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.013+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.013+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.013+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:29cd40a5:::obj549:head local-lis/les=117/118 n=113 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.013+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:28.013+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:28.013+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:28.013+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63c4000 con 0x507ac00 2026-02-11T23:31:28.013+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:2a627378:::obj100:head)]) -- 0x63c8600 con 0x507ac00 2026-02-11T23:31:28.013+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.013+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 266 ==== MOSDPGPush(4.0 118/117 [PushOp(4:2acce685:::obj583:head, version: 111'583, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2acce685:::obj583:head@111'583, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f0d00 con 0x507ac00 2026-02-11T23:31:28.013+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:2acce685:::obj583:head, version: 111'583, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2acce685:::obj583:head@111'583, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 118 2026-02-11T23:31:28.013+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:2acce685:::obj583:head, version: 111'583, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2acce685:::obj583:head@111'583, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:28.013+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.013+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 267 ==== pg_backfill(progress 4.0 e 118/118 lb 4:2a627378:::obj100:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2cb00 con 0x507ac00 2026-02-11T23:31:28.013+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:2a627378:::obj100:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:28.013+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:2acce685:::obj583:head, version: 111'583, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2acce685:::obj583:head@111'583, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:28.013+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:2a627378:::obj100:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:28.013+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.013+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:2acce685:::obj583:head, version: 111'583, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2acce685:::obj583:head@111'583, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:28.013+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:2acce685:::obj583:head, version: 111'583, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2acce685:::obj583:head@111'583, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 118/117 [PushOp(4:2acce685:::obj583:head, version: 111'583, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2acce685:::obj583:head@111'583, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:29cd40a5:::obj549:head local-lis/les=117/118 n=113 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.013+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.013+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.013+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:2a627378:::obj100:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:28.013+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:29cd40a5:::obj549:head local-lis/les=117/118 n=113 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:2acce685:::obj583:head, version: 111'583, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2acce685:::obj583:head@111'583, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.013+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:29cd40a5:::obj549:head local-lis/les=117/118 n=113 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:2acce685:::obj583:head@111'583, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.013+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:29cd40a5:::obj549:head local-lis/les=117/118 n=113 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:2acce685:::obj583:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.013+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:29cd40a5:::obj549:head local-lis/les=117/118 n=113 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:2acce685:::obj583:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.013+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:29cd40a5:::obj549:head local-lis/les=117/118 n=113 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:2acce685:::obj583:head 2026-02-11T23:31:28.013+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:29cd40a5:::obj549:head local-lis/les=117/118 n=113 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:2acce685:::obj583:head v 111'583 2026-02-11T23:31:28.013+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:29cd40a5:::obj549:head local-lis/les=117/118 n=113 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.013+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.013+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:2acce685:::obj583:head, version: 111'583, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2acce685:::obj583:head@111'583, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:28.013+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.013+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.013+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:2a627378:::obj100:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:28.013+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:2a627378:::obj100:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 4.0 e 118/118 lb 4:2a627378:::obj100:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:29cd40a5:::obj549:head local-lis/les=117/118 n=113 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.013+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.013+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:29cd40a5:::obj549:head local-lis/les=117/118 n=113 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:2a627378:::obj100:head) v3 2026-02-11T23:31:28.013+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:29cd40a5:::obj549:head local-lis/les=117/118 n=113 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:2a627378:::obj100:head) 2026-02-11T23:31:28.013+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.013+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:2a627378:::obj100:head) v3 finish latency 0.000018 2026-02-11T23:31:28.013+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2a627378:::obj100:head local-lis/les=117/118 n=114 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.013+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.013+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.013+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2a627378:::obj100:head local-lis/les=117/118 n=114 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.013+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:28.013+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:28.013+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:28.013+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63c4180 con 0x507ac00 2026-02-11T23:31:28.013+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:2acce685:::obj583:head)]) -- 0x63c8480 con 0x507ac00 2026-02-11T23:31:28.013+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.014+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 268 ==== MOSDPGPush(4.0 118/117 [PushOp(4:2b184b8d:::obj560:head, version: 111'560, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2b184b8d:::obj560:head@111'560, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56ca9c0 con 0x507ac00 2026-02-11T23:31:28.014+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:2b184b8d:::obj560:head, version: 111'560, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2b184b8d:::obj560:head@111'560, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 118 2026-02-11T23:31:28.014+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:2b184b8d:::obj560:head, version: 111'560, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2b184b8d:::obj560:head@111'560, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:28.014+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.014+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:2b184b8d:::obj560:head, version: 111'560, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2b184b8d:::obj560:head@111'560, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:28.014+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 269 ==== pg_backfill(progress 4.0 e 118/118 lb 4:2acce685:::obj583:head) ==== 1062+0+0 (unknown 0 0 0) 0x6179700 con 0x507ac00 2026-02-11T23:31:28.014+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:2acce685:::obj583:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 118 2026-02-11T23:31:28.014+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.014+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:2acce685:::obj583:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:28.014+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:2b184b8d:::obj560:head, version: 111'560, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2b184b8d:::obj560:head@111'560, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:28.014+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.014+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:2b184b8d:::obj560:head, version: 111'560, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2b184b8d:::obj560:head@111'560, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 118/117 [PushOp(4:2b184b8d:::obj560:head, version: 111'560, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2b184b8d:::obj560:head@111'560, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:2a627378:::obj100:head local-lis/les=117/118 n=114 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.014+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.014+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2a627378:::obj100:head local-lis/les=117/118 n=114 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:2b184b8d:::obj560:head, version: 111'560, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2b184b8d:::obj560:head@111'560, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.014+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2a627378:::obj100:head local-lis/les=117/118 n=114 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:2b184b8d:::obj560:head@111'560, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.014+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.014+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:2acce685:::obj583:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:28.014+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2a627378:::obj100:head local-lis/les=117/118 n=114 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:2b184b8d:::obj560:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.014+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2a627378:::obj100:head local-lis/les=117/118 n=114 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:2b184b8d:::obj560:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.014+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2a627378:::obj100:head local-lis/les=117/118 n=114 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:2b184b8d:::obj560:head 2026-02-11T23:31:28.014+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2a627378:::obj100:head local-lis/les=117/118 n=114 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:2b184b8d:::obj560:head v 111'560 2026-02-11T23:31:28.014+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2a627378:::obj100:head local-lis/les=117/118 n=114 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.014+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.014+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:2b184b8d:::obj560:head, version: 111'560, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2b184b8d:::obj560:head@111'560, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:28.014+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.014+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.014+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:2acce685:::obj583:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:28.014+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:2acce685:::obj583:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 4.0 e 118/118 lb 4:2acce685:::obj583:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:2a627378:::obj100:head local-lis/les=117/118 n=114 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.014+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.014+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2a627378:::obj100:head local-lis/les=117/118 n=114 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:2acce685:::obj583:head) v3 2026-02-11T23:31:28.014+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2a627378:::obj100:head local-lis/les=117/118 n=114 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:2acce685:::obj583:head) 2026-02-11T23:31:28.014+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.014+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:2acce685:::obj583:head) v3 finish latency 0.000015 2026-02-11T23:31:28.014+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2acce685:::obj583:head local-lis/les=117/118 n=115 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.014+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.014+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.014+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2acce685:::obj583:head local-lis/les=117/118 n=115 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.014+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:28.014+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:28.014+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:28.014+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63c4300 con 0x507ac00 2026-02-11T23:31:28.014+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:2b184b8d:::obj560:head)]) -- 0x63c8300 con 0x507ac00 2026-02-11T23:31:28.014+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.014+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 270 ==== MOSDPGPush(4.0 118/117 [PushOp(4:2c8d8a76:::obj467:head, version: 111'467, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2c8d8a76:::obj467:head@111'467, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56caea0 con 0x507ac00 2026-02-11T23:31:28.014+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:2c8d8a76:::obj467:head, version: 111'467, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2c8d8a76:::obj467:head@111'467, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 118 2026-02-11T23:31:28.014+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:2c8d8a76:::obj467:head, version: 111'467, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2c8d8a76:::obj467:head@111'467, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:28.014+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.014+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 271 ==== pg_backfill(progress 4.0 e 118/118 lb 4:2b184b8d:::obj560:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176000 con 0x507ac00 2026-02-11T23:31:28.014+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:2c8d8a76:::obj467:head, version: 111'467, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2c8d8a76:::obj467:head@111'467, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:28.014+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:2b184b8d:::obj560:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:28.014+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.014+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:2b184b8d:::obj560:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:28.014+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:2c8d8a76:::obj467:head, version: 111'467, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2c8d8a76:::obj467:head@111'467, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:28.014+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.014+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:2c8d8a76:::obj467:head, version: 111'467, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2c8d8a76:::obj467:head@111'467, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 118/117 [PushOp(4:2c8d8a76:::obj467:head, version: 111'467, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2c8d8a76:::obj467:head@111'467, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:2acce685:::obj583:head local-lis/les=117/118 n=115 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.014+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.014+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2acce685:::obj583:head local-lis/les=117/118 n=115 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:2c8d8a76:::obj467:head, version: 111'467, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2c8d8a76:::obj467:head@111'467, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.014+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2acce685:::obj583:head local-lis/les=117/118 n=115 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:2c8d8a76:::obj467:head@111'467, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.014+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.014+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:2b184b8d:::obj560:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:28.015+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2acce685:::obj583:head local-lis/les=117/118 n=115 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:2c8d8a76:::obj467:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.015+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2acce685:::obj583:head local-lis/les=117/118 n=115 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:2c8d8a76:::obj467:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.015+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2acce685:::obj583:head local-lis/les=117/118 n=115 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:2c8d8a76:::obj467:head 2026-02-11T23:31:28.015+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2acce685:::obj583:head local-lis/les=117/118 n=115 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:2c8d8a76:::obj467:head v 111'467 2026-02-11T23:31:28.015+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2acce685:::obj583:head local-lis/les=117/118 n=115 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.015+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.015+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:2c8d8a76:::obj467:head, version: 111'467, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2c8d8a76:::obj467:head@111'467, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000066 2026-02-11T23:31:28.015+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.015+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.015+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:2b184b8d:::obj560:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:28.015+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:2b184b8d:::obj560:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 4.0 e 118/118 lb 4:2b184b8d:::obj560:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:2acce685:::obj583:head local-lis/les=117/118 n=115 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.015+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.015+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2acce685:::obj583:head local-lis/les=117/118 n=115 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:2b184b8d:::obj560:head) v3 2026-02-11T23:31:28.015+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2acce685:::obj583:head local-lis/les=117/118 n=115 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:2b184b8d:::obj560:head) 2026-02-11T23:31:28.015+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.015+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:2b184b8d:::obj560:head) v3 finish latency 0.000013 2026-02-11T23:31:28.015+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2b184b8d:::obj560:head local-lis/les=117/118 n=116 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.015+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.015+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.015+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2b184b8d:::obj560:head local-lis/les=117/118 n=116 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.015+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:28.015+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:28.015+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:28.015+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63c4480 con 0x507ac00 2026-02-11T23:31:28.015+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:2c8d8a76:::obj467:head)]) -- 0x63c8180 con 0x507ac00 2026-02-11T23:31:28.015+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.015+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 272 ==== MOSDPGPush(4.0 118/117 [PushOp(4:2caf4871:::obj405:head, version: 111'405, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2caf4871:::obj405:head@111'405, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5617380 con 0x507ac00 2026-02-11T23:31:28.015+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:2caf4871:::obj405:head, version: 111'405, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2caf4871:::obj405:head@111'405, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 118 2026-02-11T23:31:28.015+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:2caf4871:::obj405:head, version: 111'405, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2caf4871:::obj405:head@111'405, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:28.015+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.015+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 273 ==== pg_backfill(progress 4.0 e 118/118 lb 4:2c8d8a76:::obj467:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9a680 con 0x507ac00 2026-02-11T23:31:28.015+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:2c8d8a76:::obj467:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:28.015+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:2caf4871:::obj405:head, version: 111'405, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2caf4871:::obj405:head@111'405, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:28.015+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:2c8d8a76:::obj467:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:28.015+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.015+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:2caf4871:::obj405:head, version: 111'405, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2caf4871:::obj405:head@111'405, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:28.015+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:2caf4871:::obj405:head, version: 111'405, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2caf4871:::obj405:head@111'405, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(4.0 118/117 [PushOp(4:2caf4871:::obj405:head, version: 111'405, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2caf4871:::obj405:head@111'405, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:2b184b8d:::obj560:head local-lis/les=117/118 n=116 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.015+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.015+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.015+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:2c8d8a76:::obj467:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:28.015+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2b184b8d:::obj560:head local-lis/les=117/118 n=116 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:2caf4871:::obj405:head, version: 111'405, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2caf4871:::obj405:head@111'405, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.015+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2b184b8d:::obj560:head local-lis/les=117/118 n=116 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:2caf4871:::obj405:head@111'405, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.015+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2b184b8d:::obj560:head local-lis/les=117/118 n=116 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:2caf4871:::obj405:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.015+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2b184b8d:::obj560:head local-lis/les=117/118 n=116 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:2caf4871:::obj405:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.015+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2b184b8d:::obj560:head local-lis/les=117/118 n=116 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:2caf4871:::obj405:head 2026-02-11T23:31:28.015+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2b184b8d:::obj560:head local-lis/les=117/118 n=116 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:2caf4871:::obj405:head v 111'405 2026-02-11T23:31:28.015+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2b184b8d:::obj560:head local-lis/les=117/118 n=116 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.015+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.015+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:2caf4871:::obj405:head, version: 111'405, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2caf4871:::obj405:head@111'405, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:28.015+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.015+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.015+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:2c8d8a76:::obj467:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:28.015+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:2c8d8a76:::obj467:head) v3 prio 15 cost 0 latency 0.000081 pg_backfill(progress 4.0 e 118/118 lb 4:2c8d8a76:::obj467:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:2b184b8d:::obj560:head local-lis/les=117/118 n=116 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.015+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.015+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2b184b8d:::obj560:head local-lis/les=117/118 n=116 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:2c8d8a76:::obj467:head) v3 2026-02-11T23:31:28.015+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2b184b8d:::obj560:head local-lis/les=117/118 n=116 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:2c8d8a76:::obj467:head) 2026-02-11T23:31:28.015+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.015+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:2c8d8a76:::obj467:head) v3 finish latency 0.000018 2026-02-11T23:31:28.015+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2c8d8a76:::obj467:head local-lis/les=117/118 n=117 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.015+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.015+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.015+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2c8d8a76:::obj467:head local-lis/les=117/118 n=117 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.015+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:28.015+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:28.015+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:28.015+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63c4600 con 0x507ac00 2026-02-11T23:31:28.015+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:2caf4871:::obj405:head)]) -- 0x63c8000 con 0x507ac00 2026-02-11T23:31:28.015+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.016+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 274 ==== MOSDPGPush(4.0 118/117 [PushOp(4:2d59679a:::obj179:head, version: 111'179, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2d59679a:::obj179:head@111'179, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a3a00 con 0x507ac00 2026-02-11T23:31:28.016+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:2d59679a:::obj179:head, version: 111'179, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2d59679a:::obj179:head@111'179, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 118 2026-02-11T23:31:28.016+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:2d59679a:::obj179:head, version: 111'179, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2d59679a:::obj179:head@111'179, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:28.016+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.016+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 275 ==== pg_backfill(progress 4.0 e 118/118 lb 4:2caf4871:::obj405:head) ==== 1062+0+0 (unknown 0 0 0) 0x5985700 con 0x507ac00 2026-02-11T23:31:28.016+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:2caf4871:::obj405:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:28.016+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:2d59679a:::obj179:head, version: 111'179, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2d59679a:::obj179:head@111'179, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:28.016+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:2caf4871:::obj405:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:28.016+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.016+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:2caf4871:::obj405:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:28.016+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.016+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:2d59679a:::obj179:head, version: 111'179, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2d59679a:::obj179:head@111'179, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:28.016+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:2d59679a:::obj179:head, version: 111'179, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2d59679a:::obj179:head@111'179, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000050 MOSDPGPush(4.0 118/117 [PushOp(4:2d59679a:::obj179:head, version: 111'179, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2d59679a:::obj179:head@111'179, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:2c8d8a76:::obj467:head local-lis/les=117/118 n=117 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.016+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.016+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2c8d8a76:::obj467:head local-lis/les=117/118 n=117 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:2d59679a:::obj179:head, version: 111'179, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2d59679a:::obj179:head@111'179, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.016+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2c8d8a76:::obj467:head local-lis/les=117/118 n=117 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:2d59679a:::obj179:head@111'179, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.016+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2c8d8a76:::obj467:head local-lis/les=117/118 n=117 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:2d59679a:::obj179:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.016+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2c8d8a76:::obj467:head local-lis/les=117/118 n=117 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:2d59679a:::obj179:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.016+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2c8d8a76:::obj467:head local-lis/les=117/118 n=117 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:2d59679a:::obj179:head 2026-02-11T23:31:28.016+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2c8d8a76:::obj467:head local-lis/les=117/118 n=117 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:2d59679a:::obj179:head v 111'179 2026-02-11T23:31:28.016+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2c8d8a76:::obj467:head local-lis/les=117/118 n=117 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.016+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.016+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:2d59679a:::obj179:head, version: 111'179, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2d59679a:::obj179:head@111'179, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:28.016+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.016+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.016+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:2caf4871:::obj405:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:28.016+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:2caf4871:::obj405:head) v3 prio 15 cost 0 latency 0.000091 pg_backfill(progress 4.0 e 118/118 lb 4:2caf4871:::obj405:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:2c8d8a76:::obj467:head local-lis/les=117/118 n=117 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.016+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.016+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2c8d8a76:::obj467:head local-lis/les=117/118 n=117 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:2caf4871:::obj405:head) v3 2026-02-11T23:31:28.016+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2c8d8a76:::obj467:head local-lis/les=117/118 n=117 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:2caf4871:::obj405:head) 2026-02-11T23:31:28.016+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.016+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:2caf4871:::obj405:head) v3 finish latency 0.000015 2026-02-11T23:31:28.016+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2caf4871:::obj405:head local-lis/les=117/118 n=118 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.016+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.016+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.016+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2caf4871:::obj405:head local-lis/les=117/118 n=118 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.016+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:28.016+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:28.016+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:28.016+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63c4780 con 0x507ac00 2026-02-11T23:31:28.016+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:2d59679a:::obj179:head)]) -- 0x649ea80 con 0x507ac00 2026-02-11T23:31:28.016+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.016+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 276 ==== MOSDPGPush(4.0 118/117 [PushOp(4:2df8f997:::obj267:head, version: 111'267, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2df8f997:::obj267:head@111'267, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x554e680 con 0x507ac00 2026-02-11T23:31:28.016+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:2df8f997:::obj267:head, version: 111'267, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2df8f997:::obj267:head@111'267, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 118 2026-02-11T23:31:28.016+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:2df8f997:::obj267:head, version: 111'267, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2df8f997:::obj267:head@111'267, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:28.016+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.016+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 277 ==== pg_backfill(progress 4.0 e 118/118 lb 4:2d59679a:::obj179:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177600 con 0x507ac00 2026-02-11T23:31:28.016+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:2d59679a:::obj179:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:28.016+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:2df8f997:::obj267:head, version: 111'267, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2df8f997:::obj267:head@111'267, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:28.016+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:2d59679a:::obj179:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:28.016+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.016+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:2df8f997:::obj267:head, version: 111'267, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2df8f997:::obj267:head@111'267, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:28.017+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.017+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:2d59679a:::obj179:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:28.017+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:2df8f997:::obj267:head, version: 111'267, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2df8f997:::obj267:head@111'267, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(4.0 118/117 [PushOp(4:2df8f997:::obj267:head, version: 111'267, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2df8f997:::obj267:head@111'267, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:2caf4871:::obj405:head local-lis/les=117/118 n=118 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.017+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.017+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2caf4871:::obj405:head local-lis/les=117/118 n=118 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:2df8f997:::obj267:head, version: 111'267, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2df8f997:::obj267:head@111'267, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.017+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2caf4871:::obj405:head local-lis/les=117/118 n=118 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:2df8f997:::obj267:head@111'267, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.017+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2caf4871:::obj405:head local-lis/les=117/118 n=118 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:2df8f997:::obj267:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.017+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2caf4871:::obj405:head local-lis/les=117/118 n=118 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:2df8f997:::obj267:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.017+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2caf4871:::obj405:head local-lis/les=117/118 n=118 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:2df8f997:::obj267:head 2026-02-11T23:31:28.017+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2caf4871:::obj405:head local-lis/les=117/118 n=118 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:2df8f997:::obj267:head v 111'267 2026-02-11T23:31:28.017+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2caf4871:::obj405:head local-lis/les=117/118 n=118 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.017+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.017+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:2df8f997:::obj267:head, version: 111'267, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2df8f997:::obj267:head@111'267, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000064 2026-02-11T23:31:28.017+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.017+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.017+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:2d59679a:::obj179:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:28.017+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:2d59679a:::obj179:head) v3 prio 15 cost 0 latency 0.000086 pg_backfill(progress 4.0 e 118/118 lb 4:2d59679a:::obj179:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:2caf4871:::obj405:head local-lis/les=117/118 n=118 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.017+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.017+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2caf4871:::obj405:head local-lis/les=117/118 n=118 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:2d59679a:::obj179:head) v3 2026-02-11T23:31:28.017+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2caf4871:::obj405:head local-lis/les=117/118 n=118 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:2d59679a:::obj179:head) 2026-02-11T23:31:28.017+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.017+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:2d59679a:::obj179:head) v3 finish latency 0.000013 2026-02-11T23:31:28.017+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2d59679a:::obj179:head local-lis/les=117/118 n=119 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.017+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.017+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.017+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2d59679a:::obj179:head local-lis/les=117/118 n=119 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.017+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:28.017+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:28.017+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:28.017+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63c4900 con 0x507ac00 2026-02-11T23:31:28.017+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:2df8f997:::obj267:head)]) -- 0x5f75b00 con 0x507ac00 2026-02-11T23:31:28.017+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.017+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 278 ==== MOSDPGPush(4.0 118/117 [PushOp(4:2e5f3a17:::obj520:head, version: 111'520, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e5f3a17:::obj520:head@111'520, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a24e0 con 0x507ac00 2026-02-11T23:31:28.017+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:2e5f3a17:::obj520:head, version: 111'520, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e5f3a17:::obj520:head@111'520, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 118 2026-02-11T23:31:28.017+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:2e5f3a17:::obj520:head, version: 111'520, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e5f3a17:::obj520:head@111'520, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:28.017+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.017+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 279 ==== pg_backfill(progress 4.0 e 118/118 lb 4:2df8f997:::obj267:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177b80 con 0x507ac00 2026-02-11T23:31:28.017+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:2e5f3a17:::obj520:head, version: 111'520, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e5f3a17:::obj520:head@111'520, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:28.017+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:2df8f997:::obj267:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:28.017+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:2df8f997:::obj267:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:28.017+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.017+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:2e5f3a17:::obj520:head, version: 111'520, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e5f3a17:::obj520:head@111'520, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:28.017+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:2e5f3a17:::obj520:head, version: 111'520, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e5f3a17:::obj520:head@111'520, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 118/117 [PushOp(4:2e5f3a17:::obj520:head, version: 111'520, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e5f3a17:::obj520:head@111'520, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:2d59679a:::obj179:head local-lis/les=117/118 n=119 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.017+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.017+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.017+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:2df8f997:::obj267:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:28.017+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2d59679a:::obj179:head local-lis/les=117/118 n=119 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:2e5f3a17:::obj520:head, version: 111'520, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e5f3a17:::obj520:head@111'520, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.017+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2d59679a:::obj179:head local-lis/les=117/118 n=119 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:2e5f3a17:::obj520:head@111'520, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.017+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2d59679a:::obj179:head local-lis/les=117/118 n=119 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:2e5f3a17:::obj520:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.017+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2d59679a:::obj179:head local-lis/les=117/118 n=119 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:2e5f3a17:::obj520:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.017+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2d59679a:::obj179:head local-lis/les=117/118 n=119 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:2e5f3a17:::obj520:head 2026-02-11T23:31:28.017+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2d59679a:::obj179:head local-lis/les=117/118 n=119 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:2e5f3a17:::obj520:head v 111'520 2026-02-11T23:31:28.017+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2d59679a:::obj179:head local-lis/les=117/118 n=119 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.017+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.017+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:2e5f3a17:::obj520:head, version: 111'520, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e5f3a17:::obj520:head@111'520, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:28.017+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.017+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.017+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:2df8f997:::obj267:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:28.017+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:2df8f997:::obj267:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 4.0 e 118/118 lb 4:2df8f997:::obj267:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:2d59679a:::obj179:head local-lis/les=117/118 n=119 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.017+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.017+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2d59679a:::obj179:head local-lis/les=117/118 n=119 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:2df8f997:::obj267:head) v3 2026-02-11T23:31:28.017+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2d59679a:::obj179:head local-lis/les=117/118 n=119 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:2df8f997:::obj267:head) 2026-02-11T23:31:28.017+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.017+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:2df8f997:::obj267:head) v3 finish latency 0.000014 2026-02-11T23:31:28.017+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2df8f997:::obj267:head local-lis/les=117/118 n=120 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.017+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.017+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.017+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2df8f997:::obj267:head local-lis/les=117/118 n=120 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.017+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:28.017+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:28.017+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:28.017+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63c4a80 con 0x507ac00 2026-02-11T23:31:28.017+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:2e5f3a17:::obj520:head)]) -- 0x649ef00 con 0x507ac00 2026-02-11T23:31:28.017+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.018+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 280 ==== MOSDPGPush(4.0 118/117 [PushOp(4:2e69a8fc:::obj598:head, version: 111'598, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e69a8fc:::obj598:head@111'598, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2000 con 0x507ac00 2026-02-11T23:31:28.018+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:2e69a8fc:::obj598:head, version: 111'598, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e69a8fc:::obj598:head@111'598, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 118 2026-02-11T23:31:28.018+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:2e69a8fc:::obj598:head, version: 111'598, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e69a8fc:::obj598:head@111'598, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:28.018+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.018+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 281 ==== pg_backfill(progress 4.0 e 118/118 lb 4:2e5f3a17:::obj520:head) ==== 1062+0+0 (unknown 0 0 0) 0x6178100 con 0x507ac00 2026-02-11T23:31:28.018+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:2e69a8fc:::obj598:head, version: 111'598, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e69a8fc:::obj598:head@111'598, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:28.018+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:2e5f3a17:::obj520:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:28.018+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.018+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:2e5f3a17:::obj520:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:28.018+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:2e69a8fc:::obj598:head, version: 111'598, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e69a8fc:::obj598:head@111'598, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:28.018+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.018+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:2e69a8fc:::obj598:head, version: 111'598, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e69a8fc:::obj598:head@111'598, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 118/117 [PushOp(4:2e69a8fc:::obj598:head, version: 111'598, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e69a8fc:::obj598:head@111'598, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:2df8f997:::obj267:head local-lis/les=117/118 n=120 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.018+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.018+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2df8f997:::obj267:head local-lis/les=117/118 n=120 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:2e69a8fc:::obj598:head, version: 111'598, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e69a8fc:::obj598:head@111'598, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.018+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2df8f997:::obj267:head local-lis/les=117/118 n=120 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:2e69a8fc:::obj598:head@111'598, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.018+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.018+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:2e5f3a17:::obj520:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:28.018+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2df8f997:::obj267:head local-lis/les=117/118 n=120 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:2e69a8fc:::obj598:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.018+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2df8f997:::obj267:head local-lis/les=117/118 n=120 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:2e69a8fc:::obj598:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.018+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2df8f997:::obj267:head local-lis/les=117/118 n=120 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:2e69a8fc:::obj598:head 2026-02-11T23:31:28.018+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2df8f997:::obj267:head local-lis/les=117/118 n=120 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:2e69a8fc:::obj598:head v 111'598 2026-02-11T23:31:28.018+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2df8f997:::obj267:head local-lis/les=117/118 n=120 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.018+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.018+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:2e69a8fc:::obj598:head, version: 111'598, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e69a8fc:::obj598:head@111'598, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:28.018+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.018+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.018+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:2e5f3a17:::obj520:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:28.018+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:2e5f3a17:::obj520:head) v3 prio 15 cost 0 latency 0.000079 pg_backfill(progress 4.0 e 118/118 lb 4:2e5f3a17:::obj520:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:2df8f997:::obj267:head local-lis/les=117/118 n=120 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.018+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.018+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2df8f997:::obj267:head local-lis/les=117/118 n=120 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:2e5f3a17:::obj520:head) v3 2026-02-11T23:31:28.018+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2df8f997:::obj267:head local-lis/les=117/118 n=120 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:2e5f3a17:::obj520:head) 2026-02-11T23:31:28.018+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.018+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:2e5f3a17:::obj520:head) v3 finish latency 0.000015 2026-02-11T23:31:28.018+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2e5f3a17:::obj520:head local-lis/les=117/118 n=121 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.018+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.018+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.018+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2e5f3a17:::obj520:head local-lis/les=117/118 n=121 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.018+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:28.018+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:28.018+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:28.018+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63c4c00 con 0x507ac00 2026-02-11T23:31:28.018+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.018+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:2e69a8fc:::obj598:head)]) -- 0x649ec00 con 0x507ac00 2026-02-11T23:31:28.018+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.019+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 282 ==== MOSDPGPush(4.0 118/117 [PushOp(4:2e72dfbc:::obj232:head, version: 111'232, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e72dfbc:::obj232:head@111'232, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f1a00 con 0x507ac00 2026-02-11T23:31:28.019+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:2e72dfbc:::obj232:head, version: 111'232, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e72dfbc:::obj232:head@111'232, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 118 2026-02-11T23:31:28.019+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:2e72dfbc:::obj232:head, version: 111'232, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e72dfbc:::obj232:head@111'232, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:28.019+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.019+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 283 ==== pg_backfill(progress 4.0 e 118/118 lb 4:2e69a8fc:::obj598:head) ==== 1062+0+0 (unknown 0 0 0) 0x6179180 con 0x507ac00 2026-02-11T23:31:28.019+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:2e69a8fc:::obj598:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:28.019+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:2e72dfbc:::obj232:head, version: 111'232, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e72dfbc:::obj232:head@111'232, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:28.019+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:2e69a8fc:::obj598:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:28.019+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.019+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:2e69a8fc:::obj598:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:28.019+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.019+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:2e72dfbc:::obj232:head, version: 111'232, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e72dfbc:::obj232:head@111'232, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:28.019+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:2e72dfbc:::obj232:head, version: 111'232, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e72dfbc:::obj232:head@111'232, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000047 MOSDPGPush(4.0 118/117 [PushOp(4:2e72dfbc:::obj232:head, version: 111'232, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e72dfbc:::obj232:head@111'232, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:2e5f3a17:::obj520:head local-lis/les=117/118 n=121 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.019+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.019+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2e5f3a17:::obj520:head local-lis/les=117/118 n=121 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:2e72dfbc:::obj232:head, version: 111'232, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e72dfbc:::obj232:head@111'232, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.019+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2e5f3a17:::obj520:head local-lis/les=117/118 n=121 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:2e72dfbc:::obj232:head@111'232, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.019+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2e5f3a17:::obj520:head local-lis/les=117/118 n=121 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:2e72dfbc:::obj232:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.019+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2e5f3a17:::obj520:head local-lis/les=117/118 n=121 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:2e72dfbc:::obj232:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.019+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2e5f3a17:::obj520:head local-lis/les=117/118 n=121 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:2e72dfbc:::obj232:head 2026-02-11T23:31:28.019+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2e5f3a17:::obj520:head local-lis/les=117/118 n=121 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:2e72dfbc:::obj232:head v 111'232 2026-02-11T23:31:28.019+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2e5f3a17:::obj520:head local-lis/les=117/118 n=121 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.019+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.019+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:2e72dfbc:::obj232:head, version: 111'232, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e72dfbc:::obj232:head@111'232, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:28.019+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.019+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.019+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:2e69a8fc:::obj598:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:28.019+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:2e69a8fc:::obj598:head) v3 prio 15 cost 0 latency 0.000093 pg_backfill(progress 4.0 e 118/118 lb 4:2e69a8fc:::obj598:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:2e5f3a17:::obj520:head local-lis/les=117/118 n=121 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.019+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.019+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2e5f3a17:::obj520:head local-lis/les=117/118 n=121 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:2e69a8fc:::obj598:head) v3 2026-02-11T23:31:28.019+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2e5f3a17:::obj520:head local-lis/les=117/118 n=121 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:2e69a8fc:::obj598:head) 2026-02-11T23:31:28.019+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.019+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:2e69a8fc:::obj598:head) v3 finish latency 0.000014 2026-02-11T23:31:28.019+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2e69a8fc:::obj598:head local-lis/les=117/118 n=122 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.019+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.019+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.019+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2e69a8fc:::obj598:head local-lis/les=117/118 n=122 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.019+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:28.019+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:28.019+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:28.019+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63c4d80 con 0x507ac00 2026-02-11T23:31:28.019+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:2e72dfbc:::obj232:head)]) -- 0x649f080 con 0x507ac00 2026-02-11T23:31:28.019+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.019+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 284 ==== MOSDPGPush(4.0 118/117 [PushOp(4:2eeda62f:::obj154:head, version: 111'154, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2eeda62f:::obj154:head@111'154, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2b60 con 0x507ac00 2026-02-11T23:31:28.019+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:2eeda62f:::obj154:head, version: 111'154, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2eeda62f:::obj154:head@111'154, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 118 2026-02-11T23:31:28.019+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:2eeda62f:::obj154:head, version: 111'154, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2eeda62f:::obj154:head@111'154, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:28.019+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.019+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 285 ==== pg_backfill(progress 4.0 e 118/118 lb 4:2e72dfbc:::obj232:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2db80 con 0x507ac00 2026-02-11T23:31:28.019+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:2eeda62f:::obj154:head, version: 111'154, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2eeda62f:::obj154:head@111'154, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:28.019+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:2e72dfbc:::obj232:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:28.019+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:2e72dfbc:::obj232:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:28.019+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.019+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:2eeda62f:::obj154:head, version: 111'154, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2eeda62f:::obj154:head@111'154, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:28.019+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:2eeda62f:::obj154:head, version: 111'154, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2eeda62f:::obj154:head@111'154, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(4.0 118/117 [PushOp(4:2eeda62f:::obj154:head, version: 111'154, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2eeda62f:::obj154:head@111'154, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:2e69a8fc:::obj598:head local-lis/les=117/118 n=122 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.019+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.019+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.019+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:2e72dfbc:::obj232:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:28.019+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2e69a8fc:::obj598:head local-lis/les=117/118 n=122 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:2eeda62f:::obj154:head, version: 111'154, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2eeda62f:::obj154:head@111'154, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.019+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2e69a8fc:::obj598:head local-lis/les=117/118 n=122 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:2eeda62f:::obj154:head@111'154, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.019+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2e69a8fc:::obj598:head local-lis/les=117/118 n=122 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:2eeda62f:::obj154:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.019+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2e69a8fc:::obj598:head local-lis/les=117/118 n=122 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:2eeda62f:::obj154:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.019+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2e69a8fc:::obj598:head local-lis/les=117/118 n=122 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:2eeda62f:::obj154:head 2026-02-11T23:31:28.019+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2e69a8fc:::obj598:head local-lis/les=117/118 n=122 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:2eeda62f:::obj154:head v 111'154 2026-02-11T23:31:28.019+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2e69a8fc:::obj598:head local-lis/les=117/118 n=122 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.019+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.019+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:2eeda62f:::obj154:head, version: 111'154, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2eeda62f:::obj154:head@111'154, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:28.019+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.019+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.019+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:2e72dfbc:::obj232:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:28.019+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:2e72dfbc:::obj232:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 4.0 e 118/118 lb 4:2e72dfbc:::obj232:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:2e69a8fc:::obj598:head local-lis/les=117/118 n=122 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.019+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.019+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2e69a8fc:::obj598:head local-lis/les=117/118 n=122 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:2e72dfbc:::obj232:head) v3 2026-02-11T23:31:28.019+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2e69a8fc:::obj598:head local-lis/les=117/118 n=122 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:2e72dfbc:::obj232:head) 2026-02-11T23:31:28.019+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.019+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:2e72dfbc:::obj232:head) v3 finish latency 0.000015 2026-02-11T23:31:28.019+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2e72dfbc:::obj232:head local-lis/les=117/118 n=123 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.019+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.019+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2e72dfbc:::obj232:head local-lis/les=117/118 n=123 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.019+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:28.019+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:28.019+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.019+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:28.019+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63c4f00 con 0x507ac00 2026-02-11T23:31:28.019+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:2eeda62f:::obj154:head)]) -- 0x5f2dc80 con 0x507ac00 2026-02-11T23:31:28.019+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.020+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 286 ==== MOSDPGPush(4.0 118/117 [PushOp(4:2f478301:::obj479:head, version: 111'479, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2f478301:::obj479:head@111'479, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x554e680 con 0x507ac00 2026-02-11T23:31:28.020+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:2f478301:::obj479:head, version: 111'479, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2f478301:::obj479:head@111'479, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 118 2026-02-11T23:31:28.020+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:2f478301:::obj479:head, version: 111'479, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2f478301:::obj479:head@111'479, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:28.020+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.020+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 287 ==== pg_backfill(progress 4.0 e 118/118 lb 4:2eeda62f:::obj154:head) ==== 1062+0+0 (unknown 0 0 0) 0x6178c00 con 0x507ac00 2026-02-11T23:31:28.020+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:2eeda62f:::obj154:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:28.020+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:2f478301:::obj479:head, version: 111'479, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2f478301:::obj479:head@111'479, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:28.020+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:2eeda62f:::obj154:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:28.020+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.020+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:2f478301:::obj479:head, version: 111'479, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2f478301:::obj479:head@111'479, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:28.020+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:2f478301:::obj479:head, version: 111'479, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2f478301:::obj479:head@111'479, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 118/117 [PushOp(4:2f478301:::obj479:head, version: 111'479, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2f478301:::obj479:head@111'479, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:2e72dfbc:::obj232:head local-lis/les=117/118 n=123 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.020+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.020+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.020+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:2eeda62f:::obj154:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:28.020+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2e72dfbc:::obj232:head local-lis/les=117/118 n=123 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:2f478301:::obj479:head, version: 111'479, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2f478301:::obj479:head@111'479, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.020+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2e72dfbc:::obj232:head local-lis/les=117/118 n=123 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:2f478301:::obj479:head@111'479, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.020+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2e72dfbc:::obj232:head local-lis/les=117/118 n=123 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:2f478301:::obj479:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.020+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2e72dfbc:::obj232:head local-lis/les=117/118 n=123 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:2f478301:::obj479:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.020+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2e72dfbc:::obj232:head local-lis/les=117/118 n=123 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:2f478301:::obj479:head 2026-02-11T23:31:28.020+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2e72dfbc:::obj232:head local-lis/les=117/118 n=123 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:2f478301:::obj479:head v 111'479 2026-02-11T23:31:28.020+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2e72dfbc:::obj232:head local-lis/les=117/118 n=123 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.020+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.020+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:2f478301:::obj479:head, version: 111'479, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2f478301:::obj479:head@111'479, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:28.020+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.020+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.020+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:2eeda62f:::obj154:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:28.020+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:2eeda62f:::obj154:head) v3 prio 15 cost 0 latency 0.000079 pg_backfill(progress 4.0 e 118/118 lb 4:2eeda62f:::obj154:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:2e72dfbc:::obj232:head local-lis/les=117/118 n=123 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.020+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.020+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2e72dfbc:::obj232:head local-lis/les=117/118 n=123 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:2eeda62f:::obj154:head) v3 2026-02-11T23:31:28.020+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2e72dfbc:::obj232:head local-lis/les=117/118 n=123 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:2eeda62f:::obj154:head) 2026-02-11T23:31:28.020+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.020+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:2eeda62f:::obj154:head) v3 finish latency 0.000017 2026-02-11T23:31:28.020+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2eeda62f:::obj154:head local-lis/les=117/118 n=124 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.020+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.020+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.020+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2eeda62f:::obj154:head local-lis/les=117/118 n=124 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.020+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:28.020+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:28.020+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:28.020+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63c5080 con 0x507ac00 2026-02-11T23:31:28.020+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:2f478301:::obj479:head)]) -- 0x649f200 con 0x507ac00 2026-02-11T23:31:28.020+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.021+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 288 ==== MOSDPGPush(4.0 118/117 [PushOp(4:303427ce:::obj485:head, version: 111'485, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:303427ce:::obj485:head@111'485, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a3a00 con 0x507ac00 2026-02-11T23:31:28.021+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:303427ce:::obj485:head, version: 111'485, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:303427ce:::obj485:head@111'485, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 118 2026-02-11T23:31:28.021+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:303427ce:::obj485:head, version: 111'485, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:303427ce:::obj485:head@111'485, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:28.021+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.021+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 289 ==== pg_backfill(progress 4.0 e 118/118 lb 4:2f478301:::obj479:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9b180 con 0x507ac00 2026-02-11T23:31:28.021+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:2f478301:::obj479:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:28.021+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:303427ce:::obj485:head, version: 111'485, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:303427ce:::obj485:head@111'485, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:28.021+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:2f478301:::obj479:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:28.021+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.021+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:303427ce:::obj485:head, version: 111'485, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:303427ce:::obj485:head@111'485, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:28.021+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.021+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:2f478301:::obj479:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:28.021+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:303427ce:::obj485:head, version: 111'485, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:303427ce:::obj485:head@111'485, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(4.0 118/117 [PushOp(4:303427ce:::obj485:head, version: 111'485, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:303427ce:::obj485:head@111'485, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:2eeda62f:::obj154:head local-lis/les=117/118 n=124 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.021+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.021+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2eeda62f:::obj154:head local-lis/les=117/118 n=124 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:303427ce:::obj485:head, version: 111'485, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:303427ce:::obj485:head@111'485, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.021+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2eeda62f:::obj154:head local-lis/les=117/118 n=124 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:303427ce:::obj485:head@111'485, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.021+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2eeda62f:::obj154:head local-lis/les=117/118 n=124 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:303427ce:::obj485:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.021+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2eeda62f:::obj154:head local-lis/les=117/118 n=124 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:303427ce:::obj485:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.021+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2eeda62f:::obj154:head local-lis/les=117/118 n=124 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:303427ce:::obj485:head 2026-02-11T23:31:28.021+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2eeda62f:::obj154:head local-lis/les=117/118 n=124 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:303427ce:::obj485:head v 111'485 2026-02-11T23:31:28.021+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2eeda62f:::obj154:head local-lis/les=117/118 n=124 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.021+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.021+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:303427ce:::obj485:head, version: 111'485, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:303427ce:::obj485:head@111'485, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000062 2026-02-11T23:31:28.021+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.021+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.021+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:2f478301:::obj479:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:28.021+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:2f478301:::obj479:head) v3 prio 15 cost 0 latency 0.000081 pg_backfill(progress 4.0 e 118/118 lb 4:2f478301:::obj479:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:2eeda62f:::obj154:head local-lis/les=117/118 n=124 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.021+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.021+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2eeda62f:::obj154:head local-lis/les=117/118 n=124 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:2f478301:::obj479:head) v3 2026-02-11T23:31:28.021+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2eeda62f:::obj154:head local-lis/les=117/118 n=124 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:2f478301:::obj479:head) 2026-02-11T23:31:28.021+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.021+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:2f478301:::obj479:head) v3 finish latency 0.000016 2026-02-11T23:31:28.021+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2f478301:::obj479:head local-lis/les=117/118 n=125 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.021+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.021+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.021+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2f478301:::obj479:head local-lis/les=117/118 n=125 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.021+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:28.021+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:28.021+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:28.021+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63c5200 con 0x507ac00 2026-02-11T23:31:28.021+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:303427ce:::obj485:head)]) -- 0x5f75c80 con 0x507ac00 2026-02-11T23:31:28.021+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.021+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 290 ==== MOSDPGPush(4.0 118/117 [PushOp(4:320b4068:::obj138:head, version: 111'138, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:320b4068:::obj138:head@111'138, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5617380 con 0x507ac00 2026-02-11T23:31:28.021+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:320b4068:::obj138:head, version: 111'138, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:320b4068:::obj138:head@111'138, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 118 2026-02-11T23:31:28.021+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:320b4068:::obj138:head, version: 111'138, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:320b4068:::obj138:head@111'138, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:28.021+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.021+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:320b4068:::obj138:head, version: 111'138, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:320b4068:::obj138:head@111'138, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:28.021+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 291 ==== pg_backfill(progress 4.0 e 118/118 lb 4:303427ce:::obj485:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98000 con 0x507ac00 2026-02-11T23:31:28.021+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:303427ce:::obj485:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 118 2026-02-11T23:31:28.021+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.021+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:303427ce:::obj485:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:28.021+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:320b4068:::obj138:head, version: 111'138, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:320b4068:::obj138:head@111'138, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:28.021+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.021+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:320b4068:::obj138:head, version: 111'138, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:320b4068:::obj138:head@111'138, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(4.0 118/117 [PushOp(4:320b4068:::obj138:head, version: 111'138, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:320b4068:::obj138:head@111'138, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:2f478301:::obj479:head local-lis/les=117/118 n=125 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.021+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.021+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2f478301:::obj479:head local-lis/les=117/118 n=125 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:320b4068:::obj138:head, version: 111'138, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:320b4068:::obj138:head@111'138, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.021+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2f478301:::obj479:head local-lis/les=117/118 n=125 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:320b4068:::obj138:head@111'138, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.021+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.021+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:303427ce:::obj485:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:28.021+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2f478301:::obj479:head local-lis/les=117/118 n=125 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:320b4068:::obj138:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.021+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2f478301:::obj479:head local-lis/les=117/118 n=125 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:320b4068:::obj138:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.021+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2f478301:::obj479:head local-lis/les=117/118 n=125 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:320b4068:::obj138:head 2026-02-11T23:31:28.021+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2f478301:::obj479:head local-lis/les=117/118 n=125 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:320b4068:::obj138:head v 111'138 2026-02-11T23:31:28.021+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2f478301:::obj479:head local-lis/les=117/118 n=125 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.021+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.021+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:320b4068:::obj138:head, version: 111'138, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:320b4068:::obj138:head@111'138, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:28.021+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.021+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.021+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:303427ce:::obj485:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:28.021+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:303427ce:::obj485:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 4.0 e 118/118 lb 4:303427ce:::obj485:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:2f478301:::obj479:head local-lis/les=117/118 n=125 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.021+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.021+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2f478301:::obj479:head local-lis/les=117/118 n=125 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:303427ce:::obj485:head) v3 2026-02-11T23:31:28.021+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:2f478301:::obj479:head local-lis/les=117/118 n=125 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:303427ce:::obj485:head) 2026-02-11T23:31:28.021+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.021+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:303427ce:::obj485:head) v3 finish latency 0.000014 2026-02-11T23:31:28.021+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:303427ce:::obj485:head local-lis/les=117/118 n=126 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.021+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.021+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:303427ce:::obj485:head local-lis/les=117/118 n=126 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.021+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:28.021+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:28.021+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:28.021+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.021+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63c5380 con 0x507ac00 2026-02-11T23:31:28.021+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:320b4068:::obj138:head)]) -- 0x649f380 con 0x507ac00 2026-02-11T23:31:28.021+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.022+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 292 ==== MOSDPGPush(4.0 118/117 [PushOp(4:325eabfb:::obj310:head, version: 111'310, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:325eabfb:::obj310:head@111'310, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56caea0 con 0x507ac00 2026-02-11T23:31:28.022+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:325eabfb:::obj310:head, version: 111'310, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:325eabfb:::obj310:head@111'310, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 118 2026-02-11T23:31:28.022+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:325eabfb:::obj310:head, version: 111'310, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:325eabfb:::obj310:head@111'310, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:28.022+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.022+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 293 ==== pg_backfill(progress 4.0 e 118/118 lb 4:320b4068:::obj138:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98580 con 0x507ac00 2026-02-11T23:31:28.022+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:320b4068:::obj138:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:28.022+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:325eabfb:::obj310:head, version: 111'310, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:325eabfb:::obj310:head@111'310, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:28.022+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:320b4068:::obj138:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:28.022+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.022+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:325eabfb:::obj310:head, version: 111'310, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:325eabfb:::obj310:head@111'310, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:28.022+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.022+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:320b4068:::obj138:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:28.022+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:325eabfb:::obj310:head, version: 111'310, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:325eabfb:::obj310:head@111'310, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 118/117 [PushOp(4:325eabfb:::obj310:head, version: 111'310, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:325eabfb:::obj310:head@111'310, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:303427ce:::obj485:head local-lis/les=117/118 n=126 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.022+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.022+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:303427ce:::obj485:head local-lis/les=117/118 n=126 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:325eabfb:::obj310:head, version: 111'310, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:325eabfb:::obj310:head@111'310, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.022+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:303427ce:::obj485:head local-lis/les=117/118 n=126 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:325eabfb:::obj310:head@111'310, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.022+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:303427ce:::obj485:head local-lis/les=117/118 n=126 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:325eabfb:::obj310:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.022+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:303427ce:::obj485:head local-lis/les=117/118 n=126 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:325eabfb:::obj310:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.022+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:303427ce:::obj485:head local-lis/les=117/118 n=126 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:325eabfb:::obj310:head 2026-02-11T23:31:28.022+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:303427ce:::obj485:head local-lis/les=117/118 n=126 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:325eabfb:::obj310:head v 111'310 2026-02-11T23:31:28.022+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:303427ce:::obj485:head local-lis/les=117/118 n=126 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.022+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.022+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:325eabfb:::obj310:head, version: 111'310, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:325eabfb:::obj310:head@111'310, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:28.022+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.022+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.022+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:320b4068:::obj138:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:28.022+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:320b4068:::obj138:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 4.0 e 118/118 lb 4:320b4068:::obj138:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:303427ce:::obj485:head local-lis/les=117/118 n=126 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.022+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.022+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:303427ce:::obj485:head local-lis/les=117/118 n=126 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:320b4068:::obj138:head) v3 2026-02-11T23:31:28.022+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:303427ce:::obj485:head local-lis/les=117/118 n=126 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:320b4068:::obj138:head) 2026-02-11T23:31:28.022+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.022+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:320b4068:::obj138:head) v3 finish latency 0.000019 2026-02-11T23:31:28.022+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:320b4068:::obj138:head local-lis/les=117/118 n=127 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.022+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.022+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.022+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:320b4068:::obj138:head local-lis/les=117/118 n=127 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.022+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:28.022+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:28.022+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:28.022+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63c5500 con 0x507ac00 2026-02-11T23:31:28.022+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:325eabfb:::obj310:head)]) -- 0x63c4180 con 0x507ac00 2026-02-11T23:31:28.022+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.023+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 294 ==== MOSDPGPush(4.0 118/117 [PushOp(4:3263b485:::obj202:head, version: 111'202, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3263b485:::obj202:head@111'202, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56ca9c0 con 0x507ac00 2026-02-11T23:31:28.023+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:3263b485:::obj202:head, version: 111'202, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3263b485:::obj202:head@111'202, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000021 epoch 118 2026-02-11T23:31:28.023+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:3263b485:::obj202:head, version: 111'202, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3263b485:::obj202:head@111'202, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:28.023+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.023+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:3263b485:::obj202:head, version: 111'202, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3263b485:::obj202:head@111'202, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:28.023+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 295 ==== pg_backfill(progress 4.0 e 118/118 lb 4:325eabfb:::obj310:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176b00 con 0x507ac00 2026-02-11T23:31:28.023+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.023+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:325eabfb:::obj310:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 118 2026-02-11T23:31:28.023+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:325eabfb:::obj310:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:28.023+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:3263b485:::obj202:head, version: 111'202, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3263b485:::obj202:head@111'202, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:28.023+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.023+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:3263b485:::obj202:head, version: 111'202, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3263b485:::obj202:head@111'202, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000048 MOSDPGPush(4.0 118/117 [PushOp(4:3263b485:::obj202:head, version: 111'202, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3263b485:::obj202:head@111'202, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:320b4068:::obj138:head local-lis/les=117/118 n=127 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.023+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.023+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:320b4068:::obj138:head local-lis/les=117/118 n=127 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:3263b485:::obj202:head, version: 111'202, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3263b485:::obj202:head@111'202, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.023+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.023+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:325eabfb:::obj310:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:28.023+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:320b4068:::obj138:head local-lis/les=117/118 n=127 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:3263b485:::obj202:head@111'202, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.023+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:320b4068:::obj138:head local-lis/les=117/118 n=127 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:3263b485:::obj202:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.023+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:320b4068:::obj138:head local-lis/les=117/118 n=127 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:3263b485:::obj202:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.023+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:320b4068:::obj138:head local-lis/les=117/118 n=127 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:3263b485:::obj202:head 2026-02-11T23:31:28.023+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:320b4068:::obj138:head local-lis/les=117/118 n=127 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:3263b485:::obj202:head v 111'202 2026-02-11T23:31:28.023+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:320b4068:::obj138:head local-lis/les=117/118 n=127 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.023+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.023+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:3263b485:::obj202:head, version: 111'202, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3263b485:::obj202:head@111'202, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000073 2026-02-11T23:31:28.023+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.023+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.023+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:325eabfb:::obj310:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:28.023+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:325eabfb:::obj310:head) v3 prio 15 cost 0 latency 0.000093 pg_backfill(progress 4.0 e 118/118 lb 4:325eabfb:::obj310:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:320b4068:::obj138:head local-lis/les=117/118 n=127 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.023+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.023+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:320b4068:::obj138:head local-lis/les=117/118 n=127 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:325eabfb:::obj310:head) v3 2026-02-11T23:31:28.023+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:320b4068:::obj138:head local-lis/les=117/118 n=127 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:325eabfb:::obj310:head) 2026-02-11T23:31:28.023+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.023+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:325eabfb:::obj310:head) v3 finish latency 0.000018 2026-02-11T23:31:28.023+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:325eabfb:::obj310:head local-lis/les=117/118 n=128 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.023+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.023+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.023+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:325eabfb:::obj310:head local-lis/les=117/118 n=128 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.023+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:28.023+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:28.023+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:28.023+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63c5680 con 0x507ac00 2026-02-11T23:31:28.023+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:3263b485:::obj202:head)]) -- 0x63c4000 con 0x507ac00 2026-02-11T23:31:28.023+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.023+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 296 ==== MOSDPGPush(4.0 118/117 [PushOp(4:326c82ab:::obj527:head, version: 111'527, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:326c82ab:::obj527:head@111'527, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f0d00 con 0x507ac00 2026-02-11T23:31:28.023+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:326c82ab:::obj527:head, version: 111'527, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:326c82ab:::obj527:head@111'527, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 118 2026-02-11T23:31:28.023+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:326c82ab:::obj527:head, version: 111'527, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:326c82ab:::obj527:head@111'527, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:28.023+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.023+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 297 ==== pg_backfill(progress 4.0 e 118/118 lb 4:3263b485:::obj202:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f84000 con 0x507ac00 2026-02-11T23:31:28.023+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:326c82ab:::obj527:head, version: 111'527, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:326c82ab:::obj527:head@111'527, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:28.023+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:3263b485:::obj202:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 118 2026-02-11T23:31:28.023+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:3263b485:::obj202:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:28.023+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.023+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:326c82ab:::obj527:head, version: 111'527, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:326c82ab:::obj527:head@111'527, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:28.023+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.023+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:3263b485:::obj202:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:28.023+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:326c82ab:::obj527:head, version: 111'527, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:326c82ab:::obj527:head@111'527, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000041 MOSDPGPush(4.0 118/117 [PushOp(4:326c82ab:::obj527:head, version: 111'527, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:326c82ab:::obj527:head@111'527, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:325eabfb:::obj310:head local-lis/les=117/118 n=128 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.023+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.023+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:325eabfb:::obj310:head local-lis/les=117/118 n=128 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:326c82ab:::obj527:head, version: 111'527, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:326c82ab:::obj527:head@111'527, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.023+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:325eabfb:::obj310:head local-lis/les=117/118 n=128 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:326c82ab:::obj527:head@111'527, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.023+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:325eabfb:::obj310:head local-lis/les=117/118 n=128 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:326c82ab:::obj527:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.023+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:325eabfb:::obj310:head local-lis/les=117/118 n=128 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:326c82ab:::obj527:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.023+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:325eabfb:::obj310:head local-lis/les=117/118 n=128 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:326c82ab:::obj527:head 2026-02-11T23:31:28.023+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:325eabfb:::obj310:head local-lis/les=117/118 n=128 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:326c82ab:::obj527:head v 111'527 2026-02-11T23:31:28.023+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:325eabfb:::obj310:head local-lis/les=117/118 n=128 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.023+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.023+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:326c82ab:::obj527:head, version: 111'527, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:326c82ab:::obj527:head@111'527, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000065 2026-02-11T23:31:28.023+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.023+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.023+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:3263b485:::obj202:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:28.023+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:3263b485:::obj202:head) v3 prio 15 cost 0 latency 0.000087 pg_backfill(progress 4.0 e 118/118 lb 4:3263b485:::obj202:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:325eabfb:::obj310:head local-lis/les=117/118 n=128 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.023+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.023+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:325eabfb:::obj310:head local-lis/les=117/118 n=128 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:3263b485:::obj202:head) v3 2026-02-11T23:31:28.023+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:325eabfb:::obj310:head local-lis/les=117/118 n=128 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:3263b485:::obj202:head) 2026-02-11T23:31:28.023+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.023+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:3263b485:::obj202:head) v3 finish latency 0.000018 2026-02-11T23:31:28.023+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3263b485:::obj202:head local-lis/les=117/118 n=129 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.023+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.024+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.024+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3263b485:::obj202:head local-lis/les=117/118 n=129 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.024+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:28.024+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:28.024+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:28.024+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63c5800 con 0x507ac00 2026-02-11T23:31:28.024+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:326c82ab:::obj527:head)]) -- 0x63c9e00 con 0x507ac00 2026-02-11T23:31:28.024+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.024+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 298 ==== MOSDPGPush(4.0 118/117 [PushOp(4:3286c571:::obj350:head, version: 111'350, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3286c571:::obj350:head@111'350, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c4e0 con 0x507ac00 2026-02-11T23:31:28.024+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:3286c571:::obj350:head, version: 111'350, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3286c571:::obj350:head@111'350, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 118 2026-02-11T23:31:28.024+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:3286c571:::obj350:head, version: 111'350, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3286c571:::obj350:head@111'350, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:28.024+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.024+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:3286c571:::obj350:head, version: 111'350, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3286c571:::obj350:head@111'350, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:28.024+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 299 ==== pg_backfill(progress 4.0 e 118/118 lb 4:326c82ab:::obj527:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99b80 con 0x507ac00 2026-02-11T23:31:28.024+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:326c82ab:::obj527:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 118 2026-02-11T23:31:28.024+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.024+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:326c82ab:::obj527:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:28.024+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:3286c571:::obj350:head, version: 111'350, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3286c571:::obj350:head@111'350, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:28.024+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.024+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:3286c571:::obj350:head, version: 111'350, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3286c571:::obj350:head@111'350, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(4.0 118/117 [PushOp(4:3286c571:::obj350:head, version: 111'350, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3286c571:::obj350:head@111'350, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:3263b485:::obj202:head local-lis/les=117/118 n=129 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.024+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.024+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3263b485:::obj202:head local-lis/les=117/118 n=129 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:3286c571:::obj350:head, version: 111'350, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3286c571:::obj350:head@111'350, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.024+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3263b485:::obj202:head local-lis/les=117/118 n=129 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:3286c571:::obj350:head@111'350, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.024+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.024+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:326c82ab:::obj527:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:28.024+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3263b485:::obj202:head local-lis/les=117/118 n=129 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:3286c571:::obj350:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.024+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3263b485:::obj202:head local-lis/les=117/118 n=129 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:3286c571:::obj350:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.024+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3263b485:::obj202:head local-lis/les=117/118 n=129 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:3286c571:::obj350:head 2026-02-11T23:31:28.024+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3263b485:::obj202:head local-lis/les=117/118 n=129 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:3286c571:::obj350:head v 111'350 2026-02-11T23:31:28.024+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3263b485:::obj202:head local-lis/les=117/118 n=129 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.024+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.024+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:3286c571:::obj350:head, version: 111'350, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3286c571:::obj350:head@111'350, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:28.024+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.024+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.024+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:326c82ab:::obj527:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:28.024+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:326c82ab:::obj527:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 4.0 e 118/118 lb 4:326c82ab:::obj527:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:3263b485:::obj202:head local-lis/les=117/118 n=129 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.024+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.024+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3263b485:::obj202:head local-lis/les=117/118 n=129 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:326c82ab:::obj527:head) v3 2026-02-11T23:31:28.024+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3263b485:::obj202:head local-lis/les=117/118 n=129 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:326c82ab:::obj527:head) 2026-02-11T23:31:28.024+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.024+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:326c82ab:::obj527:head) v3 finish latency 0.000017 2026-02-11T23:31:28.024+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:326c82ab:::obj527:head local-lis/les=117/118 n=130 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.024+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.024+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:326c82ab:::obj527:head local-lis/les=117/118 n=130 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.024+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:28.024+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:28.024+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:28.024+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.024+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63c5980 con 0x507ac00 2026-02-11T23:31:28.024+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:3286c571:::obj350:head)]) -- 0x63c9c80 con 0x507ac00 2026-02-11T23:31:28.024+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.025+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 300 ==== MOSDPGPush(4.0 118/117 [PushOp(4:32db648a:::obj568:head, version: 111'568, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:32db648a:::obj568:head@111'568, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f0ea0 con 0x507ac00 2026-02-11T23:31:28.025+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:32db648a:::obj568:head, version: 111'568, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:32db648a:::obj568:head@111'568, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 118 2026-02-11T23:31:28.025+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:32db648a:::obj568:head, version: 111'568, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:32db648a:::obj568:head@111'568, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:28.025+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.025+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 301 ==== pg_backfill(progress 4.0 e 118/118 lb 4:3286c571:::obj350:head) ==== 1062+0+0 (unknown 0 0 0) 0x5982b00 con 0x507ac00 2026-02-11T23:31:28.025+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:32db648a:::obj568:head, version: 111'568, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:32db648a:::obj568:head@111'568, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:28.025+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:3286c571:::obj350:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:28.025+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.025+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:3286c571:::obj350:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:28.025+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:32db648a:::obj568:head, version: 111'568, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:32db648a:::obj568:head@111'568, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:28.025+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.025+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:32db648a:::obj568:head, version: 111'568, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:32db648a:::obj568:head@111'568, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 118/117 [PushOp(4:32db648a:::obj568:head, version: 111'568, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:32db648a:::obj568:head@111'568, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:326c82ab:::obj527:head local-lis/les=117/118 n=130 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.025+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:3286c571:::obj350:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:28.025+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.025+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:326c82ab:::obj527:head local-lis/les=117/118 n=130 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:32db648a:::obj568:head, version: 111'568, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:32db648a:::obj568:head@111'568, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.025+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:326c82ab:::obj527:head local-lis/les=117/118 n=130 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:32db648a:::obj568:head@111'568, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.025+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:326c82ab:::obj527:head local-lis/les=117/118 n=130 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:32db648a:::obj568:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.025+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:326c82ab:::obj527:head local-lis/les=117/118 n=130 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:32db648a:::obj568:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.025+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:326c82ab:::obj527:head local-lis/les=117/118 n=130 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:32db648a:::obj568:head 2026-02-11T23:31:28.025+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:326c82ab:::obj527:head local-lis/les=117/118 n=130 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:32db648a:::obj568:head v 111'568 2026-02-11T23:31:28.025+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:326c82ab:::obj527:head local-lis/les=117/118 n=130 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.025+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.025+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:32db648a:::obj568:head, version: 111'568, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:32db648a:::obj568:head@111'568, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:28.025+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.025+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.025+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:3286c571:::obj350:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:28.025+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:3286c571:::obj350:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 4.0 e 118/118 lb 4:3286c571:::obj350:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:326c82ab:::obj527:head local-lis/les=117/118 n=130 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.025+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.025+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:326c82ab:::obj527:head local-lis/les=117/118 n=130 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:3286c571:::obj350:head) v3 2026-02-11T23:31:28.025+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:326c82ab:::obj527:head local-lis/les=117/118 n=130 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:3286c571:::obj350:head) 2026-02-11T23:31:28.025+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.025+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:3286c571:::obj350:head) v3 finish latency 0.000016 2026-02-11T23:31:28.025+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3286c571:::obj350:head local-lis/les=117/118 n=131 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.025+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.025+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3286c571:::obj350:head local-lis/les=117/118 n=131 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.025+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:28.025+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:28.025+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:28.025+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.025+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63c5b00 con 0x507ac00 2026-02-11T23:31:28.025+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:32db648a:::obj568:head)]) -- 0x63c9b00 con 0x507ac00 2026-02-11T23:31:28.025+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.025+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 302 ==== MOSDPGPush(4.0 118/117 [PushOp(4:3372ec01:::obj225:head, version: 111'225, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3372ec01:::obj225:head@111'225, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e261a0 con 0x507ac00 2026-02-11T23:31:28.025+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:3372ec01:::obj225:head, version: 111'225, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3372ec01:::obj225:head@111'225, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 118 2026-02-11T23:31:28.025+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:3372ec01:::obj225:head, version: 111'225, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3372ec01:::obj225:head@111'225, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:28.025+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.025+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 303 ==== pg_backfill(progress 4.0 e 118/118 lb 4:32db648a:::obj568:head) ==== 1062+0+0 (unknown 0 0 0) 0x4a2c000 con 0x507ac00 2026-02-11T23:31:28.025+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:32db648a:::obj568:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:28.025+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:3372ec01:::obj225:head, version: 111'225, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3372ec01:::obj225:head@111'225, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:28.025+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:32db648a:::obj568:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:28.025+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.025+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:3372ec01:::obj225:head, version: 111'225, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3372ec01:::obj225:head@111'225, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:28.025+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.025+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:32db648a:::obj568:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:28.025+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:3372ec01:::obj225:head, version: 111'225, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3372ec01:::obj225:head@111'225, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 118/117 [PushOp(4:3372ec01:::obj225:head, version: 111'225, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3372ec01:::obj225:head@111'225, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:3286c571:::obj350:head local-lis/les=117/118 n=131 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.025+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.025+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3286c571:::obj350:head local-lis/les=117/118 n=131 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:3372ec01:::obj225:head, version: 111'225, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3372ec01:::obj225:head@111'225, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.025+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3286c571:::obj350:head local-lis/les=117/118 n=131 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:3372ec01:::obj225:head@111'225, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.025+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3286c571:::obj350:head local-lis/les=117/118 n=131 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:3372ec01:::obj225:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.025+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3286c571:::obj350:head local-lis/les=117/118 n=131 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:3372ec01:::obj225:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.025+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3286c571:::obj350:head local-lis/les=117/118 n=131 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:3372ec01:::obj225:head 2026-02-11T23:31:28.025+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3286c571:::obj350:head local-lis/les=117/118 n=131 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:3372ec01:::obj225:head v 111'225 2026-02-11T23:31:28.025+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3286c571:::obj350:head local-lis/les=117/118 n=131 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.025+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.025+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:3372ec01:::obj225:head, version: 111'225, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3372ec01:::obj225:head@111'225, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000062 2026-02-11T23:31:28.025+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.025+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.025+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:32db648a:::obj568:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:28.025+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:32db648a:::obj568:head) v3 prio 15 cost 0 latency 0.000084 pg_backfill(progress 4.0 e 118/118 lb 4:32db648a:::obj568:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:3286c571:::obj350:head local-lis/les=117/118 n=131 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.025+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.025+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3286c571:::obj350:head local-lis/les=117/118 n=131 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:32db648a:::obj568:head) v3 2026-02-11T23:31:28.025+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3286c571:::obj350:head local-lis/les=117/118 n=131 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:32db648a:::obj568:head) 2026-02-11T23:31:28.025+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.025+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:32db648a:::obj568:head) v3 finish latency 0.000014 2026-02-11T23:31:28.025+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:32db648a:::obj568:head local-lis/les=117/118 n=132 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.025+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.026+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.026+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:32db648a:::obj568:head local-lis/les=117/118 n=132 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.026+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:28.026+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:28.026+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:28.026+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63c5c80 con 0x507ac00 2026-02-11T23:31:28.026+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:3372ec01:::obj225:head)]) -- 0x63c9980 con 0x507ac00 2026-02-11T23:31:28.026+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.026+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 304 ==== MOSDPGPush(4.0 118/117 [PushOp(4:339fd60b:::obj93:head, version: 111'93, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:339fd60b:::obj93:head@111'93, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x3814d00 con 0x507ac00 2026-02-11T23:31:28.026+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:339fd60b:::obj93:head, version: 111'93, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:339fd60b:::obj93:head@111'93, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 118 2026-02-11T23:31:28.026+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:339fd60b:::obj93:head, version: 111'93, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:339fd60b:::obj93:head@111'93, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:28.026+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.026+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 305 ==== pg_backfill(progress 4.0 e 118/118 lb 4:3372ec01:::obj225:head) ==== 1062+0+0 (unknown 0 0 0) 0x4a2cb00 con 0x507ac00 2026-02-11T23:31:28.026+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:3372ec01:::obj225:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:28.026+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:339fd60b:::obj93:head, version: 111'93, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:339fd60b:::obj93:head@111'93, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:28.026+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:3372ec01:::obj225:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:28.026+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.026+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:339fd60b:::obj93:head, version: 111'93, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:339fd60b:::obj93:head@111'93, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:28.026+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:339fd60b:::obj93:head, version: 111'93, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:339fd60b:::obj93:head@111'93, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(4.0 118/117 [PushOp(4:339fd60b:::obj93:head, version: 111'93, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:339fd60b:::obj93:head@111'93, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:32db648a:::obj568:head local-lis/les=117/118 n=132 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.026+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.026+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.026+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:3372ec01:::obj225:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:28.026+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:32db648a:::obj568:head local-lis/les=117/118 n=132 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:339fd60b:::obj93:head, version: 111'93, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:339fd60b:::obj93:head@111'93, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.026+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:32db648a:::obj568:head local-lis/les=117/118 n=132 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:339fd60b:::obj93:head@111'93, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.026+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:32db648a:::obj568:head local-lis/les=117/118 n=132 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:339fd60b:::obj93:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.026+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:32db648a:::obj568:head local-lis/les=117/118 n=132 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:339fd60b:::obj93:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.026+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:32db648a:::obj568:head local-lis/les=117/118 n=132 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:339fd60b:::obj93:head 2026-02-11T23:31:28.026+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:32db648a:::obj568:head local-lis/les=117/118 n=132 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:339fd60b:::obj93:head v 111'93 2026-02-11T23:31:28.026+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:32db648a:::obj568:head local-lis/les=117/118 n=132 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.026+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.026+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:339fd60b:::obj93:head, version: 111'93, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:339fd60b:::obj93:head@111'93, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:28.026+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.026+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.026+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:3372ec01:::obj225:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:28.026+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:3372ec01:::obj225:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 4.0 e 118/118 lb 4:3372ec01:::obj225:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:32db648a:::obj568:head local-lis/les=117/118 n=132 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.026+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.026+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:32db648a:::obj568:head local-lis/les=117/118 n=132 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:3372ec01:::obj225:head) v3 2026-02-11T23:31:28.026+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:32db648a:::obj568:head local-lis/les=117/118 n=132 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:3372ec01:::obj225:head) 2026-02-11T23:31:28.026+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.026+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:3372ec01:::obj225:head) v3 finish latency 0.000017 2026-02-11T23:31:28.026+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3372ec01:::obj225:head local-lis/les=117/118 n=133 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.026+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.026+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.026+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3372ec01:::obj225:head local-lis/les=117/118 n=133 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.026+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:28.026+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:28.026+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:28.026+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63c5e00 con 0x507ac00 2026-02-11T23:31:28.026+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:339fd60b:::obj93:head)]) -- 0x63c9800 con 0x507ac00 2026-02-11T23:31:28.026+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.027+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 306 ==== MOSDPGPush(4.0 118/117 [PushOp(4:33a4b6a3:::obj360:head, version: 111'360, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:33a4b6a3:::obj360:head@111'360, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c340 con 0x507ac00 2026-02-11T23:31:28.027+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:33a4b6a3:::obj360:head, version: 111'360, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:33a4b6a3:::obj360:head@111'360, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 118 2026-02-11T23:31:28.027+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:33a4b6a3:::obj360:head, version: 111'360, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:33a4b6a3:::obj360:head@111'360, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:28.027+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 307 ==== pg_backfill(progress 4.0 e 118/118 lb 4:339fd60b:::obj93:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d9ac00 con 0x507ac00 2026-02-11T23:31:28.027+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:339fd60b:::obj93:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 118 2026-02-11T23:31:28.027+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.027+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:339fd60b:::obj93:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:28.027+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:33a4b6a3:::obj360:head, version: 111'360, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:33a4b6a3:::obj360:head@111'360, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:28.027+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.027+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:33a4b6a3:::obj360:head, version: 111'360, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:33a4b6a3:::obj360:head@111'360, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:28.027+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:33a4b6a3:::obj360:head, version: 111'360, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:33a4b6a3:::obj360:head@111'360, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(4.0 118/117 [PushOp(4:33a4b6a3:::obj360:head, version: 111'360, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:33a4b6a3:::obj360:head@111'360, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:3372ec01:::obj225:head local-lis/les=117/118 n=133 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.027+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.027+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.027+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:339fd60b:::obj93:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:28.027+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3372ec01:::obj225:head local-lis/les=117/118 n=133 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:33a4b6a3:::obj360:head, version: 111'360, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:33a4b6a3:::obj360:head@111'360, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.027+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3372ec01:::obj225:head local-lis/les=117/118 n=133 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:33a4b6a3:::obj360:head@111'360, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.027+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3372ec01:::obj225:head local-lis/les=117/118 n=133 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:33a4b6a3:::obj360:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.027+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3372ec01:::obj225:head local-lis/les=117/118 n=133 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:33a4b6a3:::obj360:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.027+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3372ec01:::obj225:head local-lis/les=117/118 n=133 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:33a4b6a3:::obj360:head 2026-02-11T23:31:28.027+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3372ec01:::obj225:head local-lis/les=117/118 n=133 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:33a4b6a3:::obj360:head v 111'360 2026-02-11T23:31:28.027+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3372ec01:::obj225:head local-lis/les=117/118 n=133 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.027+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.027+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:33a4b6a3:::obj360:head, version: 111'360, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:33a4b6a3:::obj360:head@111'360, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000069 2026-02-11T23:31:28.027+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.027+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.027+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:339fd60b:::obj93:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:28.027+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:339fd60b:::obj93:head) v3 prio 15 cost 0 latency 0.000096 pg_backfill(progress 4.0 e 118/118 lb 4:339fd60b:::obj93:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:3372ec01:::obj225:head local-lis/les=117/118 n=133 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.027+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.027+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3372ec01:::obj225:head local-lis/les=117/118 n=133 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:339fd60b:::obj93:head) v3 2026-02-11T23:31:28.027+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3372ec01:::obj225:head local-lis/les=117/118 n=133 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:339fd60b:::obj93:head) 2026-02-11T23:31:28.027+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.027+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:339fd60b:::obj93:head) v3 finish latency 0.000016 2026-02-11T23:31:28.027+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:339fd60b:::obj93:head local-lis/les=117/118 n=134 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.027+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.027+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.027+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:339fd60b:::obj93:head local-lis/les=117/118 n=134 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.027+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:28.027+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:28.027+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:28.027+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63c9c80 con 0x507ac00 2026-02-11T23:31:28.027+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:33a4b6a3:::obj360:head)]) -- 0x649e480 con 0x507ac00 2026-02-11T23:31:28.027+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.027+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 308 ==== MOSDPGPush(4.0 118/117 [PushOp(4:3421eea3:::obj166:head, version: 111'166, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3421eea3:::obj166:head@111'166, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f1ba0 con 0x507ac00 2026-02-11T23:31:28.027+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:3421eea3:::obj166:head, version: 111'166, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3421eea3:::obj166:head@111'166, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 118 2026-02-11T23:31:28.027+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:3421eea3:::obj166:head, version: 111'166, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3421eea3:::obj166:head@111'166, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:28.027+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.027+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:3421eea3:::obj166:head, version: 111'166, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3421eea3:::obj166:head@111'166, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:28.027+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 309 ==== pg_backfill(progress 4.0 e 118/118 lb 4:33a4b6a3:::obj360:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99600 con 0x507ac00 2026-02-11T23:31:28.027+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:33a4b6a3:::obj360:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 118 2026-02-11T23:31:28.027+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.027+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:33a4b6a3:::obj360:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:28.027+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:3421eea3:::obj166:head, version: 111'166, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3421eea3:::obj166:head@111'166, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:28.027+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.027+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:3421eea3:::obj166:head, version: 111'166, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3421eea3:::obj166:head@111'166, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 118/117 [PushOp(4:3421eea3:::obj166:head, version: 111'166, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3421eea3:::obj166:head@111'166, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:339fd60b:::obj93:head local-lis/les=117/118 n=134 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.027+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:33a4b6a3:::obj360:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:28.027+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.027+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:339fd60b:::obj93:head local-lis/les=117/118 n=134 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:3421eea3:::obj166:head, version: 111'166, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3421eea3:::obj166:head@111'166, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.027+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:339fd60b:::obj93:head local-lis/les=117/118 n=134 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:3421eea3:::obj166:head@111'166, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.027+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:339fd60b:::obj93:head local-lis/les=117/118 n=134 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:3421eea3:::obj166:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.027+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:339fd60b:::obj93:head local-lis/les=117/118 n=134 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:3421eea3:::obj166:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.027+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:339fd60b:::obj93:head local-lis/les=117/118 n=134 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:3421eea3:::obj166:head 2026-02-11T23:31:28.027+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:339fd60b:::obj93:head local-lis/les=117/118 n=134 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:3421eea3:::obj166:head v 111'166 2026-02-11T23:31:28.027+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:339fd60b:::obj93:head local-lis/les=117/118 n=134 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.027+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.027+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:3421eea3:::obj166:head, version: 111'166, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3421eea3:::obj166:head@111'166, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:28.027+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.027+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.027+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:33a4b6a3:::obj360:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:28.027+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:33a4b6a3:::obj360:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 4.0 e 118/118 lb 4:33a4b6a3:::obj360:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:339fd60b:::obj93:head local-lis/les=117/118 n=134 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.027+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.027+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:339fd60b:::obj93:head local-lis/les=117/118 n=134 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:33a4b6a3:::obj360:head) v3 2026-02-11T23:31:28.027+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:339fd60b:::obj93:head local-lis/les=117/118 n=134 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:33a4b6a3:::obj360:head) 2026-02-11T23:31:28.027+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.027+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:33a4b6a3:::obj360:head) v3 finish latency 0.000014 2026-02-11T23:31:28.027+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:33a4b6a3:::obj360:head local-lis/les=117/118 n=135 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.027+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.028+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.028+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:33a4b6a3:::obj360:head local-lis/les=117/118 n=135 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.028+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:28.028+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:28.028+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:28.028+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63c5980 con 0x507ac00 2026-02-11T23:31:28.028+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:3421eea3:::obj166:head)]) -- 0x63b8000 con 0x507ac00 2026-02-11T23:31:28.028+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.028+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 310 ==== MOSDPGPush(4.0 118/117 [PushOp(4:360ee05a:::obj519:head, version: 111'519, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:360ee05a:::obj519:head@111'519, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x555cea0 con 0x507ac00 2026-02-11T23:31:28.028+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:360ee05a:::obj519:head, version: 111'519, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:360ee05a:::obj519:head@111'519, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 118 2026-02-11T23:31:28.028+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:360ee05a:::obj519:head, version: 111'519, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:360ee05a:::obj519:head@111'519, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:28.028+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.028+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 311 ==== pg_backfill(progress 4.0 e 118/118 lb 4:3421eea3:::obj166:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99080 con 0x507ac00 2026-02-11T23:31:28.028+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:360ee05a:::obj519:head, version: 111'519, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:360ee05a:::obj519:head@111'519, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:28.028+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:3421eea3:::obj166:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:28.028+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:3421eea3:::obj166:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:28.028+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.028+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:360ee05a:::obj519:head, version: 111'519, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:360ee05a:::obj519:head@111'519, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:28.028+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:360ee05a:::obj519:head, version: 111'519, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:360ee05a:::obj519:head@111'519, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 118/117 [PushOp(4:360ee05a:::obj519:head, version: 111'519, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:360ee05a:::obj519:head@111'519, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:33a4b6a3:::obj360:head local-lis/les=117/118 n=135 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.028+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.028+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.028+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:33a4b6a3:::obj360:head local-lis/les=117/118 n=135 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:360ee05a:::obj519:head, version: 111'519, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:360ee05a:::obj519:head@111'519, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.028+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:3421eea3:::obj166:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:28.028+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:33a4b6a3:::obj360:head local-lis/les=117/118 n=135 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:360ee05a:::obj519:head@111'519, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.028+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:33a4b6a3:::obj360:head local-lis/les=117/118 n=135 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:360ee05a:::obj519:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.028+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:33a4b6a3:::obj360:head local-lis/les=117/118 n=135 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:360ee05a:::obj519:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.028+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:33a4b6a3:::obj360:head local-lis/les=117/118 n=135 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:360ee05a:::obj519:head 2026-02-11T23:31:28.028+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:33a4b6a3:::obj360:head local-lis/les=117/118 n=135 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:360ee05a:::obj519:head v 111'519 2026-02-11T23:31:28.028+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:33a4b6a3:::obj360:head local-lis/les=117/118 n=135 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.028+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.028+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:360ee05a:::obj519:head, version: 111'519, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:360ee05a:::obj519:head@111'519, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:28.028+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.028+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.028+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:3421eea3:::obj166:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:28.028+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:3421eea3:::obj166:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 118/118 lb 4:3421eea3:::obj166:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:33a4b6a3:::obj360:head local-lis/les=117/118 n=135 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.028+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.028+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:33a4b6a3:::obj360:head local-lis/les=117/118 n=135 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:3421eea3:::obj166:head) v3 2026-02-11T23:31:28.028+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:33a4b6a3:::obj360:head local-lis/les=117/118 n=135 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:3421eea3:::obj166:head) 2026-02-11T23:31:28.028+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.028+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:3421eea3:::obj166:head) v3 finish latency 0.000013 2026-02-11T23:31:28.028+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3421eea3:::obj166:head local-lis/les=117/118 n=136 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.028+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.028+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.028+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3421eea3:::obj166:head local-lis/les=117/118 n=136 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.028+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:28.028+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:28.028+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:28.028+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63c9e00 con 0x507ac00 2026-02-11T23:31:28.028+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:360ee05a:::obj519:head)]) -- 0x6447800 con 0x507ac00 2026-02-11T23:31:28.028+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.029+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 312 ==== MOSDPGPush(4.0 118/117 [PushOp(4:3655ac21:::obj70:head, version: 111'70, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3655ac21:::obj70:head@111'70, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x56cad00 con 0x507ac00 2026-02-11T23:31:28.029+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:3655ac21:::obj70:head, version: 111'70, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3655ac21:::obj70:head@111'70, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 118 2026-02-11T23:31:28.029+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:3655ac21:::obj70:head, version: 111'70, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3655ac21:::obj70:head@111'70, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:28.029+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.029+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:3655ac21:::obj70:head, version: 111'70, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3655ac21:::obj70:head@111'70, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:28.029+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 313 ==== pg_backfill(progress 4.0 e 118/118 lb 4:360ee05a:::obj519:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176b00 con 0x507ac00 2026-02-11T23:31:28.029+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:360ee05a:::obj519:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:28.029+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:360ee05a:::obj519:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:28.029+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.029+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:3655ac21:::obj70:head, version: 111'70, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3655ac21:::obj70:head@111'70, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:28.029+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:3655ac21:::obj70:head, version: 111'70, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3655ac21:::obj70:head@111'70, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 118/117 [PushOp(4:3655ac21:::obj70:head, version: 111'70, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3655ac21:::obj70:head@111'70, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:3421eea3:::obj166:head local-lis/les=117/118 n=136 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.029+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.029+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.029+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:360ee05a:::obj519:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:28.029+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3421eea3:::obj166:head local-lis/les=117/118 n=136 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:3655ac21:::obj70:head, version: 111'70, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3655ac21:::obj70:head@111'70, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.029+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3421eea3:::obj166:head local-lis/les=117/118 n=136 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:3655ac21:::obj70:head@111'70, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.029+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3421eea3:::obj166:head local-lis/les=117/118 n=136 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:3655ac21:::obj70:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.029+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3421eea3:::obj166:head local-lis/les=117/118 n=136 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:3655ac21:::obj70:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.029+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3421eea3:::obj166:head local-lis/les=117/118 n=136 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:3655ac21:::obj70:head 2026-02-11T23:31:28.029+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3421eea3:::obj166:head local-lis/les=117/118 n=136 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:3655ac21:::obj70:head v 111'70 2026-02-11T23:31:28.029+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3421eea3:::obj166:head local-lis/les=117/118 n=136 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.029+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.029+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:3655ac21:::obj70:head, version: 111'70, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3655ac21:::obj70:head@111'70, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:28.029+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.029+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.029+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:360ee05a:::obj519:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:28.029+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:360ee05a:::obj519:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 4.0 e 118/118 lb 4:360ee05a:::obj519:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:3421eea3:::obj166:head local-lis/les=117/118 n=136 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.029+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.029+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3421eea3:::obj166:head local-lis/les=117/118 n=136 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:360ee05a:::obj519:head) v3 2026-02-11T23:31:28.029+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3421eea3:::obj166:head local-lis/les=117/118 n=136 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:360ee05a:::obj519:head) 2026-02-11T23:31:28.029+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.029+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:360ee05a:::obj519:head) v3 finish latency 0.000017 2026-02-11T23:31:28.029+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:360ee05a:::obj519:head local-lis/les=117/118 n=137 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.029+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.029+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.029+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:360ee05a:::obj519:head local-lis/les=117/118 n=137 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.029+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:28.029+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:28.029+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:28.029+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63c5800 con 0x507ac00 2026-02-11T23:31:28.029+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:3655ac21:::obj70:head)]) -- 0x649e600 con 0x507ac00 2026-02-11T23:31:28.029+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.029+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 314 ==== MOSDPGPush(4.0 118/117 [PushOp(4:372fd24d:::obj442:head, version: 111'442, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:372fd24d:::obj442:head@111'442, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56171e0 con 0x507ac00 2026-02-11T23:31:28.029+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:372fd24d:::obj442:head, version: 111'442, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:372fd24d:::obj442:head@111'442, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 118 2026-02-11T23:31:28.029+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:372fd24d:::obj442:head, version: 111'442, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:372fd24d:::obj442:head@111'442, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:28.029+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.029+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 315 ==== pg_backfill(progress 4.0 e 118/118 lb 4:3655ac21:::obj70:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d2d600 con 0x507ac00 2026-02-11T23:31:28.029+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:372fd24d:::obj442:head, version: 111'442, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:372fd24d:::obj442:head@111'442, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:28.029+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:3655ac21:::obj70:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 118 2026-02-11T23:31:28.029+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:3655ac21:::obj70:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:28.029+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.029+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:372fd24d:::obj442:head, version: 111'442, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:372fd24d:::obj442:head@111'442, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:28.029+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:372fd24d:::obj442:head, version: 111'442, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:372fd24d:::obj442:head@111'442, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 118/117 [PushOp(4:372fd24d:::obj442:head, version: 111'442, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:372fd24d:::obj442:head@111'442, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:360ee05a:::obj519:head local-lis/les=117/118 n=137 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.029+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.029+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.029+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:3655ac21:::obj70:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:28.029+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:360ee05a:::obj519:head local-lis/les=117/118 n=137 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:372fd24d:::obj442:head, version: 111'442, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:372fd24d:::obj442:head@111'442, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.029+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:360ee05a:::obj519:head local-lis/les=117/118 n=137 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:372fd24d:::obj442:head@111'442, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.029+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:360ee05a:::obj519:head local-lis/les=117/118 n=137 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:372fd24d:::obj442:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.029+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:360ee05a:::obj519:head local-lis/les=117/118 n=137 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:372fd24d:::obj442:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.029+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:360ee05a:::obj519:head local-lis/les=117/118 n=137 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:372fd24d:::obj442:head 2026-02-11T23:31:28.029+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:360ee05a:::obj519:head local-lis/les=117/118 n=137 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:372fd24d:::obj442:head v 111'442 2026-02-11T23:31:28.029+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:360ee05a:::obj519:head local-lis/les=117/118 n=137 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.029+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.029+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:372fd24d:::obj442:head, version: 111'442, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:372fd24d:::obj442:head@111'442, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:28.029+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.029+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.029+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:3655ac21:::obj70:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:28.029+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:3655ac21:::obj70:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 4.0 e 118/118 lb 4:3655ac21:::obj70:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:360ee05a:::obj519:head local-lis/les=117/118 n=137 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.029+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.029+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:360ee05a:::obj519:head local-lis/les=117/118 n=137 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:3655ac21:::obj70:head) v3 2026-02-11T23:31:28.029+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:360ee05a:::obj519:head local-lis/les=117/118 n=137 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:3655ac21:::obj70:head) 2026-02-11T23:31:28.029+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.029+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:3655ac21:::obj70:head) v3 finish latency 0.000016 2026-02-11T23:31:28.029+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3655ac21:::obj70:head local-lis/les=117/118 n=138 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.029+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.030+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.030+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3655ac21:::obj70:head local-lis/les=117/118 n=138 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.030+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:28.030+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:28.030+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:28.030+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63c4000 con 0x507ac00 2026-02-11T23:31:28.030+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:372fd24d:::obj442:head)]) -- 0x63c9200 con 0x507ac00 2026-02-11T23:31:28.030+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.030+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 316 ==== MOSDPGPush(4.0 118/117 [PushOp(4:37ff9db7:::obj453:head, version: 111'453, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:37ff9db7:::obj453:head@111'453, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a3860 con 0x507ac00 2026-02-11T23:31:28.030+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:37ff9db7:::obj453:head, version: 111'453, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:37ff9db7:::obj453:head@111'453, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 118 2026-02-11T23:31:28.030+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:37ff9db7:::obj453:head, version: 111'453, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:37ff9db7:::obj453:head@111'453, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:28.030+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.030+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 317 ==== pg_backfill(progress 4.0 e 118/118 lb 4:372fd24d:::obj442:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98580 con 0x507ac00 2026-02-11T23:31:28.030+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:372fd24d:::obj442:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 118 2026-02-11T23:31:28.030+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:37ff9db7:::obj453:head, version: 111'453, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:37ff9db7:::obj453:head@111'453, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:28.030+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:372fd24d:::obj442:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:28.030+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.030+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:37ff9db7:::obj453:head, version: 111'453, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:37ff9db7:::obj453:head@111'453, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:28.030+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:37ff9db7:::obj453:head, version: 111'453, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:37ff9db7:::obj453:head@111'453, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 118/117 [PushOp(4:37ff9db7:::obj453:head, version: 111'453, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:37ff9db7:::obj453:head@111'453, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:3655ac21:::obj70:head local-lis/les=117/118 n=138 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.030+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.030+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.030+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:372fd24d:::obj442:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:28.030+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3655ac21:::obj70:head local-lis/les=117/118 n=138 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:37ff9db7:::obj453:head, version: 111'453, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:37ff9db7:::obj453:head@111'453, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.030+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3655ac21:::obj70:head local-lis/les=117/118 n=138 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:37ff9db7:::obj453:head@111'453, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.030+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3655ac21:::obj70:head local-lis/les=117/118 n=138 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:37ff9db7:::obj453:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.030+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3655ac21:::obj70:head local-lis/les=117/118 n=138 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:37ff9db7:::obj453:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.030+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3655ac21:::obj70:head local-lis/les=117/118 n=138 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:37ff9db7:::obj453:head 2026-02-11T23:31:28.030+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3655ac21:::obj70:head local-lis/les=117/118 n=138 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:37ff9db7:::obj453:head v 111'453 2026-02-11T23:31:28.030+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3655ac21:::obj70:head local-lis/les=117/118 n=138 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.030+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.030+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:37ff9db7:::obj453:head, version: 111'453, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:37ff9db7:::obj453:head@111'453, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000065 2026-02-11T23:31:28.030+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.030+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.030+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:372fd24d:::obj442:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:28.030+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:372fd24d:::obj442:head) v3 prio 15 cost 0 latency 0.000084 pg_backfill(progress 4.0 e 118/118 lb 4:372fd24d:::obj442:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:3655ac21:::obj70:head local-lis/les=117/118 n=138 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.030+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.030+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3655ac21:::obj70:head local-lis/les=117/118 n=138 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:372fd24d:::obj442:head) v3 2026-02-11T23:31:28.030+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:3655ac21:::obj70:head local-lis/les=117/118 n=138 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:372fd24d:::obj442:head) 2026-02-11T23:31:28.030+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.030+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:372fd24d:::obj442:head) v3 finish latency 0.000016 2026-02-11T23:31:28.030+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:372fd24d:::obj442:head local-lis/les=117/118 n=139 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.030+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.030+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:372fd24d:::obj442:head local-lis/les=117/118 n=139 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.030+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.030+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:28.030+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:28.030+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:28.030+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63c5680 con 0x507ac00 2026-02-11T23:31:28.030+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:37ff9db7:::obj453:head)]) -- 0x63c9080 con 0x507ac00 2026-02-11T23:31:28.030+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.031+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 318 ==== MOSDPGPush(4.0 118/117 [PushOp(4:384eca0c:::obj484:head, version: 111'484, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:384eca0c:::obj484:head@111'484, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a3040 con 0x507ac00 2026-02-11T23:31:28.031+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:384eca0c:::obj484:head, version: 111'484, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:384eca0c:::obj484:head@111'484, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 118 2026-02-11T23:31:28.031+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:384eca0c:::obj484:head, version: 111'484, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:384eca0c:::obj484:head@111'484, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:28.031+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.031+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 319 ==== pg_backfill(progress 4.0 e 118/118 lb 4:37ff9db7:::obj453:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98000 con 0x507ac00 2026-02-11T23:31:28.031+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:37ff9db7:::obj453:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 118 2026-02-11T23:31:28.031+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:384eca0c:::obj484:head, version: 111'484, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:384eca0c:::obj484:head@111'484, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:28.031+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:37ff9db7:::obj453:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:28.031+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.031+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:384eca0c:::obj484:head, version: 111'484, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:384eca0c:::obj484:head@111'484, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:28.031+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.031+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:384eca0c:::obj484:head, version: 111'484, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:384eca0c:::obj484:head@111'484, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 118/117 [PushOp(4:384eca0c:::obj484:head, version: 111'484, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:384eca0c:::obj484:head@111'484, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:372fd24d:::obj442:head local-lis/les=117/118 n=139 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.031+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:37ff9db7:::obj453:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:28.031+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.031+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:372fd24d:::obj442:head local-lis/les=117/118 n=139 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:384eca0c:::obj484:head, version: 111'484, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:384eca0c:::obj484:head@111'484, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.031+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:372fd24d:::obj442:head local-lis/les=117/118 n=139 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:384eca0c:::obj484:head@111'484, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.031+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:372fd24d:::obj442:head local-lis/les=117/118 n=139 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:384eca0c:::obj484:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.031+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:372fd24d:::obj442:head local-lis/les=117/118 n=139 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:384eca0c:::obj484:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.031+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:372fd24d:::obj442:head local-lis/les=117/118 n=139 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:384eca0c:::obj484:head 2026-02-11T23:31:28.031+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:372fd24d:::obj442:head local-lis/les=117/118 n=139 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:384eca0c:::obj484:head v 111'484 2026-02-11T23:31:28.031+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:372fd24d:::obj442:head local-lis/les=117/118 n=139 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.031+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.031+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:384eca0c:::obj484:head, version: 111'484, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:384eca0c:::obj484:head@111'484, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:28.031+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.031+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.031+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:37ff9db7:::obj453:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:28.031+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:37ff9db7:::obj453:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 4.0 e 118/118 lb 4:37ff9db7:::obj453:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:372fd24d:::obj442:head local-lis/les=117/118 n=139 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.031+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.031+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:372fd24d:::obj442:head local-lis/les=117/118 n=139 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:37ff9db7:::obj453:head) v3 2026-02-11T23:31:28.031+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:372fd24d:::obj442:head local-lis/les=117/118 n=139 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:37ff9db7:::obj453:head) 2026-02-11T23:31:28.031+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.031+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:37ff9db7:::obj453:head) v3 finish latency 0.000016 2026-02-11T23:31:28.031+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:37ff9db7:::obj453:head local-lis/les=117/118 n=140 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.031+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.031+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:37ff9db7:::obj453:head local-lis/les=117/118 n=140 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.031+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:28.031+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:28.031+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:28.031+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63c4180 con 0x507ac00 2026-02-11T23:31:28.031+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.031+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:384eca0c:::obj484:head)]) -- 0x63c8f00 con 0x507ac00 2026-02-11T23:31:28.031+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.031+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 320 ==== MOSDPGPush(4.0 118/117 [PushOp(4:385733f0:::obj251:head, version: 111'251, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:385733f0:::obj251:head@111'251, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a29c0 con 0x507ac00 2026-02-11T23:31:28.031+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:385733f0:::obj251:head, version: 111'251, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:385733f0:::obj251:head@111'251, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 118 2026-02-11T23:31:28.031+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:385733f0:::obj251:head, version: 111'251, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:385733f0:::obj251:head@111'251, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:28.031+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.031+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 321 ==== pg_backfill(progress 4.0 e 118/118 lb 4:384eca0c:::obj484:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9b180 con 0x507ac00 2026-02-11T23:31:28.031+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:385733f0:::obj251:head, version: 111'251, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:385733f0:::obj251:head@111'251, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:28.031+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:384eca0c:::obj484:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 118 2026-02-11T23:31:28.031+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:384eca0c:::obj484:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:28.031+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.031+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:385733f0:::obj251:head, version: 111'251, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:385733f0:::obj251:head@111'251, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:28.031+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.031+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:385733f0:::obj251:head, version: 111'251, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:385733f0:::obj251:head@111'251, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000042 MOSDPGPush(4.0 118/117 [PushOp(4:385733f0:::obj251:head, version: 111'251, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:385733f0:::obj251:head@111'251, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:37ff9db7:::obj453:head local-lis/les=117/118 n=140 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.031+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.031+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.031+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:37ff9db7:::obj453:head local-lis/les=117/118 n=140 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:385733f0:::obj251:head, version: 111'251, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:385733f0:::obj251:head@111'251, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.031+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:384eca0c:::obj484:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:28.031+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:37ff9db7:::obj453:head local-lis/les=117/118 n=140 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:385733f0:::obj251:head@111'251, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.031+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:37ff9db7:::obj453:head local-lis/les=117/118 n=140 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:385733f0:::obj251:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.031+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:37ff9db7:::obj453:head local-lis/les=117/118 n=140 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:385733f0:::obj251:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.031+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:37ff9db7:::obj453:head local-lis/les=117/118 n=140 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:385733f0:::obj251:head 2026-02-11T23:31:28.031+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:37ff9db7:::obj453:head local-lis/les=117/118 n=140 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:385733f0:::obj251:head v 111'251 2026-02-11T23:31:28.031+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:37ff9db7:::obj453:head local-lis/les=117/118 n=140 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.031+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.032+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:385733f0:::obj251:head, version: 111'251, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:385733f0:::obj251:head@111'251, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000072 2026-02-11T23:31:28.032+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.032+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.032+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:384eca0c:::obj484:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:28.032+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:384eca0c:::obj484:head) v3 prio 15 cost 0 latency 0.000096 pg_backfill(progress 4.0 e 118/118 lb 4:384eca0c:::obj484:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:37ff9db7:::obj453:head local-lis/les=117/118 n=140 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.032+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.032+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:37ff9db7:::obj453:head local-lis/les=117/118 n=140 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:384eca0c:::obj484:head) v3 2026-02-11T23:31:28.032+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:37ff9db7:::obj453:head local-lis/les=117/118 n=140 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:384eca0c:::obj484:head) 2026-02-11T23:31:28.032+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.032+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:384eca0c:::obj484:head) v3 finish latency 0.000013 2026-02-11T23:31:28.032+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:384eca0c:::obj484:head local-lis/les=117/118 n=141 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.032+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.032+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:384eca0c:::obj484:head local-lis/les=117/118 n=141 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.032+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:28.032+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:28.032+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:28.032+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.032+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x63c5500 con 0x507ac00 2026-02-11T23:31:28.032+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:385733f0:::obj251:head)]) -- 0x63c8d80 con 0x507ac00 2026-02-11T23:31:28.032+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.032+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 322 ==== MOSDPGPush(4.0 118/117 [PushOp(4:3945af93:::obj271:head, version: 111'271, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3945af93:::obj271:head@111'271, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559cd00 con 0x507ac00 2026-02-11T23:31:28.032+0000 7f416abfa640 15 osd.0 118 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:3945af93:::obj271:head, version: 111'271, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3945af93:::obj271:head@111'271, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000018 epoch 118 2026-02-11T23:31:28.032+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:3945af93:::obj271:head, version: 111'271, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3945af93:::obj271:head@111'271, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:28.032+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.032+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 323 ==== pg_backfill(progress 4.0 e 118/118 lb 4:385733f0:::obj251:head) ==== 1062+0+0 (unknown 0 0 0) 0x6178c00 con 0x507ac00 2026-02-11T23:31:28.032+0000 7f416abfa640 15 osd.0 118 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:385733f0:::obj251:head) v3 prio 15 type 95 cost 0 latency 0.000010 epoch 118 2026-02-11T23:31:28.032+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:385733f0:::obj251:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:28.032+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:3945af93:::obj271:head, version: 111'271, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3945af93:::obj271:head@111'271, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:28.032+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.032+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:3945af93:::obj271:head, version: 111'271, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3945af93:::obj271:head@111'271, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:28.032+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:3945af93:::obj271:head, version: 111'271, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3945af93:::obj271:head@111'271, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000054 MOSDPGPush(4.0 118/117 [PushOp(4:3945af93:::obj271:head, version: 111'271, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3945af93:::obj271:head@111'271, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:384eca0c:::obj484:head local-lis/les=117/118 n=141 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.032+0000 7f414b98b640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.032+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.032+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:385733f0:::obj251:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:28.032+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:384eca0c:::obj484:head local-lis/les=117/118 n=141 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:3945af93:::obj271:head, version: 111'271, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3945af93:::obj271:head@111'271, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.032+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:384eca0c:::obj484:head local-lis/les=117/118 n=141 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:3945af93:::obj271:head@111'271, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.032+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:384eca0c:::obj484:head local-lis/les=117/118 n=141 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:3945af93:::obj271:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.032+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1242 ==== pg_info2(1.0 1.0 1.0( v 111'600 (111'500,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) e119/119) ==== 1085+0+0 (unknown 0 0 0) 0x63a8000 con 0x4acb000 2026-02-11T23:31:28.032+0000 7f415aefe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1241 ==== osd_map(119..119 src has 1..119) ==== 296+0+0 (unknown 0 0 0) 0x4a30000 con 0x4acb000 2026-02-11T23:31:28.032+0000 7f414b98b640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:384eca0c:::obj484:head local-lis/les=117/118 n=141 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:3945af93:::obj271:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.032+0000 7f415aefe640 20 osd.0 118 OSD::ms_dispatch: osd_map(119..119 src has 1..119) 2026-02-11T23:31:28.032+0000 7f416abfa640 15 osd.0 118 enqueue_peering_evt 1.0 epoch_sent: 119 epoch_requested: 119 MInfoRec from 1 info: 1.0( v 111'600 (111'500,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) pg_lease(ru 0.000000000s ub 84.697601318s int 16.000000000s) 2026-02-11T23:31:28.032+0000 7f415aefe640 20 osd.0 118 _dispatch 0x4a30000 osd_map(119..119 src has 1..119) 2026-02-11T23:31:28.032+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:384eca0c:::obj484:head local-lis/les=117/118 n=141 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:3945af93:::obj271:head 2026-02-11T23:31:28.032+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 119 epoch_requested: 119 MInfoRec from 1 info: 1.0( v 111'600 (111'500,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) pg_lease(ru 0.000000000s ub 84.697601318s int 16.000000000s)) class_id 2 prio 255 cost 10 e119) 2026-02-11T23:31:28.032+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:384eca0c:::obj484:head local-lis/les=117/118 n=141 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:3945af93:::obj271:head v 111'271 2026-02-11T23:31:28.032+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:384eca0c:::obj484:head local-lis/les=117/118 n=141 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.032+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.032+0000 7f415aefe640 3 osd.0 118 handle_osd_map epochs [119,119], i have 118, src has [1,119] 2026-02-11T23:31:28.032+0000 7f415aefe640 10 osd.0 118 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:31:28.032+0000 7f415aefe640 10 osd.0 118 handle_osd_map got inc map for epoch 119 2026-02-11T23:31:28.032+0000 7f415aefe640 20 osd.0 118 got_full_map 119, nothing requested 2026-02-11T23:31:28.032+0000 7f415aefe640 20 osd.0 118 trim_maps: min=1 oldest_map=1 2026-02-11T23:31:28.032+0000 7f414b98b640 10 osd.0 118 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:3945af93:::obj271:head, version: 111'271, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3945af93:::obj271:head@111'271, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000083 2026-02-11T23:31:28.032+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.032+0000 7f415aefe640 20 osd.0 118 handle_osd_map pg_num_history pg_num_history(e119 pg_nums {1={27=1},2={34=1},3={42=1},4={49=1},5={56=1},6={64=1},7={70=1},8={77=1},9={85=1},10={92=1}} deleted_pools 99,2,100,3,101,5,102,6,103,7,104,8,105,9,106,10) 2026-02-11T23:31:28.032+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 119 epoch_requested: 119 MInfoRec from 1 info: 1.0( v 111'600 (111'500,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) pg_lease(ru 0.000000000s ub 84.697601318s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e119) queued 2026-02-11T23:31:28.032+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.032+0000 7f415aefe640 10 snap_mapper record_purged_snaps purged_snaps {119={}} 2026-02-11T23:31:28.032+0000 7f415aefe640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:31:28.032+0000 7f415aefe640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e119 maps [1~119] lci=[5,119] tlb=1) 2026-02-11T23:31:28.032+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 119 epoch_requested: 119 MInfoRec from 1 info: 1.0( v 111'600 (111'500,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) pg_lease(ru 0.000000000s ub 84.697601318s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e119) pg 0x4ef0000 2026-02-11T23:31:28.032+0000 7f414b98b640 20 osd.0 op_wq _add_slot_waiter 1.0 peering, item epoch is 119, will wait on OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 119 epoch_requested: 119 MInfoRec from 1 info: 1.0( v 111'600 (111'500,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) pg_lease(ru 0.000000000s ub 84.697601318s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e119) 2026-02-11T23:31:28.032+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.032+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.032+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:385733f0:::obj251:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:28.032+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:385733f0:::obj251:head) v3 prio 15 cost 0 latency 0.000129 pg_backfill(progress 4.0 e 118/118 lb 4:385733f0:::obj251:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:384eca0c:::obj484:head local-lis/les=117/118 n=141 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.032+0000 7f414f993640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:28.032+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:384eca0c:::obj484:head local-lis/les=117/118 n=141 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:385733f0:::obj251:head) v3 2026-02-11T23:31:28.032+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:384eca0c:::obj484:head local-lis/les=117/118 n=141 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:385733f0:::obj251:head) 2026-02-11T23:31:28.032+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.032+0000 7f414f993640 10 osd.0 118 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:385733f0:::obj251:head) v3 finish latency 0.000014 2026-02-11T23:31:28.032+0000 7f414f993640 20 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:385733f0:::obj251:head local-lis/les=117/118 n=142 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.032+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.032+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:385733f0:::obj251:head local-lis/les=117/118 n=142 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.032+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.032+0000 7f414f993640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:28.032+0000 7f414f993640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:28.032+0000 7f414f993640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:28.032+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e118/118) -- 0x649f380 con 0x507ac00 2026-02-11T23:31:28.032+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:3945af93:::obj271:head)]) -- 0x63c8c00 con 0x507ac00 2026-02-11T23:31:28.032+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.032+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.032+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.033+0000 7f41591a6640 10 osd.0 118 _committed_osd_maps 119..119 2026-02-11T23:31:28.033+0000 7f41591a6640 10 osd.0 118 advance to epoch 119 (<= last 119 <= newest_map 119) 2026-02-11T23:31:28.033+0000 7f41591a6640 20 osd.0 119 consume_map version 119 2026-02-11T23:31:28.033+0000 7f41591a6640 20 osd.0 118 await_reserved_maps epoch:119 2026-02-11T23:31:28.033+0000 7f41591a6640 20 osd.0 118 await_reserved_maps done 0 2026-02-11T23:31:28.033+0000 7f41591a6640 20 osd.0 119 consume_map 119 -- publish done 2026-02-11T23:31:28.033+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:31:28.033+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x5efde40 2026-02-11T23:31:28.033+0000 7f41591a6640 20 osd.0 119 identify_splits_and_merges 4.0 e118 to e119 2026-02-11T23:31:28.033+0000 7f41591a6640 20 osd.0 119 identify_splits_and_merges 4.0 e118 to e119 pg_nums {49=1} 2026-02-11T23:31:28.033+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3701340 2026-02-11T23:31:28.033+0000 7f41591a6640 20 osd.0 119 identify_splits_and_merges 1.0 e118 to e119 2026-02-11T23:31:28.033+0000 7f41591a6640 20 osd.0 119 identify_splits_and_merges 1.0 e118 to e119 pg_nums {27=1} 2026-02-11T23:31:28.033+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:28.033+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:28.033+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:28.033+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:28.033+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:28.033+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:28.033+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:28.033+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:28.033+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:28.033+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:28.033+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:28.033+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:28.033+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:28.033+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:28.033+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:28.033+0000 7f41591a6640 20 osd.0 119 prune_pg_created 2026-02-11T23:31:28.033+0000 7f41591a6640 10 osd.0:0.consume_map 119 (was 118) 2026-02-11T23:31:28.033+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:31:28.033+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:31:28.033+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 pending_peering first epoch 119 <= 119, requeueing 2026-02-11T23:31:28.033+0000 7f41591a6640 20 osd.0:0._wake_pg_slot _wake_pg_slot 1.0 to_process <> waiting <> waiting_peering {119=} 2026-02-11T23:31:28.033+0000 7f41591a6640 10 osd.0:1.consume_map 119 (was 118) 2026-02-11T23:31:28.033+0000 7f41591a6640 10 osd.0:2.consume_map 119 (was 118) 2026-02-11T23:31:28.033+0000 7f41591a6640 10 osd.0:3.consume_map 119 (was 118) 2026-02-11T23:31:28.033+0000 7f41591a6640 10 osd.0:4.consume_map 119 (was 118) 2026-02-11T23:31:28.033+0000 7f41591a6640 10 osd.0:5.consume_map 119 (was 118) 2026-02-11T23:31:28.033+0000 7f41591a6640 10 osd.0:6.consume_map 119 (was 118) 2026-02-11T23:31:28.033+0000 7f41591a6640 10 osd.0:7.consume_map 119 (was 118) 2026-02-11T23:31:28.033+0000 7f41591a6640 10 osd.0 119 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:31:28.033+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.033+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 119 epoch_requested: 119 MInfoRec from 1 info: 1.0( v 111'600 (111'500,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) pg_lease(ru 0.000000000s ub 84.697601318s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e119) queued 2026-02-11T23:31:28.033+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.033+0000 7f41591a6640 15 osd.0 119 enqueue_peering_evt 4.0 epoch_sent: 119 epoch_requested: 119 NullEvt 2026-02-11T23:31:28.033+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 119 epoch_requested: 119 MInfoRec from 1 info: 1.0( v 111'600 (111'500,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) pg_lease(ru 0.000000000s ub 84.697601318s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e119) pg 0x4ef0000 2026-02-11T23:31:28.033+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 119 epoch_requested: 119 NullEvt) class_id 2 prio 255 cost 10 e119) 2026-02-11T23:31:28.033+0000 7f41591a6640 15 osd.0 119 enqueue_peering_evt 1.0 epoch_sent: 119 epoch_requested: 119 NullEvt 2026-02-11T23:31:28.033+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 119 epoch_requested: 119 NullEvt) class_id 2 prio 255 cost 10 e119) 2026-02-11T23:31:28.033+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 unknown NOTIFY mbc={}] handle_advance_map: 119 2026-02-11T23:31:28.033+0000 7f41591a6640 7 osd.0 119 activate_map version 119 2026-02-11T23:31:28.033+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 118 -> 119 2026-02-11T23:31:28.033+0000 7f414f993640 10 osd.0 pg_epoch: 118 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 unknown NOTIFY mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:31:28.033+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.033+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 unknown NOTIFY mbc={}] state: Started advmap 2026-02-11T23:31:28.033+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 119 epoch_requested: 119 NullEvt) class_id 2immediate prio 255 cost 10 e119) queued 2026-02-11T23:31:28.033+0000 7f4150194640 20 osd.0 119 agent_entry empty queue 2026-02-11T23:31:28.033+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.033+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 unknown NOTIFY mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:31:28.033+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 119 epoch_requested: 119 NullEvt) class_id 2immediate prio 255 cost 10 e119) pg 0x5978000 2026-02-11T23:31:28.033+0000 7f414f993640 20 osd.0 119 advance_pg new pool opts old pool opts 2026-02-11T23:31:28.033+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 unknown NOTIFY mbc={}] handle_activate_map: epoch range: 119..119 2026-02-11T23:31:28.033+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 unknown NOTIFY mbc={}] PeeringState::activate_map 2026-02-11T23:31:28.033+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 unknown NOTIFY mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 118 while current is 119 2026-02-11T23:31:28.033+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.033+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:385733f0:::obj251:head local-lis/les=117/118 n=142 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_advance_map: 119 2026-02-11T23:31:28.033+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 unknown NOTIFY mbc={}] do_peering_event: epoch_sent: 119 epoch_requested: 119 MInfoRec from 1 info: 1.0( v 111'600 (111'500,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) pg_lease(ru 0.000000000s ub 84.697601318s int 16.000000000s) 2026-02-11T23:31:28.033+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 4.0 118 -> 119 2026-02-11T23:31:28.033+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 unknown NOTIFY mbc={}] state: got info from osd.1 1.0( v 111'600 (111'500,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) 2026-02-11T23:31:28.033+0000 7f414b98b640 10 osd.0 pg_epoch: 118 pg[4.0( v 111'600 (111'590,111'600] lb 4:385733f0:::obj251:head local-lis/les=117/118 n=142 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4] -- 4/4 2026-02-11T23:31:28.033+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 unknown NOTIFY mbc={}] PeeringState::proc_lease pg_lease(ru 0.000000000s ub 84.697601318s int 16.000000000s) 2026-02-11T23:31:28.033+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 unknown NOTIFY mbc={}] PeeringState::proc_lease peer_clock_delta_lb -2.011766434s -> ruub 86.709373474s 2026-02-11T23:31:28.033+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:385733f0:::obj251:head local-lis/les=117/118 n=142 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] state: Started advmap 2026-02-11T23:31:28.033+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 unknown NOTIFY mbc={}] PeeringState::proc_lease readable_until_ub now 86.709373474s 2026-02-11T23:31:28.033+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:385733f0:::obj251:head local-lis/les=117/118 n=142 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:31:28.033+0000 7f414f993640 5 osd.0 pg_epoch: 119 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 unknown NOTIFY mbc={}] exit Started/Stray 0.102083 5 0.000024 2026-02-11T23:31:28.033+0000 7f414f993640 5 osd.0 pg_epoch: 119 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 unknown NOTIFY mbc={}] enter Started/ReplicaActive 2026-02-11T23:31:28.033+0000 7f414f993640 5 osd.0 pg_epoch: 119 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 unknown NOTIFY mbc={}] enter Started/ReplicaActive/RepNotRecovering 2026-02-11T23:31:28.033+0000 7f414b98b640 20 osd.0 119 advance_pg new pool opts old pool opts 2026-02-11T23:31:28.033+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 unknown NOTIFY mbc={}] state: In ReplicaActive, about to call activate 2026-02-11T23:31:28.033+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:385733f0:::obj251:head local-lis/les=117/118 n=142 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_activate_map: epoch range: 119..119 2026-02-11T23:31:28.033+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 unknown mbc={}] PeeringState::activate activate - no missing, moving last_complete 111'600 -> 111'600 2026-02-11T23:31:28.033+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:385733f0:::obj251:head local-lis/les=117/118 n=142 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::activate_map 2026-02-11T23:31:28.033+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 unknown mbc={}] scrubber: on_replica_activate 2026-02-11T23:31:28.033+0000 7f414f993640 20 scrubberFSM event: --vvvv---- ReplicaActivate 2026-02-11T23:31:28.033+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 unknown mbc={}] scrubber: FSM: -- state -->> ReplicaActive 2026-02-11T23:31:28.033+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:385733f0:::obj251:head local-lis/les=117/118 n=142 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 118 while current is 119 2026-02-11T23:31:28.033+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 unknown mbc={}] scrubber: FSM: -- state -->> ReplicaActive/ReplicaIdle 2026-02-11T23:31:28.033+0000 7f414f993640 20 scrubberFSM event: --^^^^---- ReplicaActivate 2026-02-11T23:31:28.033+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.033+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 unknown mbc={}] state: Activate Finished 2026-02-11T23:31:28.033+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:385733f0:::obj251:head local-lis/les=117/118 n=142 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_peering_event: epoch_sent: 119 epoch_requested: 119 NullEvt 2026-02-11T23:31:28.033+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.033+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.033+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.1 from_epoch 119 2026-02-11T23:31:28.033+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.033+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.033+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 119 epoch_requested: 119 NullEvt) class_id 2immediate prio 255 cost 10 e119) queued 2026-02-11T23:31:28.033+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.033+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6812/3043114013 map epoch 118 -> 119 (shared) 2026-02-11T23:31:28.033+0000 7f414f993640 10 osd.0 119 send_incremental_map epoch range: (119, 119] to v2:127.0.0.1:6812/3043114013 2026-02-11T23:31:28.033+0000 7f414f993640 10 osd.0 119 add_map_inc_bl 119 252 bytes 2026-02-11T23:31:28.033+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- osd_map(119..119 src has 1..119) -- 0x5955340 con 0x4acb000 2026-02-11T23:31:28.033+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_notify2(1.0 1.0 (query:119 sent:119 1.0( v 111'600 (111'590,111'600] local-lis/les=113/114 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) ([107,117] all_participants={0, 1} intervals=[([113,117] acting={1})])) e119/119) -- 0x5e1ce00 con 0x4acb000 2026-02-11T23:31:28.033+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.033+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.033+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 119 epoch_requested: 119 NullEvt) class_id 2immediate prio 255 cost 10 e119) pg 0x4ef0000 2026-02-11T23:31:28.033+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 unknown mbc={}] do_peering_event: epoch_sent: 119 epoch_requested: 119 NullEvt 2026-02-11T23:31:28.033+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.033+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.033+0000 7f414f993640 15 osd.0 119 enqueue_peering_evt 1.0 epoch_sent: 119 epoch_requested: 119 ActivateCommitted from 119 processed at 119 2026-02-11T23:31:28.033+0000 7f414f993640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 119 epoch_requested: 119 ActivateCommitted from 119 processed at 119) class_id 2 prio 255 cost 10 e119) 2026-02-11T23:31:28.033+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.033+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.033+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 119 epoch_requested: 119 ActivateCommitted from 119 processed at 119) class_id 2immediate prio 255 cost 10 e119) queued 2026-02-11T23:31:28.033+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.033+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 119 epoch_requested: 119 ActivateCommitted from 119 processed at 119) class_id 2immediate prio 255 cost 10 e119) pg 0x4ef0000 2026-02-11T23:31:28.033+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.033+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 unknown mbc={}] do_peering_event: epoch_sent: 119 epoch_requested: 119 ActivateCommitted from 119 processed at 119 2026-02-11T23:31:28.033+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 unknown mbc={}] state: react 119 telling primary 2026-02-11T23:31:28.033+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.033+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.1 from_epoch 119 2026-02-11T23:31:28.033+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.033+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.033+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.033+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_info2(1.0 1.0 1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/107 les/c/f=119/109/0 sis=118) e119/119) -- 0x5e1d500 con 0x4acb000 2026-02-11T23:31:28.033+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.033+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 324 ==== MOSDPGPush(4.0 118/117 [PushOp(4:3a764ab4:::obj37:head, version: 111'37, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3a764ab4:::obj37:head@111'37, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x56ca680 con 0x507ac00 2026-02-11T23:31:28.033+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:3a764ab4:::obj37:head, version: 111'37, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3a764ab4:::obj37:head@111'37, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000015 epoch 118 2026-02-11T23:31:28.033+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:3a764ab4:::obj37:head, version: 111'37, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3a764ab4:::obj37:head@111'37, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:28.033+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.033+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 325 ==== pg_backfill(progress 4.0 e 118/118 lb 4:3945af93:::obj271:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2db80 con 0x507ac00 2026-02-11T23:31:28.033+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:3a764ab4:::obj37:head, version: 111'37, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3a764ab4:::obj37:head@111'37, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:28.033+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:3945af93:::obj271:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 118 2026-02-11T23:31:28.033+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.033+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:3945af93:::obj271:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:28.033+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:3a764ab4:::obj37:head, version: 111'37, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3a764ab4:::obj37:head@111'37, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:28.033+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:3a764ab4:::obj37:head, version: 111'37, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3a764ab4:::obj37:head@111'37, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000043 MOSDPGPush(4.0 118/117 [PushOp(4:3a764ab4:::obj37:head, version: 111'37, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3a764ab4:::obj37:head@111'37, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:385733f0:::obj251:head local-lis/les=117/118 n=142 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.033+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 map epoch 118 -> 119 (shared) 2026-02-11T23:31:28.033+0000 7f414f993640 10 osd.0 119 send_incremental_map epoch range: (119, 119] to v2:127.0.0.1:6836/3331151400 2026-02-11T23:31:28.033+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.033+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- osd_map(119..119 src has 1..119) -- 0x5954c40 con 0x507ac00 2026-02-11T23:31:28.033+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:3945af93:::obj271:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:28.033+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:385733f0:::obj251:head local-lis/les=117/118 n=142 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:3a764ab4:::obj37:head, version: 111'37, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3a764ab4:::obj37:head@111'37, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.033+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1243 ==== pg_lease(1.0 pg_lease(ru 84.697601318s ub 84.698448181s int 16.000000000s) e119/119) ==== 58+0+0 (unknown 0 0 0) 0x63c8c00 con 0x4acb000 2026-02-11T23:31:28.033+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:385733f0:::obj251:head local-lis/les=117/118 n=142 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:3a764ab4:::obj37:head@111'37, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.033+0000 7f416abfa640 15 osd.0 119 enqueue_peering_evt 1.0 epoch_sent: 119 epoch_requested: 119 MLease epoch 119 from osd.1 pg_lease(ru 84.697601318s ub 84.698448181s int 16.000000000s) 2026-02-11T23:31:28.033+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 119 epoch_requested: 119 MLease epoch 119 from osd.1 pg_lease(ru 84.697601318s ub 84.698448181s int 16.000000000s)) class_id 2 prio 255 cost 10 e119) 2026-02-11T23:31:28.033+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:385733f0:::obj251:head local-lis/les=117/118 n=142 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:3a764ab4:::obj37:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.033+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:385733f0:::obj251:head local-lis/les=117/118 n=142 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:3a764ab4:::obj37:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.033+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1244 ==== pg_info2(1.0 1.0 1.0( v 111'600 (111'500,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/107 les/c/f=119/109/0 sis=118) e119/119) ==== 1137+0+0 (unknown 0 0 0) 0x5e1d500 con 0x4acb000 2026-02-11T23:31:28.033+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:385733f0:::obj251:head local-lis/les=117/118 n=142 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:3a764ab4:::obj37:head 2026-02-11T23:31:28.033+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:385733f0:::obj251:head local-lis/les=117/118 n=142 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:3a764ab4:::obj37:head v 111'37 2026-02-11T23:31:28.033+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:385733f0:::obj251:head local-lis/les=117/118 n=142 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.033+0000 7f416abfa640 15 osd.0 119 enqueue_peering_evt 1.0 epoch_sent: 119 epoch_requested: 119 MInfoRec from 1 info: 1.0( v 111'600 (111'500,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/107 les/c/f=119/109/0 sis=118) pg_lease(ru 84.697601318s ub 84.698448181s int 16.000000000s) 2026-02-11T23:31:28.033+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 119 epoch_requested: 119 MInfoRec from 1 info: 1.0( v 111'600 (111'500,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/107 les/c/f=119/109/0 sis=118) pg_lease(ru 84.697601318s ub 84.698448181s int 16.000000000s)) class_id 2 prio 255 cost 10 e119) 2026-02-11T23:31:28.033+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.033+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:3a764ab4:::obj37:head, version: 111'37, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3a764ab4:::obj37:head@111'37, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000071 2026-02-11T23:31:28.033+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.033+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 119 epoch_requested: 119 MLease epoch 119 from osd.1 pg_lease(ru 84.697601318s ub 84.698448181s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e119) queued 2026-02-11T23:31:28.033+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.033+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 119 epoch_requested: 119 MLease epoch 119 from osd.1 pg_lease(ru 84.697601318s ub 84.698448181s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e119) pg 0x4ef0000 2026-02-11T23:31:28.033+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 active mbc={}] do_peering_event: epoch_sent: 119 epoch_requested: 119 MLease epoch 119 from osd.1 pg_lease(ru 84.697601318s ub 84.698448181s int 16.000000000s) 2026-02-11T23:31:28.033+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 active mbc={}] PeeringState::proc_lease pg_lease(ru 84.697601318s ub 84.698448181s int 16.000000000s) 2026-02-11T23:31:28.033+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 active mbc={}] PeeringState::proc_lease peer_clock_delta_ub -2.011524916s -> ru 86.709121704s 2026-02-11T23:31:28.033+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.033+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 active mbc={}] PeeringState::proc_lease readable_until now 86.709121704s 2026-02-11T23:31:28.033+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:3945af93:::obj271:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:28.033+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 active mbc={}] PeeringState::proc_lease peer_clock_delta_lb -2.011766434s -> ruub 86.710212708s 2026-02-11T23:31:28.033+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 active mbc={}] PeeringState::proc_lease readable_until_ub now 86.710212708s 2026-02-11T23:31:28.033+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.1 from_epoch 119 2026-02-11T23:31:28.033+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.033+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.033+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_lease_ack(1.0 pg_lease_ack(ruub 84.698448181s) e119/119) -- 0x5e77e40 con 0x4acb000 2026-02-11T23:31:28.033+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:3945af93:::obj271:head) v3 prio 15 cost 0 latency 0.000100 pg_backfill(progress 4.0 e 118/118 lb 4:3945af93:::obj271:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:385733f0:::obj251:head local-lis/les=117/118 n=142 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.033+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.033+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:385733f0:::obj251:head local-lis/les=117/118 n=142 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:3945af93:::obj271:head) v3 2026-02-11T23:31:28.033+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.033+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:385733f0:::obj251:head local-lis/les=117/118 n=142 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:3945af93:::obj271:head) 2026-02-11T23:31:28.033+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.033+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1245 ==== pg_info2(1.0 1.0 1.0( v 111'600 (111'500,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/0 sis=118) e119/119) ==== 1137+0+0 (unknown 0 0 0) 0x5e1d500 con 0x4acb000 2026-02-11T23:31:28.033+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:3945af93:::obj271:head) v3 finish latency 0.000033 2026-02-11T23:31:28.033+0000 7f416abfa640 15 osd.0 119 enqueue_peering_evt 1.0 epoch_sent: 119 epoch_requested: 119 MInfoRec from 1 info: 1.0( v 111'600 (111'500,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/0 sis=118) pg_lease(ru 84.697601318s ub 84.698448181s int 16.000000000s) 2026-02-11T23:31:28.033+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 119 epoch_requested: 119 MInfoRec from 1 info: 1.0( v 111'600 (111'500,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/0 sis=118) pg_lease(ru 84.697601318s ub 84.698448181s int 16.000000000s)) class_id 2 prio 255 cost 10 e119) 2026-02-11T23:31:28.033+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.033+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3945af93:::obj271:head local-lis/les=117/118 n=143 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.033+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 119 epoch_requested: 119 MInfoRec from 1 info: 1.0( v 111'600 (111'500,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/107 les/c/f=119/109/0 sis=118) pg_lease(ru 84.697601318s ub 84.698448181s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e119) queued 2026-02-11T23:31:28.033+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.033+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.033+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 119 epoch_requested: 119 MInfoRec from 1 info: 1.0( v 111'600 (111'500,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/107 les/c/f=119/109/0 sis=118) pg_lease(ru 84.697601318s ub 84.698448181s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e119) pg 0x4ef0000 2026-02-11T23:31:28.033+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=113/107 les/c/f=114/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 active mbc={}] do_peering_event: epoch_sent: 119 epoch_requested: 119 MInfoRec from 1 info: 1.0( v 111'600 (111'500,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/107 les/c/f=119/109/0 sis=118) pg_lease(ru 84.697601318s ub 84.698448181s int 16.000000000s) 2026-02-11T23:31:28.033+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/107 les/c/f=119/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 active mbc={}] PeeringState::update_history advanced history from ec=27/27 lis/c=118/107 les/c/f=119/109/0 sis=118 2026-02-11T23:31:28.033+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.033+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.033+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 119 epoch_requested: 119 MInfoRec from 1 info: 1.0( v 111'600 (111'500,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/0 sis=118) pg_lease(ru 84.697601318s ub 84.698448181s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e119) queued 2026-02-11T23:31:28.033+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.033+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.033+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 119 epoch_requested: 119 MInfoRec from 1 info: 1.0( v 111'600 (111'500,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/0 sis=118) pg_lease(ru 84.697601318s ub 84.698448181s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e119) pg 0x4ef0000 2026-02-11T23:31:28.033+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/107 les/c/f=119/109/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 active mbc={}] do_peering_event: epoch_sent: 119 epoch_requested: 119 MInfoRec from 1 info: 1.0( v 111'600 (111'500,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/0 sis=118) pg_lease(ru 84.697601318s ub 84.698448181s int 16.000000000s) 2026-02-11T23:31:28.033+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 active mbc={}] PeeringState::update_history advanced history from ec=27/27 lis/c=118/118 les/c/f=119/119/0 sis=118 2026-02-11T23:31:28.033+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/0 sis=118) [1,0] r=1 lpr=118 pi=[107,118)/1 crt=111'600 active mbc={}] PeeringState::update_history clearing past_intervals 2026-02-11T23:31:28.033+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.033+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.033+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.033+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3945af93:::obj271:head local-lis/les=117/118 n=143 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.033+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.033+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.033+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.033+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x5f75c80 con 0x507ac00 2026-02-11T23:31:28.033+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:3a764ab4:::obj37:head)]) -- 0x63c5380 con 0x507ac00 2026-02-11T23:31:28.033+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.033+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.033+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.034+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 326 ==== MOSDPGPush(4.0 118/117 [PushOp(4:3b0aa4c9:::obj511:head, version: 111'511, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3b0aa4c9:::obj511:head@111'511, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2340 con 0x507ac00 2026-02-11T23:31:28.034+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 118/117 [PushOp(4:3b0aa4c9:::obj511:head, version: 111'511, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3b0aa4c9:::obj511:head@111'511, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000016 epoch 118 2026-02-11T23:31:28.034+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:3b0aa4c9:::obj511:head, version: 111'511, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3b0aa4c9:::obj511:head@111'511, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e118) 2026-02-11T23:31:28.034+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.034+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:3b0aa4c9:::obj511:head, version: 111'511, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3b0aa4c9:::obj511:head@111'511, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) queued 2026-02-11T23:31:28.034+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.034+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 118/117 [PushOp(4:3b0aa4c9:::obj511:head, version: 111'511, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3b0aa4c9:::obj511:head@111'511, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e118) pg 0x5978000 2026-02-11T23:31:28.034+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.034+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:3b0aa4c9:::obj511:head, version: 111'511, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3b0aa4c9:::obj511:head@111'511, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000047 MOSDPGPush(4.0 118/117 [PushOp(4:3b0aa4c9:::obj511:head, version: 111'511, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3b0aa4c9:::obj511:head@111'511, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:3945af93:::obj271:head local-lis/les=117/118 n=143 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.034+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.034+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3945af93:::obj271:head local-lis/les=117/118 n=143 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 118/117 [PushOp(4:3b0aa4c9:::obj511:head, version: 111'511, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3b0aa4c9:::obj511:head@111'511, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.034+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3945af93:::obj271:head local-lis/les=117/118 n=143 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:3b0aa4c9:::obj511:head@111'511, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.034+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 328 ==== pg_backfill(progress 4.0 e 118/118 lb 4:3a764ab4:::obj37:head) ==== 1061+0+0 (unknown 0 0 0) 0x6179180 con 0x507ac00 2026-02-11T23:31:28.034+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 118/118 lb 4:3a764ab4:::obj37:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 118 2026-02-11T23:31:28.034+0000 7f415aefe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 327 ==== osd_map(119..119 src has 1..119) ==== 296+0+0 (unknown 0 0 0) 0x5954c40 con 0x507ac00 2026-02-11T23:31:28.034+0000 7f415aefe640 20 osd.0 119 OSD::ms_dispatch: osd_map(119..119 src has 1..119) 2026-02-11T23:31:28.034+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:3a764ab4:::obj37:head) v3) class_id 0 prio 15 cost 0 e118) 2026-02-11T23:31:28.034+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3945af93:::obj271:head local-lis/les=117/118 n=143 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:3b0aa4c9:::obj511:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.034+0000 7f415aefe640 20 osd.0 119 _dispatch 0x5954c40 osd_map(119..119 src has 1..119) 2026-02-11T23:31:28.034+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3945af93:::obj271:head local-lis/les=117/118 n=143 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:3b0aa4c9:::obj511:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.034+0000 7f415aefe640 3 osd.0 119 handle_osd_map epochs [119,119], i have 119, src has [1,119] 2026-02-11T23:31:28.034+0000 7f415aefe640 10 osd.0 119 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:31:28.034+0000 7f415aefe640 10 osd.0 119 no new maps here, dropping 2026-02-11T23:31:28.034+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.034+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3945af93:::obj271:head local-lis/les=117/118 n=143 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:3b0aa4c9:::obj511:head 2026-02-11T23:31:28.034+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:3a764ab4:::obj37:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) queued 2026-02-11T23:31:28.034+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3945af93:::obj271:head local-lis/les=117/118 n=143 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:3b0aa4c9:::obj511:head v 111'511 2026-02-11T23:31:28.034+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3945af93:::obj271:head local-lis/les=117/118 n=143 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.034+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.034+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 118/117 [PushOp(4:3b0aa4c9:::obj511:head, version: 111'511, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3b0aa4c9:::obj511:head@111'511, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000080 2026-02-11T23:31:28.034+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.034+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.034+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 118/118 lb 4:3a764ab4:::obj37:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e118) pg 0x5978000 2026-02-11T23:31:28.034+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:3a764ab4:::obj37:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 4.0 e 118/118 lb 4:3a764ab4:::obj37:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:3945af93:::obj271:head local-lis/les=117/118 n=143 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.034+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.034+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3945af93:::obj271:head local-lis/les=117/118 n=143 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 118/118 lb 4:3a764ab4:::obj37:head) v3 2026-02-11T23:31:28.034+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3945af93:::obj271:head local-lis/les=117/118 n=143 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 118/118 lb 4:3a764ab4:::obj37:head) 2026-02-11T23:31:28.034+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.034+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 118/118 lb 4:3a764ab4:::obj37:head) v3 finish latency 0.000016 2026-02-11T23:31:28.034+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3a764ab4:::obj37:head local-lis/les=117/118 n=144 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.034+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.034+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.034+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3a764ab4:::obj37:head local-lis/les=117/118 n=144 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.034+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.034+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.034+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.034+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c5200 con 0x507ac00 2026-02-11T23:31:28.034+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 118/117 [PushReplyOp(4:3b0aa4c9:::obj511:head)]) -- 0x63c8a80 con 0x507ac00 2026-02-11T23:31:28.034+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.034+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 329 ==== MOSDPGPush(4.0 119/117 [PushOp(4:3bc05e5f:::obj343:head, version: 111'343, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3bc05e5f:::obj343:head@111'343, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x554e4e0 con 0x507ac00 2026-02-11T23:31:28.034+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:3bc05e5f:::obj343:head, version: 111'343, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3bc05e5f:::obj343:head@111'343, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000015 epoch 119 2026-02-11T23:31:28.034+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:3bc05e5f:::obj343:head, version: 111'343, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3bc05e5f:::obj343:head@111'343, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.034+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.034+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 330 ==== pg_backfill(progress 4.0 e 119/119 lb 4:3b0aa4c9:::obj511:head) ==== 1062+0+0 (unknown 0 0 0) 0x6178100 con 0x507ac00 2026-02-11T23:31:28.034+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:3bc05e5f:::obj343:head, version: 111'343, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3bc05e5f:::obj343:head@111'343, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.034+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:3b0aa4c9:::obj511:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.034+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:3b0aa4c9:::obj511:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.034+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.034+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:3bc05e5f:::obj343:head, version: 111'343, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3bc05e5f:::obj343:head@111'343, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.034+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:3bc05e5f:::obj343:head, version: 111'343, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3bc05e5f:::obj343:head@111'343, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000041 MOSDPGPush(4.0 119/117 [PushOp(4:3bc05e5f:::obj343:head, version: 111'343, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3bc05e5f:::obj343:head@111'343, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:3a764ab4:::obj37:head local-lis/les=117/118 n=144 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.034+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.034+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.034+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:3b0aa4c9:::obj511:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.034+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3a764ab4:::obj37:head local-lis/les=117/118 n=144 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:3bc05e5f:::obj343:head, version: 111'343, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3bc05e5f:::obj343:head@111'343, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.034+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3a764ab4:::obj37:head local-lis/les=117/118 n=144 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:3bc05e5f:::obj343:head@111'343, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.034+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3a764ab4:::obj37:head local-lis/les=117/118 n=144 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:3bc05e5f:::obj343:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.034+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3a764ab4:::obj37:head local-lis/les=117/118 n=144 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:3bc05e5f:::obj343:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.034+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3a764ab4:::obj37:head local-lis/les=117/118 n=144 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:3bc05e5f:::obj343:head 2026-02-11T23:31:28.034+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3a764ab4:::obj37:head local-lis/les=117/118 n=144 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:3bc05e5f:::obj343:head v 111'343 2026-02-11T23:31:28.034+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3a764ab4:::obj37:head local-lis/les=117/118 n=144 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.034+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.035+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:3bc05e5f:::obj343:head, version: 111'343, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3bc05e5f:::obj343:head@111'343, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000068 2026-02-11T23:31:28.035+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.035+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.035+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:3b0aa4c9:::obj511:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.035+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:3b0aa4c9:::obj511:head) v3 prio 15 cost 0 latency 0.000087 pg_backfill(progress 4.0 e 119/119 lb 4:3b0aa4c9:::obj511:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:3a764ab4:::obj37:head local-lis/les=117/118 n=144 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.035+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.035+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3a764ab4:::obj37:head local-lis/les=117/118 n=144 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:3b0aa4c9:::obj511:head) v3 2026-02-11T23:31:28.035+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3a764ab4:::obj37:head local-lis/les=117/118 n=144 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:3b0aa4c9:::obj511:head) 2026-02-11T23:31:28.035+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.035+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:3b0aa4c9:::obj511:head) v3 finish latency 0.000014 2026-02-11T23:31:28.035+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3b0aa4c9:::obj511:head local-lis/les=117/118 n=145 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.035+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.035+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.035+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3b0aa4c9:::obj511:head local-lis/les=117/118 n=145 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.035+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.035+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.035+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.035+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x649f200 con 0x507ac00 2026-02-11T23:31:28.035+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:3bc05e5f:::obj343:head)]) -- 0x4c9ac00 con 0x507ac00 2026-02-11T23:31:28.035+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.035+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 331 ==== MOSDPGPush(4.0 119/117 [PushOp(4:3cb06661:::obj109:head, version: 111'109, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cb06661:::obj109:head@111'109, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a3040 con 0x507ac00 2026-02-11T23:31:28.035+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:3cb06661:::obj109:head, version: 111'109, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cb06661:::obj109:head@111'109, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.035+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:3cb06661:::obj109:head, version: 111'109, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cb06661:::obj109:head@111'109, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.035+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.035+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 332 ==== pg_backfill(progress 4.0 e 119/119 lb 4:3bc05e5f:::obj343:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177b80 con 0x507ac00 2026-02-11T23:31:28.035+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:3bc05e5f:::obj343:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.035+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:3cb06661:::obj109:head, version: 111'109, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cb06661:::obj109:head@111'109, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.035+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:3bc05e5f:::obj343:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.035+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.035+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:3cb06661:::obj109:head, version: 111'109, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cb06661:::obj109:head@111'109, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.035+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.035+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:3cb06661:::obj109:head, version: 111'109, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cb06661:::obj109:head@111'109, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(4.0 119/117 [PushOp(4:3cb06661:::obj109:head, version: 111'109, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cb06661:::obj109:head@111'109, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:3b0aa4c9:::obj511:head local-lis/les=117/118 n=145 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.035+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:3bc05e5f:::obj343:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.035+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.035+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3b0aa4c9:::obj511:head local-lis/les=117/118 n=145 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:3cb06661:::obj109:head, version: 111'109, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cb06661:::obj109:head@111'109, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.035+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3b0aa4c9:::obj511:head local-lis/les=117/118 n=145 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:3cb06661:::obj109:head@111'109, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.035+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3b0aa4c9:::obj511:head local-lis/les=117/118 n=145 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:3cb06661:::obj109:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.035+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3b0aa4c9:::obj511:head local-lis/les=117/118 n=145 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:3cb06661:::obj109:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.035+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3b0aa4c9:::obj511:head local-lis/les=117/118 n=145 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:3cb06661:::obj109:head 2026-02-11T23:31:28.035+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3b0aa4c9:::obj511:head local-lis/les=117/118 n=145 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:3cb06661:::obj109:head v 111'109 2026-02-11T23:31:28.035+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3b0aa4c9:::obj511:head local-lis/les=117/118 n=145 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.035+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.035+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:3cb06661:::obj109:head, version: 111'109, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cb06661:::obj109:head@111'109, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000064 2026-02-11T23:31:28.035+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.035+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.035+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:3bc05e5f:::obj343:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.035+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:3bc05e5f:::obj343:head) v3 prio 15 cost 0 latency 0.000087 pg_backfill(progress 4.0 e 119/119 lb 4:3bc05e5f:::obj343:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:3b0aa4c9:::obj511:head local-lis/les=117/118 n=145 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.035+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.035+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3b0aa4c9:::obj511:head local-lis/les=117/118 n=145 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:3bc05e5f:::obj343:head) v3 2026-02-11T23:31:28.035+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3b0aa4c9:::obj511:head local-lis/les=117/118 n=145 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:3bc05e5f:::obj343:head) 2026-02-11T23:31:28.035+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.035+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:3bc05e5f:::obj343:head) v3 finish latency 0.000016 2026-02-11T23:31:28.035+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3bc05e5f:::obj343:head local-lis/les=117/118 n=146 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.035+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.035+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.035+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3bc05e5f:::obj343:head local-lis/les=117/118 n=146 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.035+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.035+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.035+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.035+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c5080 con 0x507ac00 2026-02-11T23:31:28.035+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:3cb06661:::obj109:head)]) -- 0x4c9bc80 con 0x507ac00 2026-02-11T23:31:28.035+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.036+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 333 ==== MOSDPGPush(4.0 119/117 [PushOp(4:3cb2c87d:::obj96:head, version: 111'96, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cb2c87d:::obj96:head@111'96, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x56a3860 con 0x507ac00 2026-02-11T23:31:28.036+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:3cb2c87d:::obj96:head, version: 111'96, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cb2c87d:::obj96:head@111'96, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.036+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:3cb2c87d:::obj96:head, version: 111'96, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cb2c87d:::obj96:head@111'96, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.036+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.036+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 334 ==== pg_backfill(progress 4.0 e 119/119 lb 4:3cb06661:::obj109:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177600 con 0x507ac00 2026-02-11T23:31:28.036+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:3cb06661:::obj109:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.036+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:3cb2c87d:::obj96:head, version: 111'96, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cb2c87d:::obj96:head@111'96, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.036+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:3cb06661:::obj109:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.036+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.036+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:3cb2c87d:::obj96:head, version: 111'96, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cb2c87d:::obj96:head@111'96, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.036+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:3cb2c87d:::obj96:head, version: 111'96, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cb2c87d:::obj96:head@111'96, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(4.0 119/117 [PushOp(4:3cb2c87d:::obj96:head, version: 111'96, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cb2c87d:::obj96:head@111'96, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:3bc05e5f:::obj343:head local-lis/les=117/118 n=146 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.036+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.036+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.036+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:3cb06661:::obj109:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.036+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3bc05e5f:::obj343:head local-lis/les=117/118 n=146 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:3cb2c87d:::obj96:head, version: 111'96, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cb2c87d:::obj96:head@111'96, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.036+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3bc05e5f:::obj343:head local-lis/les=117/118 n=146 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:3cb2c87d:::obj96:head@111'96, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.036+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3bc05e5f:::obj343:head local-lis/les=117/118 n=146 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:3cb2c87d:::obj96:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.036+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3bc05e5f:::obj343:head local-lis/les=117/118 n=146 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:3cb2c87d:::obj96:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.036+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3bc05e5f:::obj343:head local-lis/les=117/118 n=146 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:3cb2c87d:::obj96:head 2026-02-11T23:31:28.036+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3bc05e5f:::obj343:head local-lis/les=117/118 n=146 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:3cb2c87d:::obj96:head v 111'96 2026-02-11T23:31:28.036+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3bc05e5f:::obj343:head local-lis/les=117/118 n=146 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.036+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.036+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:3cb2c87d:::obj96:head, version: 111'96, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cb2c87d:::obj96:head@111'96, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000065 2026-02-11T23:31:28.036+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.036+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.036+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:3cb06661:::obj109:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.036+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:3cb06661:::obj109:head) v3 prio 15 cost 0 latency 0.000088 pg_backfill(progress 4.0 e 119/119 lb 4:3cb06661:::obj109:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:3bc05e5f:::obj343:head local-lis/les=117/118 n=146 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.036+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.036+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3bc05e5f:::obj343:head local-lis/les=117/118 n=146 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:3cb06661:::obj109:head) v3 2026-02-11T23:31:28.036+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3bc05e5f:::obj343:head local-lis/les=117/118 n=146 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:3cb06661:::obj109:head) 2026-02-11T23:31:28.036+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.036+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:3cb06661:::obj109:head) v3 finish latency 0.000016 2026-02-11T23:31:28.036+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3cb06661:::obj109:head local-lis/les=117/118 n=147 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.036+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.036+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.036+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3cb06661:::obj109:head local-lis/les=117/118 n=147 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.036+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.036+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.036+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.036+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x5f2dc80 con 0x507ac00 2026-02-11T23:31:28.036+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:3cb2c87d:::obj96:head)]) -- 0x4f42180 con 0x507ac00 2026-02-11T23:31:28.036+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.037+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 335 ==== MOSDPGPush(4.0 119/117 [PushOp(4:3cc20580:::obj177:head, version: 111'177, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cc20580:::obj177:head@111'177, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56171e0 con 0x507ac00 2026-02-11T23:31:28.037+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:3cc20580:::obj177:head, version: 111'177, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cc20580:::obj177:head@111'177, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 119 2026-02-11T23:31:28.037+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:3cc20580:::obj177:head, version: 111'177, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cc20580:::obj177:head@111'177, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.037+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.037+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 336 ==== pg_backfill(progress 4.0 e 119/119 lb 4:3cb2c87d:::obj96:head) ==== 1061+0+0 (unknown 0 0 0) 0x5985700 con 0x507ac00 2026-02-11T23:31:28.037+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:3cb2c87d:::obj96:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.037+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:3cc20580:::obj177:head, version: 111'177, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cc20580:::obj177:head@111'177, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.037+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:3cb2c87d:::obj96:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.037+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.037+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:3cc20580:::obj177:head, version: 111'177, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cc20580:::obj177:head@111'177, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.037+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.037+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:3cc20580:::obj177:head, version: 111'177, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cc20580:::obj177:head@111'177, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(4.0 119/117 [PushOp(4:3cc20580:::obj177:head, version: 111'177, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cc20580:::obj177:head@111'177, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:3cb06661:::obj109:head local-lis/les=117/118 n=147 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.037+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:3cb2c87d:::obj96:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.037+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.037+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3cb06661:::obj109:head local-lis/les=117/118 n=147 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:3cc20580:::obj177:head, version: 111'177, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cc20580:::obj177:head@111'177, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.037+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3cb06661:::obj109:head local-lis/les=117/118 n=147 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:3cc20580:::obj177:head@111'177, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.037+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3cb06661:::obj109:head local-lis/les=117/118 n=147 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:3cc20580:::obj177:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.037+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3cb06661:::obj109:head local-lis/les=117/118 n=147 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:3cc20580:::obj177:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.037+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3cb06661:::obj109:head local-lis/les=117/118 n=147 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:3cc20580:::obj177:head 2026-02-11T23:31:28.037+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3cb06661:::obj109:head local-lis/les=117/118 n=147 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:3cc20580:::obj177:head v 111'177 2026-02-11T23:31:28.037+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3cb06661:::obj109:head local-lis/les=117/118 n=147 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.037+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.037+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:3cc20580:::obj177:head, version: 111'177, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cc20580:::obj177:head@111'177, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:28.037+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.037+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.037+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:3cb2c87d:::obj96:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.037+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:3cb2c87d:::obj96:head) v3 prio 15 cost 0 latency 0.000081 pg_backfill(progress 4.0 e 119/119 lb 4:3cb2c87d:::obj96:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:3cb06661:::obj109:head local-lis/les=117/118 n=147 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.037+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.037+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3cb06661:::obj109:head local-lis/les=117/118 n=147 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:3cb2c87d:::obj96:head) v3 2026-02-11T23:31:28.037+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3cb06661:::obj109:head local-lis/les=117/118 n=147 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:3cb2c87d:::obj96:head) 2026-02-11T23:31:28.037+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.037+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:3cb2c87d:::obj96:head) v3 finish latency 0.000016 2026-02-11T23:31:28.037+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3cb2c87d:::obj96:head local-lis/les=117/118 n=148 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.037+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.037+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.037+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3cb2c87d:::obj96:head local-lis/les=117/118 n=148 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.037+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.037+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.037+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.037+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c4f00 con 0x507ac00 2026-02-11T23:31:28.037+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:3cc20580:::obj177:head)]) -- 0x4f42300 con 0x507ac00 2026-02-11T23:31:28.037+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.037+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 337 ==== MOSDPGPush(4.0 119/117 [PushOp(4:3d7e2b4d:::obj143:head, version: 111'143, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d7e2b4d:::obj143:head@111'143, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56cad00 con 0x507ac00 2026-02-11T23:31:28.037+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:3d7e2b4d:::obj143:head, version: 111'143, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d7e2b4d:::obj143:head@111'143, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.037+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:3d7e2b4d:::obj143:head, version: 111'143, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d7e2b4d:::obj143:head@111'143, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.037+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.037+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 338 ==== pg_backfill(progress 4.0 e 119/119 lb 4:3cc20580:::obj177:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9a680 con 0x507ac00 2026-02-11T23:31:28.037+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:3d7e2b4d:::obj143:head, version: 111'143, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d7e2b4d:::obj143:head@111'143, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.037+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:3cc20580:::obj177:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.037+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:3cc20580:::obj177:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.037+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.037+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:3d7e2b4d:::obj143:head, version: 111'143, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d7e2b4d:::obj143:head@111'143, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.037+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.037+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:3d7e2b4d:::obj143:head, version: 111'143, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d7e2b4d:::obj143:head@111'143, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 119/117 [PushOp(4:3d7e2b4d:::obj143:head, version: 111'143, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d7e2b4d:::obj143:head@111'143, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:3cb2c87d:::obj96:head local-lis/les=117/118 n=148 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.037+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.037+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3cb2c87d:::obj96:head local-lis/les=117/118 n=148 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:3d7e2b4d:::obj143:head, version: 111'143, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d7e2b4d:::obj143:head@111'143, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.037+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3cb2c87d:::obj96:head local-lis/les=117/118 n=148 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:3d7e2b4d:::obj143:head@111'143, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.037+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.037+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:3cc20580:::obj177:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.037+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3cb2c87d:::obj96:head local-lis/les=117/118 n=148 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:3d7e2b4d:::obj143:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.037+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3cb2c87d:::obj96:head local-lis/les=117/118 n=148 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:3d7e2b4d:::obj143:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.037+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3cb2c87d:::obj96:head local-lis/les=117/118 n=148 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:3d7e2b4d:::obj143:head 2026-02-11T23:31:28.037+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3cb2c87d:::obj96:head local-lis/les=117/118 n=148 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:3d7e2b4d:::obj143:head v 111'143 2026-02-11T23:31:28.037+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3cb2c87d:::obj96:head local-lis/les=117/118 n=148 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.037+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.037+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:3d7e2b4d:::obj143:head, version: 111'143, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d7e2b4d:::obj143:head@111'143, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:28.037+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.037+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.037+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:3cc20580:::obj177:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.037+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:3cc20580:::obj177:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 4.0 e 119/119 lb 4:3cc20580:::obj177:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:3cb2c87d:::obj96:head local-lis/les=117/118 n=148 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.037+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.037+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3cb2c87d:::obj96:head local-lis/les=117/118 n=148 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:3cc20580:::obj177:head) v3 2026-02-11T23:31:28.037+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3cb2c87d:::obj96:head local-lis/les=117/118 n=148 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:3cc20580:::obj177:head) 2026-02-11T23:31:28.037+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.037+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:3cc20580:::obj177:head) v3 finish latency 0.000014 2026-02-11T23:31:28.037+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3cc20580:::obj177:head local-lis/les=117/118 n=149 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.037+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.037+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.037+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3cc20580:::obj177:head local-lis/les=117/118 n=149 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.037+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.037+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.037+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.037+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x649f080 con 0x507ac00 2026-02-11T23:31:28.037+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:3d7e2b4d:::obj143:head)]) -- 0x4f42600 con 0x507ac00 2026-02-11T23:31:28.037+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.038+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 339 ==== MOSDPGPush(4.0 119/117 [PushOp(4:3d8750d8:::obj471:head, version: 111'471, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d8750d8:::obj471:head@111'471, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x555cea0 con 0x507ac00 2026-02-11T23:31:28.038+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:3d8750d8:::obj471:head, version: 111'471, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d8750d8:::obj471:head@111'471, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.038+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:3d8750d8:::obj471:head, version: 111'471, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d8750d8:::obj471:head@111'471, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.038+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.038+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 340 ==== pg_backfill(progress 4.0 e 119/119 lb 4:3d7e2b4d:::obj143:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176000 con 0x507ac00 2026-02-11T23:31:28.038+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:3d7e2b4d:::obj143:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.038+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:3d8750d8:::obj471:head, version: 111'471, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d8750d8:::obj471:head@111'471, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.038+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:3d7e2b4d:::obj143:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.038+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.038+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:3d8750d8:::obj471:head, version: 111'471, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d8750d8:::obj471:head@111'471, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.038+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:3d8750d8:::obj471:head, version: 111'471, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d8750d8:::obj471:head@111'471, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 119/117 [PushOp(4:3d8750d8:::obj471:head, version: 111'471, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d8750d8:::obj471:head@111'471, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:3cc20580:::obj177:head local-lis/les=117/118 n=149 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.038+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.038+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.038+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:3d7e2b4d:::obj143:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.038+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3cc20580:::obj177:head local-lis/les=117/118 n=149 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:3d8750d8:::obj471:head, version: 111'471, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d8750d8:::obj471:head@111'471, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.038+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3cc20580:::obj177:head local-lis/les=117/118 n=149 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:3d8750d8:::obj471:head@111'471, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.038+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3cc20580:::obj177:head local-lis/les=117/118 n=149 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:3d8750d8:::obj471:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.038+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3cc20580:::obj177:head local-lis/les=117/118 n=149 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:3d8750d8:::obj471:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.038+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3cc20580:::obj177:head local-lis/les=117/118 n=149 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:3d8750d8:::obj471:head 2026-02-11T23:31:28.038+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3cc20580:::obj177:head local-lis/les=117/118 n=149 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:3d8750d8:::obj471:head v 111'471 2026-02-11T23:31:28.038+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3cc20580:::obj177:head local-lis/les=117/118 n=149 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.038+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.038+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:3d8750d8:::obj471:head, version: 111'471, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d8750d8:::obj471:head@111'471, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000062 2026-02-11T23:31:28.038+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.038+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.038+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:3d7e2b4d:::obj143:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.038+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:3d7e2b4d:::obj143:head) v3 prio 15 cost 0 latency 0.000084 pg_backfill(progress 4.0 e 119/119 lb 4:3d7e2b4d:::obj143:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:3cc20580:::obj177:head local-lis/les=117/118 n=149 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.038+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.038+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3cc20580:::obj177:head local-lis/les=117/118 n=149 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:3d7e2b4d:::obj143:head) v3 2026-02-11T23:31:28.038+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3cc20580:::obj177:head local-lis/les=117/118 n=149 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:3d7e2b4d:::obj143:head) 2026-02-11T23:31:28.038+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.038+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:3d7e2b4d:::obj143:head) v3 finish latency 0.000014 2026-02-11T23:31:28.038+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3d7e2b4d:::obj143:head local-lis/les=117/118 n=150 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.038+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.038+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.038+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3d7e2b4d:::obj143:head local-lis/les=117/118 n=150 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.038+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.038+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.038+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.038+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c4d80 con 0x507ac00 2026-02-11T23:31:28.038+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:3d8750d8:::obj471:head)]) -- 0x4f42900 con 0x507ac00 2026-02-11T23:31:28.038+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.039+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 341 ==== MOSDPGPush(4.0 119/117 [PushOp(4:3d88a4f9:::obj551:head, version: 111'551, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d88a4f9:::obj551:head@111'551, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f1ba0 con 0x507ac00 2026-02-11T23:31:28.039+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:3d88a4f9:::obj551:head, version: 111'551, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d88a4f9:::obj551:head@111'551, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.039+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:3d88a4f9:::obj551:head, version: 111'551, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d88a4f9:::obj551:head@111'551, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.039+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.039+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:3d88a4f9:::obj551:head, version: 111'551, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d88a4f9:::obj551:head@111'551, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.039+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.039+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 342 ==== pg_backfill(progress 4.0 e 119/119 lb 4:3d8750d8:::obj471:head) ==== 1062+0+0 (unknown 0 0 0) 0x6179700 con 0x507ac00 2026-02-11T23:31:28.039+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:3d88a4f9:::obj551:head, version: 111'551, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d88a4f9:::obj551:head@111'551, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.039+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:3d8750d8:::obj471:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 119 2026-02-11T23:31:28.039+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:3d8750d8:::obj471:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.039+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.039+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:3d88a4f9:::obj551:head, version: 111'551, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d88a4f9:::obj551:head@111'551, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(4.0 119/117 [PushOp(4:3d88a4f9:::obj551:head, version: 111'551, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d88a4f9:::obj551:head@111'551, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:3d7e2b4d:::obj143:head local-lis/les=117/118 n=150 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.039+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.039+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:3d8750d8:::obj471:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.039+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3d7e2b4d:::obj143:head local-lis/les=117/118 n=150 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:3d88a4f9:::obj551:head, version: 111'551, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d88a4f9:::obj551:head@111'551, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.039+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3d7e2b4d:::obj143:head local-lis/les=117/118 n=150 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:3d88a4f9:::obj551:head@111'551, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.039+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3d7e2b4d:::obj143:head local-lis/les=117/118 n=150 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:3d88a4f9:::obj551:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.039+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3d7e2b4d:::obj143:head local-lis/les=117/118 n=150 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:3d88a4f9:::obj551:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.039+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3d7e2b4d:::obj143:head local-lis/les=117/118 n=150 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:3d88a4f9:::obj551:head 2026-02-11T23:31:28.039+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3d7e2b4d:::obj143:head local-lis/les=117/118 n=150 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:3d88a4f9:::obj551:head v 111'551 2026-02-11T23:31:28.039+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3d7e2b4d:::obj143:head local-lis/les=117/118 n=150 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.039+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.039+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:3d88a4f9:::obj551:head, version: 111'551, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d88a4f9:::obj551:head@111'551, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000069 2026-02-11T23:31:28.039+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.039+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.039+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:3d8750d8:::obj471:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.039+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:3d8750d8:::obj471:head) v3 prio 15 cost 0 latency 0.000087 pg_backfill(progress 4.0 e 119/119 lb 4:3d8750d8:::obj471:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:3d7e2b4d:::obj143:head local-lis/les=117/118 n=150 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.039+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.039+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3d7e2b4d:::obj143:head local-lis/les=117/118 n=150 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:3d8750d8:::obj471:head) v3 2026-02-11T23:31:28.039+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3d7e2b4d:::obj143:head local-lis/les=117/118 n=150 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:3d8750d8:::obj471:head) 2026-02-11T23:31:28.039+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.039+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:3d8750d8:::obj471:head) v3 finish latency 0.000015 2026-02-11T23:31:28.039+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3d8750d8:::obj471:head local-lis/les=117/118 n=151 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.039+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.039+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.039+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3d8750d8:::obj471:head local-lis/les=117/118 n=151 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.039+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.039+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.039+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.039+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x649ec00 con 0x507ac00 2026-02-11T23:31:28.039+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:3d88a4f9:::obj551:head)]) -- 0x4f42c00 con 0x507ac00 2026-02-11T23:31:28.039+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.039+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 343 ==== MOSDPGPush(4.0 119/117 [PushOp(4:3e07a120:::obj540:head, version: 111'540, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3e07a120:::obj540:head@111'540, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c340 con 0x507ac00 2026-02-11T23:31:28.039+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:3e07a120:::obj540:head, version: 111'540, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3e07a120:::obj540:head@111'540, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.039+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:3e07a120:::obj540:head, version: 111'540, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3e07a120:::obj540:head@111'540, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.039+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.039+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 344 ==== pg_backfill(progress 4.0 e 119/119 lb 4:3d88a4f9:::obj551:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2cb00 con 0x507ac00 2026-02-11T23:31:28.039+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:3e07a120:::obj540:head, version: 111'540, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3e07a120:::obj540:head@111'540, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.039+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:3d88a4f9:::obj551:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.039+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:3d88a4f9:::obj551:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.039+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.039+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:3e07a120:::obj540:head, version: 111'540, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3e07a120:::obj540:head@111'540, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.039+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:3e07a120:::obj540:head, version: 111'540, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3e07a120:::obj540:head@111'540, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 119/117 [PushOp(4:3e07a120:::obj540:head, version: 111'540, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3e07a120:::obj540:head@111'540, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:3d8750d8:::obj471:head local-lis/les=117/118 n=151 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.039+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.039+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.039+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3d8750d8:::obj471:head local-lis/les=117/118 n=151 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:3e07a120:::obj540:head, version: 111'540, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3e07a120:::obj540:head@111'540, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.039+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:3d88a4f9:::obj551:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.039+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3d8750d8:::obj471:head local-lis/les=117/118 n=151 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:3e07a120:::obj540:head@111'540, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.039+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3d8750d8:::obj471:head local-lis/les=117/118 n=151 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:3e07a120:::obj540:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.039+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3d8750d8:::obj471:head local-lis/les=117/118 n=151 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:3e07a120:::obj540:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.039+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3d8750d8:::obj471:head local-lis/les=117/118 n=151 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:3e07a120:::obj540:head 2026-02-11T23:31:28.039+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3d8750d8:::obj471:head local-lis/les=117/118 n=151 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:3e07a120:::obj540:head v 111'540 2026-02-11T23:31:28.039+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3d8750d8:::obj471:head local-lis/les=117/118 n=151 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.039+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.039+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:3e07a120:::obj540:head, version: 111'540, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3e07a120:::obj540:head@111'540, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:28.039+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.039+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.039+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:3d88a4f9:::obj551:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.039+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:3d88a4f9:::obj551:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 4.0 e 119/119 lb 4:3d88a4f9:::obj551:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:3d8750d8:::obj471:head local-lis/les=117/118 n=151 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.039+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.039+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3d8750d8:::obj471:head local-lis/les=117/118 n=151 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:3d88a4f9:::obj551:head) v3 2026-02-11T23:31:28.039+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3d8750d8:::obj471:head local-lis/les=117/118 n=151 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:3d88a4f9:::obj551:head) 2026-02-11T23:31:28.039+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.039+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:3d88a4f9:::obj551:head) v3 finish latency 0.000015 2026-02-11T23:31:28.039+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3d88a4f9:::obj551:head local-lis/les=117/118 n=152 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.039+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.040+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.040+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3d88a4f9:::obj551:head local-lis/les=117/118 n=152 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.040+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.040+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.040+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.040+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c4c00 con 0x507ac00 2026-02-11T23:31:28.040+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:3e07a120:::obj540:head)]) -- 0x4f42f00 con 0x507ac00 2026-02-11T23:31:28.040+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.040+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 345 ==== MOSDPGPush(4.0 119/117 [PushOp(4:3ed3e33d:::obj376:head, version: 111'376, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3ed3e33d:::obj376:head@111'376, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3814d00 con 0x507ac00 2026-02-11T23:31:28.040+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:3ed3e33d:::obj376:head, version: 111'376, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3ed3e33d:::obj376:head@111'376, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.040+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:3ed3e33d:::obj376:head, version: 111'376, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3ed3e33d:::obj376:head@111'376, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.040+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.040+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 346 ==== pg_backfill(progress 4.0 e 119/119 lb 4:3e07a120:::obj540:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2c000 con 0x507ac00 2026-02-11T23:31:28.040+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:3e07a120:::obj540:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.040+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:3ed3e33d:::obj376:head, version: 111'376, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3ed3e33d:::obj376:head@111'376, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.040+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:3e07a120:::obj540:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.040+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.040+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:3ed3e33d:::obj376:head, version: 111'376, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3ed3e33d:::obj376:head@111'376, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.040+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.040+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:3ed3e33d:::obj376:head, version: 111'376, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3ed3e33d:::obj376:head@111'376, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 119/117 [PushOp(4:3ed3e33d:::obj376:head, version: 111'376, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3ed3e33d:::obj376:head@111'376, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:3d88a4f9:::obj551:head local-lis/les=117/118 n=152 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.040+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:3e07a120:::obj540:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.040+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.040+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3d88a4f9:::obj551:head local-lis/les=117/118 n=152 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:3ed3e33d:::obj376:head, version: 111'376, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3ed3e33d:::obj376:head@111'376, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.040+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3d88a4f9:::obj551:head local-lis/les=117/118 n=152 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:3ed3e33d:::obj376:head@111'376, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.040+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3d88a4f9:::obj551:head local-lis/les=117/118 n=152 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:3ed3e33d:::obj376:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.040+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3d88a4f9:::obj551:head local-lis/les=117/118 n=152 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:3ed3e33d:::obj376:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.040+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3d88a4f9:::obj551:head local-lis/les=117/118 n=152 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:3ed3e33d:::obj376:head 2026-02-11T23:31:28.040+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3d88a4f9:::obj551:head local-lis/les=117/118 n=152 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:3ed3e33d:::obj376:head v 111'376 2026-02-11T23:31:28.040+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3d88a4f9:::obj551:head local-lis/les=117/118 n=152 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.040+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.040+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:3ed3e33d:::obj376:head, version: 111'376, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3ed3e33d:::obj376:head@111'376, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:28.040+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.040+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.040+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:3e07a120:::obj540:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.040+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:3e07a120:::obj540:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 4.0 e 119/119 lb 4:3e07a120:::obj540:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:3d88a4f9:::obj551:head local-lis/les=117/118 n=152 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.040+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.040+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3d88a4f9:::obj551:head local-lis/les=117/118 n=152 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:3e07a120:::obj540:head) v3 2026-02-11T23:31:28.040+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3d88a4f9:::obj551:head local-lis/les=117/118 n=152 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:3e07a120:::obj540:head) 2026-02-11T23:31:28.040+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.040+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:3e07a120:::obj540:head) v3 finish latency 0.000014 2026-02-11T23:31:28.040+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3e07a120:::obj540:head local-lis/les=117/118 n=153 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.040+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.040+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.040+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3e07a120:::obj540:head local-lis/les=117/118 n=153 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.040+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.040+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.040+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.040+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x649ef00 con 0x507ac00 2026-02-11T23:31:28.040+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:3ed3e33d:::obj376:head)]) -- 0x4f43200 con 0x507ac00 2026-02-11T23:31:28.040+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.041+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 347 ==== MOSDPGPush(4.0 119/117 [PushOp(4:3efbd745:::obj84:head, version: 111'84, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3efbd745:::obj84:head@111'84, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5e261a0 con 0x507ac00 2026-02-11T23:31:28.041+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:3efbd745:::obj84:head, version: 111'84, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3efbd745:::obj84:head@111'84, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.041+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:3efbd745:::obj84:head, version: 111'84, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3efbd745:::obj84:head@111'84, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.041+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.041+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 348 ==== pg_backfill(progress 4.0 e 119/119 lb 4:3ed3e33d:::obj376:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e100 con 0x507ac00 2026-02-11T23:31:28.041+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:3ed3e33d:::obj376:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.041+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:3efbd745:::obj84:head, version: 111'84, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3efbd745:::obj84:head@111'84, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.041+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:3ed3e33d:::obj376:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.041+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.041+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:3efbd745:::obj84:head, version: 111'84, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3efbd745:::obj84:head@111'84, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.041+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.041+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:3efbd745:::obj84:head, version: 111'84, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3efbd745:::obj84:head@111'84, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 119/117 [PushOp(4:3efbd745:::obj84:head, version: 111'84, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3efbd745:::obj84:head@111'84, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:3e07a120:::obj540:head local-lis/les=117/118 n=153 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.041+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:3ed3e33d:::obj376:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.041+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.041+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3e07a120:::obj540:head local-lis/les=117/118 n=153 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:3efbd745:::obj84:head, version: 111'84, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3efbd745:::obj84:head@111'84, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.041+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3e07a120:::obj540:head local-lis/les=117/118 n=153 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:3efbd745:::obj84:head@111'84, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.041+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3e07a120:::obj540:head local-lis/les=117/118 n=153 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:3efbd745:::obj84:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.041+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3e07a120:::obj540:head local-lis/les=117/118 n=153 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:3efbd745:::obj84:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.041+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3e07a120:::obj540:head local-lis/les=117/118 n=153 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:3efbd745:::obj84:head 2026-02-11T23:31:28.041+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3e07a120:::obj540:head local-lis/les=117/118 n=153 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:3efbd745:::obj84:head v 111'84 2026-02-11T23:31:28.041+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3e07a120:::obj540:head local-lis/les=117/118 n=153 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.041+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.041+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:3efbd745:::obj84:head, version: 111'84, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3efbd745:::obj84:head@111'84, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:28.041+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.041+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.041+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:3ed3e33d:::obj376:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.041+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:3ed3e33d:::obj376:head) v3 prio 15 cost 0 latency 0.000080 pg_backfill(progress 4.0 e 119/119 lb 4:3ed3e33d:::obj376:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:3e07a120:::obj540:head local-lis/les=117/118 n=153 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.041+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.041+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3e07a120:::obj540:head local-lis/les=117/118 n=153 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:3ed3e33d:::obj376:head) v3 2026-02-11T23:31:28.041+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3e07a120:::obj540:head local-lis/les=117/118 n=153 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:3ed3e33d:::obj376:head) 2026-02-11T23:31:28.041+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.041+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:3ed3e33d:::obj376:head) v3 finish latency 0.000015 2026-02-11T23:31:28.041+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3ed3e33d:::obj376:head local-lis/les=117/118 n=154 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.041+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.041+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.041+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3ed3e33d:::obj376:head local-lis/les=117/118 n=154 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.041+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.041+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.041+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.041+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c4a80 con 0x507ac00 2026-02-11T23:31:28.041+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:3efbd745:::obj84:head)]) -- 0x4f43500 con 0x507ac00 2026-02-11T23:31:28.041+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.041+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 349 ==== MOSDPGPush(4.0 119/117 [PushOp(4:3fc284d5:::obj140:head, version: 111'140, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3fc284d5:::obj140:head@111'140, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f0ea0 con 0x507ac00 2026-02-11T23:31:28.041+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:3fc284d5:::obj140:head, version: 111'140, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3fc284d5:::obj140:head@111'140, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.041+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:3fc284d5:::obj140:head, version: 111'140, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3fc284d5:::obj140:head@111'140, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.041+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.041+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 350 ==== pg_backfill(progress 4.0 e 119/119 lb 4:3efbd745:::obj84:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d2e680 con 0x507ac00 2026-02-11T23:31:28.041+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:3efbd745:::obj84:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.041+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:3fc284d5:::obj140:head, version: 111'140, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3fc284d5:::obj140:head@111'140, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.041+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:3efbd745:::obj84:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.041+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.041+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:3fc284d5:::obj140:head, version: 111'140, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3fc284d5:::obj140:head@111'140, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.041+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.041+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:3fc284d5:::obj140:head, version: 111'140, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3fc284d5:::obj140:head@111'140, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(4.0 119/117 [PushOp(4:3fc284d5:::obj140:head, version: 111'140, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3fc284d5:::obj140:head@111'140, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:3ed3e33d:::obj376:head local-lis/les=117/118 n=154 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.041+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:3efbd745:::obj84:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.041+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.041+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3ed3e33d:::obj376:head local-lis/les=117/118 n=154 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:3fc284d5:::obj140:head, version: 111'140, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3fc284d5:::obj140:head@111'140, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.041+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3ed3e33d:::obj376:head local-lis/les=117/118 n=154 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:3fc284d5:::obj140:head@111'140, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.041+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3ed3e33d:::obj376:head local-lis/les=117/118 n=154 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:3fc284d5:::obj140:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.041+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3ed3e33d:::obj376:head local-lis/les=117/118 n=154 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:3fc284d5:::obj140:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.041+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3ed3e33d:::obj376:head local-lis/les=117/118 n=154 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:3fc284d5:::obj140:head 2026-02-11T23:31:28.041+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3ed3e33d:::obj376:head local-lis/les=117/118 n=154 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:3fc284d5:::obj140:head v 111'140 2026-02-11T23:31:28.041+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3ed3e33d:::obj376:head local-lis/les=117/118 n=154 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.041+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.041+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:3fc284d5:::obj140:head, version: 111'140, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3fc284d5:::obj140:head@111'140, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:28.041+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.041+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.041+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:3efbd745:::obj84:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.041+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:3efbd745:::obj84:head) v3 prio 15 cost 0 latency 0.000080 pg_backfill(progress 4.0 e 119/119 lb 4:3efbd745:::obj84:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:3ed3e33d:::obj376:head local-lis/les=117/118 n=154 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.041+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.041+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3ed3e33d:::obj376:head local-lis/les=117/118 n=154 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:3efbd745:::obj84:head) v3 2026-02-11T23:31:28.041+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3ed3e33d:::obj376:head local-lis/les=117/118 n=154 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:3efbd745:::obj84:head) 2026-02-11T23:31:28.041+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.041+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:3efbd745:::obj84:head) v3 finish latency 0.000014 2026-02-11T23:31:28.041+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3efbd745:::obj84:head local-lis/les=117/118 n=155 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.041+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.042+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.042+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3efbd745:::obj84:head local-lis/les=117/118 n=155 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.042+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.042+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.042+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.042+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x5f75b00 con 0x507ac00 2026-02-11T23:31:28.042+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:3fc284d5:::obj140:head)]) -- 0x4f43680 con 0x507ac00 2026-02-11T23:31:28.042+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.042+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 351 ==== MOSDPGPush(4.0 119/117 [PushOp(4:3fe0a5f5:::obj46:head, version: 111'46, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3fe0a5f5:::obj46:head@111'46, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x55f1ba0 con 0x507ac00 2026-02-11T23:31:28.042+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:3fe0a5f5:::obj46:head, version: 111'46, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3fe0a5f5:::obj46:head@111'46, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 119 2026-02-11T23:31:28.042+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:3fe0a5f5:::obj46:head, version: 111'46, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3fe0a5f5:::obj46:head@111'46, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.042+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.042+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 352 ==== pg_backfill(progress 4.0 e 119/119 lb 4:3fc284d5:::obj140:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2d080 con 0x507ac00 2026-02-11T23:31:28.042+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:3fc284d5:::obj140:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.042+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:3fe0a5f5:::obj46:head, version: 111'46, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3fe0a5f5:::obj46:head@111'46, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.042+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:3fc284d5:::obj140:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.042+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.042+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:3fe0a5f5:::obj46:head, version: 111'46, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3fe0a5f5:::obj46:head@111'46, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.042+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.042+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:3fe0a5f5:::obj46:head, version: 111'46, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3fe0a5f5:::obj46:head@111'46, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000041 MOSDPGPush(4.0 119/117 [PushOp(4:3fe0a5f5:::obj46:head, version: 111'46, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3fe0a5f5:::obj46:head@111'46, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:3efbd745:::obj84:head local-lis/les=117/118 n=155 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.042+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:3fc284d5:::obj140:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.042+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.042+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3efbd745:::obj84:head local-lis/les=117/118 n=155 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:3fe0a5f5:::obj46:head, version: 111'46, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3fe0a5f5:::obj46:head@111'46, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.042+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3efbd745:::obj84:head local-lis/les=117/118 n=155 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:3fe0a5f5:::obj46:head@111'46, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.042+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3efbd745:::obj84:head local-lis/les=117/118 n=155 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:3fe0a5f5:::obj46:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.042+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3efbd745:::obj84:head local-lis/les=117/118 n=155 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:3fe0a5f5:::obj46:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.042+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3efbd745:::obj84:head local-lis/les=117/118 n=155 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:3fe0a5f5:::obj46:head 2026-02-11T23:31:28.042+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3efbd745:::obj84:head local-lis/les=117/118 n=155 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:3fe0a5f5:::obj46:head v 111'46 2026-02-11T23:31:28.042+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3efbd745:::obj84:head local-lis/les=117/118 n=155 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.042+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.042+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:3fe0a5f5:::obj46:head, version: 111'46, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3fe0a5f5:::obj46:head@111'46, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:28.042+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.042+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.042+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:3fc284d5:::obj140:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.042+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:3fc284d5:::obj140:head) v3 prio 15 cost 0 latency 0.000085 pg_backfill(progress 4.0 e 119/119 lb 4:3fc284d5:::obj140:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:3efbd745:::obj84:head local-lis/les=117/118 n=155 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.042+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.042+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3efbd745:::obj84:head local-lis/les=117/118 n=155 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:3fc284d5:::obj140:head) v3 2026-02-11T23:31:28.042+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3efbd745:::obj84:head local-lis/les=117/118 n=155 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:3fc284d5:::obj140:head) 2026-02-11T23:31:28.042+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.042+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:3fc284d5:::obj140:head) v3 finish latency 0.000015 2026-02-11T23:31:28.042+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3fc284d5:::obj140:head local-lis/les=117/118 n=156 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.042+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.042+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.042+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3fc284d5:::obj140:head local-lis/les=117/118 n=156 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.042+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.042+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.042+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.042+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c4900 con 0x507ac00 2026-02-11T23:31:28.042+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:3fe0a5f5:::obj46:head)]) -- 0x4f43980 con 0x507ac00 2026-02-11T23:31:28.042+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.043+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 353 ==== MOSDPGPush(4.0 119/117 [PushOp(4:401e0147:::obj213:head, version: 111'213, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:401e0147:::obj213:head@111'213, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x555cea0 con 0x507ac00 2026-02-11T23:31:28.043+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:401e0147:::obj213:head, version: 111'213, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:401e0147:::obj213:head@111'213, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 119 2026-02-11T23:31:28.043+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:401e0147:::obj213:head, version: 111'213, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:401e0147:::obj213:head@111'213, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.043+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.043+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 354 ==== pg_backfill(progress 4.0 e 119/119 lb 4:3fe0a5f5:::obj46:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d2f700 con 0x507ac00 2026-02-11T23:31:28.043+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:3fe0a5f5:::obj46:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.043+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:401e0147:::obj213:head, version: 111'213, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:401e0147:::obj213:head@111'213, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.043+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:3fe0a5f5:::obj46:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.043+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.043+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:401e0147:::obj213:head, version: 111'213, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:401e0147:::obj213:head@111'213, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.043+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:401e0147:::obj213:head, version: 111'213, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:401e0147:::obj213:head@111'213, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 119/117 [PushOp(4:401e0147:::obj213:head, version: 111'213, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:401e0147:::obj213:head@111'213, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:3fc284d5:::obj140:head local-lis/les=117/118 n=156 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.043+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.043+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.043+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:3fe0a5f5:::obj46:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.043+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3fc284d5:::obj140:head local-lis/les=117/118 n=156 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:401e0147:::obj213:head, version: 111'213, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:401e0147:::obj213:head@111'213, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.043+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3fc284d5:::obj140:head local-lis/les=117/118 n=156 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:401e0147:::obj213:head@111'213, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.043+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3fc284d5:::obj140:head local-lis/les=117/118 n=156 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:401e0147:::obj213:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.043+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3fc284d5:::obj140:head local-lis/les=117/118 n=156 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:401e0147:::obj213:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.043+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3fc284d5:::obj140:head local-lis/les=117/118 n=156 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:401e0147:::obj213:head 2026-02-11T23:31:28.043+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3fc284d5:::obj140:head local-lis/les=117/118 n=156 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:401e0147:::obj213:head v 111'213 2026-02-11T23:31:28.043+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3fc284d5:::obj140:head local-lis/les=117/118 n=156 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.043+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.043+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:401e0147:::obj213:head, version: 111'213, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:401e0147:::obj213:head@111'213, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:28.043+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.043+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.043+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:3fe0a5f5:::obj46:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.043+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:3fe0a5f5:::obj46:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 4.0 e 119/119 lb 4:3fe0a5f5:::obj46:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:3fc284d5:::obj140:head local-lis/les=117/118 n=156 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.043+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.043+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3fc284d5:::obj140:head local-lis/les=117/118 n=156 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:3fe0a5f5:::obj46:head) v3 2026-02-11T23:31:28.043+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3fc284d5:::obj140:head local-lis/les=117/118 n=156 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:3fe0a5f5:::obj46:head) 2026-02-11T23:31:28.043+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.043+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:3fe0a5f5:::obj46:head) v3 finish latency 0.000016 2026-02-11T23:31:28.043+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3fe0a5f5:::obj46:head local-lis/les=117/118 n=157 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.043+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.043+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.043+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3fe0a5f5:::obj46:head local-lis/les=117/118 n=157 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.043+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.043+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.043+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.043+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x649ea80 con 0x507ac00 2026-02-11T23:31:28.043+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:401e0147:::obj213:head)]) -- 0x4f43b00 con 0x507ac00 2026-02-11T23:31:28.043+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.043+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 355 ==== MOSDPGPush(4.0 119/117 [PushOp(4:401e941b:::obj398:head, version: 111'398, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:401e941b:::obj398:head@111'398, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56cad00 con 0x507ac00 2026-02-11T23:31:28.043+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:401e941b:::obj398:head, version: 111'398, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:401e941b:::obj398:head@111'398, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.043+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:401e941b:::obj398:head, version: 111'398, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:401e941b:::obj398:head@111'398, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.043+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.043+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 356 ==== pg_backfill(progress 4.0 e 119/119 lb 4:401e0147:::obj213:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f11600 con 0x507ac00 2026-02-11T23:31:28.043+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:401e941b:::obj398:head, version: 111'398, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:401e941b:::obj398:head@111'398, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.043+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:401e0147:::obj213:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.043+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:401e0147:::obj213:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.043+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.043+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:401e941b:::obj398:head, version: 111'398, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:401e941b:::obj398:head@111'398, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.043+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:401e941b:::obj398:head, version: 111'398, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:401e941b:::obj398:head@111'398, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 119/117 [PushOp(4:401e941b:::obj398:head, version: 111'398, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:401e941b:::obj398:head@111'398, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:3fe0a5f5:::obj46:head local-lis/les=117/118 n=157 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.043+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.043+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.043+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:401e0147:::obj213:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.043+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3fe0a5f5:::obj46:head local-lis/les=117/118 n=157 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:401e941b:::obj398:head, version: 111'398, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:401e941b:::obj398:head@111'398, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.043+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3fe0a5f5:::obj46:head local-lis/les=117/118 n=157 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:401e941b:::obj398:head@111'398, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.043+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3fe0a5f5:::obj46:head local-lis/les=117/118 n=157 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:401e941b:::obj398:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.043+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3fe0a5f5:::obj46:head local-lis/les=117/118 n=157 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:401e941b:::obj398:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.043+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3fe0a5f5:::obj46:head local-lis/les=117/118 n=157 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:401e941b:::obj398:head 2026-02-11T23:31:28.043+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3fe0a5f5:::obj46:head local-lis/les=117/118 n=157 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:401e941b:::obj398:head v 111'398 2026-02-11T23:31:28.043+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3fe0a5f5:::obj46:head local-lis/les=117/118 n=157 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.043+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.043+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:401e941b:::obj398:head, version: 111'398, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:401e941b:::obj398:head@111'398, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:28.043+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.043+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.043+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:401e0147:::obj213:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.043+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:401e0147:::obj213:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 4.0 e 119/119 lb 4:401e0147:::obj213:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:3fe0a5f5:::obj46:head local-lis/les=117/118 n=157 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.043+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.043+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3fe0a5f5:::obj46:head local-lis/les=117/118 n=157 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:401e0147:::obj213:head) v3 2026-02-11T23:31:28.043+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:3fe0a5f5:::obj46:head local-lis/les=117/118 n=157 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:401e0147:::obj213:head) 2026-02-11T23:31:28.044+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.044+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:401e0147:::obj213:head) v3 finish latency 0.000018 2026-02-11T23:31:28.044+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:401e0147:::obj213:head local-lis/les=117/118 n=158 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.044+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.044+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.044+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:401e0147:::obj213:head local-lis/les=117/118 n=158 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.044+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.044+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.044+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.044+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c4780 con 0x507ac00 2026-02-11T23:31:28.044+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:401e941b:::obj398:head)]) -- 0x4f43e00 con 0x507ac00 2026-02-11T23:31:28.044+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.044+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 357 ==== MOSDPGPush(4.0 119/117 [PushOp(4:40d1abdf:::obj254:head, version: 111'254, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:40d1abdf:::obj254:head@111'254, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56171e0 con 0x507ac00 2026-02-11T23:31:28.044+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:40d1abdf:::obj254:head, version: 111'254, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:40d1abdf:::obj254:head@111'254, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.044+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:40d1abdf:::obj254:head, version: 111'254, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:40d1abdf:::obj254:head@111'254, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.044+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.044+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:40d1abdf:::obj254:head, version: 111'254, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:40d1abdf:::obj254:head@111'254, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.044+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.044+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 358 ==== pg_backfill(progress 4.0 e 119/119 lb 4:401e941b:::obj398:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f11080 con 0x507ac00 2026-02-11T23:31:28.044+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:40d1abdf:::obj254:head, version: 111'254, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:40d1abdf:::obj254:head@111'254, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.044+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:401e941b:::obj398:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.044+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:401e941b:::obj398:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.044+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.044+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:40d1abdf:::obj254:head, version: 111'254, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:40d1abdf:::obj254:head@111'254, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(4.0 119/117 [PushOp(4:40d1abdf:::obj254:head, version: 111'254, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:40d1abdf:::obj254:head@111'254, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:401e0147:::obj213:head local-lis/les=117/118 n=158 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.044+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.044+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:401e941b:::obj398:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.044+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:401e0147:::obj213:head local-lis/les=117/118 n=158 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:40d1abdf:::obj254:head, version: 111'254, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:40d1abdf:::obj254:head@111'254, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.044+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:401e0147:::obj213:head local-lis/les=117/118 n=158 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:40d1abdf:::obj254:head@111'254, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.044+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:401e0147:::obj213:head local-lis/les=117/118 n=158 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:40d1abdf:::obj254:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.044+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:401e0147:::obj213:head local-lis/les=117/118 n=158 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:40d1abdf:::obj254:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.044+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:401e0147:::obj213:head local-lis/les=117/118 n=158 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:40d1abdf:::obj254:head 2026-02-11T23:31:28.044+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:401e0147:::obj213:head local-lis/les=117/118 n=158 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:40d1abdf:::obj254:head v 111'254 2026-02-11T23:31:28.044+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:401e0147:::obj213:head local-lis/les=117/118 n=158 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.044+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.044+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:40d1abdf:::obj254:head, version: 111'254, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:40d1abdf:::obj254:head@111'254, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000065 2026-02-11T23:31:28.044+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:401e0147:::obj213:head local-lis/les=117/118 n=158 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.044+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.044+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:401e941b:::obj398:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.044+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:401e941b:::obj398:head) v3 prio 15 cost 0 latency 0.000081 pg_backfill(progress 4.0 e 119/119 lb 4:401e941b:::obj398:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:401e0147:::obj213:head local-lis/les=117/118 n=158 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.044+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.044+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.044+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:401e0147:::obj213:head local-lis/les=117/118 n=158 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:401e941b:::obj398:head) v3 2026-02-11T23:31:28.044+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:401e0147:::obj213:head local-lis/les=117/118 n=158 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:401e941b:::obj398:head) 2026-02-11T23:31:28.044+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.044+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:401e941b:::obj398:head) v3 finish latency 0.000021 2026-02-11T23:31:28.044+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.044+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.044+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:401e941b:::obj398:head local-lis/les=117/118 n=159 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.044+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.044+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.044+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.044+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c8180 con 0x507ac00 2026-02-11T23:31:28.044+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:40d1abdf:::obj254:head)]) -- 0x63c8000 con 0x507ac00 2026-02-11T23:31:28.044+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.045+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 359 ==== MOSDPGPush(4.0 119/117 [PushOp(4:41892471:::obj524:head, version: 111'524, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:41892471:::obj524:head@111'524, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a3860 con 0x507ac00 2026-02-11T23:31:28.045+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:41892471:::obj524:head, version: 111'524, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:41892471:::obj524:head@111'524, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.045+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:41892471:::obj524:head, version: 111'524, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:41892471:::obj524:head@111'524, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.045+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.045+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 360 ==== pg_backfill(progress 4.0 e 119/119 lb 4:40d1abdf:::obj254:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f10b00 con 0x507ac00 2026-02-11T23:31:28.045+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:40d1abdf:::obj254:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.045+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:41892471:::obj524:head, version: 111'524, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:41892471:::obj524:head@111'524, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.045+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:40d1abdf:::obj254:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.045+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.045+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:41892471:::obj524:head, version: 111'524, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:41892471:::obj524:head@111'524, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.045+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.045+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:40d1abdf:::obj254:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.045+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:41892471:::obj524:head, version: 111'524, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:41892471:::obj524:head@111'524, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(4.0 119/117 [PushOp(4:41892471:::obj524:head, version: 111'524, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:41892471:::obj524:head@111'524, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:401e941b:::obj398:head local-lis/les=117/118 n=159 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.045+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.045+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:401e941b:::obj398:head local-lis/les=117/118 n=159 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:41892471:::obj524:head, version: 111'524, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:41892471:::obj524:head@111'524, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.045+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:401e941b:::obj398:head local-lis/les=117/118 n=159 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:41892471:::obj524:head@111'524, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.045+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:401e941b:::obj398:head local-lis/les=117/118 n=159 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:41892471:::obj524:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.045+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:401e941b:::obj398:head local-lis/les=117/118 n=159 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:41892471:::obj524:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.045+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:401e941b:::obj398:head local-lis/les=117/118 n=159 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:41892471:::obj524:head 2026-02-11T23:31:28.045+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:401e941b:::obj398:head local-lis/les=117/118 n=159 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:41892471:::obj524:head v 111'524 2026-02-11T23:31:28.045+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:401e941b:::obj398:head local-lis/les=117/118 n=159 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.045+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.045+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:41892471:::obj524:head, version: 111'524, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:41892471:::obj524:head@111'524, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000067 2026-02-11T23:31:28.045+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.045+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.045+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:40d1abdf:::obj254:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.045+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:40d1abdf:::obj254:head) v3 prio 15 cost 0 latency 0.000089 pg_backfill(progress 4.0 e 119/119 lb 4:40d1abdf:::obj254:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:401e941b:::obj398:head local-lis/les=117/118 n=159 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.045+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.045+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:401e941b:::obj398:head local-lis/les=117/118 n=159 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:40d1abdf:::obj254:head) v3 2026-02-11T23:31:28.045+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:401e941b:::obj398:head local-lis/les=117/118 n=159 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:40d1abdf:::obj254:head) 2026-02-11T23:31:28.045+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.045+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:40d1abdf:::obj254:head) v3 finish latency 0.000016 2026-02-11T23:31:28.045+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:40d1abdf:::obj254:head local-lis/les=117/118 n=160 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.045+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.045+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.045+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:40d1abdf:::obj254:head local-lis/les=117/118 n=160 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.045+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.045+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.045+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.045+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c4480 con 0x507ac00 2026-02-11T23:31:28.045+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:41892471:::obj524:head)]) -- 0x5f2dc80 con 0x507ac00 2026-02-11T23:31:28.045+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.045+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 361 ==== MOSDPGPush(4.0 119/117 [PushOp(4:41ea52ba:::obj210:head, version: 111'210, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:41ea52ba:::obj210:head@111'210, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a3040 con 0x507ac00 2026-02-11T23:31:28.045+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:41ea52ba:::obj210:head, version: 111'210, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:41ea52ba:::obj210:head@111'210, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.045+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:41ea52ba:::obj210:head, version: 111'210, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:41ea52ba:::obj210:head@111'210, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.045+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.045+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 362 ==== pg_backfill(progress 4.0 e 119/119 lb 4:41892471:::obj524:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f10580 con 0x507ac00 2026-02-11T23:31:28.045+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:41892471:::obj524:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.045+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:41ea52ba:::obj210:head, version: 111'210, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:41ea52ba:::obj210:head@111'210, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.045+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:41892471:::obj524:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.045+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.045+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:41ea52ba:::obj210:head, version: 111'210, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:41ea52ba:::obj210:head@111'210, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.045+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.045+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:41ea52ba:::obj210:head, version: 111'210, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:41ea52ba:::obj210:head@111'210, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 119/117 [PushOp(4:41ea52ba:::obj210:head, version: 111'210, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:41ea52ba:::obj210:head@111'210, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:40d1abdf:::obj254:head local-lis/les=117/118 n=160 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.045+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.045+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:41892471:::obj524:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.045+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:40d1abdf:::obj254:head local-lis/les=117/118 n=160 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:41ea52ba:::obj210:head, version: 111'210, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:41ea52ba:::obj210:head@111'210, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.045+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:40d1abdf:::obj254:head local-lis/les=117/118 n=160 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:41ea52ba:::obj210:head@111'210, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.045+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:40d1abdf:::obj254:head local-lis/les=117/118 n=160 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:41ea52ba:::obj210:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.045+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:40d1abdf:::obj254:head local-lis/les=117/118 n=160 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:41ea52ba:::obj210:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.045+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:40d1abdf:::obj254:head local-lis/les=117/118 n=160 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:41ea52ba:::obj210:head 2026-02-11T23:31:28.045+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:40d1abdf:::obj254:head local-lis/les=117/118 n=160 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:41ea52ba:::obj210:head v 111'210 2026-02-11T23:31:28.045+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:40d1abdf:::obj254:head local-lis/les=117/118 n=160 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.046+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.046+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:41ea52ba:::obj210:head, version: 111'210, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:41ea52ba:::obj210:head@111'210, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000083 2026-02-11T23:31:28.046+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.046+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.046+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:41892471:::obj524:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.046+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:41892471:::obj524:head) v3 prio 15 cost 0 latency 0.000104 pg_backfill(progress 4.0 e 119/119 lb 4:41892471:::obj524:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:40d1abdf:::obj254:head local-lis/les=117/118 n=160 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.046+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.046+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:40d1abdf:::obj254:head local-lis/les=117/118 n=160 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:41892471:::obj524:head) v3 2026-02-11T23:31:28.046+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:40d1abdf:::obj254:head local-lis/les=117/118 n=160 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:41892471:::obj524:head) 2026-02-11T23:31:28.046+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.046+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:41892471:::obj524:head) v3 finish latency 0.000016 2026-02-11T23:31:28.046+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:41892471:::obj524:head local-lis/les=117/118 n=161 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.046+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.046+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.046+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:41892471:::obj524:head local-lis/les=117/118 n=161 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.046+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.046+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.046+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.046+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c8300 con 0x507ac00 2026-02-11T23:31:28.046+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:41ea52ba:::obj210:head)]) -- 0x63c5080 con 0x507ac00 2026-02-11T23:31:28.046+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.046+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 363 ==== MOSDPGPush(4.0 119/117 [PushOp(4:424ce90f:::obj439:head, version: 111'439, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:424ce90f:::obj439:head@111'439, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x554e4e0 con 0x507ac00 2026-02-11T23:31:28.046+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:424ce90f:::obj439:head, version: 111'439, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:424ce90f:::obj439:head@111'439, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 119 2026-02-11T23:31:28.046+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:424ce90f:::obj439:head, version: 111'439, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:424ce90f:::obj439:head@111'439, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.046+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.046+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 364 ==== pg_backfill(progress 4.0 e 119/119 lb 4:41ea52ba:::obj210:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f13180 con 0x507ac00 2026-02-11T23:31:28.046+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:41ea52ba:::obj210:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.046+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:424ce90f:::obj439:head, version: 111'439, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:424ce90f:::obj439:head@111'439, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.046+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:41ea52ba:::obj210:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.046+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.046+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:424ce90f:::obj439:head, version: 111'439, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:424ce90f:::obj439:head@111'439, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.046+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:424ce90f:::obj439:head, version: 111'439, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:424ce90f:::obj439:head@111'439, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(4.0 119/117 [PushOp(4:424ce90f:::obj439:head, version: 111'439, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:424ce90f:::obj439:head@111'439, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:41892471:::obj524:head local-lis/les=117/118 n=161 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.046+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.046+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.046+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:41ea52ba:::obj210:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.046+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:41892471:::obj524:head local-lis/les=117/118 n=161 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:424ce90f:::obj439:head, version: 111'439, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:424ce90f:::obj439:head@111'439, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.046+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:41892471:::obj524:head local-lis/les=117/118 n=161 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:424ce90f:::obj439:head@111'439, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.046+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:41892471:::obj524:head local-lis/les=117/118 n=161 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:424ce90f:::obj439:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.046+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:41892471:::obj524:head local-lis/les=117/118 n=161 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:424ce90f:::obj439:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.046+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:41892471:::obj524:head local-lis/les=117/118 n=161 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:424ce90f:::obj439:head 2026-02-11T23:31:28.046+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:41892471:::obj524:head local-lis/les=117/118 n=161 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:424ce90f:::obj439:head v 111'439 2026-02-11T23:31:28.046+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:41892471:::obj524:head local-lis/les=117/118 n=161 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.046+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.046+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:424ce90f:::obj439:head, version: 111'439, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:424ce90f:::obj439:head@111'439, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:28.046+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.046+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.046+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:41ea52ba:::obj210:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.046+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:41ea52ba:::obj210:head) v3 prio 15 cost 0 latency 0.000085 pg_backfill(progress 4.0 e 119/119 lb 4:41ea52ba:::obj210:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:41892471:::obj524:head local-lis/les=117/118 n=161 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.046+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.046+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:41892471:::obj524:head local-lis/les=117/118 n=161 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:41ea52ba:::obj210:head) v3 2026-02-11T23:31:28.046+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:41892471:::obj524:head local-lis/les=117/118 n=161 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:41ea52ba:::obj210:head) 2026-02-11T23:31:28.046+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.046+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:41ea52ba:::obj210:head) v3 finish latency 0.000017 2026-02-11T23:31:28.046+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:41ea52ba:::obj210:head local-lis/les=117/118 n=162 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.046+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.046+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.046+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:41ea52ba:::obj210:head local-lis/les=117/118 n=162 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.046+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.046+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.046+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.046+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c4300 con 0x507ac00 2026-02-11T23:31:28.046+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:424ce90f:::obj439:head)]) -- 0x649f200 con 0x507ac00 2026-02-11T23:31:28.046+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.047+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 365 ==== MOSDPGPush(4.0 119/117 [PushOp(4:42d0e5ef:::obj468:head, version: 111'468, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:42d0e5ef:::obj468:head@111'468, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f1d40 con 0x507ac00 2026-02-11T23:31:28.047+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:42d0e5ef:::obj468:head, version: 111'468, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:42d0e5ef:::obj468:head@111'468, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.047+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:42d0e5ef:::obj468:head, version: 111'468, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:42d0e5ef:::obj468:head@111'468, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.047+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.047+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 366 ==== pg_backfill(progress 4.0 e 119/119 lb 4:424ce90f:::obj439:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f11b80 con 0x507ac00 2026-02-11T23:31:28.047+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:424ce90f:::obj439:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.047+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:42d0e5ef:::obj468:head, version: 111'468, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:42d0e5ef:::obj468:head@111'468, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.047+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:424ce90f:::obj439:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.047+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.047+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:42d0e5ef:::obj468:head, version: 111'468, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:42d0e5ef:::obj468:head@111'468, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.047+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.047+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:42d0e5ef:::obj468:head, version: 111'468, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:42d0e5ef:::obj468:head@111'468, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 119/117 [PushOp(4:42d0e5ef:::obj468:head, version: 111'468, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:42d0e5ef:::obj468:head@111'468, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:41ea52ba:::obj210:head local-lis/les=117/118 n=162 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.047+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:424ce90f:::obj439:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.047+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.047+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:41ea52ba:::obj210:head local-lis/les=117/118 n=162 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:42d0e5ef:::obj468:head, version: 111'468, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:42d0e5ef:::obj468:head@111'468, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.047+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:41ea52ba:::obj210:head local-lis/les=117/118 n=162 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:42d0e5ef:::obj468:head@111'468, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.047+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:41ea52ba:::obj210:head local-lis/les=117/118 n=162 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:42d0e5ef:::obj468:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.047+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:41ea52ba:::obj210:head local-lis/les=117/118 n=162 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:42d0e5ef:::obj468:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.047+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:41ea52ba:::obj210:head local-lis/les=117/118 n=162 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:42d0e5ef:::obj468:head 2026-02-11T23:31:28.047+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:41ea52ba:::obj210:head local-lis/les=117/118 n=162 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:42d0e5ef:::obj468:head v 111'468 2026-02-11T23:31:28.047+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:41ea52ba:::obj210:head local-lis/les=117/118 n=162 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.047+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.047+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:42d0e5ef:::obj468:head, version: 111'468, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:42d0e5ef:::obj468:head@111'468, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:28.047+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.047+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.047+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:424ce90f:::obj439:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.047+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:424ce90f:::obj439:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 4.0 e 119/119 lb 4:424ce90f:::obj439:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:41ea52ba:::obj210:head local-lis/les=117/118 n=162 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.047+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.047+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:41ea52ba:::obj210:head local-lis/les=117/118 n=162 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:424ce90f:::obj439:head) v3 2026-02-11T23:31:28.047+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:41ea52ba:::obj210:head local-lis/les=117/118 n=162 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:424ce90f:::obj439:head) 2026-02-11T23:31:28.047+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.047+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:424ce90f:::obj439:head) v3 finish latency 0.000015 2026-02-11T23:31:28.047+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:424ce90f:::obj439:head local-lis/les=117/118 n=163 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.047+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.047+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.047+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:424ce90f:::obj439:head local-lis/les=117/118 n=163 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.047+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.047+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.047+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.047+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7a000 con 0x507ac00 2026-02-11T23:31:28.047+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:42d0e5ef:::obj468:head)]) -- 0x63c5200 con 0x507ac00 2026-02-11T23:31:28.047+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.047+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 367 ==== MOSDPGPush(4.0 119/117 [PushOp(4:436594ec:::obj300:head, version: 111'300, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:436594ec:::obj300:head@111'300, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2000 con 0x507ac00 2026-02-11T23:31:28.047+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:436594ec:::obj300:head, version: 111'300, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:436594ec:::obj300:head@111'300, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.048+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:436594ec:::obj300:head, version: 111'300, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:436594ec:::obj300:head@111'300, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.048+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.048+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 368 ==== pg_backfill(progress 4.0 e 119/119 lb 4:42d0e5ef:::obj468:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f12100 con 0x507ac00 2026-02-11T23:31:28.048+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:436594ec:::obj300:head, version: 111'300, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:436594ec:::obj300:head@111'300, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.048+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:42d0e5ef:::obj468:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.048+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:42d0e5ef:::obj468:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.048+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.048+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:436594ec:::obj300:head, version: 111'300, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:436594ec:::obj300:head@111'300, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.048+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:436594ec:::obj300:head, version: 111'300, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:436594ec:::obj300:head@111'300, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(4.0 119/117 [PushOp(4:436594ec:::obj300:head, version: 111'300, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:436594ec:::obj300:head@111'300, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:424ce90f:::obj439:head local-lis/les=117/118 n=163 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.048+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.048+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.048+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:42d0e5ef:::obj468:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.048+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:424ce90f:::obj439:head local-lis/les=117/118 n=163 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:436594ec:::obj300:head, version: 111'300, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:436594ec:::obj300:head@111'300, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.048+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:424ce90f:::obj439:head local-lis/les=117/118 n=163 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:436594ec:::obj300:head@111'300, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.048+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:424ce90f:::obj439:head local-lis/les=117/118 n=163 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:436594ec:::obj300:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.048+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:424ce90f:::obj439:head local-lis/les=117/118 n=163 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:436594ec:::obj300:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.048+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:424ce90f:::obj439:head local-lis/les=117/118 n=163 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:436594ec:::obj300:head 2026-02-11T23:31:28.048+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:424ce90f:::obj439:head local-lis/les=117/118 n=163 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:436594ec:::obj300:head v 111'300 2026-02-11T23:31:28.048+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:424ce90f:::obj439:head local-lis/les=117/118 n=163 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.048+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.048+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:436594ec:::obj300:head, version: 111'300, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:436594ec:::obj300:head@111'300, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:28.048+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.048+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.048+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:42d0e5ef:::obj468:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.048+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:42d0e5ef:::obj468:head) v3 prio 15 cost 0 latency 0.000079 pg_backfill(progress 4.0 e 119/119 lb 4:42d0e5ef:::obj468:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:424ce90f:::obj439:head local-lis/les=117/118 n=163 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.048+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.048+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:424ce90f:::obj439:head local-lis/les=117/118 n=163 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:42d0e5ef:::obj468:head) v3 2026-02-11T23:31:28.048+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:424ce90f:::obj439:head local-lis/les=117/118 n=163 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:42d0e5ef:::obj468:head) 2026-02-11T23:31:28.048+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.048+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:42d0e5ef:::obj468:head) v3 finish latency 0.000015 2026-02-11T23:31:28.048+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:42d0e5ef:::obj468:head local-lis/les=117/118 n=164 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.048+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.048+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.048+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:42d0e5ef:::obj468:head local-lis/les=117/118 n=164 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.048+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.048+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.048+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.048+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7a180 con 0x507ac00 2026-02-11T23:31:28.048+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:436594ec:::obj300:head)]) -- 0x63c8c00 con 0x507ac00 2026-02-11T23:31:28.048+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.048+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 369 ==== MOSDPGPush(4.0 119/117 [PushOp(4:454d8265:::obj256:head, version: 111'256, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:454d8265:::obj256:head@111'256, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56aa000 con 0x507ac00 2026-02-11T23:31:28.048+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:454d8265:::obj256:head, version: 111'256, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:454d8265:::obj256:head@111'256, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.048+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:454d8265:::obj256:head, version: 111'256, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:454d8265:::obj256:head@111'256, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.048+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.048+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 370 ==== pg_backfill(progress 4.0 e 119/119 lb 4:436594ec:::obj300:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f12680 con 0x507ac00 2026-02-11T23:31:28.048+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:454d8265:::obj256:head, version: 111'256, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:454d8265:::obj256:head@111'256, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.048+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:436594ec:::obj300:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.048+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:436594ec:::obj300:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.048+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.048+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:454d8265:::obj256:head, version: 111'256, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:454d8265:::obj256:head@111'256, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.048+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:454d8265:::obj256:head, version: 111'256, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:454d8265:::obj256:head@111'256, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 119/117 [PushOp(4:454d8265:::obj256:head, version: 111'256, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:454d8265:::obj256:head@111'256, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:42d0e5ef:::obj468:head local-lis/les=117/118 n=164 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.048+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.048+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.048+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:436594ec:::obj300:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.048+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:42d0e5ef:::obj468:head local-lis/les=117/118 n=164 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:454d8265:::obj256:head, version: 111'256, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:454d8265:::obj256:head@111'256, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.048+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:42d0e5ef:::obj468:head local-lis/les=117/118 n=164 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:454d8265:::obj256:head@111'256, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.048+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:42d0e5ef:::obj468:head local-lis/les=117/118 n=164 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:454d8265:::obj256:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.048+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:42d0e5ef:::obj468:head local-lis/les=117/118 n=164 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:454d8265:::obj256:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.048+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:42d0e5ef:::obj468:head local-lis/les=117/118 n=164 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:454d8265:::obj256:head 2026-02-11T23:31:28.048+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:42d0e5ef:::obj468:head local-lis/les=117/118 n=164 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:454d8265:::obj256:head v 111'256 2026-02-11T23:31:28.048+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:42d0e5ef:::obj468:head local-lis/les=117/118 n=164 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.048+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.048+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:454d8265:::obj256:head, version: 111'256, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:454d8265:::obj256:head@111'256, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:28.048+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.048+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.048+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:436594ec:::obj300:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.048+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:436594ec:::obj300:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 4.0 e 119/119 lb 4:436594ec:::obj300:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:42d0e5ef:::obj468:head local-lis/les=117/118 n=164 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.048+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.048+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:42d0e5ef:::obj468:head local-lis/les=117/118 n=164 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:436594ec:::obj300:head) v3 2026-02-11T23:31:28.048+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:42d0e5ef:::obj468:head local-lis/les=117/118 n=164 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:436594ec:::obj300:head) 2026-02-11T23:31:28.048+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.048+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:436594ec:::obj300:head) v3 finish latency 0.000015 2026-02-11T23:31:28.048+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:436594ec:::obj300:head local-lis/les=117/118 n=165 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.048+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.048+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.048+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:436594ec:::obj300:head local-lis/les=117/118 n=165 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.048+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.048+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.048+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.048+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7a300 con 0x507ac00 2026-02-11T23:31:28.048+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:454d8265:::obj256:head)]) -- 0x63c5500 con 0x507ac00 2026-02-11T23:31:28.048+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.049+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 371 ==== MOSDPGPush(4.0 119/117 [PushOp(4:461f8b5e:::obj16:head, version: 111'16, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:461f8b5e:::obj16:head@111'16, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x56a2b60 con 0x507ac00 2026-02-11T23:31:28.049+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:461f8b5e:::obj16:head, version: 111'16, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:461f8b5e:::obj16:head@111'16, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 119 2026-02-11T23:31:28.049+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:461f8b5e:::obj16:head, version: 111'16, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:461f8b5e:::obj16:head@111'16, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.049+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.049+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 372 ==== pg_backfill(progress 4.0 e 119/119 lb 4:454d8265:::obj256:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f12c00 con 0x507ac00 2026-02-11T23:31:28.049+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:461f8b5e:::obj16:head, version: 111'16, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:461f8b5e:::obj16:head@111'16, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.049+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:454d8265:::obj256:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.049+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.049+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:461f8b5e:::obj16:head, version: 111'16, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:461f8b5e:::obj16:head@111'16, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.049+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:454d8265:::obj256:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.049+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.049+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:461f8b5e:::obj16:head, version: 111'16, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:461f8b5e:::obj16:head@111'16, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(4.0 119/117 [PushOp(4:461f8b5e:::obj16:head, version: 111'16, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:461f8b5e:::obj16:head@111'16, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:436594ec:::obj300:head local-lis/les=117/118 n=165 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.049+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:454d8265:::obj256:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.049+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.049+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:436594ec:::obj300:head local-lis/les=117/118 n=165 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:461f8b5e:::obj16:head, version: 111'16, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:461f8b5e:::obj16:head@111'16, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.049+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:436594ec:::obj300:head local-lis/les=117/118 n=165 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:461f8b5e:::obj16:head@111'16, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.049+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:436594ec:::obj300:head local-lis/les=117/118 n=165 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:461f8b5e:::obj16:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.049+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:436594ec:::obj300:head local-lis/les=117/118 n=165 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:461f8b5e:::obj16:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.049+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:436594ec:::obj300:head local-lis/les=117/118 n=165 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:461f8b5e:::obj16:head 2026-02-11T23:31:28.049+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:436594ec:::obj300:head local-lis/les=117/118 n=165 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:461f8b5e:::obj16:head v 111'16 2026-02-11T23:31:28.049+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:436594ec:::obj300:head local-lis/les=117/118 n=165 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.049+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.049+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:461f8b5e:::obj16:head, version: 111'16, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:461f8b5e:::obj16:head@111'16, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:28.049+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.049+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.049+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:454d8265:::obj256:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.049+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:454d8265:::obj256:head) v3 prio 15 cost 0 latency 0.000081 pg_backfill(progress 4.0 e 119/119 lb 4:454d8265:::obj256:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:436594ec:::obj300:head local-lis/les=117/118 n=165 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.049+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.049+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:436594ec:::obj300:head local-lis/les=117/118 n=165 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:454d8265:::obj256:head) v3 2026-02-11T23:31:28.049+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:436594ec:::obj300:head local-lis/les=117/118 n=165 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:454d8265:::obj256:head) 2026-02-11T23:31:28.049+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.049+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:454d8265:::obj256:head) v3 finish latency 0.000016 2026-02-11T23:31:28.049+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:454d8265:::obj256:head local-lis/les=117/118 n=166 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.049+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.049+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.049+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:454d8265:::obj256:head local-lis/les=117/118 n=166 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.049+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.049+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.049+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.049+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7a480 con 0x507ac00 2026-02-11T23:31:28.049+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:461f8b5e:::obj16:head)]) -- 0x63c4180 con 0x507ac00 2026-02-11T23:31:28.049+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.049+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 373 ==== MOSDPGPush(4.0 119/117 [PushOp(4:4667f6bf:::obj584:head, version: 111'584, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4667f6bf:::obj584:head@111'584, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x554e680 con 0x507ac00 2026-02-11T23:31:28.050+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:4667f6bf:::obj584:head, version: 111'584, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4667f6bf:::obj584:head@111'584, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 119 2026-02-11T23:31:28.050+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4667f6bf:::obj584:head, version: 111'584, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4667f6bf:::obj584:head@111'584, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.050+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.050+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 374 ==== pg_backfill(progress 4.0 e 119/119 lb 4:461f8b5e:::obj16:head) ==== 1061+0+0 (unknown 0 0 0) 0x5f13700 con 0x507ac00 2026-02-11T23:31:28.050+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4667f6bf:::obj584:head, version: 111'584, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4667f6bf:::obj584:head@111'584, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.050+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:461f8b5e:::obj16:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.050+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.050+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4667f6bf:::obj584:head, version: 111'584, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4667f6bf:::obj584:head@111'584, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.050+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:461f8b5e:::obj16:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.050+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.050+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:461f8b5e:::obj16:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.050+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:4667f6bf:::obj584:head, version: 111'584, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4667f6bf:::obj584:head@111'584, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(4.0 119/117 [PushOp(4:4667f6bf:::obj584:head, version: 111'584, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4667f6bf:::obj584:head@111'584, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:454d8265:::obj256:head local-lis/les=117/118 n=166 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.050+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.050+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:454d8265:::obj256:head local-lis/les=117/118 n=166 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:4667f6bf:::obj584:head, version: 111'584, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4667f6bf:::obj584:head@111'584, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.050+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:454d8265:::obj256:head local-lis/les=117/118 n=166 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:4667f6bf:::obj584:head@111'584, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.050+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:454d8265:::obj256:head local-lis/les=117/118 n=166 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:4667f6bf:::obj584:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.050+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:454d8265:::obj256:head local-lis/les=117/118 n=166 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:4667f6bf:::obj584:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.050+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:454d8265:::obj256:head local-lis/les=117/118 n=166 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:4667f6bf:::obj584:head 2026-02-11T23:31:28.050+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:454d8265:::obj256:head local-lis/les=117/118 n=166 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:4667f6bf:::obj584:head v 111'584 2026-02-11T23:31:28.050+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:454d8265:::obj256:head local-lis/les=117/118 n=166 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.050+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.050+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:4667f6bf:::obj584:head, version: 111'584, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4667f6bf:::obj584:head@111'584, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:28.050+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.050+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.050+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:461f8b5e:::obj16:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.050+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:461f8b5e:::obj16:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 4.0 e 119/119 lb 4:461f8b5e:::obj16:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:454d8265:::obj256:head local-lis/les=117/118 n=166 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.050+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.050+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:454d8265:::obj256:head local-lis/les=117/118 n=166 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:461f8b5e:::obj16:head) v3 2026-02-11T23:31:28.050+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:454d8265:::obj256:head local-lis/les=117/118 n=166 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:461f8b5e:::obj16:head) 2026-02-11T23:31:28.050+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.050+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:461f8b5e:::obj16:head) v3 finish latency 0.000015 2026-02-11T23:31:28.050+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:461f8b5e:::obj16:head local-lis/les=117/118 n=167 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.050+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.050+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.050+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:461f8b5e:::obj16:head local-lis/les=117/118 n=167 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.050+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.050+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.050+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.050+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7a600 con 0x507ac00 2026-02-11T23:31:28.050+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:4667f6bf:::obj584:head)]) -- 0x63c9080 con 0x507ac00 2026-02-11T23:31:28.050+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.050+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 375 ==== MOSDPGPush(4.0 119/117 [PushOp(4:46b5a6d0:::obj580:head, version: 111'580, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:46b5a6d0:::obj580:head@111'580, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a3a00 con 0x507ac00 2026-02-11T23:31:28.050+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:46b5a6d0:::obj580:head, version: 111'580, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:46b5a6d0:::obj580:head@111'580, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.050+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:46b5a6d0:::obj580:head, version: 111'580, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:46b5a6d0:::obj580:head@111'580, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.050+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.050+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 376 ==== pg_backfill(progress 4.0 e 119/119 lb 4:4667f6bf:::obj584:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f10000 con 0x507ac00 2026-02-11T23:31:28.050+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:46b5a6d0:::obj580:head, version: 111'580, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:46b5a6d0:::obj580:head@111'580, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.050+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:4667f6bf:::obj584:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.050+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4667f6bf:::obj584:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.050+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.050+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:46b5a6d0:::obj580:head, version: 111'580, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:46b5a6d0:::obj580:head@111'580, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.050+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.050+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:46b5a6d0:::obj580:head, version: 111'580, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:46b5a6d0:::obj580:head@111'580, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 119/117 [PushOp(4:46b5a6d0:::obj580:head, version: 111'580, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:46b5a6d0:::obj580:head@111'580, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:461f8b5e:::obj16:head local-lis/les=117/118 n=167 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.050+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.050+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.050+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4667f6bf:::obj584:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.050+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:461f8b5e:::obj16:head local-lis/les=117/118 n=167 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:46b5a6d0:::obj580:head, version: 111'580, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:46b5a6d0:::obj580:head@111'580, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.050+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:461f8b5e:::obj16:head local-lis/les=117/118 n=167 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:46b5a6d0:::obj580:head@111'580, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.050+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:461f8b5e:::obj16:head local-lis/les=117/118 n=167 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:46b5a6d0:::obj580:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.050+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:461f8b5e:::obj16:head local-lis/les=117/118 n=167 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:46b5a6d0:::obj580:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.050+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:461f8b5e:::obj16:head local-lis/les=117/118 n=167 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:46b5a6d0:::obj580:head 2026-02-11T23:31:28.050+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:461f8b5e:::obj16:head local-lis/les=117/118 n=167 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:46b5a6d0:::obj580:head v 111'580 2026-02-11T23:31:28.050+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:461f8b5e:::obj16:head local-lis/les=117/118 n=167 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.050+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.050+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:46b5a6d0:::obj580:head, version: 111'580, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:46b5a6d0:::obj580:head@111'580, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:28.050+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.050+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.050+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4667f6bf:::obj584:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.050+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:4667f6bf:::obj584:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 4.0 e 119/119 lb 4:4667f6bf:::obj584:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:461f8b5e:::obj16:head local-lis/les=117/118 n=167 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.050+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.050+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:461f8b5e:::obj16:head local-lis/les=117/118 n=167 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:4667f6bf:::obj584:head) v3 2026-02-11T23:31:28.050+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:461f8b5e:::obj16:head local-lis/les=117/118 n=167 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:4667f6bf:::obj584:head) 2026-02-11T23:31:28.050+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.050+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:4667f6bf:::obj584:head) v3 finish latency 0.000018 2026-02-11T23:31:28.050+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4667f6bf:::obj584:head local-lis/les=117/118 n=168 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.050+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.050+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.050+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4667f6bf:::obj584:head local-lis/les=117/118 n=168 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.050+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.050+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.050+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.050+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7a780 con 0x507ac00 2026-02-11T23:31:28.051+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:46b5a6d0:::obj580:head)]) -- 0x63c9200 con 0x507ac00 2026-02-11T23:31:28.051+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.051+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 377 ==== MOSDPGPush(4.0 119/117 [PushOp(4:4851a45a:::obj44:head, version: 111'44, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4851a45a:::obj44:head@111'44, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5617380 con 0x507ac00 2026-02-11T23:31:28.051+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:4851a45a:::obj44:head, version: 111'44, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4851a45a:::obj44:head@111'44, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 119 2026-02-11T23:31:28.051+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4851a45a:::obj44:head, version: 111'44, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4851a45a:::obj44:head@111'44, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.051+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.051+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 378 ==== pg_backfill(progress 4.0 e 119/119 lb 4:46b5a6d0:::obj580:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f10580 con 0x507ac00 2026-02-11T23:31:28.051+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:46b5a6d0:::obj580:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.051+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4851a45a:::obj44:head, version: 111'44, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4851a45a:::obj44:head@111'44, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.051+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:46b5a6d0:::obj580:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.051+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.051+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4851a45a:::obj44:head, version: 111'44, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4851a45a:::obj44:head@111'44, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.051+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.051+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:46b5a6d0:::obj580:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.051+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:4851a45a:::obj44:head, version: 111'44, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4851a45a:::obj44:head@111'44, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 119/117 [PushOp(4:4851a45a:::obj44:head, version: 111'44, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4851a45a:::obj44:head@111'44, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:4667f6bf:::obj584:head local-lis/les=117/118 n=168 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.051+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.051+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4667f6bf:::obj584:head local-lis/les=117/118 n=168 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:4851a45a:::obj44:head, version: 111'44, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4851a45a:::obj44:head@111'44, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.051+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4667f6bf:::obj584:head local-lis/les=117/118 n=168 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:4851a45a:::obj44:head@111'44, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.051+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4667f6bf:::obj584:head local-lis/les=117/118 n=168 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:4851a45a:::obj44:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.051+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4667f6bf:::obj584:head local-lis/les=117/118 n=168 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:4851a45a:::obj44:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.051+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4667f6bf:::obj584:head local-lis/les=117/118 n=168 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:4851a45a:::obj44:head 2026-02-11T23:31:28.051+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4667f6bf:::obj584:head local-lis/les=117/118 n=168 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:4851a45a:::obj44:head v 111'44 2026-02-11T23:31:28.051+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4667f6bf:::obj584:head local-lis/les=117/118 n=168 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.051+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.051+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:4851a45a:::obj44:head, version: 111'44, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4851a45a:::obj44:head@111'44, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:28.051+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.051+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.051+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:46b5a6d0:::obj580:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.051+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:46b5a6d0:::obj580:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 4.0 e 119/119 lb 4:46b5a6d0:::obj580:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:4667f6bf:::obj584:head local-lis/les=117/118 n=168 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.051+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.051+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4667f6bf:::obj584:head local-lis/les=117/118 n=168 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:46b5a6d0:::obj580:head) v3 2026-02-11T23:31:28.051+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4667f6bf:::obj584:head local-lis/les=117/118 n=168 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:46b5a6d0:::obj580:head) 2026-02-11T23:31:28.051+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.051+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:46b5a6d0:::obj580:head) v3 finish latency 0.000015 2026-02-11T23:31:28.051+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:46b5a6d0:::obj580:head local-lis/les=117/118 n=169 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.051+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.051+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.051+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:46b5a6d0:::obj580:head local-lis/les=117/118 n=169 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.051+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.051+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.051+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.051+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7a900 con 0x507ac00 2026-02-11T23:31:28.051+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:4851a45a:::obj44:head)]) -- 0x649e600 con 0x507ac00 2026-02-11T23:31:28.051+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.052+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 379 ==== MOSDPGPush(4.0 119/117 [PushOp(4:48c1458b:::obj274:head, version: 111'274, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:48c1458b:::obj274:head@111'274, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56caea0 con 0x507ac00 2026-02-11T23:31:28.052+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:48c1458b:::obj274:head, version: 111'274, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:48c1458b:::obj274:head@111'274, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.052+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:48c1458b:::obj274:head, version: 111'274, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:48c1458b:::obj274:head@111'274, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.052+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.052+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 380 ==== pg_backfill(progress 4.0 e 119/119 lb 4:4851a45a:::obj44:head) ==== 1061+0+0 (unknown 0 0 0) 0x5f10b00 con 0x507ac00 2026-02-11T23:31:28.052+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:4851a45a:::obj44:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.052+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:48c1458b:::obj274:head, version: 111'274, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:48c1458b:::obj274:head@111'274, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.052+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4851a45a:::obj44:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.052+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.052+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:48c1458b:::obj274:head, version: 111'274, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:48c1458b:::obj274:head@111'274, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.052+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:48c1458b:::obj274:head, version: 111'274, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:48c1458b:::obj274:head@111'274, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 119/117 [PushOp(4:48c1458b:::obj274:head, version: 111'274, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:48c1458b:::obj274:head@111'274, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:46b5a6d0:::obj580:head local-lis/les=117/118 n=169 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.052+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.052+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4851a45a:::obj44:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.052+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.052+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:46b5a6d0:::obj580:head local-lis/les=117/118 n=169 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:48c1458b:::obj274:head, version: 111'274, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:48c1458b:::obj274:head@111'274, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.052+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:46b5a6d0:::obj580:head local-lis/les=117/118 n=169 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:48c1458b:::obj274:head@111'274, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.052+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:46b5a6d0:::obj580:head local-lis/les=117/118 n=169 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:48c1458b:::obj274:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.052+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:46b5a6d0:::obj580:head local-lis/les=117/118 n=169 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:48c1458b:::obj274:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.052+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:46b5a6d0:::obj580:head local-lis/les=117/118 n=169 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:48c1458b:::obj274:head 2026-02-11T23:31:28.052+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:46b5a6d0:::obj580:head local-lis/les=117/118 n=169 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:48c1458b:::obj274:head v 111'274 2026-02-11T23:31:28.052+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:46b5a6d0:::obj580:head local-lis/les=117/118 n=169 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.052+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.052+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:48c1458b:::obj274:head, version: 111'274, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:48c1458b:::obj274:head@111'274, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:28.052+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.052+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.052+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4851a45a:::obj44:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.052+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:4851a45a:::obj44:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 4.0 e 119/119 lb 4:4851a45a:::obj44:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:46b5a6d0:::obj580:head local-lis/les=117/118 n=169 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.052+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.052+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:46b5a6d0:::obj580:head local-lis/les=117/118 n=169 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:4851a45a:::obj44:head) v3 2026-02-11T23:31:28.052+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:46b5a6d0:::obj580:head local-lis/les=117/118 n=169 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:4851a45a:::obj44:head) 2026-02-11T23:31:28.052+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.052+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:4851a45a:::obj44:head) v3 finish latency 0.000017 2026-02-11T23:31:28.052+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4851a45a:::obj44:head local-lis/les=117/118 n=170 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.052+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.052+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.052+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4851a45a:::obj44:head local-lis/les=117/118 n=170 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.052+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.052+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.052+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.052+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7aa80 con 0x507ac00 2026-02-11T23:31:28.052+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:48c1458b:::obj274:head)]) -- 0x63c5800 con 0x507ac00 2026-02-11T23:31:28.052+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.052+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 381 ==== MOSDPGPush(4.0 119/117 [PushOp(4:4911cfa9:::obj108:head, version: 111'108, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4911cfa9:::obj108:head@111'108, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56b64e0 con 0x507ac00 2026-02-11T23:31:28.052+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:4911cfa9:::obj108:head, version: 111'108, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4911cfa9:::obj108:head@111'108, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.052+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4911cfa9:::obj108:head, version: 111'108, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4911cfa9:::obj108:head@111'108, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.052+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.052+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 382 ==== pg_backfill(progress 4.0 e 119/119 lb 4:48c1458b:::obj274:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f11080 con 0x507ac00 2026-02-11T23:31:28.052+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4911cfa9:::obj108:head, version: 111'108, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4911cfa9:::obj108:head@111'108, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.052+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:48c1458b:::obj274:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.052+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:48c1458b:::obj274:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.052+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.052+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4911cfa9:::obj108:head, version: 111'108, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4911cfa9:::obj108:head@111'108, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.052+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:4911cfa9:::obj108:head, version: 111'108, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4911cfa9:::obj108:head@111'108, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 119/117 [PushOp(4:4911cfa9:::obj108:head, version: 111'108, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4911cfa9:::obj108:head@111'108, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:4851a45a:::obj44:head local-lis/les=117/118 n=170 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.052+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.052+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.052+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:48c1458b:::obj274:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.052+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4851a45a:::obj44:head local-lis/les=117/118 n=170 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:4911cfa9:::obj108:head, version: 111'108, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4911cfa9:::obj108:head@111'108, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.052+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4851a45a:::obj44:head local-lis/les=117/118 n=170 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:4911cfa9:::obj108:head@111'108, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.052+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4851a45a:::obj44:head local-lis/les=117/118 n=170 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:4911cfa9:::obj108:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.052+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4851a45a:::obj44:head local-lis/les=117/118 n=170 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:4911cfa9:::obj108:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.052+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4851a45a:::obj44:head local-lis/les=117/118 n=170 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:4911cfa9:::obj108:head 2026-02-11T23:31:28.052+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4851a45a:::obj44:head local-lis/les=117/118 n=170 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:4911cfa9:::obj108:head v 111'108 2026-02-11T23:31:28.052+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4851a45a:::obj44:head local-lis/les=117/118 n=170 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.052+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.052+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:4911cfa9:::obj108:head, version: 111'108, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4911cfa9:::obj108:head@111'108, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:28.052+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.052+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.052+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:48c1458b:::obj274:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.052+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:48c1458b:::obj274:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 4.0 e 119/119 lb 4:48c1458b:::obj274:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:4851a45a:::obj44:head local-lis/les=117/118 n=170 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.052+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.052+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4851a45a:::obj44:head local-lis/les=117/118 n=170 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:48c1458b:::obj274:head) v3 2026-02-11T23:31:28.052+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4851a45a:::obj44:head local-lis/les=117/118 n=170 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:48c1458b:::obj274:head) 2026-02-11T23:31:28.052+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.052+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:48c1458b:::obj274:head) v3 finish latency 0.000015 2026-02-11T23:31:28.052+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:48c1458b:::obj274:head local-lis/les=117/118 n=171 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.052+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.052+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.052+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:48c1458b:::obj274:head local-lis/les=117/118 n=171 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.052+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.052+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.052+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.052+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7ac00 con 0x507ac00 2026-02-11T23:31:28.052+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:4911cfa9:::obj108:head)]) -- 0x63c9e00 con 0x507ac00 2026-02-11T23:31:28.052+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.053+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 383 ==== MOSDPGPush(4.0 119/117 [PushOp(4:4918aa91:::obj303:head, version: 111'303, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4918aa91:::obj303:head@111'303, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2d00 con 0x507ac00 2026-02-11T23:31:28.053+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:4918aa91:::obj303:head, version: 111'303, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4918aa91:::obj303:head@111'303, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000017 epoch 119 2026-02-11T23:31:28.053+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4918aa91:::obj303:head, version: 111'303, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4918aa91:::obj303:head@111'303, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.053+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.053+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4918aa91:::obj303:head, version: 111'303, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4918aa91:::obj303:head@111'303, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.053+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 384 ==== pg_backfill(progress 4.0 e 119/119 lb 4:4911cfa9:::obj108:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f11600 con 0x507ac00 2026-02-11T23:31:28.053+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.053+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:4911cfa9:::obj108:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 119 2026-02-11T23:31:28.053+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4918aa91:::obj303:head, version: 111'303, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4918aa91:::obj303:head@111'303, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.053+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4911cfa9:::obj108:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.053+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.053+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:4918aa91:::obj303:head, version: 111'303, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4918aa91:::obj303:head@111'303, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000043 MOSDPGPush(4.0 119/117 [PushOp(4:4918aa91:::obj303:head, version: 111'303, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4918aa91:::obj303:head@111'303, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:48c1458b:::obj274:head local-lis/les=117/118 n=171 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.053+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.053+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4911cfa9:::obj108:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.053+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:48c1458b:::obj274:head local-lis/les=117/118 n=171 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:4918aa91:::obj303:head, version: 111'303, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4918aa91:::obj303:head@111'303, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.053+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:48c1458b:::obj274:head local-lis/les=117/118 n=171 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:4918aa91:::obj303:head@111'303, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.053+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:48c1458b:::obj274:head local-lis/les=117/118 n=171 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:4918aa91:::obj303:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.053+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:48c1458b:::obj274:head local-lis/les=117/118 n=171 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:4918aa91:::obj303:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.053+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:48c1458b:::obj274:head local-lis/les=117/118 n=171 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:4918aa91:::obj303:head 2026-02-11T23:31:28.053+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:48c1458b:::obj274:head local-lis/les=117/118 n=171 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:4918aa91:::obj303:head v 111'303 2026-02-11T23:31:28.053+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:48c1458b:::obj274:head local-lis/les=117/118 n=171 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.053+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.053+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:4918aa91:::obj303:head, version: 111'303, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4918aa91:::obj303:head@111'303, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:28.053+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.053+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.053+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4911cfa9:::obj108:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.053+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:4911cfa9:::obj108:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 4.0 e 119/119 lb 4:4911cfa9:::obj108:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:48c1458b:::obj274:head local-lis/les=117/118 n=171 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.053+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.053+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:48c1458b:::obj274:head local-lis/les=117/118 n=171 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:4911cfa9:::obj108:head) v3 2026-02-11T23:31:28.053+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:48c1458b:::obj274:head local-lis/les=117/118 n=171 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:4911cfa9:::obj108:head) 2026-02-11T23:31:28.053+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.053+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:4911cfa9:::obj108:head) v3 finish latency 0.000015 2026-02-11T23:31:28.053+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4911cfa9:::obj108:head local-lis/les=117/118 n=172 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.053+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.053+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.053+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4911cfa9:::obj108:head local-lis/les=117/118 n=172 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.053+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.053+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.053+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.053+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7ad80 con 0x507ac00 2026-02-11T23:31:28.053+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:4918aa91:::obj303:head)]) -- 0x63c5980 con 0x507ac00 2026-02-11T23:31:28.053+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.054+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 385 ==== MOSDPGPush(4.0 119/117 [PushOp(4:492a75ed:::obj72:head, version: 111'72, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:492a75ed:::obj72:head@111'72, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5647380 con 0x507ac00 2026-02-11T23:31:28.054+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:492a75ed:::obj72:head, version: 111'72, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:492a75ed:::obj72:head@111'72, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 119 2026-02-11T23:31:28.054+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:492a75ed:::obj72:head, version: 111'72, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:492a75ed:::obj72:head@111'72, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.054+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.054+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:492a75ed:::obj72:head, version: 111'72, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:492a75ed:::obj72:head@111'72, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.054+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 386 ==== pg_backfill(progress 4.0 e 119/119 lb 4:4918aa91:::obj303:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2f700 con 0x507ac00 2026-02-11T23:31:28.054+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:4918aa91:::obj303:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 119 2026-02-11T23:31:28.054+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.054+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:492a75ed:::obj72:head, version: 111'72, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:492a75ed:::obj72:head@111'72, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.054+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4918aa91:::obj303:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.054+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.054+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:492a75ed:::obj72:head, version: 111'72, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:492a75ed:::obj72:head@111'72, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000040 MOSDPGPush(4.0 119/117 [PushOp(4:492a75ed:::obj72:head, version: 111'72, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:492a75ed:::obj72:head@111'72, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:4911cfa9:::obj108:head local-lis/les=117/118 n=172 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.054+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4918aa91:::obj303:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.054+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.054+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4911cfa9:::obj108:head local-lis/les=117/118 n=172 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:492a75ed:::obj72:head, version: 111'72, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:492a75ed:::obj72:head@111'72, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.054+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4911cfa9:::obj108:head local-lis/les=117/118 n=172 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:492a75ed:::obj72:head@111'72, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.054+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4911cfa9:::obj108:head local-lis/les=117/118 n=172 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:492a75ed:::obj72:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.054+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4911cfa9:::obj108:head local-lis/les=117/118 n=172 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:492a75ed:::obj72:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.054+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4911cfa9:::obj108:head local-lis/les=117/118 n=172 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:492a75ed:::obj72:head 2026-02-11T23:31:28.054+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4911cfa9:::obj108:head local-lis/les=117/118 n=172 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:492a75ed:::obj72:head v 111'72 2026-02-11T23:31:28.054+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4911cfa9:::obj108:head local-lis/les=117/118 n=172 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.054+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.054+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:492a75ed:::obj72:head, version: 111'72, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:492a75ed:::obj72:head@111'72, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:28.054+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.054+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.054+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4918aa91:::obj303:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.054+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:4918aa91:::obj303:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 4.0 e 119/119 lb 4:4918aa91:::obj303:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:4911cfa9:::obj108:head local-lis/les=117/118 n=172 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.054+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.054+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4911cfa9:::obj108:head local-lis/les=117/118 n=172 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:4918aa91:::obj303:head) v3 2026-02-11T23:31:28.054+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4911cfa9:::obj108:head local-lis/les=117/118 n=172 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:4918aa91:::obj303:head) 2026-02-11T23:31:28.054+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.054+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:4918aa91:::obj303:head) v3 finish latency 0.000016 2026-02-11T23:31:28.054+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4918aa91:::obj303:head local-lis/les=117/118 n=173 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.054+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.054+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.054+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4918aa91:::obj303:head local-lis/les=117/118 n=173 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.054+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.054+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.054+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.054+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7af00 con 0x507ac00 2026-02-11T23:31:28.054+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:492a75ed:::obj72:head)]) -- 0x63c9c80 con 0x507ac00 2026-02-11T23:31:28.054+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.054+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 387 ==== MOSDPGPush(4.0 119/117 [PushOp(4:498092df:::obj260:head, version: 111'260, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:498092df:::obj260:head@111'260, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e26680 con 0x507ac00 2026-02-11T23:31:28.054+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:498092df:::obj260:head, version: 111'260, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:498092df:::obj260:head@111'260, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 119 2026-02-11T23:31:28.054+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:498092df:::obj260:head, version: 111'260, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:498092df:::obj260:head@111'260, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.054+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.054+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 388 ==== pg_backfill(progress 4.0 e 119/119 lb 4:492a75ed:::obj72:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d2d080 con 0x507ac00 2026-02-11T23:31:28.054+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:492a75ed:::obj72:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.054+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:498092df:::obj260:head, version: 111'260, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:498092df:::obj260:head@111'260, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.054+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:492a75ed:::obj72:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.054+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.054+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:498092df:::obj260:head, version: 111'260, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:498092df:::obj260:head@111'260, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.054+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.054+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:498092df:::obj260:head, version: 111'260, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:498092df:::obj260:head@111'260, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(4.0 119/117 [PushOp(4:498092df:::obj260:head, version: 111'260, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:498092df:::obj260:head@111'260, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:4918aa91:::obj303:head local-lis/les=117/118 n=173 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.054+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.054+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4918aa91:::obj303:head local-lis/les=117/118 n=173 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:498092df:::obj260:head, version: 111'260, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:498092df:::obj260:head@111'260, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.054+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4918aa91:::obj303:head local-lis/les=117/118 n=173 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:498092df:::obj260:head@111'260, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.054+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.054+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:492a75ed:::obj72:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.054+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4918aa91:::obj303:head local-lis/les=117/118 n=173 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:498092df:::obj260:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.054+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4918aa91:::obj303:head local-lis/les=117/118 n=173 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:498092df:::obj260:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.054+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4918aa91:::obj303:head local-lis/les=117/118 n=173 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:498092df:::obj260:head 2026-02-11T23:31:28.054+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4918aa91:::obj303:head local-lis/les=117/118 n=173 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:498092df:::obj260:head v 111'260 2026-02-11T23:31:28.054+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4918aa91:::obj303:head local-lis/les=117/118 n=173 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.054+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.054+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:498092df:::obj260:head, version: 111'260, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:498092df:::obj260:head@111'260, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:28.054+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.054+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.054+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:492a75ed:::obj72:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.054+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:492a75ed:::obj72:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 4.0 e 119/119 lb 4:492a75ed:::obj72:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:4918aa91:::obj303:head local-lis/les=117/118 n=173 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.054+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.054+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4918aa91:::obj303:head local-lis/les=117/118 n=173 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:492a75ed:::obj72:head) v3 2026-02-11T23:31:28.054+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4918aa91:::obj303:head local-lis/les=117/118 n=173 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:492a75ed:::obj72:head) 2026-02-11T23:31:28.055+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.055+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:492a75ed:::obj72:head) v3 finish latency 0.000016 2026-02-11T23:31:28.055+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:492a75ed:::obj72:head local-lis/les=117/118 n=174 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.055+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.055+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.055+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:492a75ed:::obj72:head local-lis/les=117/118 n=174 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.055+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.055+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.055+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.055+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7b080 con 0x507ac00 2026-02-11T23:31:28.055+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:498092df:::obj260:head)]) -- 0x63c9800 con 0x507ac00 2026-02-11T23:31:28.055+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.055+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 389 ==== MOSDPGPush(4.0 119/117 [PushOp(4:4987b93a:::obj56:head, version: 111'56, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4987b93a:::obj56:head@111'56, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x559cb60 con 0x507ac00 2026-02-11T23:31:28.055+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:4987b93a:::obj56:head, version: 111'56, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4987b93a:::obj56:head@111'56, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 119 2026-02-11T23:31:28.055+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4987b93a:::obj56:head, version: 111'56, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4987b93a:::obj56:head@111'56, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.055+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.055+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4987b93a:::obj56:head, version: 111'56, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4987b93a:::obj56:head@111'56, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.055+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 390 ==== pg_backfill(progress 4.0 e 119/119 lb 4:498092df:::obj260:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e680 con 0x507ac00 2026-02-11T23:31:28.055+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.055+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:498092df:::obj260:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 119 2026-02-11T23:31:28.055+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4987b93a:::obj56:head, version: 111'56, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4987b93a:::obj56:head@111'56, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.055+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:498092df:::obj260:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.055+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.055+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:4987b93a:::obj56:head, version: 111'56, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4987b93a:::obj56:head@111'56, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(4.0 119/117 [PushOp(4:4987b93a:::obj56:head, version: 111'56, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4987b93a:::obj56:head@111'56, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:492a75ed:::obj72:head local-lis/les=117/118 n=174 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.055+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:498092df:::obj260:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.055+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.055+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:492a75ed:::obj72:head local-lis/les=117/118 n=174 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:4987b93a:::obj56:head, version: 111'56, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4987b93a:::obj56:head@111'56, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.055+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:492a75ed:::obj72:head local-lis/les=117/118 n=174 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:4987b93a:::obj56:head@111'56, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.055+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:492a75ed:::obj72:head local-lis/les=117/118 n=174 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:4987b93a:::obj56:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.055+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:492a75ed:::obj72:head local-lis/les=117/118 n=174 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:4987b93a:::obj56:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.055+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:492a75ed:::obj72:head local-lis/les=117/118 n=174 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:4987b93a:::obj56:head 2026-02-11T23:31:28.055+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:492a75ed:::obj72:head local-lis/les=117/118 n=174 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:4987b93a:::obj56:head v 111'56 2026-02-11T23:31:28.055+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:492a75ed:::obj72:head local-lis/les=117/118 n=174 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.055+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.055+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:4987b93a:::obj56:head, version: 111'56, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4987b93a:::obj56:head@111'56, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:28.055+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.055+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.055+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:498092df:::obj260:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.055+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:498092df:::obj260:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 4.0 e 119/119 lb 4:498092df:::obj260:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:492a75ed:::obj72:head local-lis/les=117/118 n=174 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.055+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.055+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:492a75ed:::obj72:head local-lis/les=117/118 n=174 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:498092df:::obj260:head) v3 2026-02-11T23:31:28.055+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:492a75ed:::obj72:head local-lis/les=117/118 n=174 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:498092df:::obj260:head) 2026-02-11T23:31:28.055+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.055+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:498092df:::obj260:head) v3 finish latency 0.000015 2026-02-11T23:31:28.055+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:498092df:::obj260:head local-lis/les=117/118 n=175 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.055+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.055+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.055+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:498092df:::obj260:head local-lis/les=117/118 n=175 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.055+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.055+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.055+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.055+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7b200 con 0x507ac00 2026-02-11T23:31:28.055+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:4987b93a:::obj56:head)]) -- 0x63c9980 con 0x507ac00 2026-02-11T23:31:28.055+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.056+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 391 ==== MOSDPGPush(4.0 119/117 [PushOp(4:498e91f7:::obj463:head, version: 111'463, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:498e91f7:::obj463:head@111'463, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f0b60 con 0x507ac00 2026-02-11T23:31:28.056+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:498e91f7:::obj463:head, version: 111'463, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:498e91f7:::obj463:head@111'463, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 119 2026-02-11T23:31:28.056+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:498e91f7:::obj463:head, version: 111'463, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:498e91f7:::obj463:head@111'463, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.056+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.056+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 392 ==== pg_backfill(progress 4.0 e 119/119 lb 4:4987b93a:::obj56:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d2e100 con 0x507ac00 2026-02-11T23:31:28.056+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:498e91f7:::obj463:head, version: 111'463, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:498e91f7:::obj463:head@111'463, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.056+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:4987b93a:::obj56:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.056+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4987b93a:::obj56:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.056+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.056+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:498e91f7:::obj463:head, version: 111'463, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:498e91f7:::obj463:head@111'463, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.056+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:498e91f7:::obj463:head, version: 111'463, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:498e91f7:::obj463:head@111'463, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(4.0 119/117 [PushOp(4:498e91f7:::obj463:head, version: 111'463, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:498e91f7:::obj463:head@111'463, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:498092df:::obj260:head local-lis/les=117/118 n=175 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.056+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.056+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.056+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4987b93a:::obj56:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.056+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:498092df:::obj260:head local-lis/les=117/118 n=175 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:498e91f7:::obj463:head, version: 111'463, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:498e91f7:::obj463:head@111'463, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.056+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:498092df:::obj260:head local-lis/les=117/118 n=175 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:498e91f7:::obj463:head@111'463, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.056+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:498092df:::obj260:head local-lis/les=117/118 n=175 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:498e91f7:::obj463:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.056+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:498092df:::obj260:head local-lis/les=117/118 n=175 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:498e91f7:::obj463:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.056+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:498092df:::obj260:head local-lis/les=117/118 n=175 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:498e91f7:::obj463:head 2026-02-11T23:31:28.056+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:498092df:::obj260:head local-lis/les=117/118 n=175 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:498e91f7:::obj463:head v 111'463 2026-02-11T23:31:28.056+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:498092df:::obj260:head local-lis/les=117/118 n=175 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.056+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.056+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:498e91f7:::obj463:head, version: 111'463, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:498e91f7:::obj463:head@111'463, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:28.056+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.056+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.056+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4987b93a:::obj56:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.056+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:4987b93a:::obj56:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 4.0 e 119/119 lb 4:4987b93a:::obj56:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:498092df:::obj260:head local-lis/les=117/118 n=175 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.056+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.056+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:498092df:::obj260:head local-lis/les=117/118 n=175 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:4987b93a:::obj56:head) v3 2026-02-11T23:31:28.056+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:498092df:::obj260:head local-lis/les=117/118 n=175 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:4987b93a:::obj56:head) 2026-02-11T23:31:28.056+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.056+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:4987b93a:::obj56:head) v3 finish latency 0.000013 2026-02-11T23:31:28.056+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4987b93a:::obj56:head local-lis/les=117/118 n=176 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.056+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.056+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.056+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4987b93a:::obj56:head local-lis/les=117/118 n=176 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.056+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.056+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.056+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.056+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7b380 con 0x507ac00 2026-02-11T23:31:28.056+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:498e91f7:::obj463:head)]) -- 0x63c5c80 con 0x507ac00 2026-02-11T23:31:28.056+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.056+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 393 ==== MOSDPGPush(4.0 119/117 [PushOp(4:49f89eae:::obj182:head, version: 111'182, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:49f89eae:::obj182:head@111'182, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c4e0 con 0x507ac00 2026-02-11T23:31:28.056+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:49f89eae:::obj182:head, version: 111'182, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:49f89eae:::obj182:head@111'182, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.056+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:49f89eae:::obj182:head, version: 111'182, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:49f89eae:::obj182:head@111'182, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.056+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.056+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 394 ==== pg_backfill(progress 4.0 e 119/119 lb 4:498e91f7:::obj463:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2c000 con 0x507ac00 2026-02-11T23:31:28.056+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:49f89eae:::obj182:head, version: 111'182, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:49f89eae:::obj182:head@111'182, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.056+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:498e91f7:::obj463:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.056+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:498e91f7:::obj463:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.056+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.056+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:49f89eae:::obj182:head, version: 111'182, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:49f89eae:::obj182:head@111'182, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.056+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.056+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:49f89eae:::obj182:head, version: 111'182, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:49f89eae:::obj182:head@111'182, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 119/117 [PushOp(4:49f89eae:::obj182:head, version: 111'182, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:49f89eae:::obj182:head@111'182, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:4987b93a:::obj56:head local-lis/les=117/118 n=176 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.056+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.056+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4987b93a:::obj56:head local-lis/les=117/118 n=176 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:49f89eae:::obj182:head, version: 111'182, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:49f89eae:::obj182:head@111'182, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.056+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4987b93a:::obj56:head local-lis/les=117/118 n=176 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:49f89eae:::obj182:head@111'182, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.056+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.056+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:498e91f7:::obj463:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.056+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4987b93a:::obj56:head local-lis/les=117/118 n=176 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:49f89eae:::obj182:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.056+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4987b93a:::obj56:head local-lis/les=117/118 n=176 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:49f89eae:::obj182:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.056+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4987b93a:::obj56:head local-lis/les=117/118 n=176 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:49f89eae:::obj182:head 2026-02-11T23:31:28.056+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4987b93a:::obj56:head local-lis/les=117/118 n=176 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:49f89eae:::obj182:head v 111'182 2026-02-11T23:31:28.056+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4987b93a:::obj56:head local-lis/les=117/118 n=176 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.056+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.057+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:49f89eae:::obj182:head, version: 111'182, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:49f89eae:::obj182:head@111'182, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000062 2026-02-11T23:31:28.057+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.057+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.057+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:498e91f7:::obj463:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.057+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:498e91f7:::obj463:head) v3 prio 15 cost 0 latency 0.000081 pg_backfill(progress 4.0 e 119/119 lb 4:498e91f7:::obj463:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:4987b93a:::obj56:head local-lis/les=117/118 n=176 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.057+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.057+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4987b93a:::obj56:head local-lis/les=117/118 n=176 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:498e91f7:::obj463:head) v3 2026-02-11T23:31:28.057+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4987b93a:::obj56:head local-lis/les=117/118 n=176 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:498e91f7:::obj463:head) 2026-02-11T23:31:28.057+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.057+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:498e91f7:::obj463:head) v3 finish latency 0.000012 2026-02-11T23:31:28.057+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:498e91f7:::obj463:head local-lis/les=117/118 n=177 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.057+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.057+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.057+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:498e91f7:::obj463:head local-lis/les=117/118 n=177 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.057+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.057+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.057+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.057+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7b500 con 0x507ac00 2026-02-11T23:31:28.057+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:49f89eae:::obj182:head)]) -- 0x63c5b00 con 0x507ac00 2026-02-11T23:31:28.057+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.057+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 395 ==== MOSDPGPush(4.0 119/117 [PushOp(4:4b1d3252:::obj86:head, version: 111'86, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b1d3252:::obj86:head@111'86, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x55f0d00 con 0x507ac00 2026-02-11T23:31:28.057+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:4b1d3252:::obj86:head, version: 111'86, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b1d3252:::obj86:head@111'86, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 119 2026-02-11T23:31:28.057+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4b1d3252:::obj86:head, version: 111'86, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b1d3252:::obj86:head@111'86, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.057+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.057+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 396 ==== pg_backfill(progress 4.0 e 119/119 lb 4:49f89eae:::obj182:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2cb00 con 0x507ac00 2026-02-11T23:31:28.057+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:49f89eae:::obj182:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.057+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4b1d3252:::obj86:head, version: 111'86, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b1d3252:::obj86:head@111'86, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.057+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:49f89eae:::obj182:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.057+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.057+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:49f89eae:::obj182:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.057+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.057+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4b1d3252:::obj86:head, version: 111'86, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b1d3252:::obj86:head@111'86, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.057+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:4b1d3252:::obj86:head, version: 111'86, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b1d3252:::obj86:head@111'86, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000047 MOSDPGPush(4.0 119/117 [PushOp(4:4b1d3252:::obj86:head, version: 111'86, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b1d3252:::obj86:head@111'86, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:498e91f7:::obj463:head local-lis/les=117/118 n=177 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.057+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.057+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:498e91f7:::obj463:head local-lis/les=117/118 n=177 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:4b1d3252:::obj86:head, version: 111'86, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b1d3252:::obj86:head@111'86, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.057+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:498e91f7:::obj463:head local-lis/les=117/118 n=177 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:4b1d3252:::obj86:head@111'86, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.057+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:498e91f7:::obj463:head local-lis/les=117/118 n=177 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:4b1d3252:::obj86:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.057+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:498e91f7:::obj463:head local-lis/les=117/118 n=177 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:4b1d3252:::obj86:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.057+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:498e91f7:::obj463:head local-lis/les=117/118 n=177 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:4b1d3252:::obj86:head 2026-02-11T23:31:28.057+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:498e91f7:::obj463:head local-lis/les=117/118 n=177 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:4b1d3252:::obj86:head v 111'86 2026-02-11T23:31:28.057+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:498e91f7:::obj463:head local-lis/les=117/118 n=177 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.057+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.057+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:4b1d3252:::obj86:head, version: 111'86, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b1d3252:::obj86:head@111'86, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:28.057+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.057+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.057+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:49f89eae:::obj182:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.057+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:49f89eae:::obj182:head) v3 prio 15 cost 0 latency 0.000085 pg_backfill(progress 4.0 e 119/119 lb 4:49f89eae:::obj182:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:498e91f7:::obj463:head local-lis/les=117/118 n=177 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.057+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.057+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:498e91f7:::obj463:head local-lis/les=117/118 n=177 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:49f89eae:::obj182:head) v3 2026-02-11T23:31:28.057+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:498e91f7:::obj463:head local-lis/les=117/118 n=177 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:49f89eae:::obj182:head) 2026-02-11T23:31:28.057+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.057+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:49f89eae:::obj182:head) v3 finish latency 0.000013 2026-02-11T23:31:28.057+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:49f89eae:::obj182:head local-lis/les=117/118 n=178 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.057+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.057+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.057+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:49f89eae:::obj182:head local-lis/les=117/118 n=178 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.057+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.057+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.057+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.057+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7b680 con 0x507ac00 2026-02-11T23:31:28.057+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:4b1d3252:::obj86:head)]) -- 0x4f7a000 con 0x507ac00 2026-02-11T23:31:28.057+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.058+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 397 ==== MOSDPGPush(4.0 119/117 [PushOp(4:4b57ab44:::obj335:head, version: 111'335, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b57ab44:::obj335:head@111'335, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56cab60 con 0x507ac00 2026-02-11T23:31:28.058+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:4b57ab44:::obj335:head, version: 111'335, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b57ab44:::obj335:head@111'335, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.058+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4b57ab44:::obj335:head, version: 111'335, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b57ab44:::obj335:head@111'335, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.058+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.058+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 398 ==== pg_backfill(progress 4.0 e 119/119 lb 4:4b1d3252:::obj86:head) ==== 1061+0+0 (unknown 0 0 0) 0x6179700 con 0x507ac00 2026-02-11T23:31:28.058+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4b57ab44:::obj335:head, version: 111'335, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b57ab44:::obj335:head@111'335, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.058+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:4b1d3252:::obj86:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.058+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.058+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4b57ab44:::obj335:head, version: 111'335, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b57ab44:::obj335:head@111'335, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.058+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4b1d3252:::obj86:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.058+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.058+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:4b57ab44:::obj335:head, version: 111'335, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b57ab44:::obj335:head@111'335, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 119/117 [PushOp(4:4b57ab44:::obj335:head, version: 111'335, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b57ab44:::obj335:head@111'335, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:49f89eae:::obj182:head local-lis/les=117/118 n=178 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.058+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4b1d3252:::obj86:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.058+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.058+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:49f89eae:::obj182:head local-lis/les=117/118 n=178 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:4b57ab44:::obj335:head, version: 111'335, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b57ab44:::obj335:head@111'335, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.058+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:49f89eae:::obj182:head local-lis/les=117/118 n=178 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:4b57ab44:::obj335:head@111'335, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.058+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:49f89eae:::obj182:head local-lis/les=117/118 n=178 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:4b57ab44:::obj335:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.058+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:49f89eae:::obj182:head local-lis/les=117/118 n=178 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:4b57ab44:::obj335:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.058+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:49f89eae:::obj182:head local-lis/les=117/118 n=178 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:4b57ab44:::obj335:head 2026-02-11T23:31:28.058+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:49f89eae:::obj182:head local-lis/les=117/118 n=178 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:4b57ab44:::obj335:head v 111'335 2026-02-11T23:31:28.058+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:49f89eae:::obj182:head local-lis/les=117/118 n=178 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.058+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.058+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:4b57ab44:::obj335:head, version: 111'335, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b57ab44:::obj335:head@111'335, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:28.058+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.058+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.058+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4b1d3252:::obj86:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.058+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:4b1d3252:::obj86:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 4.0 e 119/119 lb 4:4b1d3252:::obj86:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:49f89eae:::obj182:head local-lis/les=117/118 n=178 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.058+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.058+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:49f89eae:::obj182:head local-lis/les=117/118 n=178 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:4b1d3252:::obj86:head) v3 2026-02-11T23:31:28.058+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:49f89eae:::obj182:head local-lis/les=117/118 n=178 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:4b1d3252:::obj86:head) 2026-02-11T23:31:28.058+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.058+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:4b1d3252:::obj86:head) v3 finish latency 0.000016 2026-02-11T23:31:28.058+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4b1d3252:::obj86:head local-lis/les=117/118 n=179 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.058+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.058+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.058+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4b1d3252:::obj86:head local-lis/les=117/118 n=179 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.058+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.058+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.058+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.058+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7b800 con 0x507ac00 2026-02-11T23:31:28.058+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:4b57ab44:::obj335:head)]) -- 0x649f200 con 0x507ac00 2026-02-11T23:31:28.058+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.058+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 399 ==== MOSDPGPush(4.0 119/117 [PushOp(4:4b639beb:::obj280:head, version: 111'280, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b639beb:::obj280:head@111'280, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2680 con 0x507ac00 2026-02-11T23:31:28.058+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:4b639beb:::obj280:head, version: 111'280, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b639beb:::obj280:head@111'280, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 119 2026-02-11T23:31:28.058+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4b639beb:::obj280:head, version: 111'280, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b639beb:::obj280:head@111'280, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.058+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.058+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4b639beb:::obj280:head, version: 111'280, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b639beb:::obj280:head@111'280, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.058+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 400 ==== pg_backfill(progress 4.0 e 119/119 lb 4:4b57ab44:::obj335:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176000 con 0x507ac00 2026-02-11T23:31:28.058+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:4b57ab44:::obj335:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 119 2026-02-11T23:31:28.058+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.058+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4b57ab44:::obj335:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.058+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4b639beb:::obj280:head, version: 111'280, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b639beb:::obj280:head@111'280, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.058+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.058+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:4b639beb:::obj280:head, version: 111'280, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b639beb:::obj280:head@111'280, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(4.0 119/117 [PushOp(4:4b639beb:::obj280:head, version: 111'280, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b639beb:::obj280:head@111'280, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:4b1d3252:::obj86:head local-lis/les=117/118 n=179 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.058+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.058+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4b1d3252:::obj86:head local-lis/les=117/118 n=179 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:4b639beb:::obj280:head, version: 111'280, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b639beb:::obj280:head@111'280, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.058+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4b1d3252:::obj86:head local-lis/les=117/118 n=179 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:4b639beb:::obj280:head@111'280, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.058+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.058+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4b57ab44:::obj335:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.058+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4b1d3252:::obj86:head local-lis/les=117/118 n=179 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:4b639beb:::obj280:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.058+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4b1d3252:::obj86:head local-lis/les=117/118 n=179 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:4b639beb:::obj280:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.058+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4b1d3252:::obj86:head local-lis/les=117/118 n=179 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:4b639beb:::obj280:head 2026-02-11T23:31:28.058+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4b1d3252:::obj86:head local-lis/les=117/118 n=179 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:4b639beb:::obj280:head v 111'280 2026-02-11T23:31:28.058+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4b1d3252:::obj86:head local-lis/les=117/118 n=179 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.058+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.058+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:4b639beb:::obj280:head, version: 111'280, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b639beb:::obj280:head@111'280, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:28.058+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.058+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.058+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4b57ab44:::obj335:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.058+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:4b57ab44:::obj335:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 4.0 e 119/119 lb 4:4b57ab44:::obj335:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:4b1d3252:::obj86:head local-lis/les=117/118 n=179 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.058+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.058+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4b1d3252:::obj86:head local-lis/les=117/118 n=179 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:4b57ab44:::obj335:head) v3 2026-02-11T23:31:28.058+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4b1d3252:::obj86:head local-lis/les=117/118 n=179 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:4b57ab44:::obj335:head) 2026-02-11T23:31:28.058+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.058+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:4b57ab44:::obj335:head) v3 finish latency 0.000013 2026-02-11T23:31:28.058+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4b57ab44:::obj335:head local-lis/les=117/118 n=180 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.058+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.059+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.059+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4b57ab44:::obj335:head local-lis/les=117/118 n=180 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.059+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.059+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.059+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.059+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7b980 con 0x507ac00 2026-02-11T23:31:28.059+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:4b639beb:::obj280:head)]) -- 0x63c5080 con 0x507ac00 2026-02-11T23:31:28.059+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.059+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 401 ==== MOSDPGPush(4.0 119/117 [PushOp(4:4baa8bb7:::obj594:head, version: 111'594, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4baa8bb7:::obj594:head@111'594, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db0b60 con 0x507ac00 2026-02-11T23:31:28.059+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:4baa8bb7:::obj594:head, version: 111'594, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4baa8bb7:::obj594:head@111'594, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 119 2026-02-11T23:31:28.059+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4baa8bb7:::obj594:head, version: 111'594, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4baa8bb7:::obj594:head@111'594, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.059+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.059+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 402 ==== pg_backfill(progress 4.0 e 119/119 lb 4:4b639beb:::obj280:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9a680 con 0x507ac00 2026-02-11T23:31:28.059+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4baa8bb7:::obj594:head, version: 111'594, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4baa8bb7:::obj594:head@111'594, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.059+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:4b639beb:::obj280:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.059+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4b639beb:::obj280:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.059+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.059+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4baa8bb7:::obj594:head, version: 111'594, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4baa8bb7:::obj594:head@111'594, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.059+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.059+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:4baa8bb7:::obj594:head, version: 111'594, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4baa8bb7:::obj594:head@111'594, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000040 MOSDPGPush(4.0 119/117 [PushOp(4:4baa8bb7:::obj594:head, version: 111'594, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4baa8bb7:::obj594:head@111'594, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:4b57ab44:::obj335:head local-lis/les=117/118 n=180 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.059+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.059+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4b57ab44:::obj335:head local-lis/les=117/118 n=180 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:4baa8bb7:::obj594:head, version: 111'594, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4baa8bb7:::obj594:head@111'594, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.059+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.059+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4b639beb:::obj280:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.059+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4b57ab44:::obj335:head local-lis/les=117/118 n=180 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:4baa8bb7:::obj594:head@111'594, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.059+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4b57ab44:::obj335:head local-lis/les=117/118 n=180 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:4baa8bb7:::obj594:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.059+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4b57ab44:::obj335:head local-lis/les=117/118 n=180 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:4baa8bb7:::obj594:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.059+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4b57ab44:::obj335:head local-lis/les=117/118 n=180 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:4baa8bb7:::obj594:head 2026-02-11T23:31:28.059+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4b57ab44:::obj335:head local-lis/les=117/118 n=180 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:4baa8bb7:::obj594:head v 111'594 2026-02-11T23:31:28.059+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4b57ab44:::obj335:head local-lis/les=117/118 n=180 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.059+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.059+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:4baa8bb7:::obj594:head, version: 111'594, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4baa8bb7:::obj594:head@111'594, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:28.059+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.059+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.059+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4b639beb:::obj280:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.059+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:4b639beb:::obj280:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 4.0 e 119/119 lb 4:4b639beb:::obj280:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:4b57ab44:::obj335:head local-lis/les=117/118 n=180 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.059+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.059+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4b57ab44:::obj335:head local-lis/les=117/118 n=180 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:4b639beb:::obj280:head) v3 2026-02-11T23:31:28.059+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4b57ab44:::obj335:head local-lis/les=117/118 n=180 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:4b639beb:::obj280:head) 2026-02-11T23:31:28.059+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.059+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:4b639beb:::obj280:head) v3 finish latency 0.000014 2026-02-11T23:31:28.059+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4b639beb:::obj280:head local-lis/les=117/118 n=181 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.059+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.059+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.059+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4b639beb:::obj280:head local-lis/les=117/118 n=181 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.059+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.059+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.059+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.059+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7bb00 con 0x507ac00 2026-02-11T23:31:28.059+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:4baa8bb7:::obj594:head)]) -- 0x5f2dc80 con 0x507ac00 2026-02-11T23:31:28.059+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.060+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 403 ==== MOSDPGPush(4.0 119/117 [PushOp(4:4c3f91c8:::obj470:head, version: 111'470, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c3f91c8:::obj470:head@111'470, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a3ba0 con 0x507ac00 2026-02-11T23:31:28.060+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:4c3f91c8:::obj470:head, version: 111'470, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c3f91c8:::obj470:head@111'470, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 119 2026-02-11T23:31:28.060+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4c3f91c8:::obj470:head, version: 111'470, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c3f91c8:::obj470:head@111'470, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.060+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.060+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 404 ==== pg_backfill(progress 4.0 e 119/119 lb 4:4baa8bb7:::obj594:head) ==== 1062+0+0 (unknown 0 0 0) 0x5985700 con 0x507ac00 2026-02-11T23:31:28.060+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4c3f91c8:::obj470:head, version: 111'470, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c3f91c8:::obj470:head@111'470, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.060+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:4baa8bb7:::obj594:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.060+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.060+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4baa8bb7:::obj594:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.060+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4c3f91c8:::obj470:head, version: 111'470, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c3f91c8:::obj470:head@111'470, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.060+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.060+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:4c3f91c8:::obj470:head, version: 111'470, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c3f91c8:::obj470:head@111'470, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(4.0 119/117 [PushOp(4:4c3f91c8:::obj470:head, version: 111'470, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c3f91c8:::obj470:head@111'470, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:4b639beb:::obj280:head local-lis/les=117/118 n=181 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.060+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.060+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4b639beb:::obj280:head local-lis/les=117/118 n=181 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:4c3f91c8:::obj470:head, version: 111'470, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c3f91c8:::obj470:head@111'470, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.060+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.060+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4b639beb:::obj280:head local-lis/les=117/118 n=181 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:4c3f91c8:::obj470:head@111'470, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.060+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4baa8bb7:::obj594:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.060+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4b639beb:::obj280:head local-lis/les=117/118 n=181 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:4c3f91c8:::obj470:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.060+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4b639beb:::obj280:head local-lis/les=117/118 n=181 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:4c3f91c8:::obj470:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.060+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4b639beb:::obj280:head local-lis/les=117/118 n=181 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:4c3f91c8:::obj470:head 2026-02-11T23:31:28.060+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4b639beb:::obj280:head local-lis/les=117/118 n=181 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:4c3f91c8:::obj470:head v 111'470 2026-02-11T23:31:28.060+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4b639beb:::obj280:head local-lis/les=117/118 n=181 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.060+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.060+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:4c3f91c8:::obj470:head, version: 111'470, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c3f91c8:::obj470:head@111'470, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:28.060+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.060+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.060+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4baa8bb7:::obj594:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.060+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:4baa8bb7:::obj594:head) v3 prio 15 cost 0 latency 0.000081 pg_backfill(progress 4.0 e 119/119 lb 4:4baa8bb7:::obj594:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:4b639beb:::obj280:head local-lis/les=117/118 n=181 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.060+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.060+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4b639beb:::obj280:head local-lis/les=117/118 n=181 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:4baa8bb7:::obj594:head) v3 2026-02-11T23:31:28.060+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4b639beb:::obj280:head local-lis/les=117/118 n=181 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:4baa8bb7:::obj594:head) 2026-02-11T23:31:28.060+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.060+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:4baa8bb7:::obj594:head) v3 finish latency 0.000013 2026-02-11T23:31:28.060+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4baa8bb7:::obj594:head local-lis/les=117/118 n=182 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.060+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.060+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.060+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4baa8bb7:::obj594:head local-lis/les=117/118 n=182 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.060+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.060+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.060+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.060+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7bc80 con 0x507ac00 2026-02-11T23:31:28.060+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:4c3f91c8:::obj470:head)]) -- 0x63c8000 con 0x507ac00 2026-02-11T23:31:28.060+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.060+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 405 ==== MOSDPGPush(4.0 119/117 [PushOp(4:4c59dc70:::obj344:head, version: 111'344, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c59dc70:::obj344:head@111'344, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2820 con 0x507ac00 2026-02-11T23:31:28.060+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:4c59dc70:::obj344:head, version: 111'344, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c59dc70:::obj344:head@111'344, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 119 2026-02-11T23:31:28.060+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4c59dc70:::obj344:head, version: 111'344, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c59dc70:::obj344:head@111'344, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.060+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.060+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 406 ==== pg_backfill(progress 4.0 e 119/119 lb 4:4c3f91c8:::obj470:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177600 con 0x507ac00 2026-02-11T23:31:28.060+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4c59dc70:::obj344:head, version: 111'344, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c59dc70:::obj344:head@111'344, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.060+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:4c3f91c8:::obj470:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 119 2026-02-11T23:31:28.060+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.060+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4c3f91c8:::obj470:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.060+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4c59dc70:::obj344:head, version: 111'344, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c59dc70:::obj344:head@111'344, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.060+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.060+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:4c59dc70:::obj344:head, version: 111'344, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c59dc70:::obj344:head@111'344, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000040 MOSDPGPush(4.0 119/117 [PushOp(4:4c59dc70:::obj344:head, version: 111'344, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c59dc70:::obj344:head@111'344, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:4baa8bb7:::obj594:head local-lis/les=117/118 n=182 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.060+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.060+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4baa8bb7:::obj594:head local-lis/les=117/118 n=182 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:4c59dc70:::obj344:head, version: 111'344, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c59dc70:::obj344:head@111'344, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.060+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4baa8bb7:::obj594:head local-lis/les=117/118 n=182 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:4c59dc70:::obj344:head@111'344, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.060+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.060+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4c3f91c8:::obj470:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.060+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4baa8bb7:::obj594:head local-lis/les=117/118 n=182 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:4c59dc70:::obj344:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.060+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4baa8bb7:::obj594:head local-lis/les=117/118 n=182 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:4c59dc70:::obj344:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.060+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4baa8bb7:::obj594:head local-lis/les=117/118 n=182 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:4c59dc70:::obj344:head 2026-02-11T23:31:28.060+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4baa8bb7:::obj594:head local-lis/les=117/118 n=182 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:4c59dc70:::obj344:head v 111'344 2026-02-11T23:31:28.060+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4baa8bb7:::obj594:head local-lis/les=117/118 n=182 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.060+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.060+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:4c59dc70:::obj344:head, version: 111'344, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c59dc70:::obj344:head@111'344, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:28.060+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.060+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.060+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4c3f91c8:::obj470:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.060+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:4c3f91c8:::obj470:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 4.0 e 119/119 lb 4:4c3f91c8:::obj470:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:4baa8bb7:::obj594:head local-lis/les=117/118 n=182 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.060+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.060+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4baa8bb7:::obj594:head local-lis/les=117/118 n=182 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:4c3f91c8:::obj470:head) v3 2026-02-11T23:31:28.060+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4baa8bb7:::obj594:head local-lis/les=117/118 n=182 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:4c3f91c8:::obj470:head) 2026-02-11T23:31:28.060+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.060+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:4c3f91c8:::obj470:head) v3 finish latency 0.000013 2026-02-11T23:31:28.060+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c3f91c8:::obj470:head local-lis/les=117/118 n=183 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.060+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.061+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.061+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c3f91c8:::obj470:head local-lis/les=117/118 n=183 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.061+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.061+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.061+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.061+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7be00 con 0x507ac00 2026-02-11T23:31:28.061+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:4c59dc70:::obj344:head)]) -- 0x4f43e00 con 0x507ac00 2026-02-11T23:31:28.061+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.061+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 407 ==== MOSDPGPush(4.0 119/117 [PushOp(4:4c73ba95:::obj68:head, version: 111'68, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c73ba95:::obj68:head@111'68, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x555d040 con 0x507ac00 2026-02-11T23:31:28.061+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:4c73ba95:::obj68:head, version: 111'68, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c73ba95:::obj68:head@111'68, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 119 2026-02-11T23:31:28.061+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4c73ba95:::obj68:head, version: 111'68, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c73ba95:::obj68:head@111'68, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.061+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.061+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 408 ==== pg_backfill(progress 4.0 e 119/119 lb 4:4c59dc70:::obj344:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177b80 con 0x507ac00 2026-02-11T23:31:28.061+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4c73ba95:::obj68:head, version: 111'68, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c73ba95:::obj68:head@111'68, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.061+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:4c59dc70:::obj344:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.061+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4c59dc70:::obj344:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.061+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.061+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4c59dc70:::obj344:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.061+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.061+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4c73ba95:::obj68:head, version: 111'68, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c73ba95:::obj68:head@111'68, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.061+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:4c73ba95:::obj68:head, version: 111'68, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c73ba95:::obj68:head@111'68, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000053 MOSDPGPush(4.0 119/117 [PushOp(4:4c73ba95:::obj68:head, version: 111'68, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c73ba95:::obj68:head@111'68, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:4c3f91c8:::obj470:head local-lis/les=117/118 n=183 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.061+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.061+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c3f91c8:::obj470:head local-lis/les=117/118 n=183 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:4c73ba95:::obj68:head, version: 111'68, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c73ba95:::obj68:head@111'68, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.061+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c3f91c8:::obj470:head local-lis/les=117/118 n=183 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:4c73ba95:::obj68:head@111'68, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.061+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c3f91c8:::obj470:head local-lis/les=117/118 n=183 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:4c73ba95:::obj68:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.061+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c3f91c8:::obj470:head local-lis/les=117/118 n=183 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:4c73ba95:::obj68:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.061+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c3f91c8:::obj470:head local-lis/les=117/118 n=183 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:4c73ba95:::obj68:head 2026-02-11T23:31:28.061+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c3f91c8:::obj470:head local-lis/les=117/118 n=183 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:4c73ba95:::obj68:head v 111'68 2026-02-11T23:31:28.061+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c3f91c8:::obj470:head local-lis/les=117/118 n=183 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.061+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.061+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:4c73ba95:::obj68:head, version: 111'68, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c73ba95:::obj68:head@111'68, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:28.061+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.061+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.061+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4c59dc70:::obj344:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.061+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:4c59dc70:::obj344:head) v3 prio 15 cost 0 latency 0.000086 pg_backfill(progress 4.0 e 119/119 lb 4:4c59dc70:::obj344:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:4c3f91c8:::obj470:head local-lis/les=117/118 n=183 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.061+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.061+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c3f91c8:::obj470:head local-lis/les=117/118 n=183 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:4c59dc70:::obj344:head) v3 2026-02-11T23:31:28.061+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c3f91c8:::obj470:head local-lis/les=117/118 n=183 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:4c59dc70:::obj344:head) 2026-02-11T23:31:28.061+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.061+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:4c59dc70:::obj344:head) v3 finish latency 0.000014 2026-02-11T23:31:28.061+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c59dc70:::obj344:head local-lis/les=117/118 n=184 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.061+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.061+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.061+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c59dc70:::obj344:head local-lis/les=117/118 n=184 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.061+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.061+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.061+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.061+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x649f200 con 0x507ac00 2026-02-11T23:31:28.061+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:4c73ba95:::obj68:head)]) -- 0x4f43b00 con 0x507ac00 2026-02-11T23:31:28.061+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.062+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 409 ==== MOSDPGPush(4.0 119/117 [PushOp(4:4c7e2cfa:::obj454:head, version: 111'454, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c7e2cfa:::obj454:head@111'454, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559d520 con 0x507ac00 2026-02-11T23:31:28.062+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:4c7e2cfa:::obj454:head, version: 111'454, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c7e2cfa:::obj454:head@111'454, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 119 2026-02-11T23:31:28.062+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4c7e2cfa:::obj454:head, version: 111'454, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c7e2cfa:::obj454:head@111'454, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.062+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.062+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 410 ==== pg_backfill(progress 4.0 e 119/119 lb 4:4c73ba95:::obj68:head) ==== 1061+0+0 (unknown 0 0 0) 0x6178100 con 0x507ac00 2026-02-11T23:31:28.062+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4c7e2cfa:::obj454:head, version: 111'454, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c7e2cfa:::obj454:head@111'454, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.062+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:4c73ba95:::obj68:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 119 2026-02-11T23:31:28.062+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.062+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4c73ba95:::obj68:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.062+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4c7e2cfa:::obj454:head, version: 111'454, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c7e2cfa:::obj454:head@111'454, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.062+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.062+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:4c7e2cfa:::obj454:head, version: 111'454, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c7e2cfa:::obj454:head@111'454, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(4.0 119/117 [PushOp(4:4c7e2cfa:::obj454:head, version: 111'454, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c7e2cfa:::obj454:head@111'454, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:4c59dc70:::obj344:head local-lis/les=117/118 n=184 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.062+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.062+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c59dc70:::obj344:head local-lis/les=117/118 n=184 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:4c7e2cfa:::obj454:head, version: 111'454, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c7e2cfa:::obj454:head@111'454, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.062+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.062+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c59dc70:::obj344:head local-lis/les=117/118 n=184 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:4c7e2cfa:::obj454:head@111'454, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.062+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4c73ba95:::obj68:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.062+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c59dc70:::obj344:head local-lis/les=117/118 n=184 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:4c7e2cfa:::obj454:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.062+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c59dc70:::obj344:head local-lis/les=117/118 n=184 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:4c7e2cfa:::obj454:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.062+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c59dc70:::obj344:head local-lis/les=117/118 n=184 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:4c7e2cfa:::obj454:head 2026-02-11T23:31:28.062+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c59dc70:::obj344:head local-lis/les=117/118 n=184 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:4c7e2cfa:::obj454:head v 111'454 2026-02-11T23:31:28.062+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c59dc70:::obj344:head local-lis/les=117/118 n=184 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.062+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.062+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:4c7e2cfa:::obj454:head, version: 111'454, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c7e2cfa:::obj454:head@111'454, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:28.062+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.062+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.062+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4c73ba95:::obj68:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.062+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:4c73ba95:::obj68:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 4.0 e 119/119 lb 4:4c73ba95:::obj68:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:4c59dc70:::obj344:head local-lis/les=117/118 n=184 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.062+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.062+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c59dc70:::obj344:head local-lis/les=117/118 n=184 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:4c73ba95:::obj68:head) v3 2026-02-11T23:31:28.062+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c59dc70:::obj344:head local-lis/les=117/118 n=184 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:4c73ba95:::obj68:head) 2026-02-11T23:31:28.062+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.062+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:4c73ba95:::obj68:head) v3 finish latency 0.000013 2026-02-11T23:31:28.062+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c73ba95:::obj68:head local-lis/les=117/118 n=185 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.062+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.062+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.062+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c73ba95:::obj68:head local-lis/les=117/118 n=185 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.062+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.062+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.062+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.062+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7b800 con 0x507ac00 2026-02-11T23:31:28.062+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:4c7e2cfa:::obj454:head)]) -- 0x649ea80 con 0x507ac00 2026-02-11T23:31:28.062+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.062+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 411 ==== MOSDPGPush(4.0 119/117 [PushOp(4:4c9f7fa8:::obj477:head, version: 111'477, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c9f7fa8:::obj477:head@111'477, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56ca9c0 con 0x507ac00 2026-02-11T23:31:28.062+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:4c9f7fa8:::obj477:head, version: 111'477, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c9f7fa8:::obj477:head@111'477, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 119 2026-02-11T23:31:28.062+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4c9f7fa8:::obj477:head, version: 111'477, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c9f7fa8:::obj477:head@111'477, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.062+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.062+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4c9f7fa8:::obj477:head, version: 111'477, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c9f7fa8:::obj477:head@111'477, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.062+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 412 ==== pg_backfill(progress 4.0 e 119/119 lb 4:4c7e2cfa:::obj454:head) ==== 1062+0+0 (unknown 0 0 0) 0x6179180 con 0x507ac00 2026-02-11T23:31:28.062+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.062+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:4c7e2cfa:::obj454:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 119 2026-02-11T23:31:28.062+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4c9f7fa8:::obj477:head, version: 111'477, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c9f7fa8:::obj477:head@111'477, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.062+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4c7e2cfa:::obj454:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.062+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.062+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:4c9f7fa8:::obj477:head, version: 111'477, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c9f7fa8:::obj477:head@111'477, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(4.0 119/117 [PushOp(4:4c9f7fa8:::obj477:head, version: 111'477, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c9f7fa8:::obj477:head@111'477, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:4c73ba95:::obj68:head local-lis/les=117/118 n=185 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.062+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4c7e2cfa:::obj454:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.062+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.062+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c73ba95:::obj68:head local-lis/les=117/118 n=185 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:4c9f7fa8:::obj477:head, version: 111'477, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c9f7fa8:::obj477:head@111'477, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.062+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c73ba95:::obj68:head local-lis/les=117/118 n=185 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:4c9f7fa8:::obj477:head@111'477, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.062+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c73ba95:::obj68:head local-lis/les=117/118 n=185 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:4c9f7fa8:::obj477:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.062+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c73ba95:::obj68:head local-lis/les=117/118 n=185 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:4c9f7fa8:::obj477:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.062+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c73ba95:::obj68:head local-lis/les=117/118 n=185 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:4c9f7fa8:::obj477:head 2026-02-11T23:31:28.062+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c73ba95:::obj68:head local-lis/les=117/118 n=185 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:4c9f7fa8:::obj477:head v 111'477 2026-02-11T23:31:28.062+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c73ba95:::obj68:head local-lis/les=117/118 n=185 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.062+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.062+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:4c9f7fa8:::obj477:head, version: 111'477, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c9f7fa8:::obj477:head@111'477, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:28.062+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.062+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.062+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4c7e2cfa:::obj454:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.062+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:4c7e2cfa:::obj454:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 4.0 e 119/119 lb 4:4c7e2cfa:::obj454:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:4c73ba95:::obj68:head local-lis/les=117/118 n=185 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.062+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.062+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c73ba95:::obj68:head local-lis/les=117/118 n=185 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:4c7e2cfa:::obj454:head) v3 2026-02-11T23:31:28.063+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c73ba95:::obj68:head local-lis/les=117/118 n=185 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:4c7e2cfa:::obj454:head) 2026-02-11T23:31:28.063+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.063+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:4c7e2cfa:::obj454:head) v3 finish latency 0.000018 2026-02-11T23:31:28.063+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c7e2cfa:::obj454:head local-lis/les=117/118 n=186 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.063+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.063+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.063+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c7e2cfa:::obj454:head local-lis/les=117/118 n=186 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.063+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.063+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.063+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.063+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7a000 con 0x507ac00 2026-02-11T23:31:28.063+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:4c9f7fa8:::obj477:head)]) -- 0x63c4900 con 0x507ac00 2026-02-11T23:31:28.063+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.063+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 413 ==== MOSDPGPush(4.0 119/117 [PushOp(4:4dc3027d:::obj550:head, version: 111'550, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4dc3027d:::obj550:head@111'550, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c000 con 0x507ac00 2026-02-11T23:31:28.063+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:4dc3027d:::obj550:head, version: 111'550, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4dc3027d:::obj550:head@111'550, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 119 2026-02-11T23:31:28.063+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4dc3027d:::obj550:head, version: 111'550, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4dc3027d:::obj550:head@111'550, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.063+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.063+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 414 ==== pg_backfill(progress 4.0 e 119/119 lb 4:4c9f7fa8:::obj477:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98b00 con 0x507ac00 2026-02-11T23:31:28.063+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4dc3027d:::obj550:head, version: 111'550, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4dc3027d:::obj550:head@111'550, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.063+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:4c9f7fa8:::obj477:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.063+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4c9f7fa8:::obj477:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.063+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.063+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4dc3027d:::obj550:head, version: 111'550, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4dc3027d:::obj550:head@111'550, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.063+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:4dc3027d:::obj550:head, version: 111'550, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4dc3027d:::obj550:head@111'550, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(4.0 119/117 [PushOp(4:4dc3027d:::obj550:head, version: 111'550, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4dc3027d:::obj550:head@111'550, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:4c7e2cfa:::obj454:head local-lis/les=117/118 n=186 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.063+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.063+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.063+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4c9f7fa8:::obj477:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.063+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c7e2cfa:::obj454:head local-lis/les=117/118 n=186 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:4dc3027d:::obj550:head, version: 111'550, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4dc3027d:::obj550:head@111'550, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.063+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c7e2cfa:::obj454:head local-lis/les=117/118 n=186 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:4dc3027d:::obj550:head@111'550, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.063+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c7e2cfa:::obj454:head local-lis/les=117/118 n=186 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:4dc3027d:::obj550:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.063+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c7e2cfa:::obj454:head local-lis/les=117/118 n=186 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:4dc3027d:::obj550:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.063+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c7e2cfa:::obj454:head local-lis/les=117/118 n=186 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:4dc3027d:::obj550:head 2026-02-11T23:31:28.063+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c7e2cfa:::obj454:head local-lis/les=117/118 n=186 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:4dc3027d:::obj550:head v 111'550 2026-02-11T23:31:28.063+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c7e2cfa:::obj454:head local-lis/les=117/118 n=186 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.063+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.063+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:4dc3027d:::obj550:head, version: 111'550, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4dc3027d:::obj550:head@111'550, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:28.063+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.063+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.063+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4c9f7fa8:::obj477:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.063+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:4c9f7fa8:::obj477:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 4.0 e 119/119 lb 4:4c9f7fa8:::obj477:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:4c7e2cfa:::obj454:head local-lis/les=117/118 n=186 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.063+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.063+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c7e2cfa:::obj454:head local-lis/les=117/118 n=186 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:4c9f7fa8:::obj477:head) v3 2026-02-11T23:31:28.063+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c7e2cfa:::obj454:head local-lis/les=117/118 n=186 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:4c9f7fa8:::obj477:head) 2026-02-11T23:31:28.063+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.063+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:4c9f7fa8:::obj477:head) v3 finish latency 0.000015 2026-02-11T23:31:28.063+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c9f7fa8:::obj477:head local-lis/les=117/118 n=187 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.063+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.063+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.063+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c9f7fa8:::obj477:head local-lis/les=117/118 n=187 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.063+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.063+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.063+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.063+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7b680 con 0x507ac00 2026-02-11T23:31:28.063+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:4dc3027d:::obj550:head)]) -- 0x5f75b00 con 0x507ac00 2026-02-11T23:31:28.063+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.064+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 415 ==== MOSDPGPush(4.0 119/117 [PushOp(4:4dd77094:::obj38:head, version: 111'38, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4dd77094:::obj38:head@111'38, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x56a2820 con 0x507ac00 2026-02-11T23:31:28.064+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:4dd77094:::obj38:head, version: 111'38, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4dd77094:::obj38:head@111'38, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 119 2026-02-11T23:31:28.064+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4dd77094:::obj38:head, version: 111'38, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4dd77094:::obj38:head@111'38, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.064+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.064+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 416 ==== pg_backfill(progress 4.0 e 119/119 lb 4:4dc3027d:::obj550:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9b180 con 0x507ac00 2026-02-11T23:31:28.064+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4dd77094:::obj38:head, version: 111'38, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4dd77094:::obj38:head@111'38, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.064+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:4dc3027d:::obj550:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.064+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4dc3027d:::obj550:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.064+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.064+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4dd77094:::obj38:head, version: 111'38, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4dd77094:::obj38:head@111'38, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.064+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:4dd77094:::obj38:head, version: 111'38, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4dd77094:::obj38:head@111'38, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(4.0 119/117 [PushOp(4:4dd77094:::obj38:head, version: 111'38, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4dd77094:::obj38:head@111'38, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:4c9f7fa8:::obj477:head local-lis/les=117/118 n=187 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.064+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.064+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.064+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4dc3027d:::obj550:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.064+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c9f7fa8:::obj477:head local-lis/les=117/118 n=187 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:4dd77094:::obj38:head, version: 111'38, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4dd77094:::obj38:head@111'38, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.064+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c9f7fa8:::obj477:head local-lis/les=117/118 n=187 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:4dd77094:::obj38:head@111'38, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.064+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c9f7fa8:::obj477:head local-lis/les=117/118 n=187 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:4dd77094:::obj38:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.064+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c9f7fa8:::obj477:head local-lis/les=117/118 n=187 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:4dd77094:::obj38:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.064+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c9f7fa8:::obj477:head local-lis/les=117/118 n=187 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:4dd77094:::obj38:head 2026-02-11T23:31:28.064+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c9f7fa8:::obj477:head local-lis/les=117/118 n=187 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:4dd77094:::obj38:head v 111'38 2026-02-11T23:31:28.064+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c9f7fa8:::obj477:head local-lis/les=117/118 n=187 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.064+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.064+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:4dd77094:::obj38:head, version: 111'38, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4dd77094:::obj38:head@111'38, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:28.064+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.064+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.064+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4dc3027d:::obj550:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.064+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:4dc3027d:::obj550:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 4.0 e 119/119 lb 4:4dc3027d:::obj550:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:4c9f7fa8:::obj477:head local-lis/les=117/118 n=187 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.064+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.064+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c9f7fa8:::obj477:head local-lis/les=117/118 n=187 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:4dc3027d:::obj550:head) v3 2026-02-11T23:31:28.064+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4c9f7fa8:::obj477:head local-lis/les=117/118 n=187 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:4dc3027d:::obj550:head) 2026-02-11T23:31:28.064+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.064+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:4dc3027d:::obj550:head) v3 finish latency 0.000013 2026-02-11T23:31:28.064+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4dc3027d:::obj550:head local-lis/les=117/118 n=188 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.064+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.064+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.064+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4dc3027d:::obj550:head local-lis/les=117/118 n=188 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.064+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.064+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.064+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.064+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c5b00 con 0x507ac00 2026-02-11T23:31:28.064+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:4dd77094:::obj38:head)]) -- 0x63c4a80 con 0x507ac00 2026-02-11T23:31:28.064+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.064+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 417 ==== MOSDPGPush(4.0 119/117 [PushOp(4:4eb30133:::obj30:head, version: 111'30, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4eb30133:::obj30:head@111'30, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x56a3ba0 con 0x507ac00 2026-02-11T23:31:28.064+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:4eb30133:::obj30:head, version: 111'30, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4eb30133:::obj30:head@111'30, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 119 2026-02-11T23:31:28.064+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4eb30133:::obj30:head, version: 111'30, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4eb30133:::obj30:head@111'30, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.064+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.064+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 418 ==== pg_backfill(progress 4.0 e 119/119 lb 4:4dd77094:::obj38:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d98000 con 0x507ac00 2026-02-11T23:31:28.064+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4eb30133:::obj30:head, version: 111'30, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4eb30133:::obj30:head@111'30, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.064+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:4dd77094:::obj38:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.064+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.064+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4eb30133:::obj30:head, version: 111'30, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4eb30133:::obj30:head@111'30, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.064+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4dd77094:::obj38:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.064+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.064+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4dd77094:::obj38:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.064+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:4eb30133:::obj30:head, version: 111'30, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4eb30133:::obj30:head@111'30, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(4.0 119/117 [PushOp(4:4eb30133:::obj30:head, version: 111'30, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4eb30133:::obj30:head@111'30, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:4dc3027d:::obj550:head local-lis/les=117/118 n=188 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.064+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.064+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4dc3027d:::obj550:head local-lis/les=117/118 n=188 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:4eb30133:::obj30:head, version: 111'30, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4eb30133:::obj30:head@111'30, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.064+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4dc3027d:::obj550:head local-lis/les=117/118 n=188 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:4eb30133:::obj30:head@111'30, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.064+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4dc3027d:::obj550:head local-lis/les=117/118 n=188 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:4eb30133:::obj30:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.064+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4dc3027d:::obj550:head local-lis/les=117/118 n=188 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:4eb30133:::obj30:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.064+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4dc3027d:::obj550:head local-lis/les=117/118 n=188 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:4eb30133:::obj30:head 2026-02-11T23:31:28.064+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4dc3027d:::obj550:head local-lis/les=117/118 n=188 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:4eb30133:::obj30:head v 111'30 2026-02-11T23:31:28.064+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4dc3027d:::obj550:head local-lis/les=117/118 n=188 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.064+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.064+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:4eb30133:::obj30:head, version: 111'30, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4eb30133:::obj30:head@111'30, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:28.064+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.064+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.064+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4dd77094:::obj38:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.064+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:4dd77094:::obj38:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 119/119 lb 4:4dd77094:::obj38:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:4dc3027d:::obj550:head local-lis/les=117/118 n=188 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.064+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.064+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4dc3027d:::obj550:head local-lis/les=117/118 n=188 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:4dd77094:::obj38:head) v3 2026-02-11T23:31:28.064+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4dc3027d:::obj550:head local-lis/les=117/118 n=188 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:4dd77094:::obj38:head) 2026-02-11T23:31:28.064+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.064+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:4dd77094:::obj38:head) v3 finish latency 0.000014 2026-02-11T23:31:28.064+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4dd77094:::obj38:head local-lis/les=117/118 n=189 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.064+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.065+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.065+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4dd77094:::obj38:head local-lis/les=117/118 n=189 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.065+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.065+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.065+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.065+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7b500 con 0x507ac00 2026-02-11T23:31:28.065+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:4eb30133:::obj30:head)]) -- 0x4f43200 con 0x507ac00 2026-02-11T23:31:28.065+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.065+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 419 ==== MOSDPGPush(4.0 119/117 [PushOp(4:4fbee9e6:::obj123:head, version: 111'123, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4fbee9e6:::obj123:head@111'123, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db0b60 con 0x507ac00 2026-02-11T23:31:28.065+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:4fbee9e6:::obj123:head, version: 111'123, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4fbee9e6:::obj123:head@111'123, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 119 2026-02-11T23:31:28.065+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4fbee9e6:::obj123:head, version: 111'123, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4fbee9e6:::obj123:head@111'123, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.065+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.065+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 420 ==== pg_backfill(progress 4.0 e 119/119 lb 4:4eb30133:::obj30:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d98580 con 0x507ac00 2026-02-11T23:31:28.065+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4fbee9e6:::obj123:head, version: 111'123, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4fbee9e6:::obj123:head@111'123, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.065+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:4eb30133:::obj30:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.065+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.065+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4eb30133:::obj30:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.065+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4fbee9e6:::obj123:head, version: 111'123, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4fbee9e6:::obj123:head@111'123, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.065+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.065+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:4fbee9e6:::obj123:head, version: 111'123, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4fbee9e6:::obj123:head@111'123, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(4.0 119/117 [PushOp(4:4fbee9e6:::obj123:head, version: 111'123, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4fbee9e6:::obj123:head@111'123, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:4dd77094:::obj38:head local-lis/les=117/118 n=189 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.065+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.065+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4dd77094:::obj38:head local-lis/les=117/118 n=189 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:4fbee9e6:::obj123:head, version: 111'123, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4fbee9e6:::obj123:head@111'123, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.065+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.065+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4dd77094:::obj38:head local-lis/les=117/118 n=189 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:4fbee9e6:::obj123:head@111'123, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.065+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4eb30133:::obj30:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.065+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4dd77094:::obj38:head local-lis/les=117/118 n=189 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:4fbee9e6:::obj123:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.065+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4dd77094:::obj38:head local-lis/les=117/118 n=189 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:4fbee9e6:::obj123:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.065+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4dd77094:::obj38:head local-lis/les=117/118 n=189 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:4fbee9e6:::obj123:head 2026-02-11T23:31:28.065+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4dd77094:::obj38:head local-lis/les=117/118 n=189 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:4fbee9e6:::obj123:head v 111'123 2026-02-11T23:31:28.065+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4dd77094:::obj38:head local-lis/les=117/118 n=189 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.065+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.065+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:4fbee9e6:::obj123:head, version: 111'123, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4fbee9e6:::obj123:head@111'123, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:28.065+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.065+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.065+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4eb30133:::obj30:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.065+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:4eb30133:::obj30:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 4.0 e 119/119 lb 4:4eb30133:::obj30:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:4dd77094:::obj38:head local-lis/les=117/118 n=189 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.065+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.065+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4dd77094:::obj38:head local-lis/les=117/118 n=189 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:4eb30133:::obj30:head) v3 2026-02-11T23:31:28.065+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4dd77094:::obj38:head local-lis/les=117/118 n=189 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:4eb30133:::obj30:head) 2026-02-11T23:31:28.065+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.065+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:4eb30133:::obj30:head) v3 finish latency 0.000013 2026-02-11T23:31:28.065+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4eb30133:::obj30:head local-lis/les=117/118 n=190 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.065+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.065+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.065+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4eb30133:::obj30:head local-lis/les=117/118 n=190 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.065+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.065+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.065+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.065+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c5c80 con 0x507ac00 2026-02-11T23:31:28.065+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:4fbee9e6:::obj123:head)]) -- 0x649ef00 con 0x507ac00 2026-02-11T23:31:28.065+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.066+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 421 ==== MOSDPGPush(4.0 119/117 [PushOp(4:4feb0eaf:::obj556:head, version: 111'556, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4feb0eaf:::obj556:head@111'556, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2680 con 0x507ac00 2026-02-11T23:31:28.066+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:4feb0eaf:::obj556:head, version: 111'556, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4feb0eaf:::obj556:head@111'556, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 119 2026-02-11T23:31:28.066+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4feb0eaf:::obj556:head, version: 111'556, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4feb0eaf:::obj556:head@111'556, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.066+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.066+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 422 ==== pg_backfill(progress 4.0 e 119/119 lb 4:4fbee9e6:::obj123:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2d600 con 0x507ac00 2026-02-11T23:31:28.066+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4feb0eaf:::obj556:head, version: 111'556, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4feb0eaf:::obj556:head@111'556, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.066+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:4fbee9e6:::obj123:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.066+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4fbee9e6:::obj123:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.066+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.066+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:4feb0eaf:::obj556:head, version: 111'556, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4feb0eaf:::obj556:head@111'556, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.066+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:4feb0eaf:::obj556:head, version: 111'556, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4feb0eaf:::obj556:head@111'556, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(4.0 119/117 [PushOp(4:4feb0eaf:::obj556:head, version: 111'556, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4feb0eaf:::obj556:head@111'556, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:4eb30133:::obj30:head local-lis/les=117/118 n=190 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.066+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.066+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.066+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4fbee9e6:::obj123:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.066+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4eb30133:::obj30:head local-lis/les=117/118 n=190 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:4feb0eaf:::obj556:head, version: 111'556, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4feb0eaf:::obj556:head@111'556, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.066+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4eb30133:::obj30:head local-lis/les=117/118 n=190 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:4feb0eaf:::obj556:head@111'556, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.066+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4eb30133:::obj30:head local-lis/les=117/118 n=190 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:4feb0eaf:::obj556:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.066+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4eb30133:::obj30:head local-lis/les=117/118 n=190 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:4feb0eaf:::obj556:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.066+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4eb30133:::obj30:head local-lis/les=117/118 n=190 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:4feb0eaf:::obj556:head 2026-02-11T23:31:28.066+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4eb30133:::obj30:head local-lis/les=117/118 n=190 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:4feb0eaf:::obj556:head v 111'556 2026-02-11T23:31:28.066+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4eb30133:::obj30:head local-lis/les=117/118 n=190 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.066+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.066+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:4feb0eaf:::obj556:head, version: 111'556, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4feb0eaf:::obj556:head@111'556, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:28.066+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.066+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.066+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4fbee9e6:::obj123:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.066+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:4fbee9e6:::obj123:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 4.0 e 119/119 lb 4:4fbee9e6:::obj123:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:4eb30133:::obj30:head local-lis/les=117/118 n=190 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.066+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.066+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4eb30133:::obj30:head local-lis/les=117/118 n=190 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:4fbee9e6:::obj123:head) v3 2026-02-11T23:31:28.066+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4eb30133:::obj30:head local-lis/les=117/118 n=190 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:4fbee9e6:::obj123:head) 2026-02-11T23:31:28.066+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.066+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:4fbee9e6:::obj123:head) v3 finish latency 0.000015 2026-02-11T23:31:28.066+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4fbee9e6:::obj123:head local-lis/les=117/118 n=191 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.066+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.066+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.066+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4fbee9e6:::obj123:head local-lis/les=117/118 n=191 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.066+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.066+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.066+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.066+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7b380 con 0x507ac00 2026-02-11T23:31:28.066+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:4feb0eaf:::obj556:head)]) -- 0x63c4c00 con 0x507ac00 2026-02-11T23:31:28.066+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.066+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 423 ==== MOSDPGPush(4.0 119/117 [PushOp(4:50174016:::obj321:head, version: 111'321, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:50174016:::obj321:head@111'321, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56cab60 con 0x507ac00 2026-02-11T23:31:28.066+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:50174016:::obj321:head, version: 111'321, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:50174016:::obj321:head@111'321, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 119 2026-02-11T23:31:28.066+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:50174016:::obj321:head, version: 111'321, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:50174016:::obj321:head@111'321, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.066+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.066+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 424 ==== pg_backfill(progress 4.0 e 119/119 lb 4:4feb0eaf:::obj556:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176b00 con 0x507ac00 2026-02-11T23:31:28.066+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:50174016:::obj321:head, version: 111'321, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:50174016:::obj321:head@111'321, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.066+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:4feb0eaf:::obj556:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.066+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.066+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4feb0eaf:::obj556:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.066+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:50174016:::obj321:head, version: 111'321, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:50174016:::obj321:head@111'321, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.066+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.066+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:50174016:::obj321:head, version: 111'321, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:50174016:::obj321:head@111'321, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(4.0 119/117 [PushOp(4:50174016:::obj321:head, version: 111'321, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:50174016:::obj321:head@111'321, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:4fbee9e6:::obj123:head local-lis/les=117/118 n=191 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.066+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.066+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4fbee9e6:::obj123:head local-lis/les=117/118 n=191 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:50174016:::obj321:head, version: 111'321, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:50174016:::obj321:head@111'321, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.066+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4fbee9e6:::obj123:head local-lis/les=117/118 n=191 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:50174016:::obj321:head@111'321, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.066+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.066+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4feb0eaf:::obj556:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.066+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4fbee9e6:::obj123:head local-lis/les=117/118 n=191 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:50174016:::obj321:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.066+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4fbee9e6:::obj123:head local-lis/les=117/118 n=191 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:50174016:::obj321:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.066+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4fbee9e6:::obj123:head local-lis/les=117/118 n=191 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:50174016:::obj321:head 2026-02-11T23:31:28.066+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4fbee9e6:::obj123:head local-lis/les=117/118 n=191 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:50174016:::obj321:head v 111'321 2026-02-11T23:31:28.066+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4fbee9e6:::obj123:head local-lis/les=117/118 n=191 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.066+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.066+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:50174016:::obj321:head, version: 111'321, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:50174016:::obj321:head@111'321, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:28.066+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.066+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.066+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:4feb0eaf:::obj556:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.066+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:4feb0eaf:::obj556:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 4.0 e 119/119 lb 4:4feb0eaf:::obj556:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:4fbee9e6:::obj123:head local-lis/les=117/118 n=191 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.066+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.066+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4fbee9e6:::obj123:head local-lis/les=117/118 n=191 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:4feb0eaf:::obj556:head) v3 2026-02-11T23:31:28.066+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4fbee9e6:::obj123:head local-lis/les=117/118 n=191 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:4feb0eaf:::obj556:head) 2026-02-11T23:31:28.066+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.066+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:4feb0eaf:::obj556:head) v3 finish latency 0.000013 2026-02-11T23:31:28.066+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4feb0eaf:::obj556:head local-lis/les=117/118 n=192 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.066+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.067+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.067+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4feb0eaf:::obj556:head local-lis/les=117/118 n=192 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.067+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.067+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.067+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.067+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c9980 con 0x507ac00 2026-02-11T23:31:28.067+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:50174016:::obj321:head)]) -- 0x649ec00 con 0x507ac00 2026-02-11T23:31:28.067+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.067+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 425 ==== MOSDPGPush(4.0 119/117 [PushOp(4:502bc4ef:::obj531:head, version: 111'531, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:502bc4ef:::obj531:head@111'531, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f0d00 con 0x507ac00 2026-02-11T23:31:28.067+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:502bc4ef:::obj531:head, version: 111'531, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:502bc4ef:::obj531:head@111'531, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 119 2026-02-11T23:31:28.067+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:502bc4ef:::obj531:head, version: 111'531, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:502bc4ef:::obj531:head@111'531, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.067+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.067+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:502bc4ef:::obj531:head, version: 111'531, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:502bc4ef:::obj531:head@111'531, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.067+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 426 ==== pg_backfill(progress 4.0 e 119/119 lb 4:50174016:::obj321:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99080 con 0x507ac00 2026-02-11T23:31:28.067+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.067+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:50174016:::obj321:head) v3 prio 15 type 95 cost 0 latency 0.000010 epoch 119 2026-02-11T23:31:28.067+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:502bc4ef:::obj531:head, version: 111'531, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:502bc4ef:::obj531:head@111'531, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.067+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:50174016:::obj321:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.067+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.067+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:502bc4ef:::obj531:head, version: 111'531, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:502bc4ef:::obj531:head@111'531, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(4.0 119/117 [PushOp(4:502bc4ef:::obj531:head, version: 111'531, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:502bc4ef:::obj531:head@111'531, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:4feb0eaf:::obj556:head local-lis/les=117/118 n=192 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.067+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:50174016:::obj321:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.067+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.067+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4feb0eaf:::obj556:head local-lis/les=117/118 n=192 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:502bc4ef:::obj531:head, version: 111'531, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:502bc4ef:::obj531:head@111'531, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.067+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4feb0eaf:::obj556:head local-lis/les=117/118 n=192 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:502bc4ef:::obj531:head@111'531, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.067+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4feb0eaf:::obj556:head local-lis/les=117/118 n=192 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:502bc4ef:::obj531:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.067+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4feb0eaf:::obj556:head local-lis/les=117/118 n=192 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:502bc4ef:::obj531:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.067+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4feb0eaf:::obj556:head local-lis/les=117/118 n=192 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:502bc4ef:::obj531:head 2026-02-11T23:31:28.067+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4feb0eaf:::obj556:head local-lis/les=117/118 n=192 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:502bc4ef:::obj531:head v 111'531 2026-02-11T23:31:28.067+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4feb0eaf:::obj556:head local-lis/les=117/118 n=192 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.067+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.067+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:502bc4ef:::obj531:head, version: 111'531, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:502bc4ef:::obj531:head@111'531, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:28.067+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.067+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.067+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:50174016:::obj321:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.067+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:50174016:::obj321:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 4.0 e 119/119 lb 4:50174016:::obj321:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:4feb0eaf:::obj556:head local-lis/les=117/118 n=192 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.067+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.067+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4feb0eaf:::obj556:head local-lis/les=117/118 n=192 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:50174016:::obj321:head) v3 2026-02-11T23:31:28.067+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:4feb0eaf:::obj556:head local-lis/les=117/118 n=192 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:50174016:::obj321:head) 2026-02-11T23:31:28.067+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.067+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:50174016:::obj321:head) v3 finish latency 0.000014 2026-02-11T23:31:28.067+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:50174016:::obj321:head local-lis/les=117/118 n=193 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.067+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.067+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.067+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:50174016:::obj321:head local-lis/les=117/118 n=193 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.067+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.067+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.067+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.067+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7b200 con 0x507ac00 2026-02-11T23:31:28.067+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:502bc4ef:::obj531:head)]) -- 0x63c4d80 con 0x507ac00 2026-02-11T23:31:28.067+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.068+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 427 ==== MOSDPGPush(4.0 119/117 [PushOp(4:506f5ba0:::obj253:head, version: 111'253, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:506f5ba0:::obj253:head@111'253, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c4e0 con 0x507ac00 2026-02-11T23:31:28.068+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:506f5ba0:::obj253:head, version: 111'253, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:506f5ba0:::obj253:head@111'253, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 119 2026-02-11T23:31:28.068+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:506f5ba0:::obj253:head, version: 111'253, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:506f5ba0:::obj253:head@111'253, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.068+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.068+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 428 ==== pg_backfill(progress 4.0 e 119/119 lb 4:502bc4ef:::obj531:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99600 con 0x507ac00 2026-02-11T23:31:28.068+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:506f5ba0:::obj253:head, version: 111'253, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:506f5ba0:::obj253:head@111'253, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.068+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:502bc4ef:::obj531:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.068+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.068+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:506f5ba0:::obj253:head, version: 111'253, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:506f5ba0:::obj253:head@111'253, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.068+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:502bc4ef:::obj531:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.068+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.068+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:506f5ba0:::obj253:head, version: 111'253, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:506f5ba0:::obj253:head@111'253, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000040 MOSDPGPush(4.0 119/117 [PushOp(4:506f5ba0:::obj253:head, version: 111'253, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:506f5ba0:::obj253:head@111'253, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:50174016:::obj321:head local-lis/les=117/118 n=193 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.068+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:502bc4ef:::obj531:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.068+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.068+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:50174016:::obj321:head local-lis/les=117/118 n=193 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:506f5ba0:::obj253:head, version: 111'253, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:506f5ba0:::obj253:head@111'253, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.068+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:50174016:::obj321:head local-lis/les=117/118 n=193 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:506f5ba0:::obj253:head@111'253, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.068+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:50174016:::obj321:head local-lis/les=117/118 n=193 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:506f5ba0:::obj253:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.068+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:50174016:::obj321:head local-lis/les=117/118 n=193 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:506f5ba0:::obj253:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.068+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:50174016:::obj321:head local-lis/les=117/118 n=193 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:506f5ba0:::obj253:head 2026-02-11T23:31:28.068+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:50174016:::obj321:head local-lis/les=117/118 n=193 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:506f5ba0:::obj253:head v 111'253 2026-02-11T23:31:28.068+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:50174016:::obj321:head local-lis/les=117/118 n=193 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.068+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.068+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:506f5ba0:::obj253:head, version: 111'253, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:506f5ba0:::obj253:head@111'253, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:28.068+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.068+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.068+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:502bc4ef:::obj531:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.068+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:502bc4ef:::obj531:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 4.0 e 119/119 lb 4:502bc4ef:::obj531:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:50174016:::obj321:head local-lis/les=117/118 n=193 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.068+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.068+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:50174016:::obj321:head local-lis/les=117/118 n=193 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:502bc4ef:::obj531:head) v3 2026-02-11T23:31:28.068+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:50174016:::obj321:head local-lis/les=117/118 n=193 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:502bc4ef:::obj531:head) 2026-02-11T23:31:28.068+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.068+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:502bc4ef:::obj531:head) v3 finish latency 0.000015 2026-02-11T23:31:28.068+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:502bc4ef:::obj531:head local-lis/les=117/118 n=194 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.068+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.068+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.068+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:502bc4ef:::obj531:head local-lis/les=117/118 n=194 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.068+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.068+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.068+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.068+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c9800 con 0x507ac00 2026-02-11T23:31:28.068+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:506f5ba0:::obj253:head)]) -- 0x649f080 con 0x507ac00 2026-02-11T23:31:28.068+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.068+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 429 ==== MOSDPGPush(4.0 119/117 [PushOp(4:50fb58bc:::obj174:head, version: 111'174, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:50fb58bc:::obj174:head@111'174, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f0b60 con 0x507ac00 2026-02-11T23:31:28.068+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:50fb58bc:::obj174:head, version: 111'174, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:50fb58bc:::obj174:head@111'174, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 119 2026-02-11T23:31:28.068+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:50fb58bc:::obj174:head, version: 111'174, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:50fb58bc:::obj174:head@111'174, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.068+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.068+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 430 ==== pg_backfill(progress 4.0 e 119/119 lb 4:506f5ba0:::obj253:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9ac00 con 0x507ac00 2026-02-11T23:31:28.068+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:50fb58bc:::obj174:head, version: 111'174, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:50fb58bc:::obj174:head@111'174, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.068+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:506f5ba0:::obj253:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.068+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:506f5ba0:::obj253:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.068+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.068+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:50fb58bc:::obj174:head, version: 111'174, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:50fb58bc:::obj174:head@111'174, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.068+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.068+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:50fb58bc:::obj174:head, version: 111'174, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:50fb58bc:::obj174:head@111'174, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(4.0 119/117 [PushOp(4:50fb58bc:::obj174:head, version: 111'174, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:50fb58bc:::obj174:head@111'174, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:502bc4ef:::obj531:head local-lis/les=117/118 n=194 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.068+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.068+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:502bc4ef:::obj531:head local-lis/les=117/118 n=194 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:50fb58bc:::obj174:head, version: 111'174, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:50fb58bc:::obj174:head@111'174, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.068+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:502bc4ef:::obj531:head local-lis/les=117/118 n=194 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:50fb58bc:::obj174:head@111'174, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.068+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.068+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:506f5ba0:::obj253:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.068+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:502bc4ef:::obj531:head local-lis/les=117/118 n=194 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:50fb58bc:::obj174:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.068+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:502bc4ef:::obj531:head local-lis/les=117/118 n=194 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:50fb58bc:::obj174:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.068+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:502bc4ef:::obj531:head local-lis/les=117/118 n=194 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:50fb58bc:::obj174:head 2026-02-11T23:31:28.068+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:502bc4ef:::obj531:head local-lis/les=117/118 n=194 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:50fb58bc:::obj174:head v 111'174 2026-02-11T23:31:28.068+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:502bc4ef:::obj531:head local-lis/les=117/118 n=194 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.068+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.068+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:50fb58bc:::obj174:head, version: 111'174, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:50fb58bc:::obj174:head@111'174, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:28.068+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.068+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.068+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:506f5ba0:::obj253:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.068+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:506f5ba0:::obj253:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 4.0 e 119/119 lb 4:506f5ba0:::obj253:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:502bc4ef:::obj531:head local-lis/les=117/118 n=194 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.068+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.068+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:502bc4ef:::obj531:head local-lis/les=117/118 n=194 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:506f5ba0:::obj253:head) v3 2026-02-11T23:31:28.068+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:502bc4ef:::obj531:head local-lis/les=117/118 n=194 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:506f5ba0:::obj253:head) 2026-02-11T23:31:28.068+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.068+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:506f5ba0:::obj253:head) v3 finish latency 0.000014 2026-02-11T23:31:28.068+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:506f5ba0:::obj253:head local-lis/les=117/118 n=195 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.068+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.069+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.069+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:506f5ba0:::obj253:head local-lis/les=117/118 n=195 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.069+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.069+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.069+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.069+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7b080 con 0x507ac00 2026-02-11T23:31:28.069+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:50fb58bc:::obj174:head)]) -- 0x63c4f00 con 0x507ac00 2026-02-11T23:31:28.069+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.069+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 431 ==== MOSDPGPush(4.0 119/117 [PushOp(4:51058f37:::obj394:head, version: 111'394, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:51058f37:::obj394:head@111'394, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559cb60 con 0x507ac00 2026-02-11T23:31:28.069+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:51058f37:::obj394:head, version: 111'394, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:51058f37:::obj394:head@111'394, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 119 2026-02-11T23:31:28.069+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:51058f37:::obj394:head, version: 111'394, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:51058f37:::obj394:head@111'394, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.069+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.069+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 432 ==== pg_backfill(progress 4.0 e 119/119 lb 4:50fb58bc:::obj174:head) ==== 1062+0+0 (unknown 0 0 0) 0x4a2cb00 con 0x507ac00 2026-02-11T23:31:28.069+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:51058f37:::obj394:head, version: 111'394, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:51058f37:::obj394:head@111'394, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.069+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.069+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:50fb58bc:::obj174:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 119 2026-02-11T23:31:28.069+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:51058f37:::obj394:head, version: 111'394, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:51058f37:::obj394:head@111'394, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.069+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:50fb58bc:::obj174:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.069+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.069+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:50fb58bc:::obj174:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.069+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:51058f37:::obj394:head, version: 111'394, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:51058f37:::obj394:head@111'394, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000040 MOSDPGPush(4.0 119/117 [PushOp(4:51058f37:::obj394:head, version: 111'394, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:51058f37:::obj394:head@111'394, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:506f5ba0:::obj253:head local-lis/les=117/118 n=195 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.069+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.069+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:506f5ba0:::obj253:head local-lis/les=117/118 n=195 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:51058f37:::obj394:head, version: 111'394, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:51058f37:::obj394:head@111'394, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.069+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:506f5ba0:::obj253:head local-lis/les=117/118 n=195 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:51058f37:::obj394:head@111'394, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.069+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:506f5ba0:::obj253:head local-lis/les=117/118 n=195 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:51058f37:::obj394:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.069+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:506f5ba0:::obj253:head local-lis/les=117/118 n=195 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:51058f37:::obj394:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.069+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:506f5ba0:::obj253:head local-lis/les=117/118 n=195 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:51058f37:::obj394:head 2026-02-11T23:31:28.069+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:506f5ba0:::obj253:head local-lis/les=117/118 n=195 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:51058f37:::obj394:head v 111'394 2026-02-11T23:31:28.069+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:506f5ba0:::obj253:head local-lis/les=117/118 n=195 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.069+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.069+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:51058f37:::obj394:head, version: 111'394, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:51058f37:::obj394:head@111'394, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:28.069+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.069+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.069+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:50fb58bc:::obj174:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.069+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:50fb58bc:::obj174:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 4.0 e 119/119 lb 4:50fb58bc:::obj174:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:506f5ba0:::obj253:head local-lis/les=117/118 n=195 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.069+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.069+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:506f5ba0:::obj253:head local-lis/les=117/118 n=195 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:50fb58bc:::obj174:head) v3 2026-02-11T23:31:28.069+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:506f5ba0:::obj253:head local-lis/les=117/118 n=195 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:50fb58bc:::obj174:head) 2026-02-11T23:31:28.069+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.069+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:50fb58bc:::obj174:head) v3 finish latency 0.000013 2026-02-11T23:31:28.069+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:50fb58bc:::obj174:head local-lis/les=117/118 n=196 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.069+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.069+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.069+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:50fb58bc:::obj174:head local-lis/les=117/118 n=196 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.069+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.069+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.069+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.069+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c9c80 con 0x507ac00 2026-02-11T23:31:28.069+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:51058f37:::obj394:head)]) -- 0x644c180 con 0x507ac00 2026-02-11T23:31:28.069+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.070+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 433 ==== MOSDPGPush(4.0 119/117 [PushOp(4:5170da72:::obj410:head, version: 111'410, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5170da72:::obj410:head@111'410, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e26680 con 0x507ac00 2026-02-11T23:31:28.070+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:5170da72:::obj410:head, version: 111'410, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5170da72:::obj410:head@111'410, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 119 2026-02-11T23:31:28.070+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:5170da72:::obj410:head, version: 111'410, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5170da72:::obj410:head@111'410, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.070+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.070+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 434 ==== pg_backfill(progress 4.0 e 119/119 lb 4:51058f37:::obj394:head) ==== 1062+0+0 (unknown 0 0 0) 0x4a2c000 con 0x507ac00 2026-02-11T23:31:28.070+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:5170da72:::obj410:head, version: 111'410, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5170da72:::obj410:head@111'410, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.070+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:51058f37:::obj394:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.070+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:51058f37:::obj394:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.070+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.070+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:5170da72:::obj410:head, version: 111'410, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5170da72:::obj410:head@111'410, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.070+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:5170da72:::obj410:head, version: 111'410, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5170da72:::obj410:head@111'410, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(4.0 119/117 [PushOp(4:5170da72:::obj410:head, version: 111'410, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5170da72:::obj410:head@111'410, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:50fb58bc:::obj174:head local-lis/les=117/118 n=196 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.070+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.070+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.070+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:51058f37:::obj394:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.070+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:50fb58bc:::obj174:head local-lis/les=117/118 n=196 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:5170da72:::obj410:head, version: 111'410, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5170da72:::obj410:head@111'410, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.070+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:50fb58bc:::obj174:head local-lis/les=117/118 n=196 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:5170da72:::obj410:head@111'410, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.070+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:50fb58bc:::obj174:head local-lis/les=117/118 n=196 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:5170da72:::obj410:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.070+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:50fb58bc:::obj174:head local-lis/les=117/118 n=196 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:5170da72:::obj410:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.070+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:50fb58bc:::obj174:head local-lis/les=117/118 n=196 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:5170da72:::obj410:head 2026-02-11T23:31:28.070+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:50fb58bc:::obj174:head local-lis/les=117/118 n=196 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:5170da72:::obj410:head v 111'410 2026-02-11T23:31:28.070+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:50fb58bc:::obj174:head local-lis/les=117/118 n=196 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.070+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.070+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:5170da72:::obj410:head, version: 111'410, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5170da72:::obj410:head@111'410, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:28.070+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.070+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.070+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:51058f37:::obj394:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.070+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:51058f37:::obj394:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 4.0 e 119/119 lb 4:51058f37:::obj394:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:50fb58bc:::obj174:head local-lis/les=117/118 n=196 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.070+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.070+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:50fb58bc:::obj174:head local-lis/les=117/118 n=196 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:51058f37:::obj394:head) v3 2026-02-11T23:31:28.070+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:50fb58bc:::obj174:head local-lis/les=117/118 n=196 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:51058f37:::obj394:head) 2026-02-11T23:31:28.070+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.070+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:51058f37:::obj394:head) v3 finish latency 0.000014 2026-02-11T23:31:28.070+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:51058f37:::obj394:head local-lis/les=117/118 n=197 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.070+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.070+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.070+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:51058f37:::obj394:head local-lis/les=117/118 n=197 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.070+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.070+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.070+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.070+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7af00 con 0x507ac00 2026-02-11T23:31:28.070+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:5170da72:::obj410:head)]) -- 0x644c480 con 0x507ac00 2026-02-11T23:31:28.070+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.070+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 435 ==== MOSDPGPush(4.0 119/117 [PushOp(4:5178a392:::obj281:head, version: 111'281, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5178a392:::obj281:head@111'281, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5647380 con 0x507ac00 2026-02-11T23:31:28.070+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:5178a392:::obj281:head, version: 111'281, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5178a392:::obj281:head@111'281, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 119 2026-02-11T23:31:28.070+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:5178a392:::obj281:head, version: 111'281, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5178a392:::obj281:head@111'281, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.070+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.070+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 436 ==== pg_backfill(progress 4.0 e 119/119 lb 4:5170da72:::obj410:head) ==== 1062+0+0 (unknown 0 0 0) 0x5982b00 con 0x507ac00 2026-02-11T23:31:28.070+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:5178a392:::obj281:head, version: 111'281, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5178a392:::obj281:head@111'281, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.070+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:5170da72:::obj410:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.070+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.070+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:5170da72:::obj410:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.070+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:5178a392:::obj281:head, version: 111'281, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5178a392:::obj281:head@111'281, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.070+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.070+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:5178a392:::obj281:head, version: 111'281, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5178a392:::obj281:head@111'281, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(4.0 119/117 [PushOp(4:5178a392:::obj281:head, version: 111'281, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5178a392:::obj281:head@111'281, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:51058f37:::obj394:head local-lis/les=117/118 n=197 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.070+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.070+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:51058f37:::obj394:head local-lis/les=117/118 n=197 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:5178a392:::obj281:head, version: 111'281, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5178a392:::obj281:head@111'281, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.070+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:51058f37:::obj394:head local-lis/les=117/118 n=197 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:5178a392:::obj281:head@111'281, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.070+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.070+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:5170da72:::obj410:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.070+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:51058f37:::obj394:head local-lis/les=117/118 n=197 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:5178a392:::obj281:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.070+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:51058f37:::obj394:head local-lis/les=117/118 n=197 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:5178a392:::obj281:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.070+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:51058f37:::obj394:head local-lis/les=117/118 n=197 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:5178a392:::obj281:head 2026-02-11T23:31:28.070+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:51058f37:::obj394:head local-lis/les=117/118 n=197 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:5178a392:::obj281:head v 111'281 2026-02-11T23:31:28.070+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:51058f37:::obj394:head local-lis/les=117/118 n=197 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.070+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.070+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:5178a392:::obj281:head, version: 111'281, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5178a392:::obj281:head@111'281, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:28.070+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.070+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.070+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:5170da72:::obj410:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.070+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:5170da72:::obj410:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 4.0 e 119/119 lb 4:5170da72:::obj410:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:51058f37:::obj394:head local-lis/les=117/118 n=197 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.070+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.070+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:51058f37:::obj394:head local-lis/les=117/118 n=197 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:5170da72:::obj410:head) v3 2026-02-11T23:31:28.070+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:51058f37:::obj394:head local-lis/les=117/118 n=197 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:5170da72:::obj410:head) 2026-02-11T23:31:28.070+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.070+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:5170da72:::obj410:head) v3 finish latency 0.000012 2026-02-11T23:31:28.070+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5170da72:::obj410:head local-lis/les=117/118 n=198 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.070+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.071+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.071+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5170da72:::obj410:head local-lis/les=117/118 n=198 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.071+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.071+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.071+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.071+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c5980 con 0x507ac00 2026-02-11T23:31:28.071+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:5178a392:::obj281:head)]) -- 0x644c780 con 0x507ac00 2026-02-11T23:31:28.071+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.071+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 437 ==== MOSDPGPush(4.0 119/117 [PushOp(4:52d1d495:::obj578:head, version: 111'578, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:52d1d495:::obj578:head@111'578, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2d00 con 0x507ac00 2026-02-11T23:31:28.071+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:52d1d495:::obj578:head, version: 111'578, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:52d1d495:::obj578:head@111'578, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 119 2026-02-11T23:31:28.071+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:52d1d495:::obj578:head, version: 111'578, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:52d1d495:::obj578:head@111'578, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.071+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.071+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 438 ==== pg_backfill(progress 4.0 e 119/119 lb 4:5178a392:::obj281:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99b80 con 0x507ac00 2026-02-11T23:31:28.071+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:52d1d495:::obj578:head, version: 111'578, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:52d1d495:::obj578:head@111'578, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.071+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:5178a392:::obj281:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.071+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:5178a392:::obj281:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.071+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.071+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:52d1d495:::obj578:head, version: 111'578, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:52d1d495:::obj578:head@111'578, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.071+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.071+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:52d1d495:::obj578:head, version: 111'578, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:52d1d495:::obj578:head@111'578, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000045 MOSDPGPush(4.0 119/117 [PushOp(4:52d1d495:::obj578:head, version: 111'578, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:52d1d495:::obj578:head@111'578, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:5170da72:::obj410:head local-lis/les=117/118 n=198 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.071+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.071+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5170da72:::obj410:head local-lis/les=117/118 n=198 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:52d1d495:::obj578:head, version: 111'578, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:52d1d495:::obj578:head@111'578, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.071+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5170da72:::obj410:head local-lis/les=117/118 n=198 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:52d1d495:::obj578:head@111'578, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.071+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.071+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:5178a392:::obj281:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.071+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5170da72:::obj410:head local-lis/les=117/118 n=198 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:52d1d495:::obj578:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.071+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5170da72:::obj410:head local-lis/les=117/118 n=198 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:52d1d495:::obj578:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.071+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5170da72:::obj410:head local-lis/les=117/118 n=198 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:52d1d495:::obj578:head 2026-02-11T23:31:28.071+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5170da72:::obj410:head local-lis/les=117/118 n=198 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:52d1d495:::obj578:head v 111'578 2026-02-11T23:31:28.071+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5170da72:::obj410:head local-lis/les=117/118 n=198 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.071+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.071+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:52d1d495:::obj578:head, version: 111'578, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:52d1d495:::obj578:head@111'578, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:28.071+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.071+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.071+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:5178a392:::obj281:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.071+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:5178a392:::obj281:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 4.0 e 119/119 lb 4:5178a392:::obj281:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:5170da72:::obj410:head local-lis/les=117/118 n=198 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.071+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.071+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5170da72:::obj410:head local-lis/les=117/118 n=198 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:5178a392:::obj281:head) v3 2026-02-11T23:31:28.071+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5170da72:::obj410:head local-lis/les=117/118 n=198 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:5178a392:::obj281:head) 2026-02-11T23:31:28.071+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.071+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:5178a392:::obj281:head) v3 finish latency 0.000014 2026-02-11T23:31:28.071+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5178a392:::obj281:head local-lis/les=117/118 n=199 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.071+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.071+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.071+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5178a392:::obj281:head local-lis/les=117/118 n=199 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.071+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.071+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.071+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.071+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7ad80 con 0x507ac00 2026-02-11T23:31:28.071+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:52d1d495:::obj578:head)]) -- 0x644ca80 con 0x507ac00 2026-02-11T23:31:28.071+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.072+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 439 ==== MOSDPGPush(4.0 119/117 [PushOp(4:52e063dd:::obj399:head, version: 111'399, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:52e063dd:::obj399:head@111'399, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56b64e0 con 0x507ac00 2026-02-11T23:31:28.072+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:52e063dd:::obj399:head, version: 111'399, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:52e063dd:::obj399:head@111'399, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000015 epoch 119 2026-02-11T23:31:28.072+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:52e063dd:::obj399:head, version: 111'399, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:52e063dd:::obj399:head@111'399, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.072+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.072+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 440 ==== pg_backfill(progress 4.0 e 119/119 lb 4:52d1d495:::obj578:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f84000 con 0x507ac00 2026-02-11T23:31:28.072+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:52e063dd:::obj399:head, version: 111'399, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:52e063dd:::obj399:head@111'399, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.072+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:52d1d495:::obj578:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.072+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:52d1d495:::obj578:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.072+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.072+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:52e063dd:::obj399:head, version: 111'399, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:52e063dd:::obj399:head@111'399, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.072+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:52e063dd:::obj399:head, version: 111'399, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:52e063dd:::obj399:head@111'399, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000044 MOSDPGPush(4.0 119/117 [PushOp(4:52e063dd:::obj399:head, version: 111'399, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:52e063dd:::obj399:head@111'399, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:5178a392:::obj281:head local-lis/les=117/118 n=199 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.072+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.072+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.072+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:52d1d495:::obj578:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.072+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5178a392:::obj281:head local-lis/les=117/118 n=199 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:52e063dd:::obj399:head, version: 111'399, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:52e063dd:::obj399:head@111'399, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.072+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5178a392:::obj281:head local-lis/les=117/118 n=199 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:52e063dd:::obj399:head@111'399, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.072+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5178a392:::obj281:head local-lis/les=117/118 n=199 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:52e063dd:::obj399:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.072+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5178a392:::obj281:head local-lis/les=117/118 n=199 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:52e063dd:::obj399:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.072+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5178a392:::obj281:head local-lis/les=117/118 n=199 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:52e063dd:::obj399:head 2026-02-11T23:31:28.072+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5178a392:::obj281:head local-lis/les=117/118 n=199 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:52e063dd:::obj399:head v 111'399 2026-02-11T23:31:28.072+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5178a392:::obj281:head local-lis/les=117/118 n=199 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.072+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.072+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:52e063dd:::obj399:head, version: 111'399, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:52e063dd:::obj399:head@111'399, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:28.072+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.072+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.072+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:52d1d495:::obj578:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.072+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:52d1d495:::obj578:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 4.0 e 119/119 lb 4:52d1d495:::obj578:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:5178a392:::obj281:head local-lis/les=117/118 n=199 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.072+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.072+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5178a392:::obj281:head local-lis/les=117/118 n=199 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:52d1d495:::obj578:head) v3 2026-02-11T23:31:28.072+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5178a392:::obj281:head local-lis/les=117/118 n=199 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:52d1d495:::obj578:head) 2026-02-11T23:31:28.072+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.072+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:52d1d495:::obj578:head) v3 finish latency 0.000014 2026-02-11T23:31:28.072+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:52d1d495:::obj578:head local-lis/les=117/118 n=200 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.072+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.072+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.072+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:52d1d495:::obj578:head local-lis/les=117/118 n=200 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.072+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.072+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.072+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.072+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c9e00 con 0x507ac00 2026-02-11T23:31:28.072+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:52e063dd:::obj399:head)]) -- 0x644cd80 con 0x507ac00 2026-02-11T23:31:28.072+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.072+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 441 ==== MOSDPGPush(4.0 119/117 [PushOp(4:53438dcf:::obj478:head, version: 111'478, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:53438dcf:::obj478:head@111'478, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56caea0 con 0x507ac00 2026-02-11T23:31:28.072+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:53438dcf:::obj478:head, version: 111'478, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:53438dcf:::obj478:head@111'478, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.072+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:53438dcf:::obj478:head, version: 111'478, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:53438dcf:::obj478:head@111'478, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.072+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.072+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:53438dcf:::obj478:head, version: 111'478, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:53438dcf:::obj478:head@111'478, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.072+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 442 ==== pg_backfill(progress 4.0 e 119/119 lb 4:52e063dd:::obj399:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99080 con 0x507ac00 2026-02-11T23:31:28.072+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:52e063dd:::obj399:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.072+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.072+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:52e063dd:::obj399:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.072+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:53438dcf:::obj478:head, version: 111'478, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:53438dcf:::obj478:head@111'478, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.072+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.072+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:53438dcf:::obj478:head, version: 111'478, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:53438dcf:::obj478:head@111'478, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(4.0 119/117 [PushOp(4:53438dcf:::obj478:head, version: 111'478, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:53438dcf:::obj478:head@111'478, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:52d1d495:::obj578:head local-lis/les=117/118 n=200 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.072+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.072+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:52d1d495:::obj578:head local-lis/les=117/118 n=200 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:53438dcf:::obj478:head, version: 111'478, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:53438dcf:::obj478:head@111'478, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.072+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:52d1d495:::obj578:head local-lis/les=117/118 n=200 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:53438dcf:::obj478:head@111'478, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.072+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.072+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:52e063dd:::obj399:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.072+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:52d1d495:::obj578:head local-lis/les=117/118 n=200 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:53438dcf:::obj478:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.072+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:52d1d495:::obj578:head local-lis/les=117/118 n=200 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:53438dcf:::obj478:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.072+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:52d1d495:::obj578:head local-lis/les=117/118 n=200 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:53438dcf:::obj478:head 2026-02-11T23:31:28.072+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:52d1d495:::obj578:head local-lis/les=117/118 n=200 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:53438dcf:::obj478:head v 111'478 2026-02-11T23:31:28.072+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:52d1d495:::obj578:head local-lis/les=117/118 n=200 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.072+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.072+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:53438dcf:::obj478:head, version: 111'478, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:53438dcf:::obj478:head@111'478, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:28.072+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.072+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.072+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:52e063dd:::obj399:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.072+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:52e063dd:::obj399:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 4.0 e 119/119 lb 4:52e063dd:::obj399:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:52d1d495:::obj578:head local-lis/les=117/118 n=200 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.072+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.072+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:52d1d495:::obj578:head local-lis/les=117/118 n=200 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:52e063dd:::obj399:head) v3 2026-02-11T23:31:28.072+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:52d1d495:::obj578:head local-lis/les=117/118 n=200 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:52e063dd:::obj399:head) 2026-02-11T23:31:28.072+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.072+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:52e063dd:::obj399:head) v3 finish latency 0.000013 2026-02-11T23:31:28.072+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:52e063dd:::obj399:head local-lis/les=117/118 n=201 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.072+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.073+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.073+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:52e063dd:::obj399:head local-lis/les=117/118 n=201 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.073+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.073+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.073+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.073+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7ac00 con 0x507ac00 2026-02-11T23:31:28.073+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:53438dcf:::obj478:head)]) -- 0x644d080 con 0x507ac00 2026-02-11T23:31:28.073+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.073+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 443 ==== MOSDPGPush(4.0 119/117 [PushOp(4:53c971fd:::obj132:head, version: 111'132, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:53c971fd:::obj132:head@111'132, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5617380 con 0x507ac00 2026-02-11T23:31:28.073+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:53c971fd:::obj132:head, version: 111'132, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:53c971fd:::obj132:head@111'132, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 119 2026-02-11T23:31:28.073+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:53c971fd:::obj132:head, version: 111'132, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:53c971fd:::obj132:head@111'132, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.073+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.073+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 444 ==== pg_backfill(progress 4.0 e 119/119 lb 4:53438dcf:::obj478:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176b00 con 0x507ac00 2026-02-11T23:31:28.073+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:53c971fd:::obj132:head, version: 111'132, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:53c971fd:::obj132:head@111'132, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.073+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:53438dcf:::obj478:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.073+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:53438dcf:::obj478:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.073+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.073+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:53c971fd:::obj132:head, version: 111'132, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:53c971fd:::obj132:head@111'132, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.073+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:53c971fd:::obj132:head, version: 111'132, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:53c971fd:::obj132:head@111'132, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(4.0 119/117 [PushOp(4:53c971fd:::obj132:head, version: 111'132, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:53c971fd:::obj132:head@111'132, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:52e063dd:::obj399:head local-lis/les=117/118 n=201 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.073+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.073+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.073+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:53438dcf:::obj478:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.073+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:52e063dd:::obj399:head local-lis/les=117/118 n=201 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:53c971fd:::obj132:head, version: 111'132, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:53c971fd:::obj132:head@111'132, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.073+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:52e063dd:::obj399:head local-lis/les=117/118 n=201 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:53c971fd:::obj132:head@111'132, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.073+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:52e063dd:::obj399:head local-lis/les=117/118 n=201 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:53c971fd:::obj132:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.073+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:52e063dd:::obj399:head local-lis/les=117/118 n=201 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:53c971fd:::obj132:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.073+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:52e063dd:::obj399:head local-lis/les=117/118 n=201 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:53c971fd:::obj132:head 2026-02-11T23:31:28.073+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:52e063dd:::obj399:head local-lis/les=117/118 n=201 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:53c971fd:::obj132:head v 111'132 2026-02-11T23:31:28.073+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:52e063dd:::obj399:head local-lis/les=117/118 n=201 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.073+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.073+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:53c971fd:::obj132:head, version: 111'132, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:53c971fd:::obj132:head@111'132, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:28.073+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.073+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.073+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:53438dcf:::obj478:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.073+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:53438dcf:::obj478:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 4.0 e 119/119 lb 4:53438dcf:::obj478:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:52e063dd:::obj399:head local-lis/les=117/118 n=201 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.073+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.073+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:52e063dd:::obj399:head local-lis/les=117/118 n=201 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:53438dcf:::obj478:head) v3 2026-02-11T23:31:28.073+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:52e063dd:::obj399:head local-lis/les=117/118 n=201 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:53438dcf:::obj478:head) 2026-02-11T23:31:28.073+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.073+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:53438dcf:::obj478:head) v3 finish latency 0.000013 2026-02-11T23:31:28.073+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:53438dcf:::obj478:head local-lis/les=117/118 n=202 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.073+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.073+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.073+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:53438dcf:::obj478:head local-lis/les=117/118 n=202 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.073+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.073+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.073+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.073+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c5800 con 0x507ac00 2026-02-11T23:31:28.073+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:53c971fd:::obj132:head)]) -- 0x644d380 con 0x507ac00 2026-02-11T23:31:28.073+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.074+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 445 ==== MOSDPGPush(4.0 119/117 [PushOp(4:541e28f4:::obj33:head, version: 111'33, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:541e28f4:::obj33:head@111'33, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x56a3a00 con 0x507ac00 2026-02-11T23:31:28.074+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:541e28f4:::obj33:head, version: 111'33, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:541e28f4:::obj33:head@111'33, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.074+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:541e28f4:::obj33:head, version: 111'33, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:541e28f4:::obj33:head@111'33, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.074+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.074+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 446 ==== pg_backfill(progress 4.0 e 119/119 lb 4:53c971fd:::obj132:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2d600 con 0x507ac00 2026-02-11T23:31:28.074+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:53c971fd:::obj132:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.074+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:541e28f4:::obj33:head, version: 111'33, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:541e28f4:::obj33:head@111'33, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.074+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:53c971fd:::obj132:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.074+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.074+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:541e28f4:::obj33:head, version: 111'33, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:541e28f4:::obj33:head@111'33, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.074+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.074+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:541e28f4:::obj33:head, version: 111'33, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:541e28f4:::obj33:head@111'33, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 119/117 [PushOp(4:541e28f4:::obj33:head, version: 111'33, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:541e28f4:::obj33:head@111'33, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:53438dcf:::obj478:head local-lis/les=117/118 n=202 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.074+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:53c971fd:::obj132:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.074+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.074+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:53438dcf:::obj478:head local-lis/les=117/118 n=202 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:541e28f4:::obj33:head, version: 111'33, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:541e28f4:::obj33:head@111'33, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.074+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:53438dcf:::obj478:head local-lis/les=117/118 n=202 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:541e28f4:::obj33:head@111'33, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.074+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:53438dcf:::obj478:head local-lis/les=117/118 n=202 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:541e28f4:::obj33:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.074+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:53438dcf:::obj478:head local-lis/les=117/118 n=202 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:541e28f4:::obj33:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.074+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:53438dcf:::obj478:head local-lis/les=117/118 n=202 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:541e28f4:::obj33:head 2026-02-11T23:31:28.074+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:53438dcf:::obj478:head local-lis/les=117/118 n=202 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:541e28f4:::obj33:head v 111'33 2026-02-11T23:31:28.074+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:53438dcf:::obj478:head local-lis/les=117/118 n=202 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.074+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.074+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:541e28f4:::obj33:head, version: 111'33, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:541e28f4:::obj33:head@111'33, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:28.074+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.074+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.074+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:53c971fd:::obj132:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.074+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:53c971fd:::obj132:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 4.0 e 119/119 lb 4:53c971fd:::obj132:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:53438dcf:::obj478:head local-lis/les=117/118 n=202 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.074+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.074+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:53438dcf:::obj478:head local-lis/les=117/118 n=202 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:53c971fd:::obj132:head) v3 2026-02-11T23:31:28.074+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:53438dcf:::obj478:head local-lis/les=117/118 n=202 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:53c971fd:::obj132:head) 2026-02-11T23:31:28.074+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.074+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:53c971fd:::obj132:head) v3 finish latency 0.000013 2026-02-11T23:31:28.074+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:53c971fd:::obj132:head local-lis/les=117/118 n=203 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.074+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.074+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.074+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:53c971fd:::obj132:head local-lis/les=117/118 n=203 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.074+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.074+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.074+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.074+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7aa80 con 0x507ac00 2026-02-11T23:31:28.074+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:541e28f4:::obj33:head)]) -- 0x644d680 con 0x507ac00 2026-02-11T23:31:28.074+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.074+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 447 ==== MOSDPGPush(4.0 119/117 [PushOp(4:545db4bf:::obj139:head, version: 111'139, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:545db4bf:::obj139:head@111'139, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x554e680 con 0x507ac00 2026-02-11T23:31:28.074+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:545db4bf:::obj139:head, version: 111'139, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:545db4bf:::obj139:head@111'139, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 119 2026-02-11T23:31:28.074+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:545db4bf:::obj139:head, version: 111'139, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:545db4bf:::obj139:head@111'139, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.074+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.074+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 448 ==== pg_backfill(progress 4.0 e 119/119 lb 4:541e28f4:::obj33:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d98580 con 0x507ac00 2026-02-11T23:31:28.074+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:545db4bf:::obj139:head, version: 111'139, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:545db4bf:::obj139:head@111'139, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.074+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:541e28f4:::obj33:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.074+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:541e28f4:::obj33:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.074+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.074+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:545db4bf:::obj139:head, version: 111'139, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:545db4bf:::obj139:head@111'139, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.074+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:545db4bf:::obj139:head, version: 111'139, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:545db4bf:::obj139:head@111'139, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 119/117 [PushOp(4:545db4bf:::obj139:head, version: 111'139, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:545db4bf:::obj139:head@111'139, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:53c971fd:::obj132:head local-lis/les=117/118 n=203 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.075+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.075+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.075+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:541e28f4:::obj33:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.075+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:53c971fd:::obj132:head local-lis/les=117/118 n=203 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:545db4bf:::obj139:head, version: 111'139, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:545db4bf:::obj139:head@111'139, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.075+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:53c971fd:::obj132:head local-lis/les=117/118 n=203 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:545db4bf:::obj139:head@111'139, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.075+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:53c971fd:::obj132:head local-lis/les=117/118 n=203 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:545db4bf:::obj139:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.075+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:53c971fd:::obj132:head local-lis/les=117/118 n=203 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:545db4bf:::obj139:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.075+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:53c971fd:::obj132:head local-lis/les=117/118 n=203 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:545db4bf:::obj139:head 2026-02-11T23:31:28.075+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:53c971fd:::obj132:head local-lis/les=117/118 n=203 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:545db4bf:::obj139:head v 111'139 2026-02-11T23:31:28.075+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:53c971fd:::obj132:head local-lis/les=117/118 n=203 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.075+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.075+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:545db4bf:::obj139:head, version: 111'139, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:545db4bf:::obj139:head@111'139, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000079 2026-02-11T23:31:28.075+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.075+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.075+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:541e28f4:::obj33:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.075+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:541e28f4:::obj33:head) v3 prio 15 cost 0 latency 0.000097 pg_backfill(progress 4.0 e 119/119 lb 4:541e28f4:::obj33:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:53c971fd:::obj132:head local-lis/les=117/118 n=203 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.075+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.075+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:53c971fd:::obj132:head local-lis/les=117/118 n=203 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:541e28f4:::obj33:head) v3 2026-02-11T23:31:28.075+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:53c971fd:::obj132:head local-lis/les=117/118 n=203 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:541e28f4:::obj33:head) 2026-02-11T23:31:28.075+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.075+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:541e28f4:::obj33:head) v3 finish latency 0.000015 2026-02-11T23:31:28.075+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:541e28f4:::obj33:head local-lis/les=117/118 n=204 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.075+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.075+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.075+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:541e28f4:::obj33:head local-lis/les=117/118 n=204 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.075+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.075+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.075+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.075+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x649e600 con 0x507ac00 2026-02-11T23:31:28.075+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:545db4bf:::obj139:head)]) -- 0x644d800 con 0x507ac00 2026-02-11T23:31:28.075+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.075+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 449 ==== MOSDPGPush(4.0 119/117 [PushOp(4:54a9e115:::obj404:head, version: 111'404, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:54a9e115:::obj404:head@111'404, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x554e340 con 0x507ac00 2026-02-11T23:31:28.075+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:54a9e115:::obj404:head, version: 111'404, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:54a9e115:::obj404:head@111'404, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 119 2026-02-11T23:31:28.075+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:54a9e115:::obj404:head, version: 111'404, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:54a9e115:::obj404:head@111'404, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.075+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.075+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 450 ==== pg_backfill(progress 4.0 e 119/119 lb 4:545db4bf:::obj139:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98000 con 0x507ac00 2026-02-11T23:31:28.075+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:54a9e115:::obj404:head, version: 111'404, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:54a9e115:::obj404:head@111'404, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.075+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:545db4bf:::obj139:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.075+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.075+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:545db4bf:::obj139:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.075+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:54a9e115:::obj404:head, version: 111'404, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:54a9e115:::obj404:head@111'404, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.075+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.075+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:54a9e115:::obj404:head, version: 111'404, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:54a9e115:::obj404:head@111'404, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000044 MOSDPGPush(4.0 119/117 [PushOp(4:54a9e115:::obj404:head, version: 111'404, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:54a9e115:::obj404:head@111'404, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:541e28f4:::obj33:head local-lis/les=117/118 n=204 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.075+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.075+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:541e28f4:::obj33:head local-lis/les=117/118 n=204 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:54a9e115:::obj404:head, version: 111'404, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:54a9e115:::obj404:head@111'404, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.075+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.075+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:541e28f4:::obj33:head local-lis/les=117/118 n=204 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:54a9e115:::obj404:head@111'404, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.075+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:545db4bf:::obj139:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.075+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:541e28f4:::obj33:head local-lis/les=117/118 n=204 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:54a9e115:::obj404:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.075+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:541e28f4:::obj33:head local-lis/les=117/118 n=204 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:54a9e115:::obj404:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.075+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:541e28f4:::obj33:head local-lis/les=117/118 n=204 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:54a9e115:::obj404:head 2026-02-11T23:31:28.075+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:541e28f4:::obj33:head local-lis/les=117/118 n=204 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:54a9e115:::obj404:head v 111'404 2026-02-11T23:31:28.075+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:541e28f4:::obj33:head local-lis/les=117/118 n=204 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.075+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.075+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:54a9e115:::obj404:head, version: 111'404, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:54a9e115:::obj404:head@111'404, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:28.075+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.075+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.075+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:545db4bf:::obj139:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.075+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:545db4bf:::obj139:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 4.0 e 119/119 lb 4:545db4bf:::obj139:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:541e28f4:::obj33:head local-lis/les=117/118 n=204 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.075+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.075+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:541e28f4:::obj33:head local-lis/les=117/118 n=204 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:545db4bf:::obj139:head) v3 2026-02-11T23:31:28.075+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:541e28f4:::obj33:head local-lis/les=117/118 n=204 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:545db4bf:::obj139:head) 2026-02-11T23:31:28.075+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.075+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:545db4bf:::obj139:head) v3 finish latency 0.000013 2026-02-11T23:31:28.075+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:545db4bf:::obj139:head local-lis/les=117/118 n=205 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.075+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.075+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.075+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:545db4bf:::obj139:head local-lis/les=117/118 n=205 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.075+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.075+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.075+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.075+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7a900 con 0x507ac00 2026-02-11T23:31:28.075+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:54a9e115:::obj404:head)]) -- 0x644db00 con 0x507ac00 2026-02-11T23:31:28.075+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.076+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 451 ==== MOSDPGPush(4.0 119/117 [PushOp(4:54cd4daa:::obj313:head, version: 111'313, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:54cd4daa:::obj313:head@111'313, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5646d00 con 0x507ac00 2026-02-11T23:31:28.076+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:54cd4daa:::obj313:head, version: 111'313, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:54cd4daa:::obj313:head@111'313, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.076+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:54cd4daa:::obj313:head, version: 111'313, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:54cd4daa:::obj313:head@111'313, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.076+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.076+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 452 ==== pg_backfill(progress 4.0 e 119/119 lb 4:54a9e115:::obj404:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9b180 con 0x507ac00 2026-02-11T23:31:28.076+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:54cd4daa:::obj313:head, version: 111'313, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:54cd4daa:::obj313:head@111'313, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.076+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:54a9e115:::obj404:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 119 2026-02-11T23:31:28.076+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:54a9e115:::obj404:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.076+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.076+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:54cd4daa:::obj313:head, version: 111'313, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:54cd4daa:::obj313:head@111'313, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.076+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.076+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:54cd4daa:::obj313:head, version: 111'313, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:54cd4daa:::obj313:head@111'313, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(4.0 119/117 [PushOp(4:54cd4daa:::obj313:head, version: 111'313, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:54cd4daa:::obj313:head@111'313, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:545db4bf:::obj139:head local-lis/les=117/118 n=205 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.076+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.076+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:545db4bf:::obj139:head local-lis/les=117/118 n=205 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:54cd4daa:::obj313:head, version: 111'313, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:54cd4daa:::obj313:head@111'313, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.076+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:545db4bf:::obj139:head local-lis/les=117/118 n=205 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:54cd4daa:::obj313:head@111'313, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.076+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.076+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:54a9e115:::obj404:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.076+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:545db4bf:::obj139:head local-lis/les=117/118 n=205 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:54cd4daa:::obj313:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.076+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:545db4bf:::obj139:head local-lis/les=117/118 n=205 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:54cd4daa:::obj313:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.076+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:545db4bf:::obj139:head local-lis/les=117/118 n=205 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:54cd4daa:::obj313:head 2026-02-11T23:31:28.076+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:545db4bf:::obj139:head local-lis/les=117/118 n=205 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:54cd4daa:::obj313:head v 111'313 2026-02-11T23:31:28.076+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:545db4bf:::obj139:head local-lis/les=117/118 n=205 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.076+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.076+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:54cd4daa:::obj313:head, version: 111'313, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:54cd4daa:::obj313:head@111'313, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:28.076+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.076+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.076+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:54a9e115:::obj404:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.076+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:54a9e115:::obj404:head) v3 prio 15 cost 0 latency 0.000080 pg_backfill(progress 4.0 e 119/119 lb 4:54a9e115:::obj404:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:545db4bf:::obj139:head local-lis/les=117/118 n=205 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.076+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.076+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:545db4bf:::obj139:head local-lis/les=117/118 n=205 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:54a9e115:::obj404:head) v3 2026-02-11T23:31:28.076+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:545db4bf:::obj139:head local-lis/les=117/118 n=205 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:54a9e115:::obj404:head) 2026-02-11T23:31:28.076+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.076+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:54a9e115:::obj404:head) v3 finish latency 0.000014 2026-02-11T23:31:28.076+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:54a9e115:::obj404:head local-lis/les=117/118 n=206 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.076+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.076+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.076+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:54a9e115:::obj404:head local-lis/les=117/118 n=206 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.076+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.076+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.076+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.076+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c9200 con 0x507ac00 2026-02-11T23:31:28.076+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:54cd4daa:::obj313:head)]) -- 0x644de00 con 0x507ac00 2026-02-11T23:31:28.076+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.077+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 453 ==== MOSDPGPush(4.0 119/117 [PushOp(4:554d3990:::obj276:head, version: 111'276, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:554d3990:::obj276:head@111'276, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56ca680 con 0x507ac00 2026-02-11T23:31:28.077+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:554d3990:::obj276:head, version: 111'276, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:554d3990:::obj276:head@111'276, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 119 2026-02-11T23:31:28.077+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:554d3990:::obj276:head, version: 111'276, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:554d3990:::obj276:head@111'276, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.077+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.077+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 454 ==== pg_backfill(progress 4.0 e 119/119 lb 4:54cd4daa:::obj313:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98b00 con 0x507ac00 2026-02-11T23:31:28.077+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:54cd4daa:::obj313:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.077+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:554d3990:::obj276:head, version: 111'276, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:554d3990:::obj276:head@111'276, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.077+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:54cd4daa:::obj313:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.077+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.077+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:554d3990:::obj276:head, version: 111'276, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:554d3990:::obj276:head@111'276, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.077+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.077+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:554d3990:::obj276:head, version: 111'276, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:554d3990:::obj276:head@111'276, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(4.0 119/117 [PushOp(4:554d3990:::obj276:head, version: 111'276, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:554d3990:::obj276:head@111'276, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:54a9e115:::obj404:head local-lis/les=117/118 n=206 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.077+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:54cd4daa:::obj313:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.077+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.077+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:54a9e115:::obj404:head local-lis/les=117/118 n=206 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:554d3990:::obj276:head, version: 111'276, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:554d3990:::obj276:head@111'276, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.077+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:54a9e115:::obj404:head local-lis/les=117/118 n=206 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:554d3990:::obj276:head@111'276, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.077+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:54a9e115:::obj404:head local-lis/les=117/118 n=206 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:554d3990:::obj276:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.077+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:54a9e115:::obj404:head local-lis/les=117/118 n=206 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:554d3990:::obj276:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.077+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:54a9e115:::obj404:head local-lis/les=117/118 n=206 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:554d3990:::obj276:head 2026-02-11T23:31:28.077+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:54a9e115:::obj404:head local-lis/les=117/118 n=206 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:554d3990:::obj276:head v 111'276 2026-02-11T23:31:28.077+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:54a9e115:::obj404:head local-lis/les=117/118 n=206 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.077+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.077+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:554d3990:::obj276:head, version: 111'276, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:554d3990:::obj276:head@111'276, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:28.077+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.077+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.077+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:54cd4daa:::obj313:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.077+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:54cd4daa:::obj313:head) v3 prio 15 cost 0 latency 0.000079 pg_backfill(progress 4.0 e 119/119 lb 4:54cd4daa:::obj313:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:54a9e115:::obj404:head local-lis/les=117/118 n=206 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.077+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.077+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:54a9e115:::obj404:head local-lis/les=117/118 n=206 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:54cd4daa:::obj313:head) v3 2026-02-11T23:31:28.077+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:54a9e115:::obj404:head local-lis/les=117/118 n=206 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:54cd4daa:::obj313:head) 2026-02-11T23:31:28.077+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.077+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:54cd4daa:::obj313:head) v3 finish latency 0.000014 2026-02-11T23:31:28.077+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:54cd4daa:::obj313:head local-lis/les=117/118 n=207 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.077+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.077+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.077+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:54cd4daa:::obj313:head local-lis/les=117/118 n=207 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.077+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.077+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.077+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.077+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7a780 con 0x507ac00 2026-02-11T23:31:28.077+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:554d3990:::obj276:head)]) -- 0x4f7b080 con 0x507ac00 2026-02-11T23:31:28.077+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.077+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 455 ==== MOSDPGPush(4.0 119/117 [PushOp(4:55c8626c:::obj516:head, version: 111'516, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:55c8626c:::obj516:head@111'516, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5617520 con 0x507ac00 2026-02-11T23:31:28.077+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:55c8626c:::obj516:head, version: 111'516, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:55c8626c:::obj516:head@111'516, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 119 2026-02-11T23:31:28.077+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:55c8626c:::obj516:head, version: 111'516, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:55c8626c:::obj516:head@111'516, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.077+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.077+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 456 ==== pg_backfill(progress 4.0 e 119/119 lb 4:554d3990:::obj276:head) ==== 1062+0+0 (unknown 0 0 0) 0x6179180 con 0x507ac00 2026-02-11T23:31:28.077+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:554d3990:::obj276:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.077+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:55c8626c:::obj516:head, version: 111'516, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:55c8626c:::obj516:head@111'516, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.077+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:554d3990:::obj276:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.077+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.077+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:55c8626c:::obj516:head, version: 111'516, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:55c8626c:::obj516:head@111'516, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.077+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:55c8626c:::obj516:head, version: 111'516, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:55c8626c:::obj516:head@111'516, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(4.0 119/117 [PushOp(4:55c8626c:::obj516:head, version: 111'516, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:55c8626c:::obj516:head@111'516, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:54cd4daa:::obj313:head local-lis/les=117/118 n=207 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.077+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.077+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.077+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:54cd4daa:::obj313:head local-lis/les=117/118 n=207 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:55c8626c:::obj516:head, version: 111'516, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:55c8626c:::obj516:head@111'516, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.077+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:54cd4daa:::obj313:head local-lis/les=117/118 n=207 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:55c8626c:::obj516:head@111'516, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.077+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.077+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:54cd4daa:::obj313:head local-lis/les=117/118 n=207 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:55c8626c:::obj516:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.077+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:554d3990:::obj276:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.077+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:54cd4daa:::obj313:head local-lis/les=117/118 n=207 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:55c8626c:::obj516:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.077+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:54cd4daa:::obj313:head local-lis/les=117/118 n=207 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:55c8626c:::obj516:head 2026-02-11T23:31:28.077+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:54cd4daa:::obj313:head local-lis/les=117/118 n=207 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:55c8626c:::obj516:head v 111'516 2026-02-11T23:31:28.077+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:54cd4daa:::obj313:head local-lis/les=117/118 n=207 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.077+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.077+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:55c8626c:::obj516:head, version: 111'516, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:55c8626c:::obj516:head@111'516, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:28.077+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:54cd4daa:::obj313:head local-lis/les=117/118 n=207 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.077+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.077+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.077+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:554d3990:::obj276:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.077+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:554d3990:::obj276:head) v3 prio 15 cost 0 latency 0.000079 pg_backfill(progress 4.0 e 119/119 lb 4:554d3990:::obj276:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:54cd4daa:::obj313:head local-lis/les=117/118 n=207 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.077+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.077+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:54cd4daa:::obj313:head local-lis/les=117/118 n=207 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:554d3990:::obj276:head) v3 2026-02-11T23:31:28.077+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:54cd4daa:::obj313:head local-lis/les=117/118 n=207 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:554d3990:::obj276:head) 2026-02-11T23:31:28.077+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.077+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:554d3990:::obj276:head) v3 finish latency 0.000028 2026-02-11T23:31:28.077+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.078+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.078+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:554d3990:::obj276:head local-lis/les=117/118 n=208 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.078+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.078+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.078+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.078+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c4180 con 0x507ac00 2026-02-11T23:31:28.078+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:55c8626c:::obj516:head)]) -- 0x63c9080 con 0x507ac00 2026-02-11T23:31:28.078+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.078+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 457 ==== MOSDPGPush(4.0 119/117 [PushOp(4:55d2244d:::obj498:head, version: 111'498, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:55d2244d:::obj498:head@111'498, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a29c0 con 0x507ac00 2026-02-11T23:31:28.078+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:55d2244d:::obj498:head, version: 111'498, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:55d2244d:::obj498:head@111'498, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.078+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:55d2244d:::obj498:head, version: 111'498, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:55d2244d:::obj498:head@111'498, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.078+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.078+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 458 ==== pg_backfill(progress 4.0 e 119/119 lb 4:55c8626c:::obj516:head) ==== 1062+0+0 (unknown 0 0 0) 0x6178100 con 0x507ac00 2026-02-11T23:31:28.078+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:55d2244d:::obj498:head, version: 111'498, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:55d2244d:::obj498:head@111'498, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.078+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:55c8626c:::obj516:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.078+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:55c8626c:::obj516:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.078+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.078+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:55d2244d:::obj498:head, version: 111'498, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:55d2244d:::obj498:head@111'498, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.078+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.078+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:55d2244d:::obj498:head, version: 111'498, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:55d2244d:::obj498:head@111'498, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 119/117 [PushOp(4:55d2244d:::obj498:head, version: 111'498, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:55d2244d:::obj498:head@111'498, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:554d3990:::obj276:head local-lis/les=117/118 n=208 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.078+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.078+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:554d3990:::obj276:head local-lis/les=117/118 n=208 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:55d2244d:::obj498:head, version: 111'498, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:55d2244d:::obj498:head@111'498, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.078+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.078+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:55c8626c:::obj516:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.078+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:554d3990:::obj276:head local-lis/les=117/118 n=208 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:55d2244d:::obj498:head@111'498, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.078+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:554d3990:::obj276:head local-lis/les=117/118 n=208 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:55d2244d:::obj498:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.078+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:554d3990:::obj276:head local-lis/les=117/118 n=208 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:55d2244d:::obj498:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.078+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:554d3990:::obj276:head local-lis/les=117/118 n=208 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:55d2244d:::obj498:head 2026-02-11T23:31:28.078+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:554d3990:::obj276:head local-lis/les=117/118 n=208 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:55d2244d:::obj498:head v 111'498 2026-02-11T23:31:28.078+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:554d3990:::obj276:head local-lis/les=117/118 n=208 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.078+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.078+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:55d2244d:::obj498:head, version: 111'498, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:55d2244d:::obj498:head@111'498, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000077 2026-02-11T23:31:28.078+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.078+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.078+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:55c8626c:::obj516:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.078+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:55c8626c:::obj516:head) v3 prio 15 cost 0 latency 0.000100 pg_backfill(progress 4.0 e 119/119 lb 4:55c8626c:::obj516:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:554d3990:::obj276:head local-lis/les=117/118 n=208 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.078+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.078+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:554d3990:::obj276:head local-lis/les=117/118 n=208 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:55c8626c:::obj516:head) v3 2026-02-11T23:31:28.078+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:554d3990:::obj276:head local-lis/les=117/118 n=208 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:55c8626c:::obj516:head) 2026-02-11T23:31:28.078+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.078+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:55c8626c:::obj516:head) v3 finish latency 0.000016 2026-02-11T23:31:28.078+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:55c8626c:::obj516:head local-lis/les=117/118 n=209 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.078+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.078+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.078+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:55c8626c:::obj516:head local-lis/les=117/118 n=209 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.078+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.078+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.078+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.078+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7a480 con 0x507ac00 2026-02-11T23:31:28.078+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:55d2244d:::obj498:head)]) -- 0x63c9800 con 0x507ac00 2026-02-11T23:31:28.078+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.079+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 459 ==== MOSDPGPush(4.0 119/117 [PushOp(4:565a3fbf:::obj474:head, version: 111'474, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:565a3fbf:::obj474:head@111'474, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a3040 con 0x507ac00 2026-02-11T23:31:28.079+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:565a3fbf:::obj474:head, version: 111'474, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:565a3fbf:::obj474:head@111'474, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 119 2026-02-11T23:31:28.079+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:565a3fbf:::obj474:head, version: 111'474, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:565a3fbf:::obj474:head@111'474, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.079+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.079+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 460 ==== pg_backfill(progress 4.0 e 119/119 lb 4:55d2244d:::obj498:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177b80 con 0x507ac00 2026-02-11T23:31:28.079+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:55d2244d:::obj498:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.079+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:565a3fbf:::obj474:head, version: 111'474, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:565a3fbf:::obj474:head@111'474, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.079+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:55d2244d:::obj498:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.079+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.079+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:55d2244d:::obj498:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.079+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.079+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:565a3fbf:::obj474:head, version: 111'474, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:565a3fbf:::obj474:head@111'474, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.079+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:565a3fbf:::obj474:head, version: 111'474, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:565a3fbf:::obj474:head@111'474, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000052 MOSDPGPush(4.0 119/117 [PushOp(4:565a3fbf:::obj474:head, version: 111'474, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:565a3fbf:::obj474:head@111'474, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:55c8626c:::obj516:head local-lis/les=117/118 n=209 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.079+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.079+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:55c8626c:::obj516:head local-lis/les=117/118 n=209 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:565a3fbf:::obj474:head, version: 111'474, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:565a3fbf:::obj474:head@111'474, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.079+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:55c8626c:::obj516:head local-lis/les=117/118 n=209 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:565a3fbf:::obj474:head@111'474, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.079+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:55c8626c:::obj516:head local-lis/les=117/118 n=209 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:565a3fbf:::obj474:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.079+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:55c8626c:::obj516:head local-lis/les=117/118 n=209 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:565a3fbf:::obj474:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.079+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:55c8626c:::obj516:head local-lis/les=117/118 n=209 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:565a3fbf:::obj474:head 2026-02-11T23:31:28.079+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:55c8626c:::obj516:head local-lis/les=117/118 n=209 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:565a3fbf:::obj474:head v 111'474 2026-02-11T23:31:28.079+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:55c8626c:::obj516:head local-lis/les=117/118 n=209 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.079+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.079+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:565a3fbf:::obj474:head, version: 111'474, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:565a3fbf:::obj474:head@111'474, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000070 2026-02-11T23:31:28.079+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.079+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.079+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:55d2244d:::obj498:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.079+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:55d2244d:::obj498:head) v3 prio 15 cost 0 latency 0.000106 pg_backfill(progress 4.0 e 119/119 lb 4:55d2244d:::obj498:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:55c8626c:::obj516:head local-lis/les=117/118 n=209 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.079+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.079+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:55c8626c:::obj516:head local-lis/les=117/118 n=209 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:55d2244d:::obj498:head) v3 2026-02-11T23:31:28.079+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:55c8626c:::obj516:head local-lis/les=117/118 n=209 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:55d2244d:::obj498:head) 2026-02-11T23:31:28.079+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.079+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:55d2244d:::obj498:head) v3 finish latency 0.000014 2026-02-11T23:31:28.079+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:55d2244d:::obj498:head local-lis/les=117/118 n=210 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.079+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.079+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.079+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:55d2244d:::obj498:head local-lis/les=117/118 n=210 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.079+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.079+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.079+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.079+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c5500 con 0x507ac00 2026-02-11T23:31:28.079+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:565a3fbf:::obj474:head)]) -- 0x4f7b200 con 0x507ac00 2026-02-11T23:31:28.079+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.079+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 461 ==== MOSDPGPush(4.0 119/117 [PushOp(4:5695f913:::obj241:head, version: 111'241, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5695f913:::obj241:head@111'241, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a3860 con 0x507ac00 2026-02-11T23:31:28.079+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:5695f913:::obj241:head, version: 111'241, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5695f913:::obj241:head@111'241, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 119 2026-02-11T23:31:28.079+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:5695f913:::obj241:head, version: 111'241, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5695f913:::obj241:head@111'241, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.079+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.079+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 462 ==== pg_backfill(progress 4.0 e 119/119 lb 4:565a3fbf:::obj474:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177600 con 0x507ac00 2026-02-11T23:31:28.079+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:5695f913:::obj241:head, version: 111'241, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5695f913:::obj241:head@111'241, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.079+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:565a3fbf:::obj474:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.079+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:565a3fbf:::obj474:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.079+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.079+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:5695f913:::obj241:head, version: 111'241, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5695f913:::obj241:head@111'241, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.079+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:5695f913:::obj241:head, version: 111'241, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5695f913:::obj241:head@111'241, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 119/117 [PushOp(4:5695f913:::obj241:head, version: 111'241, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5695f913:::obj241:head@111'241, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:55d2244d:::obj498:head local-lis/les=117/118 n=210 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.079+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.079+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.079+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:565a3fbf:::obj474:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.079+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:55d2244d:::obj498:head local-lis/les=117/118 n=210 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:5695f913:::obj241:head, version: 111'241, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5695f913:::obj241:head@111'241, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.079+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:55d2244d:::obj498:head local-lis/les=117/118 n=210 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:5695f913:::obj241:head@111'241, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.079+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:55d2244d:::obj498:head local-lis/les=117/118 n=210 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:5695f913:::obj241:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.079+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:55d2244d:::obj498:head local-lis/les=117/118 n=210 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:5695f913:::obj241:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.079+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:55d2244d:::obj498:head local-lis/les=117/118 n=210 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:5695f913:::obj241:head 2026-02-11T23:31:28.079+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:55d2244d:::obj498:head local-lis/les=117/118 n=210 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:5695f913:::obj241:head v 111'241 2026-02-11T23:31:28.079+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:55d2244d:::obj498:head local-lis/les=117/118 n=210 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.079+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.079+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:5695f913:::obj241:head, version: 111'241, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5695f913:::obj241:head@111'241, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000065 2026-02-11T23:31:28.079+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.079+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.079+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:565a3fbf:::obj474:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.079+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:565a3fbf:::obj474:head) v3 prio 15 cost 0 latency 0.000085 pg_backfill(progress 4.0 e 119/119 lb 4:565a3fbf:::obj474:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:55d2244d:::obj498:head local-lis/les=117/118 n=210 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.079+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.079+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:55d2244d:::obj498:head local-lis/les=117/118 n=210 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:565a3fbf:::obj474:head) v3 2026-02-11T23:31:28.079+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:55d2244d:::obj498:head local-lis/les=117/118 n=210 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:565a3fbf:::obj474:head) 2026-02-11T23:31:28.079+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.079+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:565a3fbf:::obj474:head) v3 finish latency 0.000013 2026-02-11T23:31:28.079+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:565a3fbf:::obj474:head local-lis/les=117/118 n=211 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.079+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.080+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.080+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:565a3fbf:::obj474:head local-lis/les=117/118 n=211 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.080+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.080+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.080+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.080+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7a300 con 0x507ac00 2026-02-11T23:31:28.080+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:5695f913:::obj241:head)]) -- 0x63c9980 con 0x507ac00 2026-02-11T23:31:28.080+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.080+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 463 ==== MOSDPGPush(4.0 119/117 [PushOp(4:56fc6216:::obj287:head, version: 111'287, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:56fc6216:::obj287:head@111'287, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56171e0 con 0x507ac00 2026-02-11T23:31:28.080+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:56fc6216:::obj287:head, version: 111'287, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:56fc6216:::obj287:head@111'287, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.080+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:56fc6216:::obj287:head, version: 111'287, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:56fc6216:::obj287:head@111'287, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.080+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.080+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 464 ==== pg_backfill(progress 4.0 e 119/119 lb 4:5695f913:::obj241:head) ==== 1062+0+0 (unknown 0 0 0) 0x5985700 con 0x507ac00 2026-02-11T23:31:28.080+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:5695f913:::obj241:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.080+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:56fc6216:::obj287:head, version: 111'287, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:56fc6216:::obj287:head@111'287, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.080+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:5695f913:::obj241:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.080+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.080+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:56fc6216:::obj287:head, version: 111'287, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:56fc6216:::obj287:head@111'287, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.080+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:56fc6216:::obj287:head, version: 111'287, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:56fc6216:::obj287:head@111'287, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(4.0 119/117 [PushOp(4:56fc6216:::obj287:head, version: 111'287, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:56fc6216:::obj287:head@111'287, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:565a3fbf:::obj474:head local-lis/les=117/118 n=211 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.080+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.080+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.080+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:5695f913:::obj241:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.080+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:565a3fbf:::obj474:head local-lis/les=117/118 n=211 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:56fc6216:::obj287:head, version: 111'287, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:56fc6216:::obj287:head@111'287, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.080+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:565a3fbf:::obj474:head local-lis/les=117/118 n=211 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:56fc6216:::obj287:head@111'287, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.080+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:565a3fbf:::obj474:head local-lis/les=117/118 n=211 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:56fc6216:::obj287:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.080+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:565a3fbf:::obj474:head local-lis/les=117/118 n=211 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:56fc6216:::obj287:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.080+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:565a3fbf:::obj474:head local-lis/les=117/118 n=211 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:56fc6216:::obj287:head 2026-02-11T23:31:28.080+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:565a3fbf:::obj474:head local-lis/les=117/118 n=211 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:56fc6216:::obj287:head v 111'287 2026-02-11T23:31:28.080+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:565a3fbf:::obj474:head local-lis/les=117/118 n=211 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.080+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.080+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:56fc6216:::obj287:head, version: 111'287, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:56fc6216:::obj287:head@111'287, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000062 2026-02-11T23:31:28.080+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.080+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.080+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:5695f913:::obj241:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.080+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:5695f913:::obj241:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 4.0 e 119/119 lb 4:5695f913:::obj241:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:565a3fbf:::obj474:head local-lis/les=117/118 n=211 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.080+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.080+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:565a3fbf:::obj474:head local-lis/les=117/118 n=211 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:5695f913:::obj241:head) v3 2026-02-11T23:31:28.080+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:565a3fbf:::obj474:head local-lis/les=117/118 n=211 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:5695f913:::obj241:head) 2026-02-11T23:31:28.080+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.080+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:5695f913:::obj241:head) v3 finish latency 0.000012 2026-02-11T23:31:28.080+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5695f913:::obj241:head local-lis/les=117/118 n=212 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.080+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.080+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.080+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5695f913:::obj241:head local-lis/les=117/118 n=212 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.080+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.080+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.080+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.080+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c8c00 con 0x507ac00 2026-02-11T23:31:28.080+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:56fc6216:::obj287:head)]) -- 0x4f7b380 con 0x507ac00 2026-02-11T23:31:28.080+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.081+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 465 ==== MOSDPGPush(4.0 119/117 [PushOp(4:578304e3:::obj249:head, version: 111'249, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:578304e3:::obj249:head@111'249, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56cad00 con 0x507ac00 2026-02-11T23:31:28.081+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:578304e3:::obj249:head, version: 111'249, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:578304e3:::obj249:head@111'249, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.081+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:578304e3:::obj249:head, version: 111'249, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:578304e3:::obj249:head@111'249, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.081+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.081+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 466 ==== pg_backfill(progress 4.0 e 119/119 lb 4:56fc6216:::obj287:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9a680 con 0x507ac00 2026-02-11T23:31:28.081+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:578304e3:::obj249:head, version: 111'249, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:578304e3:::obj249:head@111'249, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.081+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:56fc6216:::obj287:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.081+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.081+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:56fc6216:::obj287:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.081+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:578304e3:::obj249:head, version: 111'249, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:578304e3:::obj249:head@111'249, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.081+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.081+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:578304e3:::obj249:head, version: 111'249, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:578304e3:::obj249:head@111'249, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 119/117 [PushOp(4:578304e3:::obj249:head, version: 111'249, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:578304e3:::obj249:head@111'249, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:5695f913:::obj241:head local-lis/les=117/118 n=212 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.081+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.081+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5695f913:::obj241:head local-lis/les=117/118 n=212 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:578304e3:::obj249:head, version: 111'249, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:578304e3:::obj249:head@111'249, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.081+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.081+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5695f913:::obj241:head local-lis/les=117/118 n=212 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:578304e3:::obj249:head@111'249, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.081+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:56fc6216:::obj287:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.081+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5695f913:::obj241:head local-lis/les=117/118 n=212 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:578304e3:::obj249:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.081+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5695f913:::obj241:head local-lis/les=117/118 n=212 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:578304e3:::obj249:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.081+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5695f913:::obj241:head local-lis/les=117/118 n=212 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:578304e3:::obj249:head 2026-02-11T23:31:28.081+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5695f913:::obj241:head local-lis/les=117/118 n=212 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:578304e3:::obj249:head v 111'249 2026-02-11T23:31:28.081+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5695f913:::obj241:head local-lis/les=117/118 n=212 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.081+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.081+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:578304e3:::obj249:head, version: 111'249, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:578304e3:::obj249:head@111'249, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:28.081+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.081+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.081+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:56fc6216:::obj287:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.081+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:56fc6216:::obj287:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 4.0 e 119/119 lb 4:56fc6216:::obj287:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:5695f913:::obj241:head local-lis/les=117/118 n=212 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.081+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.081+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5695f913:::obj241:head local-lis/les=117/118 n=212 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:56fc6216:::obj287:head) v3 2026-02-11T23:31:28.081+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5695f913:::obj241:head local-lis/les=117/118 n=212 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:56fc6216:::obj287:head) 2026-02-11T23:31:28.081+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.081+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:56fc6216:::obj287:head) v3 finish latency 0.000013 2026-02-11T23:31:28.081+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:56fc6216:::obj287:head local-lis/les=117/118 n=213 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.081+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.081+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.081+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:56fc6216:::obj287:head local-lis/les=117/118 n=213 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.081+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.081+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.081+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.081+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7a180 con 0x507ac00 2026-02-11T23:31:28.081+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:578304e3:::obj249:head)]) -- 0x63c5c80 con 0x507ac00 2026-02-11T23:31:28.081+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.081+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 467 ==== MOSDPGPush(4.0 119/117 [PushOp(4:57ba9973:::obj296:head, version: 111'296, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:57ba9973:::obj296:head@111'296, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x555cea0 con 0x507ac00 2026-02-11T23:31:28.081+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:57ba9973:::obj296:head, version: 111'296, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:57ba9973:::obj296:head@111'296, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 119 2026-02-11T23:31:28.081+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:57ba9973:::obj296:head, version: 111'296, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:57ba9973:::obj296:head@111'296, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.081+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.081+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 468 ==== pg_backfill(progress 4.0 e 119/119 lb 4:578304e3:::obj249:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176000 con 0x507ac00 2026-02-11T23:31:28.081+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:578304e3:::obj249:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.081+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:57ba9973:::obj296:head, version: 111'296, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:57ba9973:::obj296:head@111'296, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.081+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:578304e3:::obj249:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.081+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.081+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:57ba9973:::obj296:head, version: 111'296, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:57ba9973:::obj296:head@111'296, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.081+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:57ba9973:::obj296:head, version: 111'296, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:57ba9973:::obj296:head@111'296, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 119/117 [PushOp(4:57ba9973:::obj296:head, version: 111'296, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:57ba9973:::obj296:head@111'296, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:56fc6216:::obj287:head local-lis/les=117/118 n=213 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.081+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.081+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.081+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:578304e3:::obj249:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.081+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:56fc6216:::obj287:head local-lis/les=117/118 n=213 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:57ba9973:::obj296:head, version: 111'296, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:57ba9973:::obj296:head@111'296, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.081+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:56fc6216:::obj287:head local-lis/les=117/118 n=213 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:57ba9973:::obj296:head@111'296, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.081+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:56fc6216:::obj287:head local-lis/les=117/118 n=213 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:57ba9973:::obj296:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.081+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:56fc6216:::obj287:head local-lis/les=117/118 n=213 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:57ba9973:::obj296:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.081+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:56fc6216:::obj287:head local-lis/les=117/118 n=213 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:57ba9973:::obj296:head 2026-02-11T23:31:28.081+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:56fc6216:::obj287:head local-lis/les=117/118 n=213 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:57ba9973:::obj296:head v 111'296 2026-02-11T23:31:28.081+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:56fc6216:::obj287:head local-lis/les=117/118 n=213 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.081+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.081+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:57ba9973:::obj296:head, version: 111'296, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:57ba9973:::obj296:head@111'296, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000064 2026-02-11T23:31:28.081+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.081+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.081+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:578304e3:::obj249:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.081+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:578304e3:::obj249:head) v3 prio 15 cost 0 latency 0.000088 pg_backfill(progress 4.0 e 119/119 lb 4:578304e3:::obj249:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:56fc6216:::obj287:head local-lis/les=117/118 n=213 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.081+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.081+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:56fc6216:::obj287:head local-lis/les=117/118 n=213 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:578304e3:::obj249:head) v3 2026-02-11T23:31:28.081+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:56fc6216:::obj287:head local-lis/les=117/118 n=213 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:578304e3:::obj249:head) 2026-02-11T23:31:28.081+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.081+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:578304e3:::obj249:head) v3 finish latency 0.000013 2026-02-11T23:31:28.081+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:578304e3:::obj249:head local-lis/les=117/118 n=214 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.081+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.081+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.081+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:578304e3:::obj249:head local-lis/les=117/118 n=214 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.081+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.081+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.081+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.081+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c9980 con 0x507ac00 2026-02-11T23:31:28.081+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:57ba9973:::obj296:head)]) -- 0x4f7b500 con 0x507ac00 2026-02-11T23:31:28.081+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.082+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 469 ==== MOSDPGPush(4.0 119/117 [PushOp(4:5880e7a7:::obj64:head, version: 111'64, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5880e7a7:::obj64:head@111'64, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x55f1ba0 con 0x507ac00 2026-02-11T23:31:28.082+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:5880e7a7:::obj64:head, version: 111'64, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5880e7a7:::obj64:head@111'64, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 119 2026-02-11T23:31:28.082+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:5880e7a7:::obj64:head, version: 111'64, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5880e7a7:::obj64:head@111'64, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.082+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.082+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 470 ==== pg_backfill(progress 4.0 e 119/119 lb 4:57ba9973:::obj296:head) ==== 1062+0+0 (unknown 0 0 0) 0x6179700 con 0x507ac00 2026-02-11T23:31:28.082+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:57ba9973:::obj296:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.082+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:5880e7a7:::obj64:head, version: 111'64, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5880e7a7:::obj64:head@111'64, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.082+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:57ba9973:::obj296:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.082+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.082+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:5880e7a7:::obj64:head, version: 111'64, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5880e7a7:::obj64:head@111'64, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.082+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.082+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:57ba9973:::obj296:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.082+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:5880e7a7:::obj64:head, version: 111'64, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5880e7a7:::obj64:head@111'64, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(4.0 119/117 [PushOp(4:5880e7a7:::obj64:head, version: 111'64, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5880e7a7:::obj64:head@111'64, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:578304e3:::obj249:head local-lis/les=117/118 n=214 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.082+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.082+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:578304e3:::obj249:head local-lis/les=117/118 n=214 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:5880e7a7:::obj64:head, version: 111'64, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5880e7a7:::obj64:head@111'64, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.082+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:578304e3:::obj249:head local-lis/les=117/118 n=214 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:5880e7a7:::obj64:head@111'64, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.082+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:578304e3:::obj249:head local-lis/les=117/118 n=214 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:5880e7a7:::obj64:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.082+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:578304e3:::obj249:head local-lis/les=117/118 n=214 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:5880e7a7:::obj64:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.082+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:578304e3:::obj249:head local-lis/les=117/118 n=214 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:5880e7a7:::obj64:head 2026-02-11T23:31:28.082+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:578304e3:::obj249:head local-lis/les=117/118 n=214 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:5880e7a7:::obj64:head v 111'64 2026-02-11T23:31:28.082+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:578304e3:::obj249:head local-lis/les=117/118 n=214 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.082+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.082+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:5880e7a7:::obj64:head, version: 111'64, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5880e7a7:::obj64:head@111'64, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000068 2026-02-11T23:31:28.082+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.082+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.082+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:57ba9973:::obj296:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.082+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:57ba9973:::obj296:head) v3 prio 15 cost 0 latency 0.000094 pg_backfill(progress 4.0 e 119/119 lb 4:57ba9973:::obj296:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:578304e3:::obj249:head local-lis/les=117/118 n=214 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.082+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.082+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:578304e3:::obj249:head local-lis/les=117/118 n=214 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:57ba9973:::obj296:head) v3 2026-02-11T23:31:28.082+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:578304e3:::obj249:head local-lis/les=117/118 n=214 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:57ba9973:::obj296:head) 2026-02-11T23:31:28.082+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.082+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:57ba9973:::obj296:head) v3 finish latency 0.000013 2026-02-11T23:31:28.082+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:57ba9973:::obj296:head local-lis/les=117/118 n=215 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.082+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.082+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.082+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:57ba9973:::obj296:head local-lis/les=117/118 n=215 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.082+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.082+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.082+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.082+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7a300 con 0x507ac00 2026-02-11T23:31:28.082+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:5880e7a7:::obj64:head)]) -- 0x63c5b00 con 0x507ac00 2026-02-11T23:31:28.082+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.083+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 471 ==== MOSDPGPush(4.0 119/117 [PushOp(4:58980d3a:::obj339:head, version: 111'339, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:58980d3a:::obj339:head@111'339, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f0ea0 con 0x507ac00 2026-02-11T23:31:28.083+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:58980d3a:::obj339:head, version: 111'339, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:58980d3a:::obj339:head@111'339, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000016 epoch 119 2026-02-11T23:31:28.083+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:58980d3a:::obj339:head, version: 111'339, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:58980d3a:::obj339:head@111'339, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.083+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.083+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 472 ==== pg_backfill(progress 4.0 e 119/119 lb 4:5880e7a7:::obj64:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d2cb00 con 0x507ac00 2026-02-11T23:31:28.083+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:58980d3a:::obj339:head, version: 111'339, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:58980d3a:::obj339:head@111'339, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.083+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:5880e7a7:::obj64:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.083+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:5880e7a7:::obj64:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.083+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.083+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:58980d3a:::obj339:head, version: 111'339, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:58980d3a:::obj339:head@111'339, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.083+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.083+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:58980d3a:::obj339:head, version: 111'339, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:58980d3a:::obj339:head@111'339, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000046 MOSDPGPush(4.0 119/117 [PushOp(4:58980d3a:::obj339:head, version: 111'339, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:58980d3a:::obj339:head@111'339, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:57ba9973:::obj296:head local-lis/les=117/118 n=215 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.083+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.083+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:57ba9973:::obj296:head local-lis/les=117/118 n=215 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:58980d3a:::obj339:head, version: 111'339, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:58980d3a:::obj339:head@111'339, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.083+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.083+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:5880e7a7:::obj64:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.083+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:57ba9973:::obj296:head local-lis/les=117/118 n=215 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:58980d3a:::obj339:head@111'339, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.083+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:57ba9973:::obj296:head local-lis/les=117/118 n=215 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:58980d3a:::obj339:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.083+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:57ba9973:::obj296:head local-lis/les=117/118 n=215 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:58980d3a:::obj339:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.083+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:57ba9973:::obj296:head local-lis/les=117/118 n=215 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:58980d3a:::obj339:head 2026-02-11T23:31:28.083+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:57ba9973:::obj296:head local-lis/les=117/118 n=215 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:58980d3a:::obj339:head v 111'339 2026-02-11T23:31:28.083+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:57ba9973:::obj296:head local-lis/les=117/118 n=215 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.083+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.083+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:58980d3a:::obj339:head, version: 111'339, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:58980d3a:::obj339:head@111'339, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000066 2026-02-11T23:31:28.083+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.083+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.083+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:5880e7a7:::obj64:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.083+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:5880e7a7:::obj64:head) v3 prio 15 cost 0 latency 0.000088 pg_backfill(progress 4.0 e 119/119 lb 4:5880e7a7:::obj64:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:57ba9973:::obj296:head local-lis/les=117/118 n=215 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.083+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.083+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:57ba9973:::obj296:head local-lis/les=117/118 n=215 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:5880e7a7:::obj64:head) v3 2026-02-11T23:31:28.083+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:57ba9973:::obj296:head local-lis/les=117/118 n=215 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:5880e7a7:::obj64:head) 2026-02-11T23:31:28.083+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.083+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:5880e7a7:::obj64:head) v3 finish latency 0.000013 2026-02-11T23:31:28.083+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5880e7a7:::obj64:head local-lis/les=117/118 n=216 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.083+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.083+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.083+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5880e7a7:::obj64:head local-lis/les=117/118 n=216 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.083+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.083+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.083+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.083+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7b200 con 0x507ac00 2026-02-11T23:31:28.083+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:58980d3a:::obj339:head)]) -- 0x5f75b00 con 0x507ac00 2026-02-11T23:31:28.083+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.083+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 473 ==== MOSDPGPush(4.0 119/117 [PushOp(4:5946d48f:::obj257:head, version: 111'257, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5946d48f:::obj257:head@111'257, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e261a0 con 0x507ac00 2026-02-11T23:31:28.083+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:5946d48f:::obj257:head, version: 111'257, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5946d48f:::obj257:head@111'257, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 119 2026-02-11T23:31:28.083+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:5946d48f:::obj257:head, version: 111'257, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5946d48f:::obj257:head@111'257, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.083+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.083+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 474 ==== pg_backfill(progress 4.0 e 119/119 lb 4:58980d3a:::obj339:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2c000 con 0x507ac00 2026-02-11T23:31:28.083+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:58980d3a:::obj339:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.083+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:5946d48f:::obj257:head, version: 111'257, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5946d48f:::obj257:head@111'257, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.083+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:58980d3a:::obj339:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.083+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.083+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:5946d48f:::obj257:head, version: 111'257, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5946d48f:::obj257:head@111'257, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.083+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:5946d48f:::obj257:head, version: 111'257, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5946d48f:::obj257:head@111'257, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(4.0 119/117 [PushOp(4:5946d48f:::obj257:head, version: 111'257, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5946d48f:::obj257:head@111'257, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:5880e7a7:::obj64:head local-lis/les=117/118 n=216 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.083+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.083+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.083+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5880e7a7:::obj64:head local-lis/les=117/118 n=216 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:5946d48f:::obj257:head, version: 111'257, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5946d48f:::obj257:head@111'257, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.083+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:58980d3a:::obj339:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.083+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5880e7a7:::obj64:head local-lis/les=117/118 n=216 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:5946d48f:::obj257:head@111'257, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.083+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5880e7a7:::obj64:head local-lis/les=117/118 n=216 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:5946d48f:::obj257:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.083+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5880e7a7:::obj64:head local-lis/les=117/118 n=216 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:5946d48f:::obj257:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.083+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5880e7a7:::obj64:head local-lis/les=117/118 n=216 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:5946d48f:::obj257:head 2026-02-11T23:31:28.083+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5880e7a7:::obj64:head local-lis/les=117/118 n=216 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:5946d48f:::obj257:head v 111'257 2026-02-11T23:31:28.083+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5880e7a7:::obj64:head local-lis/les=117/118 n=216 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.083+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.083+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:5946d48f:::obj257:head, version: 111'257, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5946d48f:::obj257:head@111'257, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:28.083+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5880e7a7:::obj64:head local-lis/les=117/118 n=216 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.083+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.083+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.083+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:58980d3a:::obj339:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.083+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:58980d3a:::obj339:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 119/119 lb 4:58980d3a:::obj339:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:5880e7a7:::obj64:head local-lis/les=117/118 n=216 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.083+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.083+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5880e7a7:::obj64:head local-lis/les=117/118 n=216 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:58980d3a:::obj339:head) v3 2026-02-11T23:31:28.083+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5880e7a7:::obj64:head local-lis/les=117/118 n=216 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:58980d3a:::obj339:head) 2026-02-11T23:31:28.083+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.083+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:58980d3a:::obj339:head) v3 finish latency 0.000024 2026-02-11T23:31:28.083+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.083+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.083+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:58980d3a:::obj339:head local-lis/les=117/118 n=217 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.083+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.083+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.083+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.083+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7a480 con 0x507ac00 2026-02-11T23:31:28.083+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:5946d48f:::obj257:head)]) -- 0x63c5500 con 0x507ac00 2026-02-11T23:31:28.083+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.083+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.083+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.084+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 475 ==== MOSDPGPush(4.0 119/117 [PushOp(4:5a01aba7:::obj211:head, version: 111'211, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a01aba7:::obj211:head@111'211, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3814d00 con 0x507ac00 2026-02-11T23:31:28.084+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:5a01aba7:::obj211:head, version: 111'211, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a01aba7:::obj211:head@111'211, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000016 epoch 119 2026-02-11T23:31:28.084+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:5a01aba7:::obj211:head, version: 111'211, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a01aba7:::obj211:head@111'211, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.084+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.084+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:5a01aba7:::obj211:head, version: 111'211, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a01aba7:::obj211:head@111'211, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.084+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 476 ==== pg_backfill(progress 4.0 e 119/119 lb 4:5946d48f:::obj257:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e100 con 0x507ac00 2026-02-11T23:31:28.084+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:5946d48f:::obj257:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.084+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.084+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:5946d48f:::obj257:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.084+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:5a01aba7:::obj211:head, version: 111'211, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a01aba7:::obj211:head@111'211, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.084+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.084+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:5a01aba7:::obj211:head, version: 111'211, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a01aba7:::obj211:head@111'211, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000041 MOSDPGPush(4.0 119/117 [PushOp(4:5a01aba7:::obj211:head, version: 111'211, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a01aba7:::obj211:head@111'211, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:58980d3a:::obj339:head local-lis/les=117/118 n=217 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.084+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.084+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:58980d3a:::obj339:head local-lis/les=117/118 n=217 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:5a01aba7:::obj211:head, version: 111'211, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a01aba7:::obj211:head@111'211, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.084+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:58980d3a:::obj339:head local-lis/les=117/118 n=217 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:5a01aba7:::obj211:head@111'211, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.084+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.084+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:5946d48f:::obj257:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.084+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:58980d3a:::obj339:head local-lis/les=117/118 n=217 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:5a01aba7:::obj211:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.084+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:58980d3a:::obj339:head local-lis/les=117/118 n=217 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:5a01aba7:::obj211:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.084+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:58980d3a:::obj339:head local-lis/les=117/118 n=217 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:5a01aba7:::obj211:head 2026-02-11T23:31:28.084+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:58980d3a:::obj339:head local-lis/les=117/118 n=217 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:5a01aba7:::obj211:head v 111'211 2026-02-11T23:31:28.084+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:58980d3a:::obj339:head local-lis/les=117/118 n=217 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.084+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.084+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:5a01aba7:::obj211:head, version: 111'211, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a01aba7:::obj211:head@111'211, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:28.084+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.084+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.084+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:5946d48f:::obj257:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.084+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:5946d48f:::obj257:head) v3 prio 15 cost 0 latency 0.000080 pg_backfill(progress 4.0 e 119/119 lb 4:5946d48f:::obj257:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:58980d3a:::obj339:head local-lis/les=117/118 n=217 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.084+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.084+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:58980d3a:::obj339:head local-lis/les=117/118 n=217 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:5946d48f:::obj257:head) v3 2026-02-11T23:31:28.084+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:58980d3a:::obj339:head local-lis/les=117/118 n=217 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:5946d48f:::obj257:head) 2026-02-11T23:31:28.084+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.084+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:5946d48f:::obj257:head) v3 finish latency 0.000015 2026-02-11T23:31:28.084+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5946d48f:::obj257:head local-lis/les=117/118 n=218 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.084+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.084+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.084+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5946d48f:::obj257:head local-lis/les=117/118 n=218 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.084+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.084+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.084+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.084+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c9080 con 0x507ac00 2026-02-11T23:31:28.084+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:5a01aba7:::obj211:head)]) -- 0x63c4900 con 0x507ac00 2026-02-11T23:31:28.084+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.085+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 477 ==== MOSDPGPush(4.0 119/117 [PushOp(4:5a322cd1:::obj402:head, version: 111'402, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a322cd1:::obj402:head@111'402, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c340 con 0x507ac00 2026-02-11T23:31:28.085+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:5a322cd1:::obj402:head, version: 111'402, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a322cd1:::obj402:head@111'402, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000022 epoch 119 2026-02-11T23:31:28.085+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:5a322cd1:::obj402:head, version: 111'402, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a322cd1:::obj402:head@111'402, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.085+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.085+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:5a322cd1:::obj402:head, version: 111'402, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a322cd1:::obj402:head@111'402, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.085+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 478 ==== pg_backfill(progress 4.0 e 119/119 lb 4:5a01aba7:::obj211:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e680 con 0x507ac00 2026-02-11T23:31:28.085+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.085+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:5a01aba7:::obj211:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 119 2026-02-11T23:31:28.085+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:5a322cd1:::obj402:head, version: 111'402, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a322cd1:::obj402:head@111'402, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.085+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:5a01aba7:::obj211:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.085+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.085+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:5a01aba7:::obj211:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.085+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:5a322cd1:::obj402:head, version: 111'402, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a322cd1:::obj402:head@111'402, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000058 MOSDPGPush(4.0 119/117 [PushOp(4:5a322cd1:::obj402:head, version: 111'402, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a322cd1:::obj402:head@111'402, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:5946d48f:::obj257:head local-lis/les=117/118 n=218 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.085+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.085+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5946d48f:::obj257:head local-lis/les=117/118 n=218 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:5a322cd1:::obj402:head, version: 111'402, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a322cd1:::obj402:head@111'402, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.085+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5946d48f:::obj257:head local-lis/les=117/118 n=218 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:5a322cd1:::obj402:head@111'402, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.085+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5946d48f:::obj257:head local-lis/les=117/118 n=218 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:5a322cd1:::obj402:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.085+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5946d48f:::obj257:head local-lis/les=117/118 n=218 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:5a322cd1:::obj402:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.085+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5946d48f:::obj257:head local-lis/les=117/118 n=218 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:5a322cd1:::obj402:head 2026-02-11T23:31:28.085+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5946d48f:::obj257:head local-lis/les=117/118 n=218 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:5a322cd1:::obj402:head v 111'402 2026-02-11T23:31:28.085+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5946d48f:::obj257:head local-lis/les=117/118 n=218 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.085+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.085+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:5a322cd1:::obj402:head, version: 111'402, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a322cd1:::obj402:head@111'402, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000067 2026-02-11T23:31:28.085+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.085+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.085+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:5a01aba7:::obj211:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.085+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:5a01aba7:::obj211:head) v3 prio 15 cost 0 latency 0.000086 pg_backfill(progress 4.0 e 119/119 lb 4:5a01aba7:::obj211:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:5946d48f:::obj257:head local-lis/les=117/118 n=218 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.085+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.085+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5946d48f:::obj257:head local-lis/les=117/118 n=218 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:5a01aba7:::obj211:head) v3 2026-02-11T23:31:28.085+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5946d48f:::obj257:head local-lis/les=117/118 n=218 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:5a01aba7:::obj211:head) 2026-02-11T23:31:28.085+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.085+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:5a01aba7:::obj211:head) v3 finish latency 0.000015 2026-02-11T23:31:28.085+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5a01aba7:::obj211:head local-lis/les=117/118 n=219 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.085+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.085+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.085+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5a01aba7:::obj211:head local-lis/les=117/118 n=219 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.085+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.085+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.085+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.085+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c4180 con 0x507ac00 2026-02-11T23:31:28.085+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:5a322cd1:::obj402:head)]) -- 0x649ea80 con 0x507ac00 2026-02-11T23:31:28.085+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.085+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 479 ==== MOSDPGPush(4.0 119/117 [PushOp(4:5a3c6224:::obj352:head, version: 111'352, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a3c6224:::obj352:head@111'352, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f1520 con 0x507ac00 2026-02-11T23:31:28.085+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:5a3c6224:::obj352:head, version: 111'352, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a3c6224:::obj352:head@111'352, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000015 epoch 119 2026-02-11T23:31:28.085+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:5a3c6224:::obj352:head, version: 111'352, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a3c6224:::obj352:head@111'352, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.085+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.085+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 480 ==== pg_backfill(progress 4.0 e 119/119 lb 4:5a322cd1:::obj402:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2d080 con 0x507ac00 2026-02-11T23:31:28.085+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:5a3c6224:::obj352:head, version: 111'352, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a3c6224:::obj352:head@111'352, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.085+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:5a322cd1:::obj402:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.085+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:5a322cd1:::obj402:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.085+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.085+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:5a3c6224:::obj352:head, version: 111'352, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a3c6224:::obj352:head@111'352, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.085+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.085+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:5a3c6224:::obj352:head, version: 111'352, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a3c6224:::obj352:head@111'352, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(4.0 119/117 [PushOp(4:5a3c6224:::obj352:head, version: 111'352, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a3c6224:::obj352:head@111'352, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:5a01aba7:::obj211:head local-lis/les=117/118 n=219 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.085+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.085+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5a01aba7:::obj211:head local-lis/les=117/118 n=219 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:5a3c6224:::obj352:head, version: 111'352, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a3c6224:::obj352:head@111'352, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.085+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5a01aba7:::obj211:head local-lis/les=117/118 n=219 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:5a3c6224:::obj352:head@111'352, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.085+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.085+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:5a322cd1:::obj402:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.085+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5a01aba7:::obj211:head local-lis/les=117/118 n=219 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:5a3c6224:::obj352:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.085+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5a01aba7:::obj211:head local-lis/les=117/118 n=219 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:5a3c6224:::obj352:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.085+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5a01aba7:::obj211:head local-lis/les=117/118 n=219 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:5a3c6224:::obj352:head 2026-02-11T23:31:28.085+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5a01aba7:::obj211:head local-lis/les=117/118 n=219 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:5a3c6224:::obj352:head v 111'352 2026-02-11T23:31:28.085+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5a01aba7:::obj211:head local-lis/les=117/118 n=219 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.085+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.085+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:5a3c6224:::obj352:head, version: 111'352, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a3c6224:::obj352:head@111'352, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:28.085+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.085+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.085+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:5a322cd1:::obj402:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.085+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:5a322cd1:::obj402:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 4.0 e 119/119 lb 4:5a322cd1:::obj402:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:5a01aba7:::obj211:head local-lis/les=117/118 n=219 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.085+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.085+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5a01aba7:::obj211:head local-lis/les=117/118 n=219 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:5a322cd1:::obj402:head) v3 2026-02-11T23:31:28.085+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5a01aba7:::obj211:head local-lis/les=117/118 n=219 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:5a322cd1:::obj402:head) 2026-02-11T23:31:28.085+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.085+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:5a322cd1:::obj402:head) v3 finish latency 0.000015 2026-02-11T23:31:28.085+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5a322cd1:::obj402:head local-lis/les=117/118 n=220 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.085+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.085+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.085+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5a322cd1:::obj402:head local-lis/les=117/118 n=220 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.085+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.085+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.085+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.085+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7b080 con 0x507ac00 2026-02-11T23:31:28.085+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:5a3c6224:::obj352:head)]) -- 0x4f43b00 con 0x507ac00 2026-02-11T23:31:28.085+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.086+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 481 ==== MOSDPGPush(4.0 119/117 [PushOp(4:5d17b8a1:::obj341:head, version: 111'341, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5d17b8a1:::obj341:head@111'341, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f0d00 con 0x507ac00 2026-02-11T23:31:28.086+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:5d17b8a1:::obj341:head, version: 111'341, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5d17b8a1:::obj341:head@111'341, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.086+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:5d17b8a1:::obj341:head, version: 111'341, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5d17b8a1:::obj341:head@111'341, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.086+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.086+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 482 ==== pg_backfill(progress 4.0 e 119/119 lb 4:5a3c6224:::obj352:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2f700 con 0x507ac00 2026-02-11T23:31:28.086+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:5d17b8a1:::obj341:head, version: 111'341, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5d17b8a1:::obj341:head@111'341, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.086+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:5a3c6224:::obj352:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.086+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:5a3c6224:::obj352:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.086+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.086+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:5d17b8a1:::obj341:head, version: 111'341, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5d17b8a1:::obj341:head@111'341, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.086+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:5d17b8a1:::obj341:head, version: 111'341, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5d17b8a1:::obj341:head@111'341, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(4.0 119/117 [PushOp(4:5d17b8a1:::obj341:head, version: 111'341, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5d17b8a1:::obj341:head@111'341, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:5a322cd1:::obj402:head local-lis/les=117/118 n=220 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.086+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.086+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.086+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:5a3c6224:::obj352:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.086+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5a322cd1:::obj402:head local-lis/les=117/118 n=220 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:5d17b8a1:::obj341:head, version: 111'341, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5d17b8a1:::obj341:head@111'341, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.086+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5a322cd1:::obj402:head local-lis/les=117/118 n=220 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:5d17b8a1:::obj341:head@111'341, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.086+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5a322cd1:::obj402:head local-lis/les=117/118 n=220 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:5d17b8a1:::obj341:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.086+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5a322cd1:::obj402:head local-lis/les=117/118 n=220 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:5d17b8a1:::obj341:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.086+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5a322cd1:::obj402:head local-lis/les=117/118 n=220 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:5d17b8a1:::obj341:head 2026-02-11T23:31:28.086+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5a322cd1:::obj402:head local-lis/les=117/118 n=220 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:5d17b8a1:::obj341:head v 111'341 2026-02-11T23:31:28.086+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5a322cd1:::obj402:head local-lis/les=117/118 n=220 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.086+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.086+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:5d17b8a1:::obj341:head, version: 111'341, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5d17b8a1:::obj341:head@111'341, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:28.086+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.086+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.086+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:5a3c6224:::obj352:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.086+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:5a3c6224:::obj352:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 4.0 e 119/119 lb 4:5a3c6224:::obj352:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:5a322cd1:::obj402:head local-lis/les=117/118 n=220 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.086+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.086+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5a322cd1:::obj402:head local-lis/les=117/118 n=220 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:5a3c6224:::obj352:head) v3 2026-02-11T23:31:28.086+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5a322cd1:::obj402:head local-lis/les=117/118 n=220 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:5a3c6224:::obj352:head) 2026-02-11T23:31:28.086+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.086+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:5a3c6224:::obj352:head) v3 finish latency 0.000013 2026-02-11T23:31:28.086+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5a3c6224:::obj352:head local-lis/les=117/118 n=221 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.086+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.086+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.086+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5a3c6224:::obj352:head local-lis/les=117/118 n=221 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.086+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.086+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.086+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.086+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7a780 con 0x507ac00 2026-02-11T23:31:28.086+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:5d17b8a1:::obj341:head)]) -- 0x4f43e00 con 0x507ac00 2026-02-11T23:31:28.086+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.087+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 483 ==== MOSDPGPush(4.0 119/117 [PushOp(4:5e27c68c:::obj582:head, version: 111'582, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5e27c68c:::obj582:head@111'582, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56cab60 con 0x507ac00 2026-02-11T23:31:28.087+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:5e27c68c:::obj582:head, version: 111'582, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5e27c68c:::obj582:head@111'582, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 119 2026-02-11T23:31:28.087+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:5e27c68c:::obj582:head, version: 111'582, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5e27c68c:::obj582:head@111'582, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.087+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.087+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 484 ==== pg_backfill(progress 4.0 e 119/119 lb 4:5d17b8a1:::obj341:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f11600 con 0x507ac00 2026-02-11T23:31:28.087+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:5e27c68c:::obj582:head, version: 111'582, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5e27c68c:::obj582:head@111'582, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.087+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:5d17b8a1:::obj341:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.087+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:5d17b8a1:::obj341:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.087+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.087+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:5e27c68c:::obj582:head, version: 111'582, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5e27c68c:::obj582:head@111'582, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.087+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:5e27c68c:::obj582:head, version: 111'582, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5e27c68c:::obj582:head@111'582, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(4.0 119/117 [PushOp(4:5e27c68c:::obj582:head, version: 111'582, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5e27c68c:::obj582:head@111'582, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:5a3c6224:::obj352:head local-lis/les=117/118 n=221 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.087+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.087+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.087+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:5d17b8a1:::obj341:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.087+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5a3c6224:::obj352:head local-lis/les=117/118 n=221 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:5e27c68c:::obj582:head, version: 111'582, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5e27c68c:::obj582:head@111'582, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.087+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5a3c6224:::obj352:head local-lis/les=117/118 n=221 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:5e27c68c:::obj582:head@111'582, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.087+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5a3c6224:::obj352:head local-lis/les=117/118 n=221 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:5e27c68c:::obj582:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.087+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5a3c6224:::obj352:head local-lis/les=117/118 n=221 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:5e27c68c:::obj582:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.087+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5a3c6224:::obj352:head local-lis/les=117/118 n=221 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:5e27c68c:::obj582:head 2026-02-11T23:31:28.087+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5a3c6224:::obj352:head local-lis/les=117/118 n=221 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:5e27c68c:::obj582:head v 111'582 2026-02-11T23:31:28.087+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5a3c6224:::obj352:head local-lis/les=117/118 n=221 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.087+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.087+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:5e27c68c:::obj582:head, version: 111'582, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5e27c68c:::obj582:head@111'582, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000062 2026-02-11T23:31:28.087+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.087+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.087+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:5d17b8a1:::obj341:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.087+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:5d17b8a1:::obj341:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 4.0 e 119/119 lb 4:5d17b8a1:::obj341:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:5a3c6224:::obj352:head local-lis/les=117/118 n=221 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.087+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.087+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5a3c6224:::obj352:head local-lis/les=117/118 n=221 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:5d17b8a1:::obj341:head) v3 2026-02-11T23:31:28.087+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5a3c6224:::obj352:head local-lis/les=117/118 n=221 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:5d17b8a1:::obj341:head) 2026-02-11T23:31:28.087+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.087+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:5d17b8a1:::obj341:head) v3 finish latency 0.000013 2026-02-11T23:31:28.087+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5d17b8a1:::obj341:head local-lis/les=117/118 n=222 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.087+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.087+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.087+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5d17b8a1:::obj341:head local-lis/les=117/118 n=222 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.087+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.087+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.087+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.087+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x644de00 con 0x507ac00 2026-02-11T23:31:28.087+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:5e27c68c:::obj582:head)]) -- 0x63c8000 con 0x507ac00 2026-02-11T23:31:28.087+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.087+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 485 ==== MOSDPGPush(4.0 119/117 [PushOp(4:6131e0b1:::obj290:head, version: 111'290, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6131e0b1:::obj290:head@111'290, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2680 con 0x507ac00 2026-02-11T23:31:28.087+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:6131e0b1:::obj290:head, version: 111'290, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6131e0b1:::obj290:head@111'290, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.087+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:6131e0b1:::obj290:head, version: 111'290, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6131e0b1:::obj290:head@111'290, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.087+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.087+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 486 ==== pg_backfill(progress 4.0 e 119/119 lb 4:5e27c68c:::obj582:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f11080 con 0x507ac00 2026-02-11T23:31:28.087+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:5e27c68c:::obj582:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.087+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:6131e0b1:::obj290:head, version: 111'290, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6131e0b1:::obj290:head@111'290, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.087+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:5e27c68c:::obj582:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.087+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.087+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:6131e0b1:::obj290:head, version: 111'290, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6131e0b1:::obj290:head@111'290, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.087+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:6131e0b1:::obj290:head, version: 111'290, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6131e0b1:::obj290:head@111'290, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 119/117 [PushOp(4:6131e0b1:::obj290:head, version: 111'290, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6131e0b1:::obj290:head@111'290, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:5d17b8a1:::obj341:head local-lis/les=117/118 n=222 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.087+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.087+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.087+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:5e27c68c:::obj582:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.087+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5d17b8a1:::obj341:head local-lis/les=117/118 n=222 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:6131e0b1:::obj290:head, version: 111'290, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6131e0b1:::obj290:head@111'290, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.087+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5d17b8a1:::obj341:head local-lis/les=117/118 n=222 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:6131e0b1:::obj290:head@111'290, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.087+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5d17b8a1:::obj341:head local-lis/les=117/118 n=222 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:6131e0b1:::obj290:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.087+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5d17b8a1:::obj341:head local-lis/les=117/118 n=222 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:6131e0b1:::obj290:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.087+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5d17b8a1:::obj341:head local-lis/les=117/118 n=222 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:6131e0b1:::obj290:head 2026-02-11T23:31:28.087+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5d17b8a1:::obj341:head local-lis/les=117/118 n=222 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:6131e0b1:::obj290:head v 111'290 2026-02-11T23:31:28.087+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5d17b8a1:::obj341:head local-lis/les=117/118 n=222 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.087+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.087+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:6131e0b1:::obj290:head, version: 111'290, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6131e0b1:::obj290:head@111'290, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:28.087+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.087+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.087+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:5e27c68c:::obj582:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.087+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:5e27c68c:::obj582:head) v3 prio 15 cost 0 latency 0.000082 pg_backfill(progress 4.0 e 119/119 lb 4:5e27c68c:::obj582:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:5d17b8a1:::obj341:head local-lis/les=117/118 n=222 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.087+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.087+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5d17b8a1:::obj341:head local-lis/les=117/118 n=222 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:5e27c68c:::obj582:head) v3 2026-02-11T23:31:28.087+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5d17b8a1:::obj341:head local-lis/les=117/118 n=222 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:5e27c68c:::obj582:head) 2026-02-11T23:31:28.087+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.087+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:5e27c68c:::obj582:head) v3 finish latency 0.000015 2026-02-11T23:31:28.087+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5e27c68c:::obj582:head local-lis/les=117/118 n=223 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.087+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.087+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.087+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5e27c68c:::obj582:head local-lis/les=117/118 n=223 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.087+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.087+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.087+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.087+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c9200 con 0x507ac00 2026-02-11T23:31:28.087+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:6131e0b1:::obj290:head)]) -- 0x5f2dc80 con 0x507ac00 2026-02-11T23:31:28.087+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.088+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 487 ==== MOSDPGPush(4.0 119/117 [PushOp(4:614752af:::obj331:head, version: 111'331, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:614752af:::obj331:head@111'331, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db0b60 con 0x507ac00 2026-02-11T23:31:28.088+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:614752af:::obj331:head, version: 111'331, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:614752af:::obj331:head@111'331, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.088+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:614752af:::obj331:head, version: 111'331, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:614752af:::obj331:head@111'331, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.088+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.088+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 488 ==== pg_backfill(progress 4.0 e 119/119 lb 4:6131e0b1:::obj290:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f10b00 con 0x507ac00 2026-02-11T23:31:28.088+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:6131e0b1:::obj290:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.088+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:614752af:::obj331:head, version: 111'331, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:614752af:::obj331:head@111'331, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.088+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:6131e0b1:::obj290:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.088+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.088+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:614752af:::obj331:head, version: 111'331, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:614752af:::obj331:head@111'331, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.088+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.088+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:614752af:::obj331:head, version: 111'331, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:614752af:::obj331:head@111'331, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 119/117 [PushOp(4:614752af:::obj331:head, version: 111'331, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:614752af:::obj331:head@111'331, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:5e27c68c:::obj582:head local-lis/les=117/118 n=223 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.088+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.088+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:6131e0b1:::obj290:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.088+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5e27c68c:::obj582:head local-lis/les=117/118 n=223 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:614752af:::obj331:head, version: 111'331, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:614752af:::obj331:head@111'331, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.088+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5e27c68c:::obj582:head local-lis/les=117/118 n=223 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:614752af:::obj331:head@111'331, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.088+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5e27c68c:::obj582:head local-lis/les=117/118 n=223 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:614752af:::obj331:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.088+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5e27c68c:::obj582:head local-lis/les=117/118 n=223 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:614752af:::obj331:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.088+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5e27c68c:::obj582:head local-lis/les=117/118 n=223 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:614752af:::obj331:head 2026-02-11T23:31:28.088+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5e27c68c:::obj582:head local-lis/les=117/118 n=223 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:614752af:::obj331:head v 111'331 2026-02-11T23:31:28.088+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5e27c68c:::obj582:head local-lis/les=117/118 n=223 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.088+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.088+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:614752af:::obj331:head, version: 111'331, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:614752af:::obj331:head@111'331, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:28.088+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.088+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.088+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:6131e0b1:::obj290:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.088+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:6131e0b1:::obj290:head) v3 prio 15 cost 0 latency 0.000085 pg_backfill(progress 4.0 e 119/119 lb 4:6131e0b1:::obj290:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:5e27c68c:::obj582:head local-lis/les=117/118 n=223 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.088+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.088+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5e27c68c:::obj582:head local-lis/les=117/118 n=223 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:6131e0b1:::obj290:head) v3 2026-02-11T23:31:28.088+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:5e27c68c:::obj582:head local-lis/les=117/118 n=223 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:6131e0b1:::obj290:head) 2026-02-11T23:31:28.088+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.088+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:6131e0b1:::obj290:head) v3 finish latency 0.000015 2026-02-11T23:31:28.088+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6131e0b1:::obj290:head local-lis/les=117/118 n=224 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.088+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.088+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.088+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6131e0b1:::obj290:head local-lis/les=117/118 n=224 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.088+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.088+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.088+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.088+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x644db00 con 0x507ac00 2026-02-11T23:31:28.088+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:614752af:::obj331:head)]) -- 0x63c5080 con 0x507ac00 2026-02-11T23:31:28.088+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.088+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 489 ==== MOSDPGPush(4.0 119/117 [PushOp(4:61741462:::obj327:head, version: 111'327, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:61741462:::obj327:head@111'327, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e26b60 con 0x507ac00 2026-02-11T23:31:28.088+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:61741462:::obj327:head, version: 111'327, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:61741462:::obj327:head@111'327, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.088+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:61741462:::obj327:head, version: 111'327, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:61741462:::obj327:head@111'327, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.088+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.088+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 490 ==== pg_backfill(progress 4.0 e 119/119 lb 4:614752af:::obj331:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f10580 con 0x507ac00 2026-02-11T23:31:28.088+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:61741462:::obj327:head, version: 111'327, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:61741462:::obj327:head@111'327, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.088+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:614752af:::obj331:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.088+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:614752af:::obj331:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.088+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.088+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:61741462:::obj327:head, version: 111'327, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:61741462:::obj327:head@111'327, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.088+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:61741462:::obj327:head, version: 111'327, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:61741462:::obj327:head@111'327, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(4.0 119/117 [PushOp(4:61741462:::obj327:head, version: 111'327, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:61741462:::obj327:head@111'327, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:6131e0b1:::obj290:head local-lis/les=117/118 n=224 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.088+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.088+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.088+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:614752af:::obj331:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.088+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6131e0b1:::obj290:head local-lis/les=117/118 n=224 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:61741462:::obj327:head, version: 111'327, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:61741462:::obj327:head@111'327, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.088+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6131e0b1:::obj290:head local-lis/les=117/118 n=224 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:61741462:::obj327:head@111'327, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.088+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6131e0b1:::obj290:head local-lis/les=117/118 n=224 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:61741462:::obj327:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.088+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6131e0b1:::obj290:head local-lis/les=117/118 n=224 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:61741462:::obj327:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.088+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6131e0b1:::obj290:head local-lis/les=117/118 n=224 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:61741462:::obj327:head 2026-02-11T23:31:28.088+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6131e0b1:::obj290:head local-lis/les=117/118 n=224 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:61741462:::obj327:head v 111'327 2026-02-11T23:31:28.088+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6131e0b1:::obj290:head local-lis/les=117/118 n=224 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.088+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.088+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:61741462:::obj327:head, version: 111'327, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:61741462:::obj327:head@111'327, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:28.088+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.088+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.088+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:614752af:::obj331:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.088+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:614752af:::obj331:head) v3 prio 15 cost 0 latency 0.000081 pg_backfill(progress 4.0 e 119/119 lb 4:614752af:::obj331:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:6131e0b1:::obj290:head local-lis/les=117/118 n=224 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.088+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.088+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6131e0b1:::obj290:head local-lis/les=117/118 n=224 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:614752af:::obj331:head) v3 2026-02-11T23:31:28.088+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6131e0b1:::obj290:head local-lis/les=117/118 n=224 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:614752af:::obj331:head) 2026-02-11T23:31:28.088+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.088+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:614752af:::obj331:head) v3 finish latency 0.000013 2026-02-11T23:31:28.088+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:614752af:::obj331:head local-lis/les=117/118 n=225 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.088+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.089+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.089+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:614752af:::obj331:head local-lis/les=117/118 n=225 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.089+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.089+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.089+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.089+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7a900 con 0x507ac00 2026-02-11T23:31:28.089+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:61741462:::obj327:head)]) -- 0x5d54000 con 0x507ac00 2026-02-11T23:31:28.089+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.089+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 491 ==== MOSDPGPush(4.0 119/117 [PushOp(4:61dc80df:::obj452:head, version: 111'452, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:61dc80df:::obj452:head@111'452, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a29c0 con 0x507ac00 2026-02-11T23:31:28.089+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:61dc80df:::obj452:head, version: 111'452, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:61dc80df:::obj452:head@111'452, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.089+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:61dc80df:::obj452:head, version: 111'452, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:61dc80df:::obj452:head@111'452, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.089+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.089+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 492 ==== pg_backfill(progress 4.0 e 119/119 lb 4:61741462:::obj327:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f10000 con 0x507ac00 2026-02-11T23:31:28.089+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:61dc80df:::obj452:head, version: 111'452, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:61dc80df:::obj452:head@111'452, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.089+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:61741462:::obj327:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.089+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:61741462:::obj327:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.089+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.089+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:61dc80df:::obj452:head, version: 111'452, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:61dc80df:::obj452:head@111'452, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.089+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:61dc80df:::obj452:head, version: 111'452, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:61dc80df:::obj452:head@111'452, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 119/117 [PushOp(4:61dc80df:::obj452:head, version: 111'452, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:61dc80df:::obj452:head@111'452, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:614752af:::obj331:head local-lis/les=117/118 n=225 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.089+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.089+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.089+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:61741462:::obj327:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.089+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:614752af:::obj331:head local-lis/les=117/118 n=225 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:61dc80df:::obj452:head, version: 111'452, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:61dc80df:::obj452:head@111'452, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.089+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:614752af:::obj331:head local-lis/les=117/118 n=225 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:61dc80df:::obj452:head@111'452, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.089+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:614752af:::obj331:head local-lis/les=117/118 n=225 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:61dc80df:::obj452:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.089+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:614752af:::obj331:head local-lis/les=117/118 n=225 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:61dc80df:::obj452:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.089+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:614752af:::obj331:head local-lis/les=117/118 n=225 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:61dc80df:::obj452:head 2026-02-11T23:31:28.089+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:614752af:::obj331:head local-lis/les=117/118 n=225 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:61dc80df:::obj452:head v 111'452 2026-02-11T23:31:28.089+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:614752af:::obj331:head local-lis/les=117/118 n=225 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.089+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.089+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:61dc80df:::obj452:head, version: 111'452, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:61dc80df:::obj452:head@111'452, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:28.089+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.089+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.089+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:61741462:::obj327:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.089+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:61741462:::obj327:head) v3 prio 15 cost 0 latency 0.000080 pg_backfill(progress 4.0 e 119/119 lb 4:61741462:::obj327:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:614752af:::obj331:head local-lis/les=117/118 n=225 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.089+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.089+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:614752af:::obj331:head local-lis/les=117/118 n=225 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:61741462:::obj327:head) v3 2026-02-11T23:31:28.089+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:614752af:::obj331:head local-lis/les=117/118 n=225 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:61741462:::obj327:head) 2026-02-11T23:31:28.089+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.089+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:61741462:::obj327:head) v3 finish latency 0.000016 2026-02-11T23:31:28.089+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:61741462:::obj327:head local-lis/les=117/118 n=226 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.089+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.089+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.089+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:61741462:::obj327:head local-lis/les=117/118 n=226 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.089+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.089+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.089+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.089+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x644d800 con 0x507ac00 2026-02-11T23:31:28.089+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:61dc80df:::obj452:head)]) -- 0x5d54300 con 0x507ac00 2026-02-11T23:31:28.089+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.090+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 493 ==== MOSDPGPush(4.0 119/117 [PushOp(4:622f1330:::obj230:head, version: 111'230, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:622f1330:::obj230:head@111'230, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5617520 con 0x507ac00 2026-02-11T23:31:28.090+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:622f1330:::obj230:head, version: 111'230, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:622f1330:::obj230:head@111'230, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.090+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:622f1330:::obj230:head, version: 111'230, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:622f1330:::obj230:head@111'230, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.090+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.090+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 494 ==== pg_backfill(progress 4.0 e 119/119 lb 4:61dc80df:::obj452:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f13700 con 0x507ac00 2026-02-11T23:31:28.090+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:61dc80df:::obj452:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.090+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:622f1330:::obj230:head, version: 111'230, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:622f1330:::obj230:head@111'230, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.090+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:61dc80df:::obj452:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.090+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.090+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:622f1330:::obj230:head, version: 111'230, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:622f1330:::obj230:head@111'230, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.090+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:622f1330:::obj230:head, version: 111'230, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:622f1330:::obj230:head@111'230, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 119/117 [PushOp(4:622f1330:::obj230:head, version: 111'230, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:622f1330:::obj230:head@111'230, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:61741462:::obj327:head local-lis/les=117/118 n=226 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.090+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.090+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.090+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:61dc80df:::obj452:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.090+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:61741462:::obj327:head local-lis/les=117/118 n=226 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:622f1330:::obj230:head, version: 111'230, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:622f1330:::obj230:head@111'230, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.090+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:61741462:::obj327:head local-lis/les=117/118 n=226 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:622f1330:::obj230:head@111'230, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.090+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:61741462:::obj327:head local-lis/les=117/118 n=226 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:622f1330:::obj230:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.090+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:61741462:::obj327:head local-lis/les=117/118 n=226 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:622f1330:::obj230:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.090+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:61741462:::obj327:head local-lis/les=117/118 n=226 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:622f1330:::obj230:head 2026-02-11T23:31:28.090+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:61741462:::obj327:head local-lis/les=117/118 n=226 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:622f1330:::obj230:head v 111'230 2026-02-11T23:31:28.090+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:61741462:::obj327:head local-lis/les=117/118 n=226 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.090+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.090+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:622f1330:::obj230:head, version: 111'230, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:622f1330:::obj230:head@111'230, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:28.090+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.090+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.090+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:61dc80df:::obj452:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.090+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:61dc80df:::obj452:head) v3 prio 15 cost 0 latency 0.000085 pg_backfill(progress 4.0 e 119/119 lb 4:61dc80df:::obj452:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:61741462:::obj327:head local-lis/les=117/118 n=226 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.090+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.090+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:61741462:::obj327:head local-lis/les=117/118 n=226 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:61dc80df:::obj452:head) v3 2026-02-11T23:31:28.090+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:61741462:::obj327:head local-lis/les=117/118 n=226 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:61dc80df:::obj452:head) 2026-02-11T23:31:28.090+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.090+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:61dc80df:::obj452:head) v3 finish latency 0.000013 2026-02-11T23:31:28.090+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:61dc80df:::obj452:head local-lis/les=117/118 n=227 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.090+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.090+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.090+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:61dc80df:::obj452:head local-lis/les=117/118 n=227 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.090+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.090+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.090+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.090+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x649e600 con 0x507ac00 2026-02-11T23:31:28.090+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:622f1330:::obj230:head)]) -- 0x5d54600 con 0x507ac00 2026-02-11T23:31:28.090+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.090+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 495 ==== MOSDPGPush(4.0 119/117 [PushOp(4:624e19c5:::obj464:head, version: 111'464, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:624e19c5:::obj464:head@111'464, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56ca680 con 0x507ac00 2026-02-11T23:31:28.090+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:624e19c5:::obj464:head, version: 111'464, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:624e19c5:::obj464:head@111'464, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.090+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:624e19c5:::obj464:head, version: 111'464, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:624e19c5:::obj464:head@111'464, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.090+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.090+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 496 ==== pg_backfill(progress 4.0 e 119/119 lb 4:622f1330:::obj230:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f12c00 con 0x507ac00 2026-02-11T23:31:28.090+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:622f1330:::obj230:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.090+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:624e19c5:::obj464:head, version: 111'464, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:624e19c5:::obj464:head@111'464, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.090+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:622f1330:::obj230:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.090+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.090+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:624e19c5:::obj464:head, version: 111'464, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:624e19c5:::obj464:head@111'464, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.090+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:624e19c5:::obj464:head, version: 111'464, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:624e19c5:::obj464:head@111'464, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(4.0 119/117 [PushOp(4:624e19c5:::obj464:head, version: 111'464, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:624e19c5:::obj464:head@111'464, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:61dc80df:::obj452:head local-lis/les=117/118 n=227 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.090+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.090+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.090+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:622f1330:::obj230:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.090+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:61dc80df:::obj452:head local-lis/les=117/118 n=227 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:624e19c5:::obj464:head, version: 111'464, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:624e19c5:::obj464:head@111'464, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.090+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:61dc80df:::obj452:head local-lis/les=117/118 n=227 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:624e19c5:::obj464:head@111'464, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.090+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:61dc80df:::obj452:head local-lis/les=117/118 n=227 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:624e19c5:::obj464:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.090+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:61dc80df:::obj452:head local-lis/les=117/118 n=227 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:624e19c5:::obj464:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.090+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:61dc80df:::obj452:head local-lis/les=117/118 n=227 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:624e19c5:::obj464:head 2026-02-11T23:31:28.090+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:61dc80df:::obj452:head local-lis/les=117/118 n=227 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:624e19c5:::obj464:head v 111'464 2026-02-11T23:31:28.090+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:61dc80df:::obj452:head local-lis/les=117/118 n=227 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.090+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.090+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:624e19c5:::obj464:head, version: 111'464, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:624e19c5:::obj464:head@111'464, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:28.090+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.090+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.090+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:622f1330:::obj230:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.090+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:622f1330:::obj230:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 4.0 e 119/119 lb 4:622f1330:::obj230:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:61dc80df:::obj452:head local-lis/les=117/118 n=227 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.090+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.090+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:61dc80df:::obj452:head local-lis/les=117/118 n=227 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:622f1330:::obj230:head) v3 2026-02-11T23:31:28.090+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:61dc80df:::obj452:head local-lis/les=117/118 n=227 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:622f1330:::obj230:head) 2026-02-11T23:31:28.090+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.090+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:622f1330:::obj230:head) v3 finish latency 0.000014 2026-02-11T23:31:28.090+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:622f1330:::obj230:head local-lis/les=117/118 n=228 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.090+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.090+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.090+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:622f1330:::obj230:head local-lis/les=117/118 n=228 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.090+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.090+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.090+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.090+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x644d680 con 0x507ac00 2026-02-11T23:31:28.090+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:624e19c5:::obj464:head)]) -- 0x5d54900 con 0x507ac00 2026-02-11T23:31:28.090+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.091+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 497 ==== MOSDPGPush(4.0 119/117 [PushOp(4:62eb9ce0:::obj205:head, version: 111'205, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:62eb9ce0:::obj205:head@111'205, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5646d00 con 0x507ac00 2026-02-11T23:31:28.091+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:62eb9ce0:::obj205:head, version: 111'205, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:62eb9ce0:::obj205:head@111'205, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 119 2026-02-11T23:31:28.091+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:62eb9ce0:::obj205:head, version: 111'205, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:62eb9ce0:::obj205:head@111'205, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.091+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.091+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 498 ==== pg_backfill(progress 4.0 e 119/119 lb 4:624e19c5:::obj464:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f12680 con 0x507ac00 2026-02-11T23:31:28.091+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:62eb9ce0:::obj205:head, version: 111'205, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:62eb9ce0:::obj205:head@111'205, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.091+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:624e19c5:::obj464:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 119 2026-02-11T23:31:28.091+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.091+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:624e19c5:::obj464:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.091+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:62eb9ce0:::obj205:head, version: 111'205, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:62eb9ce0:::obj205:head@111'205, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.091+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.091+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:62eb9ce0:::obj205:head, version: 111'205, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:62eb9ce0:::obj205:head@111'205, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 119/117 [PushOp(4:62eb9ce0:::obj205:head, version: 111'205, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:62eb9ce0:::obj205:head@111'205, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:622f1330:::obj230:head local-lis/les=117/118 n=228 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.091+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.091+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:622f1330:::obj230:head local-lis/les=117/118 n=228 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:62eb9ce0:::obj205:head, version: 111'205, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:62eb9ce0:::obj205:head@111'205, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.091+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:622f1330:::obj230:head local-lis/les=117/118 n=228 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:62eb9ce0:::obj205:head@111'205, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.091+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.091+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:624e19c5:::obj464:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.091+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:622f1330:::obj230:head local-lis/les=117/118 n=228 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:62eb9ce0:::obj205:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.091+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:622f1330:::obj230:head local-lis/les=117/118 n=228 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:62eb9ce0:::obj205:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.091+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:622f1330:::obj230:head local-lis/les=117/118 n=228 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:62eb9ce0:::obj205:head 2026-02-11T23:31:28.091+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:622f1330:::obj230:head local-lis/les=117/118 n=228 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:62eb9ce0:::obj205:head v 111'205 2026-02-11T23:31:28.091+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:622f1330:::obj230:head local-lis/les=117/118 n=228 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.091+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.091+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:62eb9ce0:::obj205:head, version: 111'205, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:62eb9ce0:::obj205:head@111'205, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000064 2026-02-11T23:31:28.091+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.091+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.091+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:624e19c5:::obj464:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.091+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:624e19c5:::obj464:head) v3 prio 15 cost 0 latency 0.000086 pg_backfill(progress 4.0 e 119/119 lb 4:624e19c5:::obj464:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:622f1330:::obj230:head local-lis/les=117/118 n=228 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.091+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.091+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:622f1330:::obj230:head local-lis/les=117/118 n=228 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:624e19c5:::obj464:head) v3 2026-02-11T23:31:28.091+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:622f1330:::obj230:head local-lis/les=117/118 n=228 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:624e19c5:::obj464:head) 2026-02-11T23:31:28.091+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.091+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:624e19c5:::obj464:head) v3 finish latency 0.000014 2026-02-11T23:31:28.091+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:624e19c5:::obj464:head local-lis/les=117/118 n=229 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.091+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.091+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.091+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:624e19c5:::obj464:head local-lis/les=117/118 n=229 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.091+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.091+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.091+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.091+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7aa80 con 0x507ac00 2026-02-11T23:31:28.091+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:62eb9ce0:::obj205:head)]) -- 0x5d54c00 con 0x507ac00 2026-02-11T23:31:28.091+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.091+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 499 ==== MOSDPGPush(4.0 119/117 [PushOp(4:63191178:::obj308:head, version: 111'308, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:63191178:::obj308:head@111'308, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x554e340 con 0x507ac00 2026-02-11T23:31:28.091+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:63191178:::obj308:head, version: 111'308, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:63191178:::obj308:head@111'308, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.091+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:63191178:::obj308:head, version: 111'308, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:63191178:::obj308:head@111'308, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.091+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.091+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 500 ==== pg_backfill(progress 4.0 e 119/119 lb 4:62eb9ce0:::obj205:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f12100 con 0x507ac00 2026-02-11T23:31:28.091+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:63191178:::obj308:head, version: 111'308, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:63191178:::obj308:head@111'308, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.092+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:62eb9ce0:::obj205:head) v3 prio 15 type 95 cost 0 latency 0.000011 epoch 119 2026-02-11T23:31:28.092+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:62eb9ce0:::obj205:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.092+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.092+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:63191178:::obj308:head, version: 111'308, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:63191178:::obj308:head@111'308, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.092+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:63191178:::obj308:head, version: 111'308, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:63191178:::obj308:head@111'308, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(4.0 119/117 [PushOp(4:63191178:::obj308:head, version: 111'308, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:63191178:::obj308:head@111'308, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:624e19c5:::obj464:head local-lis/les=117/118 n=229 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.092+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.092+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:62eb9ce0:::obj205:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.092+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.092+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:624e19c5:::obj464:head local-lis/les=117/118 n=229 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:63191178:::obj308:head, version: 111'308, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:63191178:::obj308:head@111'308, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.092+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:624e19c5:::obj464:head local-lis/les=117/118 n=229 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:63191178:::obj308:head@111'308, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.092+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:624e19c5:::obj464:head local-lis/les=117/118 n=229 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:63191178:::obj308:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.092+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:624e19c5:::obj464:head local-lis/les=117/118 n=229 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:63191178:::obj308:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.092+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:624e19c5:::obj464:head local-lis/les=117/118 n=229 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:63191178:::obj308:head 2026-02-11T23:31:28.092+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:624e19c5:::obj464:head local-lis/les=117/118 n=229 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:63191178:::obj308:head v 111'308 2026-02-11T23:31:28.092+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:624e19c5:::obj464:head local-lis/les=117/118 n=229 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.092+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.092+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:63191178:::obj308:head, version: 111'308, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:63191178:::obj308:head@111'308, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000078 2026-02-11T23:31:28.092+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.092+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.092+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:62eb9ce0:::obj205:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.092+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:62eb9ce0:::obj205:head) v3 prio 15 cost 0 latency 0.000103 pg_backfill(progress 4.0 e 119/119 lb 4:62eb9ce0:::obj205:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:624e19c5:::obj464:head local-lis/les=117/118 n=229 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.092+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.092+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:624e19c5:::obj464:head local-lis/les=117/118 n=229 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:62eb9ce0:::obj205:head) v3 2026-02-11T23:31:28.092+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:624e19c5:::obj464:head local-lis/les=117/118 n=229 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:62eb9ce0:::obj205:head) 2026-02-11T23:31:28.092+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.092+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:62eb9ce0:::obj205:head) v3 finish latency 0.000014 2026-02-11T23:31:28.092+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:62eb9ce0:::obj205:head local-lis/les=117/118 n=230 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.092+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.092+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.092+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:62eb9ce0:::obj205:head local-lis/les=117/118 n=230 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.092+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.092+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.092+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.092+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x644d380 con 0x507ac00 2026-02-11T23:31:28.092+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:63191178:::obj308:head)]) -- 0x5d54f00 con 0x507ac00 2026-02-11T23:31:28.092+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.092+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 501 ==== MOSDPGPush(4.0 119/117 [PushOp(4:640d3668:::obj286:head, version: 111'286, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:640d3668:::obj286:head@111'286, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a29c0 con 0x507ac00 2026-02-11T23:31:28.092+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:640d3668:::obj286:head, version: 111'286, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:640d3668:::obj286:head@111'286, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.092+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:640d3668:::obj286:head, version: 111'286, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:640d3668:::obj286:head@111'286, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.092+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.092+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 502 ==== pg_backfill(progress 4.0 e 119/119 lb 4:63191178:::obj308:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f11b80 con 0x507ac00 2026-02-11T23:31:28.092+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:63191178:::obj308:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.092+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:640d3668:::obj286:head, version: 111'286, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:640d3668:::obj286:head@111'286, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.092+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:63191178:::obj308:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.092+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.092+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:640d3668:::obj286:head, version: 111'286, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:640d3668:::obj286:head@111'286, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.092+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:640d3668:::obj286:head, version: 111'286, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:640d3668:::obj286:head@111'286, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(4.0 119/117 [PushOp(4:640d3668:::obj286:head, version: 111'286, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:640d3668:::obj286:head@111'286, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:62eb9ce0:::obj205:head local-lis/les=117/118 n=230 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.092+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.092+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.092+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:63191178:::obj308:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.092+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:62eb9ce0:::obj205:head local-lis/les=117/118 n=230 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:640d3668:::obj286:head, version: 111'286, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:640d3668:::obj286:head@111'286, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.092+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:62eb9ce0:::obj205:head local-lis/les=117/118 n=230 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:640d3668:::obj286:head@111'286, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.092+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:62eb9ce0:::obj205:head local-lis/les=117/118 n=230 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:640d3668:::obj286:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.092+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:62eb9ce0:::obj205:head local-lis/les=117/118 n=230 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:640d3668:::obj286:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.092+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:62eb9ce0:::obj205:head local-lis/les=117/118 n=230 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:640d3668:::obj286:head 2026-02-11T23:31:28.092+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:62eb9ce0:::obj205:head local-lis/les=117/118 n=230 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:640d3668:::obj286:head v 111'286 2026-02-11T23:31:28.092+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:62eb9ce0:::obj205:head local-lis/les=117/118 n=230 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.092+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.092+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:640d3668:::obj286:head, version: 111'286, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:640d3668:::obj286:head@111'286, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000066 2026-02-11T23:31:28.092+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.092+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.092+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:63191178:::obj308:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.092+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:63191178:::obj308:head) v3 prio 15 cost 0 latency 0.000093 pg_backfill(progress 4.0 e 119/119 lb 4:63191178:::obj308:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:62eb9ce0:::obj205:head local-lis/les=117/118 n=230 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.092+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.092+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:62eb9ce0:::obj205:head local-lis/les=117/118 n=230 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:63191178:::obj308:head) v3 2026-02-11T23:31:28.092+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:62eb9ce0:::obj205:head local-lis/les=117/118 n=230 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:63191178:::obj308:head) 2026-02-11T23:31:28.092+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.092+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:63191178:::obj308:head) v3 finish latency 0.000015 2026-02-11T23:31:28.092+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:63191178:::obj308:head local-lis/les=117/118 n=231 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.092+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.092+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.092+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:63191178:::obj308:head local-lis/les=117/118 n=231 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.092+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.092+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.092+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.092+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c5800 con 0x507ac00 2026-02-11T23:31:28.092+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:640d3668:::obj286:head)]) -- 0x5d55200 con 0x507ac00 2026-02-11T23:31:28.092+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.093+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 503 ==== MOSDPGPush(4.0 119/117 [PushOp(4:649aec78:::obj275:head, version: 111'275, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:649aec78:::obj275:head@111'275, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e26b60 con 0x507ac00 2026-02-11T23:31:28.093+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:649aec78:::obj275:head, version: 111'275, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:649aec78:::obj275:head@111'275, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.093+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:649aec78:::obj275:head, version: 111'275, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:649aec78:::obj275:head@111'275, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.093+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.093+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 504 ==== pg_backfill(progress 4.0 e 119/119 lb 4:640d3668:::obj286:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f13180 con 0x507ac00 2026-02-11T23:31:28.093+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:640d3668:::obj286:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.093+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:649aec78:::obj275:head, version: 111'275, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:649aec78:::obj275:head@111'275, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.093+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:640d3668:::obj286:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.093+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.093+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:649aec78:::obj275:head, version: 111'275, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:649aec78:::obj275:head@111'275, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.093+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.093+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:640d3668:::obj286:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.093+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:649aec78:::obj275:head, version: 111'275, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:649aec78:::obj275:head@111'275, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(4.0 119/117 [PushOp(4:649aec78:::obj275:head, version: 111'275, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:649aec78:::obj275:head@111'275, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:63191178:::obj308:head local-lis/les=117/118 n=231 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.093+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.093+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:63191178:::obj308:head local-lis/les=117/118 n=231 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:649aec78:::obj275:head, version: 111'275, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:649aec78:::obj275:head@111'275, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.093+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:63191178:::obj308:head local-lis/les=117/118 n=231 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:649aec78:::obj275:head@111'275, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.093+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:63191178:::obj308:head local-lis/les=117/118 n=231 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:649aec78:::obj275:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.093+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:63191178:::obj308:head local-lis/les=117/118 n=231 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:649aec78:::obj275:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.093+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:63191178:::obj308:head local-lis/les=117/118 n=231 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:649aec78:::obj275:head 2026-02-11T23:31:28.093+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:63191178:::obj308:head local-lis/les=117/118 n=231 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:649aec78:::obj275:head v 111'275 2026-02-11T23:31:28.093+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:63191178:::obj308:head local-lis/les=117/118 n=231 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.093+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.093+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:649aec78:::obj275:head, version: 111'275, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:649aec78:::obj275:head@111'275, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000070 2026-02-11T23:31:28.093+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.093+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.093+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:640d3668:::obj286:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.093+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:640d3668:::obj286:head) v3 prio 15 cost 0 latency 0.000099 pg_backfill(progress 4.0 e 119/119 lb 4:640d3668:::obj286:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:63191178:::obj308:head local-lis/les=117/118 n=231 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.093+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.093+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:63191178:::obj308:head local-lis/les=117/118 n=231 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:640d3668:::obj286:head) v3 2026-02-11T23:31:28.093+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:63191178:::obj308:head local-lis/les=117/118 n=231 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:640d3668:::obj286:head) 2026-02-11T23:31:28.093+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.093+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:640d3668:::obj286:head) v3 finish latency 0.000015 2026-02-11T23:31:28.093+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:640d3668:::obj286:head local-lis/les=117/118 n=232 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.093+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.093+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.093+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:640d3668:::obj286:head local-lis/les=117/118 n=232 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.093+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.093+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.093+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.093+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x644d080 con 0x507ac00 2026-02-11T23:31:28.093+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:649aec78:::obj275:head)]) -- 0x5d55500 con 0x507ac00 2026-02-11T23:31:28.093+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.093+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 505 ==== MOSDPGPush(4.0 119/117 [PushOp(4:64b337fe:::obj71:head, version: 111'71, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64b337fe:::obj71:head@111'71, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5db0b60 con 0x507ac00 2026-02-11T23:31:28.093+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:64b337fe:::obj71:head, version: 111'71, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64b337fe:::obj71:head@111'71, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.093+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:64b337fe:::obj71:head, version: 111'71, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64b337fe:::obj71:head@111'71, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.093+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.093+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 506 ==== pg_backfill(progress 4.0 e 119/119 lb 4:649aec78:::obj275:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f11080 con 0x507ac00 2026-02-11T23:31:28.093+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:64b337fe:::obj71:head, version: 111'71, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64b337fe:::obj71:head@111'71, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.093+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:649aec78:::obj275:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.093+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:649aec78:::obj275:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.093+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.093+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:64b337fe:::obj71:head, version: 111'71, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64b337fe:::obj71:head@111'71, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.093+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:64b337fe:::obj71:head, version: 111'71, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64b337fe:::obj71:head@111'71, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(4.0 119/117 [PushOp(4:64b337fe:::obj71:head, version: 111'71, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64b337fe:::obj71:head@111'71, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:640d3668:::obj286:head local-lis/les=117/118 n=232 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.093+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.093+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.093+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:649aec78:::obj275:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.093+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:640d3668:::obj286:head local-lis/les=117/118 n=232 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:64b337fe:::obj71:head, version: 111'71, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64b337fe:::obj71:head@111'71, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.093+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:640d3668:::obj286:head local-lis/les=117/118 n=232 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:64b337fe:::obj71:head@111'71, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.093+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:640d3668:::obj286:head local-lis/les=117/118 n=232 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:64b337fe:::obj71:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.093+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:640d3668:::obj286:head local-lis/les=117/118 n=232 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:64b337fe:::obj71:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.093+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:640d3668:::obj286:head local-lis/les=117/118 n=232 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:64b337fe:::obj71:head 2026-02-11T23:31:28.093+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:640d3668:::obj286:head local-lis/les=117/118 n=232 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:64b337fe:::obj71:head v 111'71 2026-02-11T23:31:28.093+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:640d3668:::obj286:head local-lis/les=117/118 n=232 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.093+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.093+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:64b337fe:::obj71:head, version: 111'71, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64b337fe:::obj71:head@111'71, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000065 2026-02-11T23:31:28.093+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.093+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.093+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:649aec78:::obj275:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.093+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:649aec78:::obj275:head) v3 prio 15 cost 0 latency 0.000084 pg_backfill(progress 4.0 e 119/119 lb 4:649aec78:::obj275:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:640d3668:::obj286:head local-lis/les=117/118 n=232 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.093+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.093+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:640d3668:::obj286:head local-lis/les=117/118 n=232 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:649aec78:::obj275:head) v3 2026-02-11T23:31:28.093+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:640d3668:::obj286:head local-lis/les=117/118 n=232 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:649aec78:::obj275:head) 2026-02-11T23:31:28.093+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.093+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:649aec78:::obj275:head) v3 finish latency 0.000015 2026-02-11T23:31:28.093+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:649aec78:::obj275:head local-lis/les=117/118 n=233 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.093+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.094+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.094+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:649aec78:::obj275:head local-lis/les=117/118 n=233 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.094+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.094+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.094+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.094+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7ac00 con 0x507ac00 2026-02-11T23:31:28.094+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:64b337fe:::obj71:head)]) -- 0x5d55800 con 0x507ac00 2026-02-11T23:31:28.094+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.094+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 507 ==== MOSDPGPush(4.0 119/117 [PushOp(4:64bc0383:::obj214:head, version: 111'214, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64bc0383:::obj214:head@111'214, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2680 con 0x507ac00 2026-02-11T23:31:28.094+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:64bc0383:::obj214:head, version: 111'214, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64bc0383:::obj214:head@111'214, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.094+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:64bc0383:::obj214:head, version: 111'214, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64bc0383:::obj214:head@111'214, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.094+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.094+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 508 ==== pg_backfill(progress 4.0 e 119/119 lb 4:64b337fe:::obj71:head) ==== 1061+0+0 (unknown 0 0 0) 0x5f11600 con 0x507ac00 2026-02-11T23:31:28.094+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:64bc0383:::obj214:head, version: 111'214, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64bc0383:::obj214:head@111'214, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.094+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:64b337fe:::obj71:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.094+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.094+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:64b337fe:::obj71:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.094+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:64bc0383:::obj214:head, version: 111'214, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64bc0383:::obj214:head@111'214, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.094+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.094+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:64bc0383:::obj214:head, version: 111'214, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64bc0383:::obj214:head@111'214, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 119/117 [PushOp(4:64bc0383:::obj214:head, version: 111'214, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64bc0383:::obj214:head@111'214, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:649aec78:::obj275:head local-lis/les=117/118 n=233 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.094+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.094+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:649aec78:::obj275:head local-lis/les=117/118 n=233 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:64bc0383:::obj214:head, version: 111'214, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64bc0383:::obj214:head@111'214, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.094+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.094+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:649aec78:::obj275:head local-lis/les=117/118 n=233 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:64bc0383:::obj214:head@111'214, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.094+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:64b337fe:::obj71:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.094+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:649aec78:::obj275:head local-lis/les=117/118 n=233 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:64bc0383:::obj214:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.094+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:649aec78:::obj275:head local-lis/les=117/118 n=233 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:64bc0383:::obj214:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.094+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:649aec78:::obj275:head local-lis/les=117/118 n=233 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:64bc0383:::obj214:head 2026-02-11T23:31:28.094+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:649aec78:::obj275:head local-lis/les=117/118 n=233 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:64bc0383:::obj214:head v 111'214 2026-02-11T23:31:28.094+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:649aec78:::obj275:head local-lis/les=117/118 n=233 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.094+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.094+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:64bc0383:::obj214:head, version: 111'214, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64bc0383:::obj214:head@111'214, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000067 2026-02-11T23:31:28.094+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.094+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.094+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:64b337fe:::obj71:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.094+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:64b337fe:::obj71:head) v3 prio 15 cost 0 latency 0.000089 pg_backfill(progress 4.0 e 119/119 lb 4:64b337fe:::obj71:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:649aec78:::obj275:head local-lis/les=117/118 n=233 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.094+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.094+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:649aec78:::obj275:head local-lis/les=117/118 n=233 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:64b337fe:::obj71:head) v3 2026-02-11T23:31:28.094+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:649aec78:::obj275:head local-lis/les=117/118 n=233 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:64b337fe:::obj71:head) 2026-02-11T23:31:28.094+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.094+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:64b337fe:::obj71:head) v3 finish latency 0.000015 2026-02-11T23:31:28.094+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:64b337fe:::obj71:head local-lis/les=117/118 n=234 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.094+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.094+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.094+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:64b337fe:::obj71:head local-lis/les=117/118 n=234 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.094+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.094+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.094+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.094+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x644cd80 con 0x507ac00 2026-02-11T23:31:28.094+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:64bc0383:::obj214:head)]) -- 0x5d55980 con 0x507ac00 2026-02-11T23:31:28.094+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.095+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 509 ==== MOSDPGPush(4.0 119/117 [PushOp(4:64c1dddc:::obj83:head, version: 111'83, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64c1dddc:::obj83:head@111'83, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x56cab60 con 0x507ac00 2026-02-11T23:31:28.095+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:64c1dddc:::obj83:head, version: 111'83, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64c1dddc:::obj83:head@111'83, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.095+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:64c1dddc:::obj83:head, version: 111'83, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64c1dddc:::obj83:head@111'83, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.095+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.095+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 510 ==== pg_backfill(progress 4.0 e 119/119 lb 4:64bc0383:::obj214:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2f700 con 0x507ac00 2026-02-11T23:31:28.095+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:64c1dddc:::obj83:head, version: 111'83, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64c1dddc:::obj83:head@111'83, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.095+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:64bc0383:::obj214:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 119 2026-02-11T23:31:28.095+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.095+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:64bc0383:::obj214:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.095+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:64c1dddc:::obj83:head, version: 111'83, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64c1dddc:::obj83:head@111'83, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.095+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.095+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:64c1dddc:::obj83:head, version: 111'83, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64c1dddc:::obj83:head@111'83, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(4.0 119/117 [PushOp(4:64c1dddc:::obj83:head, version: 111'83, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64c1dddc:::obj83:head@111'83, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:64b337fe:::obj71:head local-lis/les=117/118 n=234 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.095+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.095+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.095+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:64bc0383:::obj214:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.095+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:64b337fe:::obj71:head local-lis/les=117/118 n=234 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:64c1dddc:::obj83:head, version: 111'83, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64c1dddc:::obj83:head@111'83, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.095+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:64b337fe:::obj71:head local-lis/les=117/118 n=234 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:64c1dddc:::obj83:head@111'83, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.095+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:64b337fe:::obj71:head local-lis/les=117/118 n=234 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:64c1dddc:::obj83:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.095+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:64b337fe:::obj71:head local-lis/les=117/118 n=234 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:64c1dddc:::obj83:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.095+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:64b337fe:::obj71:head local-lis/les=117/118 n=234 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:64c1dddc:::obj83:head 2026-02-11T23:31:28.095+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:64b337fe:::obj71:head local-lis/les=117/118 n=234 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:64c1dddc:::obj83:head v 111'83 2026-02-11T23:31:28.095+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:64b337fe:::obj71:head local-lis/les=117/118 n=234 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.095+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.095+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:64c1dddc:::obj83:head, version: 111'83, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64c1dddc:::obj83:head@111'83, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000077 2026-02-11T23:31:28.095+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.095+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.095+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:64bc0383:::obj214:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.095+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:64bc0383:::obj214:head) v3 prio 15 cost 0 latency 0.000102 pg_backfill(progress 4.0 e 119/119 lb 4:64bc0383:::obj214:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:64b337fe:::obj71:head local-lis/les=117/118 n=234 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.095+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.095+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:64b337fe:::obj71:head local-lis/les=117/118 n=234 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:64bc0383:::obj214:head) v3 2026-02-11T23:31:28.095+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:64b337fe:::obj71:head local-lis/les=117/118 n=234 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:64bc0383:::obj214:head) 2026-02-11T23:31:28.095+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.095+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:64bc0383:::obj214:head) v3 finish latency 0.000014 2026-02-11T23:31:28.095+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:64bc0383:::obj214:head local-lis/les=117/118 n=235 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.095+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.095+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.095+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:64bc0383:::obj214:head local-lis/les=117/118 n=235 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.095+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.095+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.095+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.095+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c9e00 con 0x507ac00 2026-02-11T23:31:28.095+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:64c1dddc:::obj83:head)]) -- 0x5d55c80 con 0x507ac00 2026-02-11T23:31:28.095+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.095+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 511 ==== MOSDPGPush(4.0 119/117 [PushOp(4:657806de:::obj544:head, version: 111'544, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:657806de:::obj544:head@111'544, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f0d00 con 0x507ac00 2026-02-11T23:31:28.095+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:657806de:::obj544:head, version: 111'544, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:657806de:::obj544:head@111'544, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.095+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:657806de:::obj544:head, version: 111'544, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:657806de:::obj544:head@111'544, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.095+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.095+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 512 ==== pg_backfill(progress 4.0 e 119/119 lb 4:64c1dddc:::obj83:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d2d080 con 0x507ac00 2026-02-11T23:31:28.095+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:64c1dddc:::obj83:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.095+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:657806de:::obj544:head, version: 111'544, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:657806de:::obj544:head@111'544, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.095+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:64c1dddc:::obj83:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.095+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.095+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:657806de:::obj544:head, version: 111'544, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:657806de:::obj544:head@111'544, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.095+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.095+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:657806de:::obj544:head, version: 111'544, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:657806de:::obj544:head@111'544, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 119/117 [PushOp(4:657806de:::obj544:head, version: 111'544, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:657806de:::obj544:head@111'544, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:64bc0383:::obj214:head local-lis/les=117/118 n=235 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.095+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:64c1dddc:::obj83:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.095+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.095+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:64bc0383:::obj214:head local-lis/les=117/118 n=235 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:657806de:::obj544:head, version: 111'544, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:657806de:::obj544:head@111'544, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.095+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:64bc0383:::obj214:head local-lis/les=117/118 n=235 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:657806de:::obj544:head@111'544, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.095+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:64bc0383:::obj214:head local-lis/les=117/118 n=235 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:657806de:::obj544:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.095+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:64bc0383:::obj214:head local-lis/les=117/118 n=235 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:657806de:::obj544:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.095+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:64bc0383:::obj214:head local-lis/les=117/118 n=235 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:657806de:::obj544:head 2026-02-11T23:31:28.095+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:64bc0383:::obj214:head local-lis/les=117/118 n=235 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:657806de:::obj544:head v 111'544 2026-02-11T23:31:28.095+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:64bc0383:::obj214:head local-lis/les=117/118 n=235 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.095+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.095+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:657806de:::obj544:head, version: 111'544, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:657806de:::obj544:head@111'544, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000064 2026-02-11T23:31:28.095+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.095+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.095+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:64c1dddc:::obj83:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.095+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:64c1dddc:::obj83:head) v3 prio 15 cost 0 latency 0.000086 pg_backfill(progress 4.0 e 119/119 lb 4:64c1dddc:::obj83:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:64bc0383:::obj214:head local-lis/les=117/118 n=235 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.095+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.095+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:64bc0383:::obj214:head local-lis/les=117/118 n=235 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:64c1dddc:::obj83:head) v3 2026-02-11T23:31:28.095+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:64bc0383:::obj214:head local-lis/les=117/118 n=235 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:64c1dddc:::obj83:head) 2026-02-11T23:31:28.095+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.095+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:64c1dddc:::obj83:head) v3 finish latency 0.000013 2026-02-11T23:31:28.095+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:64c1dddc:::obj83:head local-lis/les=117/118 n=236 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.095+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.095+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.095+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:64c1dddc:::obj83:head local-lis/les=117/118 n=236 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.095+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.095+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.095+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.095+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x644ca80 con 0x507ac00 2026-02-11T23:31:28.095+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:657806de:::obj544:head)]) -- 0x5d55e00 con 0x507ac00 2026-02-11T23:31:28.096+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.096+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 513 ==== MOSDPGPush(4.0 119/117 [PushOp(4:65a1355c:::obj536:head, version: 111'536, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:65a1355c:::obj536:head@111'536, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f1520 con 0x507ac00 2026-02-11T23:31:28.096+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:65a1355c:::obj536:head, version: 111'536, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:65a1355c:::obj536:head@111'536, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000016 epoch 119 2026-02-11T23:31:28.096+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:65a1355c:::obj536:head, version: 111'536, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:65a1355c:::obj536:head@111'536, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.096+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.096+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:65a1355c:::obj536:head, version: 111'536, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:65a1355c:::obj536:head@111'536, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.096+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 514 ==== pg_backfill(progress 4.0 e 119/119 lb 4:657806de:::obj544:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e680 con 0x507ac00 2026-02-11T23:31:28.096+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.096+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:657806de:::obj544:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.096+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:65a1355c:::obj536:head, version: 111'536, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:65a1355c:::obj536:head@111'536, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.096+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:657806de:::obj544:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.096+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.096+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:657806de:::obj544:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.096+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:65a1355c:::obj536:head, version: 111'536, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:65a1355c:::obj536:head@111'536, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000046 MOSDPGPush(4.0 119/117 [PushOp(4:65a1355c:::obj536:head, version: 111'536, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:65a1355c:::obj536:head@111'536, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:64c1dddc:::obj83:head local-lis/les=117/118 n=236 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.096+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.096+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:64c1dddc:::obj83:head local-lis/les=117/118 n=236 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:65a1355c:::obj536:head, version: 111'536, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:65a1355c:::obj536:head@111'536, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.096+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:64c1dddc:::obj83:head local-lis/les=117/118 n=236 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:65a1355c:::obj536:head@111'536, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.096+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:64c1dddc:::obj83:head local-lis/les=117/118 n=236 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:65a1355c:::obj536:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.096+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:64c1dddc:::obj83:head local-lis/les=117/118 n=236 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:65a1355c:::obj536:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.096+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:64c1dddc:::obj83:head local-lis/les=117/118 n=236 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:65a1355c:::obj536:head 2026-02-11T23:31:28.096+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:64c1dddc:::obj83:head local-lis/les=117/118 n=236 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:65a1355c:::obj536:head v 111'536 2026-02-11T23:31:28.096+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:64c1dddc:::obj83:head local-lis/les=117/118 n=236 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.096+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.096+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:65a1355c:::obj536:head, version: 111'536, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:65a1355c:::obj536:head@111'536, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000065 2026-02-11T23:31:28.096+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.096+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.096+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:657806de:::obj544:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.096+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:657806de:::obj544:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 4.0 e 119/119 lb 4:657806de:::obj544:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:64c1dddc:::obj83:head local-lis/les=117/118 n=236 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.096+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.096+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:64c1dddc:::obj83:head local-lis/les=117/118 n=236 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:657806de:::obj544:head) v3 2026-02-11T23:31:28.096+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:64c1dddc:::obj83:head local-lis/les=117/118 n=236 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:657806de:::obj544:head) 2026-02-11T23:31:28.096+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.096+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:657806de:::obj544:head) v3 finish latency 0.000014 2026-02-11T23:31:28.096+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:657806de:::obj544:head local-lis/les=117/118 n=237 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.096+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.096+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.096+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:657806de:::obj544:head local-lis/les=117/118 n=237 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.096+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.096+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.096+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.096+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7ad80 con 0x507ac00 2026-02-11T23:31:28.096+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:65a1355c:::obj536:head)]) -- 0x649e600 con 0x507ac00 2026-02-11T23:31:28.096+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.097+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 515 ==== MOSDPGPush(4.0 119/117 [PushOp(4:6623e3eb:::obj517:head, version: 111'517, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6623e3eb:::obj517:head@111'517, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f0b60 con 0x507ac00 2026-02-11T23:31:28.097+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:6623e3eb:::obj517:head, version: 111'517, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6623e3eb:::obj517:head@111'517, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 119 2026-02-11T23:31:28.097+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:6623e3eb:::obj517:head, version: 111'517, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6623e3eb:::obj517:head@111'517, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.097+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.097+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:6623e3eb:::obj517:head, version: 111'517, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6623e3eb:::obj517:head@111'517, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.097+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 516 ==== pg_backfill(progress 4.0 e 119/119 lb 4:65a1355c:::obj536:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176580 con 0x507ac00 2026-02-11T23:31:28.097+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:65a1355c:::obj536:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 119 2026-02-11T23:31:28.097+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.097+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:65a1355c:::obj536:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.097+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:6623e3eb:::obj517:head, version: 111'517, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6623e3eb:::obj517:head@111'517, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.097+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.097+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:6623e3eb:::obj517:head, version: 111'517, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6623e3eb:::obj517:head@111'517, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 119/117 [PushOp(4:6623e3eb:::obj517:head, version: 111'517, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6623e3eb:::obj517:head@111'517, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:657806de:::obj544:head local-lis/les=117/118 n=237 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.097+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.097+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:657806de:::obj544:head local-lis/les=117/118 n=237 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:6623e3eb:::obj517:head, version: 111'517, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6623e3eb:::obj517:head@111'517, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.097+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.097+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:65a1355c:::obj536:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.097+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:657806de:::obj544:head local-lis/les=117/118 n=237 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:6623e3eb:::obj517:head@111'517, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.097+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:657806de:::obj544:head local-lis/les=117/118 n=237 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:6623e3eb:::obj517:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.097+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:657806de:::obj544:head local-lis/les=117/118 n=237 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:6623e3eb:::obj517:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.097+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:657806de:::obj544:head local-lis/les=117/118 n=237 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:6623e3eb:::obj517:head 2026-02-11T23:31:28.097+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:657806de:::obj544:head local-lis/les=117/118 n=237 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:6623e3eb:::obj517:head v 111'517 2026-02-11T23:31:28.097+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:657806de:::obj544:head local-lis/les=117/118 n=237 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.097+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.097+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:6623e3eb:::obj517:head, version: 111'517, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6623e3eb:::obj517:head@111'517, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:28.097+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.097+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.097+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:65a1355c:::obj536:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.097+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:65a1355c:::obj536:head) v3 prio 15 cost 0 latency 0.000081 pg_backfill(progress 4.0 e 119/119 lb 4:65a1355c:::obj536:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:657806de:::obj544:head local-lis/les=117/118 n=237 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.097+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.097+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:657806de:::obj544:head local-lis/les=117/118 n=237 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:65a1355c:::obj536:head) v3 2026-02-11T23:31:28.097+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:657806de:::obj544:head local-lis/les=117/118 n=237 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:65a1355c:::obj536:head) 2026-02-11T23:31:28.097+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.097+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:65a1355c:::obj536:head) v3 finish latency 0.000014 2026-02-11T23:31:28.097+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:65a1355c:::obj536:head local-lis/les=117/118 n=238 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.097+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.097+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.097+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:65a1355c:::obj536:head local-lis/les=117/118 n=238 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.097+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.097+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.097+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.097+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x644c780 con 0x507ac00 2026-02-11T23:31:28.097+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:6623e3eb:::obj517:head)]) -- 0x644d800 con 0x507ac00 2026-02-11T23:31:28.097+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.097+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 517 ==== MOSDPGPush(4.0 119/117 [PushOp(4:6681cfe7:::obj102:head, version: 111'102, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6681cfe7:::obj102:head@111'102, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559cb60 con 0x507ac00 2026-02-11T23:31:28.097+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:6681cfe7:::obj102:head, version: 111'102, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6681cfe7:::obj102:head@111'102, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.097+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:6681cfe7:::obj102:head, version: 111'102, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6681cfe7:::obj102:head@111'102, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.097+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.097+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 518 ==== pg_backfill(progress 4.0 e 119/119 lb 4:6623e3eb:::obj517:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e100 con 0x507ac00 2026-02-11T23:31:28.097+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:6623e3eb:::obj517:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.097+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:6681cfe7:::obj102:head, version: 111'102, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6681cfe7:::obj102:head@111'102, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.097+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:6623e3eb:::obj517:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.097+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.097+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:6681cfe7:::obj102:head, version: 111'102, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6681cfe7:::obj102:head@111'102, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.097+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:6681cfe7:::obj102:head, version: 111'102, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6681cfe7:::obj102:head@111'102, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 119/117 [PushOp(4:6681cfe7:::obj102:head, version: 111'102, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6681cfe7:::obj102:head@111'102, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:65a1355c:::obj536:head local-lis/les=117/118 n=238 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.097+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.097+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.097+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:6623e3eb:::obj517:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.097+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:65a1355c:::obj536:head local-lis/les=117/118 n=238 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:6681cfe7:::obj102:head, version: 111'102, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6681cfe7:::obj102:head@111'102, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.097+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:65a1355c:::obj536:head local-lis/les=117/118 n=238 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:6681cfe7:::obj102:head@111'102, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.097+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:65a1355c:::obj536:head local-lis/les=117/118 n=238 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:6681cfe7:::obj102:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.097+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:65a1355c:::obj536:head local-lis/les=117/118 n=238 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:6681cfe7:::obj102:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.097+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:65a1355c:::obj536:head local-lis/les=117/118 n=238 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:6681cfe7:::obj102:head 2026-02-11T23:31:28.097+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:65a1355c:::obj536:head local-lis/les=117/118 n=238 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:6681cfe7:::obj102:head v 111'102 2026-02-11T23:31:28.097+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:65a1355c:::obj536:head local-lis/les=117/118 n=238 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.097+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.097+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:6681cfe7:::obj102:head, version: 111'102, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6681cfe7:::obj102:head@111'102, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:28.097+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.097+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.097+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:6623e3eb:::obj517:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.097+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:6623e3eb:::obj517:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 4.0 e 119/119 lb 4:6623e3eb:::obj517:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:65a1355c:::obj536:head local-lis/les=117/118 n=238 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.097+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.097+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:65a1355c:::obj536:head local-lis/les=117/118 n=238 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:6623e3eb:::obj517:head) v3 2026-02-11T23:31:28.097+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:65a1355c:::obj536:head local-lis/les=117/118 n=238 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:6623e3eb:::obj517:head) 2026-02-11T23:31:28.097+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.097+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:6623e3eb:::obj517:head) v3 finish latency 0.000013 2026-02-11T23:31:28.097+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6623e3eb:::obj517:head local-lis/les=117/118 n=239 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.097+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.097+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.097+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6623e3eb:::obj517:head local-lis/les=117/118 n=239 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.097+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.097+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.097+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.097+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c5980 con 0x507ac00 2026-02-11T23:31:28.097+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:6681cfe7:::obj102:head)]) -- 0x4f7a900 con 0x507ac00 2026-02-11T23:31:28.097+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.098+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 519 ==== MOSDPGPush(4.0 119/117 [PushOp(4:6727c4f6:::obj39:head, version: 111'39, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6727c4f6:::obj39:head@111'39, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5e26680 con 0x507ac00 2026-02-11T23:31:28.098+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:6727c4f6:::obj39:head, version: 111'39, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6727c4f6:::obj39:head@111'39, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 119 2026-02-11T23:31:28.098+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:6727c4f6:::obj39:head, version: 111'39, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6727c4f6:::obj39:head@111'39, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.098+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.098+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 520 ==== pg_backfill(progress 4.0 e 119/119 lb 4:6681cfe7:::obj102:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2c000 con 0x507ac00 2026-02-11T23:31:28.098+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:6727c4f6:::obj39:head, version: 111'39, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6727c4f6:::obj39:head@111'39, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.098+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:6681cfe7:::obj102:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.098+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:6681cfe7:::obj102:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.098+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.098+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:6727c4f6:::obj39:head, version: 111'39, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6727c4f6:::obj39:head@111'39, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.098+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:6727c4f6:::obj39:head, version: 111'39, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6727c4f6:::obj39:head@111'39, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 119/117 [PushOp(4:6727c4f6:::obj39:head, version: 111'39, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6727c4f6:::obj39:head@111'39, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:6623e3eb:::obj517:head local-lis/les=117/118 n=239 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.098+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.098+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:6681cfe7:::obj102:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.098+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.098+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6623e3eb:::obj517:head local-lis/les=117/118 n=239 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:6727c4f6:::obj39:head, version: 111'39, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6727c4f6:::obj39:head@111'39, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.098+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6623e3eb:::obj517:head local-lis/les=117/118 n=239 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:6727c4f6:::obj39:head@111'39, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.098+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6623e3eb:::obj517:head local-lis/les=117/118 n=239 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:6727c4f6:::obj39:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.098+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6623e3eb:::obj517:head local-lis/les=117/118 n=239 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:6727c4f6:::obj39:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.098+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6623e3eb:::obj517:head local-lis/les=117/118 n=239 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:6727c4f6:::obj39:head 2026-02-11T23:31:28.098+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6623e3eb:::obj517:head local-lis/les=117/118 n=239 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:6727c4f6:::obj39:head v 111'39 2026-02-11T23:31:28.098+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6623e3eb:::obj517:head local-lis/les=117/118 n=239 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.098+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.098+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:6727c4f6:::obj39:head, version: 111'39, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6727c4f6:::obj39:head@111'39, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:28.098+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.098+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.098+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:6681cfe7:::obj102:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.098+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:6681cfe7:::obj102:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 4.0 e 119/119 lb 4:6681cfe7:::obj102:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:6623e3eb:::obj517:head local-lis/les=117/118 n=239 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.098+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.098+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6623e3eb:::obj517:head local-lis/les=117/118 n=239 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:6681cfe7:::obj102:head) v3 2026-02-11T23:31:28.098+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6623e3eb:::obj517:head local-lis/les=117/118 n=239 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:6681cfe7:::obj102:head) 2026-02-11T23:31:28.098+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.098+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:6681cfe7:::obj102:head) v3 finish latency 0.000014 2026-02-11T23:31:28.098+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6681cfe7:::obj102:head local-lis/les=117/118 n=240 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.098+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.098+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.098+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6681cfe7:::obj102:head local-lis/les=117/118 n=240 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.098+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.098+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.098+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.098+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x644c480 con 0x507ac00 2026-02-11T23:31:28.098+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:6727c4f6:::obj39:head)]) -- 0x644db00 con 0x507ac00 2026-02-11T23:31:28.098+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.098+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 521 ==== MOSDPGPush(4.0 119/117 [PushOp(4:67a26c0a:::obj53:head, version: 111'53, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:67a26c0a:::obj53:head@111'53, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5647380 con 0x507ac00 2026-02-11T23:31:28.098+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:67a26c0a:::obj53:head, version: 111'53, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:67a26c0a:::obj53:head@111'53, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 119 2026-02-11T23:31:28.099+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:67a26c0a:::obj53:head, version: 111'53, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:67a26c0a:::obj53:head@111'53, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.099+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.099+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:67a26c0a:::obj53:head, version: 111'53, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:67a26c0a:::obj53:head@111'53, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.099+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.099+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:67a26c0a:::obj53:head, version: 111'53, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:67a26c0a:::obj53:head@111'53, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.099+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 522 ==== pg_backfill(progress 4.0 e 119/119 lb 4:6727c4f6:::obj39:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d2cb00 con 0x507ac00 2026-02-11T23:31:28.099+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:6727c4f6:::obj39:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.099+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:67a26c0a:::obj53:head, version: 111'53, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:67a26c0a:::obj53:head@111'53, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 119/117 [PushOp(4:67a26c0a:::obj53:head, version: 111'53, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:67a26c0a:::obj53:head@111'53, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:6681cfe7:::obj102:head local-lis/les=117/118 n=240 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.099+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:6727c4f6:::obj39:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.099+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.099+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.099+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6681cfe7:::obj102:head local-lis/les=117/118 n=240 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:67a26c0a:::obj53:head, version: 111'53, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:67a26c0a:::obj53:head@111'53, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.099+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:6727c4f6:::obj39:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.099+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6681cfe7:::obj102:head local-lis/les=117/118 n=240 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:67a26c0a:::obj53:head@111'53, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.099+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6681cfe7:::obj102:head local-lis/les=117/118 n=240 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:67a26c0a:::obj53:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.099+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6681cfe7:::obj102:head local-lis/les=117/118 n=240 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:67a26c0a:::obj53:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.099+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6681cfe7:::obj102:head local-lis/les=117/118 n=240 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:67a26c0a:::obj53:head 2026-02-11T23:31:28.099+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6681cfe7:::obj102:head local-lis/les=117/118 n=240 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:67a26c0a:::obj53:head v 111'53 2026-02-11T23:31:28.099+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6681cfe7:::obj102:head local-lis/les=117/118 n=240 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.099+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.099+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:67a26c0a:::obj53:head, version: 111'53, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:67a26c0a:::obj53:head@111'53, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:28.099+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.099+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.099+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:6727c4f6:::obj39:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.099+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:6727c4f6:::obj39:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 4.0 e 119/119 lb 4:6727c4f6:::obj39:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:6681cfe7:::obj102:head local-lis/les=117/118 n=240 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.099+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.099+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6681cfe7:::obj102:head local-lis/les=117/118 n=240 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:6727c4f6:::obj39:head) v3 2026-02-11T23:31:28.099+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6681cfe7:::obj102:head local-lis/les=117/118 n=240 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:6727c4f6:::obj39:head) 2026-02-11T23:31:28.099+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.099+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:6727c4f6:::obj39:head) v3 finish latency 0.000016 2026-02-11T23:31:28.099+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6727c4f6:::obj39:head local-lis/les=117/118 n=241 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.099+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.099+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.099+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6727c4f6:::obj39:head local-lis/les=117/118 n=241 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.099+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.099+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.099+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.099+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7af00 con 0x507ac00 2026-02-11T23:31:28.099+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:67a26c0a:::obj53:head)]) -- 0x5f2dc80 con 0x507ac00 2026-02-11T23:31:28.099+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.099+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 523 ==== MOSDPGPush(4.0 119/117 [PushOp(4:685e1da9:::obj264:head, version: 111'264, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:685e1da9:::obj264:head@111'264, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2d00 con 0x507ac00 2026-02-11T23:31:28.099+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:685e1da9:::obj264:head, version: 111'264, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:685e1da9:::obj264:head@111'264, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 119 2026-02-11T23:31:28.099+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:685e1da9:::obj264:head, version: 111'264, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:685e1da9:::obj264:head@111'264, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.099+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.099+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 524 ==== pg_backfill(progress 4.0 e 119/119 lb 4:67a26c0a:::obj53:head) ==== 1061+0+0 (unknown 0 0 0) 0x6179700 con 0x507ac00 2026-02-11T23:31:28.099+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:67a26c0a:::obj53:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.099+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:685e1da9:::obj264:head, version: 111'264, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:685e1da9:::obj264:head@111'264, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.099+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:67a26c0a:::obj53:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.099+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.099+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:685e1da9:::obj264:head, version: 111'264, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:685e1da9:::obj264:head@111'264, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.099+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.099+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:67a26c0a:::obj53:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.099+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:685e1da9:::obj264:head, version: 111'264, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:685e1da9:::obj264:head@111'264, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 119/117 [PushOp(4:685e1da9:::obj264:head, version: 111'264, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:685e1da9:::obj264:head@111'264, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:6727c4f6:::obj39:head local-lis/les=117/118 n=241 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.099+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.099+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6727c4f6:::obj39:head local-lis/les=117/118 n=241 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:685e1da9:::obj264:head, version: 111'264, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:685e1da9:::obj264:head@111'264, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.099+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6727c4f6:::obj39:head local-lis/les=117/118 n=241 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:685e1da9:::obj264:head@111'264, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.099+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6727c4f6:::obj39:head local-lis/les=117/118 n=241 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:685e1da9:::obj264:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.099+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6727c4f6:::obj39:head local-lis/les=117/118 n=241 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:685e1da9:::obj264:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.099+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6727c4f6:::obj39:head local-lis/les=117/118 n=241 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:685e1da9:::obj264:head 2026-02-11T23:31:28.099+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6727c4f6:::obj39:head local-lis/les=117/118 n=241 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:685e1da9:::obj264:head v 111'264 2026-02-11T23:31:28.099+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6727c4f6:::obj39:head local-lis/les=117/118 n=241 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.099+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.099+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:685e1da9:::obj264:head, version: 111'264, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:685e1da9:::obj264:head@111'264, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:28.099+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.099+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.099+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:67a26c0a:::obj53:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.099+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:67a26c0a:::obj53:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 4.0 e 119/119 lb 4:67a26c0a:::obj53:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:6727c4f6:::obj39:head local-lis/les=117/118 n=241 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.099+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.099+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6727c4f6:::obj39:head local-lis/les=117/118 n=241 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:67a26c0a:::obj53:head) v3 2026-02-11T23:31:28.099+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6727c4f6:::obj39:head local-lis/les=117/118 n=241 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:67a26c0a:::obj53:head) 2026-02-11T23:31:28.099+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.099+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:67a26c0a:::obj53:head) v3 finish latency 0.000016 2026-02-11T23:31:28.099+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:67a26c0a:::obj53:head local-lis/les=117/118 n=242 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.099+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.099+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.099+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:67a26c0a:::obj53:head local-lis/les=117/118 n=242 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.099+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.099+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.099+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.099+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x644c180 con 0x507ac00 2026-02-11T23:31:28.099+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:685e1da9:::obj264:head)]) -- 0x63c9200 con 0x507ac00 2026-02-11T23:31:28.099+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.100+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 525 ==== MOSDPGPush(4.0 119/117 [PushOp(4:685e6e29:::obj176:head, version: 111'176, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:685e6e29:::obj176:head@111'176, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56b64e0 con 0x507ac00 2026-02-11T23:31:28.100+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:685e6e29:::obj176:head, version: 111'176, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:685e6e29:::obj176:head@111'176, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 119 2026-02-11T23:31:28.100+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:685e6e29:::obj176:head, version: 111'176, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:685e6e29:::obj176:head@111'176, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.100+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.100+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 526 ==== pg_backfill(progress 4.0 e 119/119 lb 4:685e1da9:::obj264:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176000 con 0x507ac00 2026-02-11T23:31:28.100+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:685e1da9:::obj264:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.100+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:685e6e29:::obj176:head, version: 111'176, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:685e6e29:::obj176:head@111'176, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.100+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:685e1da9:::obj264:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.100+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.100+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:685e6e29:::obj176:head, version: 111'176, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:685e6e29:::obj176:head@111'176, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.100+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:685e6e29:::obj176:head, version: 111'176, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:685e6e29:::obj176:head@111'176, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 119/117 [PushOp(4:685e6e29:::obj176:head, version: 111'176, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:685e6e29:::obj176:head@111'176, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:67a26c0a:::obj53:head local-lis/les=117/118 n=242 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.100+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.100+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.100+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:685e1da9:::obj264:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.100+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:67a26c0a:::obj53:head local-lis/les=117/118 n=242 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:685e6e29:::obj176:head, version: 111'176, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:685e6e29:::obj176:head@111'176, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.100+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:67a26c0a:::obj53:head local-lis/les=117/118 n=242 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:685e6e29:::obj176:head@111'176, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.100+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:67a26c0a:::obj53:head local-lis/les=117/118 n=242 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:685e6e29:::obj176:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.100+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:67a26c0a:::obj53:head local-lis/les=117/118 n=242 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:685e6e29:::obj176:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.100+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:67a26c0a:::obj53:head local-lis/les=117/118 n=242 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:685e6e29:::obj176:head 2026-02-11T23:31:28.100+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:67a26c0a:::obj53:head local-lis/les=117/118 n=242 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:685e6e29:::obj176:head v 111'176 2026-02-11T23:31:28.100+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:67a26c0a:::obj53:head local-lis/les=117/118 n=242 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.100+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.100+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:685e6e29:::obj176:head, version: 111'176, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:685e6e29:::obj176:head@111'176, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:28.100+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.100+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.100+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:685e1da9:::obj264:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.100+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:685e1da9:::obj264:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 119/119 lb 4:685e1da9:::obj264:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:67a26c0a:::obj53:head local-lis/les=117/118 n=242 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.100+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.100+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:67a26c0a:::obj53:head local-lis/les=117/118 n=242 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:685e1da9:::obj264:head) v3 2026-02-11T23:31:28.100+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:67a26c0a:::obj53:head local-lis/les=117/118 n=242 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:685e1da9:::obj264:head) 2026-02-11T23:31:28.100+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.100+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:685e1da9:::obj264:head) v3 finish latency 0.000017 2026-02-11T23:31:28.100+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:685e1da9:::obj264:head local-lis/les=117/118 n=243 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.100+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.100+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.100+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:685e1da9:::obj264:head local-lis/les=117/118 n=243 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.100+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.100+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.100+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.100+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c9c80 con 0x507ac00 2026-02-11T23:31:28.100+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:685e6e29:::obj176:head)]) -- 0x644de00 con 0x507ac00 2026-02-11T23:31:28.100+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.100+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 527 ==== MOSDPGPush(4.0 119/117 [PushOp(4:68743261:::obj370:head, version: 111'370, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:68743261:::obj370:head@111'370, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56caea0 con 0x507ac00 2026-02-11T23:31:28.100+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:68743261:::obj370:head, version: 111'370, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:68743261:::obj370:head@111'370, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.100+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:68743261:::obj370:head, version: 111'370, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:68743261:::obj370:head@111'370, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.100+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.100+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 528 ==== pg_backfill(progress 4.0 e 119/119 lb 4:685e6e29:::obj176:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9a680 con 0x507ac00 2026-02-11T23:31:28.100+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:685e6e29:::obj176:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.100+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:68743261:::obj370:head, version: 111'370, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:68743261:::obj370:head@111'370, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.100+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:685e6e29:::obj176:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.100+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.100+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:68743261:::obj370:head, version: 111'370, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:68743261:::obj370:head@111'370, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.100+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.100+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:685e6e29:::obj176:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.100+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:68743261:::obj370:head, version: 111'370, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:68743261:::obj370:head@111'370, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 119/117 [PushOp(4:68743261:::obj370:head, version: 111'370, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:68743261:::obj370:head@111'370, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:685e1da9:::obj264:head local-lis/les=117/118 n=243 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.100+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.100+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:685e1da9:::obj264:head local-lis/les=117/118 n=243 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:68743261:::obj370:head, version: 111'370, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:68743261:::obj370:head@111'370, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.100+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:685e1da9:::obj264:head local-lis/les=117/118 n=243 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:68743261:::obj370:head@111'370, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.100+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:685e1da9:::obj264:head local-lis/les=117/118 n=243 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:68743261:::obj370:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.100+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:685e1da9:::obj264:head local-lis/les=117/118 n=243 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:68743261:::obj370:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.100+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:685e1da9:::obj264:head local-lis/les=117/118 n=243 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:68743261:::obj370:head 2026-02-11T23:31:28.100+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:685e1da9:::obj264:head local-lis/les=117/118 n=243 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:68743261:::obj370:head v 111'370 2026-02-11T23:31:28.100+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:685e1da9:::obj264:head local-lis/les=117/118 n=243 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.100+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.100+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:68743261:::obj370:head, version: 111'370, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:68743261:::obj370:head@111'370, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:28.100+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.100+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.100+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:685e6e29:::obj176:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.100+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:685e6e29:::obj176:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 4.0 e 119/119 lb 4:685e6e29:::obj176:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:685e1da9:::obj264:head local-lis/les=117/118 n=243 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.100+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.100+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:685e1da9:::obj264:head local-lis/les=117/118 n=243 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:685e6e29:::obj176:head) v3 2026-02-11T23:31:28.100+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:685e1da9:::obj264:head local-lis/les=117/118 n=243 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:685e6e29:::obj176:head) 2026-02-11T23:31:28.100+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.100+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:685e6e29:::obj176:head) v3 finish latency 0.000017 2026-02-11T23:31:28.100+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:685e6e29:::obj176:head local-lis/les=117/118 n=244 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.100+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.100+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.100+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:685e6e29:::obj176:head local-lis/les=117/118 n=244 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.100+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.100+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.100+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.100+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x644de00 con 0x507ac00 2026-02-11T23:31:28.100+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:68743261:::obj370:head)]) -- 0x4f7a780 con 0x507ac00 2026-02-11T23:31:28.100+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.101+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 529 ==== MOSDPGPush(4.0 119/117 [PushOp(4:68869f42:::obj433:head, version: 111'433, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:68869f42:::obj433:head@111'433, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5617380 con 0x507ac00 2026-02-11T23:31:28.101+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:68869f42:::obj433:head, version: 111'433, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:68869f42:::obj433:head@111'433, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.101+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:68869f42:::obj433:head, version: 111'433, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:68869f42:::obj433:head@111'433, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.101+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.101+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 530 ==== pg_backfill(progress 4.0 e 119/119 lb 4:68743261:::obj370:head) ==== 1062+0+0 (unknown 0 0 0) 0x5985700 con 0x507ac00 2026-02-11T23:31:28.101+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:68743261:::obj370:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.101+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:68869f42:::obj433:head, version: 111'433, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:68869f42:::obj433:head@111'433, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.101+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:68743261:::obj370:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.101+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.101+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:68869f42:::obj433:head, version: 111'433, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:68869f42:::obj433:head@111'433, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.101+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:68869f42:::obj433:head, version: 111'433, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:68869f42:::obj433:head@111'433, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 119/117 [PushOp(4:68869f42:::obj433:head, version: 111'433, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:68869f42:::obj433:head@111'433, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:685e6e29:::obj176:head local-lis/les=117/118 n=244 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.101+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.101+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.101+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:68743261:::obj370:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.101+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:685e6e29:::obj176:head local-lis/les=117/118 n=244 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:68869f42:::obj433:head, version: 111'433, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:68869f42:::obj433:head@111'433, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.101+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:685e6e29:::obj176:head local-lis/les=117/118 n=244 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:68869f42:::obj433:head@111'433, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.101+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:685e6e29:::obj176:head local-lis/les=117/118 n=244 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:68869f42:::obj433:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.101+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:685e6e29:::obj176:head local-lis/les=117/118 n=244 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:68869f42:::obj433:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.101+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:685e6e29:::obj176:head local-lis/les=117/118 n=244 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:68869f42:::obj433:head 2026-02-11T23:31:28.101+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:685e6e29:::obj176:head local-lis/les=117/118 n=244 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:68869f42:::obj433:head v 111'433 2026-02-11T23:31:28.101+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:685e6e29:::obj176:head local-lis/les=117/118 n=244 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.101+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.101+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:68869f42:::obj433:head, version: 111'433, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:68869f42:::obj433:head@111'433, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:28.101+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.101+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.101+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:68743261:::obj370:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.101+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:68743261:::obj370:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 4.0 e 119/119 lb 4:68743261:::obj370:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:685e6e29:::obj176:head local-lis/les=117/118 n=244 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.101+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.101+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:685e6e29:::obj176:head local-lis/les=117/118 n=244 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:68743261:::obj370:head) v3 2026-02-11T23:31:28.101+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:685e6e29:::obj176:head local-lis/les=117/118 n=244 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:68743261:::obj370:head) 2026-02-11T23:31:28.101+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.101+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:68743261:::obj370:head) v3 finish latency 0.000015 2026-02-11T23:31:28.101+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:68743261:::obj370:head local-lis/les=117/118 n=245 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.101+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.101+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.101+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:68743261:::obj370:head local-lis/les=117/118 n=245 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.101+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.101+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.101+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.101+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c9c80 con 0x507ac00 2026-02-11T23:31:28.101+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:68869f42:::obj433:head)]) -- 0x4f7b080 con 0x507ac00 2026-02-11T23:31:28.101+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.101+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 531 ==== MOSDPGPush(4.0 119/117 [PushOp(4:69182b79:::obj301:head, version: 111'301, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:69182b79:::obj301:head@111'301, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a3a00 con 0x507ac00 2026-02-11T23:31:28.101+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:69182b79:::obj301:head, version: 111'301, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:69182b79:::obj301:head@111'301, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 119 2026-02-11T23:31:28.101+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:69182b79:::obj301:head, version: 111'301, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:69182b79:::obj301:head@111'301, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.101+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.101+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 532 ==== pg_backfill(progress 4.0 e 119/119 lb 4:68869f42:::obj433:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177600 con 0x507ac00 2026-02-11T23:31:28.101+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:69182b79:::obj301:head, version: 111'301, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:69182b79:::obj301:head@111'301, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.101+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:68869f42:::obj433:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.101+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:68869f42:::obj433:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.101+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.101+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:69182b79:::obj301:head, version: 111'301, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:69182b79:::obj301:head@111'301, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.102+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:69182b79:::obj301:head, version: 111'301, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:69182b79:::obj301:head@111'301, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(4.0 119/117 [PushOp(4:69182b79:::obj301:head, version: 111'301, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:69182b79:::obj301:head@111'301, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:68743261:::obj370:head local-lis/les=117/118 n=245 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.102+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.102+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.102+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:68869f42:::obj433:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.102+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:68743261:::obj370:head local-lis/les=117/118 n=245 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:69182b79:::obj301:head, version: 111'301, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:69182b79:::obj301:head@111'301, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.102+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:68743261:::obj370:head local-lis/les=117/118 n=245 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:69182b79:::obj301:head@111'301, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.102+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:68743261:::obj370:head local-lis/les=117/118 n=245 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:69182b79:::obj301:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.102+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:68743261:::obj370:head local-lis/les=117/118 n=245 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:69182b79:::obj301:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.102+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:68743261:::obj370:head local-lis/les=117/118 n=245 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:69182b79:::obj301:head 2026-02-11T23:31:28.102+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:68743261:::obj370:head local-lis/les=117/118 n=245 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:69182b79:::obj301:head v 111'301 2026-02-11T23:31:28.102+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:68743261:::obj370:head local-lis/les=117/118 n=245 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.102+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.102+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:69182b79:::obj301:head, version: 111'301, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:69182b79:::obj301:head@111'301, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000066 2026-02-11T23:31:28.102+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.102+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.102+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:68869f42:::obj433:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.102+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:68869f42:::obj433:head) v3 prio 15 cost 0 latency 0.000090 pg_backfill(progress 4.0 e 119/119 lb 4:68869f42:::obj433:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:68743261:::obj370:head local-lis/les=117/118 n=245 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.102+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.102+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:68743261:::obj370:head local-lis/les=117/118 n=245 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:68869f42:::obj433:head) v3 2026-02-11T23:31:28.102+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:68743261:::obj370:head local-lis/les=117/118 n=245 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:68869f42:::obj433:head) 2026-02-11T23:31:28.102+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.102+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:68869f42:::obj433:head) v3 finish latency 0.000025 2026-02-11T23:31:28.102+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:68869f42:::obj433:head local-lis/les=117/118 n=246 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.102+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.102+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.102+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:68869f42:::obj433:head local-lis/les=117/118 n=246 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.102+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.102+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.102+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.102+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c9200 con 0x507ac00 2026-02-11T23:31:28.102+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:69182b79:::obj301:head)]) -- 0x63c4180 con 0x507ac00 2026-02-11T23:31:28.102+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.102+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.102+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.102+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 533 ==== MOSDPGPush(4.0 119/117 [PushOp(4:6934f76e:::obj318:head, version: 111'318, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6934f76e:::obj318:head@111'318, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2820 con 0x507ac00 2026-02-11T23:31:28.102+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:6934f76e:::obj318:head, version: 111'318, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6934f76e:::obj318:head@111'318, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.102+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:6934f76e:::obj318:head, version: 111'318, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6934f76e:::obj318:head@111'318, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.102+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.102+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 534 ==== pg_backfill(progress 4.0 e 119/119 lb 4:69182b79:::obj301:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177b80 con 0x507ac00 2026-02-11T23:31:28.102+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:6934f76e:::obj318:head, version: 111'318, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6934f76e:::obj318:head@111'318, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.102+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:69182b79:::obj301:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.102+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:69182b79:::obj301:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.102+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.102+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:6934f76e:::obj318:head, version: 111'318, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6934f76e:::obj318:head@111'318, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.102+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:6934f76e:::obj318:head, version: 111'318, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6934f76e:::obj318:head@111'318, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(4.0 119/117 [PushOp(4:6934f76e:::obj318:head, version: 111'318, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6934f76e:::obj318:head@111'318, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:68869f42:::obj433:head local-lis/les=117/118 n=246 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.102+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.102+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.102+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:69182b79:::obj301:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.102+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:68869f42:::obj433:head local-lis/les=117/118 n=246 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:6934f76e:::obj318:head, version: 111'318, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6934f76e:::obj318:head@111'318, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.102+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:68869f42:::obj433:head local-lis/les=117/118 n=246 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:6934f76e:::obj318:head@111'318, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.102+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:68869f42:::obj433:head local-lis/les=117/118 n=246 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:6934f76e:::obj318:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.102+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:68869f42:::obj433:head local-lis/les=117/118 n=246 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:6934f76e:::obj318:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.102+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:68869f42:::obj433:head local-lis/les=117/118 n=246 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:6934f76e:::obj318:head 2026-02-11T23:31:28.102+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:68869f42:::obj433:head local-lis/les=117/118 n=246 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:6934f76e:::obj318:head v 111'318 2026-02-11T23:31:28.102+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:68869f42:::obj433:head local-lis/les=117/118 n=246 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.102+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.102+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:6934f76e:::obj318:head, version: 111'318, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6934f76e:::obj318:head@111'318, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:28.102+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.102+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.102+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:69182b79:::obj301:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.102+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:69182b79:::obj301:head) v3 prio 15 cost 0 latency 0.000082 pg_backfill(progress 4.0 e 119/119 lb 4:69182b79:::obj301:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:68869f42:::obj433:head local-lis/les=117/118 n=246 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.102+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.102+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:68869f42:::obj433:head local-lis/les=117/118 n=246 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:69182b79:::obj301:head) v3 2026-02-11T23:31:28.102+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:68869f42:::obj433:head local-lis/les=117/118 n=246 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:69182b79:::obj301:head) 2026-02-11T23:31:28.102+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.102+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:69182b79:::obj301:head) v3 finish latency 0.000017 2026-02-11T23:31:28.102+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:69182b79:::obj301:head local-lis/les=117/118 n=247 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.102+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.102+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.102+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:69182b79:::obj301:head local-lis/les=117/118 n=247 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.102+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.102+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.102+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.102+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x644c180 con 0x507ac00 2026-02-11T23:31:28.102+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:6934f76e:::obj318:head)]) -- 0x63c9080 con 0x507ac00 2026-02-11T23:31:28.102+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.103+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 535 ==== MOSDPGPush(4.0 119/117 [PushOp(4:6971da6a:::obj293:head, version: 111'293, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6971da6a:::obj293:head@111'293, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x554e4e0 con 0x507ac00 2026-02-11T23:31:28.103+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:6971da6a:::obj293:head, version: 111'293, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6971da6a:::obj293:head@111'293, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 119 2026-02-11T23:31:28.103+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:6971da6a:::obj293:head, version: 111'293, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6971da6a:::obj293:head@111'293, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.103+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.103+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 536 ==== pg_backfill(progress 4.0 e 119/119 lb 4:6934f76e:::obj318:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2c580 con 0x507ac00 2026-02-11T23:31:28.103+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:6971da6a:::obj293:head, version: 111'293, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6971da6a:::obj293:head@111'293, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.103+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:6934f76e:::obj318:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.103+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:6934f76e:::obj318:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.103+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.103+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:6971da6a:::obj293:head, version: 111'293, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6971da6a:::obj293:head@111'293, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.103+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:6971da6a:::obj293:head, version: 111'293, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6971da6a:::obj293:head@111'293, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 119/117 [PushOp(4:6971da6a:::obj293:head, version: 111'293, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6971da6a:::obj293:head@111'293, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:69182b79:::obj301:head local-lis/les=117/118 n=247 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.103+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.103+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.103+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:6934f76e:::obj318:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.103+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:69182b79:::obj301:head local-lis/les=117/118 n=247 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:6971da6a:::obj293:head, version: 111'293, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6971da6a:::obj293:head@111'293, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.103+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:69182b79:::obj301:head local-lis/les=117/118 n=247 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:6971da6a:::obj293:head@111'293, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.103+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:69182b79:::obj301:head local-lis/les=117/118 n=247 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:6971da6a:::obj293:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.103+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:69182b79:::obj301:head local-lis/les=117/118 n=247 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:6971da6a:::obj293:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.103+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:69182b79:::obj301:head local-lis/les=117/118 n=247 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:6971da6a:::obj293:head 2026-02-11T23:31:28.103+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:69182b79:::obj301:head local-lis/les=117/118 n=247 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:6971da6a:::obj293:head v 111'293 2026-02-11T23:31:28.103+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:69182b79:::obj301:head local-lis/les=117/118 n=247 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.103+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.103+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:6971da6a:::obj293:head, version: 111'293, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6971da6a:::obj293:head@111'293, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:28.103+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.103+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.103+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:6934f76e:::obj318:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.103+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:6934f76e:::obj318:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 4.0 e 119/119 lb 4:6934f76e:::obj318:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:69182b79:::obj301:head local-lis/les=117/118 n=247 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.103+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.103+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:69182b79:::obj301:head local-lis/les=117/118 n=247 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:6934f76e:::obj318:head) v3 2026-02-11T23:31:28.103+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:69182b79:::obj301:head local-lis/les=117/118 n=247 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:6934f76e:::obj318:head) 2026-02-11T23:31:28.103+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.103+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:6934f76e:::obj318:head) v3 finish latency 0.000015 2026-02-11T23:31:28.103+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6934f76e:::obj318:head local-lis/les=117/118 n=248 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.103+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.103+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.103+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6934f76e:::obj318:head local-lis/les=117/118 n=248 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.103+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.103+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.103+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.103+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x5f2dc80 con 0x507ac00 2026-02-11T23:31:28.103+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:6971da6a:::obj293:head)]) -- 0x4f7a480 con 0x507ac00 2026-02-11T23:31:28.103+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.103+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 537 ==== MOSDPGPush(4.0 119/117 [PushOp(4:698e6c1e:::obj207:head, version: 111'207, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:698e6c1e:::obj207:head@111'207, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f1d40 con 0x507ac00 2026-02-11T23:31:28.103+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:698e6c1e:::obj207:head, version: 111'207, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:698e6c1e:::obj207:head@111'207, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.103+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:698e6c1e:::obj207:head, version: 111'207, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:698e6c1e:::obj207:head@111'207, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.103+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.103+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 538 ==== pg_backfill(progress 4.0 e 119/119 lb 4:6971da6a:::obj293:head) ==== 1062+0+0 (unknown 0 0 0) 0x6178100 con 0x507ac00 2026-02-11T23:31:28.103+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:6971da6a:::obj293:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.103+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:698e6c1e:::obj207:head, version: 111'207, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:698e6c1e:::obj207:head@111'207, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.103+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:6971da6a:::obj293:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.103+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.103+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:6971da6a:::obj293:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.103+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.103+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:698e6c1e:::obj207:head, version: 111'207, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:698e6c1e:::obj207:head@111'207, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.103+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:698e6c1e:::obj207:head, version: 111'207, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:698e6c1e:::obj207:head@111'207, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000044 MOSDPGPush(4.0 119/117 [PushOp(4:698e6c1e:::obj207:head, version: 111'207, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:698e6c1e:::obj207:head@111'207, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:6934f76e:::obj318:head local-lis/les=117/118 n=248 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.103+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.103+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6934f76e:::obj318:head local-lis/les=117/118 n=248 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:698e6c1e:::obj207:head, version: 111'207, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:698e6c1e:::obj207:head@111'207, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.103+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6934f76e:::obj318:head local-lis/les=117/118 n=248 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:698e6c1e:::obj207:head@111'207, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.103+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6934f76e:::obj318:head local-lis/les=117/118 n=248 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:698e6c1e:::obj207:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.103+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6934f76e:::obj318:head local-lis/les=117/118 n=248 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:698e6c1e:::obj207:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.103+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6934f76e:::obj318:head local-lis/les=117/118 n=248 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:698e6c1e:::obj207:head 2026-02-11T23:31:28.103+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6934f76e:::obj318:head local-lis/les=117/118 n=248 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:698e6c1e:::obj207:head v 111'207 2026-02-11T23:31:28.103+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6934f76e:::obj318:head local-lis/les=117/118 n=248 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.103+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.104+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:698e6c1e:::obj207:head, version: 111'207, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:698e6c1e:::obj207:head@111'207, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:28.104+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.104+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.104+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:6971da6a:::obj293:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.104+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:6971da6a:::obj293:head) v3 prio 15 cost 0 latency 0.000088 pg_backfill(progress 4.0 e 119/119 lb 4:6971da6a:::obj293:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:6934f76e:::obj318:head local-lis/les=117/118 n=248 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.104+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.104+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6934f76e:::obj318:head local-lis/les=117/118 n=248 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:6971da6a:::obj293:head) v3 2026-02-11T23:31:28.104+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6934f76e:::obj318:head local-lis/les=117/118 n=248 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:6971da6a:::obj293:head) 2026-02-11T23:31:28.104+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.104+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:6971da6a:::obj293:head) v3 finish latency 0.000029 2026-02-11T23:31:28.104+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6971da6a:::obj293:head local-lis/les=117/118 n=249 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.104+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.104+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.104+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6971da6a:::obj293:head local-lis/les=117/118 n=249 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.104+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.104+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.104+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.104+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7af00 con 0x507ac00 2026-02-11T23:31:28.104+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:698e6c1e:::obj207:head)]) -- 0x4f7b200 con 0x507ac00 2026-02-11T23:31:28.104+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.104+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.104+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.104+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 539 ==== MOSDPGPush(4.0 119/117 [PushOp(4:699d4764:::obj50:head, version: 111'50, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:699d4764:::obj50:head@111'50, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x56a2000 con 0x507ac00 2026-02-11T23:31:28.104+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:699d4764:::obj50:head, version: 111'50, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:699d4764:::obj50:head@111'50, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 119 2026-02-11T23:31:28.104+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:699d4764:::obj50:head, version: 111'50, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:699d4764:::obj50:head@111'50, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.104+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.104+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:699d4764:::obj50:head, version: 111'50, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:699d4764:::obj50:head@111'50, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.104+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 540 ==== pg_backfill(progress 4.0 e 119/119 lb 4:698e6c1e:::obj207:head) ==== 1062+0+0 (unknown 0 0 0) 0x6179180 con 0x507ac00 2026-02-11T23:31:28.104+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.104+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:698e6c1e:::obj207:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 119 2026-02-11T23:31:28.104+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:699d4764:::obj50:head, version: 111'50, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:699d4764:::obj50:head@111'50, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.104+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:698e6c1e:::obj207:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.104+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.104+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:699d4764:::obj50:head, version: 111'50, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:699d4764:::obj50:head@111'50, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(4.0 119/117 [PushOp(4:699d4764:::obj50:head, version: 111'50, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:699d4764:::obj50:head@111'50, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:6971da6a:::obj293:head local-lis/les=117/118 n=249 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.104+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:698e6c1e:::obj207:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.104+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.104+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6971da6a:::obj293:head local-lis/les=117/118 n=249 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:699d4764:::obj50:head, version: 111'50, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:699d4764:::obj50:head@111'50, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.104+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6971da6a:::obj293:head local-lis/les=117/118 n=249 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:699d4764:::obj50:head@111'50, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.104+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6971da6a:::obj293:head local-lis/les=117/118 n=249 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:699d4764:::obj50:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.104+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6971da6a:::obj293:head local-lis/les=117/118 n=249 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:699d4764:::obj50:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.104+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6971da6a:::obj293:head local-lis/les=117/118 n=249 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:699d4764:::obj50:head 2026-02-11T23:31:28.104+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6971da6a:::obj293:head local-lis/les=117/118 n=249 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:699d4764:::obj50:head v 111'50 2026-02-11T23:31:28.104+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6971da6a:::obj293:head local-lis/les=117/118 n=249 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.104+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.104+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:699d4764:::obj50:head, version: 111'50, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:699d4764:::obj50:head@111'50, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:28.104+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.104+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.104+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:698e6c1e:::obj207:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.104+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:698e6c1e:::obj207:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 4.0 e 119/119 lb 4:698e6c1e:::obj207:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:6971da6a:::obj293:head local-lis/les=117/118 n=249 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.104+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.104+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6971da6a:::obj293:head local-lis/les=117/118 n=249 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:698e6c1e:::obj207:head) v3 2026-02-11T23:31:28.104+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6971da6a:::obj293:head local-lis/les=117/118 n=249 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:698e6c1e:::obj207:head) 2026-02-11T23:31:28.104+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.104+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:698e6c1e:::obj207:head) v3 finish latency 0.000019 2026-02-11T23:31:28.104+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:698e6c1e:::obj207:head local-lis/les=117/118 n=250 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.104+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.104+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.104+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:698e6c1e:::obj207:head local-lis/les=117/118 n=250 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.104+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.104+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.104+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.104+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x644db00 con 0x507ac00 2026-02-11T23:31:28.104+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:699d4764:::obj50:head)]) -- 0x4f7a300 con 0x507ac00 2026-02-11T23:31:28.104+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.105+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 541 ==== MOSDPGPush(4.0 119/117 [PushOp(4:69f21cc4:::obj383:head, version: 111'383, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:69f21cc4:::obj383:head@111'383, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56aa000 con 0x507ac00 2026-02-11T23:31:28.105+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:69f21cc4:::obj383:head, version: 111'383, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:69f21cc4:::obj383:head@111'383, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 119 2026-02-11T23:31:28.105+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:69f21cc4:::obj383:head, version: 111'383, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:69f21cc4:::obj383:head@111'383, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.105+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.105+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 542 ==== pg_backfill(progress 4.0 e 119/119 lb 4:699d4764:::obj50:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d98b00 con 0x507ac00 2026-02-11T23:31:28.105+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:699d4764:::obj50:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 119 2026-02-11T23:31:28.105+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:699d4764:::obj50:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.105+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:69f21cc4:::obj383:head, version: 111'383, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:69f21cc4:::obj383:head@111'383, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.105+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.105+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:69f21cc4:::obj383:head, version: 111'383, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:69f21cc4:::obj383:head@111'383, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.105+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:69f21cc4:::obj383:head, version: 111'383, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:69f21cc4:::obj383:head@111'383, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000045 MOSDPGPush(4.0 119/117 [PushOp(4:69f21cc4:::obj383:head, version: 111'383, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:69f21cc4:::obj383:head@111'383, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:698e6c1e:::obj207:head local-lis/les=117/118 n=250 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.105+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.105+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.105+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:699d4764:::obj50:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.105+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:698e6c1e:::obj207:head local-lis/les=117/118 n=250 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:69f21cc4:::obj383:head, version: 111'383, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:69f21cc4:::obj383:head@111'383, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.105+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:698e6c1e:::obj207:head local-lis/les=117/118 n=250 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:69f21cc4:::obj383:head@111'383, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.105+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:698e6c1e:::obj207:head local-lis/les=117/118 n=250 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:69f21cc4:::obj383:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.105+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:698e6c1e:::obj207:head local-lis/les=117/118 n=250 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:69f21cc4:::obj383:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.105+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:698e6c1e:::obj207:head local-lis/les=117/118 n=250 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:69f21cc4:::obj383:head 2026-02-11T23:31:28.105+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:698e6c1e:::obj207:head local-lis/les=117/118 n=250 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:69f21cc4:::obj383:head v 111'383 2026-02-11T23:31:28.105+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:698e6c1e:::obj207:head local-lis/les=117/118 n=250 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.105+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.105+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:69f21cc4:::obj383:head, version: 111'383, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:69f21cc4:::obj383:head@111'383, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:28.105+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.105+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.105+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:699d4764:::obj50:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.105+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:699d4764:::obj50:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 4.0 e 119/119 lb 4:699d4764:::obj50:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:698e6c1e:::obj207:head local-lis/les=117/118 n=250 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.105+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.105+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:698e6c1e:::obj207:head local-lis/les=117/118 n=250 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:699d4764:::obj50:head) v3 2026-02-11T23:31:28.105+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:698e6c1e:::obj207:head local-lis/les=117/118 n=250 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:699d4764:::obj50:head) 2026-02-11T23:31:28.105+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.105+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:699d4764:::obj50:head) v3 finish latency 0.000020 2026-02-11T23:31:28.105+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:699d4764:::obj50:head local-lis/les=117/118 n=251 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.105+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.105+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.105+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:699d4764:::obj50:head local-lis/les=117/118 n=251 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.105+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.105+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.105+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.105+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x644c480 con 0x507ac00 2026-02-11T23:31:28.105+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:69f21cc4:::obj383:head)]) -- 0x4f7b500 con 0x507ac00 2026-02-11T23:31:28.105+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.105+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 543 ==== MOSDPGPush(4.0 119/117 [PushOp(4:6c7994f7:::obj432:head, version: 111'432, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6c7994f7:::obj432:head@111'432, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2b60 con 0x507ac00 2026-02-11T23:31:28.105+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:6c7994f7:::obj432:head, version: 111'432, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6c7994f7:::obj432:head@111'432, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 119 2026-02-11T23:31:28.105+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:6c7994f7:::obj432:head, version: 111'432, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6c7994f7:::obj432:head@111'432, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.105+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.105+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 544 ==== pg_backfill(progress 4.0 e 119/119 lb 4:69f21cc4:::obj383:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9b180 con 0x507ac00 2026-02-11T23:31:28.105+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:6c7994f7:::obj432:head, version: 111'432, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6c7994f7:::obj432:head@111'432, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.105+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:69f21cc4:::obj383:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.105+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.105+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:69f21cc4:::obj383:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.105+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:6c7994f7:::obj432:head, version: 111'432, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6c7994f7:::obj432:head@111'432, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.105+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.105+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:6c7994f7:::obj432:head, version: 111'432, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6c7994f7:::obj432:head@111'432, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(4.0 119/117 [PushOp(4:6c7994f7:::obj432:head, version: 111'432, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6c7994f7:::obj432:head@111'432, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:699d4764:::obj50:head local-lis/les=117/118 n=251 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.105+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.105+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.105+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:69f21cc4:::obj383:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.105+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:699d4764:::obj50:head local-lis/les=117/118 n=251 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:6c7994f7:::obj432:head, version: 111'432, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6c7994f7:::obj432:head@111'432, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.105+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:699d4764:::obj50:head local-lis/les=117/118 n=251 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:6c7994f7:::obj432:head@111'432, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.105+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:699d4764:::obj50:head local-lis/les=117/118 n=251 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:6c7994f7:::obj432:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.105+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:699d4764:::obj50:head local-lis/les=117/118 n=251 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:6c7994f7:::obj432:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.105+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:699d4764:::obj50:head local-lis/les=117/118 n=251 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:6c7994f7:::obj432:head 2026-02-11T23:31:28.105+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:699d4764:::obj50:head local-lis/les=117/118 n=251 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:6c7994f7:::obj432:head v 111'432 2026-02-11T23:31:28.105+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:699d4764:::obj50:head local-lis/les=117/118 n=251 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.105+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.105+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:6c7994f7:::obj432:head, version: 111'432, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6c7994f7:::obj432:head@111'432, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000064 2026-02-11T23:31:28.105+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.105+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.105+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:69f21cc4:::obj383:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.105+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:69f21cc4:::obj383:head) v3 prio 15 cost 0 latency 0.000084 pg_backfill(progress 4.0 e 119/119 lb 4:69f21cc4:::obj383:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:699d4764:::obj50:head local-lis/les=117/118 n=251 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.105+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.105+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:699d4764:::obj50:head local-lis/les=117/118 n=251 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:69f21cc4:::obj383:head) v3 2026-02-11T23:31:28.105+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:699d4764:::obj50:head local-lis/les=117/118 n=251 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:69f21cc4:::obj383:head) 2026-02-11T23:31:28.105+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.105+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:69f21cc4:::obj383:head) v3 finish latency 0.000020 2026-02-11T23:31:28.105+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:69f21cc4:::obj383:head local-lis/les=117/118 n=252 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.105+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.105+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.105+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:69f21cc4:::obj383:head local-lis/les=117/118 n=252 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.105+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.105+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.105+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.105+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7a900 con 0x507ac00 2026-02-11T23:31:28.105+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:6c7994f7:::obj432:head)]) -- 0x63c5c80 con 0x507ac00 2026-02-11T23:31:28.105+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.106+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 545 ==== MOSDPGPush(4.0 119/117 [PushOp(4:6dc7b1fd:::obj49:head, version: 111'49, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6dc7b1fd:::obj49:head@111'49, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x554e680 con 0x507ac00 2026-02-11T23:31:28.106+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:6dc7b1fd:::obj49:head, version: 111'49, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6dc7b1fd:::obj49:head@111'49, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000018 epoch 119 2026-02-11T23:31:28.106+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:6dc7b1fd:::obj49:head, version: 111'49, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6dc7b1fd:::obj49:head@111'49, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.106+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.106+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:6dc7b1fd:::obj49:head, version: 111'49, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6dc7b1fd:::obj49:head@111'49, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.106+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 546 ==== pg_backfill(progress 4.0 e 119/119 lb 4:6c7994f7:::obj432:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98000 con 0x507ac00 2026-02-11T23:31:28.106+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.106+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:6c7994f7:::obj432:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 119 2026-02-11T23:31:28.106+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:6dc7b1fd:::obj49:head, version: 111'49, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6dc7b1fd:::obj49:head@111'49, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.106+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:6c7994f7:::obj432:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.106+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.106+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:6dc7b1fd:::obj49:head, version: 111'49, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6dc7b1fd:::obj49:head@111'49, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000048 MOSDPGPush(4.0 119/117 [PushOp(4:6dc7b1fd:::obj49:head, version: 111'49, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6dc7b1fd:::obj49:head@111'49, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:69f21cc4:::obj383:head local-lis/les=117/118 n=252 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.106+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.106+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:6c7994f7:::obj432:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.106+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:69f21cc4:::obj383:head local-lis/les=117/118 n=252 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:6dc7b1fd:::obj49:head, version: 111'49, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6dc7b1fd:::obj49:head@111'49, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.106+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:69f21cc4:::obj383:head local-lis/les=117/118 n=252 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:6dc7b1fd:::obj49:head@111'49, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.106+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:69f21cc4:::obj383:head local-lis/les=117/118 n=252 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:6dc7b1fd:::obj49:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.106+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:69f21cc4:::obj383:head local-lis/les=117/118 n=252 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:6dc7b1fd:::obj49:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.106+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:69f21cc4:::obj383:head local-lis/les=117/118 n=252 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:6dc7b1fd:::obj49:head 2026-02-11T23:31:28.106+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:69f21cc4:::obj383:head local-lis/les=117/118 n=252 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:6dc7b1fd:::obj49:head v 111'49 2026-02-11T23:31:28.106+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:69f21cc4:::obj383:head local-lis/les=117/118 n=252 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.106+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.106+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:6dc7b1fd:::obj49:head, version: 111'49, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6dc7b1fd:::obj49:head@111'49, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:28.106+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.106+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.106+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:6c7994f7:::obj432:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.106+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:6c7994f7:::obj432:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 4.0 e 119/119 lb 4:6c7994f7:::obj432:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:69f21cc4:::obj383:head local-lis/les=117/118 n=252 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.106+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.106+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:69f21cc4:::obj383:head local-lis/les=117/118 n=252 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:6c7994f7:::obj432:head) v3 2026-02-11T23:31:28.106+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:69f21cc4:::obj383:head local-lis/les=117/118 n=252 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:6c7994f7:::obj432:head) 2026-02-11T23:31:28.106+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.106+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:6c7994f7:::obj432:head) v3 finish latency 0.000018 2026-02-11T23:31:28.106+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6c7994f7:::obj432:head local-lis/les=117/118 n=253 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.106+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.106+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.106+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6c7994f7:::obj432:head local-lis/les=117/118 n=253 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.106+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.106+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.106+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.106+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c5980 con 0x507ac00 2026-02-11T23:31:28.106+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:6dc7b1fd:::obj49:head)]) -- 0x4f7b380 con 0x507ac00 2026-02-11T23:31:28.106+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.106+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 547 ==== MOSDPGPush(4.0 119/117 [PushOp(4:6e0bfbf2:::obj372:head, version: 111'372, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6e0bfbf2:::obj372:head@111'372, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a3860 con 0x507ac00 2026-02-11T23:31:28.107+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:6e0bfbf2:::obj372:head, version: 111'372, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6e0bfbf2:::obj372:head@111'372, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000020 epoch 119 2026-02-11T23:31:28.107+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:6e0bfbf2:::obj372:head, version: 111'372, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6e0bfbf2:::obj372:head@111'372, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.107+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.107+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 548 ==== pg_backfill(progress 4.0 e 119/119 lb 4:6dc7b1fd:::obj49:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d98580 con 0x507ac00 2026-02-11T23:31:28.107+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:6dc7b1fd:::obj49:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.107+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:6e0bfbf2:::obj372:head, version: 111'372, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6e0bfbf2:::obj372:head@111'372, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.107+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:6dc7b1fd:::obj49:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.107+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.107+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:6e0bfbf2:::obj372:head, version: 111'372, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6e0bfbf2:::obj372:head@111'372, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.107+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.107+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:6e0bfbf2:::obj372:head, version: 111'372, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6e0bfbf2:::obj372:head@111'372, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000046 MOSDPGPush(4.0 119/117 [PushOp(4:6e0bfbf2:::obj372:head, version: 111'372, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6e0bfbf2:::obj372:head@111'372, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:6c7994f7:::obj432:head local-lis/les=117/118 n=253 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.107+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.107+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:6dc7b1fd:::obj49:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.107+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6c7994f7:::obj432:head local-lis/les=117/118 n=253 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:6e0bfbf2:::obj372:head, version: 111'372, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6e0bfbf2:::obj372:head@111'372, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.107+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6c7994f7:::obj432:head local-lis/les=117/118 n=253 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:6e0bfbf2:::obj372:head@111'372, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.107+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6c7994f7:::obj432:head local-lis/les=117/118 n=253 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:6e0bfbf2:::obj372:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.107+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6c7994f7:::obj432:head local-lis/les=117/118 n=253 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:6e0bfbf2:::obj372:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.107+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6c7994f7:::obj432:head local-lis/les=117/118 n=253 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:6e0bfbf2:::obj372:head 2026-02-11T23:31:28.107+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6c7994f7:::obj432:head local-lis/les=117/118 n=253 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:6e0bfbf2:::obj372:head v 111'372 2026-02-11T23:31:28.107+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6c7994f7:::obj432:head local-lis/les=117/118 n=253 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.107+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.107+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:6e0bfbf2:::obj372:head, version: 111'372, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6e0bfbf2:::obj372:head@111'372, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000078 2026-02-11T23:31:28.107+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6c7994f7:::obj432:head local-lis/les=117/118 n=253 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.107+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.107+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.107+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:6dc7b1fd:::obj49:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.107+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:6dc7b1fd:::obj49:head) v3 prio 15 cost 0 latency 0.000104 pg_backfill(progress 4.0 e 119/119 lb 4:6dc7b1fd:::obj49:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:6c7994f7:::obj432:head local-lis/les=117/118 n=253 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.107+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.107+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6c7994f7:::obj432:head local-lis/les=117/118 n=253 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:6dc7b1fd:::obj49:head) v3 2026-02-11T23:31:28.107+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6c7994f7:::obj432:head local-lis/les=117/118 n=253 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:6dc7b1fd:::obj49:head) 2026-02-11T23:31:28.107+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.107+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:6dc7b1fd:::obj49:head) v3 finish latency 0.000018 2026-02-11T23:31:28.107+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.107+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.107+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6dc7b1fd:::obj49:head local-lis/les=117/118 n=254 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.107+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.107+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.107+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.107+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x649e600 con 0x507ac00 2026-02-11T23:31:28.107+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:6e0bfbf2:::obj372:head)]) -- 0x644d800 con 0x507ac00 2026-02-11T23:31:28.107+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.107+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 549 ==== MOSDPGPush(4.0 119/117 [PushOp(4:6ec34796:::obj408:head, version: 111'408, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec34796:::obj408:head@111'408, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56171e0 con 0x507ac00 2026-02-11T23:31:28.107+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:6ec34796:::obj408:head, version: 111'408, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec34796:::obj408:head@111'408, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000015 epoch 119 2026-02-11T23:31:28.107+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:6ec34796:::obj408:head, version: 111'408, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec34796:::obj408:head@111'408, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.107+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.107+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 550 ==== pg_backfill(progress 4.0 e 119/119 lb 4:6e0bfbf2:::obj372:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2d600 con 0x507ac00 2026-02-11T23:31:28.107+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:6ec34796:::obj408:head, version: 111'408, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec34796:::obj408:head@111'408, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.107+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:6e0bfbf2:::obj372:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.107+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.107+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:6e0bfbf2:::obj372:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.107+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:6ec34796:::obj408:head, version: 111'408, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec34796:::obj408:head@111'408, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.107+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.107+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:6ec34796:::obj408:head, version: 111'408, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec34796:::obj408:head@111'408, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(4.0 119/117 [PushOp(4:6ec34796:::obj408:head, version: 111'408, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec34796:::obj408:head@111'408, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:6dc7b1fd:::obj49:head local-lis/les=117/118 n=254 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.107+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.107+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6dc7b1fd:::obj49:head local-lis/les=117/118 n=254 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:6ec34796:::obj408:head, version: 111'408, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec34796:::obj408:head@111'408, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.107+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.107+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6dc7b1fd:::obj49:head local-lis/les=117/118 n=254 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:6ec34796:::obj408:head@111'408, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.107+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:6e0bfbf2:::obj372:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.107+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6dc7b1fd:::obj49:head local-lis/les=117/118 n=254 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:6ec34796:::obj408:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.107+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6dc7b1fd:::obj49:head local-lis/les=117/118 n=254 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:6ec34796:::obj408:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.107+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6dc7b1fd:::obj49:head local-lis/les=117/118 n=254 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:6ec34796:::obj408:head 2026-02-11T23:31:28.107+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6dc7b1fd:::obj49:head local-lis/les=117/118 n=254 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:6ec34796:::obj408:head v 111'408 2026-02-11T23:31:28.107+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6dc7b1fd:::obj49:head local-lis/les=117/118 n=254 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.107+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.107+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:6ec34796:::obj408:head, version: 111'408, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec34796:::obj408:head@111'408, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:28.107+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.107+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.107+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:6e0bfbf2:::obj372:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.107+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:6e0bfbf2:::obj372:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 4.0 e 119/119 lb 4:6e0bfbf2:::obj372:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:6dc7b1fd:::obj49:head local-lis/les=117/118 n=254 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.107+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.107+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6dc7b1fd:::obj49:head local-lis/les=117/118 n=254 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:6e0bfbf2:::obj372:head) v3 2026-02-11T23:31:28.107+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6dc7b1fd:::obj49:head local-lis/les=117/118 n=254 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:6e0bfbf2:::obj372:head) 2026-02-11T23:31:28.107+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.107+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:6e0bfbf2:::obj372:head) v3 finish latency 0.000019 2026-02-11T23:31:28.107+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6e0bfbf2:::obj372:head local-lis/les=117/118 n=255 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.107+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.107+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.107+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6e0bfbf2:::obj372:head local-lis/les=117/118 n=255 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.107+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.107+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.107+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.107+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7ad80 con 0x507ac00 2026-02-11T23:31:28.107+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:6ec34796:::obj408:head)]) -- 0x63c8c00 con 0x507ac00 2026-02-11T23:31:28.107+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.108+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 551 ==== MOSDPGPush(4.0 119/117 [PushOp(4:6ec93c40:::obj366:head, version: 111'366, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec93c40:::obj366:head@111'366, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56cad00 con 0x507ac00 2026-02-11T23:31:28.108+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:6ec93c40:::obj366:head, version: 111'366, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec93c40:::obj366:head@111'366, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 119 2026-02-11T23:31:28.108+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:6ec93c40:::obj366:head, version: 111'366, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec93c40:::obj366:head@111'366, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.108+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.108+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 552 ==== pg_backfill(progress 4.0 e 119/119 lb 4:6ec34796:::obj408:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176b00 con 0x507ac00 2026-02-11T23:31:28.108+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:6ec93c40:::obj366:head, version: 111'366, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec93c40:::obj366:head@111'366, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.108+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:6ec34796:::obj408:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.108+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:6ec34796:::obj408:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.108+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.108+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:6ec93c40:::obj366:head, version: 111'366, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec93c40:::obj366:head@111'366, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.108+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.108+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:6ec93c40:::obj366:head, version: 111'366, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec93c40:::obj366:head@111'366, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(4.0 119/117 [PushOp(4:6ec93c40:::obj366:head, version: 111'366, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec93c40:::obj366:head@111'366, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:6e0bfbf2:::obj372:head local-lis/les=117/118 n=255 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.108+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.108+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6e0bfbf2:::obj372:head local-lis/les=117/118 n=255 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:6ec93c40:::obj366:head, version: 111'366, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec93c40:::obj366:head@111'366, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.108+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.108+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6e0bfbf2:::obj372:head local-lis/les=117/118 n=255 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:6ec93c40:::obj366:head@111'366, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.108+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:6ec34796:::obj408:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.108+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6e0bfbf2:::obj372:head local-lis/les=117/118 n=255 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:6ec93c40:::obj366:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.108+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6e0bfbf2:::obj372:head local-lis/les=117/118 n=255 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:6ec93c40:::obj366:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.108+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6e0bfbf2:::obj372:head local-lis/les=117/118 n=255 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:6ec93c40:::obj366:head 2026-02-11T23:31:28.108+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6e0bfbf2:::obj372:head local-lis/les=117/118 n=255 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:6ec93c40:::obj366:head v 111'366 2026-02-11T23:31:28.108+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6e0bfbf2:::obj372:head local-lis/les=117/118 n=255 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.108+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.108+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:6ec93c40:::obj366:head, version: 111'366, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec93c40:::obj366:head@111'366, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000065 2026-02-11T23:31:28.108+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6e0bfbf2:::obj372:head local-lis/les=117/118 n=255 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.108+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.108+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.108+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:6ec34796:::obj408:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.108+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:6ec34796:::obj408:head) v3 prio 15 cost 0 latency 0.000085 pg_backfill(progress 4.0 e 119/119 lb 4:6ec34796:::obj408:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:6e0bfbf2:::obj372:head local-lis/les=117/118 n=255 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.108+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.108+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6e0bfbf2:::obj372:head local-lis/les=117/118 n=255 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:6ec34796:::obj408:head) v3 2026-02-11T23:31:28.108+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6e0bfbf2:::obj372:head local-lis/les=117/118 n=255 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:6ec34796:::obj408:head) 2026-02-11T23:31:28.108+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.108+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:6ec34796:::obj408:head) v3 finish latency 0.000016 2026-02-11T23:31:28.108+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.108+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.108+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6ec34796:::obj408:head local-lis/les=117/118 n=256 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.108+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.108+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.108+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.108+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x5d55c80 con 0x507ac00 2026-02-11T23:31:28.108+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:6ec93c40:::obj366:head)]) -- 0x5d55e00 con 0x507ac00 2026-02-11T23:31:28.108+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.108+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 553 ==== MOSDPGPush(4.0 119/117 [PushOp(4:6ec9e2eb:::obj591:head, version: 111'591, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec9e2eb:::obj591:head@111'591, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x555cea0 con 0x507ac00 2026-02-11T23:31:28.108+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:6ec9e2eb:::obj591:head, version: 111'591, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec9e2eb:::obj591:head@111'591, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 119 2026-02-11T23:31:28.108+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:6ec9e2eb:::obj591:head, version: 111'591, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec9e2eb:::obj591:head@111'591, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.108+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.108+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 554 ==== pg_backfill(progress 4.0 e 119/119 lb 4:6ec93c40:::obj366:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99080 con 0x507ac00 2026-02-11T23:31:28.108+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:6ec9e2eb:::obj591:head, version: 111'591, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec9e2eb:::obj591:head@111'591, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.109+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:6ec93c40:::obj366:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 119 2026-02-11T23:31:28.109+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:6ec93c40:::obj366:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.109+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.109+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:6ec9e2eb:::obj591:head, version: 111'591, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec9e2eb:::obj591:head@111'591, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.109+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.109+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:6ec9e2eb:::obj591:head, version: 111'591, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec9e2eb:::obj591:head@111'591, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000040 MOSDPGPush(4.0 119/117 [PushOp(4:6ec9e2eb:::obj591:head, version: 111'591, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec9e2eb:::obj591:head@111'591, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:6ec34796:::obj408:head local-lis/les=117/118 n=256 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.109+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.109+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6ec34796:::obj408:head local-lis/les=117/118 n=256 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:6ec9e2eb:::obj591:head, version: 111'591, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec9e2eb:::obj591:head@111'591, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.109+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.109+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:6ec93c40:::obj366:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.109+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6ec34796:::obj408:head local-lis/les=117/118 n=256 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:6ec9e2eb:::obj591:head@111'591, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.109+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6ec34796:::obj408:head local-lis/les=117/118 n=256 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:6ec9e2eb:::obj591:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.109+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6ec34796:::obj408:head local-lis/les=117/118 n=256 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:6ec9e2eb:::obj591:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.109+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6ec34796:::obj408:head local-lis/les=117/118 n=256 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:6ec9e2eb:::obj591:head 2026-02-11T23:31:28.109+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6ec34796:::obj408:head local-lis/les=117/118 n=256 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:6ec9e2eb:::obj591:head v 111'591 2026-02-11T23:31:28.109+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6ec34796:::obj408:head local-lis/les=117/118 n=256 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.109+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.109+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:6ec9e2eb:::obj591:head, version: 111'591, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec9e2eb:::obj591:head@111'591, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000075 2026-02-11T23:31:28.109+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6ec34796:::obj408:head local-lis/les=117/118 n=256 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.109+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.109+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.109+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:6ec93c40:::obj366:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.109+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:6ec93c40:::obj366:head) v3 prio 15 cost 0 latency 0.000104 pg_backfill(progress 4.0 e 119/119 lb 4:6ec93c40:::obj366:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:6ec34796:::obj408:head local-lis/les=117/118 n=256 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.109+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.109+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6ec34796:::obj408:head local-lis/les=117/118 n=256 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:6ec93c40:::obj366:head) v3 2026-02-11T23:31:28.109+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6ec34796:::obj408:head local-lis/les=117/118 n=256 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:6ec93c40:::obj366:head) 2026-02-11T23:31:28.109+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.109+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:6ec93c40:::obj366:head) v3 finish latency 0.000013 2026-02-11T23:31:28.109+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.109+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.109+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6ec93c40:::obj366:head local-lis/les=117/118 n=257 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.109+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.109+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.109+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.109+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x644cd80 con 0x507ac00 2026-02-11T23:31:28.109+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:6ec9e2eb:::obj591:head)]) -- 0x63c9e00 con 0x507ac00 2026-02-11T23:31:28.109+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.109+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 555 ==== MOSDPGPush(4.0 119/117 [PushOp(4:6f4176ae:::obj571:head, version: 111'571, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6f4176ae:::obj571:head@111'571, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f1ba0 con 0x507ac00 2026-02-11T23:31:28.109+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:6f4176ae:::obj571:head, version: 111'571, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6f4176ae:::obj571:head@111'571, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 119 2026-02-11T23:31:28.109+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:6f4176ae:::obj571:head, version: 111'571, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6f4176ae:::obj571:head@111'571, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.109+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.109+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 556 ==== pg_backfill(progress 4.0 e 119/119 lb 4:6ec9e2eb:::obj591:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f84000 con 0x507ac00 2026-02-11T23:31:28.109+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:6ec9e2eb:::obj591:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.109+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:6ec9e2eb:::obj591:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.109+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:6f4176ae:::obj571:head, version: 111'571, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6f4176ae:::obj571:head@111'571, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.109+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.109+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:6f4176ae:::obj571:head, version: 111'571, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6f4176ae:::obj571:head@111'571, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.109+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:6f4176ae:::obj571:head, version: 111'571, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6f4176ae:::obj571:head@111'571, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 119/117 [PushOp(4:6f4176ae:::obj571:head, version: 111'571, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6f4176ae:::obj571:head@111'571, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:6ec93c40:::obj366:head local-lis/les=117/118 n=257 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.109+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.109+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.109+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:6ec9e2eb:::obj591:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.109+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6ec93c40:::obj366:head local-lis/les=117/118 n=257 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:6f4176ae:::obj571:head, version: 111'571, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6f4176ae:::obj571:head@111'571, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.109+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6ec93c40:::obj366:head local-lis/les=117/118 n=257 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:6f4176ae:::obj571:head@111'571, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.109+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6ec93c40:::obj366:head local-lis/les=117/118 n=257 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:6f4176ae:::obj571:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.109+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6ec93c40:::obj366:head local-lis/les=117/118 n=257 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:6f4176ae:::obj571:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.109+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6ec93c40:::obj366:head local-lis/les=117/118 n=257 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:6f4176ae:::obj571:head 2026-02-11T23:31:28.109+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6ec93c40:::obj366:head local-lis/les=117/118 n=257 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:6f4176ae:::obj571:head v 111'571 2026-02-11T23:31:28.109+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6ec93c40:::obj366:head local-lis/les=117/118 n=257 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.109+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.109+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:6f4176ae:::obj571:head, version: 111'571, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6f4176ae:::obj571:head@111'571, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:28.109+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.109+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.109+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:6ec9e2eb:::obj591:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.109+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:6ec9e2eb:::obj591:head) v3 prio 15 cost 0 latency 0.000079 pg_backfill(progress 4.0 e 119/119 lb 4:6ec9e2eb:::obj591:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:6ec93c40:::obj366:head local-lis/les=117/118 n=257 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.109+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.109+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6ec93c40:::obj366:head local-lis/les=117/118 n=257 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:6ec9e2eb:::obj591:head) v3 2026-02-11T23:31:28.109+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6ec93c40:::obj366:head local-lis/les=117/118 n=257 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:6ec9e2eb:::obj591:head) 2026-02-11T23:31:28.109+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.109+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:6ec9e2eb:::obj591:head) v3 finish latency 0.000020 2026-02-11T23:31:28.109+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6ec9e2eb:::obj591:head local-lis/les=117/118 n=258 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.109+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.109+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.109+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6ec9e2eb:::obj591:head local-lis/les=117/118 n=258 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.109+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.109+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.109+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.109+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x5d55800 con 0x507ac00 2026-02-11T23:31:28.109+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:6f4176ae:::obj571:head)]) -- 0x646c180 con 0x507ac00 2026-02-11T23:31:28.109+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.110+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 557 ==== MOSDPGPush(4.0 119/117 [PushOp(4:6f5862a4:::obj430:head, version: 111'430, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6f5862a4:::obj430:head@111'430, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f0ea0 con 0x507ac00 2026-02-11T23:31:28.110+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:6f5862a4:::obj430:head, version: 111'430, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6f5862a4:::obj430:head@111'430, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 119 2026-02-11T23:31:28.110+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:6f5862a4:::obj430:head, version: 111'430, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6f5862a4:::obj430:head@111'430, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.110+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.110+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 558 ==== pg_backfill(progress 4.0 e 119/119 lb 4:6f4176ae:::obj571:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99b80 con 0x507ac00 2026-02-11T23:31:28.110+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:6f4176ae:::obj571:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 119 2026-02-11T23:31:28.110+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:6f4176ae:::obj571:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.110+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:6f5862a4:::obj430:head, version: 111'430, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6f5862a4:::obj430:head@111'430, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.110+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.110+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:6f5862a4:::obj430:head, version: 111'430, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6f5862a4:::obj430:head@111'430, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.110+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:6f5862a4:::obj430:head, version: 111'430, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6f5862a4:::obj430:head@111'430, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000042 MOSDPGPush(4.0 119/117 [PushOp(4:6f5862a4:::obj430:head, version: 111'430, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6f5862a4:::obj430:head@111'430, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:6ec9e2eb:::obj591:head local-lis/les=117/118 n=258 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.110+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.110+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.110+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6ec9e2eb:::obj591:head local-lis/les=117/118 n=258 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:6f5862a4:::obj430:head, version: 111'430, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6f5862a4:::obj430:head@111'430, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.110+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:6f4176ae:::obj571:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.110+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6ec9e2eb:::obj591:head local-lis/les=117/118 n=258 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:6f5862a4:::obj430:head@111'430, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.110+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6ec9e2eb:::obj591:head local-lis/les=117/118 n=258 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:6f5862a4:::obj430:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.110+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6ec9e2eb:::obj591:head local-lis/les=117/118 n=258 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:6f5862a4:::obj430:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.110+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6ec9e2eb:::obj591:head local-lis/les=117/118 n=258 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:6f5862a4:::obj430:head 2026-02-11T23:31:28.110+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6ec9e2eb:::obj591:head local-lis/les=117/118 n=258 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:6f5862a4:::obj430:head v 111'430 2026-02-11T23:31:28.110+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6ec9e2eb:::obj591:head local-lis/les=117/118 n=258 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.110+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.110+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:6f5862a4:::obj430:head, version: 111'430, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6f5862a4:::obj430:head@111'430, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000073 2026-02-11T23:31:28.110+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6ec9e2eb:::obj591:head local-lis/les=117/118 n=258 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.110+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.110+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.110+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:6f4176ae:::obj571:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.110+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:6f4176ae:::obj571:head) v3 prio 15 cost 0 latency 0.000105 pg_backfill(progress 4.0 e 119/119 lb 4:6f4176ae:::obj571:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:6ec9e2eb:::obj591:head local-lis/les=117/118 n=258 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.110+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.110+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6ec9e2eb:::obj591:head local-lis/les=117/118 n=258 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:6f4176ae:::obj571:head) v3 2026-02-11T23:31:28.110+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6ec9e2eb:::obj591:head local-lis/les=117/118 n=258 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:6f4176ae:::obj571:head) 2026-02-11T23:31:28.110+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.110+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:6f4176ae:::obj571:head) v3 finish latency 0.000019 2026-02-11T23:31:28.110+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.110+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.110+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6f4176ae:::obj571:head local-lis/les=117/118 n=259 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.110+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.110+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.110+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.110+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x644d080 con 0x507ac00 2026-02-11T23:31:28.110+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:6f5862a4:::obj430:head)]) -- 0x4f7ac00 con 0x507ac00 2026-02-11T23:31:28.110+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.110+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 559 ==== MOSDPGPush(4.0 119/117 [PushOp(4:70360d67:::obj288:head, version: 111'288, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:70360d67:::obj288:head@111'288, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e261a0 con 0x507ac00 2026-02-11T23:31:28.110+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:70360d67:::obj288:head, version: 111'288, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:70360d67:::obj288:head@111'288, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 119 2026-02-11T23:31:28.110+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:70360d67:::obj288:head, version: 111'288, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:70360d67:::obj288:head@111'288, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.110+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.111+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:70360d67:::obj288:head, version: 111'288, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:70360d67:::obj288:head@111'288, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.111+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 560 ==== pg_backfill(progress 4.0 e 119/119 lb 4:6f5862a4:::obj430:head) ==== 1062+0+0 (unknown 0 0 0) 0x5982b00 con 0x507ac00 2026-02-11T23:31:28.111+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:6f5862a4:::obj430:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 119 2026-02-11T23:31:28.111+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.111+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:6f5862a4:::obj430:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.111+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:70360d67:::obj288:head, version: 111'288, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:70360d67:::obj288:head@111'288, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.111+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.111+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.111+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:6f5862a4:::obj430:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.111+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:70360d67:::obj288:head, version: 111'288, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:70360d67:::obj288:head@111'288, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000049 MOSDPGPush(4.0 119/117 [PushOp(4:70360d67:::obj288:head, version: 111'288, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:70360d67:::obj288:head@111'288, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:6f4176ae:::obj571:head local-lis/les=117/118 n=259 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.111+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.111+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6f4176ae:::obj571:head local-lis/les=117/118 n=259 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:70360d67:::obj288:head, version: 111'288, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:70360d67:::obj288:head@111'288, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.111+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6f4176ae:::obj571:head local-lis/les=117/118 n=259 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:70360d67:::obj288:head@111'288, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.111+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6f4176ae:::obj571:head local-lis/les=117/118 n=259 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:70360d67:::obj288:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.111+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6f4176ae:::obj571:head local-lis/les=117/118 n=259 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:70360d67:::obj288:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.111+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6f4176ae:::obj571:head local-lis/les=117/118 n=259 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:70360d67:::obj288:head 2026-02-11T23:31:28.111+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6f4176ae:::obj571:head local-lis/les=117/118 n=259 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:70360d67:::obj288:head v 111'288 2026-02-11T23:31:28.111+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6f4176ae:::obj571:head local-lis/les=117/118 n=259 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.111+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.111+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:70360d67:::obj288:head, version: 111'288, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:70360d67:::obj288:head@111'288, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000086 2026-02-11T23:31:28.111+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.111+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.111+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:6f5862a4:::obj430:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.111+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:6f5862a4:::obj430:head) v3 prio 15 cost 0 latency 0.000117 pg_backfill(progress 4.0 e 119/119 lb 4:6f5862a4:::obj430:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:6f4176ae:::obj571:head local-lis/les=117/118 n=259 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.111+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.111+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6f4176ae:::obj571:head local-lis/les=117/118 n=259 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:6f5862a4:::obj430:head) v3 2026-02-11T23:31:28.111+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6f4176ae:::obj571:head local-lis/les=117/118 n=259 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:6f5862a4:::obj430:head) 2026-02-11T23:31:28.111+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.111+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:6f5862a4:::obj430:head) v3 finish latency 0.000018 2026-02-11T23:31:28.111+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6f5862a4:::obj430:head local-lis/les=117/118 n=260 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.111+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.111+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.111+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6f5862a4:::obj430:head local-lis/les=117/118 n=260 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.111+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.111+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.111+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.111+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x5d55200 con 0x507ac00 2026-02-11T23:31:28.111+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:70360d67:::obj288:head)]) -- 0x646c480 con 0x507ac00 2026-02-11T23:31:28.111+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.111+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 561 ==== MOSDPGPush(4.0 119/117 [PushOp(4:707a62d6:::obj465:head, version: 111'465, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:707a62d6:::obj465:head@111'465, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3814d00 con 0x507ac00 2026-02-11T23:31:28.111+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:707a62d6:::obj465:head, version: 111'465, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:707a62d6:::obj465:head@111'465, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000015 epoch 119 2026-02-11T23:31:28.111+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:707a62d6:::obj465:head, version: 111'465, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:707a62d6:::obj465:head@111'465, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.111+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.111+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 562 ==== pg_backfill(progress 4.0 e 119/119 lb 4:70360d67:::obj288:head) ==== 1062+0+0 (unknown 0 0 0) 0x4a2c000 con 0x507ac00 2026-02-11T23:31:28.111+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:70360d67:::obj288:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.111+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:707a62d6:::obj465:head, version: 111'465, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:707a62d6:::obj465:head@111'465, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.111+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:70360d67:::obj288:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.111+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.111+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:707a62d6:::obj465:head, version: 111'465, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:707a62d6:::obj465:head@111'465, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.111+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:707a62d6:::obj465:head, version: 111'465, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:707a62d6:::obj465:head@111'465, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(4.0 119/117 [PushOp(4:707a62d6:::obj465:head, version: 111'465, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:707a62d6:::obj465:head@111'465, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:6f5862a4:::obj430:head local-lis/les=117/118 n=260 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.111+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.111+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.111+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:70360d67:::obj288:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.111+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6f5862a4:::obj430:head local-lis/les=117/118 n=260 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:707a62d6:::obj465:head, version: 111'465, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:707a62d6:::obj465:head@111'465, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.111+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6f5862a4:::obj430:head local-lis/les=117/118 n=260 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:707a62d6:::obj465:head@111'465, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.111+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6f5862a4:::obj430:head local-lis/les=117/118 n=260 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:707a62d6:::obj465:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.111+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6f5862a4:::obj430:head local-lis/les=117/118 n=260 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:707a62d6:::obj465:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.111+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6f5862a4:::obj430:head local-lis/les=117/118 n=260 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:707a62d6:::obj465:head 2026-02-11T23:31:28.111+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6f5862a4:::obj430:head local-lis/les=117/118 n=260 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:707a62d6:::obj465:head v 111'465 2026-02-11T23:31:28.111+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6f5862a4:::obj430:head local-lis/les=117/118 n=260 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.111+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.111+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:707a62d6:::obj465:head, version: 111'465, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:707a62d6:::obj465:head@111'465, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000066 2026-02-11T23:31:28.111+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6f5862a4:::obj430:head local-lis/les=117/118 n=260 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.111+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.111+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.111+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:70360d67:::obj288:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.111+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:70360d67:::obj288:head) v3 prio 15 cost 0 latency 0.000088 pg_backfill(progress 4.0 e 119/119 lb 4:70360d67:::obj288:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:6f5862a4:::obj430:head local-lis/les=117/118 n=260 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.111+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.111+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6f5862a4:::obj430:head local-lis/les=117/118 n=260 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:70360d67:::obj288:head) v3 2026-02-11T23:31:28.111+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:6f5862a4:::obj430:head local-lis/les=117/118 n=260 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:70360d67:::obj288:head) 2026-02-11T23:31:28.111+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.111+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:70360d67:::obj288:head) v3 finish latency 0.000023 2026-02-11T23:31:28.111+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.111+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.111+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:70360d67:::obj288:head local-lis/les=117/118 n=261 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.111+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.111+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.111+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.111+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x644d380 con 0x507ac00 2026-02-11T23:31:28.111+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:707a62d6:::obj465:head)]) -- 0x63c5800 con 0x507ac00 2026-02-11T23:31:28.111+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.112+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 563 ==== MOSDPGPush(4.0 119/117 [PushOp(4:7109478f:::obj25:head, version: 111'25, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7109478f:::obj25:head@111'25, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x559c1a0 con 0x507ac00 2026-02-11T23:31:28.112+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:7109478f:::obj25:head, version: 111'25, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7109478f:::obj25:head@111'25, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 119 2026-02-11T23:31:28.112+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7109478f:::obj25:head, version: 111'25, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7109478f:::obj25:head@111'25, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.112+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.112+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 564 ==== pg_backfill(progress 4.0 e 119/119 lb 4:707a62d6:::obj465:head) ==== 1062+0+0 (unknown 0 0 0) 0x4a2cb00 con 0x507ac00 2026-02-11T23:31:28.112+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:707a62d6:::obj465:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 119 2026-02-11T23:31:28.112+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7109478f:::obj25:head, version: 111'25, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7109478f:::obj25:head@111'25, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.112+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:707a62d6:::obj465:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.112+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.112+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7109478f:::obj25:head, version: 111'25, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7109478f:::obj25:head@111'25, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.112+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:7109478f:::obj25:head, version: 111'25, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7109478f:::obj25:head@111'25, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(4.0 119/117 [PushOp(4:7109478f:::obj25:head, version: 111'25, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7109478f:::obj25:head@111'25, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:70360d67:::obj288:head local-lis/les=117/118 n=261 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.112+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.112+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.112+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:707a62d6:::obj465:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.112+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:70360d67:::obj288:head local-lis/les=117/118 n=261 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:7109478f:::obj25:head, version: 111'25, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7109478f:::obj25:head@111'25, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.112+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:70360d67:::obj288:head local-lis/les=117/118 n=261 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:7109478f:::obj25:head@111'25, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.112+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:70360d67:::obj288:head local-lis/les=117/118 n=261 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:7109478f:::obj25:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.112+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:70360d67:::obj288:head local-lis/les=117/118 n=261 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:7109478f:::obj25:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.112+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:70360d67:::obj288:head local-lis/les=117/118 n=261 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:7109478f:::obj25:head 2026-02-11T23:31:28.112+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:70360d67:::obj288:head local-lis/les=117/118 n=261 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:7109478f:::obj25:head v 111'25 2026-02-11T23:31:28.112+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:70360d67:::obj288:head local-lis/les=117/118 n=261 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.112+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.112+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:7109478f:::obj25:head, version: 111'25, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7109478f:::obj25:head@111'25, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000068 2026-02-11T23:31:28.112+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.112+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.112+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:707a62d6:::obj465:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.112+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:707a62d6:::obj465:head) v3 prio 15 cost 0 latency 0.000091 pg_backfill(progress 4.0 e 119/119 lb 4:707a62d6:::obj465:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:70360d67:::obj288:head local-lis/les=117/118 n=261 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.112+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.112+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:70360d67:::obj288:head local-lis/les=117/118 n=261 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:707a62d6:::obj465:head) v3 2026-02-11T23:31:28.112+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:70360d67:::obj288:head local-lis/les=117/118 n=261 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:707a62d6:::obj465:head) 2026-02-11T23:31:28.112+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.112+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:707a62d6:::obj465:head) v3 finish latency 0.000018 2026-02-11T23:31:28.112+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:707a62d6:::obj465:head local-lis/les=117/118 n=262 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.112+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.112+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.112+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:707a62d6:::obj465:head local-lis/les=117/118 n=262 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.112+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.112+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.112+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.112+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x5d54c00 con 0x507ac00 2026-02-11T23:31:28.112+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:7109478f:::obj25:head)]) -- 0x646c780 con 0x507ac00 2026-02-11T23:31:28.112+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.112+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 565 ==== MOSDPGPush(4.0 119/117 [PushOp(4:7135fd38:::obj57:head, version: 111'57, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7135fd38:::obj57:head@111'57, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x55f1ba0 con 0x507ac00 2026-02-11T23:31:28.112+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:7135fd38:::obj57:head, version: 111'57, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7135fd38:::obj57:head@111'57, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 119 2026-02-11T23:31:28.112+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7135fd38:::obj57:head, version: 111'57, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7135fd38:::obj57:head@111'57, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.112+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.112+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 566 ==== pg_backfill(progress 4.0 e 119/119 lb 4:7109478f:::obj25:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d9ac00 con 0x507ac00 2026-02-11T23:31:28.112+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7135fd38:::obj57:head, version: 111'57, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7135fd38:::obj57:head@111'57, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.112+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:7109478f:::obj25:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.112+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7109478f:::obj25:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.112+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.112+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7135fd38:::obj57:head, version: 111'57, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7135fd38:::obj57:head@111'57, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.112+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:7135fd38:::obj57:head, version: 111'57, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7135fd38:::obj57:head@111'57, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(4.0 119/117 [PushOp(4:7135fd38:::obj57:head, version: 111'57, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7135fd38:::obj57:head@111'57, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:707a62d6:::obj465:head local-lis/les=117/118 n=262 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.112+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.112+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.112+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:707a62d6:::obj465:head local-lis/les=117/118 n=262 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:7135fd38:::obj57:head, version: 111'57, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7135fd38:::obj57:head@111'57, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.112+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7109478f:::obj25:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.112+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:707a62d6:::obj465:head local-lis/les=117/118 n=262 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:7135fd38:::obj57:head@111'57, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.112+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:707a62d6:::obj465:head local-lis/les=117/118 n=262 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:7135fd38:::obj57:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.112+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:707a62d6:::obj465:head local-lis/les=117/118 n=262 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:7135fd38:::obj57:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.112+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:707a62d6:::obj465:head local-lis/les=117/118 n=262 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:7135fd38:::obj57:head 2026-02-11T23:31:28.112+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:707a62d6:::obj465:head local-lis/les=117/118 n=262 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:7135fd38:::obj57:head v 111'57 2026-02-11T23:31:28.112+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:707a62d6:::obj465:head local-lis/les=117/118 n=262 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.112+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.112+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:7135fd38:::obj57:head, version: 111'57, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7135fd38:::obj57:head@111'57, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:28.112+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.112+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.112+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7109478f:::obj25:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.112+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:7109478f:::obj25:head) v3 prio 15 cost 0 latency 0.000080 pg_backfill(progress 4.0 e 119/119 lb 4:7109478f:::obj25:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:707a62d6:::obj465:head local-lis/les=117/118 n=262 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.112+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.112+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:707a62d6:::obj465:head local-lis/les=117/118 n=262 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:7109478f:::obj25:head) v3 2026-02-11T23:31:28.112+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:707a62d6:::obj465:head local-lis/les=117/118 n=262 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:7109478f:::obj25:head) 2026-02-11T23:31:28.112+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.112+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:7109478f:::obj25:head) v3 finish latency 0.000020 2026-02-11T23:31:28.112+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7109478f:::obj25:head local-lis/les=117/118 n=263 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.112+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.113+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.113+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7109478f:::obj25:head local-lis/les=117/118 n=263 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.113+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.113+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.113+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.113+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7aa80 con 0x507ac00 2026-02-11T23:31:28.113+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:7135fd38:::obj57:head)]) -- 0x646c900 con 0x507ac00 2026-02-11T23:31:28.113+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.113+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 567 ==== MOSDPGPush(4.0 119/117 [PushOp(4:72da0e1a:::obj147:head, version: 111'147, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:72da0e1a:::obj147:head@111'147, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x555cea0 con 0x507ac00 2026-02-11T23:31:28.113+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:72da0e1a:::obj147:head, version: 111'147, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:72da0e1a:::obj147:head@111'147, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 119 2026-02-11T23:31:28.113+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:72da0e1a:::obj147:head, version: 111'147, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:72da0e1a:::obj147:head@111'147, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.113+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.113+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 568 ==== pg_backfill(progress 4.0 e 119/119 lb 4:7135fd38:::obj57:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d99600 con 0x507ac00 2026-02-11T23:31:28.113+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:7135fd38:::obj57:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.113+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:72da0e1a:::obj147:head, version: 111'147, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:72da0e1a:::obj147:head@111'147, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.113+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7135fd38:::obj57:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.113+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.113+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:72da0e1a:::obj147:head, version: 111'147, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:72da0e1a:::obj147:head@111'147, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.113+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.113+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:72da0e1a:::obj147:head, version: 111'147, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:72da0e1a:::obj147:head@111'147, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 119/117 [PushOp(4:72da0e1a:::obj147:head, version: 111'147, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:72da0e1a:::obj147:head@111'147, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:7109478f:::obj25:head local-lis/les=117/118 n=263 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.113+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7135fd38:::obj57:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.113+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.113+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7109478f:::obj25:head local-lis/les=117/118 n=263 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:72da0e1a:::obj147:head, version: 111'147, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:72da0e1a:::obj147:head@111'147, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.113+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7109478f:::obj25:head local-lis/les=117/118 n=263 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:72da0e1a:::obj147:head@111'147, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.113+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7109478f:::obj25:head local-lis/les=117/118 n=263 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:72da0e1a:::obj147:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.113+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7109478f:::obj25:head local-lis/les=117/118 n=263 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:72da0e1a:::obj147:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.113+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7109478f:::obj25:head local-lis/les=117/118 n=263 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:72da0e1a:::obj147:head 2026-02-11T23:31:28.113+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7109478f:::obj25:head local-lis/les=117/118 n=263 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:72da0e1a:::obj147:head v 111'147 2026-02-11T23:31:28.113+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7109478f:::obj25:head local-lis/les=117/118 n=263 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.113+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.113+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:72da0e1a:::obj147:head, version: 111'147, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:72da0e1a:::obj147:head@111'147, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:28.113+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.113+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.113+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7135fd38:::obj57:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.113+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:7135fd38:::obj57:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 4.0 e 119/119 lb 4:7135fd38:::obj57:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:7109478f:::obj25:head local-lis/les=117/118 n=263 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.113+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.113+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7109478f:::obj25:head local-lis/les=117/118 n=263 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:7135fd38:::obj57:head) v3 2026-02-11T23:31:28.113+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7109478f:::obj25:head local-lis/les=117/118 n=263 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:7135fd38:::obj57:head) 2026-02-11T23:31:28.113+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.113+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:7135fd38:::obj57:head) v3 finish latency 0.000018 2026-02-11T23:31:28.113+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7135fd38:::obj57:head local-lis/les=117/118 n=264 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.113+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.113+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.113+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7135fd38:::obj57:head local-lis/les=117/118 n=264 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.113+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.113+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.113+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.113+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.113+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x5d54900 con 0x507ac00 2026-02-11T23:31:28.113+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:72da0e1a:::obj147:head)]) -- 0x646ca80 con 0x507ac00 2026-02-11T23:31:28.113+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.114+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 569 ==== MOSDPGPush(4.0 119/117 [PushOp(4:73627f05:::obj80:head, version: 111'80, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:73627f05:::obj80:head@111'80, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x56cad00 con 0x507ac00 2026-02-11T23:31:28.114+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:73627f05:::obj80:head, version: 111'80, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:73627f05:::obj80:head@111'80, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 119 2026-02-11T23:31:28.114+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:73627f05:::obj80:head, version: 111'80, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:73627f05:::obj80:head@111'80, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.114+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.114+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 570 ==== pg_backfill(progress 4.0 e 119/119 lb 4:72da0e1a:::obj147:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2d600 con 0x507ac00 2026-02-11T23:31:28.114+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:72da0e1a:::obj147:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 119 2026-02-11T23:31:28.114+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:73627f05:::obj80:head, version: 111'80, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:73627f05:::obj80:head@111'80, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.114+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:72da0e1a:::obj147:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.114+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.114+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:73627f05:::obj80:head, version: 111'80, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:73627f05:::obj80:head@111'80, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.114+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.114+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:72da0e1a:::obj147:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.114+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:73627f05:::obj80:head, version: 111'80, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:73627f05:::obj80:head@111'80, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(4.0 119/117 [PushOp(4:73627f05:::obj80:head, version: 111'80, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:73627f05:::obj80:head@111'80, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:7135fd38:::obj57:head local-lis/les=117/118 n=264 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.114+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.114+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7135fd38:::obj57:head local-lis/les=117/118 n=264 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:73627f05:::obj80:head, version: 111'80, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:73627f05:::obj80:head@111'80, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.114+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7135fd38:::obj57:head local-lis/les=117/118 n=264 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:73627f05:::obj80:head@111'80, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.114+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7135fd38:::obj57:head local-lis/les=117/118 n=264 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:73627f05:::obj80:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.114+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7135fd38:::obj57:head local-lis/les=117/118 n=264 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:73627f05:::obj80:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.114+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7135fd38:::obj57:head local-lis/les=117/118 n=264 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:73627f05:::obj80:head 2026-02-11T23:31:28.114+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7135fd38:::obj57:head local-lis/les=117/118 n=264 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:73627f05:::obj80:head v 111'80 2026-02-11T23:31:28.114+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7135fd38:::obj57:head local-lis/les=117/118 n=264 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.114+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.114+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:73627f05:::obj80:head, version: 111'80, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:73627f05:::obj80:head@111'80, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000071 2026-02-11T23:31:28.114+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.114+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.114+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:72da0e1a:::obj147:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.114+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:72da0e1a:::obj147:head) v3 prio 15 cost 0 latency 0.000096 pg_backfill(progress 4.0 e 119/119 lb 4:72da0e1a:::obj147:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:7135fd38:::obj57:head local-lis/les=117/118 n=264 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.114+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.114+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7135fd38:::obj57:head local-lis/les=117/118 n=264 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:72da0e1a:::obj147:head) v3 2026-02-11T23:31:28.114+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7135fd38:::obj57:head local-lis/les=117/118 n=264 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:72da0e1a:::obj147:head) 2026-02-11T23:31:28.114+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.114+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:72da0e1a:::obj147:head) v3 finish latency 0.000012 2026-02-11T23:31:28.114+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:72da0e1a:::obj147:head local-lis/les=117/118 n=265 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.114+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.114+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.114+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:72da0e1a:::obj147:head local-lis/les=117/118 n=265 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.114+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.114+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.114+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.114+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x644d680 con 0x507ac00 2026-02-11T23:31:28.114+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:73627f05:::obj80:head)]) -- 0x646cd80 con 0x507ac00 2026-02-11T23:31:28.114+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.114+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 571 ==== MOSDPGPush(4.0 119/117 [PushOp(4:737833f6:::obj391:head, version: 111'391, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:737833f6:::obj391:head@111'391, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56171e0 con 0x507ac00 2026-02-11T23:31:28.114+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:737833f6:::obj391:head, version: 111'391, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:737833f6:::obj391:head@111'391, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 119 2026-02-11T23:31:28.114+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:737833f6:::obj391:head, version: 111'391, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:737833f6:::obj391:head@111'391, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.114+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.114+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 572 ==== pg_backfill(progress 4.0 e 119/119 lb 4:73627f05:::obj80:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d2ec00 con 0x507ac00 2026-02-11T23:31:28.114+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:737833f6:::obj391:head, version: 111'391, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:737833f6:::obj391:head@111'391, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.114+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:73627f05:::obj80:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 119 2026-02-11T23:31:28.114+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.114+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:73627f05:::obj80:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.114+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:737833f6:::obj391:head, version: 111'391, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:737833f6:::obj391:head@111'391, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.114+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.114+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:737833f6:::obj391:head, version: 111'391, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:737833f6:::obj391:head@111'391, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(4.0 119/117 [PushOp(4:737833f6:::obj391:head, version: 111'391, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:737833f6:::obj391:head@111'391, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:72da0e1a:::obj147:head local-lis/les=117/118 n=265 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.114+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.114+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:72da0e1a:::obj147:head local-lis/les=117/118 n=265 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:737833f6:::obj391:head, version: 111'391, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:737833f6:::obj391:head@111'391, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.114+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.114+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:73627f05:::obj80:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.114+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:72da0e1a:::obj147:head local-lis/les=117/118 n=265 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:737833f6:::obj391:head@111'391, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.114+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:72da0e1a:::obj147:head local-lis/les=117/118 n=265 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:737833f6:::obj391:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.114+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:72da0e1a:::obj147:head local-lis/les=117/118 n=265 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:737833f6:::obj391:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.114+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:72da0e1a:::obj147:head local-lis/les=117/118 n=265 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:737833f6:::obj391:head 2026-02-11T23:31:28.114+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:72da0e1a:::obj147:head local-lis/les=117/118 n=265 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:737833f6:::obj391:head v 111'391 2026-02-11T23:31:28.114+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:72da0e1a:::obj147:head local-lis/les=117/118 n=265 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.114+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.114+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:737833f6:::obj391:head, version: 111'391, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:737833f6:::obj391:head@111'391, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000068 2026-02-11T23:31:28.114+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.114+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.114+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:73627f05:::obj80:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.114+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:73627f05:::obj80:head) v3 prio 15 cost 0 latency 0.000091 pg_backfill(progress 4.0 e 119/119 lb 4:73627f05:::obj80:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:72da0e1a:::obj147:head local-lis/les=117/118 n=265 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.114+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.114+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:72da0e1a:::obj147:head local-lis/les=117/118 n=265 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:73627f05:::obj80:head) v3 2026-02-11T23:31:28.114+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:72da0e1a:::obj147:head local-lis/les=117/118 n=265 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:73627f05:::obj80:head) 2026-02-11T23:31:28.114+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.114+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:73627f05:::obj80:head) v3 finish latency 0.000017 2026-02-11T23:31:28.114+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:73627f05:::obj80:head local-lis/les=117/118 n=266 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.114+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.115+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.115+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:73627f05:::obj80:head local-lis/les=117/118 n=266 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.115+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.115+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.115+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.115+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7ac00 con 0x507ac00 2026-02-11T23:31:28.115+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:737833f6:::obj391:head)]) -- 0x646cf00 con 0x507ac00 2026-02-11T23:31:28.115+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.115+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 573 ==== MOSDPGPush(4.0 119/117 [PushOp(4:744a8d59:::obj243:head, version: 111'243, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:744a8d59:::obj243:head@111'243, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3815520 con 0x507ac00 2026-02-11T23:31:28.115+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:744a8d59:::obj243:head, version: 111'243, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:744a8d59:::obj243:head@111'243, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 119 2026-02-11T23:31:28.115+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:744a8d59:::obj243:head, version: 111'243, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:744a8d59:::obj243:head@111'243, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.115+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.115+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 574 ==== pg_backfill(progress 4.0 e 119/119 lb 4:737833f6:::obj391:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98000 con 0x507ac00 2026-02-11T23:31:28.115+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:744a8d59:::obj243:head, version: 111'243, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:744a8d59:::obj243:head@111'243, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.115+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:737833f6:::obj391:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 119 2026-02-11T23:31:28.115+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:737833f6:::obj391:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.115+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.115+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:744a8d59:::obj243:head, version: 111'243, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:744a8d59:::obj243:head@111'243, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.115+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:744a8d59:::obj243:head, version: 111'243, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:744a8d59:::obj243:head@111'243, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(4.0 119/117 [PushOp(4:744a8d59:::obj243:head, version: 111'243, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:744a8d59:::obj243:head@111'243, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:73627f05:::obj80:head local-lis/les=117/118 n=266 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.115+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.115+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.115+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:737833f6:::obj391:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.115+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:73627f05:::obj80:head local-lis/les=117/118 n=266 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:744a8d59:::obj243:head, version: 111'243, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:744a8d59:::obj243:head@111'243, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.115+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:73627f05:::obj80:head local-lis/les=117/118 n=266 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:744a8d59:::obj243:head@111'243, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.115+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:73627f05:::obj80:head local-lis/les=117/118 n=266 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:744a8d59:::obj243:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.115+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:73627f05:::obj80:head local-lis/les=117/118 n=266 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:744a8d59:::obj243:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.115+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:73627f05:::obj80:head local-lis/les=117/118 n=266 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:744a8d59:::obj243:head 2026-02-11T23:31:28.115+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:73627f05:::obj80:head local-lis/les=117/118 n=266 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:744a8d59:::obj243:head v 111'243 2026-02-11T23:31:28.115+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:73627f05:::obj80:head local-lis/les=117/118 n=266 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.115+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.115+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:744a8d59:::obj243:head, version: 111'243, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:744a8d59:::obj243:head@111'243, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000075 2026-02-11T23:31:28.115+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.115+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.115+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:737833f6:::obj391:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.115+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:737833f6:::obj391:head) v3 prio 15 cost 0 latency 0.000098 pg_backfill(progress 4.0 e 119/119 lb 4:737833f6:::obj391:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:73627f05:::obj80:head local-lis/les=117/118 n=266 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.115+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.115+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:73627f05:::obj80:head local-lis/les=117/118 n=266 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:737833f6:::obj391:head) v3 2026-02-11T23:31:28.115+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:73627f05:::obj80:head local-lis/les=117/118 n=266 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:737833f6:::obj391:head) 2026-02-11T23:31:28.115+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.115+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:737833f6:::obj391:head) v3 finish latency 0.000023 2026-02-11T23:31:28.115+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:737833f6:::obj391:head local-lis/les=117/118 n=267 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.115+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.115+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.115+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:737833f6:::obj391:head local-lis/les=117/118 n=267 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.115+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.115+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.115+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.115+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x644d080 con 0x507ac00 2026-02-11T23:31:28.115+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:744a8d59:::obj243:head)]) -- 0x646d200 con 0x507ac00 2026-02-11T23:31:28.115+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.116+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 575 ==== MOSDPGPush(4.0 119/117 [PushOp(4:748dbcc6:::obj428:head, version: 111'428, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:748dbcc6:::obj428:head@111'428, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x554e680 con 0x507ac00 2026-02-11T23:31:28.116+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:748dbcc6:::obj428:head, version: 111'428, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:748dbcc6:::obj428:head@111'428, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 119 2026-02-11T23:31:28.116+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:748dbcc6:::obj428:head, version: 111'428, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:748dbcc6:::obj428:head@111'428, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.116+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 576 ==== pg_backfill(progress 4.0 e 119/119 lb 4:744a8d59:::obj243:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9b180 con 0x507ac00 2026-02-11T23:31:28.116+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.116+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:744a8d59:::obj243:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.116+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:744a8d59:::obj243:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.116+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:748dbcc6:::obj428:head, version: 111'428, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:748dbcc6:::obj428:head@111'428, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.116+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.116+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:748dbcc6:::obj428:head, version: 111'428, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:748dbcc6:::obj428:head@111'428, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.116+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:748dbcc6:::obj428:head, version: 111'428, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:748dbcc6:::obj428:head@111'428, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 119/117 [PushOp(4:748dbcc6:::obj428:head, version: 111'428, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:748dbcc6:::obj428:head@111'428, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:737833f6:::obj391:head local-lis/les=117/118 n=267 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.116+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.116+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.116+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:744a8d59:::obj243:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.116+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:737833f6:::obj391:head local-lis/les=117/118 n=267 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:748dbcc6:::obj428:head, version: 111'428, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:748dbcc6:::obj428:head@111'428, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.116+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:737833f6:::obj391:head local-lis/les=117/118 n=267 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:748dbcc6:::obj428:head@111'428, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.116+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:737833f6:::obj391:head local-lis/les=117/118 n=267 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:748dbcc6:::obj428:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.116+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:737833f6:::obj391:head local-lis/les=117/118 n=267 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:748dbcc6:::obj428:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.116+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:737833f6:::obj391:head local-lis/les=117/118 n=267 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:748dbcc6:::obj428:head 2026-02-11T23:31:28.116+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:737833f6:::obj391:head local-lis/les=117/118 n=267 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:748dbcc6:::obj428:head v 111'428 2026-02-11T23:31:28.116+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:737833f6:::obj391:head local-lis/les=117/118 n=267 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.116+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.116+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:748dbcc6:::obj428:head, version: 111'428, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:748dbcc6:::obj428:head@111'428, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000075 2026-02-11T23:31:28.116+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.116+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.116+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:744a8d59:::obj243:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.116+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:744a8d59:::obj243:head) v3 prio 15 cost 0 latency 0.000097 pg_backfill(progress 4.0 e 119/119 lb 4:744a8d59:::obj243:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:737833f6:::obj391:head local-lis/les=117/118 n=267 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.116+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.116+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:737833f6:::obj391:head local-lis/les=117/118 n=267 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:744a8d59:::obj243:head) v3 2026-02-11T23:31:28.116+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:737833f6:::obj391:head local-lis/les=117/118 n=267 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:744a8d59:::obj243:head) 2026-02-11T23:31:28.116+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.116+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:744a8d59:::obj243:head) v3 finish latency 0.000018 2026-02-11T23:31:28.116+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:744a8d59:::obj243:head local-lis/les=117/118 n=268 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.116+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.116+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.116+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:744a8d59:::obj243:head local-lis/les=117/118 n=268 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.116+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.116+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.116+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.116+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x646c180 con 0x507ac00 2026-02-11T23:31:28.116+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:748dbcc6:::obj428:head)]) -- 0x646d500 con 0x507ac00 2026-02-11T23:31:28.116+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.116+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 577 ==== MOSDPGPush(4.0 119/117 [PushOp(4:7492d19a:::obj168:head, version: 111'168, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7492d19a:::obj168:head@111'168, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2b60 con 0x507ac00 2026-02-11T23:31:28.116+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:7492d19a:::obj168:head, version: 111'168, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7492d19a:::obj168:head@111'168, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 119 2026-02-11T23:31:28.116+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7492d19a:::obj168:head, version: 111'168, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7492d19a:::obj168:head@111'168, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.116+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 578 ==== pg_backfill(progress 4.0 e 119/119 lb 4:748dbcc6:::obj428:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98b00 con 0x507ac00 2026-02-11T23:31:28.116+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:748dbcc6:::obj428:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.116+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.116+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:748dbcc6:::obj428:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.116+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7492d19a:::obj168:head, version: 111'168, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7492d19a:::obj168:head@111'168, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.116+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.116+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7492d19a:::obj168:head, version: 111'168, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7492d19a:::obj168:head@111'168, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.116+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:7492d19a:::obj168:head, version: 111'168, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7492d19a:::obj168:head@111'168, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 119/117 [PushOp(4:7492d19a:::obj168:head, version: 111'168, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7492d19a:::obj168:head@111'168, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:744a8d59:::obj243:head local-lis/les=117/118 n=268 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.116+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.116+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.116+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:748dbcc6:::obj428:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.116+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:744a8d59:::obj243:head local-lis/les=117/118 n=268 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:7492d19a:::obj168:head, version: 111'168, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7492d19a:::obj168:head@111'168, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.116+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:744a8d59:::obj243:head local-lis/les=117/118 n=268 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:7492d19a:::obj168:head@111'168, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.116+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:744a8d59:::obj243:head local-lis/les=117/118 n=268 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:7492d19a:::obj168:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.116+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:744a8d59:::obj243:head local-lis/les=117/118 n=268 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:7492d19a:::obj168:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.116+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:744a8d59:::obj243:head local-lis/les=117/118 n=268 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:7492d19a:::obj168:head 2026-02-11T23:31:28.116+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:744a8d59:::obj243:head local-lis/les=117/118 n=268 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:7492d19a:::obj168:head v 111'168 2026-02-11T23:31:28.116+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:744a8d59:::obj243:head local-lis/les=117/118 n=268 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.116+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.117+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:7492d19a:::obj168:head, version: 111'168, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7492d19a:::obj168:head@111'168, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000088 2026-02-11T23:31:28.117+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.117+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.117+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:748dbcc6:::obj428:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.117+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:748dbcc6:::obj428:head) v3 prio 15 cost 0 latency 0.000114 pg_backfill(progress 4.0 e 119/119 lb 4:748dbcc6:::obj428:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:744a8d59:::obj243:head local-lis/les=117/118 n=268 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.117+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.117+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:744a8d59:::obj243:head local-lis/les=117/118 n=268 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:748dbcc6:::obj428:head) v3 2026-02-11T23:31:28.117+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:744a8d59:::obj243:head local-lis/les=117/118 n=268 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:748dbcc6:::obj428:head) 2026-02-11T23:31:28.117+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.117+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:748dbcc6:::obj428:head) v3 finish latency 0.000018 2026-02-11T23:31:28.117+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:748dbcc6:::obj428:head local-lis/les=117/118 n=269 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.117+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.117+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.117+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:748dbcc6:::obj428:head local-lis/les=117/118 n=269 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.117+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.117+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.117+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.117+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x5d55800 con 0x507ac00 2026-02-11T23:31:28.117+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:7492d19a:::obj168:head)]) -- 0x646d800 con 0x507ac00 2026-02-11T23:31:28.117+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.117+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 579 ==== MOSDPGPush(4.0 119/117 [PushOp(4:750d67aa:::obj572:head, version: 111'572, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:750d67aa:::obj572:head@111'572, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56aa000 con 0x507ac00 2026-02-11T23:31:28.117+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:750d67aa:::obj572:head, version: 111'572, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:750d67aa:::obj572:head@111'572, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 119 2026-02-11T23:31:28.117+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:750d67aa:::obj572:head, version: 111'572, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:750d67aa:::obj572:head@111'572, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.117+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 580 ==== pg_backfill(progress 4.0 e 119/119 lb 4:7492d19a:::obj168:head) ==== 1062+0+0 (unknown 0 0 0) 0x6179180 con 0x507ac00 2026-02-11T23:31:28.117+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:7492d19a:::obj168:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.117+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.117+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7492d19a:::obj168:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.117+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:750d67aa:::obj572:head, version: 111'572, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:750d67aa:::obj572:head@111'572, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.117+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.117+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:750d67aa:::obj572:head, version: 111'572, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:750d67aa:::obj572:head@111'572, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.117+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.117+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7492d19a:::obj168:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.117+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:750d67aa:::obj572:head, version: 111'572, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:750d67aa:::obj572:head@111'572, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(4.0 119/117 [PushOp(4:750d67aa:::obj572:head, version: 111'572, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:750d67aa:::obj572:head@111'572, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:748dbcc6:::obj428:head local-lis/les=117/118 n=269 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.117+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.117+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:748dbcc6:::obj428:head local-lis/les=117/118 n=269 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:750d67aa:::obj572:head, version: 111'572, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:750d67aa:::obj572:head@111'572, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.117+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:748dbcc6:::obj428:head local-lis/les=117/118 n=269 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:750d67aa:::obj572:head@111'572, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.117+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:748dbcc6:::obj428:head local-lis/les=117/118 n=269 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:750d67aa:::obj572:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.117+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:748dbcc6:::obj428:head local-lis/les=117/118 n=269 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:750d67aa:::obj572:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.117+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:748dbcc6:::obj428:head local-lis/les=117/118 n=269 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:750d67aa:::obj572:head 2026-02-11T23:31:28.117+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:748dbcc6:::obj428:head local-lis/les=117/118 n=269 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:750d67aa:::obj572:head v 111'572 2026-02-11T23:31:28.117+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:748dbcc6:::obj428:head local-lis/les=117/118 n=269 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.117+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.117+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:750d67aa:::obj572:head, version: 111'572, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:750d67aa:::obj572:head@111'572, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000076 2026-02-11T23:31:28.117+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.117+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.117+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7492d19a:::obj168:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.117+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:7492d19a:::obj168:head) v3 prio 15 cost 0 latency 0.000102 pg_backfill(progress 4.0 e 119/119 lb 4:7492d19a:::obj168:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:748dbcc6:::obj428:head local-lis/les=117/118 n=269 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.117+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.117+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:748dbcc6:::obj428:head local-lis/les=117/118 n=269 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:7492d19a:::obj168:head) v3 2026-02-11T23:31:28.117+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:748dbcc6:::obj428:head local-lis/les=117/118 n=269 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:7492d19a:::obj168:head) 2026-02-11T23:31:28.117+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.117+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:7492d19a:::obj168:head) v3 finish latency 0.000018 2026-02-11T23:31:28.117+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7492d19a:::obj168:head local-lis/les=117/118 n=270 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.117+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.117+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.117+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7492d19a:::obj168:head local-lis/les=117/118 n=270 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.117+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.117+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.117+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.117+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c9e00 con 0x507ac00 2026-02-11T23:31:28.117+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:750d67aa:::obj572:head)]) -- 0x646db00 con 0x507ac00 2026-02-11T23:31:28.117+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.118+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 581 ==== MOSDPGPush(4.0 119/117 [PushOp(4:76c71d4e:::obj92:head, version: 111'92, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:76c71d4e:::obj92:head@111'92, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x56a2000 con 0x507ac00 2026-02-11T23:31:28.118+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:76c71d4e:::obj92:head, version: 111'92, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:76c71d4e:::obj92:head@111'92, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 119 2026-02-11T23:31:28.118+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:76c71d4e:::obj92:head, version: 111'92, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:76c71d4e:::obj92:head@111'92, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.118+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.118+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 582 ==== pg_backfill(progress 4.0 e 119/119 lb 4:750d67aa:::obj572:head) ==== 1062+0+0 (unknown 0 0 0) 0x6178100 con 0x507ac00 2026-02-11T23:31:28.118+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:750d67aa:::obj572:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.118+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:76c71d4e:::obj92:head, version: 111'92, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:76c71d4e:::obj92:head@111'92, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.118+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:750d67aa:::obj572:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.118+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.118+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:76c71d4e:::obj92:head, version: 111'92, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:76c71d4e:::obj92:head@111'92, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.118+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.118+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:76c71d4e:::obj92:head, version: 111'92, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:76c71d4e:::obj92:head@111'92, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000042 MOSDPGPush(4.0 119/117 [PushOp(4:76c71d4e:::obj92:head, version: 111'92, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:76c71d4e:::obj92:head@111'92, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:7492d19a:::obj168:head local-lis/les=117/118 n=270 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.118+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:750d67aa:::obj572:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.118+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.118+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7492d19a:::obj168:head local-lis/les=117/118 n=270 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:76c71d4e:::obj92:head, version: 111'92, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:76c71d4e:::obj92:head@111'92, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.118+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7492d19a:::obj168:head local-lis/les=117/118 n=270 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:76c71d4e:::obj92:head@111'92, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.118+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7492d19a:::obj168:head local-lis/les=117/118 n=270 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:76c71d4e:::obj92:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.118+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7492d19a:::obj168:head local-lis/les=117/118 n=270 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:76c71d4e:::obj92:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.118+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7492d19a:::obj168:head local-lis/les=117/118 n=270 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:76c71d4e:::obj92:head 2026-02-11T23:31:28.118+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7492d19a:::obj168:head local-lis/les=117/118 n=270 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:76c71d4e:::obj92:head v 111'92 2026-02-11T23:31:28.118+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7492d19a:::obj168:head local-lis/les=117/118 n=270 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.118+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.118+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:76c71d4e:::obj92:head, version: 111'92, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:76c71d4e:::obj92:head@111'92, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:28.118+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.118+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.118+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:750d67aa:::obj572:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.118+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:750d67aa:::obj572:head) v3 prio 15 cost 0 latency 0.000084 pg_backfill(progress 4.0 e 119/119 lb 4:750d67aa:::obj572:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:7492d19a:::obj168:head local-lis/les=117/118 n=270 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.118+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.118+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7492d19a:::obj168:head local-lis/les=117/118 n=270 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:750d67aa:::obj572:head) v3 2026-02-11T23:31:28.118+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7492d19a:::obj168:head local-lis/les=117/118 n=270 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:750d67aa:::obj572:head) 2026-02-11T23:31:28.118+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.118+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:750d67aa:::obj572:head) v3 finish latency 0.000018 2026-02-11T23:31:28.118+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:750d67aa:::obj572:head local-lis/les=117/118 n=271 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.118+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.118+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.118+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:750d67aa:::obj572:head local-lis/les=117/118 n=271 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.118+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.118+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.118+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.118+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x644cd80 con 0x507ac00 2026-02-11T23:31:28.118+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:76c71d4e:::obj92:head)]) -- 0x646de00 con 0x507ac00 2026-02-11T23:31:28.118+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.118+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 583 ==== MOSDPGPush(4.0 119/117 [PushOp(4:76f9746b:::obj110:head, version: 111'110, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:76f9746b:::obj110:head@111'110, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f1d40 con 0x507ac00 2026-02-11T23:31:28.118+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:76f9746b:::obj110:head, version: 111'110, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:76f9746b:::obj110:head@111'110, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.118+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:76f9746b:::obj110:head, version: 111'110, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:76f9746b:::obj110:head@111'110, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.118+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.118+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 584 ==== pg_backfill(progress 4.0 e 119/119 lb 4:76c71d4e:::obj92:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d2c580 con 0x507ac00 2026-02-11T23:31:28.118+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:76f9746b:::obj110:head, version: 111'110, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:76f9746b:::obj110:head@111'110, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.118+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:76c71d4e:::obj92:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 119 2026-02-11T23:31:28.118+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:76c71d4e:::obj92:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.118+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.118+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:76f9746b:::obj110:head, version: 111'110, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:76f9746b:::obj110:head@111'110, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.118+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:76f9746b:::obj110:head, version: 111'110, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:76f9746b:::obj110:head@111'110, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(4.0 119/117 [PushOp(4:76f9746b:::obj110:head, version: 111'110, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:76f9746b:::obj110:head@111'110, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:750d67aa:::obj572:head local-lis/les=117/118 n=271 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.118+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.118+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.118+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:76c71d4e:::obj92:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.118+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:750d67aa:::obj572:head local-lis/les=117/118 n=271 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:76f9746b:::obj110:head, version: 111'110, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:76f9746b:::obj110:head@111'110, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.118+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:750d67aa:::obj572:head local-lis/les=117/118 n=271 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:76f9746b:::obj110:head@111'110, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.118+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:750d67aa:::obj572:head local-lis/les=117/118 n=271 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:76f9746b:::obj110:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.118+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:750d67aa:::obj572:head local-lis/les=117/118 n=271 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:76f9746b:::obj110:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.118+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:750d67aa:::obj572:head local-lis/les=117/118 n=271 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:76f9746b:::obj110:head 2026-02-11T23:31:28.118+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:750d67aa:::obj572:head local-lis/les=117/118 n=271 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:76f9746b:::obj110:head v 111'110 2026-02-11T23:31:28.118+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:750d67aa:::obj572:head local-lis/les=117/118 n=271 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.118+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.119+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:76f9746b:::obj110:head, version: 111'110, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:76f9746b:::obj110:head@111'110, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000083 2026-02-11T23:31:28.119+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.119+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.119+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:76c71d4e:::obj92:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.119+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:76c71d4e:::obj92:head) v3 prio 15 cost 0 latency 0.000106 pg_backfill(progress 4.0 e 119/119 lb 4:76c71d4e:::obj92:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:750d67aa:::obj572:head local-lis/les=117/118 n=271 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.119+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.119+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:750d67aa:::obj572:head local-lis/les=117/118 n=271 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:76c71d4e:::obj92:head) v3 2026-02-11T23:31:28.119+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:750d67aa:::obj572:head local-lis/les=117/118 n=271 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:76c71d4e:::obj92:head) 2026-02-11T23:31:28.119+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.119+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:76c71d4e:::obj92:head) v3 finish latency 0.000014 2026-02-11T23:31:28.119+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:76c71d4e:::obj92:head local-lis/les=117/118 n=272 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.119+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.119+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.119+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:76c71d4e:::obj92:head local-lis/les=117/118 n=272 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.119+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.119+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.119+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.119+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x5d55e00 con 0x507ac00 2026-02-11T23:31:28.119+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:76f9746b:::obj110:head)]) -- 0x63fe000 con 0x507ac00 2026-02-11T23:31:28.119+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.119+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 585 ==== MOSDPGPush(4.0 119/117 [PushOp(4:775ec905:::obj98:head, version: 111'98, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:775ec905:::obj98:head@111'98, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x554e4e0 con 0x507ac00 2026-02-11T23:31:28.119+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:775ec905:::obj98:head, version: 111'98, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:775ec905:::obj98:head@111'98, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 119 2026-02-11T23:31:28.119+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:775ec905:::obj98:head, version: 111'98, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:775ec905:::obj98:head@111'98, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.119+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.119+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 586 ==== pg_backfill(progress 4.0 e 119/119 lb 4:76f9746b:::obj110:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177b80 con 0x507ac00 2026-02-11T23:31:28.119+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:76f9746b:::obj110:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.119+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:775ec905:::obj98:head, version: 111'98, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:775ec905:::obj98:head@111'98, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.119+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:76f9746b:::obj110:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.119+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.119+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:775ec905:::obj98:head, version: 111'98, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:775ec905:::obj98:head@111'98, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.119+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:775ec905:::obj98:head, version: 111'98, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:775ec905:::obj98:head@111'98, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 119/117 [PushOp(4:775ec905:::obj98:head, version: 111'98, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:775ec905:::obj98:head@111'98, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:76c71d4e:::obj92:head local-lis/les=117/118 n=272 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.119+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.119+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.119+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:76c71d4e:::obj92:head local-lis/les=117/118 n=272 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:775ec905:::obj98:head, version: 111'98, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:775ec905:::obj98:head@111'98, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.119+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:76f9746b:::obj110:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.119+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:76c71d4e:::obj92:head local-lis/les=117/118 n=272 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:775ec905:::obj98:head@111'98, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.119+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:76c71d4e:::obj92:head local-lis/les=117/118 n=272 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:775ec905:::obj98:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.119+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:76c71d4e:::obj92:head local-lis/les=117/118 n=272 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:775ec905:::obj98:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.119+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:76c71d4e:::obj92:head local-lis/les=117/118 n=272 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:775ec905:::obj98:head 2026-02-11T23:31:28.119+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:76c71d4e:::obj92:head local-lis/les=117/118 n=272 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:775ec905:::obj98:head v 111'98 2026-02-11T23:31:28.119+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:76c71d4e:::obj92:head local-lis/les=117/118 n=272 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.119+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.119+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:775ec905:::obj98:head, version: 111'98, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:775ec905:::obj98:head@111'98, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:28.119+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:76c71d4e:::obj92:head local-lis/les=117/118 n=272 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.119+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.119+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.119+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:76f9746b:::obj110:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.119+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:76f9746b:::obj110:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 4.0 e 119/119 lb 4:76f9746b:::obj110:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:76c71d4e:::obj92:head local-lis/les=117/118 n=272 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.119+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.119+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:76c71d4e:::obj92:head local-lis/les=117/118 n=272 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:76f9746b:::obj110:head) v3 2026-02-11T23:31:28.119+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:76c71d4e:::obj92:head local-lis/les=117/118 n=272 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:76f9746b:::obj110:head) 2026-02-11T23:31:28.119+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.119+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:76f9746b:::obj110:head) v3 finish latency 0.000023 2026-02-11T23:31:28.119+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.119+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.119+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:76f9746b:::obj110:head local-lis/les=117/118 n=273 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.119+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.119+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.119+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.119+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c8c00 con 0x507ac00 2026-02-11T23:31:28.119+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:775ec905:::obj98:head)]) -- 0x5d55c80 con 0x507ac00 2026-02-11T23:31:28.119+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.120+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 587 ==== MOSDPGPush(4.0 119/117 [PushOp(4:776b5b83:::obj306:head, version: 111'306, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:776b5b83:::obj306:head@111'306, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2820 con 0x507ac00 2026-02-11T23:31:28.120+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:776b5b83:::obj306:head, version: 111'306, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:776b5b83:::obj306:head@111'306, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 119 2026-02-11T23:31:28.120+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:776b5b83:::obj306:head, version: 111'306, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:776b5b83:::obj306:head@111'306, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.120+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.120+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 588 ==== pg_backfill(progress 4.0 e 119/119 lb 4:775ec905:::obj98:head) ==== 1061+0+0 (unknown 0 0 0) 0x6177600 con 0x507ac00 2026-02-11T23:31:28.120+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:775ec905:::obj98:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.120+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:775ec905:::obj98:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.120+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:776b5b83:::obj306:head, version: 111'306, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:776b5b83:::obj306:head@111'306, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.120+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.120+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:776b5b83:::obj306:head, version: 111'306, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:776b5b83:::obj306:head@111'306, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.120+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:776b5b83:::obj306:head, version: 111'306, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:776b5b83:::obj306:head@111'306, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(4.0 119/117 [PushOp(4:776b5b83:::obj306:head, version: 111'306, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:776b5b83:::obj306:head@111'306, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:76f9746b:::obj110:head local-lis/les=117/118 n=273 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.120+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.120+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.120+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:775ec905:::obj98:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.120+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:76f9746b:::obj110:head local-lis/les=117/118 n=273 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:776b5b83:::obj306:head, version: 111'306, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:776b5b83:::obj306:head@111'306, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.120+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:76f9746b:::obj110:head local-lis/les=117/118 n=273 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:776b5b83:::obj306:head@111'306, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.120+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:76f9746b:::obj110:head local-lis/les=117/118 n=273 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:776b5b83:::obj306:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.120+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:76f9746b:::obj110:head local-lis/les=117/118 n=273 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:776b5b83:::obj306:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.120+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:76f9746b:::obj110:head local-lis/les=117/118 n=273 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:776b5b83:::obj306:head 2026-02-11T23:31:28.120+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:76f9746b:::obj110:head local-lis/les=117/118 n=273 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:776b5b83:::obj306:head v 111'306 2026-02-11T23:31:28.120+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:76f9746b:::obj110:head local-lis/les=117/118 n=273 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.120+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.120+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:776b5b83:::obj306:head, version: 111'306, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:776b5b83:::obj306:head@111'306, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000072 2026-02-11T23:31:28.120+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.120+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.120+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:775ec905:::obj98:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.120+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:775ec905:::obj98:head) v3 prio 15 cost 0 latency 0.000092 pg_backfill(progress 4.0 e 119/119 lb 4:775ec905:::obj98:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:76f9746b:::obj110:head local-lis/les=117/118 n=273 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.120+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.120+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:76f9746b:::obj110:head local-lis/les=117/118 n=273 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:775ec905:::obj98:head) v3 2026-02-11T23:31:28.120+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:76f9746b:::obj110:head local-lis/les=117/118 n=273 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:775ec905:::obj98:head) 2026-02-11T23:31:28.120+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.120+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:775ec905:::obj98:head) v3 finish latency 0.000017 2026-02-11T23:31:28.120+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:775ec905:::obj98:head local-lis/les=117/118 n=274 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.120+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.120+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.120+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:775ec905:::obj98:head local-lis/les=117/118 n=274 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.120+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.120+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.120+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.120+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7ad80 con 0x507ac00 2026-02-11T23:31:28.120+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:776b5b83:::obj306:head)]) -- 0x63fe300 con 0x507ac00 2026-02-11T23:31:28.120+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.120+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 589 ==== MOSDPGPush(4.0 119/117 [PushOp(4:779a6cde:::obj377:head, version: 111'377, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:779a6cde:::obj377:head@111'377, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a3a00 con 0x507ac00 2026-02-11T23:31:28.120+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:779a6cde:::obj377:head, version: 111'377, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:779a6cde:::obj377:head@111'377, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 119 2026-02-11T23:31:28.120+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:779a6cde:::obj377:head, version: 111'377, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:779a6cde:::obj377:head@111'377, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.120+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.120+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 590 ==== pg_backfill(progress 4.0 e 119/119 lb 4:776b5b83:::obj306:head) ==== 1062+0+0 (unknown 0 0 0) 0x5985700 con 0x507ac00 2026-02-11T23:31:28.120+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:776b5b83:::obj306:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.120+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:779a6cde:::obj377:head, version: 111'377, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:779a6cde:::obj377:head@111'377, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.120+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:776b5b83:::obj306:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.120+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.120+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:779a6cde:::obj377:head, version: 111'377, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:779a6cde:::obj377:head@111'377, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.120+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.120+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:779a6cde:::obj377:head, version: 111'377, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:779a6cde:::obj377:head@111'377, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 119/117 [PushOp(4:779a6cde:::obj377:head, version: 111'377, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:779a6cde:::obj377:head@111'377, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:775ec905:::obj98:head local-lis/les=117/118 n=274 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.120+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.120+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:775ec905:::obj98:head local-lis/les=117/118 n=274 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:779a6cde:::obj377:head, version: 111'377, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:779a6cde:::obj377:head@111'377, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.120+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:775ec905:::obj98:head local-lis/les=117/118 n=274 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:779a6cde:::obj377:head@111'377, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.120+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.120+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:776b5b83:::obj306:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.120+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:775ec905:::obj98:head local-lis/les=117/118 n=274 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:779a6cde:::obj377:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.120+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:775ec905:::obj98:head local-lis/les=117/118 n=274 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:779a6cde:::obj377:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.120+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:775ec905:::obj98:head local-lis/les=117/118 n=274 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:779a6cde:::obj377:head 2026-02-11T23:31:28.120+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:775ec905:::obj98:head local-lis/les=117/118 n=274 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:779a6cde:::obj377:head v 111'377 2026-02-11T23:31:28.120+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:775ec905:::obj98:head local-lis/les=117/118 n=274 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.120+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.120+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:779a6cde:::obj377:head, version: 111'377, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:779a6cde:::obj377:head@111'377, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:28.120+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.120+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.120+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:776b5b83:::obj306:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.120+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:776b5b83:::obj306:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 4.0 e 119/119 lb 4:776b5b83:::obj306:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:775ec905:::obj98:head local-lis/les=117/118 n=274 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.120+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.120+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:775ec905:::obj98:head local-lis/les=117/118 n=274 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:776b5b83:::obj306:head) v3 2026-02-11T23:31:28.120+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:775ec905:::obj98:head local-lis/les=117/118 n=274 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:776b5b83:::obj306:head) 2026-02-11T23:31:28.120+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.120+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:776b5b83:::obj306:head) v3 finish latency 0.000017 2026-02-11T23:31:28.120+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:776b5b83:::obj306:head local-lis/les=117/118 n=275 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.120+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.121+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.121+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:776b5b83:::obj306:head local-lis/les=117/118 n=275 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.121+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.121+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.121+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.121+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x644d800 con 0x507ac00 2026-02-11T23:31:28.121+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:779a6cde:::obj377:head)]) -- 0x63fe600 con 0x507ac00 2026-02-11T23:31:28.121+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.121+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 591 ==== MOSDPGPush(4.0 119/117 [PushOp(4:786d07d6:::obj325:head, version: 111'325, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:786d07d6:::obj325:head@111'325, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db0000 con 0x507ac00 2026-02-11T23:31:28.121+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:786d07d6:::obj325:head, version: 111'325, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:786d07d6:::obj325:head@111'325, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 119 2026-02-11T23:31:28.121+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:786d07d6:::obj325:head, version: 111'325, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:786d07d6:::obj325:head@111'325, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.121+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.121+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 592 ==== pg_backfill(progress 4.0 e 119/119 lb 4:779a6cde:::obj377:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9a680 con 0x507ac00 2026-02-11T23:31:28.121+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:779a6cde:::obj377:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.121+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:779a6cde:::obj377:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.121+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:786d07d6:::obj325:head, version: 111'325, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:786d07d6:::obj325:head@111'325, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.121+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.121+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:786d07d6:::obj325:head, version: 111'325, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:786d07d6:::obj325:head@111'325, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.121+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:786d07d6:::obj325:head, version: 111'325, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:786d07d6:::obj325:head@111'325, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(4.0 119/117 [PushOp(4:786d07d6:::obj325:head, version: 111'325, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:786d07d6:::obj325:head@111'325, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:776b5b83:::obj306:head local-lis/les=117/118 n=275 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.121+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.121+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.121+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:779a6cde:::obj377:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.121+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:776b5b83:::obj306:head local-lis/les=117/118 n=275 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:786d07d6:::obj325:head, version: 111'325, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:786d07d6:::obj325:head@111'325, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.121+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:776b5b83:::obj306:head local-lis/les=117/118 n=275 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:786d07d6:::obj325:head@111'325, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.121+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:776b5b83:::obj306:head local-lis/les=117/118 n=275 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:786d07d6:::obj325:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.121+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:776b5b83:::obj306:head local-lis/les=117/118 n=275 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:786d07d6:::obj325:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.121+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:776b5b83:::obj306:head local-lis/les=117/118 n=275 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:786d07d6:::obj325:head 2026-02-11T23:31:28.121+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:776b5b83:::obj306:head local-lis/les=117/118 n=275 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:786d07d6:::obj325:head v 111'325 2026-02-11T23:31:28.121+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:776b5b83:::obj306:head local-lis/les=117/118 n=275 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.121+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.121+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:786d07d6:::obj325:head, version: 111'325, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:786d07d6:::obj325:head@111'325, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000062 2026-02-11T23:31:28.121+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.121+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.121+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:779a6cde:::obj377:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.121+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:779a6cde:::obj377:head) v3 prio 15 cost 0 latency 0.000082 pg_backfill(progress 4.0 e 119/119 lb 4:779a6cde:::obj377:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:776b5b83:::obj306:head local-lis/les=117/118 n=275 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.121+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.121+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:776b5b83:::obj306:head local-lis/les=117/118 n=275 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:779a6cde:::obj377:head) v3 2026-02-11T23:31:28.121+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:776b5b83:::obj306:head local-lis/les=117/118 n=275 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:779a6cde:::obj377:head) 2026-02-11T23:31:28.121+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.121+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:779a6cde:::obj377:head) v3 finish latency 0.000014 2026-02-11T23:31:28.121+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:779a6cde:::obj377:head local-lis/les=117/118 n=276 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.121+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.121+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.121+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:779a6cde:::obj377:head local-lis/les=117/118 n=276 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.121+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.121+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.121+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.121+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x649e600 con 0x507ac00 2026-02-11T23:31:28.121+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:786d07d6:::obj325:head)]) -- 0x63fe900 con 0x507ac00 2026-02-11T23:31:28.121+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.122+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 593 ==== MOSDPGPush(4.0 119/117 [PushOp(4:78a5b78c:::obj388:head, version: 111'388, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:78a5b78c:::obj388:head@111'388, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56caea0 con 0x507ac00 2026-02-11T23:31:28.122+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:78a5b78c:::obj388:head, version: 111'388, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:78a5b78c:::obj388:head@111'388, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 119 2026-02-11T23:31:28.122+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:78a5b78c:::obj388:head, version: 111'388, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:78a5b78c:::obj388:head@111'388, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.122+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.122+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 594 ==== pg_backfill(progress 4.0 e 119/119 lb 4:786d07d6:::obj325:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176000 con 0x507ac00 2026-02-11T23:31:28.122+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:786d07d6:::obj325:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.122+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:78a5b78c:::obj388:head, version: 111'388, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:78a5b78c:::obj388:head@111'388, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.122+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:786d07d6:::obj325:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.122+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.122+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:78a5b78c:::obj388:head, version: 111'388, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:78a5b78c:::obj388:head@111'388, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.122+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:78a5b78c:::obj388:head, version: 111'388, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:78a5b78c:::obj388:head@111'388, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 119/117 [PushOp(4:78a5b78c:::obj388:head, version: 111'388, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:78a5b78c:::obj388:head@111'388, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:779a6cde:::obj377:head local-lis/les=117/118 n=276 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.122+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.122+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.122+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:786d07d6:::obj325:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.122+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:779a6cde:::obj377:head local-lis/les=117/118 n=276 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:78a5b78c:::obj388:head, version: 111'388, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:78a5b78c:::obj388:head@111'388, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.122+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:779a6cde:::obj377:head local-lis/les=117/118 n=276 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:78a5b78c:::obj388:head@111'388, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.122+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:779a6cde:::obj377:head local-lis/les=117/118 n=276 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:78a5b78c:::obj388:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.122+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:779a6cde:::obj377:head local-lis/les=117/118 n=276 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:78a5b78c:::obj388:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.122+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:779a6cde:::obj377:head local-lis/les=117/118 n=276 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:78a5b78c:::obj388:head 2026-02-11T23:31:28.122+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:779a6cde:::obj377:head local-lis/les=117/118 n=276 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:78a5b78c:::obj388:head v 111'388 2026-02-11T23:31:28.122+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:779a6cde:::obj377:head local-lis/les=117/118 n=276 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.122+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.122+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:78a5b78c:::obj388:head, version: 111'388, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:78a5b78c:::obj388:head@111'388, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000062 2026-02-11T23:31:28.122+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:779a6cde:::obj377:head local-lis/les=117/118 n=276 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.122+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.122+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.122+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:786d07d6:::obj325:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.122+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:786d07d6:::obj325:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 4.0 e 119/119 lb 4:786d07d6:::obj325:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:779a6cde:::obj377:head local-lis/les=117/118 n=276 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.122+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.122+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:779a6cde:::obj377:head local-lis/les=117/118 n=276 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:786d07d6:::obj325:head) v3 2026-02-11T23:31:28.122+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:779a6cde:::obj377:head local-lis/les=117/118 n=276 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:786d07d6:::obj325:head) 2026-02-11T23:31:28.122+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.122+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:786d07d6:::obj325:head) v3 finish latency 0.000022 2026-02-11T23:31:28.122+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.122+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.122+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:786d07d6:::obj325:head local-lis/les=117/118 n=277 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.122+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.122+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.122+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.122+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c5c80 con 0x507ac00 2026-02-11T23:31:28.122+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:78a5b78c:::obj388:head)]) -- 0x4f7b380 con 0x507ac00 2026-02-11T23:31:28.122+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.122+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 595 ==== MOSDPGPush(4.0 119/117 [PushOp(4:7903d8f8:::obj457:head, version: 111'457, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7903d8f8:::obj457:head@111'457, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56b64e0 con 0x507ac00 2026-02-11T23:31:28.122+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:7903d8f8:::obj457:head, version: 111'457, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7903d8f8:::obj457:head@111'457, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 119 2026-02-11T23:31:28.122+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7903d8f8:::obj457:head, version: 111'457, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7903d8f8:::obj457:head@111'457, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.122+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.122+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 596 ==== pg_backfill(progress 4.0 e 119/119 lb 4:78a5b78c:::obj388:head) ==== 1062+0+0 (unknown 0 0 0) 0x6179700 con 0x507ac00 2026-02-11T23:31:28.122+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:78a5b78c:::obj388:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.122+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7903d8f8:::obj457:head, version: 111'457, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7903d8f8:::obj457:head@111'457, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.122+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:78a5b78c:::obj388:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.122+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.122+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7903d8f8:::obj457:head, version: 111'457, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7903d8f8:::obj457:head@111'457, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.122+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.122+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:78a5b78c:::obj388:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.122+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:7903d8f8:::obj457:head, version: 111'457, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7903d8f8:::obj457:head@111'457, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(4.0 119/117 [PushOp(4:7903d8f8:::obj457:head, version: 111'457, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7903d8f8:::obj457:head@111'457, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:786d07d6:::obj325:head local-lis/les=117/118 n=277 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.122+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.122+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:786d07d6:::obj325:head local-lis/les=117/118 n=277 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:7903d8f8:::obj457:head, version: 111'457, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7903d8f8:::obj457:head@111'457, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.122+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:786d07d6:::obj325:head local-lis/les=117/118 n=277 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:7903d8f8:::obj457:head@111'457, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.122+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:786d07d6:::obj325:head local-lis/les=117/118 n=277 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:7903d8f8:::obj457:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.122+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:786d07d6:::obj325:head local-lis/les=117/118 n=277 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:7903d8f8:::obj457:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.122+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:786d07d6:::obj325:head local-lis/les=117/118 n=277 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:7903d8f8:::obj457:head 2026-02-11T23:31:28.122+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:786d07d6:::obj325:head local-lis/les=117/118 n=277 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:7903d8f8:::obj457:head v 111'457 2026-02-11T23:31:28.122+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:786d07d6:::obj325:head local-lis/les=117/118 n=277 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.122+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.122+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:7903d8f8:::obj457:head, version: 111'457, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7903d8f8:::obj457:head@111'457, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000067 2026-02-11T23:31:28.122+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.122+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.122+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:78a5b78c:::obj388:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.122+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:78a5b78c:::obj388:head) v3 prio 15 cost 0 latency 0.000086 pg_backfill(progress 4.0 e 119/119 lb 4:78a5b78c:::obj388:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:786d07d6:::obj325:head local-lis/les=117/118 n=277 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.122+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.122+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:786d07d6:::obj325:head local-lis/les=117/118 n=277 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:78a5b78c:::obj388:head) v3 2026-02-11T23:31:28.122+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:786d07d6:::obj325:head local-lis/les=117/118 n=277 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:78a5b78c:::obj388:head) 2026-02-11T23:31:28.122+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.122+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:78a5b78c:::obj388:head) v3 finish latency 0.000020 2026-02-11T23:31:28.122+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:78a5b78c:::obj388:head local-lis/les=117/118 n=278 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.122+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.123+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.123+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:78a5b78c:::obj388:head local-lis/les=117/118 n=278 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.123+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.123+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.123+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.123+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7a900 con 0x507ac00 2026-02-11T23:31:28.123+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:7903d8f8:::obj457:head)]) -- 0x63fec00 con 0x507ac00 2026-02-11T23:31:28.123+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.123+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 597 ==== MOSDPGPush(4.0 119/117 [PushOp(4:79ec8b1f:::obj508:head, version: 111'508, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:79ec8b1f:::obj508:head@111'508, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2d00 con 0x507ac00 2026-02-11T23:31:28.123+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:79ec8b1f:::obj508:head, version: 111'508, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:79ec8b1f:::obj508:head@111'508, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 119 2026-02-11T23:31:28.123+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:79ec8b1f:::obj508:head, version: 111'508, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:79ec8b1f:::obj508:head@111'508, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.123+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.123+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:79ec8b1f:::obj508:head, version: 111'508, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:79ec8b1f:::obj508:head@111'508, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.123+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 598 ==== pg_backfill(progress 4.0 e 119/119 lb 4:7903d8f8:::obj457:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2cb00 con 0x507ac00 2026-02-11T23:31:28.123+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:7903d8f8:::obj457:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.123+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.123+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7903d8f8:::obj457:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.123+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:79ec8b1f:::obj508:head, version: 111'508, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:79ec8b1f:::obj508:head@111'508, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.123+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.123+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:79ec8b1f:::obj508:head, version: 111'508, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:79ec8b1f:::obj508:head@111'508, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(4.0 119/117 [PushOp(4:79ec8b1f:::obj508:head, version: 111'508, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:79ec8b1f:::obj508:head@111'508, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:78a5b78c:::obj388:head local-lis/les=117/118 n=278 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.123+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.123+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.123+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:78a5b78c:::obj388:head local-lis/les=117/118 n=278 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:79ec8b1f:::obj508:head, version: 111'508, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:79ec8b1f:::obj508:head@111'508, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.123+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7903d8f8:::obj457:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.123+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:78a5b78c:::obj388:head local-lis/les=117/118 n=278 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:79ec8b1f:::obj508:head@111'508, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.123+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:78a5b78c:::obj388:head local-lis/les=117/118 n=278 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:79ec8b1f:::obj508:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.123+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:78a5b78c:::obj388:head local-lis/les=117/118 n=278 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:79ec8b1f:::obj508:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.123+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:78a5b78c:::obj388:head local-lis/les=117/118 n=278 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:79ec8b1f:::obj508:head 2026-02-11T23:31:28.123+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:78a5b78c:::obj388:head local-lis/les=117/118 n=278 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:79ec8b1f:::obj508:head v 111'508 2026-02-11T23:31:28.123+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:78a5b78c:::obj388:head local-lis/les=117/118 n=278 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.123+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.123+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:79ec8b1f:::obj508:head, version: 111'508, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:79ec8b1f:::obj508:head@111'508, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000064 2026-02-11T23:31:28.123+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:78a5b78c:::obj388:head local-lis/les=117/118 n=278 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.123+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.123+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.123+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7903d8f8:::obj457:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.123+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:7903d8f8:::obj457:head) v3 prio 15 cost 0 latency 0.000087 pg_backfill(progress 4.0 e 119/119 lb 4:7903d8f8:::obj457:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:78a5b78c:::obj388:head local-lis/les=117/118 n=278 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.123+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.123+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:78a5b78c:::obj388:head local-lis/les=117/118 n=278 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:7903d8f8:::obj457:head) v3 2026-02-11T23:31:28.123+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:78a5b78c:::obj388:head local-lis/les=117/118 n=278 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:7903d8f8:::obj457:head) 2026-02-11T23:31:28.123+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.123+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:7903d8f8:::obj457:head) v3 finish latency 0.000027 2026-02-11T23:31:28.123+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.123+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.123+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7903d8f8:::obj457:head local-lis/les=117/118 n=279 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.123+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.123+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.123+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.123+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7a300 con 0x507ac00 2026-02-11T23:31:28.123+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:79ec8b1f:::obj508:head)]) -- 0x4f7b500 con 0x507ac00 2026-02-11T23:31:28.123+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.123+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 599 ==== MOSDPGPush(4.0 119/117 [PushOp(4:7a057f92:::obj247:head, version: 111'247, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a057f92:::obj247:head@111'247, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c4e0 con 0x507ac00 2026-02-11T23:31:28.123+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:7a057f92:::obj247:head, version: 111'247, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a057f92:::obj247:head@111'247, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 119 2026-02-11T23:31:28.124+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7a057f92:::obj247:head, version: 111'247, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a057f92:::obj247:head@111'247, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.124+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.124+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 600 ==== pg_backfill(progress 4.0 e 119/119 lb 4:79ec8b1f:::obj508:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2c000 con 0x507ac00 2026-02-11T23:31:28.124+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:79ec8b1f:::obj508:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.124+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:79ec8b1f:::obj508:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.124+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7a057f92:::obj247:head, version: 111'247, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a057f92:::obj247:head@111'247, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.124+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.124+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7a057f92:::obj247:head, version: 111'247, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a057f92:::obj247:head@111'247, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.124+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:7a057f92:::obj247:head, version: 111'247, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a057f92:::obj247:head@111'247, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000041 MOSDPGPush(4.0 119/117 [PushOp(4:7a057f92:::obj247:head, version: 111'247, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a057f92:::obj247:head@111'247, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:7903d8f8:::obj457:head local-lis/les=117/118 n=279 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.124+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.124+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.124+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:79ec8b1f:::obj508:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.124+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7903d8f8:::obj457:head local-lis/les=117/118 n=279 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:7a057f92:::obj247:head, version: 111'247, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a057f92:::obj247:head@111'247, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.124+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7903d8f8:::obj457:head local-lis/les=117/118 n=279 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:7a057f92:::obj247:head@111'247, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.124+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7903d8f8:::obj457:head local-lis/les=117/118 n=279 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:7a057f92:::obj247:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.124+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7903d8f8:::obj457:head local-lis/les=117/118 n=279 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:7a057f92:::obj247:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.124+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7903d8f8:::obj457:head local-lis/les=117/118 n=279 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:7a057f92:::obj247:head 2026-02-11T23:31:28.124+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7903d8f8:::obj457:head local-lis/les=117/118 n=279 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:7a057f92:::obj247:head v 111'247 2026-02-11T23:31:28.124+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7903d8f8:::obj457:head local-lis/les=117/118 n=279 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.124+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.124+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:7a057f92:::obj247:head, version: 111'247, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a057f92:::obj247:head@111'247, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000073 2026-02-11T23:31:28.124+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.124+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.124+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:79ec8b1f:::obj508:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.124+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:79ec8b1f:::obj508:head) v3 prio 15 cost 0 latency 0.000093 pg_backfill(progress 4.0 e 119/119 lb 4:79ec8b1f:::obj508:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:7903d8f8:::obj457:head local-lis/les=117/118 n=279 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.124+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.124+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7903d8f8:::obj457:head local-lis/les=117/118 n=279 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:79ec8b1f:::obj508:head) v3 2026-02-11T23:31:28.124+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7903d8f8:::obj457:head local-lis/les=117/118 n=279 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:79ec8b1f:::obj508:head) 2026-02-11T23:31:28.124+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.124+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:79ec8b1f:::obj508:head) v3 finish latency 0.000018 2026-02-11T23:31:28.124+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:79ec8b1f:::obj508:head local-lis/les=117/118 n=280 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.124+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.124+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.124+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:79ec8b1f:::obj508:head local-lis/les=117/118 n=280 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.124+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.124+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.124+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.124+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x644db00 con 0x507ac00 2026-02-11T23:31:28.124+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:7a057f92:::obj247:head)]) -- 0x63fef00 con 0x507ac00 2026-02-11T23:31:28.124+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.124+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 601 ==== MOSDPGPush(4.0 119/117 [PushOp(4:7a539578:::obj175:head, version: 111'175, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a539578:::obj175:head@111'175, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3814ea0 con 0x507ac00 2026-02-11T23:31:28.124+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:7a539578:::obj175:head, version: 111'175, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a539578:::obj175:head@111'175, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 119 2026-02-11T23:31:28.124+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7a539578:::obj175:head, version: 111'175, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a539578:::obj175:head@111'175, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.124+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.124+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 602 ==== pg_backfill(progress 4.0 e 119/119 lb 4:7a057f92:::obj247:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e100 con 0x507ac00 2026-02-11T23:31:28.124+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:7a057f92:::obj247:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.124+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7a539578:::obj175:head, version: 111'175, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a539578:::obj175:head@111'175, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.124+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7a057f92:::obj247:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.124+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.124+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7a539578:::obj175:head, version: 111'175, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a539578:::obj175:head@111'175, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.124+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:7a539578:::obj175:head, version: 111'175, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a539578:::obj175:head@111'175, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(4.0 119/117 [PushOp(4:7a539578:::obj175:head, version: 111'175, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a539578:::obj175:head@111'175, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:79ec8b1f:::obj508:head local-lis/les=117/118 n=280 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.124+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.124+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.124+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:79ec8b1f:::obj508:head local-lis/les=117/118 n=280 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:7a539578:::obj175:head, version: 111'175, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a539578:::obj175:head@111'175, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.124+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7a057f92:::obj247:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.124+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:79ec8b1f:::obj508:head local-lis/les=117/118 n=280 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:7a539578:::obj175:head@111'175, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.124+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:79ec8b1f:::obj508:head local-lis/les=117/118 n=280 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:7a539578:::obj175:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.124+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:79ec8b1f:::obj508:head local-lis/les=117/118 n=280 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:7a539578:::obj175:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.124+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:79ec8b1f:::obj508:head local-lis/les=117/118 n=280 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:7a539578:::obj175:head 2026-02-11T23:31:28.124+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:79ec8b1f:::obj508:head local-lis/les=117/118 n=280 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:7a539578:::obj175:head v 111'175 2026-02-11T23:31:28.124+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:79ec8b1f:::obj508:head local-lis/les=117/118 n=280 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.124+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.124+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:7a539578:::obj175:head, version: 111'175, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a539578:::obj175:head@111'175, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:28.124+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:79ec8b1f:::obj508:head local-lis/les=117/118 n=280 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.124+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.124+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.124+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7a057f92:::obj247:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.124+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:7a057f92:::obj247:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 4.0 e 119/119 lb 4:7a057f92:::obj247:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:79ec8b1f:::obj508:head local-lis/les=117/118 n=280 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.124+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.124+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:79ec8b1f:::obj508:head local-lis/les=117/118 n=280 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:7a057f92:::obj247:head) v3 2026-02-11T23:31:28.124+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:79ec8b1f:::obj508:head local-lis/les=117/118 n=280 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:7a057f92:::obj247:head) 2026-02-11T23:31:28.124+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.124+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:7a057f92:::obj247:head) v3 finish latency 0.000018 2026-02-11T23:31:28.124+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.124+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.124+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a057f92:::obj247:head local-lis/les=117/118 n=281 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.125+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.125+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.125+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.125+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7a480 con 0x507ac00 2026-02-11T23:31:28.125+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:7a539578:::obj175:head)]) -- 0x4f7b200 con 0x507ac00 2026-02-11T23:31:28.125+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.125+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 603 ==== MOSDPGPush(4.0 119/117 [PushOp(4:7a71687f:::obj592:head, version: 111'592, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a71687f:::obj592:head@111'592, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a3ba0 con 0x507ac00 2026-02-11T23:31:28.125+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:7a71687f:::obj592:head, version: 111'592, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a71687f:::obj592:head@111'592, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 119 2026-02-11T23:31:28.125+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7a71687f:::obj592:head, version: 111'592, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a71687f:::obj592:head@111'592, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.125+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.125+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 604 ==== pg_backfill(progress 4.0 e 119/119 lb 4:7a539578:::obj175:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176580 con 0x507ac00 2026-02-11T23:31:28.125+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:7a539578:::obj175:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.125+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7a71687f:::obj592:head, version: 111'592, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a71687f:::obj592:head@111'592, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.125+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7a539578:::obj175:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.125+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.125+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7a71687f:::obj592:head, version: 111'592, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a71687f:::obj592:head@111'592, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.125+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.125+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:7a71687f:::obj592:head, version: 111'592, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a71687f:::obj592:head@111'592, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 119/117 [PushOp(4:7a71687f:::obj592:head, version: 111'592, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a71687f:::obj592:head@111'592, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:7a057f92:::obj247:head local-lis/les=117/118 n=281 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.125+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7a539578:::obj175:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.125+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.125+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a057f92:::obj247:head local-lis/les=117/118 n=281 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:7a71687f:::obj592:head, version: 111'592, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a71687f:::obj592:head@111'592, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.125+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a057f92:::obj247:head local-lis/les=117/118 n=281 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:7a71687f:::obj592:head@111'592, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.125+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a057f92:::obj247:head local-lis/les=117/118 n=281 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:7a71687f:::obj592:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.125+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a057f92:::obj247:head local-lis/les=117/118 n=281 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:7a71687f:::obj592:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.125+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a057f92:::obj247:head local-lis/les=117/118 n=281 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:7a71687f:::obj592:head 2026-02-11T23:31:28.125+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a057f92:::obj247:head local-lis/les=117/118 n=281 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:7a71687f:::obj592:head v 111'592 2026-02-11T23:31:28.125+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a057f92:::obj247:head local-lis/les=117/118 n=281 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.125+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.125+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:7a71687f:::obj592:head, version: 111'592, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a71687f:::obj592:head@111'592, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:28.125+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.125+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.125+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7a539578:::obj175:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.125+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:7a539578:::obj175:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 4.0 e 119/119 lb 4:7a539578:::obj175:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:7a057f92:::obj247:head local-lis/les=117/118 n=281 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.125+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.125+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a057f92:::obj247:head local-lis/les=117/118 n=281 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:7a539578:::obj175:head) v3 2026-02-11T23:31:28.125+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a057f92:::obj247:head local-lis/les=117/118 n=281 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:7a539578:::obj175:head) 2026-02-11T23:31:28.125+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.125+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:7a539578:::obj175:head) v3 finish latency 0.000015 2026-02-11T23:31:28.125+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a539578:::obj175:head local-lis/les=117/118 n=282 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.125+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.125+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.125+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a539578:::obj175:head local-lis/les=117/118 n=282 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.125+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.125+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.125+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.125+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x5f2dc80 con 0x507ac00 2026-02-11T23:31:28.125+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:7a71687f:::obj592:head)]) -- 0x63ff200 con 0x507ac00 2026-02-11T23:31:28.125+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.125+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 605 ==== MOSDPGPush(4.0 119/117 [PushOp(4:7a94cb37:::obj345:head, version: 111'345, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a94cb37:::obj345:head@111'345, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x38156c0 con 0x507ac00 2026-02-11T23:31:28.125+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:7a94cb37:::obj345:head, version: 111'345, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a94cb37:::obj345:head@111'345, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 119 2026-02-11T23:31:28.125+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7a94cb37:::obj345:head, version: 111'345, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a94cb37:::obj345:head@111'345, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.125+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.125+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 606 ==== pg_backfill(progress 4.0 e 119/119 lb 4:7a71687f:::obj592:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e680 con 0x507ac00 2026-02-11T23:31:28.125+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:7a71687f:::obj592:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.125+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7a94cb37:::obj345:head, version: 111'345, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a94cb37:::obj345:head@111'345, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.125+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7a71687f:::obj592:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.125+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.125+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7a94cb37:::obj345:head, version: 111'345, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a94cb37:::obj345:head@111'345, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.125+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:7a94cb37:::obj345:head, version: 111'345, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a94cb37:::obj345:head@111'345, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 119/117 [PushOp(4:7a94cb37:::obj345:head, version: 111'345, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a94cb37:::obj345:head@111'345, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:7a539578:::obj175:head local-lis/les=117/118 n=282 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.125+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.125+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.125+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7a71687f:::obj592:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.125+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a539578:::obj175:head local-lis/les=117/118 n=282 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:7a94cb37:::obj345:head, version: 111'345, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a94cb37:::obj345:head@111'345, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.125+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a539578:::obj175:head local-lis/les=117/118 n=282 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:7a94cb37:::obj345:head@111'345, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.125+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a539578:::obj175:head local-lis/les=117/118 n=282 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:7a94cb37:::obj345:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.125+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a539578:::obj175:head local-lis/les=117/118 n=282 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:7a94cb37:::obj345:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.125+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a539578:::obj175:head local-lis/les=117/118 n=282 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:7a94cb37:::obj345:head 2026-02-11T23:31:28.125+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a539578:::obj175:head local-lis/les=117/118 n=282 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:7a94cb37:::obj345:head v 111'345 2026-02-11T23:31:28.125+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a539578:::obj175:head local-lis/les=117/118 n=282 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.125+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.126+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:7a94cb37:::obj345:head, version: 111'345, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a94cb37:::obj345:head@111'345, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:28.126+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.126+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.126+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7a71687f:::obj592:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.126+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:7a71687f:::obj592:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 4.0 e 119/119 lb 4:7a71687f:::obj592:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:7a539578:::obj175:head local-lis/les=117/118 n=282 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.126+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.126+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a539578:::obj175:head local-lis/les=117/118 n=282 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:7a71687f:::obj592:head) v3 2026-02-11T23:31:28.126+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a539578:::obj175:head local-lis/les=117/118 n=282 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:7a71687f:::obj592:head) 2026-02-11T23:31:28.126+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.126+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:7a71687f:::obj592:head) v3 finish latency 0.000011 2026-02-11T23:31:28.126+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a71687f:::obj592:head local-lis/les=117/118 n=283 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.126+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.126+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.126+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a71687f:::obj592:head local-lis/les=117/118 n=283 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.126+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.126+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.126+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.126+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c9080 con 0x507ac00 2026-02-11T23:31:28.126+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:7a94cb37:::obj345:head)]) -- 0x63ff500 con 0x507ac00 2026-02-11T23:31:28.126+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.126+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 607 ==== MOSDPGPush(4.0 119/117 [PushOp(4:7a95c0e7:::obj245:head, version: 111'245, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a95c0e7:::obj245:head@111'245, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f1ba0 con 0x507ac00 2026-02-11T23:31:28.126+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:7a95c0e7:::obj245:head, version: 111'245, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a95c0e7:::obj245:head@111'245, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 119 2026-02-11T23:31:28.126+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7a95c0e7:::obj245:head, version: 111'245, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a95c0e7:::obj245:head@111'245, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.126+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.126+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 608 ==== pg_backfill(progress 4.0 e 119/119 lb 4:7a94cb37:::obj345:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2d080 con 0x507ac00 2026-02-11T23:31:28.126+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:7a94cb37:::obj345:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.126+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7a95c0e7:::obj245:head, version: 111'245, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a95c0e7:::obj245:head@111'245, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.126+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7a94cb37:::obj345:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.126+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.126+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7a95c0e7:::obj245:head, version: 111'245, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a95c0e7:::obj245:head@111'245, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.126+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:7a95c0e7:::obj245:head, version: 111'245, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a95c0e7:::obj245:head@111'245, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(4.0 119/117 [PushOp(4:7a95c0e7:::obj245:head, version: 111'245, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a95c0e7:::obj245:head@111'245, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:7a71687f:::obj592:head local-lis/les=117/118 n=283 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.126+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.126+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.126+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7a94cb37:::obj345:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.126+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a71687f:::obj592:head local-lis/les=117/118 n=283 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:7a95c0e7:::obj245:head, version: 111'245, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a95c0e7:::obj245:head@111'245, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.126+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a71687f:::obj592:head local-lis/les=117/118 n=283 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:7a95c0e7:::obj245:head@111'245, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.126+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a71687f:::obj592:head local-lis/les=117/118 n=283 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:7a95c0e7:::obj245:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.126+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a71687f:::obj592:head local-lis/les=117/118 n=283 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:7a95c0e7:::obj245:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.126+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a71687f:::obj592:head local-lis/les=117/118 n=283 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:7a95c0e7:::obj245:head 2026-02-11T23:31:28.126+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a71687f:::obj592:head local-lis/les=117/118 n=283 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:7a95c0e7:::obj245:head v 111'245 2026-02-11T23:31:28.126+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a71687f:::obj592:head local-lis/les=117/118 n=283 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.126+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.126+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:7a95c0e7:::obj245:head, version: 111'245, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a95c0e7:::obj245:head@111'245, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:28.126+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.126+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.126+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7a94cb37:::obj345:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.126+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:7a94cb37:::obj345:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 4.0 e 119/119 lb 4:7a94cb37:::obj345:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:7a71687f:::obj592:head local-lis/les=117/118 n=283 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.126+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.126+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a71687f:::obj592:head local-lis/les=117/118 n=283 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:7a94cb37:::obj345:head) v3 2026-02-11T23:31:28.126+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a71687f:::obj592:head local-lis/les=117/118 n=283 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:7a94cb37:::obj345:head) 2026-02-11T23:31:28.126+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.126+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:7a94cb37:::obj345:head) v3 finish latency 0.000013 2026-02-11T23:31:28.126+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a94cb37:::obj345:head local-lis/les=117/118 n=284 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.126+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.126+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.126+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a94cb37:::obj345:head local-lis/les=117/118 n=284 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.126+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.126+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.126+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.126+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x644c180 con 0x507ac00 2026-02-11T23:31:28.126+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:7a95c0e7:::obj245:head)]) -- 0x63ff800 con 0x507ac00 2026-02-11T23:31:28.126+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.127+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 609 ==== MOSDPGPush(4.0 119/117 [PushOp(4:7ac8ba2d:::obj506:head, version: 111'506, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ac8ba2d:::obj506:head@111'506, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x555cea0 con 0x507ac00 2026-02-11T23:31:28.127+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:7ac8ba2d:::obj506:head, version: 111'506, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ac8ba2d:::obj506:head@111'506, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 119 2026-02-11T23:31:28.127+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7ac8ba2d:::obj506:head, version: 111'506, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ac8ba2d:::obj506:head@111'506, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.127+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.127+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 610 ==== pg_backfill(progress 4.0 e 119/119 lb 4:7a95c0e7:::obj245:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2f700 con 0x507ac00 2026-02-11T23:31:28.127+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7ac8ba2d:::obj506:head, version: 111'506, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ac8ba2d:::obj506:head@111'506, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.127+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:7a95c0e7:::obj245:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.127+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7a95c0e7:::obj245:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.127+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.127+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7ac8ba2d:::obj506:head, version: 111'506, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ac8ba2d:::obj506:head@111'506, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.127+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.127+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:7ac8ba2d:::obj506:head, version: 111'506, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ac8ba2d:::obj506:head@111'506, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(4.0 119/117 [PushOp(4:7ac8ba2d:::obj506:head, version: 111'506, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ac8ba2d:::obj506:head@111'506, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:7a94cb37:::obj345:head local-lis/les=117/118 n=284 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.127+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.127+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.127+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a94cb37:::obj345:head local-lis/les=117/118 n=284 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:7ac8ba2d:::obj506:head, version: 111'506, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ac8ba2d:::obj506:head@111'506, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.127+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7a95c0e7:::obj245:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.127+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a94cb37:::obj345:head local-lis/les=117/118 n=284 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:7ac8ba2d:::obj506:head@111'506, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.127+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a94cb37:::obj345:head local-lis/les=117/118 n=284 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:7ac8ba2d:::obj506:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.127+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a94cb37:::obj345:head local-lis/les=117/118 n=284 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:7ac8ba2d:::obj506:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.127+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a94cb37:::obj345:head local-lis/les=117/118 n=284 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:7ac8ba2d:::obj506:head 2026-02-11T23:31:28.127+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a94cb37:::obj345:head local-lis/les=117/118 n=284 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:7ac8ba2d:::obj506:head v 111'506 2026-02-11T23:31:28.127+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a94cb37:::obj345:head local-lis/les=117/118 n=284 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.127+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.127+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:7ac8ba2d:::obj506:head, version: 111'506, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ac8ba2d:::obj506:head@111'506, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:28.127+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.127+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.127+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7a95c0e7:::obj245:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.127+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:7a95c0e7:::obj245:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 4.0 e 119/119 lb 4:7a95c0e7:::obj245:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:7a94cb37:::obj345:head local-lis/les=117/118 n=284 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.127+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.127+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a94cb37:::obj345:head local-lis/les=117/118 n=284 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:7a95c0e7:::obj245:head) v3 2026-02-11T23:31:28.127+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a94cb37:::obj345:head local-lis/les=117/118 n=284 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:7a95c0e7:::obj245:head) 2026-02-11T23:31:28.127+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.127+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:7a95c0e7:::obj245:head) v3 finish latency 0.000014 2026-02-11T23:31:28.127+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a95c0e7:::obj245:head local-lis/les=117/118 n=285 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.127+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.127+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.127+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a95c0e7:::obj245:head local-lis/les=117/118 n=285 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.127+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.127+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.127+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.127+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c4180 con 0x507ac00 2026-02-11T23:31:28.127+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:7ac8ba2d:::obj506:head)]) -- 0x63ffb00 con 0x507ac00 2026-02-11T23:31:28.127+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.127+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 611 ==== MOSDPGPush(4.0 119/117 [PushOp(4:7af66aa8:::obj163:head, version: 111'163, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7af66aa8:::obj163:head@111'163, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56cad00 con 0x507ac00 2026-02-11T23:31:28.127+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:7af66aa8:::obj163:head, version: 111'163, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7af66aa8:::obj163:head@111'163, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 119 2026-02-11T23:31:28.127+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7af66aa8:::obj163:head, version: 111'163, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7af66aa8:::obj163:head@111'163, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.127+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.127+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7af66aa8:::obj163:head, version: 111'163, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7af66aa8:::obj163:head@111'163, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.127+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 612 ==== pg_backfill(progress 4.0 e 119/119 lb 4:7ac8ba2d:::obj506:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f11600 con 0x507ac00 2026-02-11T23:31:28.127+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:7ac8ba2d:::obj506:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.127+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.127+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7ac8ba2d:::obj506:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.127+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7af66aa8:::obj163:head, version: 111'163, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7af66aa8:::obj163:head@111'163, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.127+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.127+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:7af66aa8:::obj163:head, version: 111'163, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7af66aa8:::obj163:head@111'163, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000043 MOSDPGPush(4.0 119/117 [PushOp(4:7af66aa8:::obj163:head, version: 111'163, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7af66aa8:::obj163:head@111'163, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:7a95c0e7:::obj245:head local-lis/les=117/118 n=285 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.127+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.128+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a95c0e7:::obj245:head local-lis/les=117/118 n=285 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:7af66aa8:::obj163:head, version: 111'163, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7af66aa8:::obj163:head@111'163, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.128+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.128+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7ac8ba2d:::obj506:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.128+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a95c0e7:::obj245:head local-lis/les=117/118 n=285 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:7af66aa8:::obj163:head@111'163, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.128+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a95c0e7:::obj245:head local-lis/les=117/118 n=285 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:7af66aa8:::obj163:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.128+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a95c0e7:::obj245:head local-lis/les=117/118 n=285 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:7af66aa8:::obj163:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.128+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a95c0e7:::obj245:head local-lis/les=117/118 n=285 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:7af66aa8:::obj163:head 2026-02-11T23:31:28.128+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a95c0e7:::obj245:head local-lis/les=117/118 n=285 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:7af66aa8:::obj163:head v 111'163 2026-02-11T23:31:28.128+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a95c0e7:::obj245:head local-lis/les=117/118 n=285 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.128+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.128+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:7af66aa8:::obj163:head, version: 111'163, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7af66aa8:::obj163:head@111'163, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:28.128+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.128+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.128+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7ac8ba2d:::obj506:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.128+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:7ac8ba2d:::obj506:head) v3 prio 15 cost 0 latency 0.000080 pg_backfill(progress 4.0 e 119/119 lb 4:7ac8ba2d:::obj506:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:7a95c0e7:::obj245:head local-lis/les=117/118 n=285 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.128+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.128+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a95c0e7:::obj245:head local-lis/les=117/118 n=285 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:7ac8ba2d:::obj506:head) v3 2026-02-11T23:31:28.128+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7a95c0e7:::obj245:head local-lis/les=117/118 n=285 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:7ac8ba2d:::obj506:head) 2026-02-11T23:31:28.128+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.128+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:7ac8ba2d:::obj506:head) v3 finish latency 0.000013 2026-02-11T23:31:28.128+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ac8ba2d:::obj506:head local-lis/les=117/118 n=286 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.128+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.128+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.128+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ac8ba2d:::obj506:head local-lis/les=117/118 n=286 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.128+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.128+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.128+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.128+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c9200 con 0x507ac00 2026-02-11T23:31:28.128+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:7af66aa8:::obj163:head)]) -- 0x63ffe00 con 0x507ac00 2026-02-11T23:31:28.128+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.128+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 613 ==== MOSDPGPush(4.0 119/117 [PushOp(4:7b326036:::obj411:head, version: 111'411, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7b326036:::obj411:head@111'411, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56171e0 con 0x507ac00 2026-02-11T23:31:28.128+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:7b326036:::obj411:head, version: 111'411, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7b326036:::obj411:head@111'411, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000016 epoch 119 2026-02-11T23:31:28.128+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7b326036:::obj411:head, version: 111'411, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7b326036:::obj411:head@111'411, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.128+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.128+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 614 ==== pg_backfill(progress 4.0 e 119/119 lb 4:7af66aa8:::obj163:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f11080 con 0x507ac00 2026-02-11T23:31:28.128+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:7af66aa8:::obj163:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.128+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7af66aa8:::obj163:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.128+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7b326036:::obj411:head, version: 111'411, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7b326036:::obj411:head@111'411, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.128+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.128+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7b326036:::obj411:head, version: 111'411, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7b326036:::obj411:head@111'411, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.128+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:7b326036:::obj411:head, version: 111'411, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7b326036:::obj411:head@111'411, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000043 MOSDPGPush(4.0 119/117 [PushOp(4:7b326036:::obj411:head, version: 111'411, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7b326036:::obj411:head@111'411, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:7ac8ba2d:::obj506:head local-lis/les=117/118 n=286 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.128+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.128+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.128+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7af66aa8:::obj163:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.128+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ac8ba2d:::obj506:head local-lis/les=117/118 n=286 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:7b326036:::obj411:head, version: 111'411, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7b326036:::obj411:head@111'411, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.128+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ac8ba2d:::obj506:head local-lis/les=117/118 n=286 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:7b326036:::obj411:head@111'411, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.128+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ac8ba2d:::obj506:head local-lis/les=117/118 n=286 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:7b326036:::obj411:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.128+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ac8ba2d:::obj506:head local-lis/les=117/118 n=286 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:7b326036:::obj411:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.128+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ac8ba2d:::obj506:head local-lis/les=117/118 n=286 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:7b326036:::obj411:head 2026-02-11T23:31:28.128+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ac8ba2d:::obj506:head local-lis/les=117/118 n=286 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:7b326036:::obj411:head v 111'411 2026-02-11T23:31:28.128+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ac8ba2d:::obj506:head local-lis/les=117/118 n=286 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.128+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.128+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:7b326036:::obj411:head, version: 111'411, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7b326036:::obj411:head@111'411, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000070 2026-02-11T23:31:28.128+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ac8ba2d:::obj506:head local-lis/les=117/118 n=286 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.128+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.128+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.128+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7af66aa8:::obj163:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.128+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:7af66aa8:::obj163:head) v3 prio 15 cost 0 latency 0.000094 pg_backfill(progress 4.0 e 119/119 lb 4:7af66aa8:::obj163:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:7ac8ba2d:::obj506:head local-lis/les=117/118 n=286 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.128+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.128+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ac8ba2d:::obj506:head local-lis/les=117/118 n=286 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:7af66aa8:::obj163:head) v3 2026-02-11T23:31:28.128+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ac8ba2d:::obj506:head local-lis/les=117/118 n=286 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:7af66aa8:::obj163:head) 2026-02-11T23:31:28.128+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.128+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:7af66aa8:::obj163:head) v3 finish latency 0.000021 2026-02-11T23:31:28.128+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.128+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.128+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7af66aa8:::obj163:head local-lis/les=117/118 n=287 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.128+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.128+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.128+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.128+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7a780 con 0x507ac00 2026-02-11T23:31:28.128+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:7b326036:::obj411:head)]) -- 0x4f7b080 con 0x507ac00 2026-02-11T23:31:28.128+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.129+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 615 ==== MOSDPGPush(4.0 119/117 [PushOp(4:7c121b42:::obj116:head, version: 111'116, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c121b42:::obj116:head@111'116, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3815520 con 0x507ac00 2026-02-11T23:31:28.129+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:7c121b42:::obj116:head, version: 111'116, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c121b42:::obj116:head@111'116, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 119 2026-02-11T23:31:28.129+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7c121b42:::obj116:head, version: 111'116, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c121b42:::obj116:head@111'116, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.129+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.129+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 616 ==== pg_backfill(progress 4.0 e 119/119 lb 4:7b326036:::obj411:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f13180 con 0x507ac00 2026-02-11T23:31:28.129+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:7b326036:::obj411:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.129+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7c121b42:::obj116:head, version: 111'116, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c121b42:::obj116:head@111'116, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.129+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7b326036:::obj411:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.129+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.129+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7c121b42:::obj116:head, version: 111'116, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c121b42:::obj116:head@111'116, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.129+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.129+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7b326036:::obj411:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.129+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:7c121b42:::obj116:head, version: 111'116, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c121b42:::obj116:head@111'116, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(4.0 119/117 [PushOp(4:7c121b42:::obj116:head, version: 111'116, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c121b42:::obj116:head@111'116, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:7af66aa8:::obj163:head local-lis/les=117/118 n=287 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.129+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.129+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7af66aa8:::obj163:head local-lis/les=117/118 n=287 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:7c121b42:::obj116:head, version: 111'116, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c121b42:::obj116:head@111'116, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.129+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7af66aa8:::obj163:head local-lis/les=117/118 n=287 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:7c121b42:::obj116:head@111'116, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.129+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7af66aa8:::obj163:head local-lis/les=117/118 n=287 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:7c121b42:::obj116:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.129+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7af66aa8:::obj163:head local-lis/les=117/118 n=287 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:7c121b42:::obj116:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.129+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7af66aa8:::obj163:head local-lis/les=117/118 n=287 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:7c121b42:::obj116:head 2026-02-11T23:31:28.129+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7af66aa8:::obj163:head local-lis/les=117/118 n=287 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:7c121b42:::obj116:head v 111'116 2026-02-11T23:31:28.129+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7af66aa8:::obj163:head local-lis/les=117/118 n=287 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.129+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.129+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:7c121b42:::obj116:head, version: 111'116, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c121b42:::obj116:head@111'116, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:28.129+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.129+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.129+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7b326036:::obj411:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.129+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:7b326036:::obj411:head) v3 prio 15 cost 0 latency 0.000079 pg_backfill(progress 4.0 e 119/119 lb 4:7b326036:::obj411:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:7af66aa8:::obj163:head local-lis/les=117/118 n=287 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.129+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.129+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7af66aa8:::obj163:head local-lis/les=117/118 n=287 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:7b326036:::obj411:head) v3 2026-02-11T23:31:28.129+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7af66aa8:::obj163:head local-lis/les=117/118 n=287 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:7b326036:::obj411:head) 2026-02-11T23:31:28.129+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.129+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:7b326036:::obj411:head) v3 finish latency 0.000016 2026-02-11T23:31:28.129+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7b326036:::obj411:head local-lis/les=117/118 n=288 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.129+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.129+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.129+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7b326036:::obj411:head local-lis/les=117/118 n=288 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.129+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.129+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.129+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.129+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x644de00 con 0x507ac00 2026-02-11T23:31:28.129+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:7c121b42:::obj116:head)]) -- 0x644d800 con 0x507ac00 2026-02-11T23:31:28.129+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.130+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 617 ==== MOSDPGPush(4.0 119/117 [PushOp(4:7c3a2962:::obj277:head, version: 111'277, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c3a2962:::obj277:head@111'277, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x554e680 con 0x507ac00 2026-02-11T23:31:28.130+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:7c3a2962:::obj277:head, version: 111'277, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c3a2962:::obj277:head@111'277, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 119 2026-02-11T23:31:28.130+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7c3a2962:::obj277:head, version: 111'277, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c3a2962:::obj277:head@111'277, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.130+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.130+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 618 ==== pg_backfill(progress 4.0 e 119/119 lb 4:7c121b42:::obj116:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f11b80 con 0x507ac00 2026-02-11T23:31:28.130+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:7c121b42:::obj116:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.130+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7c3a2962:::obj277:head, version: 111'277, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c3a2962:::obj277:head@111'277, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.130+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7c121b42:::obj116:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.130+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.130+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7c3a2962:::obj277:head, version: 111'277, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c3a2962:::obj277:head@111'277, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.130+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:7c3a2962:::obj277:head, version: 111'277, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c3a2962:::obj277:head@111'277, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(4.0 119/117 [PushOp(4:7c3a2962:::obj277:head, version: 111'277, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c3a2962:::obj277:head@111'277, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:7b326036:::obj411:head local-lis/les=117/118 n=288 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.130+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.130+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.130+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7c121b42:::obj116:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.130+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7b326036:::obj411:head local-lis/les=117/118 n=288 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:7c3a2962:::obj277:head, version: 111'277, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c3a2962:::obj277:head@111'277, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.130+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7b326036:::obj411:head local-lis/les=117/118 n=288 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:7c3a2962:::obj277:head@111'277, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.130+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7b326036:::obj411:head local-lis/les=117/118 n=288 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:7c3a2962:::obj277:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.130+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7b326036:::obj411:head local-lis/les=117/118 n=288 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:7c3a2962:::obj277:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.130+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7b326036:::obj411:head local-lis/les=117/118 n=288 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:7c3a2962:::obj277:head 2026-02-11T23:31:28.130+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7b326036:::obj411:head local-lis/les=117/118 n=288 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:7c3a2962:::obj277:head v 111'277 2026-02-11T23:31:28.130+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7b326036:::obj411:head local-lis/les=117/118 n=288 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.130+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.130+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:7c3a2962:::obj277:head, version: 111'277, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c3a2962:::obj277:head@111'277, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:28.130+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.130+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.130+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7c121b42:::obj116:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.130+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:7c121b42:::obj116:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 4.0 e 119/119 lb 4:7c121b42:::obj116:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:7b326036:::obj411:head local-lis/les=117/118 n=288 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.130+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.130+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7b326036:::obj411:head local-lis/les=117/118 n=288 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:7c121b42:::obj116:head) v3 2026-02-11T23:31:28.130+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7b326036:::obj411:head local-lis/les=117/118 n=288 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:7c121b42:::obj116:head) 2026-02-11T23:31:28.130+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.130+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:7c121b42:::obj116:head) v3 finish latency 0.000012 2026-02-11T23:31:28.130+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7c121b42:::obj116:head local-lis/les=117/118 n=289 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.130+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.130+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.130+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7c121b42:::obj116:head local-lis/les=117/118 n=289 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.130+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.130+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.130+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.130+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x640e000 con 0x507ac00 2026-02-11T23:31:28.130+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:7c3a2962:::obj277:head)]) -- 0x4f7ad80 con 0x507ac00 2026-02-11T23:31:28.130+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.130+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 619 ==== MOSDPGPush(4.0 119/117 [PushOp(4:7c406887:::obj566:head, version: 111'566, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c406887:::obj566:head@111'566, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2b60 con 0x507ac00 2026-02-11T23:31:28.130+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:7c406887:::obj566:head, version: 111'566, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c406887:::obj566:head@111'566, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 119 2026-02-11T23:31:28.130+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7c406887:::obj566:head, version: 111'566, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c406887:::obj566:head@111'566, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.130+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.130+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 620 ==== pg_backfill(progress 4.0 e 119/119 lb 4:7c3a2962:::obj277:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f12100 con 0x507ac00 2026-02-11T23:31:28.130+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:7c3a2962:::obj277:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.130+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7c406887:::obj566:head, version: 111'566, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c406887:::obj566:head@111'566, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.130+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7c3a2962:::obj277:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.130+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.130+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7c406887:::obj566:head, version: 111'566, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c406887:::obj566:head@111'566, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.130+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:7c406887:::obj566:head, version: 111'566, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c406887:::obj566:head@111'566, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(4.0 119/117 [PushOp(4:7c406887:::obj566:head, version: 111'566, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c406887:::obj566:head@111'566, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:7c121b42:::obj116:head local-lis/les=117/118 n=289 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.130+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.130+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.130+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7c3a2962:::obj277:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.130+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7c121b42:::obj116:head local-lis/les=117/118 n=289 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:7c406887:::obj566:head, version: 111'566, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c406887:::obj566:head@111'566, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.130+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7c121b42:::obj116:head local-lis/les=117/118 n=289 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:7c406887:::obj566:head@111'566, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.130+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7c121b42:::obj116:head local-lis/les=117/118 n=289 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:7c406887:::obj566:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.130+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7c121b42:::obj116:head local-lis/les=117/118 n=289 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:7c406887:::obj566:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.130+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7c121b42:::obj116:head local-lis/les=117/118 n=289 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:7c406887:::obj566:head 2026-02-11T23:31:28.130+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7c121b42:::obj116:head local-lis/les=117/118 n=289 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:7c406887:::obj566:head v 111'566 2026-02-11T23:31:28.130+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7c121b42:::obj116:head local-lis/les=117/118 n=289 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.130+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.130+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:7c406887:::obj566:head, version: 111'566, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c406887:::obj566:head@111'566, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:28.130+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.130+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.130+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7c3a2962:::obj277:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.130+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:7c3a2962:::obj277:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 4.0 e 119/119 lb 4:7c3a2962:::obj277:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:7c121b42:::obj116:head local-lis/les=117/118 n=289 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.130+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.130+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7c121b42:::obj116:head local-lis/les=117/118 n=289 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:7c3a2962:::obj277:head) v3 2026-02-11T23:31:28.130+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7c121b42:::obj116:head local-lis/les=117/118 n=289 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:7c3a2962:::obj277:head) 2026-02-11T23:31:28.130+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.130+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:7c3a2962:::obj277:head) v3 finish latency 0.000015 2026-02-11T23:31:28.130+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7c3a2962:::obj277:head local-lis/les=117/118 n=290 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.130+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.130+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.130+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7c3a2962:::obj277:head local-lis/les=117/118 n=290 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.130+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.130+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.130+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.130+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x640e180 con 0x507ac00 2026-02-11T23:31:28.130+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:7c406887:::obj566:head)]) -- 0x63c8c00 con 0x507ac00 2026-02-11T23:31:28.130+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.131+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 621 ==== MOSDPGPush(4.0 119/117 [PushOp(4:7c955dfc:::obj322:head, version: 111'322, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c955dfc:::obj322:head@111'322, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c000 con 0x507ac00 2026-02-11T23:31:28.131+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:7c955dfc:::obj322:head, version: 111'322, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c955dfc:::obj322:head@111'322, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 119 2026-02-11T23:31:28.131+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7c955dfc:::obj322:head, version: 111'322, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c955dfc:::obj322:head@111'322, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.131+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.131+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 622 ==== pg_backfill(progress 4.0 e 119/119 lb 4:7c406887:::obj566:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f12680 con 0x507ac00 2026-02-11T23:31:28.131+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:7c406887:::obj566:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.131+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7c955dfc:::obj322:head, version: 111'322, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c955dfc:::obj322:head@111'322, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.131+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7c406887:::obj566:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.131+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.131+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7c955dfc:::obj322:head, version: 111'322, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c955dfc:::obj322:head@111'322, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.131+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.131+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7c406887:::obj566:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.131+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:7c955dfc:::obj322:head, version: 111'322, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c955dfc:::obj322:head@111'322, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(4.0 119/117 [PushOp(4:7c955dfc:::obj322:head, version: 111'322, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c955dfc:::obj322:head@111'322, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:7c3a2962:::obj277:head local-lis/les=117/118 n=290 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.131+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.131+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7c3a2962:::obj277:head local-lis/les=117/118 n=290 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:7c955dfc:::obj322:head, version: 111'322, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c955dfc:::obj322:head@111'322, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.131+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7c3a2962:::obj277:head local-lis/les=117/118 n=290 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:7c955dfc:::obj322:head@111'322, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.131+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7c3a2962:::obj277:head local-lis/les=117/118 n=290 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:7c955dfc:::obj322:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.131+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7c3a2962:::obj277:head local-lis/les=117/118 n=290 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:7c955dfc:::obj322:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.131+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7c3a2962:::obj277:head local-lis/les=117/118 n=290 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:7c955dfc:::obj322:head 2026-02-11T23:31:28.131+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7c3a2962:::obj277:head local-lis/les=117/118 n=290 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:7c955dfc:::obj322:head v 111'322 2026-02-11T23:31:28.131+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7c3a2962:::obj277:head local-lis/les=117/118 n=290 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.131+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.131+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:7c955dfc:::obj322:head, version: 111'322, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c955dfc:::obj322:head@111'322, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:28.131+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.131+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.131+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7c406887:::obj566:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.131+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:7c406887:::obj566:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 4.0 e 119/119 lb 4:7c406887:::obj566:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:7c3a2962:::obj277:head local-lis/les=117/118 n=290 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.131+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.131+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7c3a2962:::obj277:head local-lis/les=117/118 n=290 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:7c406887:::obj566:head) v3 2026-02-11T23:31:28.131+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7c3a2962:::obj277:head local-lis/les=117/118 n=290 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:7c406887:::obj566:head) 2026-02-11T23:31:28.131+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.131+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:7c406887:::obj566:head) v3 finish latency 0.000012 2026-02-11T23:31:28.131+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7c406887:::obj566:head local-lis/les=117/118 n=291 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.131+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.131+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.131+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7c406887:::obj566:head local-lis/les=117/118 n=291 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.131+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.131+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.131+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.131+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x640e300 con 0x507ac00 2026-02-11T23:31:28.131+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:7c955dfc:::obj322:head)]) -- 0x5d55e00 con 0x507ac00 2026-02-11T23:31:28.131+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.132+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 623 ==== MOSDPGPush(4.0 119/117 [PushOp(4:7ccaea2b:::obj486:head, version: 111'486, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ccaea2b:::obj486:head@111'486, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56ca9c0 con 0x507ac00 2026-02-11T23:31:28.132+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:7ccaea2b:::obj486:head, version: 111'486, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ccaea2b:::obj486:head@111'486, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 119 2026-02-11T23:31:28.132+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7ccaea2b:::obj486:head, version: 111'486, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ccaea2b:::obj486:head@111'486, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.132+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.132+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7ccaea2b:::obj486:head, version: 111'486, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ccaea2b:::obj486:head@111'486, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.132+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 624 ==== pg_backfill(progress 4.0 e 119/119 lb 4:7c955dfc:::obj322:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f12c00 con 0x507ac00 2026-02-11T23:31:28.132+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.132+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7ccaea2b:::obj486:head, version: 111'486, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ccaea2b:::obj486:head@111'486, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.132+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:7c955dfc:::obj322:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 119 2026-02-11T23:31:28.132+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7c955dfc:::obj322:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.132+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.132+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:7ccaea2b:::obj486:head, version: 111'486, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ccaea2b:::obj486:head@111'486, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000040 MOSDPGPush(4.0 119/117 [PushOp(4:7ccaea2b:::obj486:head, version: 111'486, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ccaea2b:::obj486:head@111'486, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:7c406887:::obj566:head local-lis/les=117/118 n=291 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.132+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.132+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7c955dfc:::obj322:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.132+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7c406887:::obj566:head local-lis/les=117/118 n=291 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:7ccaea2b:::obj486:head, version: 111'486, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ccaea2b:::obj486:head@111'486, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.132+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7c406887:::obj566:head local-lis/les=117/118 n=291 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:7ccaea2b:::obj486:head@111'486, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.132+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7c406887:::obj566:head local-lis/les=117/118 n=291 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:7ccaea2b:::obj486:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.132+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7c406887:::obj566:head local-lis/les=117/118 n=291 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:7ccaea2b:::obj486:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.132+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7c406887:::obj566:head local-lis/les=117/118 n=291 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:7ccaea2b:::obj486:head 2026-02-11T23:31:28.132+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7c406887:::obj566:head local-lis/les=117/118 n=291 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:7ccaea2b:::obj486:head v 111'486 2026-02-11T23:31:28.132+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7c406887:::obj566:head local-lis/les=117/118 n=291 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.132+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.132+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:7ccaea2b:::obj486:head, version: 111'486, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ccaea2b:::obj486:head@111'486, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:28.132+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.132+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.132+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7c955dfc:::obj322:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.132+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:7c955dfc:::obj322:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 4.0 e 119/119 lb 4:7c955dfc:::obj322:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:7c406887:::obj566:head local-lis/les=117/118 n=291 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.132+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.132+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7c406887:::obj566:head local-lis/les=117/118 n=291 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:7c955dfc:::obj322:head) v3 2026-02-11T23:31:28.132+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7c406887:::obj566:head local-lis/les=117/118 n=291 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:7c955dfc:::obj322:head) 2026-02-11T23:31:28.132+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.132+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:7c955dfc:::obj322:head) v3 finish latency 0.000012 2026-02-11T23:31:28.132+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7c955dfc:::obj322:head local-lis/les=117/118 n=292 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.132+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.132+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.132+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7c955dfc:::obj322:head local-lis/les=117/118 n=292 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.132+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.132+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.132+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.132+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x640e480 con 0x507ac00 2026-02-11T23:31:28.132+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:7ccaea2b:::obj486:head)]) -- 0x644cd80 con 0x507ac00 2026-02-11T23:31:28.132+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.132+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 625 ==== MOSDPGPush(4.0 119/117 [PushOp(4:7e5f2af9:::obj239:head, version: 111'239, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7e5f2af9:::obj239:head@111'239, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559d520 con 0x507ac00 2026-02-11T23:31:28.132+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:7e5f2af9:::obj239:head, version: 111'239, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7e5f2af9:::obj239:head@111'239, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 119 2026-02-11T23:31:28.132+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7e5f2af9:::obj239:head, version: 111'239, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7e5f2af9:::obj239:head@111'239, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.132+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.132+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7e5f2af9:::obj239:head, version: 111'239, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7e5f2af9:::obj239:head@111'239, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.132+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 626 ==== pg_backfill(progress 4.0 e 119/119 lb 4:7ccaea2b:::obj486:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f13700 con 0x507ac00 2026-02-11T23:31:28.132+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.132+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:7ccaea2b:::obj486:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.132+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7e5f2af9:::obj239:head, version: 111'239, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7e5f2af9:::obj239:head@111'239, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.132+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7ccaea2b:::obj486:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.132+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.132+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7ccaea2b:::obj486:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.132+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:7e5f2af9:::obj239:head, version: 111'239, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7e5f2af9:::obj239:head@111'239, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(4.0 119/117 [PushOp(4:7e5f2af9:::obj239:head, version: 111'239, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7e5f2af9:::obj239:head@111'239, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:7c955dfc:::obj322:head local-lis/les=117/118 n=292 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.132+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.132+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7c955dfc:::obj322:head local-lis/les=117/118 n=292 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:7e5f2af9:::obj239:head, version: 111'239, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7e5f2af9:::obj239:head@111'239, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.132+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7c955dfc:::obj322:head local-lis/les=117/118 n=292 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:7e5f2af9:::obj239:head@111'239, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.132+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7c955dfc:::obj322:head local-lis/les=117/118 n=292 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:7e5f2af9:::obj239:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.132+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7c955dfc:::obj322:head local-lis/les=117/118 n=292 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:7e5f2af9:::obj239:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.132+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7c955dfc:::obj322:head local-lis/les=117/118 n=292 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:7e5f2af9:::obj239:head 2026-02-11T23:31:28.132+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7c955dfc:::obj322:head local-lis/les=117/118 n=292 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:7e5f2af9:::obj239:head v 111'239 2026-02-11T23:31:28.132+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7c955dfc:::obj322:head local-lis/les=117/118 n=292 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.132+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.132+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:7e5f2af9:::obj239:head, version: 111'239, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7e5f2af9:::obj239:head@111'239, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:28.132+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.132+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.132+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7ccaea2b:::obj486:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.132+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:7ccaea2b:::obj486:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 4.0 e 119/119 lb 4:7ccaea2b:::obj486:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:7c955dfc:::obj322:head local-lis/les=117/118 n=292 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.132+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.132+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7c955dfc:::obj322:head local-lis/les=117/118 n=292 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:7ccaea2b:::obj486:head) v3 2026-02-11T23:31:28.132+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7c955dfc:::obj322:head local-lis/les=117/118 n=292 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:7ccaea2b:::obj486:head) 2026-02-11T23:31:28.132+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.132+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:7ccaea2b:::obj486:head) v3 finish latency 0.000013 2026-02-11T23:31:28.132+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ccaea2b:::obj486:head local-lis/les=117/118 n=293 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.132+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.132+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.132+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ccaea2b:::obj486:head local-lis/les=117/118 n=293 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.132+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.132+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.132+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.132+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x640e600 con 0x507ac00 2026-02-11T23:31:28.132+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:7e5f2af9:::obj239:head)]) -- 0x63c9e00 con 0x507ac00 2026-02-11T23:31:28.132+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.133+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 627 ==== MOSDPGPush(4.0 119/117 [PushOp(4:7eacede5:::obj24:head, version: 111'24, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7eacede5:::obj24:head@111'24, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x55f1860 con 0x507ac00 2026-02-11T23:31:28.133+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:7eacede5:::obj24:head, version: 111'24, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7eacede5:::obj24:head@111'24, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000016 epoch 119 2026-02-11T23:31:28.133+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7eacede5:::obj24:head, version: 111'24, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7eacede5:::obj24:head@111'24, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.133+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.133+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 628 ==== pg_backfill(progress 4.0 e 119/119 lb 4:7e5f2af9:::obj239:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f10000 con 0x507ac00 2026-02-11T23:31:28.133+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:7e5f2af9:::obj239:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.133+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7eacede5:::obj24:head, version: 111'24, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7eacede5:::obj24:head@111'24, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.133+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7e5f2af9:::obj239:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.133+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.133+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7eacede5:::obj24:head, version: 111'24, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7eacede5:::obj24:head@111'24, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.133+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:7eacede5:::obj24:head, version: 111'24, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7eacede5:::obj24:head@111'24, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000043 MOSDPGPush(4.0 119/117 [PushOp(4:7eacede5:::obj24:head, version: 111'24, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7eacede5:::obj24:head@111'24, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:7ccaea2b:::obj486:head local-lis/les=117/118 n=293 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.133+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.133+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.133+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7e5f2af9:::obj239:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.133+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ccaea2b:::obj486:head local-lis/les=117/118 n=293 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:7eacede5:::obj24:head, version: 111'24, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7eacede5:::obj24:head@111'24, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.133+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ccaea2b:::obj486:head local-lis/les=117/118 n=293 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:7eacede5:::obj24:head@111'24, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.133+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ccaea2b:::obj486:head local-lis/les=117/118 n=293 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:7eacede5:::obj24:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.133+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ccaea2b:::obj486:head local-lis/les=117/118 n=293 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:7eacede5:::obj24:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.133+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ccaea2b:::obj486:head local-lis/les=117/118 n=293 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:7eacede5:::obj24:head 2026-02-11T23:31:28.133+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ccaea2b:::obj486:head local-lis/les=117/118 n=293 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:7eacede5:::obj24:head v 111'24 2026-02-11T23:31:28.133+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ccaea2b:::obj486:head local-lis/les=117/118 n=293 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.133+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.133+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:7eacede5:::obj24:head, version: 111'24, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7eacede5:::obj24:head@111'24, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:28.133+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.133+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.133+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7e5f2af9:::obj239:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.133+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:7e5f2af9:::obj239:head) v3 prio 15 cost 0 latency 0.000080 pg_backfill(progress 4.0 e 119/119 lb 4:7e5f2af9:::obj239:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:7ccaea2b:::obj486:head local-lis/les=117/118 n=293 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.133+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.133+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ccaea2b:::obj486:head local-lis/les=117/118 n=293 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:7e5f2af9:::obj239:head) v3 2026-02-11T23:31:28.133+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ccaea2b:::obj486:head local-lis/les=117/118 n=293 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:7e5f2af9:::obj239:head) 2026-02-11T23:31:28.133+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.133+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:7e5f2af9:::obj239:head) v3 finish latency 0.000024 2026-02-11T23:31:28.133+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7e5f2af9:::obj239:head local-lis/les=117/118 n=294 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.133+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.133+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.133+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7e5f2af9:::obj239:head local-lis/les=117/118 n=294 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.133+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.133+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.133+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.133+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x640e780 con 0x507ac00 2026-02-11T23:31:28.133+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:7eacede5:::obj24:head)]) -- 0x5d55800 con 0x507ac00 2026-02-11T23:31:28.133+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.134+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 629 ==== MOSDPGPush(4.0 119/117 [PushOp(4:7ec5cfba:::obj597:head, version: 111'597, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ec5cfba:::obj597:head@111'597, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a29c0 con 0x507ac00 2026-02-11T23:31:28.134+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:7ec5cfba:::obj597:head, version: 111'597, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ec5cfba:::obj597:head@111'597, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 119 2026-02-11T23:31:28.134+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7ec5cfba:::obj597:head, version: 111'597, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ec5cfba:::obj597:head@111'597, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.134+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.134+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 630 ==== pg_backfill(progress 4.0 e 119/119 lb 4:7eacede5:::obj24:head) ==== 1061+0+0 (unknown 0 0 0) 0x5f10580 con 0x507ac00 2026-02-11T23:31:28.134+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:7eacede5:::obj24:head) v3 prio 15 type 95 cost 0 latency 0.000010 epoch 119 2026-02-11T23:31:28.134+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7eacede5:::obj24:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.134+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7ec5cfba:::obj597:head, version: 111'597, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ec5cfba:::obj597:head@111'597, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.134+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.134+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7ec5cfba:::obj597:head, version: 111'597, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ec5cfba:::obj597:head@111'597, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.134+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:7ec5cfba:::obj597:head, version: 111'597, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ec5cfba:::obj597:head@111'597, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000047 MOSDPGPush(4.0 119/117 [PushOp(4:7ec5cfba:::obj597:head, version: 111'597, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ec5cfba:::obj597:head@111'597, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:7e5f2af9:::obj239:head local-lis/les=117/118 n=294 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.134+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.134+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.134+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7eacede5:::obj24:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.134+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7e5f2af9:::obj239:head local-lis/les=117/118 n=294 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:7ec5cfba:::obj597:head, version: 111'597, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ec5cfba:::obj597:head@111'597, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.134+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7e5f2af9:::obj239:head local-lis/les=117/118 n=294 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:7ec5cfba:::obj597:head@111'597, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.134+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7e5f2af9:::obj239:head local-lis/les=117/118 n=294 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:7ec5cfba:::obj597:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.134+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7e5f2af9:::obj239:head local-lis/les=117/118 n=294 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:7ec5cfba:::obj597:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.134+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7e5f2af9:::obj239:head local-lis/les=117/118 n=294 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:7ec5cfba:::obj597:head 2026-02-11T23:31:28.134+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7e5f2af9:::obj239:head local-lis/les=117/118 n=294 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:7ec5cfba:::obj597:head v 111'597 2026-02-11T23:31:28.134+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7e5f2af9:::obj239:head local-lis/les=117/118 n=294 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.134+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.134+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:7ec5cfba:::obj597:head, version: 111'597, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ec5cfba:::obj597:head@111'597, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:28.134+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.134+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.134+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7eacede5:::obj24:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.134+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:7eacede5:::obj24:head) v3 prio 15 cost 0 latency 0.000093 pg_backfill(progress 4.0 e 119/119 lb 4:7eacede5:::obj24:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:7e5f2af9:::obj239:head local-lis/les=117/118 n=294 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.134+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.134+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7e5f2af9:::obj239:head local-lis/les=117/118 n=294 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:7eacede5:::obj24:head) v3 2026-02-11T23:31:28.134+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7e5f2af9:::obj239:head local-lis/les=117/118 n=294 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:7eacede5:::obj24:head) 2026-02-11T23:31:28.134+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.134+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:7eacede5:::obj24:head) v3 finish latency 0.000018 2026-02-11T23:31:28.134+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7eacede5:::obj24:head local-lis/les=117/118 n=295 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.134+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.134+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.134+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7eacede5:::obj24:head local-lis/les=117/118 n=295 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.134+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.134+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.134+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.134+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x640e900 con 0x507ac00 2026-02-11T23:31:28.134+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:7ec5cfba:::obj597:head)]) -- 0x646d500 con 0x507ac00 2026-02-11T23:31:28.134+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.134+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 631 ==== MOSDPGPush(4.0 119/117 [PushOp(4:7ee98ed8:::obj216:head, version: 111'216, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ee98ed8:::obj216:head@111'216, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e26b60 con 0x507ac00 2026-02-11T23:31:28.134+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:7ee98ed8:::obj216:head, version: 111'216, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ee98ed8:::obj216:head@111'216, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000015 epoch 119 2026-02-11T23:31:28.134+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7ee98ed8:::obj216:head, version: 111'216, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ee98ed8:::obj216:head@111'216, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.134+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.134+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 632 ==== pg_backfill(progress 4.0 e 119/119 lb 4:7ec5cfba:::obj597:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f10b00 con 0x507ac00 2026-02-11T23:31:28.134+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:7ec5cfba:::obj597:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.134+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7ec5cfba:::obj597:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.134+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7ee98ed8:::obj216:head, version: 111'216, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ee98ed8:::obj216:head@111'216, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.134+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.134+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7ee98ed8:::obj216:head, version: 111'216, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ee98ed8:::obj216:head@111'216, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.134+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:7ee98ed8:::obj216:head, version: 111'216, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ee98ed8:::obj216:head@111'216, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000041 MOSDPGPush(4.0 119/117 [PushOp(4:7ee98ed8:::obj216:head, version: 111'216, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ee98ed8:::obj216:head@111'216, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:7eacede5:::obj24:head local-lis/les=117/118 n=295 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.134+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.134+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.134+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7ec5cfba:::obj597:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.134+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7eacede5:::obj24:head local-lis/les=117/118 n=295 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:7ee98ed8:::obj216:head, version: 111'216, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ee98ed8:::obj216:head@111'216, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.134+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7eacede5:::obj24:head local-lis/les=117/118 n=295 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:7ee98ed8:::obj216:head@111'216, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.134+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7eacede5:::obj24:head local-lis/les=117/118 n=295 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:7ee98ed8:::obj216:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.134+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7eacede5:::obj24:head local-lis/les=117/118 n=295 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:7ee98ed8:::obj216:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.134+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7eacede5:::obj24:head local-lis/les=117/118 n=295 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:7ee98ed8:::obj216:head 2026-02-11T23:31:28.134+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7eacede5:::obj24:head local-lis/les=117/118 n=295 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:7ee98ed8:::obj216:head v 111'216 2026-02-11T23:31:28.134+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7eacede5:::obj24:head local-lis/les=117/118 n=295 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.134+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.134+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:7ee98ed8:::obj216:head, version: 111'216, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ee98ed8:::obj216:head@111'216, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000076 2026-02-11T23:31:28.134+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7eacede5:::obj24:head local-lis/les=117/118 n=295 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.134+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.134+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.134+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7ec5cfba:::obj597:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.134+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:7ec5cfba:::obj597:head) v3 prio 15 cost 0 latency 0.000099 pg_backfill(progress 4.0 e 119/119 lb 4:7ec5cfba:::obj597:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:7eacede5:::obj24:head local-lis/les=117/118 n=295 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.134+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.134+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7eacede5:::obj24:head local-lis/les=117/118 n=295 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:7ec5cfba:::obj597:head) v3 2026-02-11T23:31:28.134+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7eacede5:::obj24:head local-lis/les=117/118 n=295 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:7ec5cfba:::obj597:head) 2026-02-11T23:31:28.134+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.134+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:7ec5cfba:::obj597:head) v3 finish latency 0.000019 2026-02-11T23:31:28.134+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.135+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.135+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ec5cfba:::obj597:head local-lis/les=117/118 n=296 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.135+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.135+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.135+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.135+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x640ed80 con 0x507ac00 2026-02-11T23:31:28.135+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:7ee98ed8:::obj216:head)]) -- 0x640ea80 con 0x507ac00 2026-02-11T23:31:28.135+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.135+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 633 ==== MOSDPGPush(4.0 119/117 [PushOp(4:7ee9cafa:::obj105:head, version: 111'105, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ee9cafa:::obj105:head@111'105, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3815380 con 0x507ac00 2026-02-11T23:31:28.135+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:7ee9cafa:::obj105:head, version: 111'105, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ee9cafa:::obj105:head@111'105, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 119 2026-02-11T23:31:28.135+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7ee9cafa:::obj105:head, version: 111'105, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ee9cafa:::obj105:head@111'105, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.135+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.135+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 634 ==== pg_backfill(progress 4.0 e 119/119 lb 4:7ee98ed8:::obj216:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f13180 con 0x507ac00 2026-02-11T23:31:28.135+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:7ee98ed8:::obj216:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.135+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7ee98ed8:::obj216:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.135+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7ee9cafa:::obj105:head, version: 111'105, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ee9cafa:::obj105:head@111'105, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.135+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.135+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7ee9cafa:::obj105:head, version: 111'105, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ee9cafa:::obj105:head@111'105, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.135+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:7ee9cafa:::obj105:head, version: 111'105, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ee9cafa:::obj105:head@111'105, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000040 MOSDPGPush(4.0 119/117 [PushOp(4:7ee9cafa:::obj105:head, version: 111'105, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ee9cafa:::obj105:head@111'105, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:7ec5cfba:::obj597:head local-lis/les=117/118 n=296 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.135+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.135+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.135+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7ee98ed8:::obj216:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.135+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ec5cfba:::obj597:head local-lis/les=117/118 n=296 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:7ee9cafa:::obj105:head, version: 111'105, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ee9cafa:::obj105:head@111'105, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.135+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ec5cfba:::obj597:head local-lis/les=117/118 n=296 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:7ee9cafa:::obj105:head@111'105, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.135+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ec5cfba:::obj597:head local-lis/les=117/118 n=296 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:7ee9cafa:::obj105:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.135+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ec5cfba:::obj597:head local-lis/les=117/118 n=296 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:7ee9cafa:::obj105:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.135+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ec5cfba:::obj597:head local-lis/les=117/118 n=296 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:7ee9cafa:::obj105:head 2026-02-11T23:31:28.135+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ec5cfba:::obj597:head local-lis/les=117/118 n=296 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:7ee9cafa:::obj105:head v 111'105 2026-02-11T23:31:28.135+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ec5cfba:::obj597:head local-lis/les=117/118 n=296 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.135+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.135+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:7ee9cafa:::obj105:head, version: 111'105, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ee9cafa:::obj105:head@111'105, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000066 2026-02-11T23:31:28.135+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ec5cfba:::obj597:head local-lis/les=117/118 n=296 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.135+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.135+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.135+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7ee98ed8:::obj216:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.135+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:7ee98ed8:::obj216:head) v3 prio 15 cost 0 latency 0.000090 pg_backfill(progress 4.0 e 119/119 lb 4:7ee98ed8:::obj216:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:7ec5cfba:::obj597:head local-lis/les=117/118 n=296 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.135+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.135+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ec5cfba:::obj597:head local-lis/les=117/118 n=296 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:7ee98ed8:::obj216:head) v3 2026-02-11T23:31:28.135+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ec5cfba:::obj597:head local-lis/les=117/118 n=296 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:7ee98ed8:::obj216:head) 2026-02-11T23:31:28.135+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.135+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:7ee98ed8:::obj216:head) v3 finish latency 0.000017 2026-02-11T23:31:28.135+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.135+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.135+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ee98ed8:::obj216:head local-lis/les=117/118 n=297 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.135+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.135+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.135+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.135+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x640f200 con 0x507ac00 2026-02-11T23:31:28.135+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:7ee9cafa:::obj105:head)]) -- 0x640ef00 con 0x507ac00 2026-02-11T23:31:28.135+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.136+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 635 ==== MOSDPGPush(4.0 119/117 [PushOp(4:7f12fc77:::obj195:head, version: 111'195, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f12fc77:::obj195:head@111'195, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2680 con 0x507ac00 2026-02-11T23:31:28.136+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:7f12fc77:::obj195:head, version: 111'195, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f12fc77:::obj195:head@111'195, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 119 2026-02-11T23:31:28.136+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7f12fc77:::obj195:head, version: 111'195, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f12fc77:::obj195:head@111'195, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.136+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.136+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 636 ==== pg_backfill(progress 4.0 e 119/119 lb 4:7ee9cafa:::obj105:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f11080 con 0x507ac00 2026-02-11T23:31:28.136+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:7ee9cafa:::obj105:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.136+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7f12fc77:::obj195:head, version: 111'195, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f12fc77:::obj195:head@111'195, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.136+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7ee9cafa:::obj105:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.136+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.136+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7f12fc77:::obj195:head, version: 111'195, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f12fc77:::obj195:head@111'195, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.136+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:7f12fc77:::obj195:head, version: 111'195, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f12fc77:::obj195:head@111'195, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(4.0 119/117 [PushOp(4:7f12fc77:::obj195:head, version: 111'195, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f12fc77:::obj195:head@111'195, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:7ee98ed8:::obj216:head local-lis/les=117/118 n=297 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.136+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.136+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.136+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7ee9cafa:::obj105:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.136+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ee98ed8:::obj216:head local-lis/les=117/118 n=297 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:7f12fc77:::obj195:head, version: 111'195, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f12fc77:::obj195:head@111'195, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.136+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ee98ed8:::obj216:head local-lis/les=117/118 n=297 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:7f12fc77:::obj195:head@111'195, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.136+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ee98ed8:::obj216:head local-lis/les=117/118 n=297 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:7f12fc77:::obj195:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.136+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ee98ed8:::obj216:head local-lis/les=117/118 n=297 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:7f12fc77:::obj195:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.136+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ee98ed8:::obj216:head local-lis/les=117/118 n=297 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:7f12fc77:::obj195:head 2026-02-11T23:31:28.136+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ee98ed8:::obj216:head local-lis/les=117/118 n=297 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:7f12fc77:::obj195:head v 111'195 2026-02-11T23:31:28.136+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ee98ed8:::obj216:head local-lis/les=117/118 n=297 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.136+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.136+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:7f12fc77:::obj195:head, version: 111'195, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f12fc77:::obj195:head@111'195, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000062 2026-02-11T23:31:28.136+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ee98ed8:::obj216:head local-lis/les=117/118 n=297 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.136+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.136+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.136+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7ee9cafa:::obj105:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.136+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:7ee9cafa:::obj105:head) v3 prio 15 cost 0 latency 0.000086 pg_backfill(progress 4.0 e 119/119 lb 4:7ee9cafa:::obj105:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:7ee98ed8:::obj216:head local-lis/les=117/118 n=297 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.136+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.136+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ee98ed8:::obj216:head local-lis/les=117/118 n=297 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:7ee9cafa:::obj105:head) v3 2026-02-11T23:31:28.136+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ee98ed8:::obj216:head local-lis/les=117/118 n=297 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:7ee9cafa:::obj105:head) 2026-02-11T23:31:28.136+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.136+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:7ee9cafa:::obj105:head) v3 finish latency 0.000024 2026-02-11T23:31:28.136+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.136+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.136+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ee9cafa:::obj105:head local-lis/les=117/118 n=298 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.136+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.136+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.136+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.136+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x640f680 con 0x507ac00 2026-02-11T23:31:28.136+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:7f12fc77:::obj195:head)]) -- 0x640f380 con 0x507ac00 2026-02-11T23:31:28.136+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.136+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.136+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.136+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 637 ==== MOSDPGPush(4.0 119/117 [PushOp(4:7f1dde7e:::obj491:head, version: 111'491, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f1dde7e:::obj491:head@111'491, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56cab60 con 0x507ac00 2026-02-11T23:31:28.136+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:7f1dde7e:::obj491:head, version: 111'491, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f1dde7e:::obj491:head@111'491, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 119 2026-02-11T23:31:28.136+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7f1dde7e:::obj491:head, version: 111'491, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f1dde7e:::obj491:head@111'491, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.136+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.136+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 638 ==== pg_backfill(progress 4.0 e 119/119 lb 4:7f12fc77:::obj195:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f11600 con 0x507ac00 2026-02-11T23:31:28.136+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:7f12fc77:::obj195:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.136+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7f12fc77:::obj195:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.136+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7f1dde7e:::obj491:head, version: 111'491, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f1dde7e:::obj491:head@111'491, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.136+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.136+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7f1dde7e:::obj491:head, version: 111'491, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f1dde7e:::obj491:head@111'491, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.136+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:7f1dde7e:::obj491:head, version: 111'491, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f1dde7e:::obj491:head@111'491, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 119/117 [PushOp(4:7f1dde7e:::obj491:head, version: 111'491, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f1dde7e:::obj491:head@111'491, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:7ee9cafa:::obj105:head local-lis/les=117/118 n=298 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.136+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.136+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.136+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ee9cafa:::obj105:head local-lis/les=117/118 n=298 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:7f1dde7e:::obj491:head, version: 111'491, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f1dde7e:::obj491:head@111'491, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.136+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7f12fc77:::obj195:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.136+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ee9cafa:::obj105:head local-lis/les=117/118 n=298 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:7f1dde7e:::obj491:head@111'491, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.136+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ee9cafa:::obj105:head local-lis/les=117/118 n=298 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:7f1dde7e:::obj491:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.136+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ee9cafa:::obj105:head local-lis/les=117/118 n=298 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:7f1dde7e:::obj491:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.136+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ee9cafa:::obj105:head local-lis/les=117/118 n=298 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:7f1dde7e:::obj491:head 2026-02-11T23:31:28.136+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ee9cafa:::obj105:head local-lis/les=117/118 n=298 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:7f1dde7e:::obj491:head v 111'491 2026-02-11T23:31:28.136+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ee9cafa:::obj105:head local-lis/les=117/118 n=298 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.136+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.136+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:7f1dde7e:::obj491:head, version: 111'491, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f1dde7e:::obj491:head@111'491, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:28.136+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.136+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.136+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7f12fc77:::obj195:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.136+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:7f12fc77:::obj195:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 119/119 lb 4:7f12fc77:::obj195:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:7ee9cafa:::obj105:head local-lis/les=117/118 n=298 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.136+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.136+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ee9cafa:::obj105:head local-lis/les=117/118 n=298 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:7f12fc77:::obj195:head) v3 2026-02-11T23:31:28.136+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7ee9cafa:::obj105:head local-lis/les=117/118 n=298 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:7f12fc77:::obj195:head) 2026-02-11T23:31:28.136+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.136+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:7f12fc77:::obj195:head) v3 finish latency 0.000018 2026-02-11T23:31:28.136+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7f12fc77:::obj195:head local-lis/les=117/118 n=299 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.136+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.137+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.137+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7f12fc77:::obj195:head local-lis/les=117/118 n=299 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.137+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.137+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.137+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.137+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x640f800 con 0x507ac00 2026-02-11T23:31:28.137+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:7f1dde7e:::obj491:head)]) -- 0x646d200 con 0x507ac00 2026-02-11T23:31:28.137+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.137+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 639 ==== MOSDPGPush(4.0 119/117 [PushOp(4:7f2a7765:::obj252:head, version: 111'252, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f2a7765:::obj252:head@111'252, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f0d00 con 0x507ac00 2026-02-11T23:31:28.137+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:7f2a7765:::obj252:head, version: 111'252, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f2a7765:::obj252:head@111'252, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.137+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7f2a7765:::obj252:head, version: 111'252, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f2a7765:::obj252:head@111'252, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.137+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.137+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7f2a7765:::obj252:head, version: 111'252, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f2a7765:::obj252:head@111'252, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.137+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 640 ==== pg_backfill(progress 4.0 e 119/119 lb 4:7f1dde7e:::obj491:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2f700 con 0x507ac00 2026-02-11T23:31:28.137+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:7f1dde7e:::obj491:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.137+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.137+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7f1dde7e:::obj491:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.137+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7f2a7765:::obj252:head, version: 111'252, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f2a7765:::obj252:head@111'252, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.137+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:7f2a7765:::obj252:head, version: 111'252, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f2a7765:::obj252:head@111'252, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 119/117 [PushOp(4:7f2a7765:::obj252:head, version: 111'252, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f2a7765:::obj252:head@111'252, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:7f12fc77:::obj195:head local-lis/les=117/118 n=299 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.137+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.137+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.137+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7f12fc77:::obj195:head local-lis/les=117/118 n=299 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:7f2a7765:::obj252:head, version: 111'252, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f2a7765:::obj252:head@111'252, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.137+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7f12fc77:::obj195:head local-lis/les=117/118 n=299 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:7f2a7765:::obj252:head@111'252, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.137+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.137+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7f12fc77:::obj195:head local-lis/les=117/118 n=299 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:7f2a7765:::obj252:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.137+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7f1dde7e:::obj491:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.137+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7f12fc77:::obj195:head local-lis/les=117/118 n=299 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:7f2a7765:::obj252:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.137+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7f12fc77:::obj195:head local-lis/les=117/118 n=299 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:7f2a7765:::obj252:head 2026-02-11T23:31:28.137+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7f12fc77:::obj195:head local-lis/les=117/118 n=299 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:7f2a7765:::obj252:head v 111'252 2026-02-11T23:31:28.137+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7f12fc77:::obj195:head local-lis/les=117/118 n=299 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.137+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.137+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:7f2a7765:::obj252:head, version: 111'252, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f2a7765:::obj252:head@111'252, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:28.137+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.137+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.137+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7f1dde7e:::obj491:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.137+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:7f1dde7e:::obj491:head) v3 prio 15 cost 0 latency 0.000060 pg_backfill(progress 4.0 e 119/119 lb 4:7f1dde7e:::obj491:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:7f12fc77:::obj195:head local-lis/les=117/118 n=299 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.137+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.137+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7f12fc77:::obj195:head local-lis/les=117/118 n=299 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:7f1dde7e:::obj491:head) v3 2026-02-11T23:31:28.137+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7f12fc77:::obj195:head local-lis/les=117/118 n=299 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:7f1dde7e:::obj491:head) 2026-02-11T23:31:28.137+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.137+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:7f1dde7e:::obj491:head) v3 finish latency 0.000019 2026-02-11T23:31:28.137+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7f1dde7e:::obj491:head local-lis/les=117/118 n=300 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.137+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.137+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.137+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7f1dde7e:::obj491:head local-lis/les=117/118 n=300 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.137+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.137+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.137+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.137+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x640f980 con 0x507ac00 2026-02-11T23:31:28.137+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:7f2a7765:::obj252:head)]) -- 0x646cf00 con 0x507ac00 2026-02-11T23:31:28.137+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.138+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 641 ==== MOSDPGPush(4.0 119/117 [PushOp(4:7fffd9c5:::obj553:head, version: 111'553, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7fffd9c5:::obj553:head@111'553, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5647380 con 0x507ac00 2026-02-11T23:31:28.138+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:7fffd9c5:::obj553:head, version: 111'553, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7fffd9c5:::obj553:head@111'553, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 119 2026-02-11T23:31:28.138+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7fffd9c5:::obj553:head, version: 111'553, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7fffd9c5:::obj553:head@111'553, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.138+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.138+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 642 ==== pg_backfill(progress 4.0 e 119/119 lb 4:7f2a7765:::obj252:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2d080 con 0x507ac00 2026-02-11T23:31:28.138+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7fffd9c5:::obj553:head, version: 111'553, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7fffd9c5:::obj553:head@111'553, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.138+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:7f2a7765:::obj252:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.138+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.138+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7f2a7765:::obj252:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.138+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:7fffd9c5:::obj553:head, version: 111'553, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7fffd9c5:::obj553:head@111'553, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.138+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.138+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:7fffd9c5:::obj553:head, version: 111'553, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7fffd9c5:::obj553:head@111'553, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 119/117 [PushOp(4:7fffd9c5:::obj553:head, version: 111'553, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7fffd9c5:::obj553:head@111'553, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:7f1dde7e:::obj491:head local-lis/les=117/118 n=300 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.138+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.138+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7f1dde7e:::obj491:head local-lis/les=117/118 n=300 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:7fffd9c5:::obj553:head, version: 111'553, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7fffd9c5:::obj553:head@111'553, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.138+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.138+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7f1dde7e:::obj491:head local-lis/les=117/118 n=300 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:7fffd9c5:::obj553:head@111'553, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.138+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7f2a7765:::obj252:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.138+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7f1dde7e:::obj491:head local-lis/les=117/118 n=300 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:7fffd9c5:::obj553:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.138+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7f1dde7e:::obj491:head local-lis/les=117/118 n=300 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:7fffd9c5:::obj553:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.138+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7f1dde7e:::obj491:head local-lis/les=117/118 n=300 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:7fffd9c5:::obj553:head 2026-02-11T23:31:28.138+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7f1dde7e:::obj491:head local-lis/les=117/118 n=300 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:7fffd9c5:::obj553:head v 111'553 2026-02-11T23:31:28.138+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7f1dde7e:::obj491:head local-lis/les=117/118 n=300 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.138+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.138+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:7fffd9c5:::obj553:head, version: 111'553, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7fffd9c5:::obj553:head@111'553, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:28.138+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.138+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.138+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7f2a7765:::obj252:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.138+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:7f2a7765:::obj252:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 119/119 lb 4:7f2a7765:::obj252:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:7f1dde7e:::obj491:head local-lis/les=117/118 n=300 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.138+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.138+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7f1dde7e:::obj491:head local-lis/les=117/118 n=300 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:7f2a7765:::obj252:head) v3 2026-02-11T23:31:28.138+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7f1dde7e:::obj491:head local-lis/les=117/118 n=300 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:7f2a7765:::obj252:head) 2026-02-11T23:31:28.138+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.138+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:7f2a7765:::obj252:head) v3 finish latency 0.000023 2026-02-11T23:31:28.138+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7f2a7765:::obj252:head local-lis/les=117/118 n=301 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.138+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.138+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.138+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7f2a7765:::obj252:head local-lis/les=117/118 n=301 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.138+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.138+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.138+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.138+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x640fb00 con 0x507ac00 2026-02-11T23:31:28.138+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:7fffd9c5:::obj553:head)]) -- 0x646cd80 con 0x507ac00 2026-02-11T23:31:28.138+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.138+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 643 ==== MOSDPGPush(4.0 119/117 [PushOp(4:81617860:::obj28:head, version: 111'28, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:81617860:::obj28:head@111'28, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5697ba0 con 0x507ac00 2026-02-11T23:31:28.138+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:81617860:::obj28:head, version: 111'28, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:81617860:::obj28:head@111'28, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 119 2026-02-11T23:31:28.138+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:81617860:::obj28:head, version: 111'28, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:81617860:::obj28:head@111'28, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.138+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.138+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 644 ==== pg_backfill(progress 4.0 e 119/119 lb 4:7fffd9c5:::obj553:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e680 con 0x507ac00 2026-02-11T23:31:28.138+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:7fffd9c5:::obj553:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.138+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:81617860:::obj28:head, version: 111'28, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:81617860:::obj28:head@111'28, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.138+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7fffd9c5:::obj553:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.138+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.138+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:81617860:::obj28:head, version: 111'28, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:81617860:::obj28:head@111'28, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.138+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:81617860:::obj28:head, version: 111'28, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:81617860:::obj28:head@111'28, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 119/117 [PushOp(4:81617860:::obj28:head, version: 111'28, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:81617860:::obj28:head@111'28, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:7f2a7765:::obj252:head local-lis/les=117/118 n=301 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.138+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.138+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.138+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7fffd9c5:::obj553:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.138+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7f2a7765:::obj252:head local-lis/les=117/118 n=301 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:81617860:::obj28:head, version: 111'28, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:81617860:::obj28:head@111'28, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.138+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7f2a7765:::obj252:head local-lis/les=117/118 n=301 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:81617860:::obj28:head@111'28, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.138+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7f2a7765:::obj252:head local-lis/les=117/118 n=301 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:81617860:::obj28:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.138+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7f2a7765:::obj252:head local-lis/les=117/118 n=301 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:81617860:::obj28:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.138+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7f2a7765:::obj252:head local-lis/les=117/118 n=301 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:81617860:::obj28:head 2026-02-11T23:31:28.138+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7f2a7765:::obj252:head local-lis/les=117/118 n=301 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:81617860:::obj28:head v 111'28 2026-02-11T23:31:28.138+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7f2a7765:::obj252:head local-lis/les=117/118 n=301 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.138+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.138+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:81617860:::obj28:head, version: 111'28, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:81617860:::obj28:head@111'28, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:28.138+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.138+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.138+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:7fffd9c5:::obj553:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.138+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:7fffd9c5:::obj553:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 4.0 e 119/119 lb 4:7fffd9c5:::obj553:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:7f2a7765:::obj252:head local-lis/les=117/118 n=301 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.138+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.138+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7f2a7765:::obj252:head local-lis/les=117/118 n=301 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:7fffd9c5:::obj553:head) v3 2026-02-11T23:31:28.138+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7f2a7765:::obj252:head local-lis/les=117/118 n=301 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:7fffd9c5:::obj553:head) 2026-02-11T23:31:28.138+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.138+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:7fffd9c5:::obj553:head) v3 finish latency 0.000018 2026-02-11T23:31:28.138+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7fffd9c5:::obj553:head local-lis/les=117/118 n=302 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.138+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.139+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.139+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7fffd9c5:::obj553:head local-lis/les=117/118 n=302 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.139+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.139+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.139+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.139+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x640fc80 con 0x507ac00 2026-02-11T23:31:28.139+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:81617860:::obj28:head)]) -- 0x646ca80 con 0x507ac00 2026-02-11T23:31:28.139+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.139+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 645 ==== MOSDPGPush(4.0 119/117 [PushOp(4:81fa42e1:::obj447:head, version: 111'447, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:81fa42e1:::obj447:head@111'447, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5617380 con 0x507ac00 2026-02-11T23:31:28.139+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:81fa42e1:::obj447:head, version: 111'447, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:81fa42e1:::obj447:head@111'447, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 119 2026-02-11T23:31:28.139+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:81fa42e1:::obj447:head, version: 111'447, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:81fa42e1:::obj447:head@111'447, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.139+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.139+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 646 ==== pg_backfill(progress 4.0 e 119/119 lb 4:81617860:::obj28:head) ==== 1061+0+0 (unknown 0 0 0) 0x6176580 con 0x507ac00 2026-02-11T23:31:28.139+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:81617860:::obj28:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.139+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:81fa42e1:::obj447:head, version: 111'447, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:81fa42e1:::obj447:head@111'447, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.139+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:81617860:::obj28:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.139+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.139+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:81fa42e1:::obj447:head, version: 111'447, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:81fa42e1:::obj447:head@111'447, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.139+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:81fa42e1:::obj447:head, version: 111'447, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:81fa42e1:::obj447:head@111'447, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 119/117 [PushOp(4:81fa42e1:::obj447:head, version: 111'447, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:81fa42e1:::obj447:head@111'447, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:7fffd9c5:::obj553:head local-lis/les=117/118 n=302 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.139+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.139+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.139+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:81617860:::obj28:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.139+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7fffd9c5:::obj553:head local-lis/les=117/118 n=302 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:81fa42e1:::obj447:head, version: 111'447, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:81fa42e1:::obj447:head@111'447, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.139+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7fffd9c5:::obj553:head local-lis/les=117/118 n=302 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:81fa42e1:::obj447:head@111'447, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.139+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7fffd9c5:::obj553:head local-lis/les=117/118 n=302 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:81fa42e1:::obj447:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.139+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7fffd9c5:::obj553:head local-lis/les=117/118 n=302 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:81fa42e1:::obj447:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.139+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7fffd9c5:::obj553:head local-lis/les=117/118 n=302 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:81fa42e1:::obj447:head 2026-02-11T23:31:28.139+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7fffd9c5:::obj553:head local-lis/les=117/118 n=302 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:81fa42e1:::obj447:head v 111'447 2026-02-11T23:31:28.139+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7fffd9c5:::obj553:head local-lis/les=117/118 n=302 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.139+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.139+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:81fa42e1:::obj447:head, version: 111'447, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:81fa42e1:::obj447:head@111'447, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:28.139+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.139+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.139+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:81617860:::obj28:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.139+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:81617860:::obj28:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 4.0 e 119/119 lb 4:81617860:::obj28:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:7fffd9c5:::obj553:head local-lis/les=117/118 n=302 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.139+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.139+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7fffd9c5:::obj553:head local-lis/les=117/118 n=302 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:81617860:::obj28:head) v3 2026-02-11T23:31:28.139+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:7fffd9c5:::obj553:head local-lis/les=117/118 n=302 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:81617860:::obj28:head) 2026-02-11T23:31:28.139+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.139+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:81617860:::obj28:head) v3 finish latency 0.000017 2026-02-11T23:31:28.139+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:81617860:::obj28:head local-lis/les=117/118 n=303 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.139+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.139+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.139+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:81617860:::obj28:head local-lis/les=117/118 n=303 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.139+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.139+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.139+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.139+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x640fe00 con 0x507ac00 2026-02-11T23:31:28.139+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:81fa42e1:::obj447:head)]) -- 0x5d54900 con 0x507ac00 2026-02-11T23:31:28.139+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.140+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 647 ==== MOSDPGPush(4.0 119/117 [PushOp(4:8205b5d8:::obj400:head, version: 111'400, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8205b5d8:::obj400:head@111'400, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f0b60 con 0x507ac00 2026-02-11T23:31:28.140+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:8205b5d8:::obj400:head, version: 111'400, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8205b5d8:::obj400:head@111'400, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 119 2026-02-11T23:31:28.140+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8205b5d8:::obj400:head, version: 111'400, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8205b5d8:::obj400:head@111'400, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.140+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.140+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 648 ==== pg_backfill(progress 4.0 e 119/119 lb 4:81fa42e1:::obj447:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e100 con 0x507ac00 2026-02-11T23:31:28.140+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8205b5d8:::obj400:head, version: 111'400, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8205b5d8:::obj400:head@111'400, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.140+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:81fa42e1:::obj447:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.140+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.140+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8205b5d8:::obj400:head, version: 111'400, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8205b5d8:::obj400:head@111'400, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.140+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:81fa42e1:::obj447:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.140+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.140+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8205b5d8:::obj400:head, version: 111'400, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8205b5d8:::obj400:head@111'400, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000040 MOSDPGPush(4.0 119/117 [PushOp(4:8205b5d8:::obj400:head, version: 111'400, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8205b5d8:::obj400:head@111'400, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:81617860:::obj28:head local-lis/les=117/118 n=303 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.140+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.140+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:81fa42e1:::obj447:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.140+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:81617860:::obj28:head local-lis/les=117/118 n=303 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:8205b5d8:::obj400:head, version: 111'400, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8205b5d8:::obj400:head@111'400, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.140+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:81617860:::obj28:head local-lis/les=117/118 n=303 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8205b5d8:::obj400:head@111'400, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.140+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:81617860:::obj28:head local-lis/les=117/118 n=303 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:8205b5d8:::obj400:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.140+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:81617860:::obj28:head local-lis/les=117/118 n=303 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:8205b5d8:::obj400:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.140+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:81617860:::obj28:head local-lis/les=117/118 n=303 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:8205b5d8:::obj400:head 2026-02-11T23:31:28.140+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:81617860:::obj28:head local-lis/les=117/118 n=303 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:8205b5d8:::obj400:head v 111'400 2026-02-11T23:31:28.140+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:81617860:::obj28:head local-lis/les=117/118 n=303 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.140+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.140+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8205b5d8:::obj400:head, version: 111'400, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8205b5d8:::obj400:head@111'400, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000065 2026-02-11T23:31:28.140+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:81617860:::obj28:head local-lis/les=117/118 n=303 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.140+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.140+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.140+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:81fa42e1:::obj447:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.140+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:81fa42e1:::obj447:head) v3 prio 15 cost 0 latency 0.000087 pg_backfill(progress 4.0 e 119/119 lb 4:81fa42e1:::obj447:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:81617860:::obj28:head local-lis/les=117/118 n=303 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.140+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.140+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:81617860:::obj28:head local-lis/les=117/118 n=303 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:81fa42e1:::obj447:head) v3 2026-02-11T23:31:28.140+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:81617860:::obj28:head local-lis/les=117/118 n=303 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:81fa42e1:::obj447:head) 2026-02-11T23:31:28.140+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.140+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:81fa42e1:::obj447:head) v3 finish latency 0.000019 2026-02-11T23:31:28.140+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.140+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.140+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:81fa42e1:::obj447:head local-lis/les=117/118 n=304 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.140+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.140+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.140+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.140+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c8c00 con 0x507ac00 2026-02-11T23:31:28.140+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:8205b5d8:::obj400:head)]) -- 0x5d55e00 con 0x507ac00 2026-02-11T23:31:28.140+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.140+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 649 ==== MOSDPGPush(4.0 119/117 [PushOp(4:8250b831:::obj124:head, version: 111'124, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8250b831:::obj124:head@111'124, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f1520 con 0x507ac00 2026-02-11T23:31:28.140+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:8250b831:::obj124:head, version: 111'124, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8250b831:::obj124:head@111'124, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000015 epoch 119 2026-02-11T23:31:28.140+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8250b831:::obj124:head, version: 111'124, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8250b831:::obj124:head@111'124, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.140+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.140+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 650 ==== pg_backfill(progress 4.0 e 119/119 lb 4:8205b5d8:::obj400:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2c000 con 0x507ac00 2026-02-11T23:31:28.140+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8250b831:::obj124:head, version: 111'124, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8250b831:::obj124:head@111'124, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.140+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:8205b5d8:::obj400:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.140+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.140+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8205b5d8:::obj400:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.140+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8250b831:::obj124:head, version: 111'124, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8250b831:::obj124:head@111'124, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.140+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8250b831:::obj124:head, version: 111'124, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8250b831:::obj124:head@111'124, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000042 MOSDPGPush(4.0 119/117 [PushOp(4:8250b831:::obj124:head, version: 111'124, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8250b831:::obj124:head@111'124, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:81fa42e1:::obj447:head local-lis/les=117/118 n=304 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.140+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.140+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.140+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8205b5d8:::obj400:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.140+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:81fa42e1:::obj447:head local-lis/les=117/118 n=304 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:8250b831:::obj124:head, version: 111'124, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8250b831:::obj124:head@111'124, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.140+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:81fa42e1:::obj447:head local-lis/les=117/118 n=304 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8250b831:::obj124:head@111'124, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.140+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:81fa42e1:::obj447:head local-lis/les=117/118 n=304 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:8250b831:::obj124:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.140+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:81fa42e1:::obj447:head local-lis/les=117/118 n=304 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:8250b831:::obj124:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.140+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:81fa42e1:::obj447:head local-lis/les=117/118 n=304 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:8250b831:::obj124:head 2026-02-11T23:31:28.140+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:81fa42e1:::obj447:head local-lis/les=117/118 n=304 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:8250b831:::obj124:head v 111'124 2026-02-11T23:31:28.140+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:81fa42e1:::obj447:head local-lis/les=117/118 n=304 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.140+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.140+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8250b831:::obj124:head, version: 111'124, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8250b831:::obj124:head@111'124, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:28.140+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:81fa42e1:::obj447:head local-lis/les=117/118 n=304 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.140+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.140+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.140+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8205b5d8:::obj400:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.140+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8205b5d8:::obj400:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 4.0 e 119/119 lb 4:8205b5d8:::obj400:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:81fa42e1:::obj447:head local-lis/les=117/118 n=304 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.140+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.140+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:81fa42e1:::obj447:head local-lis/les=117/118 n=304 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:8205b5d8:::obj400:head) v3 2026-02-11T23:31:28.140+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:81fa42e1:::obj447:head local-lis/les=117/118 n=304 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:8205b5d8:::obj400:head) 2026-02-11T23:31:28.140+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.140+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8205b5d8:::obj400:head) v3 finish latency 0.000024 2026-02-11T23:31:28.140+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.140+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.140+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8205b5d8:::obj400:head local-lis/les=117/118 n=305 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.140+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.140+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.140+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.140+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x640e000 con 0x507ac00 2026-02-11T23:31:28.140+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:8250b831:::obj124:head)]) -- 0x640e180 con 0x507ac00 2026-02-11T23:31:28.140+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.141+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.141+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.141+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 651 ==== MOSDPGPush(4.0 119/117 [PushOp(4:8260b93f:::obj338:head, version: 111'338, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8260b93f:::obj338:head@111'338, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f0d00 con 0x507ac00 2026-02-11T23:31:28.141+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:8260b93f:::obj338:head, version: 111'338, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8260b93f:::obj338:head@111'338, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000015 epoch 119 2026-02-11T23:31:28.141+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8260b93f:::obj338:head, version: 111'338, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8260b93f:::obj338:head@111'338, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.141+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.141+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 652 ==== pg_backfill(progress 4.0 e 119/119 lb 4:8250b831:::obj124:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2cb00 con 0x507ac00 2026-02-11T23:31:28.141+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:8250b831:::obj124:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.141+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8260b93f:::obj338:head, version: 111'338, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8260b93f:::obj338:head@111'338, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.141+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8250b831:::obj124:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.141+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.141+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8260b93f:::obj338:head, version: 111'338, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8260b93f:::obj338:head@111'338, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.141+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8260b93f:::obj338:head, version: 111'338, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8260b93f:::obj338:head@111'338, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000045 MOSDPGPush(4.0 119/117 [PushOp(4:8260b93f:::obj338:head, version: 111'338, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8260b93f:::obj338:head@111'338, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8205b5d8:::obj400:head local-lis/les=117/118 n=305 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.141+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.141+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.141+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8250b831:::obj124:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.141+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8205b5d8:::obj400:head local-lis/les=117/118 n=305 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:8260b93f:::obj338:head, version: 111'338, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8260b93f:::obj338:head@111'338, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.141+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8205b5d8:::obj400:head local-lis/les=117/118 n=305 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8260b93f:::obj338:head@111'338, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.141+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8205b5d8:::obj400:head local-lis/les=117/118 n=305 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:8260b93f:::obj338:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.141+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8205b5d8:::obj400:head local-lis/les=117/118 n=305 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:8260b93f:::obj338:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.141+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8205b5d8:::obj400:head local-lis/les=117/118 n=305 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:8260b93f:::obj338:head 2026-02-11T23:31:28.141+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8205b5d8:::obj400:head local-lis/les=117/118 n=305 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:8260b93f:::obj338:head v 111'338 2026-02-11T23:31:28.141+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8205b5d8:::obj400:head local-lis/les=117/118 n=305 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.141+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.141+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8260b93f:::obj338:head, version: 111'338, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8260b93f:::obj338:head@111'338, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:28.141+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8205b5d8:::obj400:head local-lis/les=117/118 n=305 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.141+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.141+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.141+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8250b831:::obj124:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.141+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8250b831:::obj124:head) v3 prio 15 cost 0 latency 0.000086 pg_backfill(progress 4.0 e 119/119 lb 4:8250b831:::obj124:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8205b5d8:::obj400:head local-lis/les=117/118 n=305 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.141+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.141+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8205b5d8:::obj400:head local-lis/les=117/118 n=305 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:8250b831:::obj124:head) v3 2026-02-11T23:31:28.141+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8205b5d8:::obj400:head local-lis/les=117/118 n=305 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:8250b831:::obj124:head) 2026-02-11T23:31:28.141+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.141+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8250b831:::obj124:head) v3 finish latency 0.000013 2026-02-11T23:31:28.141+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.141+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.141+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8250b831:::obj124:head local-lis/les=117/118 n=306 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.141+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.141+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.141+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.141+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7b080 con 0x507ac00 2026-02-11T23:31:28.141+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:8260b93f:::obj338:head)]) -- 0x644d800 con 0x507ac00 2026-02-11T23:31:28.141+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.142+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 653 ==== MOSDPGPush(4.0 119/117 [PushOp(4:8280535c:::obj236:head, version: 111'236, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8280535c:::obj236:head@111'236, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56cab60 con 0x507ac00 2026-02-11T23:31:28.142+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:8280535c:::obj236:head, version: 111'236, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8280535c:::obj236:head@111'236, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000015 epoch 119 2026-02-11T23:31:28.142+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8280535c:::obj236:head, version: 111'236, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8280535c:::obj236:head@111'236, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.142+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.142+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 654 ==== pg_backfill(progress 4.0 e 119/119 lb 4:8260b93f:::obj338:head) ==== 1062+0+0 (unknown 0 0 0) 0x6179700 con 0x507ac00 2026-02-11T23:31:28.142+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8280535c:::obj236:head, version: 111'236, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8280535c:::obj236:head@111'236, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.142+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:8260b93f:::obj338:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 119 2026-02-11T23:31:28.142+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.142+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8260b93f:::obj338:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.142+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8280535c:::obj236:head, version: 111'236, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8280535c:::obj236:head@111'236, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.142+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.142+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8280535c:::obj236:head, version: 111'236, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8280535c:::obj236:head@111'236, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000044 MOSDPGPush(4.0 119/117 [PushOp(4:8280535c:::obj236:head, version: 111'236, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8280535c:::obj236:head@111'236, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8250b831:::obj124:head local-lis/les=117/118 n=306 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.142+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.142+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.142+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8250b831:::obj124:head local-lis/les=117/118 n=306 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:8280535c:::obj236:head, version: 111'236, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8280535c:::obj236:head@111'236, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.142+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8260b93f:::obj338:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.142+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8250b831:::obj124:head local-lis/les=117/118 n=306 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8280535c:::obj236:head@111'236, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.142+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8250b831:::obj124:head local-lis/les=117/118 n=306 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:8280535c:::obj236:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.142+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8250b831:::obj124:head local-lis/les=117/118 n=306 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:8280535c:::obj236:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.142+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8250b831:::obj124:head local-lis/les=117/118 n=306 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:8280535c:::obj236:head 2026-02-11T23:31:28.142+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8250b831:::obj124:head local-lis/les=117/118 n=306 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:8280535c:::obj236:head v 111'236 2026-02-11T23:31:28.142+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8250b831:::obj124:head local-lis/les=117/118 n=306 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.142+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.142+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8280535c:::obj236:head, version: 111'236, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8280535c:::obj236:head@111'236, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:28.142+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8250b831:::obj124:head local-lis/les=117/118 n=306 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.142+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.142+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.142+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8260b93f:::obj338:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.142+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8260b93f:::obj338:head) v3 prio 15 cost 0 latency 0.000082 pg_backfill(progress 4.0 e 119/119 lb 4:8260b93f:::obj338:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8250b831:::obj124:head local-lis/les=117/118 n=306 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.142+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.142+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8250b831:::obj124:head local-lis/les=117/118 n=306 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:8260b93f:::obj338:head) v3 2026-02-11T23:31:28.142+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8250b831:::obj124:head local-lis/les=117/118 n=306 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:8260b93f:::obj338:head) 2026-02-11T23:31:28.142+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.142+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8260b93f:::obj338:head) v3 finish latency 0.000015 2026-02-11T23:31:28.142+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.142+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.142+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8260b93f:::obj338:head local-lis/les=117/118 n=307 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.142+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.142+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.142+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.142+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c9200 con 0x507ac00 2026-02-11T23:31:28.142+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:8280535c:::obj236:head)]) -- 0x4f7a780 con 0x507ac00 2026-02-11T23:31:28.142+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.142+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 655 ==== MOSDPGPush(4.0 119/117 [PushOp(4:8421ae49:::obj196:head, version: 111'196, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8421ae49:::obj196:head@111'196, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2680 con 0x507ac00 2026-02-11T23:31:28.142+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:8421ae49:::obj196:head, version: 111'196, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8421ae49:::obj196:head@111'196, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 119 2026-02-11T23:31:28.142+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8421ae49:::obj196:head, version: 111'196, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8421ae49:::obj196:head@111'196, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.142+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.142+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 656 ==== pg_backfill(progress 4.0 e 119/119 lb 4:8280535c:::obj236:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176000 con 0x507ac00 2026-02-11T23:31:28.142+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8421ae49:::obj196:head, version: 111'196, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8421ae49:::obj196:head@111'196, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.142+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:8280535c:::obj236:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.142+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8280535c:::obj236:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.142+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.142+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8421ae49:::obj196:head, version: 111'196, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8421ae49:::obj196:head@111'196, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.142+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8421ae49:::obj196:head, version: 111'196, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8421ae49:::obj196:head@111'196, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(4.0 119/117 [PushOp(4:8421ae49:::obj196:head, version: 111'196, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8421ae49:::obj196:head@111'196, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8260b93f:::obj338:head local-lis/les=117/118 n=307 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.142+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.142+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.142+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8280535c:::obj236:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.142+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8260b93f:::obj338:head local-lis/les=117/118 n=307 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:8421ae49:::obj196:head, version: 111'196, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8421ae49:::obj196:head@111'196, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.142+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8260b93f:::obj338:head local-lis/les=117/118 n=307 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8421ae49:::obj196:head@111'196, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.142+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8260b93f:::obj338:head local-lis/les=117/118 n=307 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:8421ae49:::obj196:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.142+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8260b93f:::obj338:head local-lis/les=117/118 n=307 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:8421ae49:::obj196:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.142+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8260b93f:::obj338:head local-lis/les=117/118 n=307 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:8421ae49:::obj196:head 2026-02-11T23:31:28.142+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8260b93f:::obj338:head local-lis/les=117/118 n=307 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:8421ae49:::obj196:head v 111'196 2026-02-11T23:31:28.142+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8260b93f:::obj338:head local-lis/les=117/118 n=307 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.142+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.142+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8421ae49:::obj196:head, version: 111'196, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8421ae49:::obj196:head@111'196, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:28.142+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8260b93f:::obj338:head local-lis/les=117/118 n=307 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.142+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.142+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8280535c:::obj236:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.142+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.142+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8280535c:::obj236:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 4.0 e 119/119 lb 4:8280535c:::obj236:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8260b93f:::obj338:head local-lis/les=117/118 n=307 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.142+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.142+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8260b93f:::obj338:head local-lis/les=117/118 n=307 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:8280535c:::obj236:head) v3 2026-02-11T23:31:28.142+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8260b93f:::obj338:head local-lis/les=117/118 n=307 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:8280535c:::obj236:head) 2026-02-11T23:31:28.142+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.142+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8280535c:::obj236:head) v3 finish latency 0.000018 2026-02-11T23:31:28.142+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.142+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.142+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8280535c:::obj236:head local-lis/les=117/118 n=308 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.142+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.142+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.142+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.142+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63ff800 con 0x507ac00 2026-02-11T23:31:28.142+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:8421ae49:::obj196:head)]) -- 0x63ffb00 con 0x507ac00 2026-02-11T23:31:28.142+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.143+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 657 ==== MOSDPGPush(4.0 119/117 [PushOp(4:84855744:::obj451:head, version: 111'451, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:84855744:::obj451:head@111'451, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3815380 con 0x507ac00 2026-02-11T23:31:28.143+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:84855744:::obj451:head, version: 111'451, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:84855744:::obj451:head@111'451, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.143+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:84855744:::obj451:head, version: 111'451, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:84855744:::obj451:head@111'451, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.143+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.143+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 658 ==== pg_backfill(progress 4.0 e 119/119 lb 4:8421ae49:::obj196:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9a680 con 0x507ac00 2026-02-11T23:31:28.143+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:84855744:::obj451:head, version: 111'451, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:84855744:::obj451:head@111'451, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.143+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:8421ae49:::obj196:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 119 2026-02-11T23:31:28.143+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8421ae49:::obj196:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.143+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.143+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:84855744:::obj451:head, version: 111'451, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:84855744:::obj451:head@111'451, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.143+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:84855744:::obj451:head, version: 111'451, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:84855744:::obj451:head@111'451, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 119/117 [PushOp(4:84855744:::obj451:head, version: 111'451, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:84855744:::obj451:head@111'451, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8280535c:::obj236:head local-lis/les=117/118 n=308 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.143+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.143+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.143+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8421ae49:::obj196:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.143+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8280535c:::obj236:head local-lis/les=117/118 n=308 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:84855744:::obj451:head, version: 111'451, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:84855744:::obj451:head@111'451, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.143+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8280535c:::obj236:head local-lis/les=117/118 n=308 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:84855744:::obj451:head@111'451, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.143+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8280535c:::obj236:head local-lis/les=117/118 n=308 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:84855744:::obj451:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.143+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8280535c:::obj236:head local-lis/les=117/118 n=308 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:84855744:::obj451:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.143+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8280535c:::obj236:head local-lis/les=117/118 n=308 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:84855744:::obj451:head 2026-02-11T23:31:28.143+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8280535c:::obj236:head local-lis/les=117/118 n=308 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:84855744:::obj451:head v 111'451 2026-02-11T23:31:28.143+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8280535c:::obj236:head local-lis/les=117/118 n=308 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.143+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.143+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:84855744:::obj451:head, version: 111'451, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:84855744:::obj451:head@111'451, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:28.143+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8280535c:::obj236:head local-lis/les=117/118 n=308 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.143+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.143+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.143+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8421ae49:::obj196:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.143+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8421ae49:::obj196:head) v3 prio 15 cost 0 latency 0.000084 pg_backfill(progress 4.0 e 119/119 lb 4:8421ae49:::obj196:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8280535c:::obj236:head local-lis/les=117/118 n=308 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.143+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.143+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8280535c:::obj236:head local-lis/les=117/118 n=308 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:8421ae49:::obj196:head) v3 2026-02-11T23:31:28.143+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8280535c:::obj236:head local-lis/les=117/118 n=308 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:8421ae49:::obj196:head) 2026-02-11T23:31:28.143+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.143+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8421ae49:::obj196:head) v3 finish latency 0.000021 2026-02-11T23:31:28.143+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.143+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.143+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8421ae49:::obj196:head local-lis/les=117/118 n=309 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.143+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.143+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.143+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.143+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c9080 con 0x507ac00 2026-02-11T23:31:28.143+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:84855744:::obj451:head)]) -- 0x644c180 con 0x507ac00 2026-02-11T23:31:28.143+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.143+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.143+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.143+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 659 ==== MOSDPGPush(4.0 119/117 [PushOp(4:84c1344d:::obj97:head, version: 111'97, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:84c1344d:::obj97:head@111'97, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5e26b60 con 0x507ac00 2026-02-11T23:31:28.143+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:84c1344d:::obj97:head, version: 111'97, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:84c1344d:::obj97:head@111'97, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 119 2026-02-11T23:31:28.143+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:84c1344d:::obj97:head, version: 111'97, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:84c1344d:::obj97:head@111'97, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.143+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.143+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 660 ==== pg_backfill(progress 4.0 e 119/119 lb 4:84855744:::obj451:head) ==== 1062+0+0 (unknown 0 0 0) 0x5985700 con 0x507ac00 2026-02-11T23:31:28.143+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:84855744:::obj451:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.143+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:84c1344d:::obj97:head, version: 111'97, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:84c1344d:::obj97:head@111'97, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.143+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:84855744:::obj451:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.143+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.143+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:84c1344d:::obj97:head, version: 111'97, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:84c1344d:::obj97:head@111'97, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.143+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:84c1344d:::obj97:head, version: 111'97, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:84c1344d:::obj97:head@111'97, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(4.0 119/117 [PushOp(4:84c1344d:::obj97:head, version: 111'97, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:84c1344d:::obj97:head@111'97, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8421ae49:::obj196:head local-lis/les=117/118 n=309 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.143+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.143+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.143+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8421ae49:::obj196:head local-lis/les=117/118 n=309 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:84c1344d:::obj97:head, version: 111'97, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:84c1344d:::obj97:head@111'97, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.143+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:84855744:::obj451:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.143+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8421ae49:::obj196:head local-lis/les=117/118 n=309 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:84c1344d:::obj97:head@111'97, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.143+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8421ae49:::obj196:head local-lis/les=117/118 n=309 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:84c1344d:::obj97:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.143+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8421ae49:::obj196:head local-lis/les=117/118 n=309 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:84c1344d:::obj97:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.143+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8421ae49:::obj196:head local-lis/les=117/118 n=309 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:84c1344d:::obj97:head 2026-02-11T23:31:28.143+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8421ae49:::obj196:head local-lis/les=117/118 n=309 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:84c1344d:::obj97:head v 111'97 2026-02-11T23:31:28.143+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8421ae49:::obj196:head local-lis/les=117/118 n=309 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.143+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.143+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:84c1344d:::obj97:head, version: 111'97, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:84c1344d:::obj97:head@111'97, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:28.143+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8421ae49:::obj196:head local-lis/les=117/118 n=309 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.143+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.143+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.143+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:84855744:::obj451:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.143+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:84855744:::obj451:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 119/119 lb 4:84855744:::obj451:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8421ae49:::obj196:head local-lis/les=117/118 n=309 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.143+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.143+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8421ae49:::obj196:head local-lis/les=117/118 n=309 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:84855744:::obj451:head) v3 2026-02-11T23:31:28.143+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8421ae49:::obj196:head local-lis/les=117/118 n=309 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:84855744:::obj451:head) 2026-02-11T23:31:28.144+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.144+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:84855744:::obj451:head) v3 finish latency 0.000031 2026-02-11T23:31:28.144+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.144+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.144+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:84855744:::obj451:head local-lis/les=117/118 n=310 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.144+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.144+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.144+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.144+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x5f2dc80 con 0x507ac00 2026-02-11T23:31:28.144+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:84c1344d:::obj97:head)]) -- 0x63ff200 con 0x507ac00 2026-02-11T23:31:28.144+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.144+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 661 ==== MOSDPGPush(4.0 119/117 [PushOp(4:86bb8a56:::obj416:head, version: 111'416, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:86bb8a56:::obj416:head@111'416, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a29c0 con 0x507ac00 2026-02-11T23:31:28.144+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:86bb8a56:::obj416:head, version: 111'416, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:86bb8a56:::obj416:head@111'416, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 119 2026-02-11T23:31:28.144+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:86bb8a56:::obj416:head, version: 111'416, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:86bb8a56:::obj416:head@111'416, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.144+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.144+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 662 ==== pg_backfill(progress 4.0 e 119/119 lb 4:84c1344d:::obj97:head) ==== 1061+0+0 (unknown 0 0 0) 0x6177600 con 0x507ac00 2026-02-11T23:31:28.144+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:84c1344d:::obj97:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.144+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:86bb8a56:::obj416:head, version: 111'416, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:86bb8a56:::obj416:head@111'416, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.144+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:84c1344d:::obj97:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.144+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.144+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:86bb8a56:::obj416:head, version: 111'416, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:86bb8a56:::obj416:head@111'416, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.144+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:86bb8a56:::obj416:head, version: 111'416, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:86bb8a56:::obj416:head@111'416, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(4.0 119/117 [PushOp(4:86bb8a56:::obj416:head, version: 111'416, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:86bb8a56:::obj416:head@111'416, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:84855744:::obj451:head local-lis/les=117/118 n=310 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.144+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.144+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.144+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:84c1344d:::obj97:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.144+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:84855744:::obj451:head local-lis/les=117/118 n=310 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:86bb8a56:::obj416:head, version: 111'416, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:86bb8a56:::obj416:head@111'416, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.144+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:84855744:::obj451:head local-lis/les=117/118 n=310 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:86bb8a56:::obj416:head@111'416, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.144+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:84855744:::obj451:head local-lis/les=117/118 n=310 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:86bb8a56:::obj416:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.144+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:84855744:::obj451:head local-lis/les=117/118 n=310 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:86bb8a56:::obj416:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.144+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:84855744:::obj451:head local-lis/les=117/118 n=310 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:86bb8a56:::obj416:head 2026-02-11T23:31:28.144+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:84855744:::obj451:head local-lis/les=117/118 n=310 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:86bb8a56:::obj416:head v 111'416 2026-02-11T23:31:28.144+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:84855744:::obj451:head local-lis/les=117/118 n=310 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.144+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.144+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:86bb8a56:::obj416:head, version: 111'416, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:86bb8a56:::obj416:head@111'416, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:28.144+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:84855744:::obj451:head local-lis/les=117/118 n=310 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.144+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.144+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.144+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:84c1344d:::obj97:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.144+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:84c1344d:::obj97:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 4.0 e 119/119 lb 4:84c1344d:::obj97:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:84855744:::obj451:head local-lis/les=117/118 n=310 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.144+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.144+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:84855744:::obj451:head local-lis/les=117/118 n=310 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:84c1344d:::obj97:head) v3 2026-02-11T23:31:28.144+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:84855744:::obj451:head local-lis/les=117/118 n=310 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:84c1344d:::obj97:head) 2026-02-11T23:31:28.144+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.144+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:84c1344d:::obj97:head) v3 finish latency 0.000016 2026-02-11T23:31:28.144+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.144+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.144+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:84c1344d:::obj97:head local-lis/les=117/118 n=311 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.144+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.144+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.144+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.144+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63fef00 con 0x507ac00 2026-02-11T23:31:28.144+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:86bb8a56:::obj416:head)]) -- 0x4f7b200 con 0x507ac00 2026-02-11T23:31:28.144+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.145+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 663 ==== MOSDPGPush(4.0 119/117 [PushOp(4:8745f65b:::obj476:head, version: 111'476, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8745f65b:::obj476:head@111'476, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f1860 con 0x507ac00 2026-02-11T23:31:28.145+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:8745f65b:::obj476:head, version: 111'476, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8745f65b:::obj476:head@111'476, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.145+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8745f65b:::obj476:head, version: 111'476, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8745f65b:::obj476:head@111'476, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.145+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.145+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 664 ==== pg_backfill(progress 4.0 e 119/119 lb 4:86bb8a56:::obj416:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177b80 con 0x507ac00 2026-02-11T23:31:28.145+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:86bb8a56:::obj416:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.145+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8745f65b:::obj476:head, version: 111'476, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8745f65b:::obj476:head@111'476, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.145+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:86bb8a56:::obj416:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.145+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.145+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8745f65b:::obj476:head, version: 111'476, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8745f65b:::obj476:head@111'476, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.145+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.145+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8745f65b:::obj476:head, version: 111'476, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8745f65b:::obj476:head@111'476, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 119/117 [PushOp(4:8745f65b:::obj476:head, version: 111'476, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8745f65b:::obj476:head@111'476, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:84c1344d:::obj97:head local-lis/les=117/118 n=311 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.145+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:86bb8a56:::obj416:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.145+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.145+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:84c1344d:::obj97:head local-lis/les=117/118 n=311 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:8745f65b:::obj476:head, version: 111'476, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8745f65b:::obj476:head@111'476, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.145+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:84c1344d:::obj97:head local-lis/les=117/118 n=311 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8745f65b:::obj476:head@111'476, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.145+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:84c1344d:::obj97:head local-lis/les=117/118 n=311 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:8745f65b:::obj476:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.145+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:84c1344d:::obj97:head local-lis/les=117/118 n=311 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:8745f65b:::obj476:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.145+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:84c1344d:::obj97:head local-lis/les=117/118 n=311 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:8745f65b:::obj476:head 2026-02-11T23:31:28.145+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:84c1344d:::obj97:head local-lis/les=117/118 n=311 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:8745f65b:::obj476:head v 111'476 2026-02-11T23:31:28.145+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:84c1344d:::obj97:head local-lis/les=117/118 n=311 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.145+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.145+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8745f65b:::obj476:head, version: 111'476, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8745f65b:::obj476:head@111'476, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:28.145+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:84c1344d:::obj97:head local-lis/les=117/118 n=311 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.145+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.145+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.145+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:86bb8a56:::obj416:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.145+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:86bb8a56:::obj416:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 4.0 e 119/119 lb 4:86bb8a56:::obj416:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:84c1344d:::obj97:head local-lis/les=117/118 n=311 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.145+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.145+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:84c1344d:::obj97:head local-lis/les=117/118 n=311 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:86bb8a56:::obj416:head) v3 2026-02-11T23:31:28.145+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:84c1344d:::obj97:head local-lis/les=117/118 n=311 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:86bb8a56:::obj416:head) 2026-02-11T23:31:28.145+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.145+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:86bb8a56:::obj416:head) v3 finish latency 0.000012 2026-02-11T23:31:28.145+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.145+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.145+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:86bb8a56:::obj416:head local-lis/les=117/118 n=312 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.145+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.145+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.145+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.145+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7a300 con 0x507ac00 2026-02-11T23:31:28.145+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:8745f65b:::obj476:head)]) -- 0x644db00 con 0x507ac00 2026-02-11T23:31:28.145+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.145+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 665 ==== MOSDPGPush(4.0 119/117 [PushOp(4:87eb7818:::obj91:head, version: 111'91, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:87eb7818:::obj91:head@111'91, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x559d520 con 0x507ac00 2026-02-11T23:31:28.145+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:87eb7818:::obj91:head, version: 111'91, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:87eb7818:::obj91:head@111'91, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 119 2026-02-11T23:31:28.145+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:87eb7818:::obj91:head, version: 111'91, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:87eb7818:::obj91:head@111'91, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.145+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.145+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 666 ==== pg_backfill(progress 4.0 e 119/119 lb 4:8745f65b:::obj476:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2c580 con 0x507ac00 2026-02-11T23:31:28.145+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:8745f65b:::obj476:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.145+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:87eb7818:::obj91:head, version: 111'91, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:87eb7818:::obj91:head@111'91, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.145+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8745f65b:::obj476:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.145+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.145+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:87eb7818:::obj91:head, version: 111'91, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:87eb7818:::obj91:head@111'91, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.145+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.145+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8745f65b:::obj476:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.145+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:87eb7818:::obj91:head, version: 111'91, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:87eb7818:::obj91:head@111'91, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(4.0 119/117 [PushOp(4:87eb7818:::obj91:head, version: 111'91, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:87eb7818:::obj91:head@111'91, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:86bb8a56:::obj416:head local-lis/les=117/118 n=312 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.145+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.145+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:86bb8a56:::obj416:head local-lis/les=117/118 n=312 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:87eb7818:::obj91:head, version: 111'91, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:87eb7818:::obj91:head@111'91, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.145+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:86bb8a56:::obj416:head local-lis/les=117/118 n=312 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:87eb7818:::obj91:head@111'91, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.145+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:86bb8a56:::obj416:head local-lis/les=117/118 n=312 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:87eb7818:::obj91:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.145+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:86bb8a56:::obj416:head local-lis/les=117/118 n=312 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:87eb7818:::obj91:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.145+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:86bb8a56:::obj416:head local-lis/les=117/118 n=312 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:87eb7818:::obj91:head 2026-02-11T23:31:28.145+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:86bb8a56:::obj416:head local-lis/les=117/118 n=312 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:87eb7818:::obj91:head v 111'91 2026-02-11T23:31:28.145+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:86bb8a56:::obj416:head local-lis/les=117/118 n=312 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.145+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.145+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:87eb7818:::obj91:head, version: 111'91, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:87eb7818:::obj91:head@111'91, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:28.145+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:86bb8a56:::obj416:head local-lis/les=117/118 n=312 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.145+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.145+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.145+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8745f65b:::obj476:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.145+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8745f65b:::obj476:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 4.0 e 119/119 lb 4:8745f65b:::obj476:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:86bb8a56:::obj416:head local-lis/les=117/118 n=312 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.145+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.145+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:86bb8a56:::obj416:head local-lis/les=117/118 n=312 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:8745f65b:::obj476:head) v3 2026-02-11T23:31:28.145+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:86bb8a56:::obj416:head local-lis/les=117/118 n=312 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:8745f65b:::obj476:head) 2026-02-11T23:31:28.145+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.145+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8745f65b:::obj476:head) v3 finish latency 0.000013 2026-02-11T23:31:28.145+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.146+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.146+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8745f65b:::obj476:head local-lis/les=117/118 n=313 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.146+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.146+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.146+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.146+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7a900 con 0x507ac00 2026-02-11T23:31:28.146+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:87eb7818:::obj91:head)]) -- 0x63fec00 con 0x507ac00 2026-02-11T23:31:28.146+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.146+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 667 ==== MOSDPGPush(4.0 119/117 [PushOp(4:881499a5:::obj460:head, version: 111'460, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:881499a5:::obj460:head@111'460, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56ca9c0 con 0x507ac00 2026-02-11T23:31:28.146+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:881499a5:::obj460:head, version: 111'460, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:881499a5:::obj460:head@111'460, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 119 2026-02-11T23:31:28.146+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:881499a5:::obj460:head, version: 111'460, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:881499a5:::obj460:head@111'460, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.146+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.146+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 668 ==== pg_backfill(progress 4.0 e 119/119 lb 4:87eb7818:::obj91:head) ==== 1061+0+0 (unknown 0 0 0) 0x6178100 con 0x507ac00 2026-02-11T23:31:28.146+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:87eb7818:::obj91:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.146+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:87eb7818:::obj91:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.146+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:881499a5:::obj460:head, version: 111'460, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:881499a5:::obj460:head@111'460, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.146+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.146+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:881499a5:::obj460:head, version: 111'460, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:881499a5:::obj460:head@111'460, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.146+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:881499a5:::obj460:head, version: 111'460, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:881499a5:::obj460:head@111'460, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(4.0 119/117 [PushOp(4:881499a5:::obj460:head, version: 111'460, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:881499a5:::obj460:head@111'460, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8745f65b:::obj476:head local-lis/les=117/118 n=313 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.146+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.146+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.146+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:87eb7818:::obj91:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.146+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8745f65b:::obj476:head local-lis/les=117/118 n=313 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:881499a5:::obj460:head, version: 111'460, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:881499a5:::obj460:head@111'460, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.146+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8745f65b:::obj476:head local-lis/les=117/118 n=313 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:881499a5:::obj460:head@111'460, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.146+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8745f65b:::obj476:head local-lis/les=117/118 n=313 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:881499a5:::obj460:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.146+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8745f65b:::obj476:head local-lis/les=117/118 n=313 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:881499a5:::obj460:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.146+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8745f65b:::obj476:head local-lis/les=117/118 n=313 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:881499a5:::obj460:head 2026-02-11T23:31:28.146+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8745f65b:::obj476:head local-lis/les=117/118 n=313 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:881499a5:::obj460:head v 111'460 2026-02-11T23:31:28.146+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8745f65b:::obj476:head local-lis/les=117/118 n=313 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.146+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.146+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:881499a5:::obj460:head, version: 111'460, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:881499a5:::obj460:head@111'460, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000068 2026-02-11T23:31:28.146+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.146+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.146+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:87eb7818:::obj91:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.146+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:87eb7818:::obj91:head) v3 prio 15 cost 0 latency 0.000089 pg_backfill(progress 4.0 e 119/119 lb 4:87eb7818:::obj91:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8745f65b:::obj476:head local-lis/les=117/118 n=313 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.146+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.146+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8745f65b:::obj476:head local-lis/les=117/118 n=313 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:87eb7818:::obj91:head) v3 2026-02-11T23:31:28.146+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8745f65b:::obj476:head local-lis/les=117/118 n=313 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:87eb7818:::obj91:head) 2026-02-11T23:31:28.146+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.146+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:87eb7818:::obj91:head) v3 finish latency 0.000015 2026-02-11T23:31:28.146+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:87eb7818:::obj91:head local-lis/les=117/118 n=314 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.146+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.146+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:87eb7818:::obj91:head local-lis/les=117/118 n=314 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.146+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.146+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.146+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.146+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7b380 con 0x507ac00 2026-02-11T23:31:28.146+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.146+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:881499a5:::obj460:head)]) -- 0x4f7aa80 con 0x507ac00 2026-02-11T23:31:28.146+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.146+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 669 ==== MOSDPGPush(4.0 119/117 [PushOp(4:8835249a:::obj346:head, version: 111'346, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8835249a:::obj346:head@111'346, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c000 con 0x507ac00 2026-02-11T23:31:28.147+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:8835249a:::obj346:head, version: 111'346, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8835249a:::obj346:head@111'346, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000017 epoch 119 2026-02-11T23:31:28.147+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8835249a:::obj346:head, version: 111'346, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8835249a:::obj346:head@111'346, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.147+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.147+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 670 ==== pg_backfill(progress 4.0 e 119/119 lb 4:881499a5:::obj460:head) ==== 1062+0+0 (unknown 0 0 0) 0x6179180 con 0x507ac00 2026-02-11T23:31:28.147+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:881499a5:::obj460:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.147+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8835249a:::obj346:head, version: 111'346, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8835249a:::obj346:head@111'346, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.147+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:881499a5:::obj460:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.147+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.147+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8835249a:::obj346:head, version: 111'346, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8835249a:::obj346:head@111'346, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.147+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.147+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8835249a:::obj346:head, version: 111'346, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8835249a:::obj346:head@111'346, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(4.0 119/117 [PushOp(4:8835249a:::obj346:head, version: 111'346, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8835249a:::obj346:head@111'346, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:87eb7818:::obj91:head local-lis/les=117/118 n=314 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.147+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.147+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:87eb7818:::obj91:head local-lis/les=117/118 n=314 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:8835249a:::obj346:head, version: 111'346, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8835249a:::obj346:head@111'346, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.147+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.147+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:87eb7818:::obj91:head local-lis/les=117/118 n=314 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8835249a:::obj346:head@111'346, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.147+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:881499a5:::obj460:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.147+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:87eb7818:::obj91:head local-lis/les=117/118 n=314 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:8835249a:::obj346:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.147+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:87eb7818:::obj91:head local-lis/les=117/118 n=314 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:8835249a:::obj346:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.147+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:87eb7818:::obj91:head local-lis/les=117/118 n=314 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:8835249a:::obj346:head 2026-02-11T23:31:28.147+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:87eb7818:::obj91:head local-lis/les=117/118 n=314 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:8835249a:::obj346:head v 111'346 2026-02-11T23:31:28.147+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:87eb7818:::obj91:head local-lis/les=117/118 n=314 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.147+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.147+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8835249a:::obj346:head, version: 111'346, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8835249a:::obj346:head@111'346, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:28.147+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:87eb7818:::obj91:head local-lis/les=117/118 n=314 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.147+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.147+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.147+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:881499a5:::obj460:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.147+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:881499a5:::obj460:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 4.0 e 119/119 lb 4:881499a5:::obj460:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:87eb7818:::obj91:head local-lis/les=117/118 n=314 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.147+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.147+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:87eb7818:::obj91:head local-lis/les=117/118 n=314 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:881499a5:::obj460:head) v3 2026-02-11T23:31:28.147+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:87eb7818:::obj91:head local-lis/les=117/118 n=314 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:881499a5:::obj460:head) 2026-02-11T23:31:28.147+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.147+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:881499a5:::obj460:head) v3 finish latency 0.000016 2026-02-11T23:31:28.147+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.147+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.147+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:881499a5:::obj460:head local-lis/les=117/118 n=315 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.147+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.147+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.147+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.147+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x649e600 con 0x507ac00 2026-02-11T23:31:28.147+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:8835249a:::obj346:head)]) -- 0x63c5c80 con 0x507ac00 2026-02-11T23:31:28.147+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.147+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 671 ==== MOSDPGPush(4.0 119/117 [PushOp(4:88c90a82:::obj574:head, version: 111'574, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:88c90a82:::obj574:head@111'574, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2b60 con 0x507ac00 2026-02-11T23:31:28.147+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:88c90a82:::obj574:head, version: 111'574, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:88c90a82:::obj574:head@111'574, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.147+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:88c90a82:::obj574:head, version: 111'574, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:88c90a82:::obj574:head@111'574, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.147+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.147+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 672 ==== pg_backfill(progress 4.0 e 119/119 lb 4:8835249a:::obj346:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9b180 con 0x507ac00 2026-02-11T23:31:28.147+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:8835249a:::obj346:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.147+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8835249a:::obj346:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.147+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:88c90a82:::obj574:head, version: 111'574, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:88c90a82:::obj574:head@111'574, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.147+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.147+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:88c90a82:::obj574:head, version: 111'574, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:88c90a82:::obj574:head@111'574, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.147+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:88c90a82:::obj574:head, version: 111'574, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:88c90a82:::obj574:head@111'574, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 119/117 [PushOp(4:88c90a82:::obj574:head, version: 111'574, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:88c90a82:::obj574:head@111'574, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:881499a5:::obj460:head local-lis/les=117/118 n=315 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.147+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.147+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.147+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8835249a:::obj346:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.147+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:881499a5:::obj460:head local-lis/les=117/118 n=315 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:88c90a82:::obj574:head, version: 111'574, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:88c90a82:::obj574:head@111'574, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.147+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:881499a5:::obj460:head local-lis/les=117/118 n=315 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:88c90a82:::obj574:head@111'574, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.147+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:881499a5:::obj460:head local-lis/les=117/118 n=315 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:88c90a82:::obj574:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.147+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:881499a5:::obj460:head local-lis/les=117/118 n=315 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:88c90a82:::obj574:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.147+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:881499a5:::obj460:head local-lis/les=117/118 n=315 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:88c90a82:::obj574:head 2026-02-11T23:31:28.147+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:881499a5:::obj460:head local-lis/les=117/118 n=315 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:88c90a82:::obj574:head v 111'574 2026-02-11T23:31:28.147+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:881499a5:::obj460:head local-lis/les=117/118 n=315 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.147+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.147+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:88c90a82:::obj574:head, version: 111'574, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:88c90a82:::obj574:head@111'574, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:28.147+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.147+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.147+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8835249a:::obj346:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.147+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8835249a:::obj346:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 4.0 e 119/119 lb 4:8835249a:::obj346:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:881499a5:::obj460:head local-lis/les=117/118 n=315 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.147+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.147+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:881499a5:::obj460:head local-lis/les=117/118 n=315 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:8835249a:::obj346:head) v3 2026-02-11T23:31:28.147+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:881499a5:::obj460:head local-lis/les=117/118 n=315 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:8835249a:::obj346:head) 2026-02-11T23:31:28.147+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.147+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8835249a:::obj346:head) v3 finish latency 0.000013 2026-02-11T23:31:28.147+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8835249a:::obj346:head local-lis/les=117/118 n=316 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.147+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.147+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.147+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8835249a:::obj346:head local-lis/les=117/118 n=316 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.147+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.147+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.147+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.147+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7a780 con 0x507ac00 2026-02-11T23:31:28.147+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:88c90a82:::obj574:head)]) -- 0x5d54c00 con 0x507ac00 2026-02-11T23:31:28.147+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.148+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 673 ==== MOSDPGPush(4.0 119/117 [PushOp(4:88e6b0fd:::obj99:head, version: 111'99, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:88e6b0fd:::obj99:head@111'99, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x554e680 con 0x507ac00 2026-02-11T23:31:28.148+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:88e6b0fd:::obj99:head, version: 111'99, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:88e6b0fd:::obj99:head@111'99, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 119 2026-02-11T23:31:28.148+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:88e6b0fd:::obj99:head, version: 111'99, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:88e6b0fd:::obj99:head@111'99, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.148+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.148+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 674 ==== pg_backfill(progress 4.0 e 119/119 lb 4:88c90a82:::obj574:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98000 con 0x507ac00 2026-02-11T23:31:28.148+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:88c90a82:::obj574:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.148+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:88c90a82:::obj574:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.148+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:88e6b0fd:::obj99:head, version: 111'99, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:88e6b0fd:::obj99:head@111'99, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.148+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.148+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:88e6b0fd:::obj99:head, version: 111'99, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:88e6b0fd:::obj99:head@111'99, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.148+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:88e6b0fd:::obj99:head, version: 111'99, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:88e6b0fd:::obj99:head@111'99, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(4.0 119/117 [PushOp(4:88e6b0fd:::obj99:head, version: 111'99, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:88e6b0fd:::obj99:head@111'99, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8835249a:::obj346:head local-lis/les=117/118 n=316 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.148+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.148+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.148+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:88c90a82:::obj574:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.148+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8835249a:::obj346:head local-lis/les=117/118 n=316 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:88e6b0fd:::obj99:head, version: 111'99, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:88e6b0fd:::obj99:head@111'99, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.148+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8835249a:::obj346:head local-lis/les=117/118 n=316 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:88e6b0fd:::obj99:head@111'99, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.148+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8835249a:::obj346:head local-lis/les=117/118 n=316 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:88e6b0fd:::obj99:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.148+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8835249a:::obj346:head local-lis/les=117/118 n=316 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:88e6b0fd:::obj99:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.148+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8835249a:::obj346:head local-lis/les=117/118 n=316 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:88e6b0fd:::obj99:head 2026-02-11T23:31:28.148+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8835249a:::obj346:head local-lis/les=117/118 n=316 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:88e6b0fd:::obj99:head v 111'99 2026-02-11T23:31:28.148+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8835249a:::obj346:head local-lis/les=117/118 n=316 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.148+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.148+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:88e6b0fd:::obj99:head, version: 111'99, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:88e6b0fd:::obj99:head@111'99, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:28.148+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8835249a:::obj346:head local-lis/les=117/118 n=316 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.148+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.148+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.148+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:88c90a82:::obj574:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.148+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:88c90a82:::obj574:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 4.0 e 119/119 lb 4:88c90a82:::obj574:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8835249a:::obj346:head local-lis/les=117/118 n=316 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.148+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.148+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8835249a:::obj346:head local-lis/les=117/118 n=316 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:88c90a82:::obj574:head) v3 2026-02-11T23:31:28.148+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8835249a:::obj346:head local-lis/les=117/118 n=316 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:88c90a82:::obj574:head) 2026-02-11T23:31:28.148+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.148+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:88c90a82:::obj574:head) v3 finish latency 0.000016 2026-02-11T23:31:28.148+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.148+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.148+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:88c90a82:::obj574:head local-lis/les=117/118 n=317 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.148+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.148+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.148+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.148+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x644d800 con 0x507ac00 2026-02-11T23:31:28.148+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:88e6b0fd:::obj99:head)]) -- 0x63c9200 con 0x507ac00 2026-02-11T23:31:28.148+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.148+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 675 ==== MOSDPGPush(4.0 119/117 [PushOp(4:89565a01:::obj330:head, version: 111'330, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:89565a01:::obj330:head@111'330, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3815520 con 0x507ac00 2026-02-11T23:31:28.148+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:89565a01:::obj330:head, version: 111'330, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:89565a01:::obj330:head@111'330, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 119 2026-02-11T23:31:28.148+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:89565a01:::obj330:head, version: 111'330, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:89565a01:::obj330:head@111'330, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.148+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.148+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 676 ==== pg_backfill(progress 4.0 e 119/119 lb 4:88e6b0fd:::obj99:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d2ec00 con 0x507ac00 2026-02-11T23:31:28.148+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:88e6b0fd:::obj99:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.148+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:88e6b0fd:::obj99:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.148+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:89565a01:::obj330:head, version: 111'330, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:89565a01:::obj330:head@111'330, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.148+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.148+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:89565a01:::obj330:head, version: 111'330, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:89565a01:::obj330:head@111'330, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.148+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:89565a01:::obj330:head, version: 111'330, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:89565a01:::obj330:head@111'330, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000040 MOSDPGPush(4.0 119/117 [PushOp(4:89565a01:::obj330:head, version: 111'330, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:89565a01:::obj330:head@111'330, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:88c90a82:::obj574:head local-lis/les=117/118 n=317 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.148+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.148+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.148+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:88e6b0fd:::obj99:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.148+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:88c90a82:::obj574:head local-lis/les=117/118 n=317 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:89565a01:::obj330:head, version: 111'330, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:89565a01:::obj330:head@111'330, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.148+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:88c90a82:::obj574:head local-lis/les=117/118 n=317 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:89565a01:::obj330:head@111'330, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.148+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:88c90a82:::obj574:head local-lis/les=117/118 n=317 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:89565a01:::obj330:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.148+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:88c90a82:::obj574:head local-lis/les=117/118 n=317 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:89565a01:::obj330:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.148+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:88c90a82:::obj574:head local-lis/les=117/118 n=317 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:89565a01:::obj330:head 2026-02-11T23:31:28.148+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:88c90a82:::obj574:head local-lis/les=117/118 n=317 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:89565a01:::obj330:head v 111'330 2026-02-11T23:31:28.148+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:88c90a82:::obj574:head local-lis/les=117/118 n=317 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.148+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.148+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:89565a01:::obj330:head, version: 111'330, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:89565a01:::obj330:head@111'330, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:28.148+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.148+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.148+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:88e6b0fd:::obj99:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.148+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:88e6b0fd:::obj99:head) v3 prio 15 cost 0 latency 0.000079 pg_backfill(progress 4.0 e 119/119 lb 4:88e6b0fd:::obj99:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:88c90a82:::obj574:head local-lis/les=117/118 n=317 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.148+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.148+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:88c90a82:::obj574:head local-lis/les=117/118 n=317 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:88e6b0fd:::obj99:head) v3 2026-02-11T23:31:28.148+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:88c90a82:::obj574:head local-lis/les=117/118 n=317 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:88e6b0fd:::obj99:head) 2026-02-11T23:31:28.148+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.148+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:88e6b0fd:::obj99:head) v3 finish latency 0.000013 2026-02-11T23:31:28.148+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:88e6b0fd:::obj99:head local-lis/les=117/118 n=318 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.148+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.149+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.3 127.0.0.1:0/2699377081 18 ==== osd_ping(ping e114 up_from 19 ping_stamp 2026-02-11T23:31:28.150260+0000/63.606197357s send_stamp 63.606197357s delta_ub -7.219326973s) ==== 2033+0+0 (crc 0 0 0) 0x4ecc300 con 0x48cc800 2026-02-11T23:31:28.149+0000 7f416cbfe640 20 osd.0 119 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219552040s,-7.219326973s]) 2026-02-11T23:31:28.149+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e119 up_from 5 ping_stamp 2026-02-11T23:31:28.150260+0000/63.606197357s send_stamp 70.825752258s delta_ub 7.219552040s) -- 0x4ec2180 con 0x48cc800 2026-02-11T23:31:28.149+0000 7f416cbfe640 20 osd.0 119 get_con_osd_cluster to osd.3 from_epoch 119 2026-02-11T23:31:28.149+0000 7f416cbfe640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.149+0000 7f416cbfe640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.149+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.3 127.0.0.1:0/2699377081 18 ==== osd_ping(ping e114 up_from 19 ping_stamp 2026-02-11T23:31:28.150260+0000/63.606197357s send_stamp 63.606197357s delta_ub -7.219326973s) ==== 2033+0+0 (crc 0 0 0) 0x4ecc780 con 0x48cc400 2026-02-11T23:31:28.149+0000 7f416cbfe640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6828/2699377081 updating session's projected_epoch from 113 to ping map epoch of 114 2026-02-11T23:31:28.149+0000 7f416cbfe640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6828/2699377081 map epoch 114 -> 119 (shared) 2026-02-11T23:31:28.149+0000 7f416cbfe640 10 osd.0 119 send_incremental_map epoch range: (115, 119] to v2:127.0.0.1:6828/2699377081 2026-02-11T23:31:28.149+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6828/2699377081,v1:127.0.0.1:6829/2699377081] -- osd_map(115..119 src has 1..119) -- 0x5954fc0 con 0x48ccc00 2026-02-11T23:31:28.149+0000 7f416c3fd640 20 osd.0 119 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219604492s,-7.219326973s]) 2026-02-11T23:31:28.149+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e119 up_from 5 ping_stamp 2026-02-11T23:31:28.150260+0000/63.606197357s send_stamp 70.825805664s delta_ub 7.219604492s) -- 0x4ecca80 con 0x48cc400 2026-02-11T23:31:28.149+0000 7f416c3fd640 20 osd.0 119 get_con_osd_cluster to osd.3 from_epoch 119 2026-02-11T23:31:28.149+0000 7f416c3fd640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.149+0000 7f416c3fd640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.149+0000 7f416c3fd640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.149+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.149+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:88e6b0fd:::obj99:head local-lis/les=117/118 n=318 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.149+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.149+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.149+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.149+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7b080 con 0x507ac00 2026-02-11T23:31:28.149+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:89565a01:::obj330:head)]) -- 0x644d380 con 0x507ac00 2026-02-11T23:31:28.149+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.149+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 677 ==== MOSDPGPush(4.0 119/117 [PushOp(4:8a5b4a06:::obj393:head, version: 111'393, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8a5b4a06:::obj393:head@111'393, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56171e0 con 0x507ac00 2026-02-11T23:31:28.149+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:8a5b4a06:::obj393:head, version: 111'393, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8a5b4a06:::obj393:head@111'393, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 119 2026-02-11T23:31:28.149+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8a5b4a06:::obj393:head, version: 111'393, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8a5b4a06:::obj393:head@111'393, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.149+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.149+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 678 ==== pg_backfill(progress 4.0 e 119/119 lb 4:89565a01:::obj330:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2d600 con 0x507ac00 2026-02-11T23:31:28.149+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:89565a01:::obj330:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.149+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8a5b4a06:::obj393:head, version: 111'393, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8a5b4a06:::obj393:head@111'393, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.149+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:89565a01:::obj330:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.149+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.149+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8a5b4a06:::obj393:head, version: 111'393, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8a5b4a06:::obj393:head@111'393, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.149+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8a5b4a06:::obj393:head, version: 111'393, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8a5b4a06:::obj393:head@111'393, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(4.0 119/117 [PushOp(4:8a5b4a06:::obj393:head, version: 111'393, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8a5b4a06:::obj393:head@111'393, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:88e6b0fd:::obj99:head local-lis/les=117/118 n=318 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.149+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.149+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:88e6b0fd:::obj99:head local-lis/les=117/118 n=318 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:8a5b4a06:::obj393:head, version: 111'393, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8a5b4a06:::obj393:head@111'393, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.149+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.149+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:88e6b0fd:::obj99:head local-lis/les=117/118 n=318 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8a5b4a06:::obj393:head@111'393, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.149+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:89565a01:::obj330:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.149+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:88e6b0fd:::obj99:head local-lis/les=117/118 n=318 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:8a5b4a06:::obj393:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.149+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:88e6b0fd:::obj99:head local-lis/les=117/118 n=318 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:8a5b4a06:::obj393:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.149+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:88e6b0fd:::obj99:head local-lis/les=117/118 n=318 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:8a5b4a06:::obj393:head 2026-02-11T23:31:28.149+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:88e6b0fd:::obj99:head local-lis/les=117/118 n=318 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:8a5b4a06:::obj393:head v 111'393 2026-02-11T23:31:28.149+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:88e6b0fd:::obj99:head local-lis/les=117/118 n=318 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.149+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.149+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8a5b4a06:::obj393:head, version: 111'393, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8a5b4a06:::obj393:head@111'393, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:28.149+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:88e6b0fd:::obj99:head local-lis/les=117/118 n=318 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.149+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.149+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.149+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:89565a01:::obj330:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.149+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:89565a01:::obj330:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 4.0 e 119/119 lb 4:89565a01:::obj330:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:88e6b0fd:::obj99:head local-lis/les=117/118 n=318 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.149+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.149+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:88e6b0fd:::obj99:head local-lis/les=117/118 n=318 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:89565a01:::obj330:head) v3 2026-02-11T23:31:28.149+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:88e6b0fd:::obj99:head local-lis/les=117/118 n=318 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:89565a01:::obj330:head) 2026-02-11T23:31:28.149+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.149+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:89565a01:::obj330:head) v3 finish latency 0.000024 2026-02-11T23:31:28.149+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.150+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.150+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:89565a01:::obj330:head local-lis/les=117/118 n=319 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.150+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.150+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.150+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.150+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x5d55e00 con 0x507ac00 2026-02-11T23:31:28.150+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:8a5b4a06:::obj393:head)]) -- 0x640e180 con 0x507ac00 2026-02-11T23:31:28.150+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.150+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 679 ==== MOSDPGPush(4.0 119/117 [PushOp(4:8a7d5b8a:::obj217:head, version: 111'217, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8a7d5b8a:::obj217:head@111'217, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56cad00 con 0x507ac00 2026-02-11T23:31:28.150+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:8a7d5b8a:::obj217:head, version: 111'217, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8a7d5b8a:::obj217:head@111'217, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 119 2026-02-11T23:31:28.150+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8a7d5b8a:::obj217:head, version: 111'217, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8a7d5b8a:::obj217:head@111'217, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.150+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.150+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 680 ==== pg_backfill(progress 4.0 e 119/119 lb 4:8a5b4a06:::obj393:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99600 con 0x507ac00 2026-02-11T23:31:28.150+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8a7d5b8a:::obj217:head, version: 111'217, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8a7d5b8a:::obj217:head@111'217, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.150+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:8a5b4a06:::obj393:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.150+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8a5b4a06:::obj393:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.150+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.150+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8a7d5b8a:::obj217:head, version: 111'217, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8a7d5b8a:::obj217:head@111'217, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.150+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8a7d5b8a:::obj217:head, version: 111'217, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8a7d5b8a:::obj217:head@111'217, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(4.0 119/117 [PushOp(4:8a7d5b8a:::obj217:head, version: 111'217, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8a7d5b8a:::obj217:head@111'217, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:89565a01:::obj330:head local-lis/les=117/118 n=319 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.150+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.150+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.150+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8a5b4a06:::obj393:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.150+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:89565a01:::obj330:head local-lis/les=117/118 n=319 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:8a7d5b8a:::obj217:head, version: 111'217, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8a7d5b8a:::obj217:head@111'217, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.150+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:89565a01:::obj330:head local-lis/les=117/118 n=319 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8a7d5b8a:::obj217:head@111'217, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.150+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:89565a01:::obj330:head local-lis/les=117/118 n=319 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:8a7d5b8a:::obj217:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.150+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:89565a01:::obj330:head local-lis/les=117/118 n=319 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:8a7d5b8a:::obj217:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.150+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:89565a01:::obj330:head local-lis/les=117/118 n=319 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:8a7d5b8a:::obj217:head 2026-02-11T23:31:28.150+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:89565a01:::obj330:head local-lis/les=117/118 n=319 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:8a7d5b8a:::obj217:head v 111'217 2026-02-11T23:31:28.150+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:89565a01:::obj330:head local-lis/les=117/118 n=319 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.150+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.150+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8a7d5b8a:::obj217:head, version: 111'217, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8a7d5b8a:::obj217:head@111'217, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:28.150+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:89565a01:::obj330:head local-lis/les=117/118 n=319 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.150+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.150+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.150+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8a5b4a06:::obj393:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.150+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8a5b4a06:::obj393:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 4.0 e 119/119 lb 4:8a5b4a06:::obj393:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:89565a01:::obj330:head local-lis/les=117/118 n=319 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.150+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.150+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:89565a01:::obj330:head local-lis/les=117/118 n=319 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:8a5b4a06:::obj393:head) v3 2026-02-11T23:31:28.150+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:89565a01:::obj330:head local-lis/les=117/118 n=319 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:8a5b4a06:::obj393:head) 2026-02-11T23:31:28.150+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.150+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8a5b4a06:::obj393:head) v3 finish latency 0.000018 2026-02-11T23:31:28.150+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.150+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.150+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8a5b4a06:::obj393:head local-lis/les=117/118 n=320 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.150+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.150+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.150+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.150+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x640fe00 con 0x507ac00 2026-02-11T23:31:28.150+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:8a7d5b8a:::obj217:head)]) -- 0x63c8c00 con 0x507ac00 2026-02-11T23:31:28.150+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.151+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 681 ==== MOSDPGPush(4.0 119/117 [PushOp(4:8aa18807:::obj353:head, version: 111'353, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8aa18807:::obj353:head@111'353, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x555cea0 con 0x507ac00 2026-02-11T23:31:28.151+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:8aa18807:::obj353:head, version: 111'353, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8aa18807:::obj353:head@111'353, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 119 2026-02-11T23:31:28.151+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8aa18807:::obj353:head, version: 111'353, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8aa18807:::obj353:head@111'353, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.151+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.151+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 682 ==== pg_backfill(progress 4.0 e 119/119 lb 4:8a7d5b8a:::obj217:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9ac00 con 0x507ac00 2026-02-11T23:31:28.151+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:8a7d5b8a:::obj217:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.151+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8a7d5b8a:::obj217:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.151+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8aa18807:::obj353:head, version: 111'353, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8aa18807:::obj353:head@111'353, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.151+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.151+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8aa18807:::obj353:head, version: 111'353, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8aa18807:::obj353:head@111'353, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.151+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8aa18807:::obj353:head, version: 111'353, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8aa18807:::obj353:head@111'353, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(4.0 119/117 [PushOp(4:8aa18807:::obj353:head, version: 111'353, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8aa18807:::obj353:head@111'353, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8a5b4a06:::obj393:head local-lis/les=117/118 n=320 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.151+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.151+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.151+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8a7d5b8a:::obj217:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.151+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8a5b4a06:::obj393:head local-lis/les=117/118 n=320 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:8aa18807:::obj353:head, version: 111'353, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8aa18807:::obj353:head@111'353, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.151+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8a5b4a06:::obj393:head local-lis/les=117/118 n=320 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8aa18807:::obj353:head@111'353, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.151+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8a5b4a06:::obj393:head local-lis/les=117/118 n=320 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:8aa18807:::obj353:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.151+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8a5b4a06:::obj393:head local-lis/les=117/118 n=320 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:8aa18807:::obj353:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.151+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8a5b4a06:::obj393:head local-lis/les=117/118 n=320 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:8aa18807:::obj353:head 2026-02-11T23:31:28.151+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8a5b4a06:::obj393:head local-lis/les=117/118 n=320 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:8aa18807:::obj353:head v 111'353 2026-02-11T23:31:28.151+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8a5b4a06:::obj393:head local-lis/les=117/118 n=320 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.151+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.151+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8aa18807:::obj353:head, version: 111'353, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8aa18807:::obj353:head@111'353, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000074 2026-02-11T23:31:28.151+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.151+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.151+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8a7d5b8a:::obj217:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.151+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8a7d5b8a:::obj217:head) v3 prio 15 cost 0 latency 0.000097 pg_backfill(progress 4.0 e 119/119 lb 4:8a7d5b8a:::obj217:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8a5b4a06:::obj393:head local-lis/les=117/118 n=320 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.151+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.151+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8a5b4a06:::obj393:head local-lis/les=117/118 n=320 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:8a7d5b8a:::obj217:head) v3 2026-02-11T23:31:28.151+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8a5b4a06:::obj393:head local-lis/les=117/118 n=320 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:8a7d5b8a:::obj217:head) 2026-02-11T23:31:28.151+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.151+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8a7d5b8a:::obj217:head) v3 finish latency 0.000016 2026-02-11T23:31:28.151+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8a7d5b8a:::obj217:head local-lis/les=117/118 n=321 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.151+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.151+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.151+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8a7d5b8a:::obj217:head local-lis/les=117/118 n=321 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.151+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.151+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.151+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.151+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x646ca80 con 0x507ac00 2026-02-11T23:31:28.151+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:8aa18807:::obj353:head)]) -- 0x5d55200 con 0x507ac00 2026-02-11T23:31:28.151+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.151+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 683 ==== MOSDPGPush(4.0 119/117 [PushOp(4:8b0915dc:::obj278:head, version: 111'278, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b0915dc:::obj278:head@111'278, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f1ba0 con 0x507ac00 2026-02-11T23:31:28.151+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:8b0915dc:::obj278:head, version: 111'278, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b0915dc:::obj278:head@111'278, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 119 2026-02-11T23:31:28.151+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8b0915dc:::obj278:head, version: 111'278, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b0915dc:::obj278:head@111'278, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.151+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.151+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 684 ==== pg_backfill(progress 4.0 e 119/119 lb 4:8aa18807:::obj353:head) ==== 1062+0+0 (unknown 0 0 0) 0x4a2cb00 con 0x507ac00 2026-02-11T23:31:28.151+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:8aa18807:::obj353:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.151+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8b0915dc:::obj278:head, version: 111'278, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b0915dc:::obj278:head@111'278, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.151+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8aa18807:::obj353:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.151+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.151+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8b0915dc:::obj278:head, version: 111'278, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b0915dc:::obj278:head@111'278, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.151+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8b0915dc:::obj278:head, version: 111'278, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b0915dc:::obj278:head@111'278, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(4.0 119/117 [PushOp(4:8b0915dc:::obj278:head, version: 111'278, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b0915dc:::obj278:head@111'278, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8a7d5b8a:::obj217:head local-lis/les=117/118 n=321 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.151+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.151+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.151+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8a7d5b8a:::obj217:head local-lis/les=117/118 n=321 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:8b0915dc:::obj278:head, version: 111'278, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b0915dc:::obj278:head@111'278, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.151+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8aa18807:::obj353:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.151+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8a7d5b8a:::obj217:head local-lis/les=117/118 n=321 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8b0915dc:::obj278:head@111'278, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.151+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8a7d5b8a:::obj217:head local-lis/les=117/118 n=321 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:8b0915dc:::obj278:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.151+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8a7d5b8a:::obj217:head local-lis/les=117/118 n=321 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:8b0915dc:::obj278:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.151+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8a7d5b8a:::obj217:head local-lis/les=117/118 n=321 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:8b0915dc:::obj278:head 2026-02-11T23:31:28.151+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8a7d5b8a:::obj217:head local-lis/les=117/118 n=321 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:8b0915dc:::obj278:head v 111'278 2026-02-11T23:31:28.151+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8a7d5b8a:::obj217:head local-lis/les=117/118 n=321 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.151+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.151+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8b0915dc:::obj278:head, version: 111'278, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b0915dc:::obj278:head@111'278, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:28.151+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8a7d5b8a:::obj217:head local-lis/les=117/118 n=321 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.151+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.151+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.151+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8aa18807:::obj353:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.151+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8aa18807:::obj353:head) v3 prio 15 cost 0 latency 0.000079 pg_backfill(progress 4.0 e 119/119 lb 4:8aa18807:::obj353:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8a7d5b8a:::obj217:head local-lis/les=117/118 n=321 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.151+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.151+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8a7d5b8a:::obj217:head local-lis/les=117/118 n=321 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:8aa18807:::obj353:head) v3 2026-02-11T23:31:28.151+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8a7d5b8a:::obj217:head local-lis/les=117/118 n=321 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:8aa18807:::obj353:head) 2026-02-11T23:31:28.151+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.151+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8aa18807:::obj353:head) v3 finish latency 0.000021 2026-02-11T23:31:28.151+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.152+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.152+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8aa18807:::obj353:head local-lis/les=117/118 n=322 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.152+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.152+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.152+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.152+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x640fb00 con 0x507ac00 2026-02-11T23:31:28.152+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:8b0915dc:::obj278:head)]) -- 0x640fc80 con 0x507ac00 2026-02-11T23:31:28.152+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.152+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 685 ==== MOSDPGPush(4.0 119/117 [PushOp(4:8b38e8a9:::obj385:head, version: 111'385, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b38e8a9:::obj385:head@111'385, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c1a0 con 0x507ac00 2026-02-11T23:31:28.152+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:8b38e8a9:::obj385:head, version: 111'385, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b38e8a9:::obj385:head@111'385, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 119 2026-02-11T23:31:28.152+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8b38e8a9:::obj385:head, version: 111'385, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b38e8a9:::obj385:head@111'385, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.152+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.152+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 686 ==== pg_backfill(progress 4.0 e 119/119 lb 4:8b0915dc:::obj278:head) ==== 1062+0+0 (unknown 0 0 0) 0x4a2c000 con 0x507ac00 2026-02-11T23:31:28.152+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:8b0915dc:::obj278:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.152+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8b38e8a9:::obj385:head, version: 111'385, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b38e8a9:::obj385:head@111'385, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.152+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8b0915dc:::obj278:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.152+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.152+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8b38e8a9:::obj385:head, version: 111'385, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b38e8a9:::obj385:head@111'385, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.152+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.152+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8b38e8a9:::obj385:head, version: 111'385, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b38e8a9:::obj385:head@111'385, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(4.0 119/117 [PushOp(4:8b38e8a9:::obj385:head, version: 111'385, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b38e8a9:::obj385:head@111'385, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8aa18807:::obj353:head local-lis/les=117/118 n=322 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.152+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.152+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.152+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8b0915dc:::obj278:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.152+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8aa18807:::obj353:head local-lis/les=117/118 n=322 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:8b38e8a9:::obj385:head, version: 111'385, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b38e8a9:::obj385:head@111'385, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.152+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8aa18807:::obj353:head local-lis/les=117/118 n=322 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8b38e8a9:::obj385:head@111'385, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.152+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8aa18807:::obj353:head local-lis/les=117/118 n=322 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:8b38e8a9:::obj385:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.152+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8aa18807:::obj353:head local-lis/les=117/118 n=322 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:8b38e8a9:::obj385:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.152+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8aa18807:::obj353:head local-lis/les=117/118 n=322 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:8b38e8a9:::obj385:head 2026-02-11T23:31:28.152+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8aa18807:::obj353:head local-lis/les=117/118 n=322 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:8b38e8a9:::obj385:head v 111'385 2026-02-11T23:31:28.152+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8aa18807:::obj353:head local-lis/les=117/118 n=322 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.152+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.152+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8b38e8a9:::obj385:head, version: 111'385, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b38e8a9:::obj385:head@111'385, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000075 2026-02-11T23:31:28.152+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.152+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.152+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8b0915dc:::obj278:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.152+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8b0915dc:::obj278:head) v3 prio 15 cost 0 latency 0.000098 pg_backfill(progress 4.0 e 119/119 lb 4:8b0915dc:::obj278:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8aa18807:::obj353:head local-lis/les=117/118 n=322 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.152+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.152+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8aa18807:::obj353:head local-lis/les=117/118 n=322 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:8b0915dc:::obj278:head) v3 2026-02-11T23:31:28.152+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8aa18807:::obj353:head local-lis/les=117/118 n=322 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:8b0915dc:::obj278:head) 2026-02-11T23:31:28.152+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.152+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8b0915dc:::obj278:head) v3 finish latency 0.000017 2026-02-11T23:31:28.152+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8b0915dc:::obj278:head local-lis/les=117/118 n=323 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.152+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.152+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.152+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8b0915dc:::obj278:head local-lis/les=117/118 n=323 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.152+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.152+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.152+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.152+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x646cf00 con 0x507ac00 2026-02-11T23:31:28.152+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:8b38e8a9:::obj385:head)]) -- 0x642c180 con 0x507ac00 2026-02-11T23:31:28.152+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.153+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 687 ==== MOSDPGPush(4.0 119/117 [PushOp(4:8b716ab6:::obj324:head, version: 111'324, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b716ab6:::obj324:head@111'324, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3814d00 con 0x507ac00 2026-02-11T23:31:28.153+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:8b716ab6:::obj324:head, version: 111'324, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b716ab6:::obj324:head@111'324, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 119 2026-02-11T23:31:28.153+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8b716ab6:::obj324:head, version: 111'324, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b716ab6:::obj324:head@111'324, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.153+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.153+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 688 ==== pg_backfill(progress 4.0 e 119/119 lb 4:8b38e8a9:::obj385:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98580 con 0x507ac00 2026-02-11T23:31:28.153+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:8b38e8a9:::obj385:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.153+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8b38e8a9:::obj385:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.153+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8b716ab6:::obj324:head, version: 111'324, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b716ab6:::obj324:head@111'324, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.153+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.153+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8b716ab6:::obj324:head, version: 111'324, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b716ab6:::obj324:head@111'324, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.153+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8b716ab6:::obj324:head, version: 111'324, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b716ab6:::obj324:head@111'324, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(4.0 119/117 [PushOp(4:8b716ab6:::obj324:head, version: 111'324, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b716ab6:::obj324:head@111'324, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8b0915dc:::obj278:head local-lis/les=117/118 n=323 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.153+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.153+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.153+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8b38e8a9:::obj385:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.153+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8b0915dc:::obj278:head local-lis/les=117/118 n=323 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:8b716ab6:::obj324:head, version: 111'324, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b716ab6:::obj324:head@111'324, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.153+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8b0915dc:::obj278:head local-lis/les=117/118 n=323 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8b716ab6:::obj324:head@111'324, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.153+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8b0915dc:::obj278:head local-lis/les=117/118 n=323 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:8b716ab6:::obj324:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.153+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8b0915dc:::obj278:head local-lis/les=117/118 n=323 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:8b716ab6:::obj324:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.153+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8b0915dc:::obj278:head local-lis/les=117/118 n=323 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:8b716ab6:::obj324:head 2026-02-11T23:31:28.153+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8b0915dc:::obj278:head local-lis/les=117/118 n=323 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:8b716ab6:::obj324:head v 111'324 2026-02-11T23:31:28.153+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8b0915dc:::obj278:head local-lis/les=117/118 n=323 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.153+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.153+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8b716ab6:::obj324:head, version: 111'324, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b716ab6:::obj324:head@111'324, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:28.153+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8b0915dc:::obj278:head local-lis/les=117/118 n=323 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.153+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.153+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.153+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8b38e8a9:::obj385:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.153+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8b38e8a9:::obj385:head) v3 prio 15 cost 0 latency 0.000089 pg_backfill(progress 4.0 e 119/119 lb 4:8b38e8a9:::obj385:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8b0915dc:::obj278:head local-lis/les=117/118 n=323 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.153+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.153+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8b0915dc:::obj278:head local-lis/les=117/118 n=323 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:8b38e8a9:::obj385:head) v3 2026-02-11T23:31:28.153+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8b0915dc:::obj278:head local-lis/les=117/118 n=323 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:8b38e8a9:::obj385:head) 2026-02-11T23:31:28.153+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.153+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8b38e8a9:::obj385:head) v3 finish latency 0.000022 2026-02-11T23:31:28.153+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.153+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.153+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8b38e8a9:::obj385:head local-lis/les=117/118 n=324 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.153+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.153+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.153+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.153+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x640f800 con 0x507ac00 2026-02-11T23:31:28.153+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:8b716ab6:::obj324:head)]) -- 0x640f980 con 0x507ac00 2026-02-11T23:31:28.153+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.153+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.153+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.153+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 689 ==== MOSDPGPush(4.0 119/117 [PushOp(4:8b7a7bb6:::obj294:head, version: 111'294, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b7a7bb6:::obj294:head@111'294, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a3040 con 0x507ac00 2026-02-11T23:31:28.153+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:8b7a7bb6:::obj294:head, version: 111'294, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b7a7bb6:::obj294:head@111'294, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 119 2026-02-11T23:31:28.153+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8b7a7bb6:::obj294:head, version: 111'294, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b7a7bb6:::obj294:head@111'294, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.153+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.153+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 690 ==== pg_backfill(progress 4.0 e 119/119 lb 4:8b716ab6:::obj324:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9b700 con 0x507ac00 2026-02-11T23:31:28.153+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:8b716ab6:::obj324:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.153+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8b716ab6:::obj324:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.153+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8b7a7bb6:::obj294:head, version: 111'294, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b7a7bb6:::obj294:head@111'294, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.153+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.153+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8b7a7bb6:::obj294:head, version: 111'294, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b7a7bb6:::obj294:head@111'294, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.153+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8b7a7bb6:::obj294:head, version: 111'294, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b7a7bb6:::obj294:head@111'294, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(4.0 119/117 [PushOp(4:8b7a7bb6:::obj294:head, version: 111'294, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b7a7bb6:::obj294:head@111'294, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8b38e8a9:::obj385:head local-lis/les=117/118 n=324 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.153+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.153+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.153+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8b716ab6:::obj324:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.153+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8b38e8a9:::obj385:head local-lis/les=117/118 n=324 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:8b7a7bb6:::obj294:head, version: 111'294, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b7a7bb6:::obj294:head@111'294, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.153+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8b38e8a9:::obj385:head local-lis/les=117/118 n=324 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8b7a7bb6:::obj294:head@111'294, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.153+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8b38e8a9:::obj385:head local-lis/les=117/118 n=324 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:8b7a7bb6:::obj294:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.153+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8b38e8a9:::obj385:head local-lis/les=117/118 n=324 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:8b7a7bb6:::obj294:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.153+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8b38e8a9:::obj385:head local-lis/les=117/118 n=324 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:8b7a7bb6:::obj294:head 2026-02-11T23:31:28.153+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8b38e8a9:::obj385:head local-lis/les=117/118 n=324 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:8b7a7bb6:::obj294:head v 111'294 2026-02-11T23:31:28.153+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8b38e8a9:::obj385:head local-lis/les=117/118 n=324 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.153+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.153+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8b7a7bb6:::obj294:head, version: 111'294, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b7a7bb6:::obj294:head@111'294, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000071 2026-02-11T23:31:28.153+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.153+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.153+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8b716ab6:::obj324:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.153+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8b716ab6:::obj324:head) v3 prio 15 cost 0 latency 0.000094 pg_backfill(progress 4.0 e 119/119 lb 4:8b716ab6:::obj324:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8b38e8a9:::obj385:head local-lis/les=117/118 n=324 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.153+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.153+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8b38e8a9:::obj385:head local-lis/les=117/118 n=324 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:8b716ab6:::obj324:head) v3 2026-02-11T23:31:28.153+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8b38e8a9:::obj385:head local-lis/les=117/118 n=324 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:8b716ab6:::obj324:head) 2026-02-11T23:31:28.153+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.153+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8b716ab6:::obj324:head) v3 finish latency 0.000019 2026-02-11T23:31:28.153+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8b716ab6:::obj324:head local-lis/les=117/118 n=325 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.153+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.153+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.153+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8b716ab6:::obj324:head local-lis/les=117/118 n=325 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.153+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.153+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.153+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.153+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x640f380 con 0x507ac00 2026-02-11T23:31:28.153+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:8b7a7bb6:::obj294:head)]) -- 0x642c480 con 0x507ac00 2026-02-11T23:31:28.153+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.154+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 691 ==== MOSDPGPush(4.0 119/117 [PushOp(4:8c123081:::obj473:head, version: 111'473, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8c123081:::obj473:head@111'473, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f0ea0 con 0x507ac00 2026-02-11T23:31:28.154+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:8c123081:::obj473:head, version: 111'473, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8c123081:::obj473:head@111'473, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.154+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8c123081:::obj473:head, version: 111'473, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8c123081:::obj473:head@111'473, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.154+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.154+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 692 ==== pg_backfill(progress 4.0 e 119/119 lb 4:8b7a7bb6:::obj294:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f84000 con 0x507ac00 2026-02-11T23:31:28.154+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8c123081:::obj473:head, version: 111'473, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8c123081:::obj473:head@111'473, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.154+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:8b7a7bb6:::obj294:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 119 2026-02-11T23:31:28.154+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8b7a7bb6:::obj294:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.154+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.154+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8c123081:::obj473:head, version: 111'473, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8c123081:::obj473:head@111'473, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.154+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8c123081:::obj473:head, version: 111'473, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8c123081:::obj473:head@111'473, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(4.0 119/117 [PushOp(4:8c123081:::obj473:head, version: 111'473, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8c123081:::obj473:head@111'473, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8b716ab6:::obj324:head local-lis/les=117/118 n=325 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.154+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.154+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.154+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8b7a7bb6:::obj294:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.154+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8b716ab6:::obj324:head local-lis/les=117/118 n=325 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:8c123081:::obj473:head, version: 111'473, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8c123081:::obj473:head@111'473, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.154+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8b716ab6:::obj324:head local-lis/les=117/118 n=325 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8c123081:::obj473:head@111'473, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.154+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8b716ab6:::obj324:head local-lis/les=117/118 n=325 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:8c123081:::obj473:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.154+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8b716ab6:::obj324:head local-lis/les=117/118 n=325 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:8c123081:::obj473:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.154+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8b716ab6:::obj324:head local-lis/les=117/118 n=325 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:8c123081:::obj473:head 2026-02-11T23:31:28.154+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8b716ab6:::obj324:head local-lis/les=117/118 n=325 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:8c123081:::obj473:head v 111'473 2026-02-11T23:31:28.154+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8b716ab6:::obj324:head local-lis/les=117/118 n=325 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.154+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.154+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8c123081:::obj473:head, version: 111'473, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8c123081:::obj473:head@111'473, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000076 2026-02-11T23:31:28.154+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.154+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.154+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8b7a7bb6:::obj294:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.154+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8b7a7bb6:::obj294:head) v3 prio 15 cost 0 latency 0.000101 pg_backfill(progress 4.0 e 119/119 lb 4:8b7a7bb6:::obj294:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8b716ab6:::obj324:head local-lis/les=117/118 n=325 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.154+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.154+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8b716ab6:::obj324:head local-lis/les=117/118 n=325 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:8b7a7bb6:::obj294:head) v3 2026-02-11T23:31:28.154+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8b716ab6:::obj324:head local-lis/les=117/118 n=325 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:8b7a7bb6:::obj294:head) 2026-02-11T23:31:28.154+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.154+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8b7a7bb6:::obj294:head) v3 finish latency 0.000028 2026-02-11T23:31:28.154+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8b7a7bb6:::obj294:head local-lis/les=117/118 n=326 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.154+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.154+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.154+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8b7a7bb6:::obj294:head local-lis/les=117/118 n=326 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.154+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.154+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.154+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.154+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x640f680 con 0x507ac00 2026-02-11T23:31:28.154+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:8c123081:::obj473:head)]) -- 0x642c780 con 0x507ac00 2026-02-11T23:31:28.154+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.154+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.154+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.154+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 693 ==== MOSDPGPush(4.0 119/117 [PushOp(4:8c741e19:::obj599:head, version: 111'599, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8c741e19:::obj599:head@111'599, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2d00 con 0x507ac00 2026-02-11T23:31:28.154+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:8c741e19:::obj599:head, version: 111'599, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8c741e19:::obj599:head@111'599, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 119 2026-02-11T23:31:28.154+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8c741e19:::obj599:head, version: 111'599, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8c741e19:::obj599:head@111'599, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.155+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.155+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 694 ==== pg_backfill(progress 4.0 e 119/119 lb 4:8c123081:::obj473:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99080 con 0x507ac00 2026-02-11T23:31:28.155+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:8c123081:::obj473:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 119 2026-02-11T23:31:28.155+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8c123081:::obj473:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.155+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8c741e19:::obj599:head, version: 111'599, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8c741e19:::obj599:head@111'599, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.155+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.155+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8c741e19:::obj599:head, version: 111'599, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8c741e19:::obj599:head@111'599, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.155+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8c741e19:::obj599:head, version: 111'599, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8c741e19:::obj599:head@111'599, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(4.0 119/117 [PushOp(4:8c741e19:::obj599:head, version: 111'599, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8c741e19:::obj599:head@111'599, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8b7a7bb6:::obj294:head local-lis/les=117/118 n=326 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.155+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.155+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.155+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8c123081:::obj473:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.155+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8b7a7bb6:::obj294:head local-lis/les=117/118 n=326 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:8c741e19:::obj599:head, version: 111'599, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8c741e19:::obj599:head@111'599, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.155+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8b7a7bb6:::obj294:head local-lis/les=117/118 n=326 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8c741e19:::obj599:head@111'599, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.155+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8b7a7bb6:::obj294:head local-lis/les=117/118 n=326 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:8c741e19:::obj599:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.155+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8b7a7bb6:::obj294:head local-lis/les=117/118 n=326 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:8c741e19:::obj599:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.155+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8b7a7bb6:::obj294:head local-lis/les=117/118 n=326 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:8c741e19:::obj599:head 2026-02-11T23:31:28.155+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8b7a7bb6:::obj294:head local-lis/les=117/118 n=326 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:8c741e19:::obj599:head v 111'599 2026-02-11T23:31:28.155+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8b7a7bb6:::obj294:head local-lis/les=117/118 n=326 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.155+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.155+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8c741e19:::obj599:head, version: 111'599, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8c741e19:::obj599:head@111'599, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000073 2026-02-11T23:31:28.155+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8b7a7bb6:::obj294:head local-lis/les=117/118 n=326 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.155+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.155+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.155+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8c123081:::obj473:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.155+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8c123081:::obj473:head) v3 prio 15 cost 0 latency 0.000106 pg_backfill(progress 4.0 e 119/119 lb 4:8c123081:::obj473:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8b7a7bb6:::obj294:head local-lis/les=117/118 n=326 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.155+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.155+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8b7a7bb6:::obj294:head local-lis/les=117/118 n=326 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:8c123081:::obj473:head) v3 2026-02-11T23:31:28.155+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8b7a7bb6:::obj294:head local-lis/les=117/118 n=326 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:8c123081:::obj473:head) 2026-02-11T23:31:28.155+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.155+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8c123081:::obj473:head) v3 finish latency 0.000021 2026-02-11T23:31:28.155+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.155+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.155+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8c123081:::obj473:head local-lis/les=117/118 n=327 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.155+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.155+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.155+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.155+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x640ea80 con 0x507ac00 2026-02-11T23:31:28.155+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:8c741e19:::obj599:head)]) -- 0x640ef00 con 0x507ac00 2026-02-11T23:31:28.155+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.155+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 695 ==== MOSDPGPush(4.0 119/117 [PushOp(4:8ca4bb13:::obj342:head, version: 111'342, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8ca4bb13:::obj342:head@111'342, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56b64e0 con 0x507ac00 2026-02-11T23:31:28.155+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:8ca4bb13:::obj342:head, version: 111'342, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8ca4bb13:::obj342:head@111'342, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 119 2026-02-11T23:31:28.155+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8ca4bb13:::obj342:head, version: 111'342, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8ca4bb13:::obj342:head@111'342, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.155+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.155+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 696 ==== pg_backfill(progress 4.0 e 119/119 lb 4:8c741e19:::obj599:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176b00 con 0x507ac00 2026-02-11T23:31:28.155+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:8c741e19:::obj599:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.155+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8c741e19:::obj599:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.155+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8ca4bb13:::obj342:head, version: 111'342, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8ca4bb13:::obj342:head@111'342, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.155+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.155+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8ca4bb13:::obj342:head, version: 111'342, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8ca4bb13:::obj342:head@111'342, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.155+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8ca4bb13:::obj342:head, version: 111'342, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8ca4bb13:::obj342:head@111'342, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 119/117 [PushOp(4:8ca4bb13:::obj342:head, version: 111'342, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8ca4bb13:::obj342:head@111'342, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8c123081:::obj473:head local-lis/les=117/118 n=327 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.155+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.155+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.155+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8c741e19:::obj599:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.155+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8c123081:::obj473:head local-lis/les=117/118 n=327 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:8ca4bb13:::obj342:head, version: 111'342, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8ca4bb13:::obj342:head@111'342, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.155+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8c123081:::obj473:head local-lis/les=117/118 n=327 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8ca4bb13:::obj342:head@111'342, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.155+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8c123081:::obj473:head local-lis/les=117/118 n=327 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:8ca4bb13:::obj342:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.155+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8c123081:::obj473:head local-lis/les=117/118 n=327 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:8ca4bb13:::obj342:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.155+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8c123081:::obj473:head local-lis/les=117/118 n=327 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:8ca4bb13:::obj342:head 2026-02-11T23:31:28.155+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8c123081:::obj473:head local-lis/les=117/118 n=327 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:8ca4bb13:::obj342:head v 111'342 2026-02-11T23:31:28.155+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8c123081:::obj473:head local-lis/les=117/118 n=327 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.155+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.155+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8ca4bb13:::obj342:head, version: 111'342, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8ca4bb13:::obj342:head@111'342, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000070 2026-02-11T23:31:28.155+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.155+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.155+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8c741e19:::obj599:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.155+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8c741e19:::obj599:head) v3 prio 15 cost 0 latency 0.000094 pg_backfill(progress 4.0 e 119/119 lb 4:8c741e19:::obj599:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8c123081:::obj473:head local-lis/les=117/118 n=327 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.155+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.155+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8c123081:::obj473:head local-lis/les=117/118 n=327 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:8c741e19:::obj599:head) v3 2026-02-11T23:31:28.155+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8c123081:::obj473:head local-lis/les=117/118 n=327 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:8c741e19:::obj599:head) 2026-02-11T23:31:28.155+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.155+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8c741e19:::obj599:head) v3 finish latency 0.000023 2026-02-11T23:31:28.155+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8c741e19:::obj599:head local-lis/les=117/118 n=328 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.155+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.155+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.155+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8c741e19:::obj599:head local-lis/les=117/118 n=328 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.155+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.155+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.155+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.155+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x640ed80 con 0x507ac00 2026-02-11T23:31:28.155+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:8ca4bb13:::obj342:head)]) -- 0x642ca80 con 0x507ac00 2026-02-11T23:31:28.155+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.156+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 697 ==== MOSDPGPush(4.0 119/117 [PushOp(4:8cdc434f:::obj77:head, version: 111'77, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8cdc434f:::obj77:head@111'77, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x56caea0 con 0x507ac00 2026-02-11T23:31:28.156+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:8cdc434f:::obj77:head, version: 111'77, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8cdc434f:::obj77:head@111'77, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 119 2026-02-11T23:31:28.156+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8cdc434f:::obj77:head, version: 111'77, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8cdc434f:::obj77:head@111'77, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.156+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.156+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 698 ==== pg_backfill(progress 4.0 e 119/119 lb 4:8ca4bb13:::obj342:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99600 con 0x507ac00 2026-02-11T23:31:28.156+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:8ca4bb13:::obj342:head) v3 prio 15 type 95 cost 0 latency 0.000002 epoch 119 2026-02-11T23:31:28.156+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8cdc434f:::obj77:head, version: 111'77, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8cdc434f:::obj77:head@111'77, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.156+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8ca4bb13:::obj342:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.156+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.156+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8cdc434f:::obj77:head, version: 111'77, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8cdc434f:::obj77:head@111'77, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.156+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8cdc434f:::obj77:head, version: 111'77, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8cdc434f:::obj77:head@111'77, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 119/117 [PushOp(4:8cdc434f:::obj77:head, version: 111'77, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8cdc434f:::obj77:head@111'77, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8c741e19:::obj599:head local-lis/les=117/118 n=328 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.156+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.156+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.156+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8c741e19:::obj599:head local-lis/les=117/118 n=328 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:8cdc434f:::obj77:head, version: 111'77, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8cdc434f:::obj77:head@111'77, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.156+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8ca4bb13:::obj342:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.156+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8c741e19:::obj599:head local-lis/les=117/118 n=328 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8cdc434f:::obj77:head@111'77, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.156+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8c741e19:::obj599:head local-lis/les=117/118 n=328 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:8cdc434f:::obj77:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.156+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8c741e19:::obj599:head local-lis/les=117/118 n=328 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:8cdc434f:::obj77:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.156+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8c741e19:::obj599:head local-lis/les=117/118 n=328 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:8cdc434f:::obj77:head 2026-02-11T23:31:28.156+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8c741e19:::obj599:head local-lis/les=117/118 n=328 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:8cdc434f:::obj77:head v 111'77 2026-02-11T23:31:28.156+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8c741e19:::obj599:head local-lis/les=117/118 n=328 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.156+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.156+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8cdc434f:::obj77:head, version: 111'77, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8cdc434f:::obj77:head@111'77, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:28.156+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8c741e19:::obj599:head local-lis/les=117/118 n=328 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.156+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.156+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.156+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8ca4bb13:::obj342:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.156+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8ca4bb13:::obj342:head) v3 prio 15 cost 0 latency 0.000082 pg_backfill(progress 4.0 e 119/119 lb 4:8ca4bb13:::obj342:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8c741e19:::obj599:head local-lis/les=117/118 n=328 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.156+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.156+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8c741e19:::obj599:head local-lis/les=117/118 n=328 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:8ca4bb13:::obj342:head) v3 2026-02-11T23:31:28.156+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8c741e19:::obj599:head local-lis/les=117/118 n=328 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:8ca4bb13:::obj342:head) 2026-02-11T23:31:28.156+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.156+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8ca4bb13:::obj342:head) v3 finish latency 0.000023 2026-02-11T23:31:28.156+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.156+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.156+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8ca4bb13:::obj342:head local-lis/les=117/118 n=329 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.156+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.156+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.156+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.156+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x640e900 con 0x507ac00 2026-02-11T23:31:28.156+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:8cdc434f:::obj77:head)]) -- 0x646d500 con 0x507ac00 2026-02-11T23:31:28.156+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.156+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.156+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.156+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 699 ==== MOSDPGPush(4.0 119/117 [PushOp(4:8d29890f:::obj305:head, version: 111'305, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d29890f:::obj305:head@111'305, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db0000 con 0x507ac00 2026-02-11T23:31:28.156+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:8d29890f:::obj305:head, version: 111'305, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d29890f:::obj305:head@111'305, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 119 2026-02-11T23:31:28.156+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8d29890f:::obj305:head, version: 111'305, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d29890f:::obj305:head@111'305, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.156+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.156+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 700 ==== pg_backfill(progress 4.0 e 119/119 lb 4:8cdc434f:::obj77:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d2d600 con 0x507ac00 2026-02-11T23:31:28.156+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8d29890f:::obj305:head, version: 111'305, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d29890f:::obj305:head@111'305, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.156+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:8cdc434f:::obj77:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 119 2026-02-11T23:31:28.156+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.156+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8cdc434f:::obj77:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.156+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8d29890f:::obj305:head, version: 111'305, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d29890f:::obj305:head@111'305, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.156+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.156+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8d29890f:::obj305:head, version: 111'305, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d29890f:::obj305:head@111'305, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 119/117 [PushOp(4:8d29890f:::obj305:head, version: 111'305, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d29890f:::obj305:head@111'305, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8ca4bb13:::obj342:head local-lis/les=117/118 n=329 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.156+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8cdc434f:::obj77:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.156+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.156+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8ca4bb13:::obj342:head local-lis/les=117/118 n=329 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:8d29890f:::obj305:head, version: 111'305, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d29890f:::obj305:head@111'305, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.156+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8ca4bb13:::obj342:head local-lis/les=117/118 n=329 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8d29890f:::obj305:head@111'305, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.156+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8ca4bb13:::obj342:head local-lis/les=117/118 n=329 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:8d29890f:::obj305:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.156+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8ca4bb13:::obj342:head local-lis/les=117/118 n=329 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:8d29890f:::obj305:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.156+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8ca4bb13:::obj342:head local-lis/les=117/118 n=329 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:8d29890f:::obj305:head 2026-02-11T23:31:28.156+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8ca4bb13:::obj342:head local-lis/les=117/118 n=329 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:8d29890f:::obj305:head v 111'305 2026-02-11T23:31:28.156+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8ca4bb13:::obj342:head local-lis/les=117/118 n=329 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.156+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.156+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8d29890f:::obj305:head, version: 111'305, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d29890f:::obj305:head@111'305, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:28.156+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8ca4bb13:::obj342:head local-lis/les=117/118 n=329 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.156+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.156+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.156+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8cdc434f:::obj77:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.156+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8cdc434f:::obj77:head) v3 prio 15 cost 0 latency 0.000080 pg_backfill(progress 4.0 e 119/119 lb 4:8cdc434f:::obj77:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8ca4bb13:::obj342:head local-lis/les=117/118 n=329 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.156+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.156+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8ca4bb13:::obj342:head local-lis/les=117/118 n=329 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:8cdc434f:::obj77:head) v3 2026-02-11T23:31:28.156+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8ca4bb13:::obj342:head local-lis/les=117/118 n=329 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:8cdc434f:::obj77:head) 2026-02-11T23:31:28.156+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.156+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8cdc434f:::obj77:head) v3 finish latency 0.000016 2026-02-11T23:31:28.156+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.157+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.157+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8cdc434f:::obj77:head local-lis/les=117/118 n=330 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.157+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.157+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.157+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.157+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c9e00 con 0x507ac00 2026-02-11T23:31:28.157+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:8d29890f:::obj305:head)]) -- 0x5d55800 con 0x507ac00 2026-02-11T23:31:28.157+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.157+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 701 ==== MOSDPGPush(4.0 119/117 [PushOp(4:8d384ce8:::obj297:head, version: 111'297, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d384ce8:::obj297:head@111'297, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a3a00 con 0x507ac00 2026-02-11T23:31:28.157+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:8d384ce8:::obj297:head, version: 111'297, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d384ce8:::obj297:head@111'297, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 119 2026-02-11T23:31:28.157+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8d384ce8:::obj297:head, version: 111'297, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d384ce8:::obj297:head@111'297, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.157+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.157+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 702 ==== pg_backfill(progress 4.0 e 119/119 lb 4:8d29890f:::obj305:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2ec00 con 0x507ac00 2026-02-11T23:31:28.157+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8d384ce8:::obj297:head, version: 111'297, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d384ce8:::obj297:head@111'297, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.157+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:8d29890f:::obj305:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 119 2026-02-11T23:31:28.157+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8d29890f:::obj305:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.157+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.157+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8d384ce8:::obj297:head, version: 111'297, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d384ce8:::obj297:head@111'297, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.157+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.157+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8d384ce8:::obj297:head, version: 111'297, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d384ce8:::obj297:head@111'297, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 119/117 [PushOp(4:8d384ce8:::obj297:head, version: 111'297, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d384ce8:::obj297:head@111'297, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8cdc434f:::obj77:head local-lis/les=117/118 n=330 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.157+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.157+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8cdc434f:::obj77:head local-lis/les=117/118 n=330 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:8d384ce8:::obj297:head, version: 111'297, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d384ce8:::obj297:head@111'297, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.157+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8cdc434f:::obj77:head local-lis/les=117/118 n=330 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8d384ce8:::obj297:head@111'297, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.157+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.157+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8d29890f:::obj305:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.157+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8cdc434f:::obj77:head local-lis/les=117/118 n=330 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:8d384ce8:::obj297:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.157+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8cdc434f:::obj77:head local-lis/les=117/118 n=330 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:8d384ce8:::obj297:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.157+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8cdc434f:::obj77:head local-lis/les=117/118 n=330 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:8d384ce8:::obj297:head 2026-02-11T23:31:28.157+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8cdc434f:::obj77:head local-lis/les=117/118 n=330 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:8d384ce8:::obj297:head v 111'297 2026-02-11T23:31:28.157+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8cdc434f:::obj77:head local-lis/les=117/118 n=330 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.157+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.157+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8d384ce8:::obj297:head, version: 111'297, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d384ce8:::obj297:head@111'297, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:28.157+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8cdc434f:::obj77:head local-lis/les=117/118 n=330 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.157+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.157+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.157+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8d29890f:::obj305:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.157+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8d29890f:::obj305:head) v3 prio 15 cost 0 latency 0.000082 pg_backfill(progress 4.0 e 119/119 lb 4:8d29890f:::obj305:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8cdc434f:::obj77:head local-lis/les=117/118 n=330 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.157+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.157+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8cdc434f:::obj77:head local-lis/les=117/118 n=330 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:8d29890f:::obj305:head) v3 2026-02-11T23:31:28.157+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8cdc434f:::obj77:head local-lis/les=117/118 n=330 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:8d29890f:::obj305:head) 2026-02-11T23:31:28.157+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.157+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8d29890f:::obj305:head) v3 finish latency 0.000015 2026-02-11T23:31:28.157+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.157+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.157+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8d29890f:::obj305:head local-lis/les=117/118 n=331 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.157+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.157+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.157+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.157+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x640e480 con 0x507ac00 2026-02-11T23:31:28.157+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:8d384ce8:::obj297:head)]) -- 0x640e600 con 0x507ac00 2026-02-11T23:31:28.157+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.158+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 703 ==== MOSDPGPush(4.0 119/117 [PushOp(4:8d6bd12e:::obj368:head, version: 111'368, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d6bd12e:::obj368:head@111'368, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2820 con 0x507ac00 2026-02-11T23:31:28.158+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:8d6bd12e:::obj368:head, version: 111'368, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d6bd12e:::obj368:head@111'368, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 119 2026-02-11T23:31:28.158+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8d6bd12e:::obj368:head, version: 111'368, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d6bd12e:::obj368:head@111'368, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.158+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.158+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 704 ==== pg_backfill(progress 4.0 e 119/119 lb 4:8d384ce8:::obj297:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98000 con 0x507ac00 2026-02-11T23:31:28.158+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:8d384ce8:::obj297:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.158+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8d384ce8:::obj297:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.158+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8d6bd12e:::obj368:head, version: 111'368, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d6bd12e:::obj368:head@111'368, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.158+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.158+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8d6bd12e:::obj368:head, version: 111'368, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d6bd12e:::obj368:head@111'368, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.158+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8d6bd12e:::obj368:head, version: 111'368, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d6bd12e:::obj368:head@111'368, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 119/117 [PushOp(4:8d6bd12e:::obj368:head, version: 111'368, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d6bd12e:::obj368:head@111'368, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8d29890f:::obj305:head local-lis/les=117/118 n=331 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.158+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.158+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.158+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8d384ce8:::obj297:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.158+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8d29890f:::obj305:head local-lis/les=117/118 n=331 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:8d6bd12e:::obj368:head, version: 111'368, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d6bd12e:::obj368:head@111'368, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.158+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8d29890f:::obj305:head local-lis/les=117/118 n=331 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8d6bd12e:::obj368:head@111'368, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.158+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8d29890f:::obj305:head local-lis/les=117/118 n=331 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:8d6bd12e:::obj368:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.158+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8d29890f:::obj305:head local-lis/les=117/118 n=331 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:8d6bd12e:::obj368:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.158+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8d29890f:::obj305:head local-lis/les=117/118 n=331 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:8d6bd12e:::obj368:head 2026-02-11T23:31:28.158+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8d29890f:::obj305:head local-lis/les=117/118 n=331 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:8d6bd12e:::obj368:head v 111'368 2026-02-11T23:31:28.158+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8d29890f:::obj305:head local-lis/les=117/118 n=331 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.158+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.158+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8d6bd12e:::obj368:head, version: 111'368, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d6bd12e:::obj368:head@111'368, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:28.158+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8d29890f:::obj305:head local-lis/les=117/118 n=331 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.158+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.158+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.158+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8d384ce8:::obj297:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.158+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8d384ce8:::obj297:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 4.0 e 119/119 lb 4:8d384ce8:::obj297:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8d29890f:::obj305:head local-lis/les=117/118 n=331 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.158+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.158+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8d29890f:::obj305:head local-lis/les=117/118 n=331 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:8d384ce8:::obj297:head) v3 2026-02-11T23:31:28.158+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8d29890f:::obj305:head local-lis/les=117/118 n=331 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:8d384ce8:::obj297:head) 2026-02-11T23:31:28.158+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.158+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8d384ce8:::obj297:head) v3 finish latency 0.000016 2026-02-11T23:31:28.158+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.158+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.158+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8d384ce8:::obj297:head local-lis/les=117/118 n=332 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.158+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.158+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.158+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.158+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x640fc80 con 0x507ac00 2026-02-11T23:31:28.158+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:8d6bd12e:::obj368:head)]) -- 0x642c180 con 0x507ac00 2026-02-11T23:31:28.158+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.158+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 705 ==== MOSDPGPush(4.0 119/117 [PushOp(4:8dd16f86:::obj3:head, version: 111'3, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8dd16f86:::obj3:head@111'3, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 4999+0+0 (unknown 0 0 0) 0x554e4e0 con 0x507ac00 2026-02-11T23:31:28.158+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:8dd16f86:::obj3:head, version: 111'3, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8dd16f86:::obj3:head@111'3, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 119 2026-02-11T23:31:28.158+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8dd16f86:::obj3:head, version: 111'3, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8dd16f86:::obj3:head@111'3, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.158+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.158+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 706 ==== pg_backfill(progress 4.0 e 119/119 lb 4:8d6bd12e:::obj368:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9b180 con 0x507ac00 2026-02-11T23:31:28.158+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8dd16f86:::obj3:head, version: 111'3, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8dd16f86:::obj3:head@111'3, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.158+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:8d6bd12e:::obj368:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 119 2026-02-11T23:31:28.158+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8d6bd12e:::obj368:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.158+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.158+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8dd16f86:::obj3:head, version: 111'3, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8dd16f86:::obj3:head@111'3, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.158+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.158+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8dd16f86:::obj3:head, version: 111'3, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8dd16f86:::obj3:head@111'3, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 119/117 [PushOp(4:8dd16f86:::obj3:head, version: 111'3, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8dd16f86:::obj3:head@111'3, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8d384ce8:::obj297:head local-lis/les=117/118 n=332 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.158+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.158+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.158+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8d6bd12e:::obj368:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.158+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8d384ce8:::obj297:head local-lis/les=117/118 n=332 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:8dd16f86:::obj3:head, version: 111'3, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8dd16f86:::obj3:head@111'3, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.158+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8d384ce8:::obj297:head local-lis/les=117/118 n=332 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8dd16f86:::obj3:head@111'3, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.158+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8d384ce8:::obj297:head local-lis/les=117/118 n=332 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:8dd16f86:::obj3:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.158+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8d384ce8:::obj297:head local-lis/les=117/118 n=332 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:8dd16f86:::obj3:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.158+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8d384ce8:::obj297:head local-lis/les=117/118 n=332 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:8dd16f86:::obj3:head 2026-02-11T23:31:28.158+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8d384ce8:::obj297:head local-lis/les=117/118 n=332 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:8dd16f86:::obj3:head v 111'3 2026-02-11T23:31:28.158+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8d384ce8:::obj297:head local-lis/les=117/118 n=332 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.158+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.158+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8dd16f86:::obj3:head, version: 111'3, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8dd16f86:::obj3:head@111'3, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000069 2026-02-11T23:31:28.158+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.158+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.158+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8d6bd12e:::obj368:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.158+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8d6bd12e:::obj368:head) v3 prio 15 cost 0 latency 0.000095 pg_backfill(progress 4.0 e 119/119 lb 4:8d6bd12e:::obj368:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8d384ce8:::obj297:head local-lis/les=117/118 n=332 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.158+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.158+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8d384ce8:::obj297:head local-lis/les=117/118 n=332 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:8d6bd12e:::obj368:head) v3 2026-02-11T23:31:28.158+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8d384ce8:::obj297:head local-lis/les=117/118 n=332 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:8d6bd12e:::obj368:head) 2026-02-11T23:31:28.158+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.158+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8d6bd12e:::obj368:head) v3 finish latency 0.000021 2026-02-11T23:31:28.158+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8d6bd12e:::obj368:head local-lis/les=117/118 n=333 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.158+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.158+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.158+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8d6bd12e:::obj368:head local-lis/les=117/118 n=333 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.158+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.158+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.158+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.158+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x640fb00 con 0x507ac00 2026-02-11T23:31:28.158+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:8dd16f86:::obj3:head)]) -- 0x642cd80 con 0x507ac00 2026-02-11T23:31:28.158+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.159+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 707 ==== MOSDPGPush(4.0 119/117 [PushOp(4:8e988874:::obj200:head, version: 111'200, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8e988874:::obj200:head@111'200, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f1d40 con 0x507ac00 2026-02-11T23:31:28.159+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:8e988874:::obj200:head, version: 111'200, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8e988874:::obj200:head@111'200, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 119 2026-02-11T23:31:28.159+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8e988874:::obj200:head, version: 111'200, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8e988874:::obj200:head@111'200, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.159+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.159+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 708 ==== pg_backfill(progress 4.0 e 119/119 lb 4:8dd16f86:::obj3:head) ==== 1060+0+0 (unknown 0 0 0) 0x6179180 con 0x507ac00 2026-02-11T23:31:28.159+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:8dd16f86:::obj3:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.159+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8e988874:::obj200:head, version: 111'200, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8e988874:::obj200:head@111'200, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.159+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.159+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8dd16f86:::obj3:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.159+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8e988874:::obj200:head, version: 111'200, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8e988874:::obj200:head@111'200, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.159+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.159+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8e988874:::obj200:head, version: 111'200, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8e988874:::obj200:head@111'200, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(4.0 119/117 [PushOp(4:8e988874:::obj200:head, version: 111'200, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8e988874:::obj200:head@111'200, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8d6bd12e:::obj368:head local-lis/les=117/118 n=333 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.159+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.159+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8d6bd12e:::obj368:head local-lis/les=117/118 n=333 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:8e988874:::obj200:head, version: 111'200, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8e988874:::obj200:head@111'200, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.159+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8d6bd12e:::obj368:head local-lis/les=117/118 n=333 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8e988874:::obj200:head@111'200, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.159+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.159+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8dd16f86:::obj3:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.159+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8d6bd12e:::obj368:head local-lis/les=117/118 n=333 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:8e988874:::obj200:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.159+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8d6bd12e:::obj368:head local-lis/les=117/118 n=333 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:8e988874:::obj200:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.159+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8d6bd12e:::obj368:head local-lis/les=117/118 n=333 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:8e988874:::obj200:head 2026-02-11T23:31:28.159+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8d6bd12e:::obj368:head local-lis/les=117/118 n=333 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:8e988874:::obj200:head v 111'200 2026-02-11T23:31:28.159+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8d6bd12e:::obj368:head local-lis/les=117/118 n=333 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.159+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.159+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8e988874:::obj200:head, version: 111'200, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8e988874:::obj200:head@111'200, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:28.159+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8d6bd12e:::obj368:head local-lis/les=117/118 n=333 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.159+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.159+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.159+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8dd16f86:::obj3:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.159+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8dd16f86:::obj3:head) v3 prio 15 cost 0 latency 0.000085 pg_backfill(progress 4.0 e 119/119 lb 4:8dd16f86:::obj3:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8d6bd12e:::obj368:head local-lis/les=117/118 n=333 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.159+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.159+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8d6bd12e:::obj368:head local-lis/les=117/118 n=333 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:8dd16f86:::obj3:head) v3 2026-02-11T23:31:28.159+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8d6bd12e:::obj368:head local-lis/les=117/118 n=333 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:8dd16f86:::obj3:head) 2026-02-11T23:31:28.159+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.159+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8dd16f86:::obj3:head) v3 finish latency 0.000020 2026-02-11T23:31:28.159+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.159+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.159+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8dd16f86:::obj3:head local-lis/les=117/118 n=334 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.159+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.159+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.159+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.159+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c8c00 con 0x507ac00 2026-02-11T23:31:28.159+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:8e988874:::obj200:head)]) -- 0x5d55200 con 0x507ac00 2026-02-11T23:31:28.159+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.160+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 709 ==== MOSDPGPush(4.0 119/117 [PushOp(4:8eb14ea4:::obj384:head, version: 111'384, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8eb14ea4:::obj384:head@111'384, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2000 con 0x507ac00 2026-02-11T23:31:28.160+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:8eb14ea4:::obj384:head, version: 111'384, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8eb14ea4:::obj384:head@111'384, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000017 epoch 119 2026-02-11T23:31:28.160+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8eb14ea4:::obj384:head, version: 111'384, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8eb14ea4:::obj384:head@111'384, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.160+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.160+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 710 ==== pg_backfill(progress 4.0 e 119/119 lb 4:8e988874:::obj200:head) ==== 1062+0+0 (unknown 0 0 0) 0x6178100 con 0x507ac00 2026-02-11T23:31:28.160+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:8e988874:::obj200:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.160+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8eb14ea4:::obj384:head, version: 111'384, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8eb14ea4:::obj384:head@111'384, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.160+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8e988874:::obj200:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.160+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.160+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8eb14ea4:::obj384:head, version: 111'384, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8eb14ea4:::obj384:head@111'384, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.160+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.160+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8e988874:::obj200:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.160+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8eb14ea4:::obj384:head, version: 111'384, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8eb14ea4:::obj384:head@111'384, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(4.0 119/117 [PushOp(4:8eb14ea4:::obj384:head, version: 111'384, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8eb14ea4:::obj384:head@111'384, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8dd16f86:::obj3:head local-lis/les=117/118 n=334 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.160+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.160+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8dd16f86:::obj3:head local-lis/les=117/118 n=334 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:8eb14ea4:::obj384:head, version: 111'384, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8eb14ea4:::obj384:head@111'384, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.160+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8dd16f86:::obj3:head local-lis/les=117/118 n=334 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8eb14ea4:::obj384:head@111'384, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.160+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8dd16f86:::obj3:head local-lis/les=117/118 n=334 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:8eb14ea4:::obj384:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.160+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8dd16f86:::obj3:head local-lis/les=117/118 n=334 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:8eb14ea4:::obj384:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.160+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8dd16f86:::obj3:head local-lis/les=117/118 n=334 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:8eb14ea4:::obj384:head 2026-02-11T23:31:28.160+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8dd16f86:::obj3:head local-lis/les=117/118 n=334 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:8eb14ea4:::obj384:head v 111'384 2026-02-11T23:31:28.160+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8dd16f86:::obj3:head local-lis/les=117/118 n=334 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.160+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.160+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8eb14ea4:::obj384:head, version: 111'384, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8eb14ea4:::obj384:head@111'384, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000081 2026-02-11T23:31:28.160+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.160+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.160+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8e988874:::obj200:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.160+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8e988874:::obj200:head) v3 prio 15 cost 0 latency 0.000103 pg_backfill(progress 4.0 e 119/119 lb 4:8e988874:::obj200:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8dd16f86:::obj3:head local-lis/les=117/118 n=334 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.160+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.160+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8dd16f86:::obj3:head local-lis/les=117/118 n=334 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:8e988874:::obj200:head) v3 2026-02-11T23:31:28.160+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8dd16f86:::obj3:head local-lis/les=117/118 n=334 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:8e988874:::obj200:head) 2026-02-11T23:31:28.160+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.160+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8e988874:::obj200:head) v3 finish latency 0.000016 2026-02-11T23:31:28.160+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8e988874:::obj200:head local-lis/les=117/118 n=335 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.160+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.160+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.160+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8e988874:::obj200:head local-lis/les=117/118 n=335 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.160+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.160+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.160+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.160+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x640fe00 con 0x507ac00 2026-02-11T23:31:28.160+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:8eb14ea4:::obj384:head)]) -- 0x642cf00 con 0x507ac00 2026-02-11T23:31:28.160+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.160+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 711 ==== MOSDPGPush(4.0 119/117 [PushOp(4:8f4c4fa9:::obj590:head, version: 111'590, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8f4c4fa9:::obj590:head@111'590, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56aa000 con 0x507ac00 2026-02-11T23:31:28.160+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:8f4c4fa9:::obj590:head, version: 111'590, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8f4c4fa9:::obj590:head@111'590, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 119 2026-02-11T23:31:28.160+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8f4c4fa9:::obj590:head, version: 111'590, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8f4c4fa9:::obj590:head@111'590, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.160+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.160+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8f4c4fa9:::obj590:head, version: 111'590, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8f4c4fa9:::obj590:head@111'590, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.160+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 712 ==== pg_backfill(progress 4.0 e 119/119 lb 4:8eb14ea4:::obj384:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2c580 con 0x507ac00 2026-02-11T23:31:28.160+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.160+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:8eb14ea4:::obj384:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.160+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8f4c4fa9:::obj590:head, version: 111'590, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8f4c4fa9:::obj590:head@111'590, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.160+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8eb14ea4:::obj384:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.160+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8f4c4fa9:::obj590:head, version: 111'590, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8f4c4fa9:::obj590:head@111'590, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(4.0 119/117 [PushOp(4:8f4c4fa9:::obj590:head, version: 111'590, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8f4c4fa9:::obj590:head@111'590, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8e988874:::obj200:head local-lis/les=117/118 n=335 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.160+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.160+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.160+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8e988874:::obj200:head local-lis/les=117/118 n=335 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:8f4c4fa9:::obj590:head, version: 111'590, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8f4c4fa9:::obj590:head@111'590, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.160+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8eb14ea4:::obj384:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.160+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8e988874:::obj200:head local-lis/les=117/118 n=335 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8f4c4fa9:::obj590:head@111'590, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.160+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8e988874:::obj200:head local-lis/les=117/118 n=335 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:8f4c4fa9:::obj590:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.160+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8e988874:::obj200:head local-lis/les=117/118 n=335 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:8f4c4fa9:::obj590:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.160+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8e988874:::obj200:head local-lis/les=117/118 n=335 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:8f4c4fa9:::obj590:head 2026-02-11T23:31:28.160+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8e988874:::obj200:head local-lis/les=117/118 n=335 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:8f4c4fa9:::obj590:head v 111'590 2026-02-11T23:31:28.160+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8e988874:::obj200:head local-lis/les=117/118 n=335 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.160+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.160+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8f4c4fa9:::obj590:head, version: 111'590, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8f4c4fa9:::obj590:head@111'590, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:28.160+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8e988874:::obj200:head local-lis/les=117/118 n=335 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.160+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.160+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.160+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8eb14ea4:::obj384:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.160+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8eb14ea4:::obj384:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 4.0 e 119/119 lb 4:8eb14ea4:::obj384:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8e988874:::obj200:head local-lis/les=117/118 n=335 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.160+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.160+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8e988874:::obj200:head local-lis/les=117/118 n=335 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:8eb14ea4:::obj384:head) v3 2026-02-11T23:31:28.160+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8e988874:::obj200:head local-lis/les=117/118 n=335 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:8eb14ea4:::obj384:head) 2026-02-11T23:31:28.160+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.160+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8eb14ea4:::obj384:head) v3 finish latency 0.000021 2026-02-11T23:31:28.160+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.160+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.160+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8eb14ea4:::obj384:head local-lis/les=117/118 n=336 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.160+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.160+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.160+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.160+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x644d380 con 0x507ac00 2026-02-11T23:31:28.160+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:8f4c4fa9:::obj590:head)]) -- 0x640e180 con 0x507ac00 2026-02-11T23:31:28.160+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.161+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 713 ==== MOSDPGPush(4.0 119/117 [PushOp(4:8f7cdb2a:::obj538:head, version: 111'538, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8f7cdb2a:::obj538:head@111'538, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x555d040 con 0x507ac00 2026-02-11T23:31:28.161+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:8f7cdb2a:::obj538:head, version: 111'538, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8f7cdb2a:::obj538:head@111'538, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.161+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8f7cdb2a:::obj538:head, version: 111'538, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8f7cdb2a:::obj538:head@111'538, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.161+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.161+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 714 ==== pg_backfill(progress 4.0 e 119/119 lb 4:8f4c4fa9:::obj590:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177b80 con 0x507ac00 2026-02-11T23:31:28.161+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8f7cdb2a:::obj538:head, version: 111'538, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8f7cdb2a:::obj538:head@111'538, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.161+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:8f4c4fa9:::obj590:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.161+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.161+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8f4c4fa9:::obj590:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.161+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8f7cdb2a:::obj538:head, version: 111'538, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8f7cdb2a:::obj538:head@111'538, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.161+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8f7cdb2a:::obj538:head, version: 111'538, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8f7cdb2a:::obj538:head@111'538, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 119/117 [PushOp(4:8f7cdb2a:::obj538:head, version: 111'538, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8f7cdb2a:::obj538:head@111'538, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8eb14ea4:::obj384:head local-lis/les=117/118 n=336 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.161+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.161+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8eb14ea4:::obj384:head local-lis/les=117/118 n=336 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:8f7cdb2a:::obj538:head, version: 111'538, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8f7cdb2a:::obj538:head@111'538, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.161+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.161+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8eb14ea4:::obj384:head local-lis/les=117/118 n=336 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8f7cdb2a:::obj538:head@111'538, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.161+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8f4c4fa9:::obj590:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.161+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8eb14ea4:::obj384:head local-lis/les=117/118 n=336 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:8f7cdb2a:::obj538:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.161+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8eb14ea4:::obj384:head local-lis/les=117/118 n=336 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:8f7cdb2a:::obj538:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.161+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8eb14ea4:::obj384:head local-lis/les=117/118 n=336 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:8f7cdb2a:::obj538:head 2026-02-11T23:31:28.161+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8eb14ea4:::obj384:head local-lis/les=117/118 n=336 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:8f7cdb2a:::obj538:head v 111'538 2026-02-11T23:31:28.161+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8eb14ea4:::obj384:head local-lis/les=117/118 n=336 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.161+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.161+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8f7cdb2a:::obj538:head, version: 111'538, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8f7cdb2a:::obj538:head@111'538, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:28.161+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8eb14ea4:::obj384:head local-lis/les=117/118 n=336 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.161+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.161+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.161+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8f4c4fa9:::obj590:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.161+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8f4c4fa9:::obj590:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 4.0 e 119/119 lb 4:8f4c4fa9:::obj590:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8eb14ea4:::obj384:head local-lis/les=117/118 n=336 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.161+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.161+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8eb14ea4:::obj384:head local-lis/les=117/118 n=336 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:8f4c4fa9:::obj590:head) v3 2026-02-11T23:31:28.161+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8eb14ea4:::obj384:head local-lis/les=117/118 n=336 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:8f4c4fa9:::obj590:head) 2026-02-11T23:31:28.161+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.161+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8f4c4fa9:::obj590:head) v3 finish latency 0.000015 2026-02-11T23:31:28.161+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.161+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.161+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8f4c4fa9:::obj590:head local-lis/les=117/118 n=337 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.161+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.161+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.161+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.161+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x644d800 con 0x507ac00 2026-02-11T23:31:28.161+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:8f7cdb2a:::obj538:head)]) -- 0x4f7b080 con 0x507ac00 2026-02-11T23:31:28.161+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.161+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 715 ==== MOSDPGPush(4.0 119/117 [PushOp(4:8fcfcc34:::obj221:head, version: 111'221, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8fcfcc34:::obj221:head@111'221, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x554e680 con 0x507ac00 2026-02-11T23:31:28.161+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:8fcfcc34:::obj221:head, version: 111'221, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8fcfcc34:::obj221:head@111'221, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.161+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8fcfcc34:::obj221:head, version: 111'221, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8fcfcc34:::obj221:head@111'221, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.161+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.161+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 716 ==== pg_backfill(progress 4.0 e 119/119 lb 4:8f7cdb2a:::obj538:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177600 con 0x507ac00 2026-02-11T23:31:28.161+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:8f7cdb2a:::obj538:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.161+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8fcfcc34:::obj221:head, version: 111'221, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8fcfcc34:::obj221:head@111'221, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.161+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.161+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:8fcfcc34:::obj221:head, version: 111'221, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8fcfcc34:::obj221:head@111'221, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.161+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8f7cdb2a:::obj538:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.161+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.161+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8fcfcc34:::obj221:head, version: 111'221, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8fcfcc34:::obj221:head@111'221, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(4.0 119/117 [PushOp(4:8fcfcc34:::obj221:head, version: 111'221, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8fcfcc34:::obj221:head@111'221, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8f4c4fa9:::obj590:head local-lis/les=117/118 n=337 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.161+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.161+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.161+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8f7cdb2a:::obj538:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.161+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8f4c4fa9:::obj590:head local-lis/les=117/118 n=337 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:8fcfcc34:::obj221:head, version: 111'221, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8fcfcc34:::obj221:head@111'221, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.161+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8f4c4fa9:::obj590:head local-lis/les=117/118 n=337 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8fcfcc34:::obj221:head@111'221, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.161+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8f4c4fa9:::obj590:head local-lis/les=117/118 n=337 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:8fcfcc34:::obj221:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.161+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8f4c4fa9:::obj590:head local-lis/les=117/118 n=337 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:8fcfcc34:::obj221:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.161+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8f4c4fa9:::obj590:head local-lis/les=117/118 n=337 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:8fcfcc34:::obj221:head 2026-02-11T23:31:28.161+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8f4c4fa9:::obj590:head local-lis/les=117/118 n=337 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:8fcfcc34:::obj221:head v 111'221 2026-02-11T23:31:28.162+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8f4c4fa9:::obj590:head local-lis/les=117/118 n=337 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.162+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.162+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:8fcfcc34:::obj221:head, version: 111'221, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8fcfcc34:::obj221:head@111'221, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000087 2026-02-11T23:31:28.162+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.162+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.162+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8f7cdb2a:::obj538:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.162+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8f7cdb2a:::obj538:head) v3 prio 15 cost 0 latency 0.000110 pg_backfill(progress 4.0 e 119/119 lb 4:8f7cdb2a:::obj538:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8f4c4fa9:::obj590:head local-lis/les=117/118 n=337 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.162+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.162+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8f4c4fa9:::obj590:head local-lis/les=117/118 n=337 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:8f7cdb2a:::obj538:head) v3 2026-02-11T23:31:28.162+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8f4c4fa9:::obj590:head local-lis/les=117/118 n=337 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:8f7cdb2a:::obj538:head) 2026-02-11T23:31:28.162+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.162+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8f7cdb2a:::obj538:head) v3 finish latency 0.000014 2026-02-11T23:31:28.162+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8f7cdb2a:::obj538:head local-lis/les=117/118 n=338 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.162+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.162+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.162+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8f7cdb2a:::obj538:head local-lis/les=117/118 n=338 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.162+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.162+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.162+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.162+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x5d54c00 con 0x507ac00 2026-02-11T23:31:28.162+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:8fcfcc34:::obj221:head)]) -- 0x642d200 con 0x507ac00 2026-02-11T23:31:28.162+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.162+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 717 ==== MOSDPGPush(4.0 119/117 [PushOp(4:902252c4:::obj397:head, version: 111'397, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:902252c4:::obj397:head@111'397, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3815520 con 0x507ac00 2026-02-11T23:31:28.162+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:902252c4:::obj397:head, version: 111'397, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:902252c4:::obj397:head@111'397, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 119 2026-02-11T23:31:28.162+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:902252c4:::obj397:head, version: 111'397, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:902252c4:::obj397:head@111'397, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.162+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.162+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:902252c4:::obj397:head, version: 111'397, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:902252c4:::obj397:head@111'397, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.162+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 718 ==== pg_backfill(progress 4.0 e 119/119 lb 4:8fcfcc34:::obj221:head) ==== 1062+0+0 (unknown 0 0 0) 0x5985700 con 0x507ac00 2026-02-11T23:31:28.162+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.162+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:8fcfcc34:::obj221:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.162+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:902252c4:::obj397:head, version: 111'397, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:902252c4:::obj397:head@111'397, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.162+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8fcfcc34:::obj221:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.162+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:902252c4:::obj397:head, version: 111'397, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:902252c4:::obj397:head@111'397, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 119/117 [PushOp(4:902252c4:::obj397:head, version: 111'397, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:902252c4:::obj397:head@111'397, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8f7cdb2a:::obj538:head local-lis/les=117/118 n=338 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.162+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.162+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.162+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8f7cdb2a:::obj538:head local-lis/les=117/118 n=338 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:902252c4:::obj397:head, version: 111'397, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:902252c4:::obj397:head@111'397, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.162+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8fcfcc34:::obj221:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.162+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8f7cdb2a:::obj538:head local-lis/les=117/118 n=338 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:902252c4:::obj397:head@111'397, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.162+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8f7cdb2a:::obj538:head local-lis/les=117/118 n=338 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:902252c4:::obj397:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.162+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8f7cdb2a:::obj538:head local-lis/les=117/118 n=338 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:902252c4:::obj397:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.162+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8f7cdb2a:::obj538:head local-lis/les=117/118 n=338 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:902252c4:::obj397:head 2026-02-11T23:31:28.162+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8f7cdb2a:::obj538:head local-lis/les=117/118 n=338 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:902252c4:::obj397:head v 111'397 2026-02-11T23:31:28.162+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8f7cdb2a:::obj538:head local-lis/les=117/118 n=338 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.162+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.162+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:902252c4:::obj397:head, version: 111'397, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:902252c4:::obj397:head@111'397, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:28.162+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8f7cdb2a:::obj538:head local-lis/les=117/118 n=338 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.162+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.162+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:8fcfcc34:::obj221:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.162+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.162+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8fcfcc34:::obj221:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 4.0 e 119/119 lb 4:8fcfcc34:::obj221:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8f7cdb2a:::obj538:head local-lis/les=117/118 n=338 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.162+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.162+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8f7cdb2a:::obj538:head local-lis/les=117/118 n=338 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:8fcfcc34:::obj221:head) v3 2026-02-11T23:31:28.162+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8f7cdb2a:::obj538:head local-lis/les=117/118 n=338 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:8fcfcc34:::obj221:head) 2026-02-11T23:31:28.162+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.162+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:8fcfcc34:::obj221:head) v3 finish latency 0.000018 2026-02-11T23:31:28.162+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.162+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.162+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8fcfcc34:::obj221:head local-lis/les=117/118 n=339 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.162+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.162+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.162+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.162+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x649e600 con 0x507ac00 2026-02-11T23:31:28.162+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:902252c4:::obj397:head)]) -- 0x4f7a780 con 0x507ac00 2026-02-11T23:31:28.162+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.163+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 719 ==== MOSDPGPush(4.0 119/117 [PushOp(4:90d5bfbc:::obj593:head, version: 111'593, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:90d5bfbc:::obj593:head@111'593, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56171e0 con 0x507ac00 2026-02-11T23:31:28.163+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:90d5bfbc:::obj593:head, version: 111'593, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:90d5bfbc:::obj593:head@111'593, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.163+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:90d5bfbc:::obj593:head, version: 111'593, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:90d5bfbc:::obj593:head@111'593, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.163+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.163+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:90d5bfbc:::obj593:head, version: 111'593, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:90d5bfbc:::obj593:head@111'593, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.163+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 720 ==== pg_backfill(progress 4.0 e 119/119 lb 4:902252c4:::obj397:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9a680 con 0x507ac00 2026-02-11T23:31:28.163+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.163+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:902252c4:::obj397:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.163+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:90d5bfbc:::obj593:head, version: 111'593, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:90d5bfbc:::obj593:head@111'593, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.163+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:902252c4:::obj397:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.163+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.163+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:90d5bfbc:::obj593:head, version: 111'593, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:90d5bfbc:::obj593:head@111'593, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 119/117 [PushOp(4:90d5bfbc:::obj593:head, version: 111'593, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:90d5bfbc:::obj593:head@111'593, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8fcfcc34:::obj221:head local-lis/les=117/118 n=339 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.163+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:902252c4:::obj397:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.163+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.163+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8fcfcc34:::obj221:head local-lis/les=117/118 n=339 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:90d5bfbc:::obj593:head, version: 111'593, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:90d5bfbc:::obj593:head@111'593, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.163+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8fcfcc34:::obj221:head local-lis/les=117/118 n=339 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:90d5bfbc:::obj593:head@111'593, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.163+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8fcfcc34:::obj221:head local-lis/les=117/118 n=339 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:90d5bfbc:::obj593:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.163+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8fcfcc34:::obj221:head local-lis/les=117/118 n=339 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:90d5bfbc:::obj593:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.163+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8fcfcc34:::obj221:head local-lis/les=117/118 n=339 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:90d5bfbc:::obj593:head 2026-02-11T23:31:28.163+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8fcfcc34:::obj221:head local-lis/les=117/118 n=339 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:90d5bfbc:::obj593:head v 111'593 2026-02-11T23:31:28.163+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8fcfcc34:::obj221:head local-lis/les=117/118 n=339 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.163+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.163+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:90d5bfbc:::obj593:head, version: 111'593, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:90d5bfbc:::obj593:head@111'593, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:28.163+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8fcfcc34:::obj221:head local-lis/les=117/118 n=339 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.163+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.163+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:902252c4:::obj397:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.163+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.163+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:902252c4:::obj397:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 4.0 e 119/119 lb 4:902252c4:::obj397:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:8fcfcc34:::obj221:head local-lis/les=117/118 n=339 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.163+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.163+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8fcfcc34:::obj221:head local-lis/les=117/118 n=339 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:902252c4:::obj397:head) v3 2026-02-11T23:31:28.163+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:8fcfcc34:::obj221:head local-lis/les=117/118 n=339 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:902252c4:::obj397:head) 2026-02-11T23:31:28.163+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.163+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:902252c4:::obj397:head) v3 finish latency 0.000013 2026-02-11T23:31:28.163+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.163+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.163+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:902252c4:::obj397:head local-lis/les=117/118 n=340 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.163+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.163+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.163+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.163+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63fec00 con 0x507ac00 2026-02-11T23:31:28.163+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:90d5bfbc:::obj593:head)]) -- 0x4f7aa80 con 0x507ac00 2026-02-11T23:31:28.163+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.163+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 721 ==== MOSDPGPush(4.0 119/117 [PushOp(4:917e3523:::obj419:head, version: 111'419, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:917e3523:::obj419:head@111'419, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56cad00 con 0x507ac00 2026-02-11T23:31:28.163+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:917e3523:::obj419:head, version: 111'419, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:917e3523:::obj419:head@111'419, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.163+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:917e3523:::obj419:head, version: 111'419, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:917e3523:::obj419:head@111'419, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.163+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.163+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:917e3523:::obj419:head, version: 111'419, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:917e3523:::obj419:head@111'419, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.163+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.163+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:917e3523:::obj419:head, version: 111'419, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:917e3523:::obj419:head@111'419, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.163+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 722 ==== pg_backfill(progress 4.0 e 119/119 lb 4:90d5bfbc:::obj593:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176000 con 0x507ac00 2026-02-11T23:31:28.163+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:90d5bfbc:::obj593:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.163+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.163+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:917e3523:::obj419:head, version: 111'419, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:917e3523:::obj419:head@111'419, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 119/117 [PushOp(4:917e3523:::obj419:head, version: 111'419, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:917e3523:::obj419:head@111'419, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:902252c4:::obj397:head local-lis/les=117/118 n=340 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.163+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.163+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:90d5bfbc:::obj593:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.163+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:902252c4:::obj397:head local-lis/les=117/118 n=340 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:917e3523:::obj419:head, version: 111'419, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:917e3523:::obj419:head@111'419, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.163+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:902252c4:::obj397:head local-lis/les=117/118 n=340 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:917e3523:::obj419:head@111'419, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.163+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.163+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:90d5bfbc:::obj593:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.163+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:902252c4:::obj397:head local-lis/les=117/118 n=340 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:917e3523:::obj419:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.163+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:902252c4:::obj397:head local-lis/les=117/118 n=340 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:917e3523:::obj419:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.163+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:902252c4:::obj397:head local-lis/les=117/118 n=340 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:917e3523:::obj419:head 2026-02-11T23:31:28.163+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:902252c4:::obj397:head local-lis/les=117/118 n=340 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:917e3523:::obj419:head v 111'419 2026-02-11T23:31:28.163+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:902252c4:::obj397:head local-lis/les=117/118 n=340 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.163+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.163+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:917e3523:::obj419:head, version: 111'419, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:917e3523:::obj419:head@111'419, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:28.163+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:902252c4:::obj397:head local-lis/les=117/118 n=340 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.163+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.163+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.163+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:90d5bfbc:::obj593:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.163+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:90d5bfbc:::obj593:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 4.0 e 119/119 lb 4:90d5bfbc:::obj593:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:902252c4:::obj397:head local-lis/les=117/118 n=340 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.163+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.163+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:902252c4:::obj397:head local-lis/les=117/118 n=340 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:90d5bfbc:::obj593:head) v3 2026-02-11T23:31:28.163+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:902252c4:::obj397:head local-lis/les=117/118 n=340 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:90d5bfbc:::obj593:head) 2026-02-11T23:31:28.163+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.163+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:90d5bfbc:::obj593:head) v3 finish latency 0.000013 2026-02-11T23:31:28.163+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.164+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.164+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:90d5bfbc:::obj593:head local-lis/les=117/118 n=341 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.164+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.164+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.164+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.164+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7a300 con 0x507ac00 2026-02-11T23:31:28.164+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:917e3523:::obj419:head)]) -- 0x4f7a900 con 0x507ac00 2026-02-11T23:31:28.164+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.164+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 723 ==== MOSDPGPush(4.0 119/117 [PushOp(4:91fc975b:::obj401:head, version: 111'401, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:91fc975b:::obj401:head@111'401, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db0b60 con 0x507ac00 2026-02-11T23:31:28.164+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:91fc975b:::obj401:head, version: 111'401, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:91fc975b:::obj401:head@111'401, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.164+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:91fc975b:::obj401:head, version: 111'401, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:91fc975b:::obj401:head@111'401, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.164+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.164+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:91fc975b:::obj401:head, version: 111'401, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:91fc975b:::obj401:head@111'401, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.164+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 724 ==== pg_backfill(progress 4.0 e 119/119 lb 4:917e3523:::obj419:head) ==== 1062+0+0 (unknown 0 0 0) 0x6179700 con 0x507ac00 2026-02-11T23:31:28.164+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:917e3523:::obj419:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.164+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.164+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:91fc975b:::obj401:head, version: 111'401, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:91fc975b:::obj401:head@111'401, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.164+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:917e3523:::obj419:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.164+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.164+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:91fc975b:::obj401:head, version: 111'401, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:91fc975b:::obj401:head@111'401, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 119/117 [PushOp(4:91fc975b:::obj401:head, version: 111'401, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:91fc975b:::obj401:head@111'401, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:90d5bfbc:::obj593:head local-lis/les=117/118 n=341 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.164+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:917e3523:::obj419:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.164+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.164+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:90d5bfbc:::obj593:head local-lis/les=117/118 n=341 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:91fc975b:::obj401:head, version: 111'401, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:91fc975b:::obj401:head@111'401, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.164+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:90d5bfbc:::obj593:head local-lis/les=117/118 n=341 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:91fc975b:::obj401:head@111'401, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.164+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:90d5bfbc:::obj593:head local-lis/les=117/118 n=341 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:91fc975b:::obj401:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.164+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:90d5bfbc:::obj593:head local-lis/les=117/118 n=341 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:91fc975b:::obj401:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.164+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:90d5bfbc:::obj593:head local-lis/les=117/118 n=341 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:91fc975b:::obj401:head 2026-02-11T23:31:28.164+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:90d5bfbc:::obj593:head local-lis/les=117/118 n=341 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:91fc975b:::obj401:head v 111'401 2026-02-11T23:31:28.164+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:90d5bfbc:::obj593:head local-lis/les=117/118 n=341 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.164+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.164+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:91fc975b:::obj401:head, version: 111'401, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:91fc975b:::obj401:head@111'401, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:28.164+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:90d5bfbc:::obj593:head local-lis/les=117/118 n=341 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.164+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.164+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:917e3523:::obj419:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.164+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.164+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:917e3523:::obj419:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 4.0 e 119/119 lb 4:917e3523:::obj419:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:90d5bfbc:::obj593:head local-lis/les=117/118 n=341 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.164+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.164+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:90d5bfbc:::obj593:head local-lis/les=117/118 n=341 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:917e3523:::obj419:head) v3 2026-02-11T23:31:28.164+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:90d5bfbc:::obj593:head local-lis/les=117/118 n=341 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:917e3523:::obj419:head) 2026-02-11T23:31:28.164+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.164+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:917e3523:::obj419:head) v3 finish latency 0.000013 2026-02-11T23:31:28.164+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.164+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.164+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:917e3523:::obj419:head local-lis/les=117/118 n=342 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.164+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.164+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.164+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.164+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63ff200 con 0x507ac00 2026-02-11T23:31:28.164+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:91fc975b:::obj401:head)]) -- 0x4f7b200 con 0x507ac00 2026-02-11T23:31:28.164+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.164+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 725 ==== MOSDPGPush(4.0 119/117 [PushOp(4:92b89151:::obj431:head, version: 111'431, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:92b89151:::obj431:head@111'431, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2d00 con 0x507ac00 2026-02-11T23:31:28.164+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:92b89151:::obj431:head, version: 111'431, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:92b89151:::obj431:head@111'431, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.164+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:92b89151:::obj431:head, version: 111'431, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:92b89151:::obj431:head@111'431, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.164+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.164+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:92b89151:::obj431:head, version: 111'431, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:92b89151:::obj431:head@111'431, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.164+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 726 ==== pg_backfill(progress 4.0 e 119/119 lb 4:91fc975b:::obj401:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2cb00 con 0x507ac00 2026-02-11T23:31:28.164+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.164+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:91fc975b:::obj401:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.164+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:92b89151:::obj431:head, version: 111'431, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:92b89151:::obj431:head@111'431, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.164+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:91fc975b:::obj401:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.164+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:92b89151:::obj431:head, version: 111'431, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:92b89151:::obj431:head@111'431, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 119/117 [PushOp(4:92b89151:::obj431:head, version: 111'431, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:92b89151:::obj431:head@111'431, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:917e3523:::obj419:head local-lis/les=117/118 n=342 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.164+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.164+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.164+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:91fc975b:::obj401:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.164+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:917e3523:::obj419:head local-lis/les=117/118 n=342 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:92b89151:::obj431:head, version: 111'431, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:92b89151:::obj431:head@111'431, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.164+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:917e3523:::obj419:head local-lis/les=117/118 n=342 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:92b89151:::obj431:head@111'431, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.164+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:917e3523:::obj419:head local-lis/les=117/118 n=342 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:92b89151:::obj431:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.164+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:917e3523:::obj419:head local-lis/les=117/118 n=342 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:92b89151:::obj431:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.165+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:917e3523:::obj419:head local-lis/les=117/118 n=342 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:92b89151:::obj431:head 2026-02-11T23:31:28.165+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:917e3523:::obj419:head local-lis/les=117/118 n=342 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:92b89151:::obj431:head v 111'431 2026-02-11T23:31:28.165+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:917e3523:::obj419:head local-lis/les=117/118 n=342 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.165+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.165+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:92b89151:::obj431:head, version: 111'431, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:92b89151:::obj431:head@111'431, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:28.165+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:917e3523:::obj419:head local-lis/les=117/118 n=342 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.165+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.165+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.165+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:91fc975b:::obj401:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.165+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:91fc975b:::obj401:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 119/119 lb 4:91fc975b:::obj401:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:917e3523:::obj419:head local-lis/les=117/118 n=342 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.165+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.165+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:917e3523:::obj419:head local-lis/les=117/118 n=342 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:91fc975b:::obj401:head) v3 2026-02-11T23:31:28.165+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:917e3523:::obj419:head local-lis/les=117/118 n=342 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:91fc975b:::obj401:head) 2026-02-11T23:31:28.165+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.165+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:91fc975b:::obj401:head) v3 finish latency 0.000014 2026-02-11T23:31:28.165+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.165+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.165+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:91fc975b:::obj401:head local-lis/les=117/118 n=343 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.165+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.165+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.165+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.165+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.165+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c9080 con 0x507ac00 2026-02-11T23:31:28.165+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:92b89151:::obj431:head)]) -- 0x5f2dc80 con 0x507ac00 2026-02-11T23:31:28.165+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.165+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 727 ==== MOSDPGPush(4.0 119/117 [PushOp(4:9315b0b3:::obj555:head, version: 111'555, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9315b0b3:::obj555:head@111'555, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f0ea0 con 0x507ac00 2026-02-11T23:31:28.165+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:9315b0b3:::obj555:head, version: 111'555, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9315b0b3:::obj555:head@111'555, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.165+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:9315b0b3:::obj555:head, version: 111'555, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9315b0b3:::obj555:head@111'555, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.165+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.165+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:9315b0b3:::obj555:head, version: 111'555, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9315b0b3:::obj555:head@111'555, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.165+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 728 ==== pg_backfill(progress 4.0 e 119/119 lb 4:92b89151:::obj431:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2c000 con 0x507ac00 2026-02-11T23:31:28.165+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:92b89151:::obj431:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.165+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.165+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:92b89151:::obj431:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.165+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:9315b0b3:::obj555:head, version: 111'555, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9315b0b3:::obj555:head@111'555, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.165+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:9315b0b3:::obj555:head, version: 111'555, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9315b0b3:::obj555:head@111'555, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 119/117 [PushOp(4:9315b0b3:::obj555:head, version: 111'555, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9315b0b3:::obj555:head@111'555, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:91fc975b:::obj401:head local-lis/les=117/118 n=343 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.165+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.165+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.165+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:92b89151:::obj431:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.165+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:91fc975b:::obj401:head local-lis/les=117/118 n=343 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:9315b0b3:::obj555:head, version: 111'555, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9315b0b3:::obj555:head@111'555, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.165+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:91fc975b:::obj401:head local-lis/les=117/118 n=343 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:9315b0b3:::obj555:head@111'555, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.165+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:91fc975b:::obj401:head local-lis/les=117/118 n=343 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:9315b0b3:::obj555:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.165+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:91fc975b:::obj401:head local-lis/les=117/118 n=343 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:9315b0b3:::obj555:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.165+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:91fc975b:::obj401:head local-lis/les=117/118 n=343 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:9315b0b3:::obj555:head 2026-02-11T23:31:28.165+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:91fc975b:::obj401:head local-lis/les=117/118 n=343 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:9315b0b3:::obj555:head v 111'555 2026-02-11T23:31:28.165+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:91fc975b:::obj401:head local-lis/les=117/118 n=343 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.165+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.165+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:9315b0b3:::obj555:head, version: 111'555, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9315b0b3:::obj555:head@111'555, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:28.165+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:91fc975b:::obj401:head local-lis/les=117/118 n=343 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.165+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.165+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.165+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:92b89151:::obj431:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.165+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:92b89151:::obj431:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 4.0 e 119/119 lb 4:92b89151:::obj431:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:91fc975b:::obj401:head local-lis/les=117/118 n=343 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.165+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.165+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:91fc975b:::obj401:head local-lis/les=117/118 n=343 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:92b89151:::obj431:head) v3 2026-02-11T23:31:28.165+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:91fc975b:::obj401:head local-lis/les=117/118 n=343 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:92b89151:::obj431:head) 2026-02-11T23:31:28.165+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.165+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:92b89151:::obj431:head) v3 finish latency 0.000014 2026-02-11T23:31:28.165+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.165+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.165+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:92b89151:::obj431:head local-lis/les=117/118 n=344 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.165+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.165+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.165+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.165+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7a780 con 0x507ac00 2026-02-11T23:31:28.165+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.165+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:9315b0b3:::obj555:head)]) -- 0x63ffb00 con 0x507ac00 2026-02-11T23:31:28.165+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.166+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 729 ==== MOSDPGPush(4.0 119/117 [PushOp(4:9397791a:::obj547:head, version: 111'547, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9397791a:::obj547:head@111'547, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a3040 con 0x507ac00 2026-02-11T23:31:28.166+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:9397791a:::obj547:head, version: 111'547, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9397791a:::obj547:head@111'547, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.166+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:9397791a:::obj547:head, version: 111'547, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9397791a:::obj547:head@111'547, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.166+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.166+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:9397791a:::obj547:head, version: 111'547, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9397791a:::obj547:head@111'547, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.166+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 730 ==== pg_backfill(progress 4.0 e 119/119 lb 4:9315b0b3:::obj555:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e100 con 0x507ac00 2026-02-11T23:31:28.166+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:9315b0b3:::obj555:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.166+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.166+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:9315b0b3:::obj555:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.166+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:9397791a:::obj547:head, version: 111'547, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9397791a:::obj547:head@111'547, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.166+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.166+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:9397791a:::obj547:head, version: 111'547, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9397791a:::obj547:head@111'547, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 119/117 [PushOp(4:9397791a:::obj547:head, version: 111'547, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9397791a:::obj547:head@111'547, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:92b89151:::obj431:head local-lis/les=117/118 n=344 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.166+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.166+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:92b89151:::obj431:head local-lis/les=117/118 n=344 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:9397791a:::obj547:head, version: 111'547, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9397791a:::obj547:head@111'547, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.166+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:92b89151:::obj431:head local-lis/les=117/118 n=344 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:9397791a:::obj547:head@111'547, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.166+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.166+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:9315b0b3:::obj555:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.166+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:92b89151:::obj431:head local-lis/les=117/118 n=344 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:9397791a:::obj547:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.166+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:92b89151:::obj431:head local-lis/les=117/118 n=344 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:9397791a:::obj547:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.166+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:92b89151:::obj431:head local-lis/les=117/118 n=344 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:9397791a:::obj547:head 2026-02-11T23:31:28.166+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:92b89151:::obj431:head local-lis/les=117/118 n=344 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:9397791a:::obj547:head v 111'547 2026-02-11T23:31:28.166+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:92b89151:::obj431:head local-lis/les=117/118 n=344 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.166+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.166+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:9397791a:::obj547:head, version: 111'547, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9397791a:::obj547:head@111'547, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:28.166+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:92b89151:::obj431:head local-lis/les=117/118 n=344 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.166+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.166+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:9315b0b3:::obj555:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.166+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.166+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:9315b0b3:::obj555:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 4.0 e 119/119 lb 4:9315b0b3:::obj555:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:92b89151:::obj431:head local-lis/les=117/118 n=344 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.166+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.166+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:92b89151:::obj431:head local-lis/les=117/118 n=344 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:9315b0b3:::obj555:head) v3 2026-02-11T23:31:28.166+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:92b89151:::obj431:head local-lis/les=117/118 n=344 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:9315b0b3:::obj555:head) 2026-02-11T23:31:28.166+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.166+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:9315b0b3:::obj555:head) v3 finish latency 0.000013 2026-02-11T23:31:28.166+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.166+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.166+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9315b0b3:::obj555:head local-lis/les=117/118 n=345 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.166+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.166+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.166+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.166+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x5d54c00 con 0x507ac00 2026-02-11T23:31:28.166+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:9397791a:::obj547:head)]) -- 0x649e600 con 0x507ac00 2026-02-11T23:31:28.166+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.166+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 731 ==== MOSDPGPush(4.0 119/117 [PushOp(4:9455f78c:::obj228:head, version: 111'228, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9455f78c:::obj228:head@111'228, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3814d00 con 0x507ac00 2026-02-11T23:31:28.166+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:9455f78c:::obj228:head, version: 111'228, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9455f78c:::obj228:head@111'228, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.166+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:9455f78c:::obj228:head, version: 111'228, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9455f78c:::obj228:head@111'228, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.166+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.166+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:9455f78c:::obj228:head, version: 111'228, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9455f78c:::obj228:head@111'228, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.166+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 732 ==== pg_backfill(progress 4.0 e 119/119 lb 4:9397791a:::obj547:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176580 con 0x507ac00 2026-02-11T23:31:28.166+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.166+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:9397791a:::obj547:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.166+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:9455f78c:::obj228:head, version: 111'228, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9455f78c:::obj228:head@111'228, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.166+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:9397791a:::obj547:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.166+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:9455f78c:::obj228:head, version: 111'228, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9455f78c:::obj228:head@111'228, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 119/117 [PushOp(4:9455f78c:::obj228:head, version: 111'228, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9455f78c:::obj228:head@111'228, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:9315b0b3:::obj555:head local-lis/les=117/118 n=345 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.166+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.166+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.166+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9315b0b3:::obj555:head local-lis/les=117/118 n=345 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:9455f78c:::obj228:head, version: 111'228, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9455f78c:::obj228:head@111'228, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.166+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:9397791a:::obj547:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.166+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9315b0b3:::obj555:head local-lis/les=117/118 n=345 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:9455f78c:::obj228:head@111'228, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.166+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9315b0b3:::obj555:head local-lis/les=117/118 n=345 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:9455f78c:::obj228:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.166+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9315b0b3:::obj555:head local-lis/les=117/118 n=345 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:9455f78c:::obj228:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.166+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9315b0b3:::obj555:head local-lis/les=117/118 n=345 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:9455f78c:::obj228:head 2026-02-11T23:31:28.166+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9315b0b3:::obj555:head local-lis/les=117/118 n=345 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:9455f78c:::obj228:head v 111'228 2026-02-11T23:31:28.166+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9315b0b3:::obj555:head local-lis/les=117/118 n=345 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.166+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.166+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:9455f78c:::obj228:head, version: 111'228, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9455f78c:::obj228:head@111'228, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:28.166+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9315b0b3:::obj555:head local-lis/les=117/118 n=345 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.166+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.166+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:9397791a:::obj547:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.166+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.166+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:9397791a:::obj547:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 4.0 e 119/119 lb 4:9397791a:::obj547:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:9315b0b3:::obj555:head local-lis/les=117/118 n=345 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.166+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.166+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9315b0b3:::obj555:head local-lis/les=117/118 n=345 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:9397791a:::obj547:head) v3 2026-02-11T23:31:28.166+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9315b0b3:::obj555:head local-lis/les=117/118 n=345 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:9397791a:::obj547:head) 2026-02-11T23:31:28.166+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.166+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:9397791a:::obj547:head) v3 finish latency 0.000013 2026-02-11T23:31:28.166+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.166+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.166+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9397791a:::obj547:head local-lis/les=117/118 n=346 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.166+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.166+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.166+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.166+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x640e180 con 0x507ac00 2026-02-11T23:31:28.166+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:9455f78c:::obj228:head)]) -- 0x4f7b080 con 0x507ac00 2026-02-11T23:31:28.166+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.167+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 733 ==== MOSDPGPush(4.0 119/117 [PushOp(4:945b6e2e:::obj114:head, version: 111'114, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:945b6e2e:::obj114:head@111'114, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c1a0 con 0x507ac00 2026-02-11T23:31:28.167+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:945b6e2e:::obj114:head, version: 111'114, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:945b6e2e:::obj114:head@111'114, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.167+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:945b6e2e:::obj114:head, version: 111'114, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:945b6e2e:::obj114:head@111'114, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.167+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.167+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:945b6e2e:::obj114:head, version: 111'114, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:945b6e2e:::obj114:head@111'114, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.167+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 734 ==== pg_backfill(progress 4.0 e 119/119 lb 4:9455f78c:::obj228:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e680 con 0x507ac00 2026-02-11T23:31:28.167+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:9455f78c:::obj228:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.167+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.167+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:9455f78c:::obj228:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.167+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:945b6e2e:::obj114:head, version: 111'114, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:945b6e2e:::obj114:head@111'114, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.167+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.167+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:945b6e2e:::obj114:head, version: 111'114, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:945b6e2e:::obj114:head@111'114, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 119/117 [PushOp(4:945b6e2e:::obj114:head, version: 111'114, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:945b6e2e:::obj114:head@111'114, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:9397791a:::obj547:head local-lis/les=117/118 n=346 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.167+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.167+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9397791a:::obj547:head local-lis/les=117/118 n=346 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:945b6e2e:::obj114:head, version: 111'114, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:945b6e2e:::obj114:head@111'114, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.167+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.167+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9397791a:::obj547:head local-lis/les=117/118 n=346 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:945b6e2e:::obj114:head@111'114, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.167+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:9455f78c:::obj228:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.167+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9397791a:::obj547:head local-lis/les=117/118 n=346 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:945b6e2e:::obj114:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.167+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9397791a:::obj547:head local-lis/les=117/118 n=346 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:945b6e2e:::obj114:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.167+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9397791a:::obj547:head local-lis/les=117/118 n=346 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:945b6e2e:::obj114:head 2026-02-11T23:31:28.167+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9397791a:::obj547:head local-lis/les=117/118 n=346 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:945b6e2e:::obj114:head v 111'114 2026-02-11T23:31:28.167+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9397791a:::obj547:head local-lis/les=117/118 n=346 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.167+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.167+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:945b6e2e:::obj114:head, version: 111'114, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:945b6e2e:::obj114:head@111'114, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:28.167+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9397791a:::obj547:head local-lis/les=117/118 n=346 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.167+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.167+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.167+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:9455f78c:::obj228:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.167+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:9455f78c:::obj228:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 4.0 e 119/119 lb 4:9455f78c:::obj228:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:9397791a:::obj547:head local-lis/les=117/118 n=346 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.167+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.167+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9397791a:::obj547:head local-lis/les=117/118 n=346 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:9455f78c:::obj228:head) v3 2026-02-11T23:31:28.167+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9397791a:::obj547:head local-lis/les=117/118 n=346 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:9455f78c:::obj228:head) 2026-02-11T23:31:28.167+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.167+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:9455f78c:::obj228:head) v3 finish latency 0.000014 2026-02-11T23:31:28.167+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.167+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.167+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9455f78c:::obj228:head local-lis/les=117/118 n=347 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.167+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.167+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.167+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.167+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x640fe00 con 0x507ac00 2026-02-11T23:31:28.167+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:945b6e2e:::obj114:head)]) -- 0x644d380 con 0x507ac00 2026-02-11T23:31:28.167+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.167+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 735 ==== MOSDPGPush(4.0 119/117 [PushOp(4:9477fafa:::obj499:head, version: 111'499, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9477fafa:::obj499:head@111'499, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2d00 con 0x507ac00 2026-02-11T23:31:28.167+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:9477fafa:::obj499:head, version: 111'499, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9477fafa:::obj499:head@111'499, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.167+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:9477fafa:::obj499:head, version: 111'499, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9477fafa:::obj499:head@111'499, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.167+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.167+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 736 ==== pg_backfill(progress 4.0 e 119/119 lb 4:945b6e2e:::obj114:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2d080 con 0x507ac00 2026-02-11T23:31:28.167+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:945b6e2e:::obj114:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.167+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:9477fafa:::obj499:head, version: 111'499, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9477fafa:::obj499:head@111'499, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.167+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:945b6e2e:::obj114:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.167+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.167+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:9477fafa:::obj499:head, version: 111'499, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9477fafa:::obj499:head@111'499, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.167+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:9477fafa:::obj499:head, version: 111'499, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9477fafa:::obj499:head@111'499, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 119/117 [PushOp(4:9477fafa:::obj499:head, version: 111'499, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9477fafa:::obj499:head@111'499, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:9455f78c:::obj228:head local-lis/les=117/118 n=347 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.167+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.167+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.167+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9455f78c:::obj228:head local-lis/les=117/118 n=347 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:9477fafa:::obj499:head, version: 111'499, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9477fafa:::obj499:head@111'499, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.167+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:945b6e2e:::obj114:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.167+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9455f78c:::obj228:head local-lis/les=117/118 n=347 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:9477fafa:::obj499:head@111'499, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.167+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9455f78c:::obj228:head local-lis/les=117/118 n=347 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:9477fafa:::obj499:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.167+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9455f78c:::obj228:head local-lis/les=117/118 n=347 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:9477fafa:::obj499:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.167+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9455f78c:::obj228:head local-lis/les=117/118 n=347 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:9477fafa:::obj499:head 2026-02-11T23:31:28.167+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9455f78c:::obj228:head local-lis/les=117/118 n=347 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:9477fafa:::obj499:head v 111'499 2026-02-11T23:31:28.167+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9455f78c:::obj228:head local-lis/les=117/118 n=347 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.167+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.167+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:9477fafa:::obj499:head, version: 111'499, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9477fafa:::obj499:head@111'499, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000043 2026-02-11T23:31:28.167+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9455f78c:::obj228:head local-lis/les=117/118 n=347 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.167+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.167+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.167+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:945b6e2e:::obj114:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.167+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:945b6e2e:::obj114:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 4.0 e 119/119 lb 4:945b6e2e:::obj114:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:9455f78c:::obj228:head local-lis/les=117/118 n=347 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.167+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.167+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9455f78c:::obj228:head local-lis/les=117/118 n=347 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:945b6e2e:::obj114:head) v3 2026-02-11T23:31:28.167+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9455f78c:::obj228:head local-lis/les=117/118 n=347 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:945b6e2e:::obj114:head) 2026-02-11T23:31:28.167+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.167+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:945b6e2e:::obj114:head) v3 finish latency 0.000013 2026-02-11T23:31:28.167+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.168+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.168+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:945b6e2e:::obj114:head local-lis/les=117/118 n=348 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.168+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.168+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.168+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.168+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x642cd80 con 0x507ac00 2026-02-11T23:31:28.168+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:9477fafa:::obj499:head)]) -- 0x5d55200 con 0x507ac00 2026-02-11T23:31:28.168+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.168+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 737 ==== MOSDPGPush(4.0 119/117 [PushOp(4:95a9a074:::obj509:head, version: 111'509, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:95a9a074:::obj509:head@111'509, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db0b60 con 0x507ac00 2026-02-11T23:31:28.168+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:95a9a074:::obj509:head, version: 111'509, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:95a9a074:::obj509:head@111'509, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.168+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:95a9a074:::obj509:head, version: 111'509, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:95a9a074:::obj509:head@111'509, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.168+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.168+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:95a9a074:::obj509:head, version: 111'509, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:95a9a074:::obj509:head@111'509, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.168+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 738 ==== pg_backfill(progress 4.0 e 119/119 lb 4:9477fafa:::obj499:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2f700 con 0x507ac00 2026-02-11T23:31:28.168+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:9477fafa:::obj499:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.168+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.168+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:95a9a074:::obj509:head, version: 111'509, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:95a9a074:::obj509:head@111'509, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.168+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:9477fafa:::obj499:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.168+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.168+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:95a9a074:::obj509:head, version: 111'509, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:95a9a074:::obj509:head@111'509, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 119/117 [PushOp(4:95a9a074:::obj509:head, version: 111'509, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:95a9a074:::obj509:head@111'509, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:945b6e2e:::obj114:head local-lis/les=117/118 n=348 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.168+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.168+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:9477fafa:::obj499:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.168+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:945b6e2e:::obj114:head local-lis/les=117/118 n=348 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:95a9a074:::obj509:head, version: 111'509, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:95a9a074:::obj509:head@111'509, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.168+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:945b6e2e:::obj114:head local-lis/les=117/118 n=348 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:95a9a074:::obj509:head@111'509, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.168+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:945b6e2e:::obj114:head local-lis/les=117/118 n=348 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:95a9a074:::obj509:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.168+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:945b6e2e:::obj114:head local-lis/les=117/118 n=348 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:95a9a074:::obj509:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.168+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:945b6e2e:::obj114:head local-lis/les=117/118 n=348 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:95a9a074:::obj509:head 2026-02-11T23:31:28.168+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:945b6e2e:::obj114:head local-lis/les=117/118 n=348 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:95a9a074:::obj509:head v 111'509 2026-02-11T23:31:28.168+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:945b6e2e:::obj114:head local-lis/les=117/118 n=348 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.168+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.168+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:95a9a074:::obj509:head, version: 111'509, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:95a9a074:::obj509:head@111'509, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:28.168+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:945b6e2e:::obj114:head local-lis/les=117/118 n=348 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.168+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.168+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:9477fafa:::obj499:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.168+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.168+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:9477fafa:::obj499:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 4.0 e 119/119 lb 4:9477fafa:::obj499:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:945b6e2e:::obj114:head local-lis/les=117/118 n=348 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.168+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.168+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:945b6e2e:::obj114:head local-lis/les=117/118 n=348 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:9477fafa:::obj499:head) v3 2026-02-11T23:31:28.168+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:945b6e2e:::obj114:head local-lis/les=117/118 n=348 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:9477fafa:::obj499:head) 2026-02-11T23:31:28.168+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.168+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:9477fafa:::obj499:head) v3 finish latency 0.000013 2026-02-11T23:31:28.168+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.168+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.168+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9477fafa:::obj499:head local-lis/les=117/118 n=349 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.168+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.168+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.168+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.168+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x640fc80 con 0x507ac00 2026-02-11T23:31:28.168+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:95a9a074:::obj509:head)]) -- 0x640fb00 con 0x507ac00 2026-02-11T23:31:28.168+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.169+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 739 ==== MOSDPGPush(4.0 119/117 [PushOp(4:9604ab91:::obj494:head, version: 111'494, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9604ab91:::obj494:head@111'494, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56cad00 con 0x507ac00 2026-02-11T23:31:28.169+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:9604ab91:::obj494:head, version: 111'494, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9604ab91:::obj494:head@111'494, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.169+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:9604ab91:::obj494:head, version: 111'494, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9604ab91:::obj494:head@111'494, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.169+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.169+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 740 ==== pg_backfill(progress 4.0 e 119/119 lb 4:95a9a074:::obj509:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f11600 con 0x507ac00 2026-02-11T23:31:28.169+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:9604ab91:::obj494:head, version: 111'494, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9604ab91:::obj494:head@111'494, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.169+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:95a9a074:::obj509:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.169+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.169+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:95a9a074:::obj509:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.169+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:9604ab91:::obj494:head, version: 111'494, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9604ab91:::obj494:head@111'494, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.169+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:9604ab91:::obj494:head, version: 111'494, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9604ab91:::obj494:head@111'494, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 119/117 [PushOp(4:9604ab91:::obj494:head, version: 111'494, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9604ab91:::obj494:head@111'494, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:9477fafa:::obj499:head local-lis/les=117/118 n=349 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.169+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.169+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.169+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9477fafa:::obj499:head local-lis/les=117/118 n=349 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:9604ab91:::obj494:head, version: 111'494, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9604ab91:::obj494:head@111'494, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.169+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:95a9a074:::obj509:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.169+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9477fafa:::obj499:head local-lis/les=117/118 n=349 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:9604ab91:::obj494:head@111'494, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.169+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9477fafa:::obj499:head local-lis/les=117/118 n=349 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:9604ab91:::obj494:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.169+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9477fafa:::obj499:head local-lis/les=117/118 n=349 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:9604ab91:::obj494:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.169+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9477fafa:::obj499:head local-lis/les=117/118 n=349 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:9604ab91:::obj494:head 2026-02-11T23:31:28.169+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9477fafa:::obj499:head local-lis/les=117/118 n=349 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:9604ab91:::obj494:head v 111'494 2026-02-11T23:31:28.169+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9477fafa:::obj499:head local-lis/les=117/118 n=349 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.169+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.169+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:9604ab91:::obj494:head, version: 111'494, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9604ab91:::obj494:head@111'494, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:28.169+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9477fafa:::obj499:head local-lis/les=117/118 n=349 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.169+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.169+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:95a9a074:::obj509:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.169+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.169+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:95a9a074:::obj509:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 4.0 e 119/119 lb 4:95a9a074:::obj509:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:9477fafa:::obj499:head local-lis/les=117/118 n=349 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.169+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.169+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9477fafa:::obj499:head local-lis/les=117/118 n=349 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:95a9a074:::obj509:head) v3 2026-02-11T23:31:28.169+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9477fafa:::obj499:head local-lis/les=117/118 n=349 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:95a9a074:::obj509:head) 2026-02-11T23:31:28.169+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.169+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:95a9a074:::obj509:head) v3 finish latency 0.000014 2026-02-11T23:31:28.169+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.169+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.169+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:95a9a074:::obj509:head local-lis/les=117/118 n=350 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.169+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.169+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.169+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.169+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x5d55800 con 0x507ac00 2026-02-11T23:31:28.169+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:9604ab91:::obj494:head)]) -- 0x640e600 con 0x507ac00 2026-02-11T23:31:28.169+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.169+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 741 ==== MOSDPGPush(4.0 119/117 [PushOp(4:96183f21:::obj585:head, version: 111'585, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:96183f21:::obj585:head@111'585, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56171e0 con 0x507ac00 2026-02-11T23:31:28.169+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:96183f21:::obj585:head, version: 111'585, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:96183f21:::obj585:head@111'585, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.169+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:96183f21:::obj585:head, version: 111'585, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:96183f21:::obj585:head@111'585, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.169+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.169+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:96183f21:::obj585:head, version: 111'585, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:96183f21:::obj585:head@111'585, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.169+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 742 ==== pg_backfill(progress 4.0 e 119/119 lb 4:9604ab91:::obj494:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f11080 con 0x507ac00 2026-02-11T23:31:28.169+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.169+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:9604ab91:::obj494:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.169+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:96183f21:::obj585:head, version: 111'585, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:96183f21:::obj585:head@111'585, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.169+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:9604ab91:::obj494:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.169+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.169+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:96183f21:::obj585:head, version: 111'585, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:96183f21:::obj585:head@111'585, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 119/117 [PushOp(4:96183f21:::obj585:head, version: 111'585, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:96183f21:::obj585:head@111'585, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:95a9a074:::obj509:head local-lis/les=117/118 n=350 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.169+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:9604ab91:::obj494:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.169+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.169+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:95a9a074:::obj509:head local-lis/les=117/118 n=350 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:96183f21:::obj585:head, version: 111'585, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:96183f21:::obj585:head@111'585, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.169+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:95a9a074:::obj509:head local-lis/les=117/118 n=350 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:96183f21:::obj585:head@111'585, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.169+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:95a9a074:::obj509:head local-lis/les=117/118 n=350 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:96183f21:::obj585:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.169+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:95a9a074:::obj509:head local-lis/les=117/118 n=350 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:96183f21:::obj585:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.169+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:95a9a074:::obj509:head local-lis/les=117/118 n=350 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:96183f21:::obj585:head 2026-02-11T23:31:28.169+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:95a9a074:::obj509:head local-lis/les=117/118 n=350 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:96183f21:::obj585:head v 111'585 2026-02-11T23:31:28.169+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:95a9a074:::obj509:head local-lis/les=117/118 n=350 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.169+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.169+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:96183f21:::obj585:head, version: 111'585, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:96183f21:::obj585:head@111'585, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:28.169+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:95a9a074:::obj509:head local-lis/les=117/118 n=350 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.169+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.169+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.169+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:9604ab91:::obj494:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.169+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:9604ab91:::obj494:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 4.0 e 119/119 lb 4:9604ab91:::obj494:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:95a9a074:::obj509:head local-lis/les=117/118 n=350 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.169+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.169+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:95a9a074:::obj509:head local-lis/les=117/118 n=350 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:9604ab91:::obj494:head) v3 2026-02-11T23:31:28.169+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:95a9a074:::obj509:head local-lis/les=117/118 n=350 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:9604ab91:::obj494:head) 2026-02-11T23:31:28.169+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.169+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:9604ab91:::obj494:head) v3 finish latency 0.000019 2026-02-11T23:31:28.169+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.169+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.169+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9604ab91:::obj494:head local-lis/les=117/118 n=351 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.169+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.169+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.169+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.169+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x640e900 con 0x507ac00 2026-02-11T23:31:28.169+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:96183f21:::obj585:head)]) -- 0x63c9e00 con 0x507ac00 2026-02-11T23:31:28.169+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.170+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 743 ==== MOSDPGPush(4.0 119/117 [PushOp(4:96d0cad0:::obj307:head, version: 111'307, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:96d0cad0:::obj307:head@111'307, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3815520 con 0x507ac00 2026-02-11T23:31:28.170+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:96d0cad0:::obj307:head, version: 111'307, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:96d0cad0:::obj307:head@111'307, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.170+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:96d0cad0:::obj307:head, version: 111'307, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:96d0cad0:::obj307:head@111'307, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.170+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.170+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:96d0cad0:::obj307:head, version: 111'307, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:96d0cad0:::obj307:head@111'307, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.170+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 744 ==== pg_backfill(progress 4.0 e 119/119 lb 4:96183f21:::obj585:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f13180 con 0x507ac00 2026-02-11T23:31:28.170+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.170+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:96183f21:::obj585:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.170+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:96d0cad0:::obj307:head, version: 111'307, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:96d0cad0:::obj307:head@111'307, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.170+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:96183f21:::obj585:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.170+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.170+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:96d0cad0:::obj307:head, version: 111'307, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:96d0cad0:::obj307:head@111'307, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 119/117 [PushOp(4:96d0cad0:::obj307:head, version: 111'307, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:96d0cad0:::obj307:head@111'307, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:9604ab91:::obj494:head local-lis/les=117/118 n=351 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.170+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.170+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:96183f21:::obj585:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.170+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9604ab91:::obj494:head local-lis/les=117/118 n=351 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:96d0cad0:::obj307:head, version: 111'307, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:96d0cad0:::obj307:head@111'307, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.170+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9604ab91:::obj494:head local-lis/les=117/118 n=351 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:96d0cad0:::obj307:head@111'307, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.170+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9604ab91:::obj494:head local-lis/les=117/118 n=351 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:96d0cad0:::obj307:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.170+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9604ab91:::obj494:head local-lis/les=117/118 n=351 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:96d0cad0:::obj307:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.170+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9604ab91:::obj494:head local-lis/les=117/118 n=351 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:96d0cad0:::obj307:head 2026-02-11T23:31:28.170+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9604ab91:::obj494:head local-lis/les=117/118 n=351 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:96d0cad0:::obj307:head v 111'307 2026-02-11T23:31:28.170+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9604ab91:::obj494:head local-lis/les=117/118 n=351 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.170+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.170+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:96d0cad0:::obj307:head, version: 111'307, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:96d0cad0:::obj307:head@111'307, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:28.170+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9604ab91:::obj494:head local-lis/les=117/118 n=351 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.170+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.170+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.170+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:96183f21:::obj585:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.170+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:96183f21:::obj585:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 4.0 e 119/119 lb 4:96183f21:::obj585:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:9604ab91:::obj494:head local-lis/les=117/118 n=351 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.170+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.170+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9604ab91:::obj494:head local-lis/les=117/118 n=351 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:96183f21:::obj585:head) v3 2026-02-11T23:31:28.170+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9604ab91:::obj494:head local-lis/les=117/118 n=351 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:96183f21:::obj585:head) 2026-02-11T23:31:28.170+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.170+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:96183f21:::obj585:head) v3 finish latency 0.000014 2026-02-11T23:31:28.170+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.170+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.170+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:96183f21:::obj585:head local-lis/les=117/118 n=352 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.170+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.170+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.170+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.170+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x640ef00 con 0x507ac00 2026-02-11T23:31:28.170+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:96d0cad0:::obj307:head)]) -- 0x642ca80 con 0x507ac00 2026-02-11T23:31:28.170+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.170+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 745 ==== MOSDPGPush(4.0 119/117 [PushOp(4:9720f72d:::obj90:head, version: 111'90, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9720f72d:::obj90:head@111'90, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x554e680 con 0x507ac00 2026-02-11T23:31:28.170+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:9720f72d:::obj90:head, version: 111'90, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9720f72d:::obj90:head@111'90, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.170+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:9720f72d:::obj90:head, version: 111'90, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9720f72d:::obj90:head@111'90, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.170+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.170+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 746 ==== pg_backfill(progress 4.0 e 119/119 lb 4:96d0cad0:::obj307:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f10b00 con 0x507ac00 2026-02-11T23:31:28.170+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:9720f72d:::obj90:head, version: 111'90, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9720f72d:::obj90:head@111'90, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.170+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:96d0cad0:::obj307:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.170+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:96d0cad0:::obj307:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.170+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.170+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:9720f72d:::obj90:head, version: 111'90, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9720f72d:::obj90:head@111'90, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.170+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:9720f72d:::obj90:head, version: 111'90, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9720f72d:::obj90:head@111'90, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(4.0 119/117 [PushOp(4:9720f72d:::obj90:head, version: 111'90, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9720f72d:::obj90:head@111'90, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:96183f21:::obj585:head local-lis/les=117/118 n=352 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.170+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.170+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.170+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:96183f21:::obj585:head local-lis/les=117/118 n=352 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:9720f72d:::obj90:head, version: 111'90, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9720f72d:::obj90:head@111'90, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.170+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:96d0cad0:::obj307:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.170+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:96183f21:::obj585:head local-lis/les=117/118 n=352 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:9720f72d:::obj90:head@111'90, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.170+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:96183f21:::obj585:head local-lis/les=117/118 n=352 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:9720f72d:::obj90:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.171+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:96183f21:::obj585:head local-lis/les=117/118 n=352 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:9720f72d:::obj90:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.171+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:96183f21:::obj585:head local-lis/les=117/118 n=352 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:9720f72d:::obj90:head 2026-02-11T23:31:28.171+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:96183f21:::obj585:head local-lis/les=117/118 n=352 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:9720f72d:::obj90:head v 111'90 2026-02-11T23:31:28.171+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:96183f21:::obj585:head local-lis/les=117/118 n=352 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.171+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.171+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:9720f72d:::obj90:head, version: 111'90, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9720f72d:::obj90:head@111'90, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:28.171+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:96183f21:::obj585:head local-lis/les=117/118 n=352 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.171+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.171+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:96d0cad0:::obj307:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.171+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.171+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:96d0cad0:::obj307:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 4.0 e 119/119 lb 4:96d0cad0:::obj307:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:96183f21:::obj585:head local-lis/les=117/118 n=352 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.171+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.171+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:96183f21:::obj585:head local-lis/les=117/118 n=352 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:96d0cad0:::obj307:head) v3 2026-02-11T23:31:28.171+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:96183f21:::obj585:head local-lis/les=117/118 n=352 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:96d0cad0:::obj307:head) 2026-02-11T23:31:28.171+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.171+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:96d0cad0:::obj307:head) v3 finish latency 0.000015 2026-02-11T23:31:28.171+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.171+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.171+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:96d0cad0:::obj307:head local-lis/les=117/118 n=353 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.171+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.171+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.171+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.171+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x642c780 con 0x507ac00 2026-02-11T23:31:28.171+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:9720f72d:::obj90:head)]) -- 0x640ea80 con 0x507ac00 2026-02-11T23:31:28.171+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.171+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 747 ==== MOSDPGPush(4.0 119/117 [PushOp(4:98019cfb:::obj381:head, version: 111'381, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:98019cfb:::obj381:head@111'381, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x555d040 con 0x507ac00 2026-02-11T23:31:28.171+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:98019cfb:::obj381:head, version: 111'381, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:98019cfb:::obj381:head@111'381, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.171+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:98019cfb:::obj381:head, version: 111'381, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:98019cfb:::obj381:head@111'381, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.171+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.171+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:98019cfb:::obj381:head, version: 111'381, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:98019cfb:::obj381:head@111'381, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.171+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 748 ==== pg_backfill(progress 4.0 e 119/119 lb 4:9720f72d:::obj90:head) ==== 1061+0+0 (unknown 0 0 0) 0x5f10580 con 0x507ac00 2026-02-11T23:31:28.171+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:9720f72d:::obj90:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.171+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.171+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:98019cfb:::obj381:head, version: 111'381, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:98019cfb:::obj381:head@111'381, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.171+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:9720f72d:::obj90:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.171+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:98019cfb:::obj381:head, version: 111'381, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:98019cfb:::obj381:head@111'381, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 119/117 [PushOp(4:98019cfb:::obj381:head, version: 111'381, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:98019cfb:::obj381:head@111'381, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:96d0cad0:::obj307:head local-lis/les=117/118 n=353 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.171+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.171+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.171+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:96d0cad0:::obj307:head local-lis/les=117/118 n=353 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:98019cfb:::obj381:head, version: 111'381, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:98019cfb:::obj381:head@111'381, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.171+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:9720f72d:::obj90:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.171+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:96d0cad0:::obj307:head local-lis/les=117/118 n=353 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:98019cfb:::obj381:head@111'381, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.171+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:96d0cad0:::obj307:head local-lis/les=117/118 n=353 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:98019cfb:::obj381:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.171+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:96d0cad0:::obj307:head local-lis/les=117/118 n=353 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:98019cfb:::obj381:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.171+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:96d0cad0:::obj307:head local-lis/les=117/118 n=353 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:98019cfb:::obj381:head 2026-02-11T23:31:28.171+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:96d0cad0:::obj307:head local-lis/les=117/118 n=353 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:98019cfb:::obj381:head v 111'381 2026-02-11T23:31:28.171+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:96d0cad0:::obj307:head local-lis/les=117/118 n=353 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.171+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.171+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:98019cfb:::obj381:head, version: 111'381, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:98019cfb:::obj381:head@111'381, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:28.171+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:96d0cad0:::obj307:head local-lis/les=117/118 n=353 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.171+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.171+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.171+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:9720f72d:::obj90:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.171+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:9720f72d:::obj90:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 4.0 e 119/119 lb 4:9720f72d:::obj90:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:96d0cad0:::obj307:head local-lis/les=117/118 n=353 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.171+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.171+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:96d0cad0:::obj307:head local-lis/les=117/118 n=353 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:9720f72d:::obj90:head) v3 2026-02-11T23:31:28.171+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:96d0cad0:::obj307:head local-lis/les=117/118 n=353 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:9720f72d:::obj90:head) 2026-02-11T23:31:28.171+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.171+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:9720f72d:::obj90:head) v3 finish latency 0.000013 2026-02-11T23:31:28.171+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.171+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.171+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9720f72d:::obj90:head local-lis/les=117/118 n=354 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.171+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.171+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.171+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.171+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x640f380 con 0x507ac00 2026-02-11T23:31:28.171+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:98019cfb:::obj381:head)]) -- 0x640f680 con 0x507ac00 2026-02-11T23:31:28.171+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.172+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 749 ==== MOSDPGPush(4.0 119/117 [PushOp(4:98b68518:::obj150:head, version: 111'150, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:98b68518:::obj150:head@111'150, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56aa000 con 0x507ac00 2026-02-11T23:31:28.172+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:98b68518:::obj150:head, version: 111'150, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:98b68518:::obj150:head@111'150, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.172+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:98b68518:::obj150:head, version: 111'150, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:98b68518:::obj150:head@111'150, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.172+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.172+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:98b68518:::obj150:head, version: 111'150, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:98b68518:::obj150:head@111'150, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.172+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 750 ==== pg_backfill(progress 4.0 e 119/119 lb 4:98019cfb:::obj381:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f10000 con 0x507ac00 2026-02-11T23:31:28.172+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:98019cfb:::obj381:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.172+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.172+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:98b68518:::obj150:head, version: 111'150, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:98b68518:::obj150:head@111'150, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.172+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:98019cfb:::obj381:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.172+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:98b68518:::obj150:head, version: 111'150, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:98b68518:::obj150:head@111'150, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 119/117 [PushOp(4:98b68518:::obj150:head, version: 111'150, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:98b68518:::obj150:head@111'150, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:9720f72d:::obj90:head local-lis/les=117/118 n=354 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.172+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.172+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.172+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:98019cfb:::obj381:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.172+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9720f72d:::obj90:head local-lis/les=117/118 n=354 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:98b68518:::obj150:head, version: 111'150, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:98b68518:::obj150:head@111'150, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.172+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9720f72d:::obj90:head local-lis/les=117/118 n=354 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:98b68518:::obj150:head@111'150, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.172+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9720f72d:::obj90:head local-lis/les=117/118 n=354 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:98b68518:::obj150:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.172+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9720f72d:::obj90:head local-lis/les=117/118 n=354 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:98b68518:::obj150:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.172+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9720f72d:::obj90:head local-lis/les=117/118 n=354 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:98b68518:::obj150:head 2026-02-11T23:31:28.172+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9720f72d:::obj90:head local-lis/les=117/118 n=354 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:98b68518:::obj150:head v 111'150 2026-02-11T23:31:28.172+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9720f72d:::obj90:head local-lis/les=117/118 n=354 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.172+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.172+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:98b68518:::obj150:head, version: 111'150, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:98b68518:::obj150:head@111'150, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:28.172+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9720f72d:::obj90:head local-lis/les=117/118 n=354 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.172+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.172+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.172+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:98019cfb:::obj381:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.172+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:98019cfb:::obj381:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 4.0 e 119/119 lb 4:98019cfb:::obj381:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:9720f72d:::obj90:head local-lis/les=117/118 n=354 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.172+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.172+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9720f72d:::obj90:head local-lis/les=117/118 n=354 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:98019cfb:::obj381:head) v3 2026-02-11T23:31:28.172+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9720f72d:::obj90:head local-lis/les=117/118 n=354 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:98019cfb:::obj381:head) 2026-02-11T23:31:28.172+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.172+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:98019cfb:::obj381:head) v3 finish latency 0.000014 2026-02-11T23:31:28.172+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.172+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.172+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:98019cfb:::obj381:head local-lis/les=117/118 n=355 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.172+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.172+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.172+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.172+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66d8000 con 0x507ac00 2026-02-11T23:31:28.172+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:98b68518:::obj150:head)]) -- 0x640f980 con 0x507ac00 2026-02-11T23:31:28.172+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.172+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 751 ==== MOSDPGPush(4.0 119/117 [PushOp(4:9956259d:::obj240:head, version: 111'240, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9956259d:::obj240:head@111'240, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2000 con 0x507ac00 2026-02-11T23:31:28.172+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:9956259d:::obj240:head, version: 111'240, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9956259d:::obj240:head@111'240, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.172+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:9956259d:::obj240:head, version: 111'240, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9956259d:::obj240:head@111'240, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.172+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.172+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:9956259d:::obj240:head, version: 111'240, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9956259d:::obj240:head@111'240, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.172+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 752 ==== pg_backfill(progress 4.0 e 119/119 lb 4:98b68518:::obj150:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f13700 con 0x507ac00 2026-02-11T23:31:28.172+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.172+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:98b68518:::obj150:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.172+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:9956259d:::obj240:head, version: 111'240, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9956259d:::obj240:head@111'240, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.172+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:98b68518:::obj150:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.172+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.172+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:9956259d:::obj240:head, version: 111'240, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9956259d:::obj240:head@111'240, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 119/117 [PushOp(4:9956259d:::obj240:head, version: 111'240, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9956259d:::obj240:head@111'240, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:98019cfb:::obj381:head local-lis/les=117/118 n=355 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.172+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.172+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:98b68518:::obj150:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.172+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:98019cfb:::obj381:head local-lis/les=117/118 n=355 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:9956259d:::obj240:head, version: 111'240, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9956259d:::obj240:head@111'240, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.172+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:98019cfb:::obj381:head local-lis/les=117/118 n=355 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:9956259d:::obj240:head@111'240, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.172+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:98019cfb:::obj381:head local-lis/les=117/118 n=355 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:9956259d:::obj240:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.172+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:98019cfb:::obj381:head local-lis/les=117/118 n=355 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:9956259d:::obj240:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.172+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:98019cfb:::obj381:head local-lis/les=117/118 n=355 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:9956259d:::obj240:head 2026-02-11T23:31:28.172+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:98019cfb:::obj381:head local-lis/les=117/118 n=355 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:9956259d:::obj240:head v 111'240 2026-02-11T23:31:28.172+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:98019cfb:::obj381:head local-lis/les=117/118 n=355 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.172+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.172+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:9956259d:::obj240:head, version: 111'240, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9956259d:::obj240:head@111'240, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:28.172+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:98019cfb:::obj381:head local-lis/les=117/118 n=355 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.172+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.172+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.172+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:98b68518:::obj150:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.172+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:98b68518:::obj150:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 4.0 e 119/119 lb 4:98b68518:::obj150:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:98019cfb:::obj381:head local-lis/les=117/118 n=355 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.172+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.172+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:98019cfb:::obj381:head local-lis/les=117/118 n=355 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:98b68518:::obj150:head) v3 2026-02-11T23:31:28.172+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:98019cfb:::obj381:head local-lis/les=117/118 n=355 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:98b68518:::obj150:head) 2026-02-11T23:31:28.172+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.172+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:98b68518:::obj150:head) v3 finish latency 0.000013 2026-02-11T23:31:28.172+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.172+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.172+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:98b68518:::obj150:head local-lis/les=117/118 n=356 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.172+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.172+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.172+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.172+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66d8480 con 0x507ac00 2026-02-11T23:31:28.172+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:9956259d:::obj240:head)]) -- 0x66d8180 con 0x507ac00 2026-02-11T23:31:28.172+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.173+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 753 ==== MOSDPGPush(4.0 119/117 [PushOp(4:996db15f:::obj326:head, version: 111'326, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:996db15f:::obj326:head@111'326, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f1d40 con 0x507ac00 2026-02-11T23:31:28.173+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:996db15f:::obj326:head, version: 111'326, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:996db15f:::obj326:head@111'326, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.173+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:996db15f:::obj326:head, version: 111'326, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:996db15f:::obj326:head@111'326, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.173+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.173+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:996db15f:::obj326:head, version: 111'326, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:996db15f:::obj326:head@111'326, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.173+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 754 ==== pg_backfill(progress 4.0 e 119/119 lb 4:9956259d:::obj240:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f12c00 con 0x507ac00 2026-02-11T23:31:28.173+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:9956259d:::obj240:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.173+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.173+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:996db15f:::obj326:head, version: 111'326, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:996db15f:::obj326:head@111'326, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.173+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:9956259d:::obj240:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.173+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.173+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:996db15f:::obj326:head, version: 111'326, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:996db15f:::obj326:head@111'326, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 119/117 [PushOp(4:996db15f:::obj326:head, version: 111'326, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:996db15f:::obj326:head@111'326, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:98b68518:::obj150:head local-lis/les=117/118 n=356 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.173+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:9956259d:::obj240:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.173+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.173+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:98b68518:::obj150:head local-lis/les=117/118 n=356 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:996db15f:::obj326:head, version: 111'326, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:996db15f:::obj326:head@111'326, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.173+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:98b68518:::obj150:head local-lis/les=117/118 n=356 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:996db15f:::obj326:head@111'326, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.173+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:98b68518:::obj150:head local-lis/les=117/118 n=356 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:996db15f:::obj326:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.173+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:98b68518:::obj150:head local-lis/les=117/118 n=356 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:996db15f:::obj326:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.173+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:98b68518:::obj150:head local-lis/les=117/118 n=356 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:996db15f:::obj326:head 2026-02-11T23:31:28.173+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:98b68518:::obj150:head local-lis/les=117/118 n=356 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:996db15f:::obj326:head v 111'326 2026-02-11T23:31:28.173+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:98b68518:::obj150:head local-lis/les=117/118 n=356 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.173+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.173+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:996db15f:::obj326:head, version: 111'326, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:996db15f:::obj326:head@111'326, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:28.173+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:98b68518:::obj150:head local-lis/les=117/118 n=356 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.173+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.173+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:9956259d:::obj240:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.173+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.173+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:9956259d:::obj240:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 4.0 e 119/119 lb 4:9956259d:::obj240:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:98b68518:::obj150:head local-lis/les=117/118 n=356 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.173+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.173+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:98b68518:::obj150:head local-lis/les=117/118 n=356 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:9956259d:::obj240:head) v3 2026-02-11T23:31:28.173+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:98b68518:::obj150:head local-lis/les=117/118 n=356 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:9956259d:::obj240:head) 2026-02-11T23:31:28.173+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.173+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:9956259d:::obj240:head) v3 finish latency 0.000014 2026-02-11T23:31:28.173+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.173+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.173+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9956259d:::obj240:head local-lis/les=117/118 n=357 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.173+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.173+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.173+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.173+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66d8900 con 0x507ac00 2026-02-11T23:31:28.173+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:996db15f:::obj326:head)]) -- 0x66d8600 con 0x507ac00 2026-02-11T23:31:28.173+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.173+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 755 ==== MOSDPGPush(4.0 119/117 [PushOp(4:99944c24:::obj170:head, version: 111'170, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:99944c24:::obj170:head@111'170, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x554e4e0 con 0x507ac00 2026-02-11T23:31:28.173+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:99944c24:::obj170:head, version: 111'170, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:99944c24:::obj170:head@111'170, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.173+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:99944c24:::obj170:head, version: 111'170, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:99944c24:::obj170:head@111'170, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.173+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.173+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:99944c24:::obj170:head, version: 111'170, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:99944c24:::obj170:head@111'170, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.173+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 756 ==== pg_backfill(progress 4.0 e 119/119 lb 4:996db15f:::obj326:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f12680 con 0x507ac00 2026-02-11T23:31:28.173+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.173+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:996db15f:::obj326:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.173+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:99944c24:::obj170:head, version: 111'170, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:99944c24:::obj170:head@111'170, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.173+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:996db15f:::obj326:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.173+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:99944c24:::obj170:head, version: 111'170, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:99944c24:::obj170:head@111'170, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 119/117 [PushOp(4:99944c24:::obj170:head, version: 111'170, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:99944c24:::obj170:head@111'170, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:9956259d:::obj240:head local-lis/les=117/118 n=357 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.173+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.173+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.173+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:996db15f:::obj326:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.173+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9956259d:::obj240:head local-lis/les=117/118 n=357 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:99944c24:::obj170:head, version: 111'170, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:99944c24:::obj170:head@111'170, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.173+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9956259d:::obj240:head local-lis/les=117/118 n=357 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:99944c24:::obj170:head@111'170, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.173+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9956259d:::obj240:head local-lis/les=117/118 n=357 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:99944c24:::obj170:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.173+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9956259d:::obj240:head local-lis/les=117/118 n=357 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:99944c24:::obj170:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.173+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9956259d:::obj240:head local-lis/les=117/118 n=357 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:99944c24:::obj170:head 2026-02-11T23:31:28.173+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9956259d:::obj240:head local-lis/les=117/118 n=357 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:99944c24:::obj170:head v 111'170 2026-02-11T23:31:28.173+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9956259d:::obj240:head local-lis/les=117/118 n=357 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.173+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.174+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:99944c24:::obj170:head, version: 111'170, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:99944c24:::obj170:head@111'170, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:28.174+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9956259d:::obj240:head local-lis/les=117/118 n=357 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.174+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.174+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:996db15f:::obj326:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.174+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.174+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:996db15f:::obj326:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 4.0 e 119/119 lb 4:996db15f:::obj326:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:9956259d:::obj240:head local-lis/les=117/118 n=357 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.174+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.174+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9956259d:::obj240:head local-lis/les=117/118 n=357 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:996db15f:::obj326:head) v3 2026-02-11T23:31:28.174+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9956259d:::obj240:head local-lis/les=117/118 n=357 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:996db15f:::obj326:head) 2026-02-11T23:31:28.174+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.174+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:996db15f:::obj326:head) v3 finish latency 0.000014 2026-02-11T23:31:28.174+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.174+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.174+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:996db15f:::obj326:head local-lis/les=117/118 n=358 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.174+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.174+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.174+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.174+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66d8d80 con 0x507ac00 2026-02-11T23:31:28.174+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:99944c24:::obj170:head)]) -- 0x66d8a80 con 0x507ac00 2026-02-11T23:31:28.174+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.174+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 757 ==== MOSDPGPush(4.0 119/117 [PushOp(4:9a09113a:::obj7:head, version: 111'7, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9a09113a:::obj7:head@111'7, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 4999+0+0 (unknown 0 0 0) 0x56a2820 con 0x507ac00 2026-02-11T23:31:28.174+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:9a09113a:::obj7:head, version: 111'7, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9a09113a:::obj7:head@111'7, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.174+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:9a09113a:::obj7:head, version: 111'7, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9a09113a:::obj7:head@111'7, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.174+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.174+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:9a09113a:::obj7:head, version: 111'7, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9a09113a:::obj7:head@111'7, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.174+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 758 ==== pg_backfill(progress 4.0 e 119/119 lb 4:99944c24:::obj170:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f12100 con 0x507ac00 2026-02-11T23:31:28.174+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:99944c24:::obj170:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.174+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.174+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:9a09113a:::obj7:head, version: 111'7, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9a09113a:::obj7:head@111'7, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.174+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:99944c24:::obj170:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.174+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.174+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:9a09113a:::obj7:head, version: 111'7, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9a09113a:::obj7:head@111'7, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 119/117 [PushOp(4:9a09113a:::obj7:head, version: 111'7, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9a09113a:::obj7:head@111'7, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:996db15f:::obj326:head local-lis/les=117/118 n=358 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.174+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.174+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:99944c24:::obj170:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.174+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:996db15f:::obj326:head local-lis/les=117/118 n=358 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:9a09113a:::obj7:head, version: 111'7, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9a09113a:::obj7:head@111'7, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.174+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:996db15f:::obj326:head local-lis/les=117/118 n=358 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:9a09113a:::obj7:head@111'7, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.174+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:996db15f:::obj326:head local-lis/les=117/118 n=358 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:9a09113a:::obj7:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.174+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:996db15f:::obj326:head local-lis/les=117/118 n=358 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:9a09113a:::obj7:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.174+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:996db15f:::obj326:head local-lis/les=117/118 n=358 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:9a09113a:::obj7:head 2026-02-11T23:31:28.174+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:996db15f:::obj326:head local-lis/les=117/118 n=358 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:9a09113a:::obj7:head v 111'7 2026-02-11T23:31:28.174+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:996db15f:::obj326:head local-lis/les=117/118 n=358 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.174+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.174+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:9a09113a:::obj7:head, version: 111'7, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9a09113a:::obj7:head@111'7, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:28.174+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:996db15f:::obj326:head local-lis/les=117/118 n=358 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.174+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.174+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.174+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:99944c24:::obj170:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.174+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:99944c24:::obj170:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 4.0 e 119/119 lb 4:99944c24:::obj170:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:996db15f:::obj326:head local-lis/les=117/118 n=358 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.174+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.174+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:996db15f:::obj326:head local-lis/les=117/118 n=358 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:99944c24:::obj170:head) v3 2026-02-11T23:31:28.174+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:996db15f:::obj326:head local-lis/les=117/118 n=358 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:99944c24:::obj170:head) 2026-02-11T23:31:28.174+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.174+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:99944c24:::obj170:head) v3 finish latency 0.000014 2026-02-11T23:31:28.174+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.174+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.174+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:99944c24:::obj170:head local-lis/les=117/118 n=359 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.174+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.174+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.174+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.174+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66d9080 con 0x507ac00 2026-02-11T23:31:28.174+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:9a09113a:::obj7:head)]) -- 0x66d8f00 con 0x507ac00 2026-02-11T23:31:28.174+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.175+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 759 ==== MOSDPGPush(4.0 119/117 [PushOp(4:9bbc8a19:::obj348:head, version: 111'348, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9bbc8a19:::obj348:head@111'348, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a3a00 con 0x507ac00 2026-02-11T23:31:28.175+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:9bbc8a19:::obj348:head, version: 111'348, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9bbc8a19:::obj348:head@111'348, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.175+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:9bbc8a19:::obj348:head, version: 111'348, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9bbc8a19:::obj348:head@111'348, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.175+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.175+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 760 ==== pg_backfill(progress 4.0 e 119/119 lb 4:9a09113a:::obj7:head) ==== 1060+0+0 (unknown 0 0 0) 0x5f11b80 con 0x507ac00 2026-02-11T23:31:28.175+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:9bbc8a19:::obj348:head, version: 111'348, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9bbc8a19:::obj348:head@111'348, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.175+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:9a09113a:::obj7:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.175+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.175+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:9a09113a:::obj7:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.175+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:9bbc8a19:::obj348:head, version: 111'348, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9bbc8a19:::obj348:head@111'348, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.175+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:9bbc8a19:::obj348:head, version: 111'348, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9bbc8a19:::obj348:head@111'348, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 119/117 [PushOp(4:9bbc8a19:::obj348:head, version: 111'348, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9bbc8a19:::obj348:head@111'348, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:99944c24:::obj170:head local-lis/les=117/118 n=359 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.175+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.175+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.175+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:99944c24:::obj170:head local-lis/les=117/118 n=359 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:9bbc8a19:::obj348:head, version: 111'348, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9bbc8a19:::obj348:head@111'348, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.175+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:99944c24:::obj170:head local-lis/les=117/118 n=359 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:9bbc8a19:::obj348:head@111'348, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.175+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.175+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:99944c24:::obj170:head local-lis/les=117/118 n=359 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:9bbc8a19:::obj348:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.175+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:9a09113a:::obj7:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.175+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:99944c24:::obj170:head local-lis/les=117/118 n=359 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:9bbc8a19:::obj348:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.175+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:99944c24:::obj170:head local-lis/les=117/118 n=359 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:9bbc8a19:::obj348:head 2026-02-11T23:31:28.175+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:99944c24:::obj170:head local-lis/les=117/118 n=359 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:9bbc8a19:::obj348:head v 111'348 2026-02-11T23:31:28.175+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:99944c24:::obj170:head local-lis/les=117/118 n=359 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.175+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.175+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:9bbc8a19:::obj348:head, version: 111'348, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9bbc8a19:::obj348:head@111'348, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:28.175+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:99944c24:::obj170:head local-lis/les=117/118 n=359 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.175+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.175+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.175+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:9a09113a:::obj7:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.175+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:9a09113a:::obj7:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 4.0 e 119/119 lb 4:9a09113a:::obj7:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:99944c24:::obj170:head local-lis/les=117/118 n=359 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.175+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.175+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:99944c24:::obj170:head local-lis/les=117/118 n=359 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:9a09113a:::obj7:head) v3 2026-02-11T23:31:28.175+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:99944c24:::obj170:head local-lis/les=117/118 n=359 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:9a09113a:::obj7:head) 2026-02-11T23:31:28.175+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.175+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:9a09113a:::obj7:head) v3 finish latency 0.000014 2026-02-11T23:31:28.175+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.175+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.175+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9a09113a:::obj7:head local-lis/les=117/118 n=360 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.175+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.175+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.175+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.175+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66d9500 con 0x507ac00 2026-02-11T23:31:28.175+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:9bbc8a19:::obj348:head)]) -- 0x66d9200 con 0x507ac00 2026-02-11T23:31:28.175+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.175+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 761 ==== MOSDPGPush(4.0 119/117 [PushOp(4:9c3090f1:::obj522:head, version: 111'522, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c3090f1:::obj522:head@111'522, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db0000 con 0x507ac00 2026-02-11T23:31:28.175+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:9c3090f1:::obj522:head, version: 111'522, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c3090f1:::obj522:head@111'522, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.175+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:9c3090f1:::obj522:head, version: 111'522, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c3090f1:::obj522:head@111'522, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.175+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.175+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:9c3090f1:::obj522:head, version: 111'522, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c3090f1:::obj522:head@111'522, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.175+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 762 ==== pg_backfill(progress 4.0 e 119/119 lb 4:9bbc8a19:::obj348:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f10b00 con 0x507ac00 2026-02-11T23:31:28.175+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.175+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:9c3090f1:::obj522:head, version: 111'522, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c3090f1:::obj522:head@111'522, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.175+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:9bbc8a19:::obj348:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.175+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:9bbc8a19:::obj348:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.175+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:9c3090f1:::obj522:head, version: 111'522, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c3090f1:::obj522:head@111'522, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 119/117 [PushOp(4:9c3090f1:::obj522:head, version: 111'522, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c3090f1:::obj522:head@111'522, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:9a09113a:::obj7:head local-lis/les=117/118 n=360 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.175+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.175+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.175+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:9bbc8a19:::obj348:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.175+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9a09113a:::obj7:head local-lis/les=117/118 n=360 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:9c3090f1:::obj522:head, version: 111'522, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c3090f1:::obj522:head@111'522, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.175+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9a09113a:::obj7:head local-lis/les=117/118 n=360 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:9c3090f1:::obj522:head@111'522, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.175+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9a09113a:::obj7:head local-lis/les=117/118 n=360 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:9c3090f1:::obj522:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.175+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9a09113a:::obj7:head local-lis/les=117/118 n=360 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:9c3090f1:::obj522:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.175+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9a09113a:::obj7:head local-lis/les=117/118 n=360 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:9c3090f1:::obj522:head 2026-02-11T23:31:28.175+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9a09113a:::obj7:head local-lis/les=117/118 n=360 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:9c3090f1:::obj522:head v 111'522 2026-02-11T23:31:28.175+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9a09113a:::obj7:head local-lis/les=117/118 n=360 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.175+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.175+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:9c3090f1:::obj522:head, version: 111'522, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c3090f1:::obj522:head@111'522, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:28.175+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9a09113a:::obj7:head local-lis/les=117/118 n=360 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.175+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.175+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.175+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:9bbc8a19:::obj348:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.175+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:9bbc8a19:::obj348:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 4.0 e 119/119 lb 4:9bbc8a19:::obj348:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:9a09113a:::obj7:head local-lis/les=117/118 n=360 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.175+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.175+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9a09113a:::obj7:head local-lis/les=117/118 n=360 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:9bbc8a19:::obj348:head) v3 2026-02-11T23:31:28.175+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9a09113a:::obj7:head local-lis/les=117/118 n=360 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:9bbc8a19:::obj348:head) 2026-02-11T23:31:28.175+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.175+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:9bbc8a19:::obj348:head) v3 finish latency 0.000018 2026-02-11T23:31:28.175+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.175+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.175+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9bbc8a19:::obj348:head local-lis/les=117/118 n=361 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.175+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.175+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.175+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.175+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66d9980 con 0x507ac00 2026-02-11T23:31:28.175+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.175+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:9c3090f1:::obj522:head)]) -- 0x66d9680 con 0x507ac00 2026-02-11T23:31:28.175+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.176+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 763 ==== MOSDPGPush(4.0 119/117 [PushOp(4:9c626db4:::obj119:head, version: 111'119, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c626db4:::obj119:head@111'119, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56caea0 con 0x507ac00 2026-02-11T23:31:28.176+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:9c626db4:::obj119:head, version: 111'119, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c626db4:::obj119:head@111'119, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.176+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:9c626db4:::obj119:head, version: 111'119, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c626db4:::obj119:head@111'119, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.176+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.176+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:9c626db4:::obj119:head, version: 111'119, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c626db4:::obj119:head@111'119, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.176+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 764 ==== pg_backfill(progress 4.0 e 119/119 lb 4:9c3090f1:::obj522:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f13180 con 0x507ac00 2026-02-11T23:31:28.176+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:9c3090f1:::obj522:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.176+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.176+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:9c626db4:::obj119:head, version: 111'119, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c626db4:::obj119:head@111'119, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.176+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:9c3090f1:::obj522:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.176+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.176+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:9c626db4:::obj119:head, version: 111'119, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c626db4:::obj119:head@111'119, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 119/117 [PushOp(4:9c626db4:::obj119:head, version: 111'119, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c626db4:::obj119:head@111'119, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:9bbc8a19:::obj348:head local-lis/les=117/118 n=361 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.176+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.176+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:9c3090f1:::obj522:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.176+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9bbc8a19:::obj348:head local-lis/les=117/118 n=361 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:9c626db4:::obj119:head, version: 111'119, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c626db4:::obj119:head@111'119, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.176+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9bbc8a19:::obj348:head local-lis/les=117/118 n=361 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:9c626db4:::obj119:head@111'119, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.176+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9bbc8a19:::obj348:head local-lis/les=117/118 n=361 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:9c626db4:::obj119:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.176+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9bbc8a19:::obj348:head local-lis/les=117/118 n=361 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:9c626db4:::obj119:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.176+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9bbc8a19:::obj348:head local-lis/les=117/118 n=361 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:9c626db4:::obj119:head 2026-02-11T23:31:28.176+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9bbc8a19:::obj348:head local-lis/les=117/118 n=361 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:9c626db4:::obj119:head v 111'119 2026-02-11T23:31:28.176+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9bbc8a19:::obj348:head local-lis/les=117/118 n=361 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.176+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.176+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:9c626db4:::obj119:head, version: 111'119, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c626db4:::obj119:head@111'119, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:28.176+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9bbc8a19:::obj348:head local-lis/les=117/118 n=361 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.176+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.176+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.176+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:9c3090f1:::obj522:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.176+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:9c3090f1:::obj522:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 4.0 e 119/119 lb 4:9c3090f1:::obj522:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:9bbc8a19:::obj348:head local-lis/les=117/118 n=361 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.176+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.176+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9bbc8a19:::obj348:head local-lis/les=117/118 n=361 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:9c3090f1:::obj522:head) v3 2026-02-11T23:31:28.176+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9bbc8a19:::obj348:head local-lis/les=117/118 n=361 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:9c3090f1:::obj522:head) 2026-02-11T23:31:28.176+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.176+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:9c3090f1:::obj522:head) v3 finish latency 0.000014 2026-02-11T23:31:28.176+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.176+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.176+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9c3090f1:::obj522:head local-lis/les=117/118 n=362 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.176+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.176+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.176+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.176+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66d9e00 con 0x507ac00 2026-02-11T23:31:28.176+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.176+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:9c626db4:::obj119:head)]) -- 0x66d9b00 con 0x507ac00 2026-02-11T23:31:28.176+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.177+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 765 ==== MOSDPGPush(4.0 119/117 [PushOp(4:9c9438f5:::obj513:head, version: 111'513, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c9438f5:::obj513:head@111'513, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56b64e0 con 0x507ac00 2026-02-11T23:31:28.177+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:9c9438f5:::obj513:head, version: 111'513, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c9438f5:::obj513:head@111'513, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 119 2026-02-11T23:31:28.177+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:9c9438f5:::obj513:head, version: 111'513, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c9438f5:::obj513:head@111'513, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.177+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.177+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 766 ==== pg_backfill(progress 4.0 e 119/119 lb 4:9c626db4:::obj119:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f11080 con 0x507ac00 2026-02-11T23:31:28.177+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:9c9438f5:::obj513:head, version: 111'513, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c9438f5:::obj513:head@111'513, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.177+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:9c626db4:::obj119:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.177+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:9c626db4:::obj119:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.177+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.177+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:9c9438f5:::obj513:head, version: 111'513, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c9438f5:::obj513:head@111'513, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.177+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:9c9438f5:::obj513:head, version: 111'513, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c9438f5:::obj513:head@111'513, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 119/117 [PushOp(4:9c9438f5:::obj513:head, version: 111'513, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c9438f5:::obj513:head@111'513, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:9c3090f1:::obj522:head local-lis/les=117/118 n=362 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.177+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.177+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.177+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:9c626db4:::obj119:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.177+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9c3090f1:::obj522:head local-lis/les=117/118 n=362 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:9c9438f5:::obj513:head, version: 111'513, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c9438f5:::obj513:head@111'513, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.177+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9c3090f1:::obj522:head local-lis/les=117/118 n=362 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:9c9438f5:::obj513:head@111'513, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.177+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9c3090f1:::obj522:head local-lis/les=117/118 n=362 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:9c9438f5:::obj513:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.177+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9c3090f1:::obj522:head local-lis/les=117/118 n=362 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:9c9438f5:::obj513:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.177+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9c3090f1:::obj522:head local-lis/les=117/118 n=362 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:9c9438f5:::obj513:head 2026-02-11T23:31:28.177+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9c3090f1:::obj522:head local-lis/les=117/118 n=362 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:9c9438f5:::obj513:head v 111'513 2026-02-11T23:31:28.177+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9c3090f1:::obj522:head local-lis/les=117/118 n=362 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.177+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.177+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:9c9438f5:::obj513:head, version: 111'513, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c9438f5:::obj513:head@111'513, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:28.177+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9c3090f1:::obj522:head local-lis/les=117/118 n=362 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.177+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.177+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:9c626db4:::obj119:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.177+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.177+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:9c626db4:::obj119:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 4.0 e 119/119 lb 4:9c626db4:::obj119:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:9c3090f1:::obj522:head local-lis/les=117/118 n=362 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.177+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.177+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9c3090f1:::obj522:head local-lis/les=117/118 n=362 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:9c626db4:::obj119:head) v3 2026-02-11T23:31:28.177+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9c3090f1:::obj522:head local-lis/les=117/118 n=362 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:9c626db4:::obj119:head) 2026-02-11T23:31:28.177+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.177+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:9c626db4:::obj119:head) v3 finish latency 0.000018 2026-02-11T23:31:28.177+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.177+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.177+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9c626db4:::obj119:head local-lis/les=117/118 n=363 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.177+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.177+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.177+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.177+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x640f680 con 0x507ac00 2026-02-11T23:31:28.177+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.177+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:9c9438f5:::obj513:head)]) -- 0x640f980 con 0x507ac00 2026-02-11T23:31:28.177+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.177+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 767 ==== MOSDPGPush(4.0 119/117 [PushOp(4:9f41ce81:::obj424:head, version: 111'424, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f41ce81:::obj424:head@111'424, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f0d00 con 0x507ac00 2026-02-11T23:31:28.177+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:9f41ce81:::obj424:head, version: 111'424, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f41ce81:::obj424:head@111'424, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.177+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:9f41ce81:::obj424:head, version: 111'424, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f41ce81:::obj424:head@111'424, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.177+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.177+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:9f41ce81:::obj424:head, version: 111'424, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f41ce81:::obj424:head@111'424, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.177+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.177+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:9f41ce81:::obj424:head, version: 111'424, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f41ce81:::obj424:head@111'424, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.177+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 768 ==== pg_backfill(progress 4.0 e 119/119 lb 4:9c9438f5:::obj513:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f11600 con 0x507ac00 2026-02-11T23:31:28.177+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:9c9438f5:::obj513:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.177+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.177+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:9f41ce81:::obj424:head, version: 111'424, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f41ce81:::obj424:head@111'424, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 119/117 [PushOp(4:9f41ce81:::obj424:head, version: 111'424, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f41ce81:::obj424:head@111'424, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:9c626db4:::obj119:head local-lis/les=117/118 n=363 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.177+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:9c9438f5:::obj513:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.177+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.177+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9c626db4:::obj119:head local-lis/les=117/118 n=363 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:9f41ce81:::obj424:head, version: 111'424, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f41ce81:::obj424:head@111'424, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.177+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9c626db4:::obj119:head local-lis/les=117/118 n=363 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:9f41ce81:::obj424:head@111'424, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.177+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.177+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:9c9438f5:::obj513:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.177+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9c626db4:::obj119:head local-lis/les=117/118 n=363 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:9f41ce81:::obj424:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.177+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9c626db4:::obj119:head local-lis/les=117/118 n=363 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:9f41ce81:::obj424:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.177+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9c626db4:::obj119:head local-lis/les=117/118 n=363 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:9f41ce81:::obj424:head 2026-02-11T23:31:28.177+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9c626db4:::obj119:head local-lis/les=117/118 n=363 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:9f41ce81:::obj424:head v 111'424 2026-02-11T23:31:28.177+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9c626db4:::obj119:head local-lis/les=117/118 n=363 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.177+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.177+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:9f41ce81:::obj424:head, version: 111'424, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f41ce81:::obj424:head@111'424, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:28.177+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9c626db4:::obj119:head local-lis/les=117/118 n=363 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.177+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.177+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:9c9438f5:::obj513:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.177+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.177+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:9c9438f5:::obj513:head) v3 prio 15 cost 0 latency 0.000057 pg_backfill(progress 4.0 e 119/119 lb 4:9c9438f5:::obj513:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:9c626db4:::obj119:head local-lis/les=117/118 n=363 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.177+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.177+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9c626db4:::obj119:head local-lis/les=117/118 n=363 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:9c9438f5:::obj513:head) v3 2026-02-11T23:31:28.177+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9c626db4:::obj119:head local-lis/les=117/118 n=363 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:9c9438f5:::obj513:head) 2026-02-11T23:31:28.177+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.177+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:9c9438f5:::obj513:head) v3 finish latency 0.000016 2026-02-11T23:31:28.177+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.177+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.177+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9c9438f5:::obj513:head local-lis/les=117/118 n=364 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.177+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.177+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.177+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.177+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x642c780 con 0x507ac00 2026-02-11T23:31:28.177+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:9f41ce81:::obj424:head)]) -- 0x640f380 con 0x507ac00 2026-02-11T23:31:28.177+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.178+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 769 ==== MOSDPGPush(4.0 119/117 [PushOp(4:9f7e105e:::obj535:head, version: 111'535, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f7e105e:::obj535:head@111'535, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c4e0 con 0x507ac00 2026-02-11T23:31:28.178+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:9f7e105e:::obj535:head, version: 111'535, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f7e105e:::obj535:head@111'535, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.178+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:9f7e105e:::obj535:head, version: 111'535, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f7e105e:::obj535:head@111'535, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.178+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.178+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:9f7e105e:::obj535:head, version: 111'535, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f7e105e:::obj535:head@111'535, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.178+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 770 ==== pg_backfill(progress 4.0 e 119/119 lb 4:9f41ce81:::obj424:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2f700 con 0x507ac00 2026-02-11T23:31:28.178+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:9f41ce81:::obj424:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.178+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.178+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:9f7e105e:::obj535:head, version: 111'535, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f7e105e:::obj535:head@111'535, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.178+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:9f41ce81:::obj424:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.178+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.178+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:9f41ce81:::obj424:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.178+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:9f7e105e:::obj535:head, version: 111'535, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f7e105e:::obj535:head@111'535, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 119/117 [PushOp(4:9f7e105e:::obj535:head, version: 111'535, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f7e105e:::obj535:head@111'535, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:9c9438f5:::obj513:head local-lis/les=117/118 n=364 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.178+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.178+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9c9438f5:::obj513:head local-lis/les=117/118 n=364 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:9f7e105e:::obj535:head, version: 111'535, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f7e105e:::obj535:head@111'535, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.178+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9c9438f5:::obj513:head local-lis/les=117/118 n=364 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:9f7e105e:::obj535:head@111'535, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.178+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9c9438f5:::obj513:head local-lis/les=117/118 n=364 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:9f7e105e:::obj535:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.178+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9c9438f5:::obj513:head local-lis/les=117/118 n=364 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:9f7e105e:::obj535:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.178+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9c9438f5:::obj513:head local-lis/les=117/118 n=364 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:9f7e105e:::obj535:head 2026-02-11T23:31:28.178+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9c9438f5:::obj513:head local-lis/les=117/118 n=364 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:9f7e105e:::obj535:head v 111'535 2026-02-11T23:31:28.178+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9c9438f5:::obj513:head local-lis/les=117/118 n=364 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.178+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.178+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:9f7e105e:::obj535:head, version: 111'535, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f7e105e:::obj535:head@111'535, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:28.178+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9c9438f5:::obj513:head local-lis/les=117/118 n=364 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.178+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.178+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.178+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:9f41ce81:::obj424:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.178+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:9f41ce81:::obj424:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 4.0 e 119/119 lb 4:9f41ce81:::obj424:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:9c9438f5:::obj513:head local-lis/les=117/118 n=364 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.178+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.178+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9c9438f5:::obj513:head local-lis/les=117/118 n=364 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:9f41ce81:::obj424:head) v3 2026-02-11T23:31:28.178+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9c9438f5:::obj513:head local-lis/les=117/118 n=364 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:9f41ce81:::obj424:head) 2026-02-11T23:31:28.178+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.178+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:9f41ce81:::obj424:head) v3 finish latency 0.000014 2026-02-11T23:31:28.178+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.178+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.178+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9f41ce81:::obj424:head local-lis/les=117/118 n=365 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.178+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.178+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.178+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.178+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c9e00 con 0x507ac00 2026-02-11T23:31:28.178+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:9f7e105e:::obj535:head)]) -- 0x642ca80 con 0x507ac00 2026-02-11T23:31:28.178+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.178+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 771 ==== MOSDPGPush(4.0 119/117 [PushOp(4:9f8e934c:::obj456:head, version: 111'456, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f8e934c:::obj456:head@111'456, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3814ea0 con 0x507ac00 2026-02-11T23:31:28.178+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:9f8e934c:::obj456:head, version: 111'456, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f8e934c:::obj456:head@111'456, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.178+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:9f8e934c:::obj456:head, version: 111'456, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f8e934c:::obj456:head@111'456, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.178+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.178+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:9f8e934c:::obj456:head, version: 111'456, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f8e934c:::obj456:head@111'456, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.178+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 772 ==== pg_backfill(progress 4.0 e 119/119 lb 4:9f7e105e:::obj535:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2d080 con 0x507ac00 2026-02-11T23:31:28.178+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.178+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:9f7e105e:::obj535:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.178+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:9f8e934c:::obj456:head, version: 111'456, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f8e934c:::obj456:head@111'456, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.178+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:9f7e105e:::obj535:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.178+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:9f8e934c:::obj456:head, version: 111'456, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f8e934c:::obj456:head@111'456, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 119/117 [PushOp(4:9f8e934c:::obj456:head, version: 111'456, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f8e934c:::obj456:head@111'456, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:9f41ce81:::obj424:head local-lis/les=117/118 n=365 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.178+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.178+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9f41ce81:::obj424:head local-lis/les=117/118 n=365 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:9f8e934c:::obj456:head, version: 111'456, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f8e934c:::obj456:head@111'456, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.178+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.178+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9f41ce81:::obj424:head local-lis/les=117/118 n=365 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:9f8e934c:::obj456:head@111'456, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.178+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:9f7e105e:::obj535:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.178+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9f41ce81:::obj424:head local-lis/les=117/118 n=365 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:9f8e934c:::obj456:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.178+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9f41ce81:::obj424:head local-lis/les=117/118 n=365 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:9f8e934c:::obj456:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.178+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9f41ce81:::obj424:head local-lis/les=117/118 n=365 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:9f8e934c:::obj456:head 2026-02-11T23:31:28.178+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9f41ce81:::obj424:head local-lis/les=117/118 n=365 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:9f8e934c:::obj456:head v 111'456 2026-02-11T23:31:28.178+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9f41ce81:::obj424:head local-lis/les=117/118 n=365 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.178+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.178+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:9f8e934c:::obj456:head, version: 111'456, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f8e934c:::obj456:head@111'456, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:28.178+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9f41ce81:::obj424:head local-lis/les=117/118 n=365 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.178+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.178+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.178+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:9f7e105e:::obj535:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.178+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:9f7e105e:::obj535:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 4.0 e 119/119 lb 4:9f7e105e:::obj535:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:9f41ce81:::obj424:head local-lis/les=117/118 n=365 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.178+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.178+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9f41ce81:::obj424:head local-lis/les=117/118 n=365 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:9f7e105e:::obj535:head) v3 2026-02-11T23:31:28.178+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9f41ce81:::obj424:head local-lis/les=117/118 n=365 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:9f7e105e:::obj535:head) 2026-02-11T23:31:28.178+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.178+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:9f7e105e:::obj535:head) v3 finish latency 0.000013 2026-02-11T23:31:28.178+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.178+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.178+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9f7e105e:::obj535:head local-lis/les=117/118 n=366 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.178+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.178+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.178+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.178+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x5d55800 con 0x507ac00 2026-02-11T23:31:28.178+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:9f8e934c:::obj456:head)]) -- 0x640e900 con 0x507ac00 2026-02-11T23:31:28.178+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.179+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 773 ==== MOSDPGPush(4.0 119/117 [PushOp(4:a0060720:::obj493:head, version: 111'493, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0060720:::obj493:head@111'493, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a3ba0 con 0x507ac00 2026-02-11T23:31:28.179+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:a0060720:::obj493:head, version: 111'493, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0060720:::obj493:head@111'493, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.179+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a0060720:::obj493:head, version: 111'493, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0060720:::obj493:head@111'493, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.179+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.179+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a0060720:::obj493:head, version: 111'493, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0060720:::obj493:head@111'493, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.179+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 774 ==== pg_backfill(progress 4.0 e 119/119 lb 4:9f8e934c:::obj456:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e680 con 0x507ac00 2026-02-11T23:31:28.179+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:9f8e934c:::obj456:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.179+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.179+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a0060720:::obj493:head, version: 111'493, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0060720:::obj493:head@111'493, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.179+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:9f8e934c:::obj456:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.179+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a0060720:::obj493:head, version: 111'493, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0060720:::obj493:head@111'493, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 119/117 [PushOp(4:a0060720:::obj493:head, version: 111'493, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0060720:::obj493:head@111'493, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:9f7e105e:::obj535:head local-lis/les=117/118 n=366 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.179+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.179+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.179+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9f7e105e:::obj535:head local-lis/les=117/118 n=366 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:a0060720:::obj493:head, version: 111'493, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0060720:::obj493:head@111'493, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.179+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:9f8e934c:::obj456:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.179+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9f7e105e:::obj535:head local-lis/les=117/118 n=366 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a0060720:::obj493:head@111'493, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.179+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9f7e105e:::obj535:head local-lis/les=117/118 n=366 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:a0060720:::obj493:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.179+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9f7e105e:::obj535:head local-lis/les=117/118 n=366 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:a0060720:::obj493:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.179+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9f7e105e:::obj535:head local-lis/les=117/118 n=366 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:a0060720:::obj493:head 2026-02-11T23:31:28.179+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9f7e105e:::obj535:head local-lis/les=117/118 n=366 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:a0060720:::obj493:head v 111'493 2026-02-11T23:31:28.179+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9f7e105e:::obj535:head local-lis/les=117/118 n=366 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.179+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.179+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a0060720:::obj493:head, version: 111'493, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0060720:::obj493:head@111'493, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:28.179+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9f7e105e:::obj535:head local-lis/les=117/118 n=366 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.179+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.179+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.179+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:9f8e934c:::obj456:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.179+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:9f8e934c:::obj456:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 4.0 e 119/119 lb 4:9f8e934c:::obj456:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:9f7e105e:::obj535:head local-lis/les=117/118 n=366 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.179+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.179+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9f7e105e:::obj535:head local-lis/les=117/118 n=366 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:9f8e934c:::obj456:head) v3 2026-02-11T23:31:28.179+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9f7e105e:::obj535:head local-lis/les=117/118 n=366 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:9f8e934c:::obj456:head) 2026-02-11T23:31:28.179+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.179+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:9f8e934c:::obj456:head) v3 finish latency 0.000013 2026-02-11T23:31:28.179+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.179+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.179+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9f8e934c:::obj456:head local-lis/les=117/118 n=367 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.179+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.179+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.179+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.179+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x5d55200 con 0x507ac00 2026-02-11T23:31:28.179+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:a0060720:::obj493:head)]) -- 0x640fb00 con 0x507ac00 2026-02-11T23:31:28.179+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.179+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 775 ==== MOSDPGPush(4.0 119/117 [PushOp(4:a0217245:::obj31:head, version: 111'31, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0217245:::obj31:head@111'31, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x38156c0 con 0x507ac00 2026-02-11T23:31:28.179+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:a0217245:::obj31:head, version: 111'31, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0217245:::obj31:head@111'31, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.179+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a0217245:::obj31:head, version: 111'31, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0217245:::obj31:head@111'31, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.179+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.179+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a0217245:::obj31:head, version: 111'31, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0217245:::obj31:head@111'31, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.179+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 776 ==== pg_backfill(progress 4.0 e 119/119 lb 4:a0060720:::obj493:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176580 con 0x507ac00 2026-02-11T23:31:28.179+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:a0060720:::obj493:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.179+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.179+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a0217245:::obj31:head, version: 111'31, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0217245:::obj31:head@111'31, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.179+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a0060720:::obj493:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.179+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.179+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a0217245:::obj31:head, version: 111'31, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0217245:::obj31:head@111'31, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 119/117 [PushOp(4:a0217245:::obj31:head, version: 111'31, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0217245:::obj31:head@111'31, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:9f8e934c:::obj456:head local-lis/les=117/118 n=367 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.179+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.179+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a0060720:::obj493:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.179+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9f8e934c:::obj456:head local-lis/les=117/118 n=367 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:a0217245:::obj31:head, version: 111'31, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0217245:::obj31:head@111'31, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.179+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9f8e934c:::obj456:head local-lis/les=117/118 n=367 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a0217245:::obj31:head@111'31, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.179+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9f8e934c:::obj456:head local-lis/les=117/118 n=367 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:a0217245:::obj31:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.179+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9f8e934c:::obj456:head local-lis/les=117/118 n=367 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:a0217245:::obj31:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.179+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9f8e934c:::obj456:head local-lis/les=117/118 n=367 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:a0217245:::obj31:head 2026-02-11T23:31:28.179+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9f8e934c:::obj456:head local-lis/les=117/118 n=367 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:a0217245:::obj31:head v 111'31 2026-02-11T23:31:28.179+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9f8e934c:::obj456:head local-lis/les=117/118 n=367 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.179+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.179+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a0217245:::obj31:head, version: 111'31, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0217245:::obj31:head@111'31, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:28.179+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9f8e934c:::obj456:head local-lis/les=117/118 n=367 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.179+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.179+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.179+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a0060720:::obj493:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.179+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a0060720:::obj493:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 4.0 e 119/119 lb 4:a0060720:::obj493:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:9f8e934c:::obj456:head local-lis/les=117/118 n=367 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.179+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.179+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9f8e934c:::obj456:head local-lis/les=117/118 n=367 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:a0060720:::obj493:head) v3 2026-02-11T23:31:28.179+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:9f8e934c:::obj456:head local-lis/les=117/118 n=367 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:a0060720:::obj493:head) 2026-02-11T23:31:28.179+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.179+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a0060720:::obj493:head) v3 finish latency 0.000013 2026-02-11T23:31:28.179+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.180+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.180+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a0060720:::obj493:head local-lis/les=117/118 n=368 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.180+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.180+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.180+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.180+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x644d380 con 0x507ac00 2026-02-11T23:31:28.180+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:a0217245:::obj31:head)]) -- 0x642cd80 con 0x507ac00 2026-02-11T23:31:28.180+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.180+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 777 ==== MOSDPGPush(4.0 119/117 [PushOp(4:a0e9da24:::obj323:head, version: 111'323, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0e9da24:::obj323:head@111'323, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e26680 con 0x507ac00 2026-02-11T23:31:28.180+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:a0e9da24:::obj323:head, version: 111'323, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0e9da24:::obj323:head@111'323, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.180+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a0e9da24:::obj323:head, version: 111'323, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0e9da24:::obj323:head@111'323, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.180+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.180+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a0e9da24:::obj323:head, version: 111'323, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0e9da24:::obj323:head@111'323, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.180+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 778 ==== pg_backfill(progress 4.0 e 119/119 lb 4:a0217245:::obj31:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d2e100 con 0x507ac00 2026-02-11T23:31:28.180+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:a0217245:::obj31:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.180+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.180+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a0e9da24:::obj323:head, version: 111'323, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0e9da24:::obj323:head@111'323, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.180+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a0217245:::obj31:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.180+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a0e9da24:::obj323:head, version: 111'323, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0e9da24:::obj323:head@111'323, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 119/117 [PushOp(4:a0e9da24:::obj323:head, version: 111'323, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0e9da24:::obj323:head@111'323, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a0060720:::obj493:head local-lis/les=117/118 n=368 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.180+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.180+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.180+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a0217245:::obj31:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.180+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a0060720:::obj493:head local-lis/les=117/118 n=368 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:a0e9da24:::obj323:head, version: 111'323, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0e9da24:::obj323:head@111'323, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.180+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a0060720:::obj493:head local-lis/les=117/118 n=368 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a0e9da24:::obj323:head@111'323, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.180+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a0060720:::obj493:head local-lis/les=117/118 n=368 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:a0e9da24:::obj323:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.180+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a0060720:::obj493:head local-lis/les=117/118 n=368 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:a0e9da24:::obj323:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.180+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a0060720:::obj493:head local-lis/les=117/118 n=368 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:a0e9da24:::obj323:head 2026-02-11T23:31:28.180+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a0060720:::obj493:head local-lis/les=117/118 n=368 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:a0e9da24:::obj323:head v 111'323 2026-02-11T23:31:28.180+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a0060720:::obj493:head local-lis/les=117/118 n=368 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.180+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.180+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a0e9da24:::obj323:head, version: 111'323, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0e9da24:::obj323:head@111'323, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:28.180+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a0060720:::obj493:head local-lis/les=117/118 n=368 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.180+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.180+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.180+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a0217245:::obj31:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.180+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a0217245:::obj31:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 4.0 e 119/119 lb 4:a0217245:::obj31:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a0060720:::obj493:head local-lis/les=117/118 n=368 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.180+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.180+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a0060720:::obj493:head local-lis/les=117/118 n=368 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:a0217245:::obj31:head) v3 2026-02-11T23:31:28.180+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a0060720:::obj493:head local-lis/les=117/118 n=368 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:a0217245:::obj31:head) 2026-02-11T23:31:28.180+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.180+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a0217245:::obj31:head) v3 finish latency 0.000014 2026-02-11T23:31:28.180+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.180+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.180+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a0217245:::obj31:head local-lis/les=117/118 n=369 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.180+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.180+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.180+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.180+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x640e180 con 0x507ac00 2026-02-11T23:31:28.180+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:a0e9da24:::obj323:head)]) -- 0x640fe00 con 0x507ac00 2026-02-11T23:31:28.180+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.181+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 779 ==== MOSDPGPush(4.0 119/117 [PushOp(4:a10cad2d:::obj340:head, version: 111'340, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a10cad2d:::obj340:head@111'340, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a3860 con 0x507ac00 2026-02-11T23:31:28.181+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:a10cad2d:::obj340:head, version: 111'340, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a10cad2d:::obj340:head@111'340, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.181+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a10cad2d:::obj340:head, version: 111'340, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a10cad2d:::obj340:head@111'340, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.181+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.181+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a10cad2d:::obj340:head, version: 111'340, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a10cad2d:::obj340:head@111'340, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.181+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 780 ==== pg_backfill(progress 4.0 e 119/119 lb 4:a0e9da24:::obj323:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2c000 con 0x507ac00 2026-02-11T23:31:28.181+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:a0e9da24:::obj323:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.181+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.181+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a10cad2d:::obj340:head, version: 111'340, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a10cad2d:::obj340:head@111'340, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.181+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.181+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a10cad2d:::obj340:head, version: 111'340, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a10cad2d:::obj340:head@111'340, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 119/117 [PushOp(4:a10cad2d:::obj340:head, version: 111'340, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a10cad2d:::obj340:head@111'340, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a0217245:::obj31:head local-lis/les=117/118 n=369 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.181+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.181+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a0e9da24:::obj323:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.181+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a0217245:::obj31:head local-lis/les=117/118 n=369 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:a10cad2d:::obj340:head, version: 111'340, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a10cad2d:::obj340:head@111'340, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.181+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a0217245:::obj31:head local-lis/les=117/118 n=369 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a10cad2d:::obj340:head@111'340, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.181+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.181+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a0e9da24:::obj323:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.181+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a0217245:::obj31:head local-lis/les=117/118 n=369 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:a10cad2d:::obj340:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.181+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a0217245:::obj31:head local-lis/les=117/118 n=369 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:a10cad2d:::obj340:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.181+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a0217245:::obj31:head local-lis/les=117/118 n=369 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:a10cad2d:::obj340:head 2026-02-11T23:31:28.181+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a0217245:::obj31:head local-lis/les=117/118 n=369 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:a10cad2d:::obj340:head v 111'340 2026-02-11T23:31:28.181+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a0217245:::obj31:head local-lis/les=117/118 n=369 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.181+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.181+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a10cad2d:::obj340:head, version: 111'340, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a10cad2d:::obj340:head@111'340, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:28.181+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a0217245:::obj31:head local-lis/les=117/118 n=369 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.181+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.181+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.181+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a0e9da24:::obj323:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.181+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a0e9da24:::obj323:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 4.0 e 119/119 lb 4:a0e9da24:::obj323:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a0217245:::obj31:head local-lis/les=117/118 n=369 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.181+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.181+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a0217245:::obj31:head local-lis/les=117/118 n=369 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:a0e9da24:::obj323:head) v3 2026-02-11T23:31:28.181+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a0217245:::obj31:head local-lis/les=117/118 n=369 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:a0e9da24:::obj323:head) 2026-02-11T23:31:28.181+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.181+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a0e9da24:::obj323:head) v3 finish latency 0.000014 2026-02-11T23:31:28.181+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.181+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.181+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a0e9da24:::obj323:head local-lis/les=117/118 n=370 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.181+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.181+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.181+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.181+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63ffb00 con 0x507ac00 2026-02-11T23:31:28.181+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:a10cad2d:::obj340:head)]) -- 0x649e600 con 0x507ac00 2026-02-11T23:31:28.181+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.181+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 781 ==== MOSDPGPush(4.0 119/117 [PushOp(4:a16a9a6a:::obj427:head, version: 111'427, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a16a9a6a:::obj427:head@111'427, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56cab60 con 0x507ac00 2026-02-11T23:31:28.181+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:a16a9a6a:::obj427:head, version: 111'427, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a16a9a6a:::obj427:head@111'427, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.181+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a16a9a6a:::obj427:head, version: 111'427, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a16a9a6a:::obj427:head@111'427, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.181+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.181+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 782 ==== pg_backfill(progress 4.0 e 119/119 lb 4:a10cad2d:::obj340:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2cb00 con 0x507ac00 2026-02-11T23:31:28.181+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a16a9a6a:::obj427:head, version: 111'427, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a16a9a6a:::obj427:head@111'427, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.181+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:a10cad2d:::obj340:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.181+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a10cad2d:::obj340:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.181+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.181+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a16a9a6a:::obj427:head, version: 111'427, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a16a9a6a:::obj427:head@111'427, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.181+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.181+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a16a9a6a:::obj427:head, version: 111'427, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a16a9a6a:::obj427:head@111'427, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 119/117 [PushOp(4:a16a9a6a:::obj427:head, version: 111'427, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a16a9a6a:::obj427:head@111'427, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a0e9da24:::obj323:head local-lis/les=117/118 n=370 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.181+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.181+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.181+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a10cad2d:::obj340:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.181+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a0e9da24:::obj323:head local-lis/les=117/118 n=370 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:a16a9a6a:::obj427:head, version: 111'427, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a16a9a6a:::obj427:head@111'427, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.181+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a0e9da24:::obj323:head local-lis/les=117/118 n=370 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a16a9a6a:::obj427:head@111'427, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.181+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a0e9da24:::obj323:head local-lis/les=117/118 n=370 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:a16a9a6a:::obj427:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.181+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a0e9da24:::obj323:head local-lis/les=117/118 n=370 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:a16a9a6a:::obj427:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.181+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a0e9da24:::obj323:head local-lis/les=117/118 n=370 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:a16a9a6a:::obj427:head 2026-02-11T23:31:28.181+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a0e9da24:::obj323:head local-lis/les=117/118 n=370 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:a16a9a6a:::obj427:head v 111'427 2026-02-11T23:31:28.181+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a0e9da24:::obj323:head local-lis/les=117/118 n=370 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.181+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.181+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a16a9a6a:::obj427:head, version: 111'427, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a16a9a6a:::obj427:head@111'427, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000071 2026-02-11T23:31:28.181+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.181+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.181+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a10cad2d:::obj340:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.181+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a10cad2d:::obj340:head) v3 prio 15 cost 0 latency 0.000092 pg_backfill(progress 4.0 e 119/119 lb 4:a10cad2d:::obj340:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a0e9da24:::obj323:head local-lis/les=117/118 n=370 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.181+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.181+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a0e9da24:::obj323:head local-lis/les=117/118 n=370 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:a10cad2d:::obj340:head) v3 2026-02-11T23:31:28.181+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a0e9da24:::obj323:head local-lis/les=117/118 n=370 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:a10cad2d:::obj340:head) 2026-02-11T23:31:28.181+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.181+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a10cad2d:::obj340:head) v3 finish latency 0.000012 2026-02-11T23:31:28.181+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a10cad2d:::obj340:head local-lis/les=117/118 n=371 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.181+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.182+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.182+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a10cad2d:::obj340:head local-lis/les=117/118 n=371 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.182+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.182+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.182+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.182+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7a780 con 0x507ac00 2026-02-11T23:31:28.182+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:a16a9a6a:::obj427:head)]) -- 0x642d500 con 0x507ac00 2026-02-11T23:31:28.182+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.182+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 783 ==== MOSDPGPush(4.0 119/117 [PushOp(4:a16dc95c:::obj269:head, version: 111'269, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a16dc95c:::obj269:head@111'269, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2680 con 0x507ac00 2026-02-11T23:31:28.182+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:a16dc95c:::obj269:head, version: 111'269, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a16dc95c:::obj269:head@111'269, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 119 2026-02-11T23:31:28.182+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a16dc95c:::obj269:head, version: 111'269, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a16dc95c:::obj269:head@111'269, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.182+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.182+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a16dc95c:::obj269:head, version: 111'269, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a16dc95c:::obj269:head@111'269, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.182+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 784 ==== pg_backfill(progress 4.0 e 119/119 lb 4:a16a9a6a:::obj427:head) ==== 1062+0+0 (unknown 0 0 0) 0x6179700 con 0x507ac00 2026-02-11T23:31:28.182+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.182+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:a16a9a6a:::obj427:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.182+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a16dc95c:::obj269:head, version: 111'269, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a16dc95c:::obj269:head@111'269, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.182+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a16a9a6a:::obj427:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.182+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a16dc95c:::obj269:head, version: 111'269, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a16dc95c:::obj269:head@111'269, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 119/117 [PushOp(4:a16dc95c:::obj269:head, version: 111'269, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a16dc95c:::obj269:head@111'269, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a10cad2d:::obj340:head local-lis/les=117/118 n=371 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.182+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.182+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.182+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a16a9a6a:::obj427:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.182+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a10cad2d:::obj340:head local-lis/les=117/118 n=371 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:a16dc95c:::obj269:head, version: 111'269, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a16dc95c:::obj269:head@111'269, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.182+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a10cad2d:::obj340:head local-lis/les=117/118 n=371 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a16dc95c:::obj269:head@111'269, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.182+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a10cad2d:::obj340:head local-lis/les=117/118 n=371 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:a16dc95c:::obj269:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.182+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a10cad2d:::obj340:head local-lis/les=117/118 n=371 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:a16dc95c:::obj269:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.182+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a10cad2d:::obj340:head local-lis/les=117/118 n=371 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:a16dc95c:::obj269:head 2026-02-11T23:31:28.182+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a10cad2d:::obj340:head local-lis/les=117/118 n=371 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:a16dc95c:::obj269:head v 111'269 2026-02-11T23:31:28.182+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a10cad2d:::obj340:head local-lis/les=117/118 n=371 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.182+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.182+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a16dc95c:::obj269:head, version: 111'269, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a16dc95c:::obj269:head@111'269, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:28.182+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a10cad2d:::obj340:head local-lis/les=117/118 n=371 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.182+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.182+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.182+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a16a9a6a:::obj427:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.182+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a16a9a6a:::obj427:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 4.0 e 119/119 lb 4:a16a9a6a:::obj427:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a10cad2d:::obj340:head local-lis/les=117/118 n=371 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.182+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.182+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a10cad2d:::obj340:head local-lis/les=117/118 n=371 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:a16a9a6a:::obj427:head) v3 2026-02-11T23:31:28.182+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a10cad2d:::obj340:head local-lis/les=117/118 n=371 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:a16a9a6a:::obj427:head) 2026-02-11T23:31:28.182+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.182+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a16a9a6a:::obj427:head) v3 finish latency 0.000017 2026-02-11T23:31:28.182+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.182+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.182+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a16a9a6a:::obj427:head local-lis/les=117/118 n=372 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.182+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.182+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.182+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.182+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7b200 con 0x507ac00 2026-02-11T23:31:28.182+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.182+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:a16dc95c:::obj269:head)]) -- 0x5f2dc80 con 0x507ac00 2026-02-11T23:31:28.182+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.182+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 785 ==== MOSDPGPush(4.0 119/117 [PushOp(4:a22120c4:::obj223:head, version: 111'223, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a22120c4:::obj223:head@111'223, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3815380 con 0x507ac00 2026-02-11T23:31:28.182+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:a22120c4:::obj223:head, version: 111'223, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a22120c4:::obj223:head@111'223, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.182+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a22120c4:::obj223:head, version: 111'223, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a22120c4:::obj223:head@111'223, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.182+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.182+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a22120c4:::obj223:head, version: 111'223, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a22120c4:::obj223:head@111'223, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.182+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 786 ==== pg_backfill(progress 4.0 e 119/119 lb 4:a16dc95c:::obj269:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176000 con 0x507ac00 2026-02-11T23:31:28.182+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.182+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:a16dc95c:::obj269:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.182+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a22120c4:::obj223:head, version: 111'223, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a22120c4:::obj223:head@111'223, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.182+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a16dc95c:::obj269:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.182+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a22120c4:::obj223:head, version: 111'223, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a22120c4:::obj223:head@111'223, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 119/117 [PushOp(4:a22120c4:::obj223:head, version: 111'223, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a22120c4:::obj223:head@111'223, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a16a9a6a:::obj427:head local-lis/les=117/118 n=372 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.182+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.182+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.182+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a16dc95c:::obj269:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.182+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a16a9a6a:::obj427:head local-lis/les=117/118 n=372 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:a22120c4:::obj223:head, version: 111'223, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a22120c4:::obj223:head@111'223, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.182+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a16a9a6a:::obj427:head local-lis/les=117/118 n=372 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a22120c4:::obj223:head@111'223, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.182+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a16a9a6a:::obj427:head local-lis/les=117/118 n=372 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:a22120c4:::obj223:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.182+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a16a9a6a:::obj427:head local-lis/les=117/118 n=372 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:a22120c4:::obj223:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.182+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a16a9a6a:::obj427:head local-lis/les=117/118 n=372 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:a22120c4:::obj223:head 2026-02-11T23:31:28.182+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a16a9a6a:::obj427:head local-lis/les=117/118 n=372 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:a22120c4:::obj223:head v 111'223 2026-02-11T23:31:28.182+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a16a9a6a:::obj427:head local-lis/les=117/118 n=372 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.182+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.182+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a22120c4:::obj223:head, version: 111'223, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a22120c4:::obj223:head@111'223, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:28.182+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a16a9a6a:::obj427:head local-lis/les=117/118 n=372 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.182+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.182+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.182+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a16dc95c:::obj269:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.182+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a16dc95c:::obj269:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 4.0 e 119/119 lb 4:a16dc95c:::obj269:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a16a9a6a:::obj427:head local-lis/les=117/118 n=372 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.182+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.182+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a16a9a6a:::obj427:head local-lis/les=117/118 n=372 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:a16dc95c:::obj269:head) v3 2026-02-11T23:31:28.182+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a16a9a6a:::obj427:head local-lis/les=117/118 n=372 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:a16dc95c:::obj269:head) 2026-02-11T23:31:28.182+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.182+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a16dc95c:::obj269:head) v3 finish latency 0.000013 2026-02-11T23:31:28.182+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.183+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.183+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a16dc95c:::obj269:head local-lis/les=117/118 n=373 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.183+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.183+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.183+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.183+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7a300 con 0x507ac00 2026-02-11T23:31:28.183+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:a22120c4:::obj223:head)]) -- 0x63ff200 con 0x507ac00 2026-02-11T23:31:28.183+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.183+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 787 ==== MOSDPGPush(4.0 119/117 [PushOp(4:a25cd2bd:::obj42:head, version: 111'42, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a25cd2bd:::obj42:head@111'42, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5e26b60 con 0x507ac00 2026-02-11T23:31:28.183+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:a25cd2bd:::obj42:head, version: 111'42, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a25cd2bd:::obj42:head@111'42, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.183+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a25cd2bd:::obj42:head, version: 111'42, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a25cd2bd:::obj42:head@111'42, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.183+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.183+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a25cd2bd:::obj42:head, version: 111'42, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a25cd2bd:::obj42:head@111'42, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.183+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 788 ==== pg_backfill(progress 4.0 e 119/119 lb 4:a22120c4:::obj223:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9a680 con 0x507ac00 2026-02-11T23:31:28.183+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:a22120c4:::obj223:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.183+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.183+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a25cd2bd:::obj42:head, version: 111'42, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a25cd2bd:::obj42:head@111'42, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.183+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a22120c4:::obj223:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.183+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.183+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a25cd2bd:::obj42:head, version: 111'42, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a25cd2bd:::obj42:head@111'42, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 119/117 [PushOp(4:a25cd2bd:::obj42:head, version: 111'42, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a25cd2bd:::obj42:head@111'42, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a16dc95c:::obj269:head local-lis/les=117/118 n=373 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.183+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.183+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a22120c4:::obj223:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.183+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a16dc95c:::obj269:head local-lis/les=117/118 n=373 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:a25cd2bd:::obj42:head, version: 111'42, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a25cd2bd:::obj42:head@111'42, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.183+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a16dc95c:::obj269:head local-lis/les=117/118 n=373 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a25cd2bd:::obj42:head@111'42, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.183+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a16dc95c:::obj269:head local-lis/les=117/118 n=373 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:a25cd2bd:::obj42:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.183+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a16dc95c:::obj269:head local-lis/les=117/118 n=373 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:a25cd2bd:::obj42:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.183+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a16dc95c:::obj269:head local-lis/les=117/118 n=373 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:a25cd2bd:::obj42:head 2026-02-11T23:31:28.183+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a16dc95c:::obj269:head local-lis/les=117/118 n=373 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:a25cd2bd:::obj42:head v 111'42 2026-02-11T23:31:28.183+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a16dc95c:::obj269:head local-lis/les=117/118 n=373 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.183+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.183+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a25cd2bd:::obj42:head, version: 111'42, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a25cd2bd:::obj42:head@111'42, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:28.183+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a16dc95c:::obj269:head local-lis/les=117/118 n=373 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.183+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.183+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a22120c4:::obj223:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.183+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.183+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a22120c4:::obj223:head) v3 prio 15 cost 0 latency 0.000060 pg_backfill(progress 4.0 e 119/119 lb 4:a22120c4:::obj223:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a16dc95c:::obj269:head local-lis/les=117/118 n=373 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.183+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.183+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a16dc95c:::obj269:head local-lis/les=117/118 n=373 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:a22120c4:::obj223:head) v3 2026-02-11T23:31:28.183+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a16dc95c:::obj269:head local-lis/les=117/118 n=373 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:a22120c4:::obj223:head) 2026-02-11T23:31:28.183+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.183+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a22120c4:::obj223:head) v3 finish latency 0.000014 2026-02-11T23:31:28.183+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.183+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.183+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a22120c4:::obj223:head local-lis/les=117/118 n=374 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.183+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.183+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.183+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.183+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63fec00 con 0x507ac00 2026-02-11T23:31:28.183+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:a25cd2bd:::obj42:head)]) -- 0x4f7aa80 con 0x507ac00 2026-02-11T23:31:28.183+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.184+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 789 ==== MOSDPGPush(4.0 119/117 [PushOp(4:a2bd80aa:::obj577:head, version: 111'577, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a2bd80aa:::obj577:head@111'577, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a29c0 con 0x507ac00 2026-02-11T23:31:28.184+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:a2bd80aa:::obj577:head, version: 111'577, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a2bd80aa:::obj577:head@111'577, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.184+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a2bd80aa:::obj577:head, version: 111'577, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a2bd80aa:::obj577:head@111'577, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.184+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.184+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a2bd80aa:::obj577:head, version: 111'577, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a2bd80aa:::obj577:head@111'577, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.184+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 790 ==== pg_backfill(progress 4.0 e 119/119 lb 4:a25cd2bd:::obj42:head) ==== 1061+0+0 (unknown 0 0 0) 0x5985700 con 0x507ac00 2026-02-11T23:31:28.184+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.184+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:a25cd2bd:::obj42:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.184+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a2bd80aa:::obj577:head, version: 111'577, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a2bd80aa:::obj577:head@111'577, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.184+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a25cd2bd:::obj42:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.184+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a2bd80aa:::obj577:head, version: 111'577, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a2bd80aa:::obj577:head@111'577, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 119/117 [PushOp(4:a2bd80aa:::obj577:head, version: 111'577, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a2bd80aa:::obj577:head@111'577, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a22120c4:::obj223:head local-lis/les=117/118 n=374 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.184+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.184+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.184+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a25cd2bd:::obj42:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.184+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a22120c4:::obj223:head local-lis/les=117/118 n=374 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:a2bd80aa:::obj577:head, version: 111'577, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a2bd80aa:::obj577:head@111'577, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.184+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a22120c4:::obj223:head local-lis/les=117/118 n=374 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a2bd80aa:::obj577:head@111'577, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.184+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a22120c4:::obj223:head local-lis/les=117/118 n=374 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:a2bd80aa:::obj577:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.184+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a22120c4:::obj223:head local-lis/les=117/118 n=374 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:a2bd80aa:::obj577:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.184+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a22120c4:::obj223:head local-lis/les=117/118 n=374 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:a2bd80aa:::obj577:head 2026-02-11T23:31:28.184+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a22120c4:::obj223:head local-lis/les=117/118 n=374 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:a2bd80aa:::obj577:head v 111'577 2026-02-11T23:31:28.184+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a22120c4:::obj223:head local-lis/les=117/118 n=374 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.184+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.184+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a2bd80aa:::obj577:head, version: 111'577, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a2bd80aa:::obj577:head@111'577, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:28.184+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a22120c4:::obj223:head local-lis/les=117/118 n=374 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.184+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.184+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a25cd2bd:::obj42:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.184+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.184+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a25cd2bd:::obj42:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 4.0 e 119/119 lb 4:a25cd2bd:::obj42:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a22120c4:::obj223:head local-lis/les=117/118 n=374 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.184+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.184+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a22120c4:::obj223:head local-lis/les=117/118 n=374 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:a25cd2bd:::obj42:head) v3 2026-02-11T23:31:28.184+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a22120c4:::obj223:head local-lis/les=117/118 n=374 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:a25cd2bd:::obj42:head) 2026-02-11T23:31:28.184+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.184+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a25cd2bd:::obj42:head) v3 finish latency 0.000014 2026-02-11T23:31:28.184+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.184+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.184+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a25cd2bd:::obj42:head local-lis/les=117/118 n=375 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.184+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.184+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.184+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.184+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x649e600 con 0x507ac00 2026-02-11T23:31:28.184+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.184+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:a2bd80aa:::obj577:head)]) -- 0x642d500 con 0x507ac00 2026-02-11T23:31:28.184+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.184+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 791 ==== MOSDPGPush(4.0 119/117 [PushOp(4:a3278361:::obj425:head, version: 111'425, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3278361:::obj425:head@111'425, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x554e340 con 0x507ac00 2026-02-11T23:31:28.184+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:a3278361:::obj425:head, version: 111'425, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3278361:::obj425:head@111'425, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.184+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a3278361:::obj425:head, version: 111'425, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3278361:::obj425:head@111'425, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.184+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.184+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a3278361:::obj425:head, version: 111'425, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3278361:::obj425:head@111'425, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.184+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 792 ==== pg_backfill(progress 4.0 e 119/119 lb 4:a2bd80aa:::obj577:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177600 con 0x507ac00 2026-02-11T23:31:28.184+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.184+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:a2bd80aa:::obj577:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.184+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a3278361:::obj425:head, version: 111'425, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3278361:::obj425:head@111'425, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.184+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a2bd80aa:::obj577:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.184+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a3278361:::obj425:head, version: 111'425, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3278361:::obj425:head@111'425, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 119/117 [PushOp(4:a3278361:::obj425:head, version: 111'425, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3278361:::obj425:head@111'425, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a25cd2bd:::obj42:head local-lis/les=117/118 n=375 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.184+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.184+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a25cd2bd:::obj42:head local-lis/les=117/118 n=375 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:a3278361:::obj425:head, version: 111'425, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3278361:::obj425:head@111'425, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.184+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.184+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a25cd2bd:::obj42:head local-lis/les=117/118 n=375 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a3278361:::obj425:head@111'425, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.184+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a2bd80aa:::obj577:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.184+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a25cd2bd:::obj42:head local-lis/les=117/118 n=375 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:a3278361:::obj425:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.184+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a25cd2bd:::obj42:head local-lis/les=117/118 n=375 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:a3278361:::obj425:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.184+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a25cd2bd:::obj42:head local-lis/les=117/118 n=375 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:a3278361:::obj425:head 2026-02-11T23:31:28.184+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a25cd2bd:::obj42:head local-lis/les=117/118 n=375 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:a3278361:::obj425:head v 111'425 2026-02-11T23:31:28.184+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a25cd2bd:::obj42:head local-lis/les=117/118 n=375 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.184+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.184+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a3278361:::obj425:head, version: 111'425, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3278361:::obj425:head@111'425, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:28.184+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a25cd2bd:::obj42:head local-lis/les=117/118 n=375 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.184+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.184+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.184+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a2bd80aa:::obj577:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.184+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a2bd80aa:::obj577:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 4.0 e 119/119 lb 4:a2bd80aa:::obj577:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a25cd2bd:::obj42:head local-lis/les=117/118 n=375 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.184+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.184+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a25cd2bd:::obj42:head local-lis/les=117/118 n=375 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:a2bd80aa:::obj577:head) v3 2026-02-11T23:31:28.184+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a25cd2bd:::obj42:head local-lis/les=117/118 n=375 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:a2bd80aa:::obj577:head) 2026-02-11T23:31:28.184+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.184+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a2bd80aa:::obj577:head) v3 finish latency 0.000013 2026-02-11T23:31:28.184+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.184+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.184+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a2bd80aa:::obj577:head local-lis/les=117/118 n=376 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.184+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.184+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.184+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.184+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x640e180 con 0x507ac00 2026-02-11T23:31:28.184+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.184+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:a3278361:::obj425:head)]) -- 0x63ffb00 con 0x507ac00 2026-02-11T23:31:28.184+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.185+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 793 ==== MOSDPGPush(4.0 119/117 [PushOp(4:a3411a0c:::obj160:head, version: 111'160, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3411a0c:::obj160:head@111'160, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5646d00 con 0x507ac00 2026-02-11T23:31:28.185+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:a3411a0c:::obj160:head, version: 111'160, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3411a0c:::obj160:head@111'160, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.185+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a3411a0c:::obj160:head, version: 111'160, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3411a0c:::obj160:head@111'160, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.185+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.185+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 794 ==== pg_backfill(progress 4.0 e 119/119 lb 4:a3278361:::obj425:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177b80 con 0x507ac00 2026-02-11T23:31:28.185+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a3411a0c:::obj160:head, version: 111'160, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3411a0c:::obj160:head@111'160, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.185+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:a3278361:::obj425:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.185+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.185+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a3278361:::obj425:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.185+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a3411a0c:::obj160:head, version: 111'160, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3411a0c:::obj160:head@111'160, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.185+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.185+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a3411a0c:::obj160:head, version: 111'160, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3411a0c:::obj160:head@111'160, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 119/117 [PushOp(4:a3411a0c:::obj160:head, version: 111'160, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3411a0c:::obj160:head@111'160, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a2bd80aa:::obj577:head local-lis/les=117/118 n=376 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.185+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.185+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a2bd80aa:::obj577:head local-lis/les=117/118 n=376 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:a3411a0c:::obj160:head, version: 111'160, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3411a0c:::obj160:head@111'160, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.185+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a2bd80aa:::obj577:head local-lis/les=117/118 n=376 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a3411a0c:::obj160:head@111'160, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.185+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.185+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a3278361:::obj425:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.185+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a2bd80aa:::obj577:head local-lis/les=117/118 n=376 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:a3411a0c:::obj160:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.185+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a2bd80aa:::obj577:head local-lis/les=117/118 n=376 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:a3411a0c:::obj160:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.185+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a2bd80aa:::obj577:head local-lis/les=117/118 n=376 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:a3411a0c:::obj160:head 2026-02-11T23:31:28.185+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a2bd80aa:::obj577:head local-lis/les=117/118 n=376 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:a3411a0c:::obj160:head v 111'160 2026-02-11T23:31:28.185+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a2bd80aa:::obj577:head local-lis/les=117/118 n=376 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.185+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.185+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a3411a0c:::obj160:head, version: 111'160, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3411a0c:::obj160:head@111'160, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:28.185+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a2bd80aa:::obj577:head local-lis/les=117/118 n=376 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.185+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.185+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a3278361:::obj425:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.185+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.185+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a3278361:::obj425:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 119/119 lb 4:a3278361:::obj425:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a2bd80aa:::obj577:head local-lis/les=117/118 n=376 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.185+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.185+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a2bd80aa:::obj577:head local-lis/les=117/118 n=376 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:a3278361:::obj425:head) v3 2026-02-11T23:31:28.185+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a2bd80aa:::obj577:head local-lis/les=117/118 n=376 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:a3278361:::obj425:head) 2026-02-11T23:31:28.185+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.185+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a3278361:::obj425:head) v3 finish latency 0.000014 2026-02-11T23:31:28.185+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.185+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.185+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a3278361:::obj425:head local-lis/les=117/118 n=377 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.185+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.185+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.185+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.185+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x640fb00 con 0x507ac00 2026-02-11T23:31:28.185+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:a3411a0c:::obj160:head)]) -- 0x642cd80 con 0x507ac00 2026-02-11T23:31:28.185+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.185+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 795 ==== MOSDPGPush(4.0 119/117 [PushOp(4:a37ff878:::obj107:head, version: 111'107, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a37ff878:::obj107:head@111'107, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56ca680 con 0x507ac00 2026-02-11T23:31:28.185+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:a37ff878:::obj107:head, version: 111'107, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a37ff878:::obj107:head@111'107, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.185+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a37ff878:::obj107:head, version: 111'107, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a37ff878:::obj107:head@111'107, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.185+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.185+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a37ff878:::obj107:head, version: 111'107, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a37ff878:::obj107:head@111'107, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.185+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 796 ==== pg_backfill(progress 4.0 e 119/119 lb 4:a3411a0c:::obj160:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2c580 con 0x507ac00 2026-02-11T23:31:28.185+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.185+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:a3411a0c:::obj160:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.185+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a37ff878:::obj107:head, version: 111'107, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a37ff878:::obj107:head@111'107, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.185+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a3411a0c:::obj160:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.185+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a37ff878:::obj107:head, version: 111'107, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a37ff878:::obj107:head@111'107, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 119/117 [PushOp(4:a37ff878:::obj107:head, version: 111'107, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a37ff878:::obj107:head@111'107, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a3278361:::obj425:head local-lis/les=117/118 n=377 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.185+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.185+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.185+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a3411a0c:::obj160:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.185+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a3278361:::obj425:head local-lis/les=117/118 n=377 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:a37ff878:::obj107:head, version: 111'107, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a37ff878:::obj107:head@111'107, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.185+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a3278361:::obj425:head local-lis/les=117/118 n=377 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a37ff878:::obj107:head@111'107, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.185+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a3278361:::obj425:head local-lis/les=117/118 n=377 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:a37ff878:::obj107:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.185+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a3278361:::obj425:head local-lis/les=117/118 n=377 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:a37ff878:::obj107:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.185+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a3278361:::obj425:head local-lis/les=117/118 n=377 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:a37ff878:::obj107:head 2026-02-11T23:31:28.185+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a3278361:::obj425:head local-lis/les=117/118 n=377 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:a37ff878:::obj107:head v 111'107 2026-02-11T23:31:28.185+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a3278361:::obj425:head local-lis/les=117/118 n=377 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.185+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.185+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a37ff878:::obj107:head, version: 111'107, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a37ff878:::obj107:head@111'107, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:28.185+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a3278361:::obj425:head local-lis/les=117/118 n=377 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.185+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.185+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.185+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a3411a0c:::obj160:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.185+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a3411a0c:::obj160:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 4.0 e 119/119 lb 4:a3411a0c:::obj160:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a3278361:::obj425:head local-lis/les=117/118 n=377 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.185+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.185+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a3278361:::obj425:head local-lis/les=117/118 n=377 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:a3411a0c:::obj160:head) v3 2026-02-11T23:31:28.185+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a3278361:::obj425:head local-lis/les=117/118 n=377 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:a3411a0c:::obj160:head) 2026-02-11T23:31:28.185+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.185+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a3411a0c:::obj160:head) v3 finish latency 0.000015 2026-02-11T23:31:28.185+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.186+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a3411a0c:::obj160:head local-lis/les=117/118 n=378 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.186+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.186+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.186+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.186+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.186+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x5d55800 con 0x507ac00 2026-02-11T23:31:28.186+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:a37ff878:::obj107:head)]) -- 0x5d55200 con 0x507ac00 2026-02-11T23:31:28.186+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.186+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 797 ==== MOSDPGPush(4.0 119/117 [PushOp(4:a383681b:::obj41:head, version: 111'41, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a383681b:::obj41:head@111'41, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x55f09c0 con 0x507ac00 2026-02-11T23:31:28.186+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:a383681b:::obj41:head, version: 111'41, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a383681b:::obj41:head@111'41, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.186+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a383681b:::obj41:head, version: 111'41, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a383681b:::obj41:head@111'41, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.186+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.186+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 798 ==== pg_backfill(progress 4.0 e 119/119 lb 4:a37ff878:::obj107:head) ==== 1062+0+0 (unknown 0 0 0) 0x6178100 con 0x507ac00 2026-02-11T23:31:28.186+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a383681b:::obj41:head, version: 111'41, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a383681b:::obj41:head@111'41, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.186+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:a37ff878:::obj107:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.186+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.186+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a37ff878:::obj107:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.186+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a383681b:::obj41:head, version: 111'41, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a383681b:::obj41:head@111'41, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.186+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a383681b:::obj41:head, version: 111'41, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a383681b:::obj41:head@111'41, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 119/117 [PushOp(4:a383681b:::obj41:head, version: 111'41, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a383681b:::obj41:head@111'41, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a3411a0c:::obj160:head local-lis/les=117/118 n=378 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.186+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.186+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.186+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a3411a0c:::obj160:head local-lis/les=117/118 n=378 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:a383681b:::obj41:head, version: 111'41, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a383681b:::obj41:head@111'41, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.186+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a37ff878:::obj107:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.186+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a3411a0c:::obj160:head local-lis/les=117/118 n=378 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a383681b:::obj41:head@111'41, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.186+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a3411a0c:::obj160:head local-lis/les=117/118 n=378 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:a383681b:::obj41:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.186+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a3411a0c:::obj160:head local-lis/les=117/118 n=378 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:a383681b:::obj41:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.186+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a3411a0c:::obj160:head local-lis/les=117/118 n=378 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:a383681b:::obj41:head 2026-02-11T23:31:28.186+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a3411a0c:::obj160:head local-lis/les=117/118 n=378 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:a383681b:::obj41:head v 111'41 2026-02-11T23:31:28.186+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a3411a0c:::obj160:head local-lis/les=117/118 n=378 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.186+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.186+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a383681b:::obj41:head, version: 111'41, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a383681b:::obj41:head@111'41, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:28.186+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a3411a0c:::obj160:head local-lis/les=117/118 n=378 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.186+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.186+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a37ff878:::obj107:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.186+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.186+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a37ff878:::obj107:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 4.0 e 119/119 lb 4:a37ff878:::obj107:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a3411a0c:::obj160:head local-lis/les=117/118 n=378 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.186+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.186+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a3411a0c:::obj160:head local-lis/les=117/118 n=378 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:a37ff878:::obj107:head) v3 2026-02-11T23:31:28.186+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a3411a0c:::obj160:head local-lis/les=117/118 n=378 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:a37ff878:::obj107:head) 2026-02-11T23:31:28.186+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.186+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a37ff878:::obj107:head) v3 finish latency 0.000014 2026-02-11T23:31:28.186+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.186+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.186+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a37ff878:::obj107:head local-lis/les=117/118 n=379 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.186+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.186+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.186+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.186+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c9e00 con 0x507ac00 2026-02-11T23:31:28.186+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:a383681b:::obj41:head)]) -- 0x642ca80 con 0x507ac00 2026-02-11T23:31:28.186+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.187+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 799 ==== MOSDPGPush(4.0 119/117 [PushOp(4:a39a40f0:::obj561:head, version: 111'561, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a39a40f0:::obj561:head@111'561, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a29c0 con 0x507ac00 2026-02-11T23:31:28.187+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:a39a40f0:::obj561:head, version: 111'561, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a39a40f0:::obj561:head@111'561, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.187+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a39a40f0:::obj561:head, version: 111'561, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a39a40f0:::obj561:head@111'561, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.187+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.187+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a39a40f0:::obj561:head, version: 111'561, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a39a40f0:::obj561:head@111'561, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.187+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 800 ==== pg_backfill(progress 4.0 e 119/119 lb 4:a383681b:::obj41:head) ==== 1061+0+0 (unknown 0 0 0) 0x6179180 con 0x507ac00 2026-02-11T23:31:28.187+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:a383681b:::obj41:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.187+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.187+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a39a40f0:::obj561:head, version: 111'561, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a39a40f0:::obj561:head@111'561, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.187+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a383681b:::obj41:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.187+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a39a40f0:::obj561:head, version: 111'561, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a39a40f0:::obj561:head@111'561, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 119/117 [PushOp(4:a39a40f0:::obj561:head, version: 111'561, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a39a40f0:::obj561:head@111'561, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a37ff878:::obj107:head local-lis/les=117/118 n=379 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.187+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.187+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.187+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a383681b:::obj41:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.187+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a37ff878:::obj107:head local-lis/les=117/118 n=379 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:a39a40f0:::obj561:head, version: 111'561, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a39a40f0:::obj561:head@111'561, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.187+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a37ff878:::obj107:head local-lis/les=117/118 n=379 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a39a40f0:::obj561:head@111'561, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.187+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a37ff878:::obj107:head local-lis/les=117/118 n=379 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:a39a40f0:::obj561:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.187+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a37ff878:::obj107:head local-lis/les=117/118 n=379 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:a39a40f0:::obj561:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.187+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a37ff878:::obj107:head local-lis/les=117/118 n=379 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:a39a40f0:::obj561:head 2026-02-11T23:31:28.187+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a37ff878:::obj107:head local-lis/les=117/118 n=379 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:a39a40f0:::obj561:head v 111'561 2026-02-11T23:31:28.187+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a37ff878:::obj107:head local-lis/les=117/118 n=379 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.187+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.187+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a39a40f0:::obj561:head, version: 111'561, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a39a40f0:::obj561:head@111'561, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:28.187+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a37ff878:::obj107:head local-lis/les=117/118 n=379 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.187+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.187+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.187+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a383681b:::obj41:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.187+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a383681b:::obj41:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 4.0 e 119/119 lb 4:a383681b:::obj41:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a37ff878:::obj107:head local-lis/les=117/118 n=379 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.187+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.187+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a37ff878:::obj107:head local-lis/les=117/118 n=379 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:a383681b:::obj41:head) v3 2026-02-11T23:31:28.187+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a37ff878:::obj107:head local-lis/les=117/118 n=379 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:a383681b:::obj41:head) 2026-02-11T23:31:28.187+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.187+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a383681b:::obj41:head) v3 finish latency 0.000014 2026-02-11T23:31:28.187+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.187+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.187+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a383681b:::obj41:head local-lis/les=117/118 n=380 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.187+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.187+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.187+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.187+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x640f980 con 0x507ac00 2026-02-11T23:31:28.187+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:a39a40f0:::obj561:head)]) -- 0x640f380 con 0x507ac00 2026-02-11T23:31:28.187+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.187+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 801 ==== MOSDPGPush(4.0 119/117 [PushOp(4:a3cf8cd5:::obj363:head, version: 111'363, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3cf8cd5:::obj363:head@111'363, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e26b60 con 0x507ac00 2026-02-11T23:31:28.187+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:a3cf8cd5:::obj363:head, version: 111'363, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3cf8cd5:::obj363:head@111'363, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.187+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a3cf8cd5:::obj363:head, version: 111'363, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3cf8cd5:::obj363:head@111'363, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.187+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.187+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a3cf8cd5:::obj363:head, version: 111'363, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3cf8cd5:::obj363:head@111'363, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.187+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 802 ==== pg_backfill(progress 4.0 e 119/119 lb 4:a39a40f0:::obj561:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9b180 con 0x507ac00 2026-02-11T23:31:28.187+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.187+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:a39a40f0:::obj561:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.187+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a3cf8cd5:::obj363:head, version: 111'363, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3cf8cd5:::obj363:head@111'363, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.187+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a39a40f0:::obj561:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.187+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a3cf8cd5:::obj363:head, version: 111'363, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3cf8cd5:::obj363:head@111'363, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 119/117 [PushOp(4:a3cf8cd5:::obj363:head, version: 111'363, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3cf8cd5:::obj363:head@111'363, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a383681b:::obj41:head local-lis/les=117/118 n=380 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.187+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.187+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.187+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a39a40f0:::obj561:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.187+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a383681b:::obj41:head local-lis/les=117/118 n=380 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:a3cf8cd5:::obj363:head, version: 111'363, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3cf8cd5:::obj363:head@111'363, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.187+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a383681b:::obj41:head local-lis/les=117/118 n=380 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a3cf8cd5:::obj363:head@111'363, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.187+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a383681b:::obj41:head local-lis/les=117/118 n=380 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:a3cf8cd5:::obj363:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.187+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a383681b:::obj41:head local-lis/les=117/118 n=380 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:a3cf8cd5:::obj363:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.187+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a383681b:::obj41:head local-lis/les=117/118 n=380 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:a3cf8cd5:::obj363:head 2026-02-11T23:31:28.187+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a383681b:::obj41:head local-lis/les=117/118 n=380 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:a3cf8cd5:::obj363:head v 111'363 2026-02-11T23:31:28.187+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a383681b:::obj41:head local-lis/les=117/118 n=380 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.187+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.187+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a3cf8cd5:::obj363:head, version: 111'363, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3cf8cd5:::obj363:head@111'363, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:28.187+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a383681b:::obj41:head local-lis/les=117/118 n=380 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.187+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.187+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.187+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a39a40f0:::obj561:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.187+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a39a40f0:::obj561:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 4.0 e 119/119 lb 4:a39a40f0:::obj561:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a383681b:::obj41:head local-lis/les=117/118 n=380 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.187+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.187+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a383681b:::obj41:head local-lis/les=117/118 n=380 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:a39a40f0:::obj561:head) v3 2026-02-11T23:31:28.187+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a383681b:::obj41:head local-lis/les=117/118 n=380 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:a39a40f0:::obj561:head) 2026-02-11T23:31:28.187+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.187+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a39a40f0:::obj561:head) v3 finish latency 0.000013 2026-02-11T23:31:28.187+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.187+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.187+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a39a40f0:::obj561:head local-lis/les=117/118 n=381 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.187+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.187+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.187+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.187+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66d9e00 con 0x507ac00 2026-02-11T23:31:28.187+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:a3cf8cd5:::obj363:head)]) -- 0x640f680 con 0x507ac00 2026-02-11T23:31:28.187+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.188+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 803 ==== MOSDPGPush(4.0 119/117 [PushOp(4:a42b336e:::obj435:head, version: 111'435, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a42b336e:::obj435:head@111'435, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3815380 con 0x507ac00 2026-02-11T23:31:28.188+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:a42b336e:::obj435:head, version: 111'435, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a42b336e:::obj435:head@111'435, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.188+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a42b336e:::obj435:head, version: 111'435, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a42b336e:::obj435:head@111'435, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.188+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.188+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a42b336e:::obj435:head, version: 111'435, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a42b336e:::obj435:head@111'435, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.188+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 804 ==== pg_backfill(progress 4.0 e 119/119 lb 4:a3cf8cd5:::obj363:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98000 con 0x507ac00 2026-02-11T23:31:28.188+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:a3cf8cd5:::obj363:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.188+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.188+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a42b336e:::obj435:head, version: 111'435, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a42b336e:::obj435:head@111'435, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.188+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a3cf8cd5:::obj363:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.188+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a42b336e:::obj435:head, version: 111'435, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a42b336e:::obj435:head@111'435, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 119/117 [PushOp(4:a42b336e:::obj435:head, version: 111'435, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a42b336e:::obj435:head@111'435, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a39a40f0:::obj561:head local-lis/les=117/118 n=381 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.188+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.188+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.188+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a3cf8cd5:::obj363:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.188+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a39a40f0:::obj561:head local-lis/les=117/118 n=381 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:a42b336e:::obj435:head, version: 111'435, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a42b336e:::obj435:head@111'435, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.188+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a39a40f0:::obj561:head local-lis/les=117/118 n=381 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a42b336e:::obj435:head@111'435, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.188+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a39a40f0:::obj561:head local-lis/les=117/118 n=381 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:a42b336e:::obj435:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.188+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a39a40f0:::obj561:head local-lis/les=117/118 n=381 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:a42b336e:::obj435:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.188+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a39a40f0:::obj561:head local-lis/les=117/118 n=381 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:a42b336e:::obj435:head 2026-02-11T23:31:28.188+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a39a40f0:::obj561:head local-lis/les=117/118 n=381 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:a42b336e:::obj435:head v 111'435 2026-02-11T23:31:28.188+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a39a40f0:::obj561:head local-lis/les=117/118 n=381 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.188+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.188+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a42b336e:::obj435:head, version: 111'435, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a42b336e:::obj435:head@111'435, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:28.188+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a39a40f0:::obj561:head local-lis/les=117/118 n=381 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.188+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.188+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a3cf8cd5:::obj363:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.188+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.188+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a3cf8cd5:::obj363:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 4.0 e 119/119 lb 4:a3cf8cd5:::obj363:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a39a40f0:::obj561:head local-lis/les=117/118 n=381 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.188+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.188+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a39a40f0:::obj561:head local-lis/les=117/118 n=381 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:a3cf8cd5:::obj363:head) v3 2026-02-11T23:31:28.188+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a39a40f0:::obj561:head local-lis/les=117/118 n=381 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:a3cf8cd5:::obj363:head) 2026-02-11T23:31:28.188+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.188+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a3cf8cd5:::obj363:head) v3 finish latency 0.000014 2026-02-11T23:31:28.188+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.188+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.188+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a3cf8cd5:::obj363:head local-lis/les=117/118 n=382 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.188+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.188+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.188+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.188+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66d9200 con 0x507ac00 2026-02-11T23:31:28.188+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.188+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:a42b336e:::obj435:head)]) -- 0x66d9680 con 0x507ac00 2026-02-11T23:31:28.188+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.188+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 805 ==== MOSDPGPush(4.0 119/117 [PushOp(4:a4383432:::obj559:head, version: 111'559, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4383432:::obj559:head@111'559, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2680 con 0x507ac00 2026-02-11T23:31:28.188+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:a4383432:::obj559:head, version: 111'559, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4383432:::obj559:head@111'559, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.188+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a4383432:::obj559:head, version: 111'559, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4383432:::obj559:head@111'559, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.188+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.188+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 806 ==== pg_backfill(progress 4.0 e 119/119 lb 4:a42b336e:::obj435:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2ec00 con 0x507ac00 2026-02-11T23:31:28.188+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a4383432:::obj559:head, version: 111'559, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4383432:::obj559:head@111'559, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.188+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:a42b336e:::obj435:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.188+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.188+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a42b336e:::obj435:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.188+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a4383432:::obj559:head, version: 111'559, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4383432:::obj559:head@111'559, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.188+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.188+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a4383432:::obj559:head, version: 111'559, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4383432:::obj559:head@111'559, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 119/117 [PushOp(4:a4383432:::obj559:head, version: 111'559, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4383432:::obj559:head@111'559, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a3cf8cd5:::obj363:head local-lis/les=117/118 n=382 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.188+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.188+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a3cf8cd5:::obj363:head local-lis/les=117/118 n=382 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:a4383432:::obj559:head, version: 111'559, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4383432:::obj559:head@111'559, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.188+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a3cf8cd5:::obj363:head local-lis/les=117/118 n=382 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a4383432:::obj559:head@111'559, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.188+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.188+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a42b336e:::obj435:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.188+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a3cf8cd5:::obj363:head local-lis/les=117/118 n=382 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:a4383432:::obj559:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.188+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a3cf8cd5:::obj363:head local-lis/les=117/118 n=382 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:a4383432:::obj559:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.188+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a3cf8cd5:::obj363:head local-lis/les=117/118 n=382 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:a4383432:::obj559:head 2026-02-11T23:31:28.188+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a3cf8cd5:::obj363:head local-lis/les=117/118 n=382 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:a4383432:::obj559:head v 111'559 2026-02-11T23:31:28.188+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a3cf8cd5:::obj363:head local-lis/les=117/118 n=382 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.188+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.188+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a4383432:::obj559:head, version: 111'559, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4383432:::obj559:head@111'559, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:28.188+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a3cf8cd5:::obj363:head local-lis/les=117/118 n=382 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.188+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.188+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a42b336e:::obj435:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.188+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.188+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a42b336e:::obj435:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 4.0 e 119/119 lb 4:a42b336e:::obj435:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a3cf8cd5:::obj363:head local-lis/les=117/118 n=382 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.188+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.188+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a3cf8cd5:::obj363:head local-lis/les=117/118 n=382 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:a42b336e:::obj435:head) v3 2026-02-11T23:31:28.188+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a3cf8cd5:::obj363:head local-lis/les=117/118 n=382 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:a42b336e:::obj435:head) 2026-02-11T23:31:28.188+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.188+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a42b336e:::obj435:head) v3 finish latency 0.000013 2026-02-11T23:31:28.188+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.188+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.188+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a42b336e:::obj435:head local-lis/les=117/118 n=383 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.188+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.188+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.188+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.188+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66d9080 con 0x507ac00 2026-02-11T23:31:28.188+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:a4383432:::obj559:head)]) -- 0x66d9500 con 0x507ac00 2026-02-11T23:31:28.188+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.189+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 807 ==== MOSDPGPush(4.0 119/117 [PushOp(4:a4518626:::obj581:head, version: 111'581, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4518626:::obj581:head@111'581, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56cab60 con 0x507ac00 2026-02-11T23:31:28.189+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:a4518626:::obj581:head, version: 111'581, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4518626:::obj581:head@111'581, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.189+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a4518626:::obj581:head, version: 111'581, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4518626:::obj581:head@111'581, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.189+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.189+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a4518626:::obj581:head, version: 111'581, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4518626:::obj581:head@111'581, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.189+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 808 ==== pg_backfill(progress 4.0 e 119/119 lb 4:a4383432:::obj559:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2d600 con 0x507ac00 2026-02-11T23:31:28.189+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:a4383432:::obj559:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.189+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.189+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a4383432:::obj559:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.189+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a4518626:::obj581:head, version: 111'581, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4518626:::obj581:head@111'581, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.189+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a4518626:::obj581:head, version: 111'581, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4518626:::obj581:head@111'581, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 119/117 [PushOp(4:a4518626:::obj581:head, version: 111'581, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4518626:::obj581:head@111'581, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a42b336e:::obj435:head local-lis/les=117/118 n=383 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.189+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.189+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.189+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a4383432:::obj559:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.189+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a42b336e:::obj435:head local-lis/les=117/118 n=383 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:a4518626:::obj581:head, version: 111'581, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4518626:::obj581:head@111'581, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.189+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a42b336e:::obj435:head local-lis/les=117/118 n=383 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a4518626:::obj581:head@111'581, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.189+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a42b336e:::obj435:head local-lis/les=117/118 n=383 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:a4518626:::obj581:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.189+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a42b336e:::obj435:head local-lis/les=117/118 n=383 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:a4518626:::obj581:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.189+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a42b336e:::obj435:head local-lis/les=117/118 n=383 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:a4518626:::obj581:head 2026-02-11T23:31:28.189+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a42b336e:::obj435:head local-lis/les=117/118 n=383 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:a4518626:::obj581:head v 111'581 2026-02-11T23:31:28.189+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a42b336e:::obj435:head local-lis/les=117/118 n=383 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.189+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.189+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a4518626:::obj581:head, version: 111'581, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4518626:::obj581:head@111'581, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:28.189+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a42b336e:::obj435:head local-lis/les=117/118 n=383 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.189+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.189+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.189+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a4383432:::obj559:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.189+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a4383432:::obj559:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 4.0 e 119/119 lb 4:a4383432:::obj559:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a42b336e:::obj435:head local-lis/les=117/118 n=383 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.189+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.189+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a42b336e:::obj435:head local-lis/les=117/118 n=383 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:a4383432:::obj559:head) v3 2026-02-11T23:31:28.189+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a42b336e:::obj435:head local-lis/les=117/118 n=383 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:a4383432:::obj559:head) 2026-02-11T23:31:28.189+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.189+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a4383432:::obj559:head) v3 finish latency 0.000015 2026-02-11T23:31:28.189+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.189+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.189+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a4383432:::obj559:head local-lis/les=117/118 n=384 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.189+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.189+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.189+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.189+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66d8600 con 0x507ac00 2026-02-11T23:31:28.189+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:a4518626:::obj581:head)]) -- 0x66d8a80 con 0x507ac00 2026-02-11T23:31:28.189+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.189+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 809 ==== MOSDPGPush(4.0 119/117 [PushOp(4:a4664b16:::obj367:head, version: 111'367, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4664b16:::obj367:head@111'367, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a3860 con 0x507ac00 2026-02-11T23:31:28.189+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:a4664b16:::obj367:head, version: 111'367, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4664b16:::obj367:head@111'367, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.189+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a4664b16:::obj367:head, version: 111'367, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4664b16:::obj367:head@111'367, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.189+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.189+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a4664b16:::obj367:head, version: 111'367, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4664b16:::obj367:head@111'367, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.189+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 810 ==== pg_backfill(progress 4.0 e 119/119 lb 4:a4518626:::obj581:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99600 con 0x507ac00 2026-02-11T23:31:28.189+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:a4518626:::obj581:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.189+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.189+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a4664b16:::obj367:head, version: 111'367, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4664b16:::obj367:head@111'367, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.189+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a4518626:::obj581:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.189+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a4664b16:::obj367:head, version: 111'367, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4664b16:::obj367:head@111'367, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 119/117 [PushOp(4:a4664b16:::obj367:head, version: 111'367, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4664b16:::obj367:head@111'367, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a4383432:::obj559:head local-lis/les=117/118 n=384 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.189+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.189+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.189+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a4383432:::obj559:head local-lis/les=117/118 n=384 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:a4664b16:::obj367:head, version: 111'367, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4664b16:::obj367:head@111'367, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.189+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a4518626:::obj581:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.189+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a4383432:::obj559:head local-lis/les=117/118 n=384 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a4664b16:::obj367:head@111'367, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.189+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a4383432:::obj559:head local-lis/les=117/118 n=384 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:a4664b16:::obj367:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.189+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a4383432:::obj559:head local-lis/les=117/118 n=384 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:a4664b16:::obj367:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.189+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a4383432:::obj559:head local-lis/les=117/118 n=384 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:a4664b16:::obj367:head 2026-02-11T23:31:28.189+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a4383432:::obj559:head local-lis/les=117/118 n=384 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:a4664b16:::obj367:head v 111'367 2026-02-11T23:31:28.189+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a4383432:::obj559:head local-lis/les=117/118 n=384 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.189+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.189+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a4664b16:::obj367:head, version: 111'367, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4664b16:::obj367:head@111'367, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:28.190+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a4383432:::obj559:head local-lis/les=117/118 n=384 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.190+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.190+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.190+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a4518626:::obj581:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.190+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a4518626:::obj581:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 4.0 e 119/119 lb 4:a4518626:::obj581:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a4383432:::obj559:head local-lis/les=117/118 n=384 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.190+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.190+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a4383432:::obj559:head local-lis/les=117/118 n=384 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:a4518626:::obj581:head) v3 2026-02-11T23:31:28.190+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a4383432:::obj559:head local-lis/les=117/118 n=384 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:a4518626:::obj581:head) 2026-02-11T23:31:28.190+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.190+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a4518626:::obj581:head) v3 finish latency 0.000014 2026-02-11T23:31:28.190+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.190+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.190+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a4518626:::obj581:head local-lis/les=117/118 n=385 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.190+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.190+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.190+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.190+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66d8480 con 0x507ac00 2026-02-11T23:31:28.190+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:a4664b16:::obj367:head)]) -- 0x66d8900 con 0x507ac00 2026-02-11T23:31:28.190+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.190+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 811 ==== MOSDPGPush(4.0 119/117 [PushOp(4:a52fdb40:::obj548:head, version: 111'548, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a52fdb40:::obj548:head@111'548, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e26680 con 0x507ac00 2026-02-11T23:31:28.190+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:a52fdb40:::obj548:head, version: 111'548, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a52fdb40:::obj548:head@111'548, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.190+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a52fdb40:::obj548:head, version: 111'548, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a52fdb40:::obj548:head@111'548, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.190+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.190+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a52fdb40:::obj548:head, version: 111'548, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a52fdb40:::obj548:head@111'548, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.190+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 812 ==== pg_backfill(progress 4.0 e 119/119 lb 4:a4664b16:::obj367:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176b00 con 0x507ac00 2026-02-11T23:31:28.190+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:a4664b16:::obj367:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.190+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.190+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a52fdb40:::obj548:head, version: 111'548, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a52fdb40:::obj548:head@111'548, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.190+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a4664b16:::obj367:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.190+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.190+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a52fdb40:::obj548:head, version: 111'548, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a52fdb40:::obj548:head@111'548, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 119/117 [PushOp(4:a52fdb40:::obj548:head, version: 111'548, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a52fdb40:::obj548:head@111'548, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a4518626:::obj581:head local-lis/les=117/118 n=385 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.190+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.190+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a4664b16:::obj367:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.190+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a4518626:::obj581:head local-lis/les=117/118 n=385 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:a52fdb40:::obj548:head, version: 111'548, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a52fdb40:::obj548:head@111'548, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.190+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a4518626:::obj581:head local-lis/les=117/118 n=385 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a52fdb40:::obj548:head@111'548, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.190+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a4518626:::obj581:head local-lis/les=117/118 n=385 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:a52fdb40:::obj548:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.190+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a4518626:::obj581:head local-lis/les=117/118 n=385 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:a52fdb40:::obj548:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.190+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a4518626:::obj581:head local-lis/les=117/118 n=385 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:a52fdb40:::obj548:head 2026-02-11T23:31:28.190+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a4518626:::obj581:head local-lis/les=117/118 n=385 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:a52fdb40:::obj548:head v 111'548 2026-02-11T23:31:28.190+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a4518626:::obj581:head local-lis/les=117/118 n=385 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.190+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.190+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a52fdb40:::obj548:head, version: 111'548, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a52fdb40:::obj548:head@111'548, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:28.190+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a4518626:::obj581:head local-lis/les=117/118 n=385 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.190+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.190+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a4664b16:::obj367:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.190+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.190+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a4664b16:::obj367:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 4.0 e 119/119 lb 4:a4664b16:::obj367:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a4518626:::obj581:head local-lis/les=117/118 n=385 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.190+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.190+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a4518626:::obj581:head local-lis/les=117/118 n=385 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:a4664b16:::obj367:head) v3 2026-02-11T23:31:28.190+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a4518626:::obj581:head local-lis/les=117/118 n=385 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:a4664b16:::obj367:head) 2026-02-11T23:31:28.190+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.190+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a4664b16:::obj367:head) v3 finish latency 0.000014 2026-02-11T23:31:28.190+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.190+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.190+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a4664b16:::obj367:head local-lis/les=117/118 n=386 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.190+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.190+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.190+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.190+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.190+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66fa300 con 0x507ac00 2026-02-11T23:31:28.190+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:a52fdb40:::obj548:head)]) -- 0x66fa000 con 0x507ac00 2026-02-11T23:31:28.190+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.191+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 813 ==== MOSDPGPush(4.0 119/117 [PushOp(4:a53b2263:::obj137:head, version: 111'137, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a53b2263:::obj137:head@111'137, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x38156c0 con 0x507ac00 2026-02-11T23:31:28.191+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:a53b2263:::obj137:head, version: 111'137, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a53b2263:::obj137:head@111'137, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.191+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a53b2263:::obj137:head, version: 111'137, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a53b2263:::obj137:head@111'137, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.191+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.191+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a53b2263:::obj137:head, version: 111'137, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a53b2263:::obj137:head@111'137, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.191+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 814 ==== pg_backfill(progress 4.0 e 119/119 lb 4:a52fdb40:::obj548:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99b80 con 0x507ac00 2026-02-11T23:31:28.191+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:a52fdb40:::obj548:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.191+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.191+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a53b2263:::obj137:head, version: 111'137, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a53b2263:::obj137:head@111'137, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.191+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a52fdb40:::obj548:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.191+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a53b2263:::obj137:head, version: 111'137, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a53b2263:::obj137:head@111'137, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 119/117 [PushOp(4:a53b2263:::obj137:head, version: 111'137, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a53b2263:::obj137:head@111'137, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a4664b16:::obj367:head local-lis/les=117/118 n=386 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.191+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.191+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.191+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a52fdb40:::obj548:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.191+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a4664b16:::obj367:head local-lis/les=117/118 n=386 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:a53b2263:::obj137:head, version: 111'137, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a53b2263:::obj137:head@111'137, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.191+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a4664b16:::obj367:head local-lis/les=117/118 n=386 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a53b2263:::obj137:head@111'137, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.191+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a4664b16:::obj367:head local-lis/les=117/118 n=386 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:a53b2263:::obj137:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.191+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a4664b16:::obj367:head local-lis/les=117/118 n=386 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:a53b2263:::obj137:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.191+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a4664b16:::obj367:head local-lis/les=117/118 n=386 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:a53b2263:::obj137:head 2026-02-11T23:31:28.191+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a4664b16:::obj367:head local-lis/les=117/118 n=386 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:a53b2263:::obj137:head v 111'137 2026-02-11T23:31:28.191+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a4664b16:::obj367:head local-lis/les=117/118 n=386 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.191+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.191+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a53b2263:::obj137:head, version: 111'137, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a53b2263:::obj137:head@111'137, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:28.191+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a4664b16:::obj367:head local-lis/les=117/118 n=386 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.191+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.191+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a52fdb40:::obj548:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.191+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.191+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a52fdb40:::obj548:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 4.0 e 119/119 lb 4:a52fdb40:::obj548:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a4664b16:::obj367:head local-lis/les=117/118 n=386 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.191+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.191+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a4664b16:::obj367:head local-lis/les=117/118 n=386 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:a52fdb40:::obj548:head) v3 2026-02-11T23:31:28.191+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a4664b16:::obj367:head local-lis/les=117/118 n=386 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:a52fdb40:::obj548:head) 2026-02-11T23:31:28.191+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.191+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a52fdb40:::obj548:head) v3 finish latency 0.000013 2026-02-11T23:31:28.191+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.191+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.191+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a52fdb40:::obj548:head local-lis/les=117/118 n=387 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.191+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.191+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.191+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.191+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66fa780 con 0x507ac00 2026-02-11T23:31:28.191+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:a53b2263:::obj137:head)]) -- 0x66fa480 con 0x507ac00 2026-02-11T23:31:28.191+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.191+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 815 ==== MOSDPGPush(4.0 119/117 [PushOp(4:a555be2b:::obj75:head, version: 111'75, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a555be2b:::obj75:head@111'75, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x56a3ba0 con 0x507ac00 2026-02-11T23:31:28.191+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:a555be2b:::obj75:head, version: 111'75, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a555be2b:::obj75:head@111'75, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.191+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a555be2b:::obj75:head, version: 111'75, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a555be2b:::obj75:head@111'75, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.191+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.191+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a555be2b:::obj75:head, version: 111'75, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a555be2b:::obj75:head@111'75, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.191+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 816 ==== pg_backfill(progress 4.0 e 119/119 lb 4:a53b2263:::obj137:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9b700 con 0x507ac00 2026-02-11T23:31:28.191+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:a53b2263:::obj137:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.191+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.191+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a53b2263:::obj137:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.191+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a555be2b:::obj75:head, version: 111'75, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a555be2b:::obj75:head@111'75, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.191+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.191+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a555be2b:::obj75:head, version: 111'75, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a555be2b:::obj75:head@111'75, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 119/117 [PushOp(4:a555be2b:::obj75:head, version: 111'75, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a555be2b:::obj75:head@111'75, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a52fdb40:::obj548:head local-lis/les=117/118 n=387 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.191+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.191+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a52fdb40:::obj548:head local-lis/les=117/118 n=387 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:a555be2b:::obj75:head, version: 111'75, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a555be2b:::obj75:head@111'75, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.191+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a52fdb40:::obj548:head local-lis/les=117/118 n=387 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a555be2b:::obj75:head@111'75, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.191+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.191+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a53b2263:::obj137:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.191+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a52fdb40:::obj548:head local-lis/les=117/118 n=387 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:a555be2b:::obj75:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.191+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a52fdb40:::obj548:head local-lis/les=117/118 n=387 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:a555be2b:::obj75:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.191+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a52fdb40:::obj548:head local-lis/les=117/118 n=387 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:a555be2b:::obj75:head 2026-02-11T23:31:28.191+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a52fdb40:::obj548:head local-lis/les=117/118 n=387 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:a555be2b:::obj75:head v 111'75 2026-02-11T23:31:28.191+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a52fdb40:::obj548:head local-lis/les=117/118 n=387 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.191+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.191+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a555be2b:::obj75:head, version: 111'75, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a555be2b:::obj75:head@111'75, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:28.191+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a52fdb40:::obj548:head local-lis/les=117/118 n=387 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.191+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.191+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.191+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a53b2263:::obj137:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.191+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a53b2263:::obj137:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 4.0 e 119/119 lb 4:a53b2263:::obj137:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a52fdb40:::obj548:head local-lis/les=117/118 n=387 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.191+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.191+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a52fdb40:::obj548:head local-lis/les=117/118 n=387 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:a53b2263:::obj137:head) v3 2026-02-11T23:31:28.191+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a52fdb40:::obj548:head local-lis/les=117/118 n=387 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:a53b2263:::obj137:head) 2026-02-11T23:31:28.191+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.191+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a53b2263:::obj137:head) v3 finish latency 0.000013 2026-02-11T23:31:28.191+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.191+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.191+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a53b2263:::obj137:head local-lis/les=117/118 n=388 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.191+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.191+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.191+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.191+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66faa80 con 0x507ac00 2026-02-11T23:31:28.191+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:a555be2b:::obj75:head)]) -- 0x66fa900 con 0x507ac00 2026-02-11T23:31:28.191+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.192+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 817 ==== MOSDPGPush(4.0 119/117 [PushOp(4:a557efb1:::obj8:head, version: 111'8, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a557efb1:::obj8:head@111'8, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 4999+0+0 (unknown 0 0 0) 0x3814ea0 con 0x507ac00 2026-02-11T23:31:28.192+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:a557efb1:::obj8:head, version: 111'8, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a557efb1:::obj8:head@111'8, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.192+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a557efb1:::obj8:head, version: 111'8, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a557efb1:::obj8:head@111'8, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.192+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.192+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a557efb1:::obj8:head, version: 111'8, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a557efb1:::obj8:head@111'8, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.192+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 818 ==== pg_backfill(progress 4.0 e 119/119 lb 4:a555be2b:::obj75:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d98580 con 0x507ac00 2026-02-11T23:31:28.192+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:a555be2b:::obj75:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.192+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.192+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a557efb1:::obj8:head, version: 111'8, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a557efb1:::obj8:head@111'8, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.192+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a555be2b:::obj75:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.192+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a557efb1:::obj8:head, version: 111'8, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a557efb1:::obj8:head@111'8, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 119/117 [PushOp(4:a557efb1:::obj8:head, version: 111'8, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a557efb1:::obj8:head@111'8, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a53b2263:::obj137:head local-lis/les=117/118 n=388 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.192+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.192+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.192+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a555be2b:::obj75:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.192+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a53b2263:::obj137:head local-lis/les=117/118 n=388 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:a557efb1:::obj8:head, version: 111'8, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a557efb1:::obj8:head@111'8, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.192+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a53b2263:::obj137:head local-lis/les=117/118 n=388 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a557efb1:::obj8:head@111'8, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.192+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a53b2263:::obj137:head local-lis/les=117/118 n=388 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:a557efb1:::obj8:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.192+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a53b2263:::obj137:head local-lis/les=117/118 n=388 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:a557efb1:::obj8:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.192+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a53b2263:::obj137:head local-lis/les=117/118 n=388 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:a557efb1:::obj8:head 2026-02-11T23:31:28.192+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a53b2263:::obj137:head local-lis/les=117/118 n=388 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:a557efb1:::obj8:head v 111'8 2026-02-11T23:31:28.192+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a53b2263:::obj137:head local-lis/les=117/118 n=388 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.192+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.192+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a557efb1:::obj8:head, version: 111'8, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a557efb1:::obj8:head@111'8, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:28.192+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a53b2263:::obj137:head local-lis/les=117/118 n=388 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.192+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.192+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.192+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a555be2b:::obj75:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.192+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a555be2b:::obj75:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 4.0 e 119/119 lb 4:a555be2b:::obj75:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a53b2263:::obj137:head local-lis/les=117/118 n=388 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.192+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.192+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a53b2263:::obj137:head local-lis/les=117/118 n=388 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:a555be2b:::obj75:head) v3 2026-02-11T23:31:28.192+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a53b2263:::obj137:head local-lis/les=117/118 n=388 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:a555be2b:::obj75:head) 2026-02-11T23:31:28.192+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.192+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a555be2b:::obj75:head) v3 finish latency 0.000014 2026-02-11T23:31:28.192+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.192+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.192+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a555be2b:::obj75:head local-lis/les=117/118 n=389 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.192+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.192+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.192+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.192+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66fad80 con 0x507ac00 2026-02-11T23:31:28.192+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:a557efb1:::obj8:head)]) -- 0x66fac00 con 0x507ac00 2026-02-11T23:31:28.192+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.192+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 819 ==== MOSDPGPush(4.0 119/117 [PushOp(4:a5c2f6e2:::obj111:head, version: 111'111, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a5c2f6e2:::obj111:head@111'111, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c4e0 con 0x507ac00 2026-02-11T23:31:28.192+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:a5c2f6e2:::obj111:head, version: 111'111, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a5c2f6e2:::obj111:head@111'111, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.192+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a5c2f6e2:::obj111:head, version: 111'111, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a5c2f6e2:::obj111:head@111'111, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.192+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.192+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a5c2f6e2:::obj111:head, version: 111'111, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a5c2f6e2:::obj111:head@111'111, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.192+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 820 ==== pg_backfill(progress 4.0 e 119/119 lb 4:a557efb1:::obj8:head) ==== 1060+0+0 (unknown 0 0 0) 0x4a2c000 con 0x507ac00 2026-02-11T23:31:28.192+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.192+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:a557efb1:::obj8:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.192+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a5c2f6e2:::obj111:head, version: 111'111, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a5c2f6e2:::obj111:head@111'111, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.192+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a557efb1:::obj8:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.192+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a5c2f6e2:::obj111:head, version: 111'111, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a5c2f6e2:::obj111:head@111'111, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 119/117 [PushOp(4:a5c2f6e2:::obj111:head, version: 111'111, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a5c2f6e2:::obj111:head@111'111, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a555be2b:::obj75:head local-lis/les=117/118 n=389 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.192+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.192+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.192+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a557efb1:::obj8:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.192+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a555be2b:::obj75:head local-lis/les=117/118 n=389 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:a5c2f6e2:::obj111:head, version: 111'111, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a5c2f6e2:::obj111:head@111'111, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.192+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a555be2b:::obj75:head local-lis/les=117/118 n=389 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a5c2f6e2:::obj111:head@111'111, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.192+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a555be2b:::obj75:head local-lis/les=117/118 n=389 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:a5c2f6e2:::obj111:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.192+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a555be2b:::obj75:head local-lis/les=117/118 n=389 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:a5c2f6e2:::obj111:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.192+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a555be2b:::obj75:head local-lis/les=117/118 n=389 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:a5c2f6e2:::obj111:head 2026-02-11T23:31:28.192+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a555be2b:::obj75:head local-lis/les=117/118 n=389 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:a5c2f6e2:::obj111:head v 111'111 2026-02-11T23:31:28.192+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a555be2b:::obj75:head local-lis/les=117/118 n=389 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.192+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.192+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a5c2f6e2:::obj111:head, version: 111'111, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a5c2f6e2:::obj111:head@111'111, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000044 2026-02-11T23:31:28.192+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a555be2b:::obj75:head local-lis/les=117/118 n=389 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.192+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.192+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.192+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a557efb1:::obj8:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.192+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a557efb1:::obj8:head) v3 prio 15 cost 0 latency 0.000060 pg_backfill(progress 4.0 e 119/119 lb 4:a557efb1:::obj8:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a555be2b:::obj75:head local-lis/les=117/118 n=389 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.192+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.192+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a555be2b:::obj75:head local-lis/les=117/118 n=389 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:a557efb1:::obj8:head) v3 2026-02-11T23:31:28.192+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a555be2b:::obj75:head local-lis/les=117/118 n=389 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:a557efb1:::obj8:head) 2026-02-11T23:31:28.192+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.192+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a557efb1:::obj8:head) v3 finish latency 0.000013 2026-02-11T23:31:28.192+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.193+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.193+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a557efb1:::obj8:head local-lis/les=117/118 n=390 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.193+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.193+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.193+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.193+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66fb200 con 0x507ac00 2026-02-11T23:31:28.193+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:a5c2f6e2:::obj111:head)]) -- 0x66faf00 con 0x507ac00 2026-02-11T23:31:28.193+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.193+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 821 ==== MOSDPGPush(4.0 119/117 [PushOp(4:a5ea1435:::obj539:head, version: 111'539, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a5ea1435:::obj539:head@111'539, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f0d00 con 0x507ac00 2026-02-11T23:31:28.193+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:a5ea1435:::obj539:head, version: 111'539, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a5ea1435:::obj539:head@111'539, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.193+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a5ea1435:::obj539:head, version: 111'539, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a5ea1435:::obj539:head@111'539, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.193+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.193+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 822 ==== pg_backfill(progress 4.0 e 119/119 lb 4:a5c2f6e2:::obj111:head) ==== 1062+0+0 (unknown 0 0 0) 0x4a2cb00 con 0x507ac00 2026-02-11T23:31:28.193+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a5ea1435:::obj539:head, version: 111'539, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a5ea1435:::obj539:head@111'539, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.193+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:a5c2f6e2:::obj111:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.193+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.193+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a5c2f6e2:::obj111:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.193+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a5ea1435:::obj539:head, version: 111'539, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a5ea1435:::obj539:head@111'539, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.193+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a5ea1435:::obj539:head, version: 111'539, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a5ea1435:::obj539:head@111'539, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 119/117 [PushOp(4:a5ea1435:::obj539:head, version: 111'539, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a5ea1435:::obj539:head@111'539, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a557efb1:::obj8:head local-lis/les=117/118 n=390 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.193+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.193+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.193+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a557efb1:::obj8:head local-lis/les=117/118 n=390 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:a5ea1435:::obj539:head, version: 111'539, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a5ea1435:::obj539:head@111'539, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.193+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a557efb1:::obj8:head local-lis/les=117/118 n=390 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a5ea1435:::obj539:head@111'539, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.193+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.193+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a5c2f6e2:::obj111:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.193+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a557efb1:::obj8:head local-lis/les=117/118 n=390 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:a5ea1435:::obj539:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.193+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a557efb1:::obj8:head local-lis/les=117/118 n=390 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:a5ea1435:::obj539:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.193+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a557efb1:::obj8:head local-lis/les=117/118 n=390 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:a5ea1435:::obj539:head 2026-02-11T23:31:28.193+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a557efb1:::obj8:head local-lis/les=117/118 n=390 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:a5ea1435:::obj539:head v 111'539 2026-02-11T23:31:28.193+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a557efb1:::obj8:head local-lis/les=117/118 n=390 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.193+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.193+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a5ea1435:::obj539:head, version: 111'539, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a5ea1435:::obj539:head@111'539, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:28.193+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a557efb1:::obj8:head local-lis/les=117/118 n=390 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.193+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.193+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.193+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a5c2f6e2:::obj111:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.193+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a5c2f6e2:::obj111:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 4.0 e 119/119 lb 4:a5c2f6e2:::obj111:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a557efb1:::obj8:head local-lis/les=117/118 n=390 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.193+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.193+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a557efb1:::obj8:head local-lis/les=117/118 n=390 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:a5c2f6e2:::obj111:head) v3 2026-02-11T23:31:28.193+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a557efb1:::obj8:head local-lis/les=117/118 n=390 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:a5c2f6e2:::obj111:head) 2026-02-11T23:31:28.193+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.193+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a5c2f6e2:::obj111:head) v3 finish latency 0.000013 2026-02-11T23:31:28.193+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.193+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.193+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a5c2f6e2:::obj111:head local-lis/les=117/118 n=391 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.193+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.193+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.193+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.193+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66fb680 con 0x507ac00 2026-02-11T23:31:28.193+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:a5ea1435:::obj539:head)]) -- 0x66fb380 con 0x507ac00 2026-02-11T23:31:28.193+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.193+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 823 ==== MOSDPGPush(4.0 119/117 [PushOp(4:a601bbe9:::obj562:head, version: 111'562, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a601bbe9:::obj562:head@111'562, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56b64e0 con 0x507ac00 2026-02-11T23:31:28.193+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:a601bbe9:::obj562:head, version: 111'562, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a601bbe9:::obj562:head@111'562, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.193+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a601bbe9:::obj562:head, version: 111'562, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a601bbe9:::obj562:head@111'562, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.193+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.193+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a601bbe9:::obj562:head, version: 111'562, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a601bbe9:::obj562:head@111'562, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.193+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 824 ==== pg_backfill(progress 4.0 e 119/119 lb 4:a5ea1435:::obj539:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9ac00 con 0x507ac00 2026-02-11T23:31:28.193+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.193+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:a5ea1435:::obj539:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.193+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a601bbe9:::obj562:head, version: 111'562, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a601bbe9:::obj562:head@111'562, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.193+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a5ea1435:::obj539:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.193+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.193+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a601bbe9:::obj562:head, version: 111'562, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a601bbe9:::obj562:head@111'562, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 119/117 [PushOp(4:a601bbe9:::obj562:head, version: 111'562, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a601bbe9:::obj562:head@111'562, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a5c2f6e2:::obj111:head local-lis/les=117/118 n=391 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.193+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a5ea1435:::obj539:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.193+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.193+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a5c2f6e2:::obj111:head local-lis/les=117/118 n=391 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:a601bbe9:::obj562:head, version: 111'562, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a601bbe9:::obj562:head@111'562, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.193+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a5c2f6e2:::obj111:head local-lis/les=117/118 n=391 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a601bbe9:::obj562:head@111'562, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.193+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a5c2f6e2:::obj111:head local-lis/les=117/118 n=391 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:a601bbe9:::obj562:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.193+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a5c2f6e2:::obj111:head local-lis/les=117/118 n=391 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:a601bbe9:::obj562:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.193+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a5c2f6e2:::obj111:head local-lis/les=117/118 n=391 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:a601bbe9:::obj562:head 2026-02-11T23:31:28.193+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a5c2f6e2:::obj111:head local-lis/les=117/118 n=391 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:a601bbe9:::obj562:head v 111'562 2026-02-11T23:31:28.193+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a5c2f6e2:::obj111:head local-lis/les=117/118 n=391 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.193+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.194+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a601bbe9:::obj562:head, version: 111'562, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a601bbe9:::obj562:head@111'562, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:28.194+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a5c2f6e2:::obj111:head local-lis/les=117/118 n=391 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.194+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.194+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a5ea1435:::obj539:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.194+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.194+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a5ea1435:::obj539:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 4.0 e 119/119 lb 4:a5ea1435:::obj539:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a5c2f6e2:::obj111:head local-lis/les=117/118 n=391 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.194+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.194+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a5c2f6e2:::obj111:head local-lis/les=117/118 n=391 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:a5ea1435:::obj539:head) v3 2026-02-11T23:31:28.194+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a5c2f6e2:::obj111:head local-lis/les=117/118 n=391 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:a5ea1435:::obj539:head) 2026-02-11T23:31:28.194+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.194+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a5ea1435:::obj539:head) v3 finish latency 0.000013 2026-02-11T23:31:28.194+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.194+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a5ea1435:::obj539:head local-lis/les=117/118 n=392 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.194+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.194+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.194+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.194+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.194+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66fbb00 con 0x507ac00 2026-02-11T23:31:28.194+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:a601bbe9:::obj562:head)]) -- 0x66fb800 con 0x507ac00 2026-02-11T23:31:28.194+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.194+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 825 ==== MOSDPGPush(4.0 119/117 [PushOp(4:a61ad63e:::obj13:head, version: 111'13, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a61ad63e:::obj13:head@111'13, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x56caea0 con 0x507ac00 2026-02-11T23:31:28.194+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:a61ad63e:::obj13:head, version: 111'13, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a61ad63e:::obj13:head@111'13, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.194+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a61ad63e:::obj13:head, version: 111'13, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a61ad63e:::obj13:head@111'13, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.194+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.194+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a61ad63e:::obj13:head, version: 111'13, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a61ad63e:::obj13:head@111'13, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.194+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 826 ==== pg_backfill(progress 4.0 e 119/119 lb 4:a601bbe9:::obj562:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99600 con 0x507ac00 2026-02-11T23:31:28.194+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:a601bbe9:::obj562:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.194+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.194+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a61ad63e:::obj13:head, version: 111'13, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a61ad63e:::obj13:head@111'13, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.194+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a601bbe9:::obj562:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.194+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a61ad63e:::obj13:head, version: 111'13, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a61ad63e:::obj13:head@111'13, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 119/117 [PushOp(4:a61ad63e:::obj13:head, version: 111'13, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a61ad63e:::obj13:head@111'13, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a5ea1435:::obj539:head local-lis/les=117/118 n=392 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.194+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.194+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.194+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a601bbe9:::obj562:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.194+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a5ea1435:::obj539:head local-lis/les=117/118 n=392 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:a61ad63e:::obj13:head, version: 111'13, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a61ad63e:::obj13:head@111'13, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.194+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a5ea1435:::obj539:head local-lis/les=117/118 n=392 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a61ad63e:::obj13:head@111'13, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.194+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a5ea1435:::obj539:head local-lis/les=117/118 n=392 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:a61ad63e:::obj13:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.194+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a5ea1435:::obj539:head local-lis/les=117/118 n=392 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:a61ad63e:::obj13:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.194+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a5ea1435:::obj539:head local-lis/les=117/118 n=392 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:a61ad63e:::obj13:head 2026-02-11T23:31:28.194+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a5ea1435:::obj539:head local-lis/les=117/118 n=392 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:a61ad63e:::obj13:head v 111'13 2026-02-11T23:31:28.194+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a5ea1435:::obj539:head local-lis/les=117/118 n=392 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.194+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.194+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a61ad63e:::obj13:head, version: 111'13, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a61ad63e:::obj13:head@111'13, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:28.194+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a5ea1435:::obj539:head local-lis/les=117/118 n=392 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.194+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.194+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.194+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a601bbe9:::obj562:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.194+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a601bbe9:::obj562:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 4.0 e 119/119 lb 4:a601bbe9:::obj562:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a5ea1435:::obj539:head local-lis/les=117/118 n=392 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.194+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.194+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a5ea1435:::obj539:head local-lis/les=117/118 n=392 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:a601bbe9:::obj562:head) v3 2026-02-11T23:31:28.194+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a5ea1435:::obj539:head local-lis/les=117/118 n=392 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:a601bbe9:::obj562:head) 2026-02-11T23:31:28.194+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.194+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a601bbe9:::obj562:head) v3 finish latency 0.000015 2026-02-11T23:31:28.194+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.194+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.194+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a601bbe9:::obj562:head local-lis/les=117/118 n=393 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.194+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.194+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.194+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.194+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66fbe00 con 0x507ac00 2026-02-11T23:31:28.194+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.194+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:a61ad63e:::obj13:head)]) -- 0x66fbc80 con 0x507ac00 2026-02-11T23:31:28.194+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.195+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 827 ==== MOSDPGPush(4.0 119/117 [PushOp(4:a630f391:::obj375:head, version: 111'375, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a630f391:::obj375:head@111'375, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db0000 con 0x507ac00 2026-02-11T23:31:28.195+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:a630f391:::obj375:head, version: 111'375, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a630f391:::obj375:head@111'375, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.195+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a630f391:::obj375:head, version: 111'375, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a630f391:::obj375:head@111'375, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.195+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.195+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a630f391:::obj375:head, version: 111'375, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a630f391:::obj375:head@111'375, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.195+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 828 ==== pg_backfill(progress 4.0 e 119/119 lb 4:a61ad63e:::obj13:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d2d600 con 0x507ac00 2026-02-11T23:31:28.195+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:a61ad63e:::obj13:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.195+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.195+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a630f391:::obj375:head, version: 111'375, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a630f391:::obj375:head@111'375, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.195+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a61ad63e:::obj13:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.195+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.195+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a630f391:::obj375:head, version: 111'375, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a630f391:::obj375:head@111'375, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 119/117 [PushOp(4:a630f391:::obj375:head, version: 111'375, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a630f391:::obj375:head@111'375, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a601bbe9:::obj562:head local-lis/les=117/118 n=393 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.195+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.195+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a61ad63e:::obj13:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.195+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a601bbe9:::obj562:head local-lis/les=117/118 n=393 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:a630f391:::obj375:head, version: 111'375, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a630f391:::obj375:head@111'375, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.195+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a601bbe9:::obj562:head local-lis/les=117/118 n=393 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a630f391:::obj375:head@111'375, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.195+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a601bbe9:::obj562:head local-lis/les=117/118 n=393 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:a630f391:::obj375:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.195+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a601bbe9:::obj562:head local-lis/les=117/118 n=393 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:a630f391:::obj375:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.195+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a601bbe9:::obj562:head local-lis/les=117/118 n=393 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:a630f391:::obj375:head 2026-02-11T23:31:28.195+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a601bbe9:::obj562:head local-lis/les=117/118 n=393 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:a630f391:::obj375:head v 111'375 2026-02-11T23:31:28.195+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a601bbe9:::obj562:head local-lis/les=117/118 n=393 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.195+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.195+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a630f391:::obj375:head, version: 111'375, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a630f391:::obj375:head@111'375, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:28.195+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a601bbe9:::obj562:head local-lis/les=117/118 n=393 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.195+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.195+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a61ad63e:::obj13:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.195+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.195+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a61ad63e:::obj13:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 4.0 e 119/119 lb 4:a61ad63e:::obj13:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a601bbe9:::obj562:head local-lis/les=117/118 n=393 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.195+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.195+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a601bbe9:::obj562:head local-lis/les=117/118 n=393 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:a61ad63e:::obj13:head) v3 2026-02-11T23:31:28.195+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a601bbe9:::obj562:head local-lis/les=117/118 n=393 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:a61ad63e:::obj13:head) 2026-02-11T23:31:28.195+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.195+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a61ad63e:::obj13:head) v3 finish latency 0.000013 2026-02-11T23:31:28.195+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.195+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.195+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a61ad63e:::obj13:head local-lis/les=117/118 n=394 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.195+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.195+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.195+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.195+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66fa000 con 0x507ac00 2026-02-11T23:31:28.195+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:a630f391:::obj375:head)]) -- 0x66fa480 con 0x507ac00 2026-02-11T23:31:28.195+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.195+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 829 ==== MOSDPGPush(4.0 119/117 [PushOp(4:a6b27c5f:::obj420:head, version: 111'420, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a6b27c5f:::obj420:head@111'420, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a3a00 con 0x507ac00 2026-02-11T23:31:28.195+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:a6b27c5f:::obj420:head, version: 111'420, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a6b27c5f:::obj420:head@111'420, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.195+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a6b27c5f:::obj420:head, version: 111'420, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a6b27c5f:::obj420:head@111'420, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.195+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.195+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a6b27c5f:::obj420:head, version: 111'420, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a6b27c5f:::obj420:head@111'420, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.195+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 830 ==== pg_backfill(progress 4.0 e 119/119 lb 4:a630f391:::obj375:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2ec00 con 0x507ac00 2026-02-11T23:31:28.195+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.195+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:a630f391:::obj375:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.195+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a6b27c5f:::obj420:head, version: 111'420, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a6b27c5f:::obj420:head@111'420, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.195+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a630f391:::obj375:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.195+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a6b27c5f:::obj420:head, version: 111'420, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a6b27c5f:::obj420:head@111'420, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 119/117 [PushOp(4:a6b27c5f:::obj420:head, version: 111'420, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a6b27c5f:::obj420:head@111'420, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a61ad63e:::obj13:head local-lis/les=117/118 n=394 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.195+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.195+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.195+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a61ad63e:::obj13:head local-lis/les=117/118 n=394 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:a6b27c5f:::obj420:head, version: 111'420, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a6b27c5f:::obj420:head@111'420, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.195+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a630f391:::obj375:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.195+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a61ad63e:::obj13:head local-lis/les=117/118 n=394 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a6b27c5f:::obj420:head@111'420, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.195+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a61ad63e:::obj13:head local-lis/les=117/118 n=394 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:a6b27c5f:::obj420:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.195+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a61ad63e:::obj13:head local-lis/les=117/118 n=394 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:a6b27c5f:::obj420:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.195+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a61ad63e:::obj13:head local-lis/les=117/118 n=394 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:a6b27c5f:::obj420:head 2026-02-11T23:31:28.195+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a61ad63e:::obj13:head local-lis/les=117/118 n=394 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:a6b27c5f:::obj420:head v 111'420 2026-02-11T23:31:28.195+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a61ad63e:::obj13:head local-lis/les=117/118 n=394 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.195+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.195+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a6b27c5f:::obj420:head, version: 111'420, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a6b27c5f:::obj420:head@111'420, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:28.195+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a61ad63e:::obj13:head local-lis/les=117/118 n=394 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.195+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.195+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a630f391:::obj375:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.195+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.195+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a630f391:::obj375:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 4.0 e 119/119 lb 4:a630f391:::obj375:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a61ad63e:::obj13:head local-lis/les=117/118 n=394 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.195+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.195+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a61ad63e:::obj13:head local-lis/les=117/118 n=394 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:a630f391:::obj375:head) v3 2026-02-11T23:31:28.195+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a61ad63e:::obj13:head local-lis/les=117/118 n=394 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:a630f391:::obj375:head) 2026-02-11T23:31:28.195+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.195+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a630f391:::obj375:head) v3 finish latency 0.000013 2026-02-11T23:31:28.195+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.195+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.195+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a630f391:::obj375:head local-lis/les=117/118 n=395 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.196+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.196+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.196+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.196+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66d8480 con 0x507ac00 2026-02-11T23:31:28.196+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:a6b27c5f:::obj420:head)]) -- 0x66fa300 con 0x507ac00 2026-02-11T23:31:28.196+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.196+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 831 ==== MOSDPGPush(4.0 119/117 [PushOp(4:a6f174aa:::obj203:head, version: 111'203, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a6f174aa:::obj203:head@111'203, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2820 con 0x507ac00 2026-02-11T23:31:28.196+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:a6f174aa:::obj203:head, version: 111'203, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a6f174aa:::obj203:head@111'203, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.196+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a6f174aa:::obj203:head, version: 111'203, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a6f174aa:::obj203:head@111'203, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.196+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.196+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 832 ==== pg_backfill(progress 4.0 e 119/119 lb 4:a6b27c5f:::obj420:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98000 con 0x507ac00 2026-02-11T23:31:28.196+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a6f174aa:::obj203:head, version: 111'203, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a6f174aa:::obj203:head@111'203, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.196+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:a6b27c5f:::obj420:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.196+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.196+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a6b27c5f:::obj420:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.196+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a6f174aa:::obj203:head, version: 111'203, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a6f174aa:::obj203:head@111'203, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.196+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.196+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a6f174aa:::obj203:head, version: 111'203, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a6f174aa:::obj203:head@111'203, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 119/117 [PushOp(4:a6f174aa:::obj203:head, version: 111'203, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a6f174aa:::obj203:head@111'203, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a630f391:::obj375:head local-lis/les=117/118 n=395 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.196+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.196+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a630f391:::obj375:head local-lis/les=117/118 n=395 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:a6f174aa:::obj203:head, version: 111'203, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a6f174aa:::obj203:head@111'203, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.196+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a630f391:::obj375:head local-lis/les=117/118 n=395 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a6f174aa:::obj203:head@111'203, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.196+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.196+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a6b27c5f:::obj420:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.196+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a630f391:::obj375:head local-lis/les=117/118 n=395 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:a6f174aa:::obj203:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.196+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a630f391:::obj375:head local-lis/les=117/118 n=395 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:a6f174aa:::obj203:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.196+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a630f391:::obj375:head local-lis/les=117/118 n=395 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:a6f174aa:::obj203:head 2026-02-11T23:31:28.196+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a630f391:::obj375:head local-lis/les=117/118 n=395 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:a6f174aa:::obj203:head v 111'203 2026-02-11T23:31:28.196+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a630f391:::obj375:head local-lis/les=117/118 n=395 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.196+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.196+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a6f174aa:::obj203:head, version: 111'203, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a6f174aa:::obj203:head@111'203, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:28.196+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a630f391:::obj375:head local-lis/les=117/118 n=395 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.196+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.196+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.196+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a6b27c5f:::obj420:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.196+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a6b27c5f:::obj420:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 4.0 e 119/119 lb 4:a6b27c5f:::obj420:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a630f391:::obj375:head local-lis/les=117/118 n=395 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.196+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.196+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a630f391:::obj375:head local-lis/les=117/118 n=395 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:a6b27c5f:::obj420:head) v3 2026-02-11T23:31:28.196+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a630f391:::obj375:head local-lis/les=117/118 n=395 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:a6b27c5f:::obj420:head) 2026-02-11T23:31:28.196+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.196+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a6b27c5f:::obj420:head) v3 finish latency 0.000013 2026-02-11T23:31:28.196+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.196+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.196+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a6b27c5f:::obj420:head local-lis/les=117/118 n=396 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.196+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.196+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.196+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.196+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66d9500 con 0x507ac00 2026-02-11T23:31:28.196+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:a6f174aa:::obj203:head)]) -- 0x66d8a80 con 0x507ac00 2026-02-11T23:31:28.196+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.196+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 833 ==== MOSDPGPush(4.0 119/117 [PushOp(4:a8709cbe:::obj458:head, version: 111'458, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a8709cbe:::obj458:head@111'458, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f1860 con 0x507ac00 2026-02-11T23:31:28.196+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:a8709cbe:::obj458:head, version: 111'458, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a8709cbe:::obj458:head@111'458, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.196+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a8709cbe:::obj458:head, version: 111'458, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a8709cbe:::obj458:head@111'458, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.196+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.196+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a8709cbe:::obj458:head, version: 111'458, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a8709cbe:::obj458:head@111'458, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.196+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 834 ==== pg_backfill(progress 4.0 e 119/119 lb 4:a6f174aa:::obj203:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9b180 con 0x507ac00 2026-02-11T23:31:28.196+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:a6f174aa:::obj203:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.196+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.196+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a6f174aa:::obj203:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.196+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a8709cbe:::obj458:head, version: 111'458, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a8709cbe:::obj458:head@111'458, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.196+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a8709cbe:::obj458:head, version: 111'458, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a8709cbe:::obj458:head@111'458, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 119/117 [PushOp(4:a8709cbe:::obj458:head, version: 111'458, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a8709cbe:::obj458:head@111'458, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a6b27c5f:::obj420:head local-lis/les=117/118 n=396 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.196+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.196+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.196+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a6b27c5f:::obj420:head local-lis/les=117/118 n=396 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:a8709cbe:::obj458:head, version: 111'458, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a8709cbe:::obj458:head@111'458, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.196+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a6b27c5f:::obj420:head local-lis/les=117/118 n=396 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a8709cbe:::obj458:head@111'458, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.196+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.196+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a6f174aa:::obj203:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.196+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a6b27c5f:::obj420:head local-lis/les=117/118 n=396 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:a8709cbe:::obj458:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.197+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a6b27c5f:::obj420:head local-lis/les=117/118 n=396 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:a8709cbe:::obj458:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.197+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a6b27c5f:::obj420:head local-lis/les=117/118 n=396 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:a8709cbe:::obj458:head 2026-02-11T23:31:28.197+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a6b27c5f:::obj420:head local-lis/les=117/118 n=396 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:a8709cbe:::obj458:head v 111'458 2026-02-11T23:31:28.197+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a6b27c5f:::obj420:head local-lis/les=117/118 n=396 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.197+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.197+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a8709cbe:::obj458:head, version: 111'458, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a8709cbe:::obj458:head@111'458, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000062 2026-02-11T23:31:28.197+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a6b27c5f:::obj420:head local-lis/les=117/118 n=396 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.197+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.197+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.197+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a6f174aa:::obj203:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.197+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a6f174aa:::obj203:head) v3 prio 15 cost 0 latency 0.000081 pg_backfill(progress 4.0 e 119/119 lb 4:a6f174aa:::obj203:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a6b27c5f:::obj420:head local-lis/les=117/118 n=396 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.197+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.197+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a6b27c5f:::obj420:head local-lis/les=117/118 n=396 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:a6f174aa:::obj203:head) v3 2026-02-11T23:31:28.197+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a6b27c5f:::obj420:head local-lis/les=117/118 n=396 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:a6f174aa:::obj203:head) 2026-02-11T23:31:28.197+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.197+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a6f174aa:::obj203:head) v3 finish latency 0.000013 2026-02-11T23:31:28.197+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.197+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.197+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a6f174aa:::obj203:head local-lis/les=117/118 n=397 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.197+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.197+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.197+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.197+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66d9200 con 0x507ac00 2026-02-11T23:31:28.197+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:a8709cbe:::obj458:head)]) -- 0x66d9080 con 0x507ac00 2026-02-11T23:31:28.197+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.197+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 835 ==== MOSDPGPush(4.0 119/117 [PushOp(4:a87532ae:::obj194:head, version: 111'194, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a87532ae:::obj194:head@111'194, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559d520 con 0x507ac00 2026-02-11T23:31:28.197+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:a87532ae:::obj194:head, version: 111'194, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a87532ae:::obj194:head@111'194, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.197+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a87532ae:::obj194:head, version: 111'194, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a87532ae:::obj194:head@111'194, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.197+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.197+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a87532ae:::obj194:head, version: 111'194, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a87532ae:::obj194:head@111'194, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.197+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 836 ==== pg_backfill(progress 4.0 e 119/119 lb 4:a8709cbe:::obj458:head) ==== 1062+0+0 (unknown 0 0 0) 0x6179180 con 0x507ac00 2026-02-11T23:31:28.197+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:a8709cbe:::obj458:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.197+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.197+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a8709cbe:::obj458:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.197+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a87532ae:::obj194:head, version: 111'194, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a87532ae:::obj194:head@111'194, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.197+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.197+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a87532ae:::obj194:head, version: 111'194, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a87532ae:::obj194:head@111'194, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 119/117 [PushOp(4:a87532ae:::obj194:head, version: 111'194, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a87532ae:::obj194:head@111'194, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a6f174aa:::obj203:head local-lis/les=117/118 n=397 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.197+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.197+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a6f174aa:::obj203:head local-lis/les=117/118 n=397 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:a87532ae:::obj194:head, version: 111'194, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a87532ae:::obj194:head@111'194, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.197+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a6f174aa:::obj203:head local-lis/les=117/118 n=397 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a87532ae:::obj194:head@111'194, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.197+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.197+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a8709cbe:::obj458:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.197+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a6f174aa:::obj203:head local-lis/les=117/118 n=397 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:a87532ae:::obj194:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.197+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a6f174aa:::obj203:head local-lis/les=117/118 n=397 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:a87532ae:::obj194:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.197+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a6f174aa:::obj203:head local-lis/les=117/118 n=397 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:a87532ae:::obj194:head 2026-02-11T23:31:28.197+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a6f174aa:::obj203:head local-lis/les=117/118 n=397 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:a87532ae:::obj194:head v 111'194 2026-02-11T23:31:28.197+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a6f174aa:::obj203:head local-lis/les=117/118 n=397 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.197+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.197+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a87532ae:::obj194:head, version: 111'194, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a87532ae:::obj194:head@111'194, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:28.197+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a6f174aa:::obj203:head local-lis/les=117/118 n=397 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.197+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.197+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.197+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a8709cbe:::obj458:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.197+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a8709cbe:::obj458:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 4.0 e 119/119 lb 4:a8709cbe:::obj458:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a6f174aa:::obj203:head local-lis/les=117/118 n=397 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.197+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.197+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a6f174aa:::obj203:head local-lis/les=117/118 n=397 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:a8709cbe:::obj458:head) v3 2026-02-11T23:31:28.197+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a6f174aa:::obj203:head local-lis/les=117/118 n=397 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:a8709cbe:::obj458:head) 2026-02-11T23:31:28.197+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.197+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a8709cbe:::obj458:head) v3 finish latency 0.000013 2026-02-11T23:31:28.197+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.197+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.197+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a8709cbe:::obj458:head local-lis/les=117/118 n=398 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.197+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.197+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.197+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.197+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x640f380 con 0x507ac00 2026-02-11T23:31:28.197+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:a87532ae:::obj194:head)]) -- 0x640f680 con 0x507ac00 2026-02-11T23:31:28.197+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.198+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 837 ==== MOSDPGPush(4.0 119/117 [PushOp(4:a8d6b24d:::obj229:head, version: 111'229, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a8d6b24d:::obj229:head@111'229, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56ca9c0 con 0x507ac00 2026-02-11T23:31:28.198+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:a8d6b24d:::obj229:head, version: 111'229, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a8d6b24d:::obj229:head@111'229, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.198+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a8d6b24d:::obj229:head, version: 111'229, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a8d6b24d:::obj229:head@111'229, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.198+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.198+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a8d6b24d:::obj229:head, version: 111'229, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a8d6b24d:::obj229:head@111'229, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.198+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 838 ==== pg_backfill(progress 4.0 e 119/119 lb 4:a87532ae:::obj194:head) ==== 1062+0+0 (unknown 0 0 0) 0x6178100 con 0x507ac00 2026-02-11T23:31:28.198+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.198+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:a87532ae:::obj194:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.198+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:a8d6b24d:::obj229:head, version: 111'229, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a8d6b24d:::obj229:head@111'229, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.198+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a87532ae:::obj194:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.198+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a8d6b24d:::obj229:head, version: 111'229, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a8d6b24d:::obj229:head@111'229, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 119/117 [PushOp(4:a8d6b24d:::obj229:head, version: 111'229, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a8d6b24d:::obj229:head@111'229, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a8709cbe:::obj458:head local-lis/les=117/118 n=398 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.198+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.198+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.198+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a8709cbe:::obj458:head local-lis/les=117/118 n=398 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:a8d6b24d:::obj229:head, version: 111'229, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a8d6b24d:::obj229:head@111'229, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.198+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a87532ae:::obj194:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.198+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a8709cbe:::obj458:head local-lis/les=117/118 n=398 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a8d6b24d:::obj229:head@111'229, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.198+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a8709cbe:::obj458:head local-lis/les=117/118 n=398 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:a8d6b24d:::obj229:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.198+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a8709cbe:::obj458:head local-lis/les=117/118 n=398 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:a8d6b24d:::obj229:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.198+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a8709cbe:::obj458:head local-lis/les=117/118 n=398 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:a8d6b24d:::obj229:head 2026-02-11T23:31:28.198+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a8709cbe:::obj458:head local-lis/les=117/118 n=398 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:a8d6b24d:::obj229:head v 111'229 2026-02-11T23:31:28.198+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a8709cbe:::obj458:head local-lis/les=117/118 n=398 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.198+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.198+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:a8d6b24d:::obj229:head, version: 111'229, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a8d6b24d:::obj229:head@111'229, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000043 2026-02-11T23:31:28.198+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a8709cbe:::obj458:head local-lis/les=117/118 n=398 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.198+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.198+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.198+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a87532ae:::obj194:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.198+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a87532ae:::obj194:head) v3 prio 15 cost 0 latency 0.000059 pg_backfill(progress 4.0 e 119/119 lb 4:a87532ae:::obj194:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a8709cbe:::obj458:head local-lis/les=117/118 n=398 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.198+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.198+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a8709cbe:::obj458:head local-lis/les=117/118 n=398 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:a87532ae:::obj194:head) v3 2026-02-11T23:31:28.198+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a8709cbe:::obj458:head local-lis/les=117/118 n=398 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:a87532ae:::obj194:head) 2026-02-11T23:31:28.198+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.198+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a87532ae:::obj194:head) v3 finish latency 0.000014 2026-02-11T23:31:28.198+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.198+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.198+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a87532ae:::obj194:head local-lis/les=117/118 n=399 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.198+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.198+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.198+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.198+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c9e00 con 0x507ac00 2026-02-11T23:31:28.198+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:a8d6b24d:::obj229:head)]) -- 0x640f980 con 0x507ac00 2026-02-11T23:31:28.198+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.198+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.198+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 839 ==== MOSDPGPush(4.0 119/117 [PushOp(4:aa1e2680:::obj587:head, version: 111'587, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aa1e2680:::obj587:head@111'587, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c000 con 0x507ac00 2026-02-11T23:31:28.198+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:aa1e2680:::obj587:head, version: 111'587, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aa1e2680:::obj587:head@111'587, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.198+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:aa1e2680:::obj587:head, version: 111'587, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aa1e2680:::obj587:head@111'587, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.198+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.198+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:aa1e2680:::obj587:head, version: 111'587, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aa1e2680:::obj587:head@111'587, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.198+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.198+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:aa1e2680:::obj587:head, version: 111'587, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aa1e2680:::obj587:head@111'587, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.198+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.198+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:aa1e2680:::obj587:head, version: 111'587, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aa1e2680:::obj587:head@111'587, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 119/117 [PushOp(4:aa1e2680:::obj587:head, version: 111'587, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aa1e2680:::obj587:head@111'587, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a87532ae:::obj194:head local-lis/les=117/118 n=399 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.198+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.198+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 840 ==== pg_backfill(progress 4.0 e 119/119 lb 4:a8d6b24d:::obj229:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2c580 con 0x507ac00 2026-02-11T23:31:28.198+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a87532ae:::obj194:head local-lis/les=117/118 n=399 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:aa1e2680:::obj587:head, version: 111'587, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aa1e2680:::obj587:head@111'587, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.198+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:a8d6b24d:::obj229:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.198+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a87532ae:::obj194:head local-lis/les=117/118 n=399 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:aa1e2680:::obj587:head@111'587, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.198+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a8d6b24d:::obj229:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.198+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a87532ae:::obj194:head local-lis/les=117/118 n=399 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:aa1e2680:::obj587:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.198+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.198+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a87532ae:::obj194:head local-lis/les=117/118 n=399 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:aa1e2680:::obj587:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.198+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a8d6b24d:::obj229:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.198+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a87532ae:::obj194:head local-lis/les=117/118 n=399 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:aa1e2680:::obj587:head 2026-02-11T23:31:28.198+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a87532ae:::obj194:head local-lis/les=117/118 n=399 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:aa1e2680:::obj587:head v 111'587 2026-02-11T23:31:28.198+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a87532ae:::obj194:head local-lis/les=117/118 n=399 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.198+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.198+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:aa1e2680:::obj587:head, version: 111'587, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aa1e2680:::obj587:head@111'587, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000042 2026-02-11T23:31:28.198+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a87532ae:::obj194:head local-lis/les=117/118 n=399 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.198+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.198+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.198+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:a8d6b24d:::obj229:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.198+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a8d6b24d:::obj229:head) v3 prio 15 cost 0 latency 0.000046 pg_backfill(progress 4.0 e 119/119 lb 4:a8d6b24d:::obj229:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a87532ae:::obj194:head local-lis/les=117/118 n=399 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.198+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.198+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a87532ae:::obj194:head local-lis/les=117/118 n=399 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:a8d6b24d:::obj229:head) v3 2026-02-11T23:31:28.198+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a87532ae:::obj194:head local-lis/les=117/118 n=399 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:a8d6b24d:::obj229:head) 2026-02-11T23:31:28.198+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.198+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:a8d6b24d:::obj229:head) v3 finish latency 0.000013 2026-02-11T23:31:28.198+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.199+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.199+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a8d6b24d:::obj229:head local-lis/les=117/118 n=400 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.199+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.199+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.199+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.199+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x642cd80 con 0x507ac00 2026-02-11T23:31:28.199+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:aa1e2680:::obj587:head)]) -- 0x5d55200 con 0x507ac00 2026-02-11T23:31:28.199+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.199+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.199+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 841 ==== MOSDPGPush(4.0 119/117 [PushOp(4:ab0f5189:::obj85:head, version: 111'85, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ab0f5189:::obj85:head@111'85, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x56a2b60 con 0x507ac00 2026-02-11T23:31:28.199+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:ab0f5189:::obj85:head, version: 111'85, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ab0f5189:::obj85:head@111'85, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.199+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ab0f5189:::obj85:head, version: 111'85, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ab0f5189:::obj85:head@111'85, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.199+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.199+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ab0f5189:::obj85:head, version: 111'85, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ab0f5189:::obj85:head@111'85, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.199+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 842 ==== pg_backfill(progress 4.0 e 119/119 lb 4:aa1e2680:::obj587:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177b80 con 0x507ac00 2026-02-11T23:31:28.199+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.199+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:aa1e2680:::obj587:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.199+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ab0f5189:::obj85:head, version: 111'85, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ab0f5189:::obj85:head@111'85, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.199+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:aa1e2680:::obj587:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.199+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.199+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:ab0f5189:::obj85:head, version: 111'85, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ab0f5189:::obj85:head@111'85, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 119/117 [PushOp(4:ab0f5189:::obj85:head, version: 111'85, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ab0f5189:::obj85:head@111'85, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a8d6b24d:::obj229:head local-lis/les=117/118 n=400 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.199+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.199+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:aa1e2680:::obj587:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.199+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a8d6b24d:::obj229:head local-lis/les=117/118 n=400 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:ab0f5189:::obj85:head, version: 111'85, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ab0f5189:::obj85:head@111'85, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.199+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a8d6b24d:::obj229:head local-lis/les=117/118 n=400 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:ab0f5189:::obj85:head@111'85, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.199+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a8d6b24d:::obj229:head local-lis/les=117/118 n=400 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:ab0f5189:::obj85:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.199+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a8d6b24d:::obj229:head local-lis/les=117/118 n=400 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:ab0f5189:::obj85:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.199+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a8d6b24d:::obj229:head local-lis/les=117/118 n=400 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:ab0f5189:::obj85:head 2026-02-11T23:31:28.199+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a8d6b24d:::obj229:head local-lis/les=117/118 n=400 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:ab0f5189:::obj85:head v 111'85 2026-02-11T23:31:28.199+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a8d6b24d:::obj229:head local-lis/les=117/118 n=400 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.199+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.199+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:ab0f5189:::obj85:head, version: 111'85, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ab0f5189:::obj85:head@111'85, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000043 2026-02-11T23:31:28.199+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a8d6b24d:::obj229:head local-lis/les=117/118 n=400 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.199+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.199+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.199+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:aa1e2680:::obj587:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.199+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:aa1e2680:::obj587:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 4.0 e 119/119 lb 4:aa1e2680:::obj587:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:a8d6b24d:::obj229:head local-lis/les=117/118 n=400 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.199+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.199+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a8d6b24d:::obj229:head local-lis/les=117/118 n=400 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:aa1e2680:::obj587:head) v3 2026-02-11T23:31:28.199+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:a8d6b24d:::obj229:head local-lis/les=117/118 n=400 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:aa1e2680:::obj587:head) 2026-02-11T23:31:28.199+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.199+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:aa1e2680:::obj587:head) v3 finish latency 0.000013 2026-02-11T23:31:28.199+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.199+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.199+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:aa1e2680:::obj587:head local-lis/les=117/118 n=401 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.199+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.199+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.199+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.199+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63ffb00 con 0x507ac00 2026-02-11T23:31:28.199+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:ab0f5189:::obj85:head)]) -- 0x640fb00 con 0x507ac00 2026-02-11T23:31:28.199+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.199+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.200+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 843 ==== MOSDPGPush(4.0 119/117 [PushOp(4:ab946124:::obj15:head, version: 111'15, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ab946124:::obj15:head@111'15, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x554e680 con 0x507ac00 2026-02-11T23:31:28.200+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:ab946124:::obj15:head, version: 111'15, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ab946124:::obj15:head@111'15, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.200+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ab946124:::obj15:head, version: 111'15, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ab946124:::obj15:head@111'15, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.200+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.200+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ab946124:::obj15:head, version: 111'15, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ab946124:::obj15:head@111'15, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.200+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 844 ==== pg_backfill(progress 4.0 e 119/119 lb 4:ab0f5189:::obj85:head) ==== 1061+0+0 (unknown 0 0 0) 0x6177600 con 0x507ac00 2026-02-11T23:31:28.200+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:ab0f5189:::obj85:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.200+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.200+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ab946124:::obj15:head, version: 111'15, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ab946124:::obj15:head@111'15, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.200+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:ab0f5189:::obj85:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.200+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.200+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:ab946124:::obj15:head, version: 111'15, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ab946124:::obj15:head@111'15, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 119/117 [PushOp(4:ab946124:::obj15:head, version: 111'15, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ab946124:::obj15:head@111'15, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:aa1e2680:::obj587:head local-lis/les=117/118 n=401 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.200+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:ab0f5189:::obj85:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.200+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.200+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:aa1e2680:::obj587:head local-lis/les=117/118 n=401 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:ab946124:::obj15:head, version: 111'15, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ab946124:::obj15:head@111'15, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.200+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:aa1e2680:::obj587:head local-lis/les=117/118 n=401 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:ab946124:::obj15:head@111'15, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.200+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:aa1e2680:::obj587:head local-lis/les=117/118 n=401 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:ab946124:::obj15:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.200+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:aa1e2680:::obj587:head local-lis/les=117/118 n=401 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:ab946124:::obj15:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.200+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:aa1e2680:::obj587:head local-lis/les=117/118 n=401 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:ab946124:::obj15:head 2026-02-11T23:31:28.200+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:aa1e2680:::obj587:head local-lis/les=117/118 n=401 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:ab946124:::obj15:head v 111'15 2026-02-11T23:31:28.200+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:aa1e2680:::obj587:head local-lis/les=117/118 n=401 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.200+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.200+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:ab946124:::obj15:head, version: 111'15, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ab946124:::obj15:head@111'15, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:28.200+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:aa1e2680:::obj587:head local-lis/les=117/118 n=401 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.200+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.200+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.200+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:ab0f5189:::obj85:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.200+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:ab0f5189:::obj85:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 4.0 e 119/119 lb 4:ab0f5189:::obj85:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:aa1e2680:::obj587:head local-lis/les=117/118 n=401 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.200+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.200+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:aa1e2680:::obj587:head local-lis/les=117/118 n=401 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:ab0f5189:::obj85:head) v3 2026-02-11T23:31:28.200+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:aa1e2680:::obj587:head local-lis/les=117/118 n=401 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:ab0f5189:::obj85:head) 2026-02-11T23:31:28.200+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.200+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:ab0f5189:::obj85:head) v3 finish latency 0.000015 2026-02-11T23:31:28.200+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.200+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.200+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ab0f5189:::obj85:head local-lis/les=117/118 n=402 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.200+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.200+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.200+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.200+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x642d500 con 0x507ac00 2026-02-11T23:31:28.200+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:ab946124:::obj15:head)]) -- 0x640e180 con 0x507ac00 2026-02-11T23:31:28.200+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.200+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 845 ==== MOSDPGPush(4.0 119/117 [PushOp(4:ac7304c6:::obj483:head, version: 111'483, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ac7304c6:::obj483:head@111'483, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3815520 con 0x507ac00 2026-02-11T23:31:28.200+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:ac7304c6:::obj483:head, version: 111'483, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ac7304c6:::obj483:head@111'483, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.200+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ac7304c6:::obj483:head, version: 111'483, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ac7304c6:::obj483:head@111'483, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.200+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.200+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 846 ==== pg_backfill(progress 4.0 e 119/119 lb 4:ab946124:::obj15:head) ==== 1061+0+0 (unknown 0 0 0) 0x5985700 con 0x507ac00 2026-02-11T23:31:28.200+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ac7304c6:::obj483:head, version: 111'483, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ac7304c6:::obj483:head@111'483, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.200+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:ab946124:::obj15:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.200+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:ab946124:::obj15:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.200+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.200+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ac7304c6:::obj483:head, version: 111'483, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ac7304c6:::obj483:head@111'483, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.200+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.200+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:ac7304c6:::obj483:head, version: 111'483, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ac7304c6:::obj483:head@111'483, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(4.0 119/117 [PushOp(4:ac7304c6:::obj483:head, version: 111'483, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ac7304c6:::obj483:head@111'483, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:ab0f5189:::obj85:head local-lis/les=117/118 n=402 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.200+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.200+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.200+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:ab946124:::obj15:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.200+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ab0f5189:::obj85:head local-lis/les=117/118 n=402 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:ac7304c6:::obj483:head, version: 111'483, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ac7304c6:::obj483:head@111'483, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.200+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ab0f5189:::obj85:head local-lis/les=117/118 n=402 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:ac7304c6:::obj483:head@111'483, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.200+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ab0f5189:::obj85:head local-lis/les=117/118 n=402 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:ac7304c6:::obj483:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.200+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ab0f5189:::obj85:head local-lis/les=117/118 n=402 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:ac7304c6:::obj483:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.200+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ab0f5189:::obj85:head local-lis/les=117/118 n=402 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:ac7304c6:::obj483:head 2026-02-11T23:31:28.200+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ab0f5189:::obj85:head local-lis/les=117/118 n=402 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:ac7304c6:::obj483:head v 111'483 2026-02-11T23:31:28.200+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ab0f5189:::obj85:head local-lis/les=117/118 n=402 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.200+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.200+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:ac7304c6:::obj483:head, version: 111'483, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ac7304c6:::obj483:head@111'483, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000066 2026-02-11T23:31:28.200+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.200+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.200+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:ab946124:::obj15:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.200+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:ab946124:::obj15:head) v3 prio 15 cost 0 latency 0.000085 pg_backfill(progress 4.0 e 119/119 lb 4:ab946124:::obj15:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:ab0f5189:::obj85:head local-lis/les=117/118 n=402 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.200+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.200+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ab0f5189:::obj85:head local-lis/les=117/118 n=402 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:ab946124:::obj15:head) v3 2026-02-11T23:31:28.200+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ab0f5189:::obj85:head local-lis/les=117/118 n=402 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:ab946124:::obj15:head) 2026-02-11T23:31:28.200+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.200+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:ab946124:::obj15:head) v3 finish latency 0.000018 2026-02-11T23:31:28.200+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ab946124:::obj15:head local-lis/les=117/118 n=403 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.200+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.201+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.201+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ab946124:::obj15:head local-lis/les=117/118 n=403 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.201+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.201+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.201+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.201+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x649e600 con 0x507ac00 2026-02-11T23:31:28.201+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:ac7304c6:::obj483:head)]) -- 0x642d800 con 0x507ac00 2026-02-11T23:31:28.201+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.201+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 847 ==== MOSDPGPush(4.0 119/117 [PushOp(4:acb8a831:::obj238:head, version: 111'238, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:acb8a831:::obj238:head@111'238, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56171e0 con 0x507ac00 2026-02-11T23:31:28.201+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:acb8a831:::obj238:head, version: 111'238, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:acb8a831:::obj238:head@111'238, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 119 2026-02-11T23:31:28.201+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:acb8a831:::obj238:head, version: 111'238, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:acb8a831:::obj238:head@111'238, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.201+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.201+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:acb8a831:::obj238:head, version: 111'238, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:acb8a831:::obj238:head@111'238, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.201+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 848 ==== pg_backfill(progress 4.0 e 119/119 lb 4:ac7304c6:::obj483:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9a680 con 0x507ac00 2026-02-11T23:31:28.201+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:ac7304c6:::obj483:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.201+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.201+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:ac7304c6:::obj483:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.201+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:acb8a831:::obj238:head, version: 111'238, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:acb8a831:::obj238:head@111'238, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.201+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.201+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:ac7304c6:::obj483:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.201+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:acb8a831:::obj238:head, version: 111'238, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:acb8a831:::obj238:head@111'238, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 119/117 [PushOp(4:acb8a831:::obj238:head, version: 111'238, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:acb8a831:::obj238:head@111'238, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:ab946124:::obj15:head local-lis/les=117/118 n=403 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.201+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.201+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ab946124:::obj15:head local-lis/les=117/118 n=403 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:acb8a831:::obj238:head, version: 111'238, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:acb8a831:::obj238:head@111'238, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.201+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ab946124:::obj15:head local-lis/les=117/118 n=403 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:acb8a831:::obj238:head@111'238, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.201+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ab946124:::obj15:head local-lis/les=117/118 n=403 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:acb8a831:::obj238:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.201+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ab946124:::obj15:head local-lis/les=117/118 n=403 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:acb8a831:::obj238:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.201+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ab946124:::obj15:head local-lis/les=117/118 n=403 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:acb8a831:::obj238:head 2026-02-11T23:31:28.201+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ab946124:::obj15:head local-lis/les=117/118 n=403 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:acb8a831:::obj238:head v 111'238 2026-02-11T23:31:28.201+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ab946124:::obj15:head local-lis/les=117/118 n=403 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.201+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.201+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:acb8a831:::obj238:head, version: 111'238, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:acb8a831:::obj238:head@111'238, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:28.201+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ab946124:::obj15:head local-lis/les=117/118 n=403 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.201+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.201+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.201+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:ac7304c6:::obj483:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.201+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:ac7304c6:::obj483:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 4.0 e 119/119 lb 4:ac7304c6:::obj483:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:ab946124:::obj15:head local-lis/les=117/118 n=403 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.201+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.201+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ab946124:::obj15:head local-lis/les=117/118 n=403 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:ac7304c6:::obj483:head) v3 2026-02-11T23:31:28.201+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ab946124:::obj15:head local-lis/les=117/118 n=403 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:ac7304c6:::obj483:head) 2026-02-11T23:31:28.201+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.201+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:ac7304c6:::obj483:head) v3 finish latency 0.000017 2026-02-11T23:31:28.201+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.201+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.201+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ac7304c6:::obj483:head local-lis/les=117/118 n=404 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.201+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.201+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.201+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.201+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63ff200 con 0x507ac00 2026-02-11T23:31:28.201+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:acb8a831:::obj238:head)]) -- 0x4f7aa80 con 0x507ac00 2026-02-11T23:31:28.201+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.202+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 849 ==== MOSDPGPush(4.0 119/117 [PushOp(4:acfc8dd9:::obj74:head, version: 111'74, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:acfc8dd9:::obj74:head@111'74, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x56cad00 con 0x507ac00 2026-02-11T23:31:28.202+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:acfc8dd9:::obj74:head, version: 111'74, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:acfc8dd9:::obj74:head@111'74, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.202+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:acfc8dd9:::obj74:head, version: 111'74, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:acfc8dd9:::obj74:head@111'74, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.202+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.202+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 850 ==== pg_backfill(progress 4.0 e 119/119 lb 4:acb8a831:::obj238:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176000 con 0x507ac00 2026-02-11T23:31:28.202+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:acfc8dd9:::obj74:head, version: 111'74, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:acfc8dd9:::obj74:head@111'74, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.202+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:acb8a831:::obj238:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.202+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:acb8a831:::obj238:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.202+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.202+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:acfc8dd9:::obj74:head, version: 111'74, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:acfc8dd9:::obj74:head@111'74, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.202+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:acfc8dd9:::obj74:head, version: 111'74, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:acfc8dd9:::obj74:head@111'74, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 119/117 [PushOp(4:acfc8dd9:::obj74:head, version: 111'74, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:acfc8dd9:::obj74:head@111'74, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:ac7304c6:::obj483:head local-lis/les=117/118 n=404 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.202+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.202+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.202+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:acb8a831:::obj238:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.202+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ac7304c6:::obj483:head local-lis/les=117/118 n=404 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:acfc8dd9:::obj74:head, version: 111'74, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:acfc8dd9:::obj74:head@111'74, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.202+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ac7304c6:::obj483:head local-lis/les=117/118 n=404 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:acfc8dd9:::obj74:head@111'74, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.202+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ac7304c6:::obj483:head local-lis/les=117/118 n=404 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:acfc8dd9:::obj74:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.202+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ac7304c6:::obj483:head local-lis/les=117/118 n=404 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:acfc8dd9:::obj74:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.202+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ac7304c6:::obj483:head local-lis/les=117/118 n=404 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:acfc8dd9:::obj74:head 2026-02-11T23:31:28.202+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ac7304c6:::obj483:head local-lis/les=117/118 n=404 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:acfc8dd9:::obj74:head v 111'74 2026-02-11T23:31:28.202+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ac7304c6:::obj483:head local-lis/les=117/118 n=404 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.202+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.202+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:acfc8dd9:::obj74:head, version: 111'74, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:acfc8dd9:::obj74:head@111'74, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:28.202+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ac7304c6:::obj483:head local-lis/les=117/118 n=404 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.202+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.202+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.202+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:acb8a831:::obj238:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.202+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:acb8a831:::obj238:head) v3 prio 15 cost 0 latency 0.000081 pg_backfill(progress 4.0 e 119/119 lb 4:acb8a831:::obj238:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:ac7304c6:::obj483:head local-lis/les=117/118 n=404 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.202+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.202+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ac7304c6:::obj483:head local-lis/les=117/118 n=404 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:acb8a831:::obj238:head) v3 2026-02-11T23:31:28.202+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ac7304c6:::obj483:head local-lis/les=117/118 n=404 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:acb8a831:::obj238:head) 2026-02-11T23:31:28.202+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.202+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:acb8a831:::obj238:head) v3 finish latency 0.000014 2026-02-11T23:31:28.202+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.202+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.202+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:acb8a831:::obj238:head local-lis/les=117/118 n=405 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.202+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.202+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.202+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.202+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x5f2dc80 con 0x507ac00 2026-02-11T23:31:28.202+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:acfc8dd9:::obj74:head)]) -- 0x4f7a300 con 0x507ac00 2026-02-11T23:31:28.202+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.202+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 851 ==== MOSDPGPush(4.0 119/117 [PushOp(4:ad830abd:::obj496:head, version: 111'496, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ad830abd:::obj496:head@111'496, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db0b60 con 0x507ac00 2026-02-11T23:31:28.202+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:ad830abd:::obj496:head, version: 111'496, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ad830abd:::obj496:head@111'496, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.202+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ad830abd:::obj496:head, version: 111'496, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ad830abd:::obj496:head@111'496, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.202+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.202+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 852 ==== pg_backfill(progress 4.0 e 119/119 lb 4:acfc8dd9:::obj74:head) ==== 1061+0+0 (unknown 0 0 0) 0x6179700 con 0x507ac00 2026-02-11T23:31:28.202+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ad830abd:::obj496:head, version: 111'496, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ad830abd:::obj496:head@111'496, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.202+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:acfc8dd9:::obj74:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.202+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:acfc8dd9:::obj74:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.202+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.202+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ad830abd:::obj496:head, version: 111'496, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ad830abd:::obj496:head@111'496, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.202+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.202+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:ad830abd:::obj496:head, version: 111'496, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ad830abd:::obj496:head@111'496, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(4.0 119/117 [PushOp(4:ad830abd:::obj496:head, version: 111'496, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ad830abd:::obj496:head@111'496, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:acb8a831:::obj238:head local-lis/les=117/118 n=405 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.202+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:acfc8dd9:::obj74:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.202+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.202+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:acb8a831:::obj238:head local-lis/les=117/118 n=405 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:ad830abd:::obj496:head, version: 111'496, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ad830abd:::obj496:head@111'496, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.202+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:acb8a831:::obj238:head local-lis/les=117/118 n=405 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:ad830abd:::obj496:head@111'496, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.202+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:acb8a831:::obj238:head local-lis/les=117/118 n=405 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:ad830abd:::obj496:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.202+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:acb8a831:::obj238:head local-lis/les=117/118 n=405 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:ad830abd:::obj496:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.202+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:acb8a831:::obj238:head local-lis/les=117/118 n=405 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:ad830abd:::obj496:head 2026-02-11T23:31:28.202+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:acb8a831:::obj238:head local-lis/les=117/118 n=405 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:ad830abd:::obj496:head v 111'496 2026-02-11T23:31:28.202+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:acb8a831:::obj238:head local-lis/les=117/118 n=405 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.202+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.202+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:ad830abd:::obj496:head, version: 111'496, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ad830abd:::obj496:head@111'496, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000064 2026-02-11T23:31:28.202+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.202+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.202+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:acfc8dd9:::obj74:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.202+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:acfc8dd9:::obj74:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 4.0 e 119/119 lb 4:acfc8dd9:::obj74:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:acb8a831:::obj238:head local-lis/les=117/118 n=405 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.202+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.202+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:acb8a831:::obj238:head local-lis/les=117/118 n=405 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:acfc8dd9:::obj74:head) v3 2026-02-11T23:31:28.202+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:acb8a831:::obj238:head local-lis/les=117/118 n=405 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:acfc8dd9:::obj74:head) 2026-02-11T23:31:28.202+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.202+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:acfc8dd9:::obj74:head) v3 finish latency 0.000012 2026-02-11T23:31:28.202+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:acfc8dd9:::obj74:head local-lis/les=117/118 n=406 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.202+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.203+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.203+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:acfc8dd9:::obj74:head local-lis/les=117/118 n=406 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.203+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.203+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.203+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.203+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7b200 con 0x507ac00 2026-02-11T23:31:28.203+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:ad830abd:::obj496:head)]) -- 0x642db00 con 0x507ac00 2026-02-11T23:31:28.203+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.203+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 853 ==== MOSDPGPush(4.0 119/117 [PushOp(4:adc1b1a4:::obj497:head, version: 111'497, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:adc1b1a4:::obj497:head@111'497, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2d00 con 0x507ac00 2026-02-11T23:31:28.203+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:adc1b1a4:::obj497:head, version: 111'497, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:adc1b1a4:::obj497:head@111'497, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 119 2026-02-11T23:31:28.203+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:adc1b1a4:::obj497:head, version: 111'497, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:adc1b1a4:::obj497:head@111'497, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.203+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.203+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:adc1b1a4:::obj497:head, version: 111'497, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:adc1b1a4:::obj497:head@111'497, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.203+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 854 ==== pg_backfill(progress 4.0 e 119/119 lb 4:ad830abd:::obj496:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2cb00 con 0x507ac00 2026-02-11T23:31:28.203+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:ad830abd:::obj496:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.203+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.203+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:ad830abd:::obj496:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.203+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:adc1b1a4:::obj497:head, version: 111'497, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:adc1b1a4:::obj497:head@111'497, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.203+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.203+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:adc1b1a4:::obj497:head, version: 111'497, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:adc1b1a4:::obj497:head@111'497, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 119/117 [PushOp(4:adc1b1a4:::obj497:head, version: 111'497, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:adc1b1a4:::obj497:head@111'497, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:acfc8dd9:::obj74:head local-lis/les=117/118 n=406 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.203+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.203+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:acfc8dd9:::obj74:head local-lis/les=117/118 n=406 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:adc1b1a4:::obj497:head, version: 111'497, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:adc1b1a4:::obj497:head@111'497, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.203+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:acfc8dd9:::obj74:head local-lis/les=117/118 n=406 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:adc1b1a4:::obj497:head@111'497, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.203+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.203+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:ad830abd:::obj496:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.203+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:acfc8dd9:::obj74:head local-lis/les=117/118 n=406 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:adc1b1a4:::obj497:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.203+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:acfc8dd9:::obj74:head local-lis/les=117/118 n=406 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:adc1b1a4:::obj497:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.203+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:acfc8dd9:::obj74:head local-lis/les=117/118 n=406 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:adc1b1a4:::obj497:head 2026-02-11T23:31:28.203+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:acfc8dd9:::obj74:head local-lis/les=117/118 n=406 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:adc1b1a4:::obj497:head v 111'497 2026-02-11T23:31:28.203+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:acfc8dd9:::obj74:head local-lis/les=117/118 n=406 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.203+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.203+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:adc1b1a4:::obj497:head, version: 111'497, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:adc1b1a4:::obj497:head@111'497, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:28.203+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:acfc8dd9:::obj74:head local-lis/les=117/118 n=406 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.203+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.203+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.203+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:ad830abd:::obj496:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.203+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:ad830abd:::obj496:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 4.0 e 119/119 lb 4:ad830abd:::obj496:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:acfc8dd9:::obj74:head local-lis/les=117/118 n=406 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.203+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.203+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:acfc8dd9:::obj74:head local-lis/les=117/118 n=406 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:ad830abd:::obj496:head) v3 2026-02-11T23:31:28.203+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:acfc8dd9:::obj74:head local-lis/les=117/118 n=406 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:ad830abd:::obj496:head) 2026-02-11T23:31:28.203+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.203+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:ad830abd:::obj496:head) v3 finish latency 0.000017 2026-02-11T23:31:28.203+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.203+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.203+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ad830abd:::obj496:head local-lis/les=117/118 n=407 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.203+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.203+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.203+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.203+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x640e180 con 0x507ac00 2026-02-11T23:31:28.203+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:adc1b1a4:::obj497:head)]) -- 0x642d800 con 0x507ac00 2026-02-11T23:31:28.203+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.203+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 855 ==== MOSDPGPush(4.0 119/117 [PushOp(4:ae2ef588:::obj234:head, version: 111'234, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ae2ef588:::obj234:head@111'234, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c1a0 con 0x507ac00 2026-02-11T23:31:28.203+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:ae2ef588:::obj234:head, version: 111'234, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ae2ef588:::obj234:head@111'234, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.203+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ae2ef588:::obj234:head, version: 111'234, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ae2ef588:::obj234:head@111'234, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.203+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.203+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 856 ==== pg_backfill(progress 4.0 e 119/119 lb 4:adc1b1a4:::obj497:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2c000 con 0x507ac00 2026-02-11T23:31:28.203+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ae2ef588:::obj234:head, version: 111'234, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ae2ef588:::obj234:head@111'234, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.203+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:adc1b1a4:::obj497:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.203+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:adc1b1a4:::obj497:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.203+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.203+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ae2ef588:::obj234:head, version: 111'234, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ae2ef588:::obj234:head@111'234, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.203+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.203+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:ae2ef588:::obj234:head, version: 111'234, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ae2ef588:::obj234:head@111'234, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 119/117 [PushOp(4:ae2ef588:::obj234:head, version: 111'234, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ae2ef588:::obj234:head@111'234, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:ad830abd:::obj496:head local-lis/les=117/118 n=407 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.203+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.203+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.203+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:adc1b1a4:::obj497:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.203+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ad830abd:::obj496:head local-lis/les=117/118 n=407 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:ae2ef588:::obj234:head, version: 111'234, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ae2ef588:::obj234:head@111'234, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.203+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ad830abd:::obj496:head local-lis/les=117/118 n=407 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:ae2ef588:::obj234:head@111'234, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.204+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ad830abd:::obj496:head local-lis/les=117/118 n=407 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:ae2ef588:::obj234:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.204+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ad830abd:::obj496:head local-lis/les=117/118 n=407 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:ae2ef588:::obj234:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.204+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ad830abd:::obj496:head local-lis/les=117/118 n=407 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:ae2ef588:::obj234:head 2026-02-11T23:31:28.204+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ad830abd:::obj496:head local-lis/les=117/118 n=407 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:ae2ef588:::obj234:head v 111'234 2026-02-11T23:31:28.204+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ad830abd:::obj496:head local-lis/les=117/118 n=407 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.204+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.204+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:ae2ef588:::obj234:head, version: 111'234, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ae2ef588:::obj234:head@111'234, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000077 2026-02-11T23:31:28.204+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.204+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.204+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:adc1b1a4:::obj497:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.204+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:adc1b1a4:::obj497:head) v3 prio 15 cost 0 latency 0.000097 pg_backfill(progress 4.0 e 119/119 lb 4:adc1b1a4:::obj497:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:ad830abd:::obj496:head local-lis/les=117/118 n=407 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.204+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.204+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ad830abd:::obj496:head local-lis/les=117/118 n=407 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:adc1b1a4:::obj497:head) v3 2026-02-11T23:31:28.204+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ad830abd:::obj496:head local-lis/les=117/118 n=407 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:adc1b1a4:::obj497:head) 2026-02-11T23:31:28.204+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.204+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:adc1b1a4:::obj497:head) v3 finish latency 0.000021 2026-02-11T23:31:28.204+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:adc1b1a4:::obj497:head local-lis/les=117/118 n=408 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.204+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.204+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.204+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:adc1b1a4:::obj497:head local-lis/les=117/118 n=408 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.204+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.204+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.204+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.204+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x642d500 con 0x507ac00 2026-02-11T23:31:28.204+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:ae2ef588:::obj234:head)]) -- 0x642de00 con 0x507ac00 2026-02-11T23:31:28.204+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.204+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 857 ==== MOSDPGPush(4.0 119/117 [PushOp(4:aec2b241:::obj347:head, version: 111'347, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aec2b241:::obj347:head@111'347, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3814d00 con 0x507ac00 2026-02-11T23:31:28.204+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:aec2b241:::obj347:head, version: 111'347, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aec2b241:::obj347:head@111'347, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 119 2026-02-11T23:31:28.204+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:aec2b241:::obj347:head, version: 111'347, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aec2b241:::obj347:head@111'347, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.204+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.204+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 858 ==== pg_backfill(progress 4.0 e 119/119 lb 4:ae2ef588:::obj234:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e100 con 0x507ac00 2026-02-11T23:31:28.204+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:ae2ef588:::obj234:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.204+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:aec2b241:::obj347:head, version: 111'347, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aec2b241:::obj347:head@111'347, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.204+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:ae2ef588:::obj234:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.204+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.204+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:aec2b241:::obj347:head, version: 111'347, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aec2b241:::obj347:head@111'347, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.204+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.204+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:aec2b241:::obj347:head, version: 111'347, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aec2b241:::obj347:head@111'347, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(4.0 119/117 [PushOp(4:aec2b241:::obj347:head, version: 111'347, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aec2b241:::obj347:head@111'347, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:adc1b1a4:::obj497:head local-lis/les=117/118 n=408 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.204+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:ae2ef588:::obj234:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.204+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.204+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:adc1b1a4:::obj497:head local-lis/les=117/118 n=408 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:aec2b241:::obj347:head, version: 111'347, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aec2b241:::obj347:head@111'347, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.204+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:adc1b1a4:::obj497:head local-lis/les=117/118 n=408 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:aec2b241:::obj347:head@111'347, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.204+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:adc1b1a4:::obj497:head local-lis/les=117/118 n=408 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:aec2b241:::obj347:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.204+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:adc1b1a4:::obj497:head local-lis/les=117/118 n=408 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:aec2b241:::obj347:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.204+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:adc1b1a4:::obj497:head local-lis/les=117/118 n=408 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:aec2b241:::obj347:head 2026-02-11T23:31:28.204+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:adc1b1a4:::obj497:head local-lis/les=117/118 n=408 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:aec2b241:::obj347:head v 111'347 2026-02-11T23:31:28.204+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:adc1b1a4:::obj497:head local-lis/les=117/118 n=408 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.204+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.204+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:aec2b241:::obj347:head, version: 111'347, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aec2b241:::obj347:head@111'347, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000076 2026-02-11T23:31:28.204+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:adc1b1a4:::obj497:head local-lis/les=117/118 n=408 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.204+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.204+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.204+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:ae2ef588:::obj234:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.204+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:ae2ef588:::obj234:head) v3 prio 15 cost 0 latency 0.000101 pg_backfill(progress 4.0 e 119/119 lb 4:ae2ef588:::obj234:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:adc1b1a4:::obj497:head local-lis/les=117/118 n=408 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.204+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.204+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:adc1b1a4:::obj497:head local-lis/les=117/118 n=408 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:ae2ef588:::obj234:head) v3 2026-02-11T23:31:28.204+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:adc1b1a4:::obj497:head local-lis/les=117/118 n=408 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:ae2ef588:::obj234:head) 2026-02-11T23:31:28.204+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.204+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:ae2ef588:::obj234:head) v3 finish latency 0.000016 2026-02-11T23:31:28.204+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.204+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.204+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ae2ef588:::obj234:head local-lis/les=117/118 n=409 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.204+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.204+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.204+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.204+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x5d55200 con 0x507ac00 2026-02-11T23:31:28.204+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:aec2b241:::obj347:head)]) -- 0x640fb00 con 0x507ac00 2026-02-11T23:31:28.205+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.205+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 859 ==== MOSDPGPush(4.0 119/117 [PushOp(4:aecaa835:::obj157:head, version: 111'157, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aecaa835:::obj157:head@111'157, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a3040 con 0x507ac00 2026-02-11T23:31:28.205+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:aecaa835:::obj157:head, version: 111'157, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aecaa835:::obj157:head@111'157, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.205+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:aecaa835:::obj157:head, version: 111'157, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aecaa835:::obj157:head@111'157, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.205+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 860 ==== pg_backfill(progress 4.0 e 119/119 lb 4:aec2b241:::obj347:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176580 con 0x507ac00 2026-02-11T23:31:28.205+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.205+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:aec2b241:::obj347:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.205+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:aec2b241:::obj347:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.205+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:aecaa835:::obj157:head, version: 111'157, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aecaa835:::obj157:head@111'157, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.205+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.205+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:aecaa835:::obj157:head, version: 111'157, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aecaa835:::obj157:head@111'157, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.205+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.205+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:aec2b241:::obj347:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.205+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:aecaa835:::obj157:head, version: 111'157, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aecaa835:::obj157:head@111'157, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(4.0 119/117 [PushOp(4:aecaa835:::obj157:head, version: 111'157, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aecaa835:::obj157:head@111'157, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:ae2ef588:::obj234:head local-lis/les=117/118 n=409 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.205+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.205+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ae2ef588:::obj234:head local-lis/les=117/118 n=409 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:aecaa835:::obj157:head, version: 111'157, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aecaa835:::obj157:head@111'157, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.205+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ae2ef588:::obj234:head local-lis/les=117/118 n=409 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:aecaa835:::obj157:head@111'157, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.205+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ae2ef588:::obj234:head local-lis/les=117/118 n=409 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:aecaa835:::obj157:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.205+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ae2ef588:::obj234:head local-lis/les=117/118 n=409 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:aecaa835:::obj157:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.205+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ae2ef588:::obj234:head local-lis/les=117/118 n=409 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:aecaa835:::obj157:head 2026-02-11T23:31:28.205+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ae2ef588:::obj234:head local-lis/les=117/118 n=409 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:aecaa835:::obj157:head v 111'157 2026-02-11T23:31:28.205+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ae2ef588:::obj234:head local-lis/les=117/118 n=409 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.205+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.205+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:aecaa835:::obj157:head, version: 111'157, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aecaa835:::obj157:head@111'157, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000071 2026-02-11T23:31:28.205+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ae2ef588:::obj234:head local-lis/les=117/118 n=409 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.205+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.205+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.205+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:aec2b241:::obj347:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.205+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:aec2b241:::obj347:head) v3 prio 15 cost 0 latency 0.000101 pg_backfill(progress 4.0 e 119/119 lb 4:aec2b241:::obj347:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:ae2ef588:::obj234:head local-lis/les=117/118 n=409 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.205+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.205+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ae2ef588:::obj234:head local-lis/les=117/118 n=409 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:aec2b241:::obj347:head) v3 2026-02-11T23:31:28.205+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ae2ef588:::obj234:head local-lis/les=117/118 n=409 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:aec2b241:::obj347:head) 2026-02-11T23:31:28.205+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.205+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:aec2b241:::obj347:head) v3 finish latency 0.000016 2026-02-11T23:31:28.205+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.205+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.205+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:aec2b241:::obj347:head local-lis/les=117/118 n=410 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.205+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.205+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.205+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.205+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c9e00 con 0x507ac00 2026-02-11T23:31:28.205+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:aecaa835:::obj157:head)]) -- 0x642cd80 con 0x507ac00 2026-02-11T23:31:28.205+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.205+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 861 ==== MOSDPGPush(4.0 119/117 [PushOp(4:af2b834d:::obj153:head, version: 111'153, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af2b834d:::obj153:head@111'153, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f0ea0 con 0x507ac00 2026-02-11T23:31:28.205+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:af2b834d:::obj153:head, version: 111'153, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af2b834d:::obj153:head@111'153, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 119 2026-02-11T23:31:28.205+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:af2b834d:::obj153:head, version: 111'153, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af2b834d:::obj153:head@111'153, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.206+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.206+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 862 ==== pg_backfill(progress 4.0 e 119/119 lb 4:aecaa835:::obj157:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e680 con 0x507ac00 2026-02-11T23:31:28.206+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:af2b834d:::obj153:head, version: 111'153, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af2b834d:::obj153:head@111'153, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.206+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:aecaa835:::obj157:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.206+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:aecaa835:::obj157:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.206+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.206+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:af2b834d:::obj153:head, version: 111'153, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af2b834d:::obj153:head@111'153, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.206+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:af2b834d:::obj153:head, version: 111'153, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af2b834d:::obj153:head@111'153, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(4.0 119/117 [PushOp(4:af2b834d:::obj153:head, version: 111'153, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af2b834d:::obj153:head@111'153, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:aec2b241:::obj347:head local-lis/les=117/118 n=410 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.206+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.206+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.206+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:aecaa835:::obj157:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.206+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:aec2b241:::obj347:head local-lis/les=117/118 n=410 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:af2b834d:::obj153:head, version: 111'153, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af2b834d:::obj153:head@111'153, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.206+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:aec2b241:::obj347:head local-lis/les=117/118 n=410 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:af2b834d:::obj153:head@111'153, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.206+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:aec2b241:::obj347:head local-lis/les=117/118 n=410 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:af2b834d:::obj153:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.206+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:aec2b241:::obj347:head local-lis/les=117/118 n=410 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:af2b834d:::obj153:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.206+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:aec2b241:::obj347:head local-lis/les=117/118 n=410 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:af2b834d:::obj153:head 2026-02-11T23:31:28.206+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:aec2b241:::obj347:head local-lis/les=117/118 n=410 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:af2b834d:::obj153:head v 111'153 2026-02-11T23:31:28.206+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:aec2b241:::obj347:head local-lis/les=117/118 n=410 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.206+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.206+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:af2b834d:::obj153:head, version: 111'153, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af2b834d:::obj153:head@111'153, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:28.206+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:aec2b241:::obj347:head local-lis/les=117/118 n=410 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.206+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.206+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.206+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:aecaa835:::obj157:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.206+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:aecaa835:::obj157:head) v3 prio 15 cost 0 latency 0.000091 pg_backfill(progress 4.0 e 119/119 lb 4:aecaa835:::obj157:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:aec2b241:::obj347:head local-lis/les=117/118 n=410 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.206+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.206+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:aec2b241:::obj347:head local-lis/les=117/118 n=410 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:aecaa835:::obj157:head) v3 2026-02-11T23:31:28.206+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:aec2b241:::obj347:head local-lis/les=117/118 n=410 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:aecaa835:::obj157:head) 2026-02-11T23:31:28.206+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.206+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:aecaa835:::obj157:head) v3 finish latency 0.000019 2026-02-11T23:31:28.206+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.206+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.206+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:aecaa835:::obj157:head local-lis/les=117/118 n=411 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.206+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.206+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.206+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.206+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66d9080 con 0x507ac00 2026-02-11T23:31:28.206+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:af2b834d:::obj153:head)]) -- 0x640f680 con 0x507ac00 2026-02-11T23:31:28.206+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.206+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.206+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.206+0000 7f415b6ff640 20 osd.0 119 reports for 0 queries 2026-02-11T23:31:28.206+0000 7f415b6ff640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] -- mgrreport(unknown.0 +0-0 packed 3294 daemon_metrics=2) -- 0x4f9f180 con 0x4aca800 2026-02-11T23:31:28.206+0000 7f415b6ff640 15 osd.0 119 collect_pg_stats 2026-02-11T23:31:28.206+0000 7f415b6ff640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] -- pg_stats(0 pgs seq 21474836499 v 0) -- 0x4aa1200 con 0x4aca800 2026-02-11T23:31:28.206+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 863 ==== MOSDPGPush(4.0 119/117 [PushOp(4:af2dbe17:::obj414:head, version: 111'414, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af2dbe17:::obj414:head@111'414, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2d00 con 0x507ac00 2026-02-11T23:31:28.206+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:af2dbe17:::obj414:head, version: 111'414, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af2dbe17:::obj414:head@111'414, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.206+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:af2dbe17:::obj414:head, version: 111'414, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af2dbe17:::obj414:head@111'414, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.206+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.206+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 864 ==== pg_backfill(progress 4.0 e 119/119 lb 4:af2b834d:::obj153:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2d080 con 0x507ac00 2026-02-11T23:31:28.206+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:af2dbe17:::obj414:head, version: 111'414, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af2dbe17:::obj414:head@111'414, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.206+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:af2b834d:::obj153:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.206+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:af2b834d:::obj153:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.206+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.206+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:af2dbe17:::obj414:head, version: 111'414, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af2dbe17:::obj414:head@111'414, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.206+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.206+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:af2dbe17:::obj414:head, version: 111'414, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af2dbe17:::obj414:head@111'414, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(4.0 119/117 [PushOp(4:af2dbe17:::obj414:head, version: 111'414, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af2dbe17:::obj414:head@111'414, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:aecaa835:::obj157:head local-lis/les=117/118 n=411 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.206+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.206+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:aecaa835:::obj157:head local-lis/les=117/118 n=411 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:af2dbe17:::obj414:head, version: 111'414, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af2dbe17:::obj414:head@111'414, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.206+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.206+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:af2b834d:::obj153:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.206+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:aecaa835:::obj157:head local-lis/les=117/118 n=411 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:af2dbe17:::obj414:head@111'414, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.206+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:aecaa835:::obj157:head local-lis/les=117/118 n=411 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:af2dbe17:::obj414:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.206+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:aecaa835:::obj157:head local-lis/les=117/118 n=411 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:af2dbe17:::obj414:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.206+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:aecaa835:::obj157:head local-lis/les=117/118 n=411 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:af2dbe17:::obj414:head 2026-02-11T23:31:28.206+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:aecaa835:::obj157:head local-lis/les=117/118 n=411 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:af2dbe17:::obj414:head v 111'414 2026-02-11T23:31:28.206+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:aecaa835:::obj157:head local-lis/les=117/118 n=411 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.206+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.206+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:af2dbe17:::obj414:head, version: 111'414, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af2dbe17:::obj414:head@111'414, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000064 2026-02-11T23:31:28.206+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:aecaa835:::obj157:head local-lis/les=117/118 n=411 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.206+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.206+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.206+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:af2b834d:::obj153:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.206+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:af2b834d:::obj153:head) v3 prio 15 cost 0 latency 0.000090 pg_backfill(progress 4.0 e 119/119 lb 4:af2b834d:::obj153:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:aecaa835:::obj157:head local-lis/les=117/118 n=411 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.206+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.206+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:aecaa835:::obj157:head local-lis/les=117/118 n=411 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:af2b834d:::obj153:head) v3 2026-02-11T23:31:28.206+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:aecaa835:::obj157:head local-lis/les=117/118 n=411 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:af2b834d:::obj153:head) 2026-02-11T23:31:28.206+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.206+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:af2b834d:::obj153:head) v3 finish latency 0.000014 2026-02-11T23:31:28.206+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.206+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.206+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af2b834d:::obj153:head local-lis/les=117/118 n=412 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.206+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.207+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.207+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.207+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66d9500 con 0x507ac00 2026-02-11T23:31:28.207+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:af2dbe17:::obj414:head)]) -- 0x66d9200 con 0x507ac00 2026-02-11T23:31:28.207+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.207+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 865 ==== MOSDPGPush(4.0 119/117 [PushOp(4:af44a1d6:::obj35:head, version: 111'35, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af44a1d6:::obj35:head@111'35, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5db0b60 con 0x507ac00 2026-02-11T23:31:28.207+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:af44a1d6:::obj35:head, version: 111'35, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af44a1d6:::obj35:head@111'35, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.207+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:af44a1d6:::obj35:head, version: 111'35, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af44a1d6:::obj35:head@111'35, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.207+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.207+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 866 ==== pg_backfill(progress 4.0 e 119/119 lb 4:af2dbe17:::obj414:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2f700 con 0x507ac00 2026-02-11T23:31:28.207+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:af2dbe17:::obj414:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.207+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:af44a1d6:::obj35:head, version: 111'35, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af44a1d6:::obj35:head@111'35, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.207+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:af2dbe17:::obj414:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.207+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.207+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:af44a1d6:::obj35:head, version: 111'35, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af44a1d6:::obj35:head@111'35, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.207+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:af44a1d6:::obj35:head, version: 111'35, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af44a1d6:::obj35:head@111'35, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(4.0 119/117 [PushOp(4:af44a1d6:::obj35:head, version: 111'35, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af44a1d6:::obj35:head@111'35, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:af2b834d:::obj153:head local-lis/les=117/118 n=412 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.207+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.207+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.207+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:af2dbe17:::obj414:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.207+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af2b834d:::obj153:head local-lis/les=117/118 n=412 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:af44a1d6:::obj35:head, version: 111'35, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af44a1d6:::obj35:head@111'35, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.207+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af2b834d:::obj153:head local-lis/les=117/118 n=412 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:af44a1d6:::obj35:head@111'35, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.207+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af2b834d:::obj153:head local-lis/les=117/118 n=412 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:af44a1d6:::obj35:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.207+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af2b834d:::obj153:head local-lis/les=117/118 n=412 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:af44a1d6:::obj35:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.207+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af2b834d:::obj153:head local-lis/les=117/118 n=412 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:af44a1d6:::obj35:head 2026-02-11T23:31:28.207+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af2b834d:::obj153:head local-lis/les=117/118 n=412 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:af44a1d6:::obj35:head v 111'35 2026-02-11T23:31:28.207+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af2b834d:::obj153:head local-lis/les=117/118 n=412 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.207+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.207+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:af44a1d6:::obj35:head, version: 111'35, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af44a1d6:::obj35:head@111'35, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000070 2026-02-11T23:31:28.207+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af2b834d:::obj153:head local-lis/les=117/118 n=412 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.207+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.207+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.207+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:af2dbe17:::obj414:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.207+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:af2dbe17:::obj414:head) v3 prio 15 cost 0 latency 0.000098 pg_backfill(progress 4.0 e 119/119 lb 4:af2dbe17:::obj414:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:af2b834d:::obj153:head local-lis/les=117/118 n=412 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.207+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.207+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af2b834d:::obj153:head local-lis/les=117/118 n=412 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:af2dbe17:::obj414:head) v3 2026-02-11T23:31:28.207+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af2b834d:::obj153:head local-lis/les=117/118 n=412 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:af2dbe17:::obj414:head) 2026-02-11T23:31:28.207+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.207+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:af2dbe17:::obj414:head) v3 finish latency 0.000020 2026-02-11T23:31:28.207+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.207+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.207+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af2dbe17:::obj414:head local-lis/les=117/118 n=413 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.207+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.207+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.207+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.207+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66d8480 con 0x507ac00 2026-02-11T23:31:28.207+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:af44a1d6:::obj35:head)]) -- 0x66fa300 con 0x507ac00 2026-02-11T23:31:28.207+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.207+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.207+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.207+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 867 ==== MOSDPGPush(4.0 119/117 [PushOp(4:af5f95cb:::obj12:head, version: 111'12, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af5f95cb:::obj12:head@111'12, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x56cad00 con 0x507ac00 2026-02-11T23:31:28.207+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:af5f95cb:::obj12:head, version: 111'12, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af5f95cb:::obj12:head@111'12, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 119 2026-02-11T23:31:28.207+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:af5f95cb:::obj12:head, version: 111'12, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af5f95cb:::obj12:head@111'12, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.207+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.207+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:af5f95cb:::obj12:head, version: 111'12, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af5f95cb:::obj12:head@111'12, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.207+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 868 ==== pg_backfill(progress 4.0 e 119/119 lb 4:af44a1d6:::obj35:head) ==== 1061+0+0 (unknown 0 0 0) 0x5f11600 con 0x507ac00 2026-02-11T23:31:28.207+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:af44a1d6:::obj35:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.207+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.207+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:af44a1d6:::obj35:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.207+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:af5f95cb:::obj12:head, version: 111'12, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af5f95cb:::obj12:head@111'12, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.207+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.207+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:af5f95cb:::obj12:head, version: 111'12, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af5f95cb:::obj12:head@111'12, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 119/117 [PushOp(4:af5f95cb:::obj12:head, version: 111'12, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af5f95cb:::obj12:head@111'12, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:af2dbe17:::obj414:head local-lis/les=117/118 n=413 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.207+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.207+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.207+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af2dbe17:::obj414:head local-lis/les=117/118 n=413 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:af5f95cb:::obj12:head, version: 111'12, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af5f95cb:::obj12:head@111'12, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.207+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:af44a1d6:::obj35:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.207+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af2dbe17:::obj414:head local-lis/les=117/118 n=413 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:af5f95cb:::obj12:head@111'12, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.207+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af2dbe17:::obj414:head local-lis/les=117/118 n=413 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:af5f95cb:::obj12:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.207+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af2dbe17:::obj414:head local-lis/les=117/118 n=413 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:af5f95cb:::obj12:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.207+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af2dbe17:::obj414:head local-lis/les=117/118 n=413 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:af5f95cb:::obj12:head 2026-02-11T23:31:28.207+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af2dbe17:::obj414:head local-lis/les=117/118 n=413 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:af5f95cb:::obj12:head v 111'12 2026-02-11T23:31:28.207+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af2dbe17:::obj414:head local-lis/les=117/118 n=413 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.207+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.208+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:af5f95cb:::obj12:head, version: 111'12, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af5f95cb:::obj12:head@111'12, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:28.208+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af2dbe17:::obj414:head local-lis/les=117/118 n=413 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.208+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.208+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.208+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:af44a1d6:::obj35:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.208+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:af44a1d6:::obj35:head) v3 prio 15 cost 0 latency 0.000082 pg_backfill(progress 4.0 e 119/119 lb 4:af44a1d6:::obj35:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:af2dbe17:::obj414:head local-lis/les=117/118 n=413 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.208+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.208+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af2dbe17:::obj414:head local-lis/les=117/118 n=413 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:af44a1d6:::obj35:head) v3 2026-02-11T23:31:28.208+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af2dbe17:::obj414:head local-lis/les=117/118 n=413 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:af44a1d6:::obj35:head) 2026-02-11T23:31:28.208+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.208+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:af44a1d6:::obj35:head) v3 finish latency 0.000011 2026-02-11T23:31:28.208+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.208+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.208+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af44a1d6:::obj35:head local-lis/les=117/118 n=414 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.208+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.208+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.208+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.208+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66fa000 con 0x507ac00 2026-02-11T23:31:28.208+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:af5f95cb:::obj12:head)]) -- 0x66fa480 con 0x507ac00 2026-02-11T23:31:28.208+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.208+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 869 ==== MOSDPGPush(4.0 119/117 [PushOp(4:af93a04b:::obj437:head, version: 111'437, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af93a04b:::obj437:head@111'437, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56171e0 con 0x507ac00 2026-02-11T23:31:28.208+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:af93a04b:::obj437:head, version: 111'437, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af93a04b:::obj437:head@111'437, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.208+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:af93a04b:::obj437:head, version: 111'437, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af93a04b:::obj437:head@111'437, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.208+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.208+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:af93a04b:::obj437:head, version: 111'437, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af93a04b:::obj437:head@111'437, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.208+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 870 ==== pg_backfill(progress 4.0 e 119/119 lb 4:af5f95cb:::obj12:head) ==== 1061+0+0 (unknown 0 0 0) 0x5f11080 con 0x507ac00 2026-02-11T23:31:28.208+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:af5f95cb:::obj12:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 119 2026-02-11T23:31:28.208+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.208+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:af5f95cb:::obj12:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.208+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:af93a04b:::obj437:head, version: 111'437, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af93a04b:::obj437:head@111'437, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.208+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.208+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:af93a04b:::obj437:head, version: 111'437, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af93a04b:::obj437:head@111'437, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(4.0 119/117 [PushOp(4:af93a04b:::obj437:head, version: 111'437, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af93a04b:::obj437:head@111'437, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:af44a1d6:::obj35:head local-lis/les=117/118 n=414 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.208+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.208+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.208+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af44a1d6:::obj35:head local-lis/les=117/118 n=414 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:af93a04b:::obj437:head, version: 111'437, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af93a04b:::obj437:head@111'437, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.208+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:af5f95cb:::obj12:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.208+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af44a1d6:::obj35:head local-lis/les=117/118 n=414 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:af93a04b:::obj437:head@111'437, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.208+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af44a1d6:::obj35:head local-lis/les=117/118 n=414 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:af93a04b:::obj437:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.208+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af44a1d6:::obj35:head local-lis/les=117/118 n=414 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:af93a04b:::obj437:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.208+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af44a1d6:::obj35:head local-lis/les=117/118 n=414 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:af93a04b:::obj437:head 2026-02-11T23:31:28.208+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af44a1d6:::obj35:head local-lis/les=117/118 n=414 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:af93a04b:::obj437:head v 111'437 2026-02-11T23:31:28.208+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af44a1d6:::obj35:head local-lis/les=117/118 n=414 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.208+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.208+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:af93a04b:::obj437:head, version: 111'437, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af93a04b:::obj437:head@111'437, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:28.208+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af44a1d6:::obj35:head local-lis/les=117/118 n=414 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.208+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.208+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.208+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:af5f95cb:::obj12:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.208+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:af5f95cb:::obj12:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 4.0 e 119/119 lb 4:af5f95cb:::obj12:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:af44a1d6:::obj35:head local-lis/les=117/118 n=414 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.208+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.208+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af44a1d6:::obj35:head local-lis/les=117/118 n=414 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:af5f95cb:::obj12:head) v3 2026-02-11T23:31:28.208+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af44a1d6:::obj35:head local-lis/les=117/118 n=414 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:af5f95cb:::obj12:head) 2026-02-11T23:31:28.208+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.208+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:af5f95cb:::obj12:head) v3 finish latency 0.000011 2026-02-11T23:31:28.208+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.208+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.208+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af5f95cb:::obj12:head local-lis/les=117/118 n=415 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.208+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.208+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.208+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.208+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66fb800 con 0x507ac00 2026-02-11T23:31:28.208+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:af93a04b:::obj437:head)]) -- 0x66fbc80 con 0x507ac00 2026-02-11T23:31:28.208+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.209+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 871 ==== MOSDPGPush(4.0 119/117 [PushOp(4:af9ff462:::obj165:head, version: 111'165, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af9ff462:::obj165:head@111'165, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3815520 con 0x507ac00 2026-02-11T23:31:28.209+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:af9ff462:::obj165:head, version: 111'165, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af9ff462:::obj165:head@111'165, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 119 2026-02-11T23:31:28.209+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:af9ff462:::obj165:head, version: 111'165, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af9ff462:::obj165:head@111'165, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.209+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.209+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:af9ff462:::obj165:head, version: 111'165, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af9ff462:::obj165:head@111'165, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.209+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 872 ==== pg_backfill(progress 4.0 e 119/119 lb 4:af93a04b:::obj437:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f13180 con 0x507ac00 2026-02-11T23:31:28.209+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:af93a04b:::obj437:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.209+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.209+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:af93a04b:::obj437:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.209+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:af9ff462:::obj165:head, version: 111'165, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af9ff462:::obj165:head@111'165, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.209+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.209+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:af9ff462:::obj165:head, version: 111'165, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af9ff462:::obj165:head@111'165, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 119/117 [PushOp(4:af9ff462:::obj165:head, version: 111'165, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af9ff462:::obj165:head@111'165, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:af5f95cb:::obj12:head local-lis/les=117/118 n=415 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.209+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.209+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.209+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:af93a04b:::obj437:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.209+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af5f95cb:::obj12:head local-lis/les=117/118 n=415 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:af9ff462:::obj165:head, version: 111'165, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af9ff462:::obj165:head@111'165, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.209+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af5f95cb:::obj12:head local-lis/les=117/118 n=415 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:af9ff462:::obj165:head@111'165, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.209+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af5f95cb:::obj12:head local-lis/les=117/118 n=415 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:af9ff462:::obj165:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.209+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af5f95cb:::obj12:head local-lis/les=117/118 n=415 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:af9ff462:::obj165:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.209+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af5f95cb:::obj12:head local-lis/les=117/118 n=415 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:af9ff462:::obj165:head 2026-02-11T23:31:28.209+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af5f95cb:::obj12:head local-lis/les=117/118 n=415 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:af9ff462:::obj165:head v 111'165 2026-02-11T23:31:28.209+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af5f95cb:::obj12:head local-lis/les=117/118 n=415 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.209+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.209+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:af9ff462:::obj165:head, version: 111'165, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af9ff462:::obj165:head@111'165, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:28.209+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.209+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.209+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:af93a04b:::obj437:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.209+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:af93a04b:::obj437:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 4.0 e 119/119 lb 4:af93a04b:::obj437:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:af5f95cb:::obj12:head local-lis/les=117/118 n=415 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.209+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.209+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af5f95cb:::obj12:head local-lis/les=117/118 n=415 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:af93a04b:::obj437:head) v3 2026-02-11T23:31:28.209+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af5f95cb:::obj12:head local-lis/les=117/118 n=415 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:af93a04b:::obj437:head) 2026-02-11T23:31:28.209+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.209+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:af93a04b:::obj437:head) v3 finish latency 0.000010 2026-02-11T23:31:28.209+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af93a04b:::obj437:head local-lis/les=117/118 n=416 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.209+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.209+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.209+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af93a04b:::obj437:head local-lis/les=117/118 n=416 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.209+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.209+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.209+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.209+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66fbb00 con 0x507ac00 2026-02-11T23:31:28.209+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:af9ff462:::obj165:head)]) -- 0x670a180 con 0x507ac00 2026-02-11T23:31:28.209+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.209+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 873 ==== MOSDPGPush(4.0 119/117 [PushOp(4:afacbcad:::obj315:head, version: 111'315, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afacbcad:::obj315:head@111'315, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x554e680 con 0x507ac00 2026-02-11T23:31:28.209+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:afacbcad:::obj315:head, version: 111'315, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afacbcad:::obj315:head@111'315, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.209+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:afacbcad:::obj315:head, version: 111'315, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afacbcad:::obj315:head@111'315, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.209+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.209+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:afacbcad:::obj315:head, version: 111'315, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afacbcad:::obj315:head@111'315, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.209+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 874 ==== pg_backfill(progress 4.0 e 119/119 lb 4:af9ff462:::obj165:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f10b00 con 0x507ac00 2026-02-11T23:31:28.209+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:af9ff462:::obj165:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.209+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.209+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:af9ff462:::obj165:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.209+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:afacbcad:::obj315:head, version: 111'315, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afacbcad:::obj315:head@111'315, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.209+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.209+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:afacbcad:::obj315:head, version: 111'315, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afacbcad:::obj315:head@111'315, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 119/117 [PushOp(4:afacbcad:::obj315:head, version: 111'315, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afacbcad:::obj315:head@111'315, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:af93a04b:::obj437:head local-lis/les=117/118 n=416 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.209+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.209+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.209+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:af9ff462:::obj165:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.209+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af93a04b:::obj437:head local-lis/les=117/118 n=416 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:afacbcad:::obj315:head, version: 111'315, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afacbcad:::obj315:head@111'315, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.209+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af93a04b:::obj437:head local-lis/les=117/118 n=416 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:afacbcad:::obj315:head@111'315, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.209+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af93a04b:::obj437:head local-lis/les=117/118 n=416 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:afacbcad:::obj315:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.209+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af93a04b:::obj437:head local-lis/les=117/118 n=416 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:afacbcad:::obj315:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.209+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af93a04b:::obj437:head local-lis/les=117/118 n=416 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:afacbcad:::obj315:head 2026-02-11T23:31:28.209+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af93a04b:::obj437:head local-lis/les=117/118 n=416 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:afacbcad:::obj315:head v 111'315 2026-02-11T23:31:28.209+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af93a04b:::obj437:head local-lis/les=117/118 n=416 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.209+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.209+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:afacbcad:::obj315:head, version: 111'315, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afacbcad:::obj315:head@111'315, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:28.209+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af93a04b:::obj437:head local-lis/les=117/118 n=416 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.209+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.209+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.209+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:af9ff462:::obj165:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.209+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:af9ff462:::obj165:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 4.0 e 119/119 lb 4:af9ff462:::obj165:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:af93a04b:::obj437:head local-lis/les=117/118 n=416 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.209+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.209+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af93a04b:::obj437:head local-lis/les=117/118 n=416 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:af9ff462:::obj165:head) v3 2026-02-11T23:31:28.209+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af93a04b:::obj437:head local-lis/les=117/118 n=416 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:af9ff462:::obj165:head) 2026-02-11T23:31:28.209+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.209+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:af9ff462:::obj165:head) v3 finish latency 0.000010 2026-02-11T23:31:28.209+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.209+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.209+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af9ff462:::obj165:head local-lis/les=117/118 n=417 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.209+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.209+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.209+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.209+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66faf00 con 0x507ac00 2026-02-11T23:31:28.209+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:afacbcad:::obj315:head)]) -- 0x66fb380 con 0x507ac00 2026-02-11T23:31:28.209+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.210+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 875 ==== MOSDPGPush(4.0 119/117 [PushOp(4:afbdc8ea:::obj55:head, version: 111'55, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afbdc8ea:::obj55:head@111'55, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x56a2b60 con 0x507ac00 2026-02-11T23:31:28.210+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:afbdc8ea:::obj55:head, version: 111'55, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afbdc8ea:::obj55:head@111'55, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 119 2026-02-11T23:31:28.210+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:afbdc8ea:::obj55:head, version: 111'55, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afbdc8ea:::obj55:head@111'55, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.210+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.210+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:afbdc8ea:::obj55:head, version: 111'55, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afbdc8ea:::obj55:head@111'55, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.210+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 876 ==== pg_backfill(progress 4.0 e 119/119 lb 4:afacbcad:::obj315:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f11b80 con 0x507ac00 2026-02-11T23:31:28.210+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:afacbcad:::obj315:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.210+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.210+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:afacbcad:::obj315:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.210+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:afbdc8ea:::obj55:head, version: 111'55, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afbdc8ea:::obj55:head@111'55, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.210+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.210+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:afbdc8ea:::obj55:head, version: 111'55, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afbdc8ea:::obj55:head@111'55, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(4.0 119/117 [PushOp(4:afbdc8ea:::obj55:head, version: 111'55, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afbdc8ea:::obj55:head@111'55, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:af9ff462:::obj165:head local-lis/les=117/118 n=417 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.210+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.210+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.210+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:afacbcad:::obj315:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.210+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af9ff462:::obj165:head local-lis/les=117/118 n=417 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:afbdc8ea:::obj55:head, version: 111'55, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afbdc8ea:::obj55:head@111'55, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.210+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af9ff462:::obj165:head local-lis/les=117/118 n=417 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:afbdc8ea:::obj55:head@111'55, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.210+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af9ff462:::obj165:head local-lis/les=117/118 n=417 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:afbdc8ea:::obj55:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.210+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af9ff462:::obj165:head local-lis/les=117/118 n=417 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:afbdc8ea:::obj55:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.210+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af9ff462:::obj165:head local-lis/les=117/118 n=417 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:afbdc8ea:::obj55:head 2026-02-11T23:31:28.210+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af9ff462:::obj165:head local-lis/les=117/118 n=417 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:afbdc8ea:::obj55:head v 111'55 2026-02-11T23:31:28.210+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af9ff462:::obj165:head local-lis/les=117/118 n=417 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.210+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.210+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:afbdc8ea:::obj55:head, version: 111'55, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afbdc8ea:::obj55:head@111'55, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:28.210+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af9ff462:::obj165:head local-lis/les=117/118 n=417 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.210+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.210+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.210+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:afacbcad:::obj315:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.210+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:afacbcad:::obj315:head) v3 prio 15 cost 0 latency 0.000085 pg_backfill(progress 4.0 e 119/119 lb 4:afacbcad:::obj315:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:af9ff462:::obj165:head local-lis/les=117/118 n=417 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.210+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.210+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af9ff462:::obj165:head local-lis/les=117/118 n=417 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:afacbcad:::obj315:head) v3 2026-02-11T23:31:28.210+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:af9ff462:::obj165:head local-lis/les=117/118 n=417 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:afacbcad:::obj315:head) 2026-02-11T23:31:28.210+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.210+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:afacbcad:::obj315:head) v3 finish latency 0.000012 2026-02-11T23:31:28.210+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.210+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.210+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:afacbcad:::obj315:head local-lis/les=117/118 n=418 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.210+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.210+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.210+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.210+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66fac00 con 0x507ac00 2026-02-11T23:31:28.210+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:afbdc8ea:::obj55:head)]) -- 0x66fb200 con 0x507ac00 2026-02-11T23:31:28.210+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.210+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 877 ==== MOSDPGPush(4.0 119/117 [PushOp(4:afd348d8:::obj184:head, version: 111'184, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afd348d8:::obj184:head@111'184, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c000 con 0x507ac00 2026-02-11T23:31:28.210+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:afd348d8:::obj184:head, version: 111'184, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afd348d8:::obj184:head@111'184, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 119 2026-02-11T23:31:28.210+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:afd348d8:::obj184:head, version: 111'184, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afd348d8:::obj184:head@111'184, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.210+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.210+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 878 ==== pg_backfill(progress 4.0 e 119/119 lb 4:afbdc8ea:::obj55:head) ==== 1061+0+0 (unknown 0 0 0) 0x5f12100 con 0x507ac00 2026-02-11T23:31:28.210+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:afd348d8:::obj184:head, version: 111'184, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afd348d8:::obj184:head@111'184, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.210+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:afbdc8ea:::obj55:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.210+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.210+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:afbdc8ea:::obj55:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.210+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:afd348d8:::obj184:head, version: 111'184, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afd348d8:::obj184:head@111'184, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.210+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.210+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:afd348d8:::obj184:head, version: 111'184, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afd348d8:::obj184:head@111'184, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(4.0 119/117 [PushOp(4:afd348d8:::obj184:head, version: 111'184, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afd348d8:::obj184:head@111'184, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:afacbcad:::obj315:head local-lis/les=117/118 n=418 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.210+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.210+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:afacbcad:::obj315:head local-lis/les=117/118 n=418 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:afd348d8:::obj184:head, version: 111'184, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afd348d8:::obj184:head@111'184, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.210+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.210+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:afacbcad:::obj315:head local-lis/les=117/118 n=418 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:afd348d8:::obj184:head@111'184, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.210+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:afbdc8ea:::obj55:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.210+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:afacbcad:::obj315:head local-lis/les=117/118 n=418 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:afd348d8:::obj184:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.210+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:afacbcad:::obj315:head local-lis/les=117/118 n=418 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:afd348d8:::obj184:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.210+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:afacbcad:::obj315:head local-lis/les=117/118 n=418 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:afd348d8:::obj184:head 2026-02-11T23:31:28.210+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:afacbcad:::obj315:head local-lis/les=117/118 n=418 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:afd348d8:::obj184:head v 111'184 2026-02-11T23:31:28.210+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:afacbcad:::obj315:head local-lis/les=117/118 n=418 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.210+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.211+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:afd348d8:::obj184:head, version: 111'184, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afd348d8:::obj184:head@111'184, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000066 2026-02-11T23:31:28.211+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.211+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.211+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:afbdc8ea:::obj55:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.211+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:afbdc8ea:::obj55:head) v3 prio 15 cost 0 latency 0.000084 pg_backfill(progress 4.0 e 119/119 lb 4:afbdc8ea:::obj55:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:afacbcad:::obj315:head local-lis/les=117/118 n=418 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.211+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.211+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:afacbcad:::obj315:head local-lis/les=117/118 n=418 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:afbdc8ea:::obj55:head) v3 2026-02-11T23:31:28.211+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:afacbcad:::obj315:head local-lis/les=117/118 n=418 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:afbdc8ea:::obj55:head) 2026-02-11T23:31:28.211+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.211+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:afbdc8ea:::obj55:head) v3 finish latency 0.000015 2026-02-11T23:31:28.211+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:afbdc8ea:::obj55:head local-lis/les=117/118 n=419 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.211+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.211+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.211+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:afbdc8ea:::obj55:head local-lis/les=117/118 n=419 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.211+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.211+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.211+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.211+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66fad80 con 0x507ac00 2026-02-11T23:31:28.211+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:afd348d8:::obj184:head)]) -- 0x670a480 con 0x507ac00 2026-02-11T23:31:28.211+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.211+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 879 ==== MOSDPGPush(4.0 119/117 [PushOp(4:b0395c4b:::obj158:head, version: 111'158, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b0395c4b:::obj158:head@111'158, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56ca9c0 con 0x507ac00 2026-02-11T23:31:28.211+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:b0395c4b:::obj158:head, version: 111'158, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b0395c4b:::obj158:head@111'158, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.211+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b0395c4b:::obj158:head, version: 111'158, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b0395c4b:::obj158:head@111'158, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.211+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.211+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 880 ==== pg_backfill(progress 4.0 e 119/119 lb 4:afd348d8:::obj184:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f12680 con 0x507ac00 2026-02-11T23:31:28.211+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:afd348d8:::obj184:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 119 2026-02-11T23:31:28.211+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b0395c4b:::obj158:head, version: 111'158, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b0395c4b:::obj158:head@111'158, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.211+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:afd348d8:::obj184:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.211+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.211+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b0395c4b:::obj158:head, version: 111'158, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b0395c4b:::obj158:head@111'158, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.211+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.211+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:afd348d8:::obj184:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.211+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b0395c4b:::obj158:head, version: 111'158, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b0395c4b:::obj158:head@111'158, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(4.0 119/117 [PushOp(4:b0395c4b:::obj158:head, version: 111'158, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b0395c4b:::obj158:head@111'158, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:afbdc8ea:::obj55:head local-lis/les=117/118 n=419 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.211+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.211+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:afbdc8ea:::obj55:head local-lis/les=117/118 n=419 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:b0395c4b:::obj158:head, version: 111'158, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b0395c4b:::obj158:head@111'158, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.211+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:afbdc8ea:::obj55:head local-lis/les=117/118 n=419 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b0395c4b:::obj158:head@111'158, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.211+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:afbdc8ea:::obj55:head local-lis/les=117/118 n=419 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:b0395c4b:::obj158:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.211+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:afbdc8ea:::obj55:head local-lis/les=117/118 n=419 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:b0395c4b:::obj158:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.211+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:afbdc8ea:::obj55:head local-lis/les=117/118 n=419 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:b0395c4b:::obj158:head 2026-02-11T23:31:28.211+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:afbdc8ea:::obj55:head local-lis/les=117/118 n=419 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:b0395c4b:::obj158:head v 111'158 2026-02-11T23:31:28.211+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:afbdc8ea:::obj55:head local-lis/les=117/118 n=419 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.211+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.211+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b0395c4b:::obj158:head, version: 111'158, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b0395c4b:::obj158:head@111'158, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000068 2026-02-11T23:31:28.211+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:afbdc8ea:::obj55:head local-lis/les=117/118 n=419 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.211+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.211+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.211+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:afd348d8:::obj184:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.211+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:afd348d8:::obj184:head) v3 prio 15 cost 0 latency 0.000095 pg_backfill(progress 4.0 e 119/119 lb 4:afd348d8:::obj184:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:afbdc8ea:::obj55:head local-lis/les=117/118 n=419 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.211+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.211+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:afbdc8ea:::obj55:head local-lis/les=117/118 n=419 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:afd348d8:::obj184:head) v3 2026-02-11T23:31:28.211+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:afbdc8ea:::obj55:head local-lis/les=117/118 n=419 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:afd348d8:::obj184:head) 2026-02-11T23:31:28.211+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.211+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:afd348d8:::obj184:head) v3 finish latency 0.000022 2026-02-11T23:31:28.211+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.211+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.211+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:afd348d8:::obj184:head local-lis/les=117/118 n=420 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.211+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.211+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.211+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.211+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x670a480 con 0x507ac00 2026-02-11T23:31:28.211+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:b0395c4b:::obj158:head)]) -- 0x66fa900 con 0x507ac00 2026-02-11T23:31:28.211+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.212+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 881 ==== MOSDPGPush(4.0 119/117 [PushOp(4:b133d16d:::obj222:head, version: 111'222, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b133d16d:::obj222:head@111'222, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559d520 con 0x507ac00 2026-02-11T23:31:28.212+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:b133d16d:::obj222:head, version: 111'222, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b133d16d:::obj222:head@111'222, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 119 2026-02-11T23:31:28.212+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b133d16d:::obj222:head, version: 111'222, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b133d16d:::obj222:head@111'222, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.212+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.212+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 882 ==== pg_backfill(progress 4.0 e 119/119 lb 4:b0395c4b:::obj158:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f12c00 con 0x507ac00 2026-02-11T23:31:28.212+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:b0395c4b:::obj158:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.212+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b133d16d:::obj222:head, version: 111'222, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b133d16d:::obj222:head@111'222, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.212+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b0395c4b:::obj158:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.212+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.212+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b133d16d:::obj222:head, version: 111'222, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b133d16d:::obj222:head@111'222, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.212+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b133d16d:::obj222:head, version: 111'222, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b133d16d:::obj222:head@111'222, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(4.0 119/117 [PushOp(4:b133d16d:::obj222:head, version: 111'222, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b133d16d:::obj222:head@111'222, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:afd348d8:::obj184:head local-lis/les=117/118 n=420 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.212+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.212+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.212+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b0395c4b:::obj158:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.212+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:afd348d8:::obj184:head local-lis/les=117/118 n=420 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:b133d16d:::obj222:head, version: 111'222, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b133d16d:::obj222:head@111'222, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.212+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:afd348d8:::obj184:head local-lis/les=117/118 n=420 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b133d16d:::obj222:head@111'222, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.212+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:afd348d8:::obj184:head local-lis/les=117/118 n=420 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:b133d16d:::obj222:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.212+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:afd348d8:::obj184:head local-lis/les=117/118 n=420 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:b133d16d:::obj222:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.212+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:afd348d8:::obj184:head local-lis/les=117/118 n=420 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:b133d16d:::obj222:head 2026-02-11T23:31:28.212+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:afd348d8:::obj184:head local-lis/les=117/118 n=420 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:b133d16d:::obj222:head v 111'222 2026-02-11T23:31:28.212+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:afd348d8:::obj184:head local-lis/les=117/118 n=420 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.212+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.212+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b133d16d:::obj222:head, version: 111'222, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b133d16d:::obj222:head@111'222, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000081 2026-02-11T23:31:28.212+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.212+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.212+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b0395c4b:::obj158:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.212+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b0395c4b:::obj158:head) v3 prio 15 cost 0 latency 0.000100 pg_backfill(progress 4.0 e 119/119 lb 4:b0395c4b:::obj158:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:afd348d8:::obj184:head local-lis/les=117/118 n=420 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.212+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.212+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:afd348d8:::obj184:head local-lis/les=117/118 n=420 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:b0395c4b:::obj158:head) v3 2026-02-11T23:31:28.212+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:afd348d8:::obj184:head local-lis/les=117/118 n=420 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:b0395c4b:::obj158:head) 2026-02-11T23:31:28.212+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.212+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b0395c4b:::obj158:head) v3 finish latency 0.000019 2026-02-11T23:31:28.212+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b0395c4b:::obj158:head local-lis/les=117/118 n=421 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.212+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.212+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.212+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b0395c4b:::obj158:head local-lis/les=117/118 n=421 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.212+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.212+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.212+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.212+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66fad80 con 0x507ac00 2026-02-11T23:31:28.212+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:b133d16d:::obj222:head)]) -- 0x670a780 con 0x507ac00 2026-02-11T23:31:28.212+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.212+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 883 ==== MOSDPGPush(4.0 119/117 [PushOp(4:b164ceff:::obj421:head, version: 111'421, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b164ceff:::obj421:head@111'421, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f1860 con 0x507ac00 2026-02-11T23:31:28.212+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:b164ceff:::obj421:head, version: 111'421, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b164ceff:::obj421:head@111'421, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.212+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b164ceff:::obj421:head, version: 111'421, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b164ceff:::obj421:head@111'421, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.212+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.212+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b164ceff:::obj421:head, version: 111'421, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b164ceff:::obj421:head@111'421, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.212+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 884 ==== pg_backfill(progress 4.0 e 119/119 lb 4:b133d16d:::obj222:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f13700 con 0x507ac00 2026-02-11T23:31:28.212+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.212+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:b133d16d:::obj222:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 119 2026-02-11T23:31:28.212+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b164ceff:::obj421:head, version: 111'421, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b164ceff:::obj421:head@111'421, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.212+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b133d16d:::obj222:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.212+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.212+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b133d16d:::obj222:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.212+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b164ceff:::obj421:head, version: 111'421, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b164ceff:::obj421:head@111'421, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 119/117 [PushOp(4:b164ceff:::obj421:head, version: 111'421, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b164ceff:::obj421:head@111'421, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b0395c4b:::obj158:head local-lis/les=117/118 n=421 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.212+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.212+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b0395c4b:::obj158:head local-lis/les=117/118 n=421 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:b164ceff:::obj421:head, version: 111'421, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b164ceff:::obj421:head@111'421, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.212+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b0395c4b:::obj158:head local-lis/les=117/118 n=421 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b164ceff:::obj421:head@111'421, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.212+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b0395c4b:::obj158:head local-lis/les=117/118 n=421 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:b164ceff:::obj421:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.212+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b0395c4b:::obj158:head local-lis/les=117/118 n=421 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:b164ceff:::obj421:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.212+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b0395c4b:::obj158:head local-lis/les=117/118 n=421 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:b164ceff:::obj421:head 2026-02-11T23:31:28.212+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b0395c4b:::obj158:head local-lis/les=117/118 n=421 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:b164ceff:::obj421:head v 111'421 2026-02-11T23:31:28.212+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b0395c4b:::obj158:head local-lis/les=117/118 n=421 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.212+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.213+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b164ceff:::obj421:head, version: 111'421, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b164ceff:::obj421:head@111'421, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000079 2026-02-11T23:31:28.213+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.213+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.213+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b133d16d:::obj222:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.213+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b133d16d:::obj222:head) v3 prio 15 cost 0 latency 0.000097 pg_backfill(progress 4.0 e 119/119 lb 4:b133d16d:::obj222:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b0395c4b:::obj158:head local-lis/les=117/118 n=421 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.213+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.213+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b0395c4b:::obj158:head local-lis/les=117/118 n=421 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:b133d16d:::obj222:head) v3 2026-02-11T23:31:28.213+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b0395c4b:::obj158:head local-lis/les=117/118 n=421 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:b133d16d:::obj222:head) 2026-02-11T23:31:28.213+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.213+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b133d16d:::obj222:head) v3 finish latency 0.000014 2026-02-11T23:31:28.213+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b133d16d:::obj222:head local-lis/les=117/118 n=422 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.213+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.213+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.213+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b133d16d:::obj222:head local-lis/les=117/118 n=422 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.213+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.213+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.213+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.213+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66fb200 con 0x507ac00 2026-02-11T23:31:28.213+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:b164ceff:::obj421:head)]) -- 0x670aa80 con 0x507ac00 2026-02-11T23:31:28.213+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.213+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 885 ==== MOSDPGPush(4.0 119/117 [PushOp(4:b1721c5a:::obj526:head, version: 111'526, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1721c5a:::obj526:head@111'526, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2820 con 0x507ac00 2026-02-11T23:31:28.213+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:b1721c5a:::obj526:head, version: 111'526, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1721c5a:::obj526:head@111'526, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.213+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b1721c5a:::obj526:head, version: 111'526, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1721c5a:::obj526:head@111'526, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.213+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.213+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 886 ==== pg_backfill(progress 4.0 e 119/119 lb 4:b164ceff:::obj421:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f10000 con 0x507ac00 2026-02-11T23:31:28.213+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b1721c5a:::obj526:head, version: 111'526, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1721c5a:::obj526:head@111'526, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.213+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:b164ceff:::obj421:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.213+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b164ceff:::obj421:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.213+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.213+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b1721c5a:::obj526:head, version: 111'526, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1721c5a:::obj526:head@111'526, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.213+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b1721c5a:::obj526:head, version: 111'526, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1721c5a:::obj526:head@111'526, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 119/117 [PushOp(4:b1721c5a:::obj526:head, version: 111'526, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1721c5a:::obj526:head@111'526, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b133d16d:::obj222:head local-lis/les=117/118 n=422 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.213+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.213+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.213+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b164ceff:::obj421:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.213+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b133d16d:::obj222:head local-lis/les=117/118 n=422 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:b1721c5a:::obj526:head, version: 111'526, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1721c5a:::obj526:head@111'526, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.213+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b133d16d:::obj222:head local-lis/les=117/118 n=422 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b1721c5a:::obj526:head@111'526, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.213+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b133d16d:::obj222:head local-lis/les=117/118 n=422 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:b1721c5a:::obj526:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.213+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b133d16d:::obj222:head local-lis/les=117/118 n=422 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:b1721c5a:::obj526:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.213+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b133d16d:::obj222:head local-lis/les=117/118 n=422 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:b1721c5a:::obj526:head 2026-02-11T23:31:28.213+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b133d16d:::obj222:head local-lis/les=117/118 n=422 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:b1721c5a:::obj526:head v 111'526 2026-02-11T23:31:28.213+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b133d16d:::obj222:head local-lis/les=117/118 n=422 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.213+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.213+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b1721c5a:::obj526:head, version: 111'526, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1721c5a:::obj526:head@111'526, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000071 2026-02-11T23:31:28.213+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.213+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.213+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b164ceff:::obj421:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.213+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b164ceff:::obj421:head) v3 prio 15 cost 0 latency 0.000088 pg_backfill(progress 4.0 e 119/119 lb 4:b164ceff:::obj421:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b133d16d:::obj222:head local-lis/les=117/118 n=422 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.213+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.213+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b133d16d:::obj222:head local-lis/les=117/118 n=422 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:b164ceff:::obj421:head) v3 2026-02-11T23:31:28.213+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b133d16d:::obj222:head local-lis/les=117/118 n=422 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:b164ceff:::obj421:head) 2026-02-11T23:31:28.213+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.213+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b164ceff:::obj421:head) v3 finish latency 0.000014 2026-02-11T23:31:28.213+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b164ceff:::obj421:head local-lis/les=117/118 n=423 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.213+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.213+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.213+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b164ceff:::obj421:head local-lis/les=117/118 n=423 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.213+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.213+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.213+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.213+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66fac00 con 0x507ac00 2026-02-11T23:31:28.213+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:b1721c5a:::obj526:head)]) -- 0x670ad80 con 0x507ac00 2026-02-11T23:31:28.213+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.214+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 887 ==== MOSDPGPush(4.0 119/117 [PushOp(4:b1ca68cf:::obj358:head, version: 111'358, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1ca68cf:::obj358:head@111'358, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a3a00 con 0x507ac00 2026-02-11T23:31:28.214+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:b1ca68cf:::obj358:head, version: 111'358, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1ca68cf:::obj358:head@111'358, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.214+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b1ca68cf:::obj358:head, version: 111'358, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1ca68cf:::obj358:head@111'358, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.214+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.214+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 888 ==== pg_backfill(progress 4.0 e 119/119 lb 4:b1721c5a:::obj526:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f10580 con 0x507ac00 2026-02-11T23:31:28.214+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:b1721c5a:::obj526:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.214+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b1ca68cf:::obj358:head, version: 111'358, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1ca68cf:::obj358:head@111'358, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.214+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b1721c5a:::obj526:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.214+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.214+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b1ca68cf:::obj358:head, version: 111'358, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1ca68cf:::obj358:head@111'358, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.214+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b1ca68cf:::obj358:head, version: 111'358, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1ca68cf:::obj358:head@111'358, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 119/117 [PushOp(4:b1ca68cf:::obj358:head, version: 111'358, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1ca68cf:::obj358:head@111'358, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b164ceff:::obj421:head local-lis/les=117/118 n=423 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.214+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.214+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.214+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b1721c5a:::obj526:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.214+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b164ceff:::obj421:head local-lis/les=117/118 n=423 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:b1ca68cf:::obj358:head, version: 111'358, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1ca68cf:::obj358:head@111'358, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.214+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b164ceff:::obj421:head local-lis/les=117/118 n=423 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b1ca68cf:::obj358:head@111'358, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.214+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b164ceff:::obj421:head local-lis/les=117/118 n=423 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:b1ca68cf:::obj358:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.214+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b164ceff:::obj421:head local-lis/les=117/118 n=423 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:b1ca68cf:::obj358:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.214+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b164ceff:::obj421:head local-lis/les=117/118 n=423 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:b1ca68cf:::obj358:head 2026-02-11T23:31:28.214+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b164ceff:::obj421:head local-lis/les=117/118 n=423 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:b1ca68cf:::obj358:head v 111'358 2026-02-11T23:31:28.214+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b164ceff:::obj421:head local-lis/les=117/118 n=423 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.214+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.214+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b1ca68cf:::obj358:head, version: 111'358, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1ca68cf:::obj358:head@111'358, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000067 2026-02-11T23:31:28.214+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.214+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.214+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b1721c5a:::obj526:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.214+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b1721c5a:::obj526:head) v3 prio 15 cost 0 latency 0.000085 pg_backfill(progress 4.0 e 119/119 lb 4:b1721c5a:::obj526:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b164ceff:::obj421:head local-lis/les=117/118 n=423 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.214+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.214+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b164ceff:::obj421:head local-lis/les=117/118 n=423 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:b1721c5a:::obj526:head) v3 2026-02-11T23:31:28.214+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b164ceff:::obj421:head local-lis/les=117/118 n=423 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:b1721c5a:::obj526:head) 2026-02-11T23:31:28.214+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.214+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b1721c5a:::obj526:head) v3 finish latency 0.000014 2026-02-11T23:31:28.214+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b1721c5a:::obj526:head local-lis/les=117/118 n=424 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.214+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.214+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.214+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b1721c5a:::obj526:head local-lis/les=117/118 n=424 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.214+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.214+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.214+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.214+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66fb380 con 0x507ac00 2026-02-11T23:31:28.214+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:b1ca68cf:::obj358:head)]) -- 0x670b080 con 0x507ac00 2026-02-11T23:31:28.214+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.214+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 889 ==== MOSDPGPush(4.0 119/117 [PushOp(4:b1e2ea00:::obj289:head, version: 111'289, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1e2ea00:::obj289:head@111'289, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db0000 con 0x507ac00 2026-02-11T23:31:28.214+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:b1e2ea00:::obj289:head, version: 111'289, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1e2ea00:::obj289:head@111'289, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.214+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b1e2ea00:::obj289:head, version: 111'289, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1e2ea00:::obj289:head@111'289, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.214+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.214+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 890 ==== pg_backfill(progress 4.0 e 119/119 lb 4:b1ca68cf:::obj358:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f10b00 con 0x507ac00 2026-02-11T23:31:28.214+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:b1ca68cf:::obj358:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.214+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b1e2ea00:::obj289:head, version: 111'289, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1e2ea00:::obj289:head@111'289, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.214+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b1ca68cf:::obj358:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.214+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.214+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b1e2ea00:::obj289:head, version: 111'289, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1e2ea00:::obj289:head@111'289, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.214+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b1e2ea00:::obj289:head, version: 111'289, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1e2ea00:::obj289:head@111'289, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 119/117 [PushOp(4:b1e2ea00:::obj289:head, version: 111'289, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1e2ea00:::obj289:head@111'289, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b1721c5a:::obj526:head local-lis/les=117/118 n=424 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.214+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.214+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.214+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b1ca68cf:::obj358:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.214+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b1721c5a:::obj526:head local-lis/les=117/118 n=424 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:b1e2ea00:::obj289:head, version: 111'289, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1e2ea00:::obj289:head@111'289, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.214+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b1721c5a:::obj526:head local-lis/les=117/118 n=424 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b1e2ea00:::obj289:head@111'289, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.214+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b1721c5a:::obj526:head local-lis/les=117/118 n=424 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:b1e2ea00:::obj289:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.214+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b1721c5a:::obj526:head local-lis/les=117/118 n=424 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:b1e2ea00:::obj289:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.214+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b1721c5a:::obj526:head local-lis/les=117/118 n=424 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:b1e2ea00:::obj289:head 2026-02-11T23:31:28.214+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b1721c5a:::obj526:head local-lis/les=117/118 n=424 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:b1e2ea00:::obj289:head v 111'289 2026-02-11T23:31:28.214+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b1721c5a:::obj526:head local-lis/les=117/118 n=424 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.214+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.214+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b1e2ea00:::obj289:head, version: 111'289, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1e2ea00:::obj289:head@111'289, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000072 2026-02-11T23:31:28.214+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.214+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.214+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b1ca68cf:::obj358:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.214+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b1ca68cf:::obj358:head) v3 prio 15 cost 0 latency 0.000090 pg_backfill(progress 4.0 e 119/119 lb 4:b1ca68cf:::obj358:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b1721c5a:::obj526:head local-lis/les=117/118 n=424 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.214+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.214+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b1721c5a:::obj526:head local-lis/les=117/118 n=424 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:b1ca68cf:::obj358:head) v3 2026-02-11T23:31:28.214+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b1721c5a:::obj526:head local-lis/les=117/118 n=424 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:b1ca68cf:::obj358:head) 2026-02-11T23:31:28.214+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.214+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b1ca68cf:::obj358:head) v3 finish latency 0.000016 2026-02-11T23:31:28.214+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b1ca68cf:::obj358:head local-lis/les=117/118 n=425 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.214+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.215+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.215+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b1ca68cf:::obj358:head local-lis/les=117/118 n=425 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.215+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.215+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.215+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.215+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66faf00 con 0x507ac00 2026-02-11T23:31:28.215+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:b1e2ea00:::obj289:head)]) -- 0x670b380 con 0x507ac00 2026-02-11T23:31:28.215+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.215+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 891 ==== MOSDPGPush(4.0 119/117 [PushOp(4:b26ae718:::obj317:head, version: 111'317, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b26ae718:::obj317:head@111'317, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56caea0 con 0x507ac00 2026-02-11T23:31:28.215+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:b26ae718:::obj317:head, version: 111'317, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b26ae718:::obj317:head@111'317, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.215+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b26ae718:::obj317:head, version: 111'317, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b26ae718:::obj317:head@111'317, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.215+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.215+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 892 ==== pg_backfill(progress 4.0 e 119/119 lb 4:b1e2ea00:::obj289:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f13180 con 0x507ac00 2026-02-11T23:31:28.215+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b26ae718:::obj317:head, version: 111'317, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b26ae718:::obj317:head@111'317, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.215+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:b1e2ea00:::obj289:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.215+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b1e2ea00:::obj289:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.215+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.215+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b26ae718:::obj317:head, version: 111'317, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b26ae718:::obj317:head@111'317, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.215+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b26ae718:::obj317:head, version: 111'317, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b26ae718:::obj317:head@111'317, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 119/117 [PushOp(4:b26ae718:::obj317:head, version: 111'317, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b26ae718:::obj317:head@111'317, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b1ca68cf:::obj358:head local-lis/les=117/118 n=425 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.215+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.215+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.215+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b1e2ea00:::obj289:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.215+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b1ca68cf:::obj358:head local-lis/les=117/118 n=425 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:b26ae718:::obj317:head, version: 111'317, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b26ae718:::obj317:head@111'317, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.215+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b1ca68cf:::obj358:head local-lis/les=117/118 n=425 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b26ae718:::obj317:head@111'317, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.215+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b1ca68cf:::obj358:head local-lis/les=117/118 n=425 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:b26ae718:::obj317:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.215+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b1ca68cf:::obj358:head local-lis/les=117/118 n=425 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:b26ae718:::obj317:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.215+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b1ca68cf:::obj358:head local-lis/les=117/118 n=425 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:b26ae718:::obj317:head 2026-02-11T23:31:28.215+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b1ca68cf:::obj358:head local-lis/les=117/118 n=425 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:b26ae718:::obj317:head v 111'317 2026-02-11T23:31:28.215+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b1ca68cf:::obj358:head local-lis/les=117/118 n=425 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.215+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.215+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b26ae718:::obj317:head, version: 111'317, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b26ae718:::obj317:head@111'317, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000073 2026-02-11T23:31:28.215+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.215+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.215+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b1e2ea00:::obj289:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.215+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b1e2ea00:::obj289:head) v3 prio 15 cost 0 latency 0.000090 pg_backfill(progress 4.0 e 119/119 lb 4:b1e2ea00:::obj289:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b1ca68cf:::obj358:head local-lis/les=117/118 n=425 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.215+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.215+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b1ca68cf:::obj358:head local-lis/les=117/118 n=425 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:b1e2ea00:::obj289:head) v3 2026-02-11T23:31:28.215+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b1ca68cf:::obj358:head local-lis/les=117/118 n=425 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:b1e2ea00:::obj289:head) 2026-02-11T23:31:28.215+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.215+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b1e2ea00:::obj289:head) v3 finish latency 0.000015 2026-02-11T23:31:28.215+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b1e2ea00:::obj289:head local-lis/les=117/118 n=426 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.215+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.215+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.215+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b1e2ea00:::obj289:head local-lis/les=117/118 n=426 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.215+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.215+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.215+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.215+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x670a180 con 0x507ac00 2026-02-11T23:31:28.215+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:b26ae718:::obj317:head)]) -- 0x670b680 con 0x507ac00 2026-02-11T23:31:28.215+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.215+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 893 ==== MOSDPGPush(4.0 119/117 [PushOp(4:b2b26856:::obj586:head, version: 111'586, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b2b26856:::obj586:head@111'586, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56b64e0 con 0x507ac00 2026-02-11T23:31:28.215+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:b2b26856:::obj586:head, version: 111'586, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b2b26856:::obj586:head@111'586, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.215+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b2b26856:::obj586:head, version: 111'586, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b2b26856:::obj586:head@111'586, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.215+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.215+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b2b26856:::obj586:head, version: 111'586, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b2b26856:::obj586:head@111'586, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.216+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 894 ==== pg_backfill(progress 4.0 e 119/119 lb 4:b26ae718:::obj317:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f11080 con 0x507ac00 2026-02-11T23:31:28.216+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:b26ae718:::obj317:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 119 2026-02-11T23:31:28.216+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.216+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b26ae718:::obj317:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.216+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b2b26856:::obj586:head, version: 111'586, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b2b26856:::obj586:head@111'586, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.216+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.216+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b2b26856:::obj586:head, version: 111'586, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b2b26856:::obj586:head@111'586, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 119/117 [PushOp(4:b2b26856:::obj586:head, version: 111'586, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b2b26856:::obj586:head@111'586, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b1e2ea00:::obj289:head local-lis/les=117/118 n=426 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.216+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.216+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b1e2ea00:::obj289:head local-lis/les=117/118 n=426 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:b2b26856:::obj586:head, version: 111'586, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b2b26856:::obj586:head@111'586, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.216+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b1e2ea00:::obj289:head local-lis/les=117/118 n=426 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b2b26856:::obj586:head@111'586, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.216+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.216+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b26ae718:::obj317:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.216+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b1e2ea00:::obj289:head local-lis/les=117/118 n=426 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:b2b26856:::obj586:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.216+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b1e2ea00:::obj289:head local-lis/les=117/118 n=426 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:b2b26856:::obj586:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.216+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b1e2ea00:::obj289:head local-lis/les=117/118 n=426 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:b2b26856:::obj586:head 2026-02-11T23:31:28.216+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b1e2ea00:::obj289:head local-lis/les=117/118 n=426 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:b2b26856:::obj586:head v 111'586 2026-02-11T23:31:28.216+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b1e2ea00:::obj289:head local-lis/les=117/118 n=426 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.216+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.216+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b2b26856:::obj586:head, version: 111'586, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b2b26856:::obj586:head@111'586, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000069 2026-02-11T23:31:28.216+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.216+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.216+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b26ae718:::obj317:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.216+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b26ae718:::obj317:head) v3 prio 15 cost 0 latency 0.000089 pg_backfill(progress 4.0 e 119/119 lb 4:b26ae718:::obj317:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b1e2ea00:::obj289:head local-lis/les=117/118 n=426 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.216+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.216+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b1e2ea00:::obj289:head local-lis/les=117/118 n=426 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:b26ae718:::obj317:head) v3 2026-02-11T23:31:28.216+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b1e2ea00:::obj289:head local-lis/les=117/118 n=426 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:b26ae718:::obj317:head) 2026-02-11T23:31:28.216+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.216+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b26ae718:::obj317:head) v3 finish latency 0.000015 2026-02-11T23:31:28.216+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b26ae718:::obj317:head local-lis/les=117/118 n=427 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.216+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.216+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.216+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b26ae718:::obj317:head local-lis/les=117/118 n=427 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.216+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.216+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.216+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.216+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66fbb00 con 0x507ac00 2026-02-11T23:31:28.216+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:b2b26856:::obj586:head)]) -- 0x670b980 con 0x507ac00 2026-02-11T23:31:28.216+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.216+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 895 ==== MOSDPGPush(4.0 119/117 [PushOp(4:b2ec4718:::obj500:head, version: 111'500, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b2ec4718:::obj500:head@111'500, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f0d00 con 0x507ac00 2026-02-11T23:31:28.216+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:b2ec4718:::obj500:head, version: 111'500, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b2ec4718:::obj500:head@111'500, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.216+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b2ec4718:::obj500:head, version: 111'500, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b2ec4718:::obj500:head@111'500, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.216+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.216+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 896 ==== pg_backfill(progress 4.0 e 119/119 lb 4:b2b26856:::obj586:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f11600 con 0x507ac00 2026-02-11T23:31:28.216+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:b2b26856:::obj586:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 119 2026-02-11T23:31:28.216+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b2ec4718:::obj500:head, version: 111'500, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b2ec4718:::obj500:head@111'500, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.216+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b2b26856:::obj586:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.216+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.216+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b2ec4718:::obj500:head, version: 111'500, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b2ec4718:::obj500:head@111'500, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.216+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.216+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b2b26856:::obj586:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.216+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b2ec4718:::obj500:head, version: 111'500, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b2ec4718:::obj500:head@111'500, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(4.0 119/117 [PushOp(4:b2ec4718:::obj500:head, version: 111'500, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b2ec4718:::obj500:head@111'500, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b26ae718:::obj317:head local-lis/les=117/118 n=427 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.216+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.216+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b26ae718:::obj317:head local-lis/les=117/118 n=427 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:b2ec4718:::obj500:head, version: 111'500, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b2ec4718:::obj500:head@111'500, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.216+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b26ae718:::obj317:head local-lis/les=117/118 n=427 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b2ec4718:::obj500:head@111'500, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.216+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b26ae718:::obj317:head local-lis/les=117/118 n=427 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:b2ec4718:::obj500:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.216+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b26ae718:::obj317:head local-lis/les=117/118 n=427 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:b2ec4718:::obj500:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.216+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b26ae718:::obj317:head local-lis/les=117/118 n=427 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:b2ec4718:::obj500:head 2026-02-11T23:31:28.216+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b26ae718:::obj317:head local-lis/les=117/118 n=427 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:b2ec4718:::obj500:head v 111'500 2026-02-11T23:31:28.216+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b26ae718:::obj317:head local-lis/les=117/118 n=427 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.216+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.216+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b2ec4718:::obj500:head, version: 111'500, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b2ec4718:::obj500:head@111'500, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000073 2026-02-11T23:31:28.216+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.216+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.216+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b2b26856:::obj586:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.216+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b2b26856:::obj586:head) v3 prio 15 cost 0 latency 0.000094 pg_backfill(progress 4.0 e 119/119 lb 4:b2b26856:::obj586:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b26ae718:::obj317:head local-lis/les=117/118 n=427 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.216+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.216+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b26ae718:::obj317:head local-lis/les=117/118 n=427 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:b2b26856:::obj586:head) v3 2026-02-11T23:31:28.216+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b26ae718:::obj317:head local-lis/les=117/118 n=427 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:b2b26856:::obj586:head) 2026-02-11T23:31:28.216+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.216+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b2b26856:::obj586:head) v3 finish latency 0.000017 2026-02-11T23:31:28.216+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b2b26856:::obj586:head local-lis/les=117/118 n=428 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.216+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.216+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.216+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b2b26856:::obj586:head local-lis/les=117/118 n=428 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.216+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.216+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.216+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.216+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66fbc80 con 0x507ac00 2026-02-11T23:31:28.216+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:b2ec4718:::obj500:head)]) -- 0x670bc80 con 0x507ac00 2026-02-11T23:31:28.216+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.216+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.217+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 897 ==== MOSDPGPush(4.0 119/117 [PushOp(4:b33086a8:::obj36:head, version: 111'36, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b33086a8:::obj36:head@111'36, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5647380 con 0x507ac00 2026-02-11T23:31:28.217+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:b33086a8:::obj36:head, version: 111'36, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b33086a8:::obj36:head@111'36, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.217+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b33086a8:::obj36:head, version: 111'36, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b33086a8:::obj36:head@111'36, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.217+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.217+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 898 ==== pg_backfill(progress 4.0 e 119/119 lb 4:b2ec4718:::obj500:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2f700 con 0x507ac00 2026-02-11T23:31:28.217+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:b2ec4718:::obj500:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.217+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b33086a8:::obj36:head, version: 111'36, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b33086a8:::obj36:head@111'36, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.217+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b2ec4718:::obj500:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.217+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.217+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b33086a8:::obj36:head, version: 111'36, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b33086a8:::obj36:head@111'36, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.217+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b33086a8:::obj36:head, version: 111'36, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b33086a8:::obj36:head@111'36, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 119/117 [PushOp(4:b33086a8:::obj36:head, version: 111'36, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b33086a8:::obj36:head@111'36, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b2b26856:::obj586:head local-lis/les=117/118 n=428 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.217+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.217+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.217+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b2ec4718:::obj500:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.217+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b2b26856:::obj586:head local-lis/les=117/118 n=428 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:b33086a8:::obj36:head, version: 111'36, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b33086a8:::obj36:head@111'36, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.217+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b2b26856:::obj586:head local-lis/les=117/118 n=428 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b33086a8:::obj36:head@111'36, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.217+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b2b26856:::obj586:head local-lis/les=117/118 n=428 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:b33086a8:::obj36:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.217+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b2b26856:::obj586:head local-lis/les=117/118 n=428 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:b33086a8:::obj36:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.217+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b2b26856:::obj586:head local-lis/les=117/118 n=428 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:b33086a8:::obj36:head 2026-02-11T23:31:28.217+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b2b26856:::obj586:head local-lis/les=117/118 n=428 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:b33086a8:::obj36:head v 111'36 2026-02-11T23:31:28.217+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b2b26856:::obj586:head local-lis/les=117/118 n=428 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.217+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.217+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b33086a8:::obj36:head, version: 111'36, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b33086a8:::obj36:head@111'36, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000080 2026-02-11T23:31:28.217+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.217+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.217+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b2ec4718:::obj500:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.217+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b2ec4718:::obj500:head) v3 prio 15 cost 0 latency 0.000097 pg_backfill(progress 4.0 e 119/119 lb 4:b2ec4718:::obj500:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b2b26856:::obj586:head local-lis/les=117/118 n=428 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.217+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.217+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b2b26856:::obj586:head local-lis/les=117/118 n=428 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:b2ec4718:::obj500:head) v3 2026-02-11T23:31:28.217+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b2b26856:::obj586:head local-lis/les=117/118 n=428 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:b2ec4718:::obj500:head) 2026-02-11T23:31:28.217+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.217+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b2ec4718:::obj500:head) v3 finish latency 0.000015 2026-02-11T23:31:28.217+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b2ec4718:::obj500:head local-lis/les=117/118 n=429 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.217+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.217+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.217+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b2ec4718:::obj500:head local-lis/les=117/118 n=429 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.217+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.217+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.217+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.217+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66fb800 con 0x507ac00 2026-02-11T23:31:28.217+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:b33086a8:::obj36:head)]) -- 0x671c000 con 0x507ac00 2026-02-11T23:31:28.217+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.217+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 899 ==== MOSDPGPush(4.0 119/117 [PushOp(4:b4726009:::obj503:head, version: 111'503, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b4726009:::obj503:head@111'503, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5697ba0 con 0x507ac00 2026-02-11T23:31:28.217+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:b4726009:::obj503:head, version: 111'503, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b4726009:::obj503:head@111'503, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.217+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b4726009:::obj503:head, version: 111'503, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b4726009:::obj503:head@111'503, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.217+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.217+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 900 ==== pg_backfill(progress 4.0 e 119/119 lb 4:b33086a8:::obj36:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d2d080 con 0x507ac00 2026-02-11T23:31:28.217+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:b33086a8:::obj36:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.217+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b4726009:::obj503:head, version: 111'503, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b4726009:::obj503:head@111'503, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.217+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b33086a8:::obj36:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.217+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.217+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b4726009:::obj503:head, version: 111'503, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b4726009:::obj503:head@111'503, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.217+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b4726009:::obj503:head, version: 111'503, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b4726009:::obj503:head@111'503, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 119/117 [PushOp(4:b4726009:::obj503:head, version: 111'503, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b4726009:::obj503:head@111'503, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b2ec4718:::obj500:head local-lis/les=117/118 n=429 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.217+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.217+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.217+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b33086a8:::obj36:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.217+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b2ec4718:::obj500:head local-lis/les=117/118 n=429 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:b4726009:::obj503:head, version: 111'503, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b4726009:::obj503:head@111'503, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.217+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b2ec4718:::obj500:head local-lis/les=117/118 n=429 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b4726009:::obj503:head@111'503, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.217+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b2ec4718:::obj500:head local-lis/les=117/118 n=429 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:b4726009:::obj503:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.217+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b2ec4718:::obj500:head local-lis/les=117/118 n=429 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:b4726009:::obj503:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.217+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b2ec4718:::obj500:head local-lis/les=117/118 n=429 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:b4726009:::obj503:head 2026-02-11T23:31:28.217+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b2ec4718:::obj500:head local-lis/les=117/118 n=429 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:b4726009:::obj503:head v 111'503 2026-02-11T23:31:28.217+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b2ec4718:::obj500:head local-lis/les=117/118 n=429 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.217+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.217+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b4726009:::obj503:head, version: 111'503, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b4726009:::obj503:head@111'503, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000076 2026-02-11T23:31:28.217+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.217+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.217+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b33086a8:::obj36:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.217+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b33086a8:::obj36:head) v3 prio 15 cost 0 latency 0.000093 pg_backfill(progress 4.0 e 119/119 lb 4:b33086a8:::obj36:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b2ec4718:::obj500:head local-lis/les=117/118 n=429 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.217+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.217+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b2ec4718:::obj500:head local-lis/les=117/118 n=429 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:b33086a8:::obj36:head) v3 2026-02-11T23:31:28.217+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b2ec4718:::obj500:head local-lis/les=117/118 n=429 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:b33086a8:::obj36:head) 2026-02-11T23:31:28.217+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.217+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b33086a8:::obj36:head) v3 finish latency 0.000014 2026-02-11T23:31:28.217+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b33086a8:::obj36:head local-lis/les=117/118 n=430 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.217+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.218+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.218+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b33086a8:::obj36:head local-lis/les=117/118 n=430 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.218+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.218+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.218+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.218+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66fa480 con 0x507ac00 2026-02-11T23:31:28.218+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:b4726009:::obj503:head)]) -- 0x671c180 con 0x507ac00 2026-02-11T23:31:28.218+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.218+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 901 ==== MOSDPGPush(4.0 119/117 [PushOp(4:b5298834:::obj386:head, version: 111'386, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5298834:::obj386:head@111'386, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5617380 con 0x507ac00 2026-02-11T23:31:28.218+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:b5298834:::obj386:head, version: 111'386, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5298834:::obj386:head@111'386, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 119 2026-02-11T23:31:28.218+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b5298834:::obj386:head, version: 111'386, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5298834:::obj386:head@111'386, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.218+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.218+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 902 ==== pg_backfill(progress 4.0 e 119/119 lb 4:b4726009:::obj503:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e680 con 0x507ac00 2026-02-11T23:31:28.218+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:b4726009:::obj503:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.218+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b5298834:::obj386:head, version: 111'386, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5298834:::obj386:head@111'386, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.218+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b4726009:::obj503:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.218+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.218+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b5298834:::obj386:head, version: 111'386, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5298834:::obj386:head@111'386, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.218+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b5298834:::obj386:head, version: 111'386, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5298834:::obj386:head@111'386, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(4.0 119/117 [PushOp(4:b5298834:::obj386:head, version: 111'386, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5298834:::obj386:head@111'386, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b33086a8:::obj36:head local-lis/les=117/118 n=430 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.218+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.218+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.218+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b4726009:::obj503:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.218+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b33086a8:::obj36:head local-lis/les=117/118 n=430 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:b5298834:::obj386:head, version: 111'386, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5298834:::obj386:head@111'386, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.218+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b33086a8:::obj36:head local-lis/les=117/118 n=430 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b5298834:::obj386:head@111'386, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.218+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b33086a8:::obj36:head local-lis/les=117/118 n=430 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:b5298834:::obj386:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.218+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b33086a8:::obj36:head local-lis/les=117/118 n=430 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:b5298834:::obj386:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.218+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b33086a8:::obj36:head local-lis/les=117/118 n=430 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:b5298834:::obj386:head 2026-02-11T23:31:28.218+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b33086a8:::obj36:head local-lis/les=117/118 n=430 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:b5298834:::obj386:head v 111'386 2026-02-11T23:31:28.218+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b33086a8:::obj36:head local-lis/les=117/118 n=430 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.218+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.218+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b5298834:::obj386:head, version: 111'386, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5298834:::obj386:head@111'386, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000070 2026-02-11T23:31:28.218+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.218+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.218+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b4726009:::obj503:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.218+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b4726009:::obj503:head) v3 prio 15 cost 0 latency 0.000088 pg_backfill(progress 4.0 e 119/119 lb 4:b4726009:::obj503:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b33086a8:::obj36:head local-lis/les=117/118 n=430 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.218+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.218+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b33086a8:::obj36:head local-lis/les=117/118 n=430 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:b4726009:::obj503:head) v3 2026-02-11T23:31:28.218+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b33086a8:::obj36:head local-lis/les=117/118 n=430 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:b4726009:::obj503:head) 2026-02-11T23:31:28.218+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.218+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b4726009:::obj503:head) v3 finish latency 0.000015 2026-02-11T23:31:28.218+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b4726009:::obj503:head local-lis/les=117/118 n=431 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.218+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.218+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.218+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b4726009:::obj503:head local-lis/les=117/118 n=431 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.218+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.218+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.218+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.218+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66fa000 con 0x507ac00 2026-02-11T23:31:28.218+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:b5298834:::obj386:head)]) -- 0x671c480 con 0x507ac00 2026-02-11T23:31:28.218+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.219+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 903 ==== MOSDPGPush(4.0 119/117 [PushOp(4:b5339e65:::obj355:head, version: 111'355, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5339e65:::obj355:head@111'355, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f0b60 con 0x507ac00 2026-02-11T23:31:28.219+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:b5339e65:::obj355:head, version: 111'355, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5339e65:::obj355:head@111'355, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.219+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b5339e65:::obj355:head, version: 111'355, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5339e65:::obj355:head@111'355, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.219+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.219+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 904 ==== pg_backfill(progress 4.0 e 119/119 lb 4:b5298834:::obj386:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176580 con 0x507ac00 2026-02-11T23:31:28.219+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:b5298834:::obj386:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.219+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b5339e65:::obj355:head, version: 111'355, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5339e65:::obj355:head@111'355, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.219+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b5298834:::obj386:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.219+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.219+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b5339e65:::obj355:head, version: 111'355, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5339e65:::obj355:head@111'355, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.219+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b5339e65:::obj355:head, version: 111'355, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5339e65:::obj355:head@111'355, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 119/117 [PushOp(4:b5339e65:::obj355:head, version: 111'355, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5339e65:::obj355:head@111'355, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b4726009:::obj503:head local-lis/les=117/118 n=431 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.219+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.219+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.219+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b5298834:::obj386:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.219+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b4726009:::obj503:head local-lis/les=117/118 n=431 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:b5339e65:::obj355:head, version: 111'355, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5339e65:::obj355:head@111'355, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.219+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b4726009:::obj503:head local-lis/les=117/118 n=431 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b5339e65:::obj355:head@111'355, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.219+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b4726009:::obj503:head local-lis/les=117/118 n=431 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:b5339e65:::obj355:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.219+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b4726009:::obj503:head local-lis/les=117/118 n=431 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:b5339e65:::obj355:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.219+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b4726009:::obj503:head local-lis/les=117/118 n=431 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:b5339e65:::obj355:head 2026-02-11T23:31:28.219+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b4726009:::obj503:head local-lis/les=117/118 n=431 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:b5339e65:::obj355:head v 111'355 2026-02-11T23:31:28.219+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b4726009:::obj503:head local-lis/les=117/118 n=431 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.219+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.219+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b5339e65:::obj355:head, version: 111'355, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5339e65:::obj355:head@111'355, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000069 2026-02-11T23:31:28.219+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.219+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.219+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b5298834:::obj386:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.219+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b5298834:::obj386:head) v3 prio 15 cost 0 latency 0.000087 pg_backfill(progress 4.0 e 119/119 lb 4:b5298834:::obj386:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b4726009:::obj503:head local-lis/les=117/118 n=431 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.219+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.219+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b4726009:::obj503:head local-lis/les=117/118 n=431 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:b5298834:::obj386:head) v3 2026-02-11T23:31:28.219+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b4726009:::obj503:head local-lis/les=117/118 n=431 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:b5298834:::obj386:head) 2026-02-11T23:31:28.219+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.219+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b5298834:::obj386:head) v3 finish latency 0.000015 2026-02-11T23:31:28.219+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b5298834:::obj386:head local-lis/les=117/118 n=432 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.219+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.219+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.219+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b5298834:::obj386:head local-lis/les=117/118 n=432 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.219+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.219+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.219+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.219+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66fa300 con 0x507ac00 2026-02-11T23:31:28.219+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:b5339e65:::obj355:head)]) -- 0x671c780 con 0x507ac00 2026-02-11T23:31:28.219+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.219+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 905 ==== MOSDPGPush(4.0 119/117 [PushOp(4:b5445b0d:::obj455:head, version: 111'455, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5445b0d:::obj455:head@111'455, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f1520 con 0x507ac00 2026-02-11T23:31:28.219+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:b5445b0d:::obj455:head, version: 111'455, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5445b0d:::obj455:head@111'455, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.219+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b5445b0d:::obj455:head, version: 111'455, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5445b0d:::obj455:head@111'455, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.219+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.219+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 906 ==== pg_backfill(progress 4.0 e 119/119 lb 4:b5339e65:::obj355:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e100 con 0x507ac00 2026-02-11T23:31:28.219+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:b5339e65:::obj355:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.219+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b5445b0d:::obj455:head, version: 111'455, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5445b0d:::obj455:head@111'455, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.219+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b5339e65:::obj355:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.219+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.219+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b5445b0d:::obj455:head, version: 111'455, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5445b0d:::obj455:head@111'455, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.219+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.219+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b5339e65:::obj355:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.219+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b5445b0d:::obj455:head, version: 111'455, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5445b0d:::obj455:head@111'455, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 119/117 [PushOp(4:b5445b0d:::obj455:head, version: 111'455, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5445b0d:::obj455:head@111'455, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b5298834:::obj386:head local-lis/les=117/118 n=432 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.219+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.219+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b5298834:::obj386:head local-lis/les=117/118 n=432 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:b5445b0d:::obj455:head, version: 111'455, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5445b0d:::obj455:head@111'455, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.219+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b5298834:::obj386:head local-lis/les=117/118 n=432 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b5445b0d:::obj455:head@111'455, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.219+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b5298834:::obj386:head local-lis/les=117/118 n=432 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:b5445b0d:::obj455:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.219+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b5298834:::obj386:head local-lis/les=117/118 n=432 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:b5445b0d:::obj455:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.219+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b5298834:::obj386:head local-lis/les=117/118 n=432 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:b5445b0d:::obj455:head 2026-02-11T23:31:28.219+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b5298834:::obj386:head local-lis/les=117/118 n=432 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:b5445b0d:::obj455:head v 111'455 2026-02-11T23:31:28.219+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b5298834:::obj386:head local-lis/les=117/118 n=432 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.219+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.219+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b5445b0d:::obj455:head, version: 111'455, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5445b0d:::obj455:head@111'455, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000066 2026-02-11T23:31:28.219+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.219+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.219+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b5339e65:::obj355:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.219+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b5339e65:::obj355:head) v3 prio 15 cost 0 latency 0.000084 pg_backfill(progress 4.0 e 119/119 lb 4:b5339e65:::obj355:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b5298834:::obj386:head local-lis/les=117/118 n=432 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.219+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.219+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b5298834:::obj386:head local-lis/les=117/118 n=432 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:b5339e65:::obj355:head) v3 2026-02-11T23:31:28.219+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b5298834:::obj386:head local-lis/les=117/118 n=432 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:b5339e65:::obj355:head) 2026-02-11T23:31:28.219+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.219+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b5339e65:::obj355:head) v3 finish latency 0.000015 2026-02-11T23:31:28.219+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b5339e65:::obj355:head local-lis/les=117/118 n=433 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.219+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.219+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.219+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b5339e65:::obj355:head local-lis/les=117/118 n=433 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.219+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.219+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.219+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.219+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66d8480 con 0x507ac00 2026-02-11T23:31:28.219+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:b5445b0d:::obj455:head)]) -- 0x671ca80 con 0x507ac00 2026-02-11T23:31:28.219+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.220+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 907 ==== MOSDPGPush(4.0 119/117 [PushOp(4:b59cea89:::obj183:head, version: 111'183, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b59cea89:::obj183:head@111'183, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a3860 con 0x507ac00 2026-02-11T23:31:28.220+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:b59cea89:::obj183:head, version: 111'183, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b59cea89:::obj183:head@111'183, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.220+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b59cea89:::obj183:head, version: 111'183, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b59cea89:::obj183:head@111'183, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.220+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.220+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 908 ==== pg_backfill(progress 4.0 e 119/119 lb 4:b5445b0d:::obj455:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2c000 con 0x507ac00 2026-02-11T23:31:28.220+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:b5445b0d:::obj455:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.220+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b59cea89:::obj183:head, version: 111'183, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b59cea89:::obj183:head@111'183, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.220+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b5445b0d:::obj455:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.220+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.220+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b59cea89:::obj183:head, version: 111'183, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b59cea89:::obj183:head@111'183, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.220+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b59cea89:::obj183:head, version: 111'183, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b59cea89:::obj183:head@111'183, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 119/117 [PushOp(4:b59cea89:::obj183:head, version: 111'183, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b59cea89:::obj183:head@111'183, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b5339e65:::obj355:head local-lis/les=117/118 n=433 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.220+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.220+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.220+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b5445b0d:::obj455:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.220+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b5339e65:::obj355:head local-lis/les=117/118 n=433 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:b59cea89:::obj183:head, version: 111'183, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b59cea89:::obj183:head@111'183, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.220+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b5339e65:::obj355:head local-lis/les=117/118 n=433 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b59cea89:::obj183:head@111'183, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.220+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b5339e65:::obj355:head local-lis/les=117/118 n=433 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:b59cea89:::obj183:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.220+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b5339e65:::obj355:head local-lis/les=117/118 n=433 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:b59cea89:::obj183:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.220+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b5339e65:::obj355:head local-lis/les=117/118 n=433 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:b59cea89:::obj183:head 2026-02-11T23:31:28.220+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b5339e65:::obj355:head local-lis/les=117/118 n=433 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:b59cea89:::obj183:head v 111'183 2026-02-11T23:31:28.220+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b5339e65:::obj355:head local-lis/les=117/118 n=433 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.220+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.220+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b59cea89:::obj183:head, version: 111'183, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b59cea89:::obj183:head@111'183, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000068 2026-02-11T23:31:28.220+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.220+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.220+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b5445b0d:::obj455:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.220+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b5445b0d:::obj455:head) v3 prio 15 cost 0 latency 0.000085 pg_backfill(progress 4.0 e 119/119 lb 4:b5445b0d:::obj455:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b5339e65:::obj355:head local-lis/les=117/118 n=433 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.220+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.220+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b5339e65:::obj355:head local-lis/les=117/118 n=433 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:b5445b0d:::obj455:head) v3 2026-02-11T23:31:28.220+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b5339e65:::obj355:head local-lis/les=117/118 n=433 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:b5445b0d:::obj455:head) 2026-02-11T23:31:28.220+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.220+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b5445b0d:::obj455:head) v3 finish latency 0.000016 2026-02-11T23:31:28.220+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b5445b0d:::obj455:head local-lis/les=117/118 n=434 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.220+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.220+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.220+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b5445b0d:::obj455:head local-lis/les=117/118 n=434 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.220+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.220+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.220+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.220+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66d9200 con 0x507ac00 2026-02-11T23:31:28.220+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:b59cea89:::obj183:head)]) -- 0x671cd80 con 0x507ac00 2026-02-11T23:31:28.220+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.220+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 909 ==== MOSDPGPush(4.0 119/117 [PushOp(4:b5a67629:::obj219:head, version: 111'219, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5a67629:::obj219:head@111'219, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56cab60 con 0x507ac00 2026-02-11T23:31:28.220+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:b5a67629:::obj219:head, version: 111'219, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5a67629:::obj219:head@111'219, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.220+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b5a67629:::obj219:head, version: 111'219, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5a67629:::obj219:head@111'219, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.220+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.220+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b5a67629:::obj219:head, version: 111'219, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5a67629:::obj219:head@111'219, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.220+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.220+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 910 ==== pg_backfill(progress 4.0 e 119/119 lb 4:b59cea89:::obj183:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2cb00 con 0x507ac00 2026-02-11T23:31:28.220+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b5a67629:::obj219:head, version: 111'219, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5a67629:::obj219:head@111'219, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.220+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:b59cea89:::obj183:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 119 2026-02-11T23:31:28.220+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b59cea89:::obj183:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.220+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.220+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b59cea89:::obj183:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.220+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b5a67629:::obj219:head, version: 111'219, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5a67629:::obj219:head@111'219, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(4.0 119/117 [PushOp(4:b5a67629:::obj219:head, version: 111'219, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5a67629:::obj219:head@111'219, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b5445b0d:::obj455:head local-lis/les=117/118 n=434 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.220+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.220+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b5445b0d:::obj455:head local-lis/les=117/118 n=434 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:b5a67629:::obj219:head, version: 111'219, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5a67629:::obj219:head@111'219, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.220+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b5445b0d:::obj455:head local-lis/les=117/118 n=434 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b5a67629:::obj219:head@111'219, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.220+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b5445b0d:::obj455:head local-lis/les=117/118 n=434 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:b5a67629:::obj219:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.220+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b5445b0d:::obj455:head local-lis/les=117/118 n=434 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:b5a67629:::obj219:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.220+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b5445b0d:::obj455:head local-lis/les=117/118 n=434 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:b5a67629:::obj219:head 2026-02-11T23:31:28.220+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b5445b0d:::obj455:head local-lis/les=117/118 n=434 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:b5a67629:::obj219:head v 111'219 2026-02-11T23:31:28.220+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b5445b0d:::obj455:head local-lis/les=117/118 n=434 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.220+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.220+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b5a67629:::obj219:head, version: 111'219, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5a67629:::obj219:head@111'219, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000066 2026-02-11T23:31:28.220+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.220+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.220+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b59cea89:::obj183:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.220+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b59cea89:::obj183:head) v3 prio 15 cost 0 latency 0.000081 pg_backfill(progress 4.0 e 119/119 lb 4:b59cea89:::obj183:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b5445b0d:::obj455:head local-lis/les=117/118 n=434 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.220+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.220+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b5445b0d:::obj455:head local-lis/les=117/118 n=434 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:b59cea89:::obj183:head) v3 2026-02-11T23:31:28.220+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b5445b0d:::obj455:head local-lis/les=117/118 n=434 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:b59cea89:::obj183:head) 2026-02-11T23:31:28.220+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.220+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b59cea89:::obj183:head) v3 finish latency 0.000017 2026-02-11T23:31:28.220+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b59cea89:::obj183:head local-lis/les=117/118 n=435 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.220+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.221+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.221+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b59cea89:::obj183:head local-lis/les=117/118 n=435 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.221+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.221+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.221+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.221+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66d9500 con 0x507ac00 2026-02-11T23:31:28.221+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:b5a67629:::obj219:head)]) -- 0x671d080 con 0x507ac00 2026-02-11T23:31:28.221+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.221+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 911 ==== MOSDPGPush(4.0 119/117 [PushOp(4:b609c5a3:::obj261:head, version: 111'261, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b609c5a3:::obj261:head@111'261, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2680 con 0x507ac00 2026-02-11T23:31:28.221+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:b609c5a3:::obj261:head, version: 111'261, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b609c5a3:::obj261:head@111'261, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.221+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b609c5a3:::obj261:head, version: 111'261, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b609c5a3:::obj261:head@111'261, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.221+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.221+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 912 ==== pg_backfill(progress 4.0 e 119/119 lb 4:b5a67629:::obj219:head) ==== 1062+0+0 (unknown 0 0 0) 0x6179700 con 0x507ac00 2026-02-11T23:31:28.221+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b609c5a3:::obj261:head, version: 111'261, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b609c5a3:::obj261:head@111'261, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.221+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:b5a67629:::obj219:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.221+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b5a67629:::obj219:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.221+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.221+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b609c5a3:::obj261:head, version: 111'261, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b609c5a3:::obj261:head@111'261, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.221+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b609c5a3:::obj261:head, version: 111'261, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b609c5a3:::obj261:head@111'261, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 119/117 [PushOp(4:b609c5a3:::obj261:head, version: 111'261, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b609c5a3:::obj261:head@111'261, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b59cea89:::obj183:head local-lis/les=117/118 n=435 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.221+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.221+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.221+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b5a67629:::obj219:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.221+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b59cea89:::obj183:head local-lis/les=117/118 n=435 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:b609c5a3:::obj261:head, version: 111'261, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b609c5a3:::obj261:head@111'261, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.221+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b59cea89:::obj183:head local-lis/les=117/118 n=435 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b609c5a3:::obj261:head@111'261, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.221+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b59cea89:::obj183:head local-lis/les=117/118 n=435 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:b609c5a3:::obj261:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.221+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b59cea89:::obj183:head local-lis/les=117/118 n=435 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:b609c5a3:::obj261:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.221+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b59cea89:::obj183:head local-lis/les=117/118 n=435 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:b609c5a3:::obj261:head 2026-02-11T23:31:28.221+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b59cea89:::obj183:head local-lis/les=117/118 n=435 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:b609c5a3:::obj261:head v 111'261 2026-02-11T23:31:28.221+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b59cea89:::obj183:head local-lis/les=117/118 n=435 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.221+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.221+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b609c5a3:::obj261:head, version: 111'261, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b609c5a3:::obj261:head@111'261, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000065 2026-02-11T23:31:28.221+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.221+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.221+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b5a67629:::obj219:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.221+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b5a67629:::obj219:head) v3 prio 15 cost 0 latency 0.000082 pg_backfill(progress 4.0 e 119/119 lb 4:b5a67629:::obj219:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b59cea89:::obj183:head local-lis/les=117/118 n=435 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.221+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.221+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b59cea89:::obj183:head local-lis/les=117/118 n=435 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:b5a67629:::obj219:head) v3 2026-02-11T23:31:28.221+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b59cea89:::obj183:head local-lis/les=117/118 n=435 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:b5a67629:::obj219:head) 2026-02-11T23:31:28.221+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.221+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b5a67629:::obj219:head) v3 finish latency 0.000014 2026-02-11T23:31:28.221+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b5a67629:::obj219:head local-lis/les=117/118 n=436 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.221+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.221+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.221+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b5a67629:::obj219:head local-lis/les=117/118 n=436 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.221+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.221+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.221+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.221+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x640f680 con 0x507ac00 2026-02-11T23:31:28.221+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:b609c5a3:::obj261:head)]) -- 0x671d380 con 0x507ac00 2026-02-11T23:31:28.221+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.222+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 913 ==== MOSDPGPush(4.0 119/117 [PushOp(4:b66eab34:::obj112:head, version: 111'112, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b66eab34:::obj112:head@111'112, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3815380 con 0x507ac00 2026-02-11T23:31:28.222+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:b66eab34:::obj112:head, version: 111'112, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b66eab34:::obj112:head@111'112, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.222+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b66eab34:::obj112:head, version: 111'112, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b66eab34:::obj112:head@111'112, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.222+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.222+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 914 ==== pg_backfill(progress 4.0 e 119/119 lb 4:b609c5a3:::obj261:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176000 con 0x507ac00 2026-02-11T23:31:28.222+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b66eab34:::obj112:head, version: 111'112, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b66eab34:::obj112:head@111'112, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.222+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:b609c5a3:::obj261:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.222+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b609c5a3:::obj261:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.222+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.222+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b66eab34:::obj112:head, version: 111'112, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b66eab34:::obj112:head@111'112, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.222+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b66eab34:::obj112:head, version: 111'112, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b66eab34:::obj112:head@111'112, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 119/117 [PushOp(4:b66eab34:::obj112:head, version: 111'112, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b66eab34:::obj112:head@111'112, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b5a67629:::obj219:head local-lis/les=117/118 n=436 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.222+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.222+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.222+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b609c5a3:::obj261:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.222+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b5a67629:::obj219:head local-lis/les=117/118 n=436 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:b66eab34:::obj112:head, version: 111'112, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b66eab34:::obj112:head@111'112, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.222+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b5a67629:::obj219:head local-lis/les=117/118 n=436 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b66eab34:::obj112:head@111'112, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.222+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b5a67629:::obj219:head local-lis/les=117/118 n=436 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:b66eab34:::obj112:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.222+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b5a67629:::obj219:head local-lis/les=117/118 n=436 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:b66eab34:::obj112:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.222+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b5a67629:::obj219:head local-lis/les=117/118 n=436 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:b66eab34:::obj112:head 2026-02-11T23:31:28.222+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b5a67629:::obj219:head local-lis/les=117/118 n=436 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:b66eab34:::obj112:head v 111'112 2026-02-11T23:31:28.222+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b5a67629:::obj219:head local-lis/les=117/118 n=436 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.222+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.222+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b66eab34:::obj112:head, version: 111'112, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b66eab34:::obj112:head@111'112, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000070 2026-02-11T23:31:28.222+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.222+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.222+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b609c5a3:::obj261:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.222+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b609c5a3:::obj261:head) v3 prio 15 cost 0 latency 0.000088 pg_backfill(progress 4.0 e 119/119 lb 4:b609c5a3:::obj261:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b5a67629:::obj219:head local-lis/les=117/118 n=436 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.222+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.222+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b5a67629:::obj219:head local-lis/les=117/118 n=436 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:b609c5a3:::obj261:head) v3 2026-02-11T23:31:28.222+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b5a67629:::obj219:head local-lis/les=117/118 n=436 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:b609c5a3:::obj261:head) 2026-02-11T23:31:28.222+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.222+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b609c5a3:::obj261:head) v3 finish latency 0.000014 2026-02-11T23:31:28.222+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b609c5a3:::obj261:head local-lis/les=117/118 n=437 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.222+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.222+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.222+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b609c5a3:::obj261:head local-lis/les=117/118 n=437 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.222+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.222+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.222+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.222+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66d9080 con 0x507ac00 2026-02-11T23:31:28.222+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:b66eab34:::obj112:head)]) -- 0x671d680 con 0x507ac00 2026-02-11T23:31:28.222+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.222+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 915 ==== MOSDPGPush(4.0 119/117 [PushOp(4:b6a90992:::obj552:head, version: 111'552, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b6a90992:::obj552:head@111'552, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e26b60 con 0x507ac00 2026-02-11T23:31:28.222+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:b6a90992:::obj552:head, version: 111'552, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b6a90992:::obj552:head@111'552, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.222+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b6a90992:::obj552:head, version: 111'552, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b6a90992:::obj552:head@111'552, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.222+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.222+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 916 ==== pg_backfill(progress 4.0 e 119/119 lb 4:b66eab34:::obj112:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9a680 con 0x507ac00 2026-02-11T23:31:28.222+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:b66eab34:::obj112:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 119 2026-02-11T23:31:28.222+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b6a90992:::obj552:head, version: 111'552, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b6a90992:::obj552:head@111'552, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.222+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b66eab34:::obj112:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.222+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.222+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b6a90992:::obj552:head, version: 111'552, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b6a90992:::obj552:head@111'552, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.222+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.222+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b66eab34:::obj112:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.222+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b6a90992:::obj552:head, version: 111'552, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b6a90992:::obj552:head@111'552, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(4.0 119/117 [PushOp(4:b6a90992:::obj552:head, version: 111'552, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b6a90992:::obj552:head@111'552, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b609c5a3:::obj261:head local-lis/les=117/118 n=437 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.222+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.222+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b609c5a3:::obj261:head local-lis/les=117/118 n=437 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:b6a90992:::obj552:head, version: 111'552, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b6a90992:::obj552:head@111'552, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.222+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b609c5a3:::obj261:head local-lis/les=117/118 n=437 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b6a90992:::obj552:head@111'552, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.222+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b609c5a3:::obj261:head local-lis/les=117/118 n=437 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:b6a90992:::obj552:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.222+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b609c5a3:::obj261:head local-lis/les=117/118 n=437 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:b6a90992:::obj552:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.222+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b609c5a3:::obj261:head local-lis/les=117/118 n=437 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:b6a90992:::obj552:head 2026-02-11T23:31:28.222+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b609c5a3:::obj261:head local-lis/les=117/118 n=437 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:b6a90992:::obj552:head v 111'552 2026-02-11T23:31:28.222+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b609c5a3:::obj261:head local-lis/les=117/118 n=437 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.222+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.222+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b6a90992:::obj552:head, version: 111'552, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b6a90992:::obj552:head@111'552, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000067 2026-02-11T23:31:28.222+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.222+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.222+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b66eab34:::obj112:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.222+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b66eab34:::obj112:head) v3 prio 15 cost 0 latency 0.000086 pg_backfill(progress 4.0 e 119/119 lb 4:b66eab34:::obj112:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b609c5a3:::obj261:head local-lis/les=117/118 n=437 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.222+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.222+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b609c5a3:::obj261:head local-lis/les=117/118 n=437 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:b66eab34:::obj112:head) v3 2026-02-11T23:31:28.222+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b609c5a3:::obj261:head local-lis/les=117/118 n=437 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:b66eab34:::obj112:head) 2026-02-11T23:31:28.222+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.222+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b66eab34:::obj112:head) v3 finish latency 0.000013 2026-02-11T23:31:28.222+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b66eab34:::obj112:head local-lis/les=117/118 n=438 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.222+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.223+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.223+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b66eab34:::obj112:head local-lis/les=117/118 n=438 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.223+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.223+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.223+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.223+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x642cd80 con 0x507ac00 2026-02-11T23:31:28.223+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:b6a90992:::obj552:head)]) -- 0x671d980 con 0x507ac00 2026-02-11T23:31:28.223+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.223+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 917 ==== MOSDPGPush(4.0 119/117 [PushOp(4:b6dfbaf7:::obj426:head, version: 111'426, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b6dfbaf7:::obj426:head@111'426, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a29c0 con 0x507ac00 2026-02-11T23:31:28.223+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:b6dfbaf7:::obj426:head, version: 111'426, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b6dfbaf7:::obj426:head@111'426, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.223+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b6dfbaf7:::obj426:head, version: 111'426, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b6dfbaf7:::obj426:head@111'426, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.223+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.223+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 918 ==== pg_backfill(progress 4.0 e 119/119 lb 4:b6a90992:::obj552:head) ==== 1062+0+0 (unknown 0 0 0) 0x5985700 con 0x507ac00 2026-02-11T23:31:28.223+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b6dfbaf7:::obj426:head, version: 111'426, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b6dfbaf7:::obj426:head@111'426, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.223+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:b6a90992:::obj552:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.223+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.223+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b6a90992:::obj552:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.223+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b6dfbaf7:::obj426:head, version: 111'426, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b6dfbaf7:::obj426:head@111'426, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.223+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.223+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b6dfbaf7:::obj426:head, version: 111'426, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b6dfbaf7:::obj426:head@111'426, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 119/117 [PushOp(4:b6dfbaf7:::obj426:head, version: 111'426, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b6dfbaf7:::obj426:head@111'426, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b66eab34:::obj112:head local-lis/les=117/118 n=438 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.223+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.223+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.223+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b6a90992:::obj552:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.223+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b66eab34:::obj112:head local-lis/les=117/118 n=438 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:b6dfbaf7:::obj426:head, version: 111'426, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b6dfbaf7:::obj426:head@111'426, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.223+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b66eab34:::obj112:head local-lis/les=117/118 n=438 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b6dfbaf7:::obj426:head@111'426, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.223+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b66eab34:::obj112:head local-lis/les=117/118 n=438 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:b6dfbaf7:::obj426:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.223+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b66eab34:::obj112:head local-lis/les=117/118 n=438 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:b6dfbaf7:::obj426:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.223+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b66eab34:::obj112:head local-lis/les=117/118 n=438 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:b6dfbaf7:::obj426:head 2026-02-11T23:31:28.223+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b66eab34:::obj112:head local-lis/les=117/118 n=438 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:b6dfbaf7:::obj426:head v 111'426 2026-02-11T23:31:28.223+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b66eab34:::obj112:head local-lis/les=117/118 n=438 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.223+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.223+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b6dfbaf7:::obj426:head, version: 111'426, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b6dfbaf7:::obj426:head@111'426, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000078 2026-02-11T23:31:28.223+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.223+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.223+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b6a90992:::obj552:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.223+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b6a90992:::obj552:head) v3 prio 15 cost 0 latency 0.000094 pg_backfill(progress 4.0 e 119/119 lb 4:b6a90992:::obj552:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b66eab34:::obj112:head local-lis/les=117/118 n=438 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.223+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.223+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b66eab34:::obj112:head local-lis/les=117/118 n=438 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:b6a90992:::obj552:head) v3 2026-02-11T23:31:28.223+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b66eab34:::obj112:head local-lis/les=117/118 n=438 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:b6a90992:::obj552:head) 2026-02-11T23:31:28.223+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.223+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b6a90992:::obj552:head) v3 finish latency 0.000014 2026-02-11T23:31:28.223+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b6a90992:::obj552:head local-lis/les=117/118 n=439 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.223+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.223+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.223+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b6a90992:::obj552:head local-lis/les=117/118 n=439 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.223+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.223+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.223+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.223+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c9e00 con 0x507ac00 2026-02-11T23:31:28.223+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:b6dfbaf7:::obj426:head)]) -- 0x671dc80 con 0x507ac00 2026-02-11T23:31:28.223+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.224+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 919 ==== MOSDPGPush(4.0 119/117 [PushOp(4:b70eecc2:::obj59:head, version: 111'59, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b70eecc2:::obj59:head@111'59, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x55f09c0 con 0x507ac00 2026-02-11T23:31:28.224+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:b70eecc2:::obj59:head, version: 111'59, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b70eecc2:::obj59:head@111'59, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.224+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b70eecc2:::obj59:head, version: 111'59, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b70eecc2:::obj59:head@111'59, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.224+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.224+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 920 ==== pg_backfill(progress 4.0 e 119/119 lb 4:b6dfbaf7:::obj426:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177600 con 0x507ac00 2026-02-11T23:31:28.224+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b70eecc2:::obj59:head, version: 111'59, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b70eecc2:::obj59:head@111'59, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.224+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:b6dfbaf7:::obj426:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.224+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b6dfbaf7:::obj426:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.224+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.224+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b70eecc2:::obj59:head, version: 111'59, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b70eecc2:::obj59:head@111'59, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.224+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b70eecc2:::obj59:head, version: 111'59, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b70eecc2:::obj59:head@111'59, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 119/117 [PushOp(4:b70eecc2:::obj59:head, version: 111'59, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b70eecc2:::obj59:head@111'59, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b6a90992:::obj552:head local-lis/les=117/118 n=439 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.224+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.224+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.224+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b6dfbaf7:::obj426:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.224+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b6a90992:::obj552:head local-lis/les=117/118 n=439 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:b70eecc2:::obj59:head, version: 111'59, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b70eecc2:::obj59:head@111'59, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.224+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b6a90992:::obj552:head local-lis/les=117/118 n=439 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b70eecc2:::obj59:head@111'59, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.224+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b6a90992:::obj552:head local-lis/les=117/118 n=439 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:b70eecc2:::obj59:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.224+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b6a90992:::obj552:head local-lis/les=117/118 n=439 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:b70eecc2:::obj59:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.224+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b6a90992:::obj552:head local-lis/les=117/118 n=439 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:b70eecc2:::obj59:head 2026-02-11T23:31:28.224+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b6a90992:::obj552:head local-lis/les=117/118 n=439 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:b70eecc2:::obj59:head v 111'59 2026-02-11T23:31:28.224+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b6a90992:::obj552:head local-lis/les=117/118 n=439 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.224+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.224+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b70eecc2:::obj59:head, version: 111'59, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b70eecc2:::obj59:head@111'59, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000070 2026-02-11T23:31:28.224+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.224+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.224+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b6dfbaf7:::obj426:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.224+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b6dfbaf7:::obj426:head) v3 prio 15 cost 0 latency 0.000087 pg_backfill(progress 4.0 e 119/119 lb 4:b6dfbaf7:::obj426:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b6a90992:::obj552:head local-lis/les=117/118 n=439 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.224+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.224+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b6a90992:::obj552:head local-lis/les=117/118 n=439 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:b6dfbaf7:::obj426:head) v3 2026-02-11T23:31:28.224+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b6a90992:::obj552:head local-lis/les=117/118 n=439 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:b6dfbaf7:::obj426:head) 2026-02-11T23:31:28.224+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.224+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b6dfbaf7:::obj426:head) v3 finish latency 0.000013 2026-02-11T23:31:28.224+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b6dfbaf7:::obj426:head local-lis/les=117/118 n=440 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.224+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.224+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.224+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b6dfbaf7:::obj426:head local-lis/les=117/118 n=440 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.224+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.224+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.224+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.224+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x640fb00 con 0x507ac00 2026-02-11T23:31:28.224+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:b70eecc2:::obj59:head)]) -- 0x671d080 con 0x507ac00 2026-02-11T23:31:28.224+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.224+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 921 ==== MOSDPGPush(4.0 119/117 [PushOp(4:b75b9722:::obj495:head, version: 111'495, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b75b9722:::obj495:head@111'495, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56ca680 con 0x507ac00 2026-02-11T23:31:28.224+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:b75b9722:::obj495:head, version: 111'495, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b75b9722:::obj495:head@111'495, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.224+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b75b9722:::obj495:head, version: 111'495, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b75b9722:::obj495:head@111'495, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.224+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.224+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 922 ==== pg_backfill(progress 4.0 e 119/119 lb 4:b70eecc2:::obj59:head) ==== 1061+0+0 (unknown 0 0 0) 0x6177b80 con 0x507ac00 2026-02-11T23:31:28.224+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b75b9722:::obj495:head, version: 111'495, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b75b9722:::obj495:head@111'495, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.224+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:b70eecc2:::obj59:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.224+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b70eecc2:::obj59:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.224+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.224+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b75b9722:::obj495:head, version: 111'495, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b75b9722:::obj495:head@111'495, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.224+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b75b9722:::obj495:head, version: 111'495, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b75b9722:::obj495:head@111'495, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 119/117 [PushOp(4:b75b9722:::obj495:head, version: 111'495, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b75b9722:::obj495:head@111'495, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b6dfbaf7:::obj426:head local-lis/les=117/118 n=440 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.224+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.224+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b70eecc2:::obj59:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.224+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.224+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b6dfbaf7:::obj426:head local-lis/les=117/118 n=440 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:b75b9722:::obj495:head, version: 111'495, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b75b9722:::obj495:head@111'495, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.224+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b6dfbaf7:::obj426:head local-lis/les=117/118 n=440 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b75b9722:::obj495:head@111'495, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.224+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b6dfbaf7:::obj426:head local-lis/les=117/118 n=440 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:b75b9722:::obj495:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.224+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b6dfbaf7:::obj426:head local-lis/les=117/118 n=440 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:b75b9722:::obj495:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.224+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b6dfbaf7:::obj426:head local-lis/les=117/118 n=440 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:b75b9722:::obj495:head 2026-02-11T23:31:28.224+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b6dfbaf7:::obj426:head local-lis/les=117/118 n=440 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:b75b9722:::obj495:head v 111'495 2026-02-11T23:31:28.224+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b6dfbaf7:::obj426:head local-lis/les=117/118 n=440 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.224+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.224+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b75b9722:::obj495:head, version: 111'495, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b75b9722:::obj495:head@111'495, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000070 2026-02-11T23:31:28.224+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.224+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.224+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b70eecc2:::obj59:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.224+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b70eecc2:::obj59:head) v3 prio 15 cost 0 latency 0.000087 pg_backfill(progress 4.0 e 119/119 lb 4:b70eecc2:::obj59:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b6dfbaf7:::obj426:head local-lis/les=117/118 n=440 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.224+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.224+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b6dfbaf7:::obj426:head local-lis/les=117/118 n=440 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:b70eecc2:::obj59:head) v3 2026-02-11T23:31:28.224+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b6dfbaf7:::obj426:head local-lis/les=117/118 n=440 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:b70eecc2:::obj59:head) 2026-02-11T23:31:28.224+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.224+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b70eecc2:::obj59:head) v3 finish latency 0.000014 2026-02-11T23:31:28.224+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b70eecc2:::obj59:head local-lis/les=117/118 n=441 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.224+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.224+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.224+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b70eecc2:::obj59:head local-lis/les=117/118 n=441 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.224+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.224+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.224+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.224+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x5d55200 con 0x507ac00 2026-02-11T23:31:28.224+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:b75b9722:::obj495:head)]) -- 0x66d9500 con 0x507ac00 2026-02-11T23:31:28.224+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.225+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 923 ==== MOSDPGPush(4.0 119/117 [PushOp(4:b841f58a:::obj361:head, version: 111'361, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b841f58a:::obj361:head@111'361, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5646d00 con 0x507ac00 2026-02-11T23:31:28.225+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:b841f58a:::obj361:head, version: 111'361, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b841f58a:::obj361:head@111'361, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.225+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b841f58a:::obj361:head, version: 111'361, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b841f58a:::obj361:head@111'361, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.225+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.225+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 924 ==== pg_backfill(progress 4.0 e 119/119 lb 4:b75b9722:::obj495:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2c580 con 0x507ac00 2026-02-11T23:31:28.225+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:b75b9722:::obj495:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 119 2026-02-11T23:31:28.225+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b841f58a:::obj361:head, version: 111'361, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b841f58a:::obj361:head@111'361, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.225+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b75b9722:::obj495:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.225+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.225+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b841f58a:::obj361:head, version: 111'361, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b841f58a:::obj361:head@111'361, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.225+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.225+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b75b9722:::obj495:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.225+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b841f58a:::obj361:head, version: 111'361, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b841f58a:::obj361:head@111'361, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 119/117 [PushOp(4:b841f58a:::obj361:head, version: 111'361, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b841f58a:::obj361:head@111'361, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b70eecc2:::obj59:head local-lis/les=117/118 n=441 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.225+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.225+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b70eecc2:::obj59:head local-lis/les=117/118 n=441 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:b841f58a:::obj361:head, version: 111'361, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b841f58a:::obj361:head@111'361, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.225+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b70eecc2:::obj59:head local-lis/les=117/118 n=441 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b841f58a:::obj361:head@111'361, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.225+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b70eecc2:::obj59:head local-lis/les=117/118 n=441 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:b841f58a:::obj361:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.225+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b70eecc2:::obj59:head local-lis/les=117/118 n=441 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:b841f58a:::obj361:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.225+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b70eecc2:::obj59:head local-lis/les=117/118 n=441 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:b841f58a:::obj361:head 2026-02-11T23:31:28.225+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b70eecc2:::obj59:head local-lis/les=117/118 n=441 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:b841f58a:::obj361:head v 111'361 2026-02-11T23:31:28.225+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b70eecc2:::obj59:head local-lis/les=117/118 n=441 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.225+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.225+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b841f58a:::obj361:head, version: 111'361, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b841f58a:::obj361:head@111'361, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000067 2026-02-11T23:31:28.225+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.225+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.225+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b75b9722:::obj495:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.225+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b75b9722:::obj495:head) v3 prio 15 cost 0 latency 0.000088 pg_backfill(progress 4.0 e 119/119 lb 4:b75b9722:::obj495:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b70eecc2:::obj59:head local-lis/les=117/118 n=441 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.225+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.225+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b70eecc2:::obj59:head local-lis/les=117/118 n=441 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:b75b9722:::obj495:head) v3 2026-02-11T23:31:28.225+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b70eecc2:::obj59:head local-lis/les=117/118 n=441 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:b75b9722:::obj495:head) 2026-02-11T23:31:28.225+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.225+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b75b9722:::obj495:head) v3 finish latency 0.000015 2026-02-11T23:31:28.225+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b75b9722:::obj495:head local-lis/les=117/118 n=442 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.225+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.225+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.225+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b75b9722:::obj495:head local-lis/les=117/118 n=442 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.225+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.225+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.225+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.225+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x642de00 con 0x507ac00 2026-02-11T23:31:28.225+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:b841f58a:::obj361:head)]) -- 0x66d9200 con 0x507ac00 2026-02-11T23:31:28.225+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.225+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 925 ==== MOSDPGPush(4.0 119/117 [PushOp(4:b8510467:::obj40:head, version: 111'40, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b8510467:::obj40:head@111'40, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x554e340 con 0x507ac00 2026-02-11T23:31:28.225+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:b8510467:::obj40:head, version: 111'40, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b8510467:::obj40:head@111'40, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.225+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b8510467:::obj40:head, version: 111'40, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b8510467:::obj40:head@111'40, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.225+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.225+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 926 ==== pg_backfill(progress 4.0 e 119/119 lb 4:b841f58a:::obj361:head) ==== 1062+0+0 (unknown 0 0 0) 0x6178100 con 0x507ac00 2026-02-11T23:31:28.225+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:b841f58a:::obj361:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.225+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b8510467:::obj40:head, version: 111'40, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b8510467:::obj40:head@111'40, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.225+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b841f58a:::obj361:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.225+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.225+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b8510467:::obj40:head, version: 111'40, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b8510467:::obj40:head@111'40, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.225+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b8510467:::obj40:head, version: 111'40, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b8510467:::obj40:head@111'40, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 119/117 [PushOp(4:b8510467:::obj40:head, version: 111'40, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b8510467:::obj40:head@111'40, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b75b9722:::obj495:head local-lis/les=117/118 n=442 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.225+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.225+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.225+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b841f58a:::obj361:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.225+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b75b9722:::obj495:head local-lis/les=117/118 n=442 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:b8510467:::obj40:head, version: 111'40, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b8510467:::obj40:head@111'40, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.225+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b75b9722:::obj495:head local-lis/les=117/118 n=442 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b8510467:::obj40:head@111'40, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.225+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b75b9722:::obj495:head local-lis/les=117/118 n=442 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:b8510467:::obj40:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.225+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b75b9722:::obj495:head local-lis/les=117/118 n=442 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:b8510467:::obj40:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.225+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b75b9722:::obj495:head local-lis/les=117/118 n=442 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:b8510467:::obj40:head 2026-02-11T23:31:28.225+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b75b9722:::obj495:head local-lis/les=117/118 n=442 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:b8510467:::obj40:head v 111'40 2026-02-11T23:31:28.225+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b75b9722:::obj495:head local-lis/les=117/118 n=442 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.225+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.225+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b8510467:::obj40:head, version: 111'40, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b8510467:::obj40:head@111'40, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000070 2026-02-11T23:31:28.226+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.226+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.226+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b841f58a:::obj361:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.226+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b841f58a:::obj361:head) v3 prio 15 cost 0 latency 0.000088 pg_backfill(progress 4.0 e 119/119 lb 4:b841f58a:::obj361:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b75b9722:::obj495:head local-lis/les=117/118 n=442 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.226+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.226+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b75b9722:::obj495:head local-lis/les=117/118 n=442 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:b841f58a:::obj361:head) v3 2026-02-11T23:31:28.226+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b75b9722:::obj495:head local-lis/les=117/118 n=442 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:b841f58a:::obj361:head) 2026-02-11T23:31:28.226+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.226+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b841f58a:::obj361:head) v3 finish latency 0.000014 2026-02-11T23:31:28.226+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b841f58a:::obj361:head local-lis/les=117/118 n=443 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.226+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.226+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.226+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b841f58a:::obj361:head local-lis/les=117/118 n=443 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.226+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.226+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.226+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.226+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x642d500 con 0x507ac00 2026-02-11T23:31:28.226+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:b8510467:::obj40:head)]) -- 0x66d8480 con 0x507ac00 2026-02-11T23:31:28.226+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.226+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 927 ==== MOSDPGPush(4.0 119/117 [PushOp(4:b8f63b78:::obj291:head, version: 111'291, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b8f63b78:::obj291:head@111'291, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a29c0 con 0x507ac00 2026-02-11T23:31:28.226+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:b8f63b78:::obj291:head, version: 111'291, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b8f63b78:::obj291:head@111'291, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.226+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b8f63b78:::obj291:head, version: 111'291, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b8f63b78:::obj291:head@111'291, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.226+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.226+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 928 ==== pg_backfill(progress 4.0 e 119/119 lb 4:b8510467:::obj40:head) ==== 1061+0+0 (unknown 0 0 0) 0x6179180 con 0x507ac00 2026-02-11T23:31:28.226+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:b8510467:::obj40:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.226+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b8f63b78:::obj291:head, version: 111'291, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b8f63b78:::obj291:head@111'291, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.226+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b8510467:::obj40:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.226+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.226+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b8f63b78:::obj291:head, version: 111'291, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b8f63b78:::obj291:head@111'291, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.226+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b8f63b78:::obj291:head, version: 111'291, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b8f63b78:::obj291:head@111'291, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 119/117 [PushOp(4:b8f63b78:::obj291:head, version: 111'291, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b8f63b78:::obj291:head@111'291, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b841f58a:::obj361:head local-lis/les=117/118 n=443 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.226+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.226+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.226+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b8510467:::obj40:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.226+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b841f58a:::obj361:head local-lis/les=117/118 n=443 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:b8f63b78:::obj291:head, version: 111'291, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b8f63b78:::obj291:head@111'291, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.226+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b841f58a:::obj361:head local-lis/les=117/118 n=443 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b8f63b78:::obj291:head@111'291, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.226+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b841f58a:::obj361:head local-lis/les=117/118 n=443 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:b8f63b78:::obj291:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.226+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b841f58a:::obj361:head local-lis/les=117/118 n=443 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:b8f63b78:::obj291:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.226+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b841f58a:::obj361:head local-lis/les=117/118 n=443 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:b8f63b78:::obj291:head 2026-02-11T23:31:28.226+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b841f58a:::obj361:head local-lis/les=117/118 n=443 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:b8f63b78:::obj291:head v 111'291 2026-02-11T23:31:28.226+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b841f58a:::obj361:head local-lis/les=117/118 n=443 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.226+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.226+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b8f63b78:::obj291:head, version: 111'291, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b8f63b78:::obj291:head@111'291, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000068 2026-02-11T23:31:28.226+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.226+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.226+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b8510467:::obj40:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.226+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b8510467:::obj40:head) v3 prio 15 cost 0 latency 0.000085 pg_backfill(progress 4.0 e 119/119 lb 4:b8510467:::obj40:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b841f58a:::obj361:head local-lis/les=117/118 n=443 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.226+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.226+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b841f58a:::obj361:head local-lis/les=117/118 n=443 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:b8510467:::obj40:head) v3 2026-02-11T23:31:28.226+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b841f58a:::obj361:head local-lis/les=117/118 n=443 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:b8510467:::obj40:head) 2026-02-11T23:31:28.226+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.226+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b8510467:::obj40:head) v3 finish latency 0.000015 2026-02-11T23:31:28.226+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b8510467:::obj40:head local-lis/les=117/118 n=444 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.226+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.226+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.226+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b8510467:::obj40:head local-lis/les=117/118 n=444 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.226+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.226+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.226+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.226+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x642d800 con 0x507ac00 2026-02-11T23:31:28.226+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:b8f63b78:::obj291:head)]) -- 0x671c780 con 0x507ac00 2026-02-11T23:31:28.226+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.227+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 929 ==== MOSDPGPush(4.0 119/117 [PushOp(4:b92d6f18:::obj505:head, version: 111'505, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b92d6f18:::obj505:head@111'505, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e26b60 con 0x507ac00 2026-02-11T23:31:28.227+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:b92d6f18:::obj505:head, version: 111'505, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b92d6f18:::obj505:head@111'505, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.227+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b92d6f18:::obj505:head, version: 111'505, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b92d6f18:::obj505:head@111'505, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.227+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.227+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b92d6f18:::obj505:head, version: 111'505, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b92d6f18:::obj505:head@111'505, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.227+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 930 ==== pg_backfill(progress 4.0 e 119/119 lb 4:b8f63b78:::obj291:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9b180 con 0x507ac00 2026-02-11T23:31:28.227+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:b8f63b78:::obj291:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.227+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.227+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b8f63b78:::obj291:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.227+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b92d6f18:::obj505:head, version: 111'505, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b92d6f18:::obj505:head@111'505, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.227+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.227+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b92d6f18:::obj505:head, version: 111'505, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b92d6f18:::obj505:head@111'505, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 119/117 [PushOp(4:b92d6f18:::obj505:head, version: 111'505, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b92d6f18:::obj505:head@111'505, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b8510467:::obj40:head local-lis/les=117/118 n=444 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.227+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.227+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b8510467:::obj40:head local-lis/les=117/118 n=444 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:b92d6f18:::obj505:head, version: 111'505, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b92d6f18:::obj505:head@111'505, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.227+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.227+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b8f63b78:::obj291:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.227+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b8510467:::obj40:head local-lis/les=117/118 n=444 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b92d6f18:::obj505:head@111'505, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.227+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b8510467:::obj40:head local-lis/les=117/118 n=444 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:b92d6f18:::obj505:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.227+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b8510467:::obj40:head local-lis/les=117/118 n=444 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:b92d6f18:::obj505:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.227+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b8510467:::obj40:head local-lis/les=117/118 n=444 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:b92d6f18:::obj505:head 2026-02-11T23:31:28.227+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b8510467:::obj40:head local-lis/les=117/118 n=444 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:b92d6f18:::obj505:head v 111'505 2026-02-11T23:31:28.227+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b8510467:::obj40:head local-lis/les=117/118 n=444 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.227+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.227+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b92d6f18:::obj505:head, version: 111'505, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b92d6f18:::obj505:head@111'505, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000067 2026-02-11T23:31:28.227+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.227+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.227+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b8f63b78:::obj291:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.227+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b8f63b78:::obj291:head) v3 prio 15 cost 0 latency 0.000082 pg_backfill(progress 4.0 e 119/119 lb 4:b8f63b78:::obj291:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b8510467:::obj40:head local-lis/les=117/118 n=444 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.227+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.227+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b8510467:::obj40:head local-lis/les=117/118 n=444 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:b8f63b78:::obj291:head) v3 2026-02-11T23:31:28.227+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b8510467:::obj40:head local-lis/les=117/118 n=444 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:b8f63b78:::obj291:head) 2026-02-11T23:31:28.227+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.227+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b8f63b78:::obj291:head) v3 finish latency 0.000014 2026-02-11T23:31:28.227+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b8f63b78:::obj291:head local-lis/les=117/118 n=445 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.227+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.227+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.227+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b8f63b78:::obj291:head local-lis/les=117/118 n=445 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.227+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.227+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.227+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.227+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x640e180 con 0x507ac00 2026-02-11T23:31:28.227+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:b92d6f18:::obj505:head)]) -- 0x671c480 con 0x507ac00 2026-02-11T23:31:28.227+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.227+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 931 ==== MOSDPGPush(4.0 119/117 [PushOp(4:b9836a99:::obj18:head, version: 111'18, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b9836a99:::obj18:head@111'18, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x3815380 con 0x507ac00 2026-02-11T23:31:28.227+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:b9836a99:::obj18:head, version: 111'18, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b9836a99:::obj18:head@111'18, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.227+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b9836a99:::obj18:head, version: 111'18, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b9836a99:::obj18:head@111'18, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.227+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.227+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 932 ==== pg_backfill(progress 4.0 e 119/119 lb 4:b92d6f18:::obj505:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98000 con 0x507ac00 2026-02-11T23:31:28.227+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b9836a99:::obj18:head, version: 111'18, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b9836a99:::obj18:head@111'18, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.227+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:b92d6f18:::obj505:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.227+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b92d6f18:::obj505:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.227+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.227+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:b9836a99:::obj18:head, version: 111'18, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b9836a99:::obj18:head@111'18, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.227+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b9836a99:::obj18:head, version: 111'18, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b9836a99:::obj18:head@111'18, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 119/117 [PushOp(4:b9836a99:::obj18:head, version: 111'18, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b9836a99:::obj18:head@111'18, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b8f63b78:::obj291:head local-lis/les=117/118 n=445 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.227+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.227+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.227+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b92d6f18:::obj505:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.227+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b8f63b78:::obj291:head local-lis/les=117/118 n=445 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:b9836a99:::obj18:head, version: 111'18, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b9836a99:::obj18:head@111'18, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.227+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b8f63b78:::obj291:head local-lis/les=117/118 n=445 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b9836a99:::obj18:head@111'18, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.227+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b8f63b78:::obj291:head local-lis/les=117/118 n=445 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:b9836a99:::obj18:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.227+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b8f63b78:::obj291:head local-lis/les=117/118 n=445 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:b9836a99:::obj18:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.227+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b8f63b78:::obj291:head local-lis/les=117/118 n=445 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:b9836a99:::obj18:head 2026-02-11T23:31:28.227+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b8f63b78:::obj291:head local-lis/les=117/118 n=445 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:b9836a99:::obj18:head v 111'18 2026-02-11T23:31:28.227+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b8f63b78:::obj291:head local-lis/les=117/118 n=445 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.227+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.227+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:b9836a99:::obj18:head, version: 111'18, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b9836a99:::obj18:head@111'18, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000066 2026-02-11T23:31:28.227+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.227+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.227+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b92d6f18:::obj505:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.227+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b92d6f18:::obj505:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 4.0 e 119/119 lb 4:b92d6f18:::obj505:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b8f63b78:::obj291:head local-lis/les=117/118 n=445 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.227+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.227+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b8f63b78:::obj291:head local-lis/les=117/118 n=445 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:b92d6f18:::obj505:head) v3 2026-02-11T23:31:28.227+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b8f63b78:::obj291:head local-lis/les=117/118 n=445 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:b92d6f18:::obj505:head) 2026-02-11T23:31:28.227+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.227+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b92d6f18:::obj505:head) v3 finish latency 0.000014 2026-02-11T23:31:28.227+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b92d6f18:::obj505:head local-lis/les=117/118 n=446 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.227+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.227+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.227+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b92d6f18:::obj505:head local-lis/les=117/118 n=446 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.227+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.227+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.227+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.227+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x642db00 con 0x507ac00 2026-02-11T23:31:28.227+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.227+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:b9836a99:::obj18:head)]) -- 0x671c180 con 0x507ac00 2026-02-11T23:31:28.227+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.228+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 933 ==== MOSDPGPush(4.0 119/117 [PushOp(4:ba083999:::obj295:head, version: 111'295, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba083999:::obj295:head@111'295, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2680 con 0x507ac00 2026-02-11T23:31:28.228+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:ba083999:::obj295:head, version: 111'295, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba083999:::obj295:head@111'295, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.228+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ba083999:::obj295:head, version: 111'295, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba083999:::obj295:head@111'295, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.228+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.228+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 934 ==== pg_backfill(progress 4.0 e 119/119 lb 4:b9836a99:::obj18:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d2ec00 con 0x507ac00 2026-02-11T23:31:28.228+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ba083999:::obj295:head, version: 111'295, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba083999:::obj295:head@111'295, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.228+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:b9836a99:::obj18:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.228+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.228+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b9836a99:::obj18:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.228+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ba083999:::obj295:head, version: 111'295, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba083999:::obj295:head@111'295, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.228+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.228+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:ba083999:::obj295:head, version: 111'295, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba083999:::obj295:head@111'295, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 119/117 [PushOp(4:ba083999:::obj295:head, version: 111'295, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba083999:::obj295:head@111'295, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b92d6f18:::obj505:head local-lis/les=117/118 n=446 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.228+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.228+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.228+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b92d6f18:::obj505:head local-lis/les=117/118 n=446 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:ba083999:::obj295:head, version: 111'295, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba083999:::obj295:head@111'295, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.228+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b9836a99:::obj18:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.228+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b92d6f18:::obj505:head local-lis/les=117/118 n=446 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:ba083999:::obj295:head@111'295, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.228+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b92d6f18:::obj505:head local-lis/les=117/118 n=446 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:ba083999:::obj295:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.228+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b92d6f18:::obj505:head local-lis/les=117/118 n=446 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:ba083999:::obj295:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.228+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b92d6f18:::obj505:head local-lis/les=117/118 n=446 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:ba083999:::obj295:head 2026-02-11T23:31:28.228+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b92d6f18:::obj505:head local-lis/les=117/118 n=446 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:ba083999:::obj295:head v 111'295 2026-02-11T23:31:28.228+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b92d6f18:::obj505:head local-lis/les=117/118 n=446 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.228+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.228+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:ba083999:::obj295:head, version: 111'295, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba083999:::obj295:head@111'295, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000067 2026-02-11T23:31:28.228+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.228+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.228+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:b9836a99:::obj18:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.228+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b9836a99:::obj18:head) v3 prio 15 cost 0 latency 0.000084 pg_backfill(progress 4.0 e 119/119 lb 4:b9836a99:::obj18:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b92d6f18:::obj505:head local-lis/les=117/118 n=446 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.228+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.228+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b92d6f18:::obj505:head local-lis/les=117/118 n=446 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:b9836a99:::obj18:head) v3 2026-02-11T23:31:28.228+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b92d6f18:::obj505:head local-lis/les=117/118 n=446 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:b9836a99:::obj18:head) 2026-02-11T23:31:28.228+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.228+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:b9836a99:::obj18:head) v3 finish latency 0.000016 2026-02-11T23:31:28.228+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b9836a99:::obj18:head local-lis/les=117/118 n=447 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.228+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.228+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.228+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b9836a99:::obj18:head local-lis/les=117/118 n=447 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.228+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.228+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.228+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.228+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7b200 con 0x507ac00 2026-02-11T23:31:28.228+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:ba083999:::obj295:head)]) -- 0x66fa480 con 0x507ac00 2026-02-11T23:31:28.228+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.228+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 935 ==== MOSDPGPush(4.0 119/117 [PushOp(4:ba3d236d:::obj529:head, version: 111'529, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba3d236d:::obj529:head@111'529, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56cab60 con 0x507ac00 2026-02-11T23:31:28.228+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:ba3d236d:::obj529:head, version: 111'529, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba3d236d:::obj529:head@111'529, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.228+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ba3d236d:::obj529:head, version: 111'529, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba3d236d:::obj529:head@111'529, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.228+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.228+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 936 ==== pg_backfill(progress 4.0 e 119/119 lb 4:ba083999:::obj295:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2d600 con 0x507ac00 2026-02-11T23:31:28.228+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ba3d236d:::obj529:head, version: 111'529, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba3d236d:::obj529:head@111'529, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.228+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:ba083999:::obj295:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.228+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:ba083999:::obj295:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.228+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.228+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ba3d236d:::obj529:head, version: 111'529, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba3d236d:::obj529:head@111'529, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.228+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:ba3d236d:::obj529:head, version: 111'529, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba3d236d:::obj529:head@111'529, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 119/117 [PushOp(4:ba3d236d:::obj529:head, version: 111'529, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba3d236d:::obj529:head@111'529, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b9836a99:::obj18:head local-lis/les=117/118 n=447 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.228+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.228+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.228+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:ba083999:::obj295:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.228+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b9836a99:::obj18:head local-lis/les=117/118 n=447 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:ba3d236d:::obj529:head, version: 111'529, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba3d236d:::obj529:head@111'529, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.228+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b9836a99:::obj18:head local-lis/les=117/118 n=447 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:ba3d236d:::obj529:head@111'529, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.228+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b9836a99:::obj18:head local-lis/les=117/118 n=447 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:ba3d236d:::obj529:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.228+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b9836a99:::obj18:head local-lis/les=117/118 n=447 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:ba3d236d:::obj529:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.228+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b9836a99:::obj18:head local-lis/les=117/118 n=447 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:ba3d236d:::obj529:head 2026-02-11T23:31:28.228+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b9836a99:::obj18:head local-lis/les=117/118 n=447 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:ba3d236d:::obj529:head v 111'529 2026-02-11T23:31:28.228+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b9836a99:::obj18:head local-lis/les=117/118 n=447 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.228+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.228+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:ba3d236d:::obj529:head, version: 111'529, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba3d236d:::obj529:head@111'529, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000074 2026-02-11T23:31:28.228+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.228+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.228+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:ba083999:::obj295:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.228+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:ba083999:::obj295:head) v3 prio 15 cost 0 latency 0.000092 pg_backfill(progress 4.0 e 119/119 lb 4:ba083999:::obj295:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:b9836a99:::obj18:head local-lis/les=117/118 n=447 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.228+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.228+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b9836a99:::obj18:head local-lis/les=117/118 n=447 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:ba083999:::obj295:head) v3 2026-02-11T23:31:28.228+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:b9836a99:::obj18:head local-lis/les=117/118 n=447 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:ba083999:::obj295:head) 2026-02-11T23:31:28.228+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.228+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:ba083999:::obj295:head) v3 finish latency 0.000014 2026-02-11T23:31:28.228+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ba083999:::obj295:head local-lis/les=117/118 n=448 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.228+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.229+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.229+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ba083999:::obj295:head local-lis/les=117/118 n=448 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.229+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.229+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.229+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.229+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7a300 con 0x507ac00 2026-02-11T23:31:28.229+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:ba3d236d:::obj529:head)]) -- 0x66fb800 con 0x507ac00 2026-02-11T23:31:28.229+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.229+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 937 ==== MOSDPGPush(4.0 119/117 [PushOp(4:ba98db39:::obj542:head, version: 111'542, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba98db39:::obj542:head@111'542, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a3860 con 0x507ac00 2026-02-11T23:31:28.229+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:ba98db39:::obj542:head, version: 111'542, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba98db39:::obj542:head@111'542, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.229+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ba98db39:::obj542:head, version: 111'542, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba98db39:::obj542:head@111'542, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.229+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.229+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 938 ==== pg_backfill(progress 4.0 e 119/119 lb 4:ba3d236d:::obj529:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99600 con 0x507ac00 2026-02-11T23:31:28.229+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ba98db39:::obj542:head, version: 111'542, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba98db39:::obj542:head@111'542, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.229+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:ba3d236d:::obj529:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.229+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:ba3d236d:::obj529:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.229+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.229+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ba98db39:::obj542:head, version: 111'542, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba98db39:::obj542:head@111'542, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.229+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:ba98db39:::obj542:head, version: 111'542, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba98db39:::obj542:head@111'542, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 119/117 [PushOp(4:ba98db39:::obj542:head, version: 111'542, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba98db39:::obj542:head@111'542, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:ba083999:::obj295:head local-lis/les=117/118 n=448 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.229+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.229+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.229+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:ba3d236d:::obj529:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.229+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ba083999:::obj295:head local-lis/les=117/118 n=448 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:ba98db39:::obj542:head, version: 111'542, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba98db39:::obj542:head@111'542, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.229+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ba083999:::obj295:head local-lis/les=117/118 n=448 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:ba98db39:::obj542:head@111'542, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.229+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ba083999:::obj295:head local-lis/les=117/118 n=448 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:ba98db39:::obj542:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.229+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ba083999:::obj295:head local-lis/les=117/118 n=448 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:ba98db39:::obj542:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.229+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ba083999:::obj295:head local-lis/les=117/118 n=448 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:ba98db39:::obj542:head 2026-02-11T23:31:28.229+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ba083999:::obj295:head local-lis/les=117/118 n=448 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:ba98db39:::obj542:head v 111'542 2026-02-11T23:31:28.229+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ba083999:::obj295:head local-lis/les=117/118 n=448 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.229+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.229+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:ba98db39:::obj542:head, version: 111'542, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba98db39:::obj542:head@111'542, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000067 2026-02-11T23:31:28.229+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.229+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.229+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:ba3d236d:::obj529:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.229+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:ba3d236d:::obj529:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 4.0 e 119/119 lb 4:ba3d236d:::obj529:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:ba083999:::obj295:head local-lis/les=117/118 n=448 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.229+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.229+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ba083999:::obj295:head local-lis/les=117/118 n=448 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:ba3d236d:::obj529:head) v3 2026-02-11T23:31:28.229+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ba083999:::obj295:head local-lis/les=117/118 n=448 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:ba3d236d:::obj529:head) 2026-02-11T23:31:28.229+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.229+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:ba3d236d:::obj529:head) v3 finish latency 0.000014 2026-02-11T23:31:28.229+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ba3d236d:::obj529:head local-lis/les=117/118 n=449 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.229+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.229+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.229+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ba3d236d:::obj529:head local-lis/les=117/118 n=449 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.229+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.229+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.229+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.229+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x5f2dc80 con 0x507ac00 2026-02-11T23:31:28.229+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:ba98db39:::obj542:head)]) -- 0x66fbc80 con 0x507ac00 2026-02-11T23:31:28.229+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.229+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 939 ==== MOSDPGPush(4.0 119/117 [PushOp(4:bb038e88:::obj54:head, version: 111'54, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bb038e88:::obj54:head@111'54, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x55f1520 con 0x507ac00 2026-02-11T23:31:28.229+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:bb038e88:::obj54:head, version: 111'54, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bb038e88:::obj54:head@111'54, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.229+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:bb038e88:::obj54:head, version: 111'54, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bb038e88:::obj54:head@111'54, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.229+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.229+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 940 ==== pg_backfill(progress 4.0 e 119/119 lb 4:ba98db39:::obj542:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9ac00 con 0x507ac00 2026-02-11T23:31:28.229+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:ba98db39:::obj542:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.229+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:bb038e88:::obj54:head, version: 111'54, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bb038e88:::obj54:head@111'54, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.229+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:ba98db39:::obj542:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.229+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.229+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:bb038e88:::obj54:head, version: 111'54, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bb038e88:::obj54:head@111'54, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.229+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:bb038e88:::obj54:head, version: 111'54, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bb038e88:::obj54:head@111'54, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 119/117 [PushOp(4:bb038e88:::obj54:head, version: 111'54, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bb038e88:::obj54:head@111'54, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:ba3d236d:::obj529:head local-lis/les=117/118 n=449 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.229+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.229+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.229+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:ba98db39:::obj542:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.229+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ba3d236d:::obj529:head local-lis/les=117/118 n=449 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:bb038e88:::obj54:head, version: 111'54, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bb038e88:::obj54:head@111'54, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.229+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ba3d236d:::obj529:head local-lis/les=117/118 n=449 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:bb038e88:::obj54:head@111'54, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.229+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ba3d236d:::obj529:head local-lis/les=117/118 n=449 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:bb038e88:::obj54:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.229+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ba3d236d:::obj529:head local-lis/les=117/118 n=449 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:bb038e88:::obj54:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.229+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ba3d236d:::obj529:head local-lis/les=117/118 n=449 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:bb038e88:::obj54:head 2026-02-11T23:31:28.230+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ba3d236d:::obj529:head local-lis/les=117/118 n=449 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:bb038e88:::obj54:head v 111'54 2026-02-11T23:31:28.230+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ba3d236d:::obj529:head local-lis/les=117/118 n=449 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.230+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.230+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:bb038e88:::obj54:head, version: 111'54, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bb038e88:::obj54:head@111'54, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000079 2026-02-11T23:31:28.230+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.230+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.230+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:ba98db39:::obj542:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.230+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:ba98db39:::obj542:head) v3 prio 15 cost 0 latency 0.000096 pg_backfill(progress 4.0 e 119/119 lb 4:ba98db39:::obj542:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:ba3d236d:::obj529:head local-lis/les=117/118 n=449 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.230+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.230+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ba3d236d:::obj529:head local-lis/les=117/118 n=449 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:ba98db39:::obj542:head) v3 2026-02-11T23:31:28.230+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ba3d236d:::obj529:head local-lis/les=117/118 n=449 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:ba98db39:::obj542:head) 2026-02-11T23:31:28.230+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.230+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:ba98db39:::obj542:head) v3 finish latency 0.000016 2026-02-11T23:31:28.230+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ba98db39:::obj542:head local-lis/les=117/118 n=450 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.230+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.230+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.230+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ba98db39:::obj542:head local-lis/les=117/118 n=450 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.230+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.230+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.230+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.230+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7aa80 con 0x507ac00 2026-02-11T23:31:28.230+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:bb038e88:::obj54:head)]) -- 0x66fbb00 con 0x507ac00 2026-02-11T23:31:28.230+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.230+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 941 ==== MOSDPGPush(4.0 119/117 [PushOp(4:bbf67374:::obj187:head, version: 111'187, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bbf67374:::obj187:head@111'187, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f0b60 con 0x507ac00 2026-02-11T23:31:28.230+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:bbf67374:::obj187:head, version: 111'187, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bbf67374:::obj187:head@111'187, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.230+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:bbf67374:::obj187:head, version: 111'187, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bbf67374:::obj187:head@111'187, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.230+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.230+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 942 ==== pg_backfill(progress 4.0 e 119/119 lb 4:bb038e88:::obj54:head) ==== 1061+0+0 (unknown 0 0 0) 0x4a2cb00 con 0x507ac00 2026-02-11T23:31:28.230+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:bb038e88:::obj54:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.230+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:bbf67374:::obj187:head, version: 111'187, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bbf67374:::obj187:head@111'187, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.230+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:bb038e88:::obj54:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.230+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.230+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:bbf67374:::obj187:head, version: 111'187, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bbf67374:::obj187:head@111'187, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.230+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:bbf67374:::obj187:head, version: 111'187, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bbf67374:::obj187:head@111'187, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 119/117 [PushOp(4:bbf67374:::obj187:head, version: 111'187, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bbf67374:::obj187:head@111'187, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:ba98db39:::obj542:head local-lis/les=117/118 n=450 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.230+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.230+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.230+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:bb038e88:::obj54:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.230+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ba98db39:::obj542:head local-lis/les=117/118 n=450 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:bbf67374:::obj187:head, version: 111'187, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bbf67374:::obj187:head@111'187, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.230+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ba98db39:::obj542:head local-lis/les=117/118 n=450 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:bbf67374:::obj187:head@111'187, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.230+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ba98db39:::obj542:head local-lis/les=117/118 n=450 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:bbf67374:::obj187:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.230+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ba98db39:::obj542:head local-lis/les=117/118 n=450 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:bbf67374:::obj187:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.230+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ba98db39:::obj542:head local-lis/les=117/118 n=450 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:bbf67374:::obj187:head 2026-02-11T23:31:28.230+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ba98db39:::obj542:head local-lis/les=117/118 n=450 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:bbf67374:::obj187:head v 111'187 2026-02-11T23:31:28.230+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ba98db39:::obj542:head local-lis/les=117/118 n=450 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.230+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.230+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:bbf67374:::obj187:head, version: 111'187, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bbf67374:::obj187:head@111'187, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000073 2026-02-11T23:31:28.230+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.230+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.230+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:bb038e88:::obj54:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.230+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:bb038e88:::obj54:head) v3 prio 15 cost 0 latency 0.000090 pg_backfill(progress 4.0 e 119/119 lb 4:bb038e88:::obj54:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:ba98db39:::obj542:head local-lis/les=117/118 n=450 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.230+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.230+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ba98db39:::obj542:head local-lis/les=117/118 n=450 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:bb038e88:::obj54:head) v3 2026-02-11T23:31:28.230+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ba98db39:::obj542:head local-lis/les=117/118 n=450 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:bb038e88:::obj54:head) 2026-02-11T23:31:28.230+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.230+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:bb038e88:::obj54:head) v3 finish latency 0.000013 2026-02-11T23:31:28.230+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bb038e88:::obj54:head local-lis/les=117/118 n=451 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.230+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.230+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.230+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bb038e88:::obj54:head local-lis/les=117/118 n=451 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.230+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.230+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.230+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.230+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63ff200 con 0x507ac00 2026-02-11T23:31:28.230+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:bbf67374:::obj187:head)]) -- 0x670b680 con 0x507ac00 2026-02-11T23:31:28.230+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.231+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 943 ==== MOSDPGPush(4.0 119/117 [PushOp(4:bc55eba4:::obj534:head, version: 111'534, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bc55eba4:::obj534:head@111'534, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5617380 con 0x507ac00 2026-02-11T23:31:28.231+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:bc55eba4:::obj534:head, version: 111'534, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bc55eba4:::obj534:head@111'534, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.231+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:bc55eba4:::obj534:head, version: 111'534, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bc55eba4:::obj534:head@111'534, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.231+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.231+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 944 ==== pg_backfill(progress 4.0 e 119/119 lb 4:bbf67374:::obj187:head) ==== 1062+0+0 (unknown 0 0 0) 0x4a2c000 con 0x507ac00 2026-02-11T23:31:28.231+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:bbf67374:::obj187:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.231+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:bc55eba4:::obj534:head, version: 111'534, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bc55eba4:::obj534:head@111'534, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.231+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:bbf67374:::obj187:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.231+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.231+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:bc55eba4:::obj534:head, version: 111'534, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bc55eba4:::obj534:head@111'534, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.231+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.231+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:bbf67374:::obj187:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.231+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:bc55eba4:::obj534:head, version: 111'534, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bc55eba4:::obj534:head@111'534, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 119/117 [PushOp(4:bc55eba4:::obj534:head, version: 111'534, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bc55eba4:::obj534:head@111'534, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:bb038e88:::obj54:head local-lis/les=117/118 n=451 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.231+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.231+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bb038e88:::obj54:head local-lis/les=117/118 n=451 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:bc55eba4:::obj534:head, version: 111'534, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bc55eba4:::obj534:head@111'534, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.231+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bb038e88:::obj54:head local-lis/les=117/118 n=451 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:bc55eba4:::obj534:head@111'534, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.231+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bb038e88:::obj54:head local-lis/les=117/118 n=451 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:bc55eba4:::obj534:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.231+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bb038e88:::obj54:head local-lis/les=117/118 n=451 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:bc55eba4:::obj534:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.231+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bb038e88:::obj54:head local-lis/les=117/118 n=451 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:bc55eba4:::obj534:head 2026-02-11T23:31:28.231+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bb038e88:::obj54:head local-lis/les=117/118 n=451 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:bc55eba4:::obj534:head v 111'534 2026-02-11T23:31:28.231+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bb038e88:::obj54:head local-lis/les=117/118 n=451 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.231+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.231+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:bc55eba4:::obj534:head, version: 111'534, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bc55eba4:::obj534:head@111'534, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000069 2026-02-11T23:31:28.231+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.231+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.231+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:bbf67374:::obj187:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.231+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:bbf67374:::obj187:head) v3 prio 15 cost 0 latency 0.000087 pg_backfill(progress 4.0 e 119/119 lb 4:bbf67374:::obj187:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:bb038e88:::obj54:head local-lis/les=117/118 n=451 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.231+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.231+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bb038e88:::obj54:head local-lis/les=117/118 n=451 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:bbf67374:::obj187:head) v3 2026-02-11T23:31:28.231+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bb038e88:::obj54:head local-lis/les=117/118 n=451 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:bbf67374:::obj187:head) 2026-02-11T23:31:28.231+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.231+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:bbf67374:::obj187:head) v3 finish latency 0.000021 2026-02-11T23:31:28.231+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bbf67374:::obj187:head local-lis/les=117/118 n=452 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.231+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.231+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.231+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bbf67374:::obj187:head local-lis/les=117/118 n=452 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.231+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.231+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.231+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.231+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x670b680 con 0x507ac00 2026-02-11T23:31:28.231+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:bc55eba4:::obj534:head)]) -- 0x670b380 con 0x507ac00 2026-02-11T23:31:28.231+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.231+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 945 ==== MOSDPGPush(4.0 119/117 [PushOp(4:bd84d997:::obj292:head, version: 111'292, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bd84d997:::obj292:head@111'292, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5697ba0 con 0x507ac00 2026-02-11T23:31:28.231+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:bd84d997:::obj292:head, version: 111'292, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bd84d997:::obj292:head@111'292, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.231+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:bd84d997:::obj292:head, version: 111'292, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bd84d997:::obj292:head@111'292, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.231+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.231+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 946 ==== pg_backfill(progress 4.0 e 119/119 lb 4:bc55eba4:::obj534:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98580 con 0x507ac00 2026-02-11T23:31:28.231+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:bd84d997:::obj292:head, version: 111'292, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bd84d997:::obj292:head@111'292, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.231+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:bc55eba4:::obj534:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.231+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:bc55eba4:::obj534:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.231+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.231+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:bd84d997:::obj292:head, version: 111'292, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bd84d997:::obj292:head@111'292, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.231+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:bd84d997:::obj292:head, version: 111'292, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bd84d997:::obj292:head@111'292, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 119/117 [PushOp(4:bd84d997:::obj292:head, version: 111'292, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bd84d997:::obj292:head@111'292, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:bbf67374:::obj187:head local-lis/les=117/118 n=452 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.231+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.231+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.231+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:bc55eba4:::obj534:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.231+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bbf67374:::obj187:head local-lis/les=117/118 n=452 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:bd84d997:::obj292:head, version: 111'292, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bd84d997:::obj292:head@111'292, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.231+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bbf67374:::obj187:head local-lis/les=117/118 n=452 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:bd84d997:::obj292:head@111'292, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.231+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bbf67374:::obj187:head local-lis/les=117/118 n=452 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:bd84d997:::obj292:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.231+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bbf67374:::obj187:head local-lis/les=117/118 n=452 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:bd84d997:::obj292:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.231+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bbf67374:::obj187:head local-lis/les=117/118 n=452 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:bd84d997:::obj292:head 2026-02-11T23:31:28.231+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bbf67374:::obj187:head local-lis/les=117/118 n=452 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:bd84d997:::obj292:head v 111'292 2026-02-11T23:31:28.231+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bbf67374:::obj187:head local-lis/les=117/118 n=452 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.231+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.231+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:bd84d997:::obj292:head, version: 111'292, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bd84d997:::obj292:head@111'292, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000068 2026-02-11T23:31:28.231+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.231+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.231+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:bc55eba4:::obj534:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.231+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:bc55eba4:::obj534:head) v3 prio 15 cost 0 latency 0.000085 pg_backfill(progress 4.0 e 119/119 lb 4:bc55eba4:::obj534:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:bbf67374:::obj187:head local-lis/les=117/118 n=452 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.231+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.231+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bbf67374:::obj187:head local-lis/les=117/118 n=452 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:bc55eba4:::obj534:head) v3 2026-02-11T23:31:28.231+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bbf67374:::obj187:head local-lis/les=117/118 n=452 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:bc55eba4:::obj534:head) 2026-02-11T23:31:28.231+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.231+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:bc55eba4:::obj534:head) v3 finish latency 0.000015 2026-02-11T23:31:28.231+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bc55eba4:::obj534:head local-lis/les=117/118 n=453 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.231+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.231+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bc55eba4:::obj534:head local-lis/les=117/118 n=453 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.231+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.231+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.231+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.231+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.231+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63ff200 con 0x507ac00 2026-02-11T23:31:28.231+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:bd84d997:::obj292:head)]) -- 0x670b080 con 0x507ac00 2026-02-11T23:31:28.231+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.232+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 947 ==== MOSDPGPush(4.0 119/117 [PushOp(4:bdc5b82e:::obj52:head, version: 111'52, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bdc5b82e:::obj52:head@111'52, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5647380 con 0x507ac00 2026-02-11T23:31:28.232+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:bdc5b82e:::obj52:head, version: 111'52, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bdc5b82e:::obj52:head@111'52, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.232+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:bdc5b82e:::obj52:head, version: 111'52, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bdc5b82e:::obj52:head@111'52, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.232+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.232+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:bdc5b82e:::obj52:head, version: 111'52, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bdc5b82e:::obj52:head@111'52, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.232+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 948 ==== pg_backfill(progress 4.0 e 119/119 lb 4:bd84d997:::obj292:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9b700 con 0x507ac00 2026-02-11T23:31:28.232+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.232+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:bd84d997:::obj292:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 119 2026-02-11T23:31:28.232+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:bdc5b82e:::obj52:head, version: 111'52, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bdc5b82e:::obj52:head@111'52, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.232+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:bd84d997:::obj292:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.232+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.232+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:bd84d997:::obj292:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.232+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:bdc5b82e:::obj52:head, version: 111'52, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bdc5b82e:::obj52:head@111'52, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 119/117 [PushOp(4:bdc5b82e:::obj52:head, version: 111'52, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bdc5b82e:::obj52:head@111'52, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:bc55eba4:::obj534:head local-lis/les=117/118 n=453 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.232+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.232+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bc55eba4:::obj534:head local-lis/les=117/118 n=453 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:bdc5b82e:::obj52:head, version: 111'52, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bdc5b82e:::obj52:head@111'52, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.232+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bc55eba4:::obj534:head local-lis/les=117/118 n=453 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:bdc5b82e:::obj52:head@111'52, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.232+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bc55eba4:::obj534:head local-lis/les=117/118 n=453 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:bdc5b82e:::obj52:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.232+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bc55eba4:::obj534:head local-lis/les=117/118 n=453 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:bdc5b82e:::obj52:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.232+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bc55eba4:::obj534:head local-lis/les=117/118 n=453 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:bdc5b82e:::obj52:head 2026-02-11T23:31:28.232+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bc55eba4:::obj534:head local-lis/les=117/118 n=453 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:bdc5b82e:::obj52:head v 111'52 2026-02-11T23:31:28.232+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bc55eba4:::obj534:head local-lis/les=117/118 n=453 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.232+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.232+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:bdc5b82e:::obj52:head, version: 111'52, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bdc5b82e:::obj52:head@111'52, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000071 2026-02-11T23:31:28.232+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.232+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:bd84d997:::obj292:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.232+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.232+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:bd84d997:::obj292:head) v3 prio 15 cost 0 latency 0.000091 pg_backfill(progress 4.0 e 119/119 lb 4:bd84d997:::obj292:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:bc55eba4:::obj534:head local-lis/les=117/118 n=453 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.232+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.232+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bc55eba4:::obj534:head local-lis/les=117/118 n=453 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:bd84d997:::obj292:head) v3 2026-02-11T23:31:28.232+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bc55eba4:::obj534:head local-lis/les=117/118 n=453 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:bd84d997:::obj292:head) 2026-02-11T23:31:28.232+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.232+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:bd84d997:::obj292:head) v3 finish latency 0.000017 2026-02-11T23:31:28.232+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bd84d997:::obj292:head local-lis/les=117/118 n=454 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.232+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.232+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.232+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bd84d997:::obj292:head local-lis/les=117/118 n=454 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.232+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.232+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.232+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.232+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66fbb00 con 0x507ac00 2026-02-11T23:31:28.232+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:bdc5b82e:::obj52:head)]) -- 0x670ad80 con 0x507ac00 2026-02-11T23:31:28.232+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.232+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 949 ==== MOSDPGPush(4.0 119/117 [PushOp(4:bdf2c185:::obj359:head, version: 111'359, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bdf2c185:::obj359:head@111'359, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f0d00 con 0x507ac00 2026-02-11T23:31:28.232+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:bdf2c185:::obj359:head, version: 111'359, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bdf2c185:::obj359:head@111'359, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.232+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:bdf2c185:::obj359:head, version: 111'359, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bdf2c185:::obj359:head@111'359, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.232+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.232+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 950 ==== pg_backfill(progress 4.0 e 119/119 lb 4:bdc5b82e:::obj52:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d99b80 con 0x507ac00 2026-02-11T23:31:28.232+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:bdf2c185:::obj359:head, version: 111'359, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bdf2c185:::obj359:head@111'359, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.232+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:bdc5b82e:::obj52:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.232+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:bdc5b82e:::obj52:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.232+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.232+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:bdf2c185:::obj359:head, version: 111'359, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bdf2c185:::obj359:head@111'359, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.232+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:bdf2c185:::obj359:head, version: 111'359, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bdf2c185:::obj359:head@111'359, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 119/117 [PushOp(4:bdf2c185:::obj359:head, version: 111'359, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bdf2c185:::obj359:head@111'359, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:bd84d997:::obj292:head local-lis/les=117/118 n=454 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.232+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.232+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.232+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:bdc5b82e:::obj52:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.232+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bd84d997:::obj292:head local-lis/les=117/118 n=454 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:bdf2c185:::obj359:head, version: 111'359, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bdf2c185:::obj359:head@111'359, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.232+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bd84d997:::obj292:head local-lis/les=117/118 n=454 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:bdf2c185:::obj359:head@111'359, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.232+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bd84d997:::obj292:head local-lis/les=117/118 n=454 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:bdf2c185:::obj359:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.232+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bd84d997:::obj292:head local-lis/les=117/118 n=454 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:bdf2c185:::obj359:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.232+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bd84d997:::obj292:head local-lis/les=117/118 n=454 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:bdf2c185:::obj359:head 2026-02-11T23:31:28.232+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bd84d997:::obj292:head local-lis/les=117/118 n=454 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:bdf2c185:::obj359:head v 111'359 2026-02-11T23:31:28.232+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bd84d997:::obj292:head local-lis/les=117/118 n=454 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.232+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.232+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:bdf2c185:::obj359:head, version: 111'359, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bdf2c185:::obj359:head@111'359, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000065 2026-02-11T23:31:28.232+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.232+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.232+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:bdc5b82e:::obj52:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.232+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:bdc5b82e:::obj52:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 4.0 e 119/119 lb 4:bdc5b82e:::obj52:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:bd84d997:::obj292:head local-lis/les=117/118 n=454 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.232+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.232+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bd84d997:::obj292:head local-lis/les=117/118 n=454 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:bdc5b82e:::obj52:head) v3 2026-02-11T23:31:28.232+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bd84d997:::obj292:head local-lis/les=117/118 n=454 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:bdc5b82e:::obj52:head) 2026-02-11T23:31:28.232+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.232+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:bdc5b82e:::obj52:head) v3 finish latency 0.000015 2026-02-11T23:31:28.232+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bdc5b82e:::obj52:head local-lis/les=117/118 n=455 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.232+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.233+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.233+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bdc5b82e:::obj52:head local-lis/les=117/118 n=455 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.233+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.233+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.233+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.233+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7aa80 con 0x507ac00 2026-02-11T23:31:28.233+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:bdf2c185:::obj359:head)]) -- 0x66fac00 con 0x507ac00 2026-02-11T23:31:28.233+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.233+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 951 ==== MOSDPGPush(4.0 119/117 [PushOp(4:be10f7c9:::obj475:head, version: 111'475, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be10f7c9:::obj475:head@111'475, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56b64e0 con 0x507ac00 2026-02-11T23:31:28.233+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:be10f7c9:::obj475:head, version: 111'475, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be10f7c9:::obj475:head@111'475, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.233+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:be10f7c9:::obj475:head, version: 111'475, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be10f7c9:::obj475:head@111'475, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.233+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.233+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 952 ==== pg_backfill(progress 4.0 e 119/119 lb 4:bdf2c185:::obj359:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176b00 con 0x507ac00 2026-02-11T23:31:28.233+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:bdf2c185:::obj359:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.233+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:be10f7c9:::obj475:head, version: 111'475, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be10f7c9:::obj475:head@111'475, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.233+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:bdf2c185:::obj359:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.233+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.233+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:be10f7c9:::obj475:head, version: 111'475, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be10f7c9:::obj475:head@111'475, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.233+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:be10f7c9:::obj475:head, version: 111'475, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be10f7c9:::obj475:head@111'475, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 119/117 [PushOp(4:be10f7c9:::obj475:head, version: 111'475, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be10f7c9:::obj475:head@111'475, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:bdc5b82e:::obj52:head local-lis/les=117/118 n=455 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.233+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.233+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.233+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:bdf2c185:::obj359:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.233+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bdc5b82e:::obj52:head local-lis/les=117/118 n=455 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:be10f7c9:::obj475:head, version: 111'475, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be10f7c9:::obj475:head@111'475, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.233+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bdc5b82e:::obj52:head local-lis/les=117/118 n=455 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:be10f7c9:::obj475:head@111'475, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.233+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bdc5b82e:::obj52:head local-lis/les=117/118 n=455 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:be10f7c9:::obj475:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.233+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bdc5b82e:::obj52:head local-lis/les=117/118 n=455 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:be10f7c9:::obj475:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.233+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bdc5b82e:::obj52:head local-lis/les=117/118 n=455 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:be10f7c9:::obj475:head 2026-02-11T23:31:28.233+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bdc5b82e:::obj52:head local-lis/les=117/118 n=455 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:be10f7c9:::obj475:head v 111'475 2026-02-11T23:31:28.233+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bdc5b82e:::obj52:head local-lis/les=117/118 n=455 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.233+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.233+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:be10f7c9:::obj475:head, version: 111'475, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be10f7c9:::obj475:head@111'475, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000069 2026-02-11T23:31:28.233+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.233+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.233+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:bdf2c185:::obj359:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.233+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:bdf2c185:::obj359:head) v3 prio 15 cost 0 latency 0.000086 pg_backfill(progress 4.0 e 119/119 lb 4:bdf2c185:::obj359:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:bdc5b82e:::obj52:head local-lis/les=117/118 n=455 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.233+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.233+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bdc5b82e:::obj52:head local-lis/les=117/118 n=455 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:bdf2c185:::obj359:head) v3 2026-02-11T23:31:28.233+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bdc5b82e:::obj52:head local-lis/les=117/118 n=455 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:bdf2c185:::obj359:head) 2026-02-11T23:31:28.233+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.233+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:bdf2c185:::obj359:head) v3 finish latency 0.000016 2026-02-11T23:31:28.233+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bdf2c185:::obj359:head local-lis/les=117/118 n=456 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.233+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.233+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bdf2c185:::obj359:head local-lis/les=117/118 n=456 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.233+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.233+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.233+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.233+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.233+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66fbc80 con 0x507ac00 2026-02-11T23:31:28.233+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:be10f7c9:::obj475:head)]) -- 0x66fb200 con 0x507ac00 2026-02-11T23:31:28.233+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.233+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 953 ==== MOSDPGPush(4.0 119/117 [PushOp(4:be1f0c3b:::obj423:head, version: 111'423, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be1f0c3b:::obj423:head@111'423, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56caea0 con 0x507ac00 2026-02-11T23:31:28.233+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:be1f0c3b:::obj423:head, version: 111'423, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be1f0c3b:::obj423:head@111'423, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.233+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:be1f0c3b:::obj423:head, version: 111'423, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be1f0c3b:::obj423:head@111'423, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.233+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.233+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 954 ==== pg_backfill(progress 4.0 e 119/119 lb 4:be10f7c9:::obj475:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99600 con 0x507ac00 2026-02-11T23:31:28.233+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:be10f7c9:::obj475:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.233+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:be1f0c3b:::obj423:head, version: 111'423, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be1f0c3b:::obj423:head@111'423, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.233+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:be10f7c9:::obj475:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.233+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.233+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:be1f0c3b:::obj423:head, version: 111'423, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be1f0c3b:::obj423:head@111'423, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.233+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:be1f0c3b:::obj423:head, version: 111'423, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be1f0c3b:::obj423:head@111'423, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 119/117 [PushOp(4:be1f0c3b:::obj423:head, version: 111'423, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be1f0c3b:::obj423:head@111'423, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:bdf2c185:::obj359:head local-lis/les=117/118 n=456 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.233+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.233+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.233+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:be10f7c9:::obj475:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.233+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bdf2c185:::obj359:head local-lis/les=117/118 n=456 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:be1f0c3b:::obj423:head, version: 111'423, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be1f0c3b:::obj423:head@111'423, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.233+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bdf2c185:::obj359:head local-lis/les=117/118 n=456 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:be1f0c3b:::obj423:head@111'423, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.233+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bdf2c185:::obj359:head local-lis/les=117/118 n=456 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:be1f0c3b:::obj423:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.233+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bdf2c185:::obj359:head local-lis/les=117/118 n=456 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:be1f0c3b:::obj423:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.233+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bdf2c185:::obj359:head local-lis/les=117/118 n=456 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:be1f0c3b:::obj423:head 2026-02-11T23:31:28.233+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bdf2c185:::obj359:head local-lis/les=117/118 n=456 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:be1f0c3b:::obj423:head v 111'423 2026-02-11T23:31:28.233+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bdf2c185:::obj359:head local-lis/les=117/118 n=456 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.233+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.233+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:be1f0c3b:::obj423:head, version: 111'423, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be1f0c3b:::obj423:head@111'423, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000077 2026-02-11T23:31:28.233+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.233+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.233+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:be10f7c9:::obj475:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.233+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:be10f7c9:::obj475:head) v3 prio 15 cost 0 latency 0.000094 pg_backfill(progress 4.0 e 119/119 lb 4:be10f7c9:::obj475:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:bdf2c185:::obj359:head local-lis/les=117/118 n=456 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.233+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.233+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bdf2c185:::obj359:head local-lis/les=117/118 n=456 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:be10f7c9:::obj475:head) v3 2026-02-11T23:31:28.233+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bdf2c185:::obj359:head local-lis/les=117/118 n=456 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:be10f7c9:::obj475:head) 2026-02-11T23:31:28.233+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.233+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:be10f7c9:::obj475:head) v3 finish latency 0.000015 2026-02-11T23:31:28.233+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:be10f7c9:::obj475:head local-lis/les=117/118 n=457 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.233+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.234+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.234+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:be10f7c9:::obj475:head local-lis/les=117/118 n=457 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.234+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.234+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.234+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.234+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x5f2dc80 con 0x507ac00 2026-02-11T23:31:28.234+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.234+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:be1f0c3b:::obj423:head)]) -- 0x66fad80 con 0x507ac00 2026-02-11T23:31:28.234+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.234+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 955 ==== MOSDPGPush(4.0 119/117 [PushOp(4:be604f9b:::obj512:head, version: 111'512, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be604f9b:::obj512:head@111'512, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db0000 con 0x507ac00 2026-02-11T23:31:28.234+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:be604f9b:::obj512:head, version: 111'512, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be604f9b:::obj512:head@111'512, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.234+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:be604f9b:::obj512:head, version: 111'512, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be604f9b:::obj512:head@111'512, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.234+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.234+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 956 ==== pg_backfill(progress 4.0 e 119/119 lb 4:be1f0c3b:::obj423:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2d600 con 0x507ac00 2026-02-11T23:31:28.234+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:be604f9b:::obj512:head, version: 111'512, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be604f9b:::obj512:head@111'512, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.234+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:be1f0c3b:::obj423:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.234+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:be1f0c3b:::obj423:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.234+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.234+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:be604f9b:::obj512:head, version: 111'512, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be604f9b:::obj512:head@111'512, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.234+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:be604f9b:::obj512:head, version: 111'512, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be604f9b:::obj512:head@111'512, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 119/117 [PushOp(4:be604f9b:::obj512:head, version: 111'512, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be604f9b:::obj512:head@111'512, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:be10f7c9:::obj475:head local-lis/les=117/118 n=457 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.234+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.234+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.234+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:be1f0c3b:::obj423:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.234+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:be10f7c9:::obj475:head local-lis/les=117/118 n=457 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:be604f9b:::obj512:head, version: 111'512, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be604f9b:::obj512:head@111'512, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.234+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:be10f7c9:::obj475:head local-lis/les=117/118 n=457 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:be604f9b:::obj512:head@111'512, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.234+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:be10f7c9:::obj475:head local-lis/les=117/118 n=457 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:be604f9b:::obj512:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.234+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:be10f7c9:::obj475:head local-lis/les=117/118 n=457 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:be604f9b:::obj512:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.234+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:be10f7c9:::obj475:head local-lis/les=117/118 n=457 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:be604f9b:::obj512:head 2026-02-11T23:31:28.234+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:be10f7c9:::obj475:head local-lis/les=117/118 n=457 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:be604f9b:::obj512:head v 111'512 2026-02-11T23:31:28.234+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:be10f7c9:::obj475:head local-lis/les=117/118 n=457 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.234+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.234+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:be604f9b:::obj512:head, version: 111'512, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be604f9b:::obj512:head@111'512, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000074 2026-02-11T23:31:28.234+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.234+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.234+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:be1f0c3b:::obj423:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.234+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:be1f0c3b:::obj423:head) v3 prio 15 cost 0 latency 0.000091 pg_backfill(progress 4.0 e 119/119 lb 4:be1f0c3b:::obj423:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:be10f7c9:::obj475:head local-lis/les=117/118 n=457 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.234+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.234+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:be10f7c9:::obj475:head local-lis/les=117/118 n=457 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:be1f0c3b:::obj423:head) v3 2026-02-11T23:31:28.234+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:be10f7c9:::obj475:head local-lis/les=117/118 n=457 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:be1f0c3b:::obj423:head) 2026-02-11T23:31:28.234+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.234+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:be1f0c3b:::obj423:head) v3 finish latency 0.000015 2026-02-11T23:31:28.234+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:be1f0c3b:::obj423:head local-lis/les=117/118 n=458 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.234+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.234+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.234+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:be1f0c3b:::obj423:head local-lis/les=117/118 n=458 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.234+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.234+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.234+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.234+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66fb800 con 0x507ac00 2026-02-11T23:31:28.234+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:be604f9b:::obj512:head)]) -- 0x670a480 con 0x507ac00 2026-02-11T23:31:28.234+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.235+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 957 ==== MOSDPGPush(4.0 119/117 [PushOp(4:beac82f5:::obj51:head, version: 111'51, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:beac82f5:::obj51:head@111'51, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x56a3a00 con 0x507ac00 2026-02-11T23:31:28.235+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:beac82f5:::obj51:head, version: 111'51, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:beac82f5:::obj51:head@111'51, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 119 2026-02-11T23:31:28.235+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:beac82f5:::obj51:head, version: 111'51, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:beac82f5:::obj51:head@111'51, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.235+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.235+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 958 ==== pg_backfill(progress 4.0 e 119/119 lb 4:be604f9b:::obj512:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2ec00 con 0x507ac00 2026-02-11T23:31:28.235+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:beac82f5:::obj51:head, version: 111'51, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:beac82f5:::obj51:head@111'51, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.235+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:be604f9b:::obj512:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.235+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:be604f9b:::obj512:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.235+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.235+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:beac82f5:::obj51:head, version: 111'51, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:beac82f5:::obj51:head@111'51, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.235+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.235+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:beac82f5:::obj51:head, version: 111'51, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:beac82f5:::obj51:head@111'51, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 119/117 [PushOp(4:beac82f5:::obj51:head, version: 111'51, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:beac82f5:::obj51:head@111'51, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:be1f0c3b:::obj423:head local-lis/les=117/118 n=458 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.235+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.235+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:be1f0c3b:::obj423:head local-lis/les=117/118 n=458 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:beac82f5:::obj51:head, version: 111'51, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:beac82f5:::obj51:head@111'51, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.235+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.235+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:be604f9b:::obj512:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.235+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:be1f0c3b:::obj423:head local-lis/les=117/118 n=458 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:beac82f5:::obj51:head@111'51, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.235+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:be1f0c3b:::obj423:head local-lis/les=117/118 n=458 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:beac82f5:::obj51:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.235+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:be1f0c3b:::obj423:head local-lis/les=117/118 n=458 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:beac82f5:::obj51:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.235+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:be1f0c3b:::obj423:head local-lis/les=117/118 n=458 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:beac82f5:::obj51:head 2026-02-11T23:31:28.235+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:be1f0c3b:::obj423:head local-lis/les=117/118 n=458 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:beac82f5:::obj51:head v 111'51 2026-02-11T23:31:28.235+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:be1f0c3b:::obj423:head local-lis/les=117/118 n=458 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.235+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.235+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:beac82f5:::obj51:head, version: 111'51, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:beac82f5:::obj51:head@111'51, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:28.235+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:be1f0c3b:::obj423:head local-lis/les=117/118 n=458 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.235+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.235+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.235+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:be604f9b:::obj512:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.235+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:be604f9b:::obj512:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 4.0 e 119/119 lb 4:be604f9b:::obj512:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:be1f0c3b:::obj423:head local-lis/les=117/118 n=458 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.235+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.235+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:be1f0c3b:::obj423:head local-lis/les=117/118 n=458 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:be604f9b:::obj512:head) v3 2026-02-11T23:31:28.235+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:be1f0c3b:::obj423:head local-lis/les=117/118 n=458 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:be604f9b:::obj512:head) 2026-02-11T23:31:28.235+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.235+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:be604f9b:::obj512:head) v3 finish latency 0.000021 2026-02-11T23:31:28.235+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.235+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.235+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:be604f9b:::obj512:head local-lis/les=117/118 n=459 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.235+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.235+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.235+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.235+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66fa480 con 0x507ac00 2026-02-11T23:31:28.235+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:beac82f5:::obj51:head)]) -- 0x4f7a300 con 0x507ac00 2026-02-11T23:31:28.235+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.235+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.235+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.235+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 959 ==== MOSDPGPush(4.0 119/117 [PushOp(4:bec6553a:::obj65:head, version: 111'65, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bec6553a:::obj65:head@111'65, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x56a2820 con 0x507ac00 2026-02-11T23:31:28.235+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:bec6553a:::obj65:head, version: 111'65, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bec6553a:::obj65:head@111'65, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.235+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:bec6553a:::obj65:head, version: 111'65, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bec6553a:::obj65:head@111'65, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.235+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.235+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 960 ==== pg_backfill(progress 4.0 e 119/119 lb 4:beac82f5:::obj51:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d98000 con 0x507ac00 2026-02-11T23:31:28.235+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:bec6553a:::obj65:head, version: 111'65, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bec6553a:::obj65:head@111'65, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.235+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:beac82f5:::obj51:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.235+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:beac82f5:::obj51:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.235+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.235+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:bec6553a:::obj65:head, version: 111'65, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bec6553a:::obj65:head@111'65, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.235+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.235+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:bec6553a:::obj65:head, version: 111'65, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bec6553a:::obj65:head@111'65, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 119/117 [PushOp(4:bec6553a:::obj65:head, version: 111'65, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bec6553a:::obj65:head@111'65, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:be604f9b:::obj512:head local-lis/les=117/118 n=459 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.235+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:beac82f5:::obj51:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.235+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.235+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:be604f9b:::obj512:head local-lis/les=117/118 n=459 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:bec6553a:::obj65:head, version: 111'65, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bec6553a:::obj65:head@111'65, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.235+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:be604f9b:::obj512:head local-lis/les=117/118 n=459 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:bec6553a:::obj65:head@111'65, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.235+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:be604f9b:::obj512:head local-lis/les=117/118 n=459 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:bec6553a:::obj65:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.235+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:be604f9b:::obj512:head local-lis/les=117/118 n=459 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:bec6553a:::obj65:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.235+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:be604f9b:::obj512:head local-lis/les=117/118 n=459 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:bec6553a:::obj65:head 2026-02-11T23:31:28.235+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:be604f9b:::obj512:head local-lis/les=117/118 n=459 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:bec6553a:::obj65:head v 111'65 2026-02-11T23:31:28.235+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:be604f9b:::obj512:head local-lis/les=117/118 n=459 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.235+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.235+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:bec6553a:::obj65:head, version: 111'65, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bec6553a:::obj65:head@111'65, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:28.235+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:be604f9b:::obj512:head local-lis/les=117/118 n=459 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.235+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.235+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.235+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:beac82f5:::obj51:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.235+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:beac82f5:::obj51:head) v3 prio 15 cost 0 latency 0.000082 pg_backfill(progress 4.0 e 119/119 lb 4:beac82f5:::obj51:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:be604f9b:::obj512:head local-lis/les=117/118 n=459 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.235+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.235+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:be604f9b:::obj512:head local-lis/les=117/118 n=459 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:beac82f5:::obj51:head) v3 2026-02-11T23:31:28.235+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:be604f9b:::obj512:head local-lis/les=117/118 n=459 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:beac82f5:::obj51:head) 2026-02-11T23:31:28.235+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.235+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:beac82f5:::obj51:head) v3 finish latency 0.000013 2026-02-11T23:31:28.235+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.235+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.235+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:beac82f5:::obj51:head local-lis/les=117/118 n=460 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.235+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.235+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.235+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.235+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x671c180 con 0x507ac00 2026-02-11T23:31:28.235+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:bec6553a:::obj65:head)]) -- 0x4f7b200 con 0x507ac00 2026-02-11T23:31:28.235+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.236+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 961 ==== MOSDPGPush(4.0 119/117 [PushOp(4:bf41e297:::obj573:head, version: 111'573, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bf41e297:::obj573:head@111'573, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x554e4e0 con 0x507ac00 2026-02-11T23:31:28.236+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:bf41e297:::obj573:head, version: 111'573, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bf41e297:::obj573:head@111'573, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 119 2026-02-11T23:31:28.236+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:bf41e297:::obj573:head, version: 111'573, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bf41e297:::obj573:head@111'573, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.236+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.236+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 962 ==== pg_backfill(progress 4.0 e 119/119 lb 4:bec6553a:::obj65:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d9b180 con 0x507ac00 2026-02-11T23:31:28.236+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:bec6553a:::obj65:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 119 2026-02-11T23:31:28.236+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:bf41e297:::obj573:head, version: 111'573, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bf41e297:::obj573:head@111'573, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.236+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:bec6553a:::obj65:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.236+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.236+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:bf41e297:::obj573:head, version: 111'573, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bf41e297:::obj573:head@111'573, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.236+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.236+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:bec6553a:::obj65:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.236+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:bf41e297:::obj573:head, version: 111'573, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bf41e297:::obj573:head@111'573, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(4.0 119/117 [PushOp(4:bf41e297:::obj573:head, version: 111'573, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bf41e297:::obj573:head@111'573, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:beac82f5:::obj51:head local-lis/les=117/118 n=460 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.236+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.236+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:beac82f5:::obj51:head local-lis/les=117/118 n=460 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:bf41e297:::obj573:head, version: 111'573, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bf41e297:::obj573:head@111'573, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.236+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:beac82f5:::obj51:head local-lis/les=117/118 n=460 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:bf41e297:::obj573:head@111'573, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.236+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:beac82f5:::obj51:head local-lis/les=117/118 n=460 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:bf41e297:::obj573:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.236+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:beac82f5:::obj51:head local-lis/les=117/118 n=460 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:bf41e297:::obj573:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.236+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:beac82f5:::obj51:head local-lis/les=117/118 n=460 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:bf41e297:::obj573:head 2026-02-11T23:31:28.236+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:beac82f5:::obj51:head local-lis/les=117/118 n=460 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:bf41e297:::obj573:head v 111'573 2026-02-11T23:31:28.236+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:beac82f5:::obj51:head local-lis/les=117/118 n=460 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.236+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.236+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:bf41e297:::obj573:head, version: 111'573, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bf41e297:::obj573:head@111'573, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000070 2026-02-11T23:31:28.236+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.236+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.236+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:bec6553a:::obj65:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.236+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:bec6553a:::obj65:head) v3 prio 15 cost 0 latency 0.000091 pg_backfill(progress 4.0 e 119/119 lb 4:bec6553a:::obj65:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:beac82f5:::obj51:head local-lis/les=117/118 n=460 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.236+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.236+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:beac82f5:::obj51:head local-lis/les=117/118 n=460 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:bec6553a:::obj65:head) v3 2026-02-11T23:31:28.236+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:beac82f5:::obj51:head local-lis/les=117/118 n=460 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:bec6553a:::obj65:head) 2026-02-11T23:31:28.236+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.236+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:bec6553a:::obj65:head) v3 finish latency 0.000017 2026-02-11T23:31:28.236+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bec6553a:::obj65:head local-lis/les=117/118 n=461 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.236+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.236+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.236+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bec6553a:::obj65:head local-lis/les=117/118 n=461 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.236+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.236+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.236+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.236+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x642db00 con 0x507ac00 2026-02-11T23:31:28.236+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:bf41e297:::obj573:head)]) -- 0x6738180 con 0x507ac00 2026-02-11T23:31:28.236+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.236+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 963 ==== MOSDPGPush(4.0 119/117 [PushOp(4:bf770068:::obj268:head, version: 111'268, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bf770068:::obj268:head@111'268, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f1d40 con 0x507ac00 2026-02-11T23:31:28.236+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:bf770068:::obj268:head, version: 111'268, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bf770068:::obj268:head@111'268, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.236+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:bf770068:::obj268:head, version: 111'268, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bf770068:::obj268:head@111'268, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.236+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.236+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 964 ==== pg_backfill(progress 4.0 e 119/119 lb 4:bf41e297:::obj573:head) ==== 1062+0+0 (unknown 0 0 0) 0x6179180 con 0x507ac00 2026-02-11T23:31:28.236+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:bf41e297:::obj573:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.236+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:bf770068:::obj268:head, version: 111'268, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bf770068:::obj268:head@111'268, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.236+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:bf41e297:::obj573:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.236+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.236+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:bf770068:::obj268:head, version: 111'268, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bf770068:::obj268:head@111'268, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.236+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:bf770068:::obj268:head, version: 111'268, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bf770068:::obj268:head@111'268, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 119/117 [PushOp(4:bf770068:::obj268:head, version: 111'268, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bf770068:::obj268:head@111'268, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:bec6553a:::obj65:head local-lis/les=117/118 n=461 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.236+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.236+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.236+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:bf41e297:::obj573:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.236+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bec6553a:::obj65:head local-lis/les=117/118 n=461 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:bf770068:::obj268:head, version: 111'268, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bf770068:::obj268:head@111'268, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.236+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bec6553a:::obj65:head local-lis/les=117/118 n=461 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:bf770068:::obj268:head@111'268, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.236+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bec6553a:::obj65:head local-lis/les=117/118 n=461 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:bf770068:::obj268:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.236+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bec6553a:::obj65:head local-lis/les=117/118 n=461 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:bf770068:::obj268:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.236+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bec6553a:::obj65:head local-lis/les=117/118 n=461 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:bf770068:::obj268:head 2026-02-11T23:31:28.236+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bec6553a:::obj65:head local-lis/les=117/118 n=461 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:bf770068:::obj268:head v 111'268 2026-02-11T23:31:28.236+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bec6553a:::obj65:head local-lis/les=117/118 n=461 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.236+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.236+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:bf770068:::obj268:head, version: 111'268, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bf770068:::obj268:head@111'268, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000067 2026-02-11T23:31:28.236+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.236+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.236+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:bf41e297:::obj573:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.236+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:bf41e297:::obj573:head) v3 prio 15 cost 0 latency 0.000085 pg_backfill(progress 4.0 e 119/119 lb 4:bf41e297:::obj573:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:bec6553a:::obj65:head local-lis/les=117/118 n=461 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.236+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.236+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bec6553a:::obj65:head local-lis/les=117/118 n=461 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:bf41e297:::obj573:head) v3 2026-02-11T23:31:28.236+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bec6553a:::obj65:head local-lis/les=117/118 n=461 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:bf41e297:::obj573:head) 2026-02-11T23:31:28.236+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.236+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:bf41e297:::obj573:head) v3 finish latency 0.000015 2026-02-11T23:31:28.236+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bf41e297:::obj573:head local-lis/les=117/118 n=462 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.236+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.237+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.237+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bf41e297:::obj573:head local-lis/les=117/118 n=462 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.237+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.237+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.237+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.237+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x671c480 con 0x507ac00 2026-02-11T23:31:28.237+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:bf770068:::obj268:head)]) -- 0x6738480 con 0x507ac00 2026-02-11T23:31:28.237+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.237+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 965 ==== MOSDPGPush(4.0 119/117 [PushOp(4:c0848ef4:::obj545:head, version: 111'545, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c0848ef4:::obj545:head@111'545, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2000 con 0x507ac00 2026-02-11T23:31:28.237+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:c0848ef4:::obj545:head, version: 111'545, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c0848ef4:::obj545:head@111'545, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.237+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c0848ef4:::obj545:head, version: 111'545, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c0848ef4:::obj545:head@111'545, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.237+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.237+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 966 ==== pg_backfill(progress 4.0 e 119/119 lb 4:bf770068:::obj268:head) ==== 1062+0+0 (unknown 0 0 0) 0x6178100 con 0x507ac00 2026-02-11T23:31:28.237+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:bf770068:::obj268:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.237+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c0848ef4:::obj545:head, version: 111'545, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c0848ef4:::obj545:head@111'545, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.237+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:bf770068:::obj268:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.237+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.237+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c0848ef4:::obj545:head, version: 111'545, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c0848ef4:::obj545:head@111'545, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.237+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:c0848ef4:::obj545:head, version: 111'545, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c0848ef4:::obj545:head@111'545, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 119/117 [PushOp(4:c0848ef4:::obj545:head, version: 111'545, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c0848ef4:::obj545:head@111'545, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:bf41e297:::obj573:head local-lis/les=117/118 n=462 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.237+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.237+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.237+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:bf770068:::obj268:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.237+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bf41e297:::obj573:head local-lis/les=117/118 n=462 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:c0848ef4:::obj545:head, version: 111'545, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c0848ef4:::obj545:head@111'545, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.237+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bf41e297:::obj573:head local-lis/les=117/118 n=462 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:c0848ef4:::obj545:head@111'545, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.237+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bf41e297:::obj573:head local-lis/les=117/118 n=462 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:c0848ef4:::obj545:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.237+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bf41e297:::obj573:head local-lis/les=117/118 n=462 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:c0848ef4:::obj545:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.237+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bf41e297:::obj573:head local-lis/les=117/118 n=462 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:c0848ef4:::obj545:head 2026-02-11T23:31:28.237+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bf41e297:::obj573:head local-lis/les=117/118 n=462 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:c0848ef4:::obj545:head v 111'545 2026-02-11T23:31:28.237+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bf41e297:::obj573:head local-lis/les=117/118 n=462 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.237+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.237+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:c0848ef4:::obj545:head, version: 111'545, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c0848ef4:::obj545:head@111'545, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000065 2026-02-11T23:31:28.237+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.237+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.237+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:bf770068:::obj268:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.237+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:bf770068:::obj268:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 4.0 e 119/119 lb 4:bf770068:::obj268:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:bf41e297:::obj573:head local-lis/les=117/118 n=462 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.237+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.237+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bf41e297:::obj573:head local-lis/les=117/118 n=462 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:bf770068:::obj268:head) v3 2026-02-11T23:31:28.237+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bf41e297:::obj573:head local-lis/les=117/118 n=462 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:bf770068:::obj268:head) 2026-02-11T23:31:28.237+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.237+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:bf770068:::obj268:head) v3 finish latency 0.000015 2026-02-11T23:31:28.237+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bf770068:::obj268:head local-lis/les=117/118 n=463 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.237+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.237+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.237+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bf770068:::obj268:head local-lis/les=117/118 n=463 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.237+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.237+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.237+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.237+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x640e180 con 0x507ac00 2026-02-11T23:31:28.237+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:c0848ef4:::obj545:head)]) -- 0x6738780 con 0x507ac00 2026-02-11T23:31:28.237+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.237+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 967 ==== MOSDPGPush(4.0 119/117 [PushOp(4:c0c49def:::obj22:head, version: 111'22, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c0c49def:::obj22:head@111'22, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x56aa000 con 0x507ac00 2026-02-11T23:31:28.237+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:c0c49def:::obj22:head, version: 111'22, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c0c49def:::obj22:head@111'22, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.237+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c0c49def:::obj22:head, version: 111'22, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c0c49def:::obj22:head@111'22, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.237+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.237+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 968 ==== pg_backfill(progress 4.0 e 119/119 lb 4:c0848ef4:::obj545:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2c580 con 0x507ac00 2026-02-11T23:31:28.237+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c0c49def:::obj22:head, version: 111'22, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c0c49def:::obj22:head@111'22, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.237+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:c0848ef4:::obj545:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.237+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c0848ef4:::obj545:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.237+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.237+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c0c49def:::obj22:head, version: 111'22, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c0c49def:::obj22:head@111'22, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.237+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:c0c49def:::obj22:head, version: 111'22, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c0c49def:::obj22:head@111'22, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 119/117 [PushOp(4:c0c49def:::obj22:head, version: 111'22, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c0c49def:::obj22:head@111'22, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:bf770068:::obj268:head local-lis/les=117/118 n=463 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.237+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.238+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c0848ef4:::obj545:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.238+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.238+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bf770068:::obj268:head local-lis/les=117/118 n=463 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:c0c49def:::obj22:head, version: 111'22, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c0c49def:::obj22:head@111'22, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.238+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bf770068:::obj268:head local-lis/les=117/118 n=463 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:c0c49def:::obj22:head@111'22, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.238+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bf770068:::obj268:head local-lis/les=117/118 n=463 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:c0c49def:::obj22:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.238+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bf770068:::obj268:head local-lis/les=117/118 n=463 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:c0c49def:::obj22:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.238+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bf770068:::obj268:head local-lis/les=117/118 n=463 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:c0c49def:::obj22:head 2026-02-11T23:31:28.238+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bf770068:::obj268:head local-lis/les=117/118 n=463 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:c0c49def:::obj22:head v 111'22 2026-02-11T23:31:28.238+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bf770068:::obj268:head local-lis/les=117/118 n=463 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.238+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.238+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:c0c49def:::obj22:head, version: 111'22, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c0c49def:::obj22:head@111'22, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000080 2026-02-11T23:31:28.238+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.238+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.238+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c0848ef4:::obj545:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.238+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:c0848ef4:::obj545:head) v3 prio 15 cost 0 latency 0.000098 pg_backfill(progress 4.0 e 119/119 lb 4:c0848ef4:::obj545:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:bf770068:::obj268:head local-lis/les=117/118 n=463 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.238+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.238+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bf770068:::obj268:head local-lis/les=117/118 n=463 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:c0848ef4:::obj545:head) v3 2026-02-11T23:31:28.238+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:bf770068:::obj268:head local-lis/les=117/118 n=463 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:c0848ef4:::obj545:head) 2026-02-11T23:31:28.238+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.238+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:c0848ef4:::obj545:head) v3 finish latency 0.000015 2026-02-11T23:31:28.238+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c0848ef4:::obj545:head local-lis/les=117/118 n=464 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.238+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.238+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.238+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c0848ef4:::obj545:head local-lis/les=117/118 n=464 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.238+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.238+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.238+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.238+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x671c780 con 0x507ac00 2026-02-11T23:31:28.238+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:c0c49def:::obj22:head)]) -- 0x6738a80 con 0x507ac00 2026-02-11T23:31:28.238+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.238+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 969 ==== MOSDPGPush(4.0 119/117 [PushOp(4:c138e0af:::obj576:head, version: 111'576, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c138e0af:::obj576:head@111'576, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x555d040 con 0x507ac00 2026-02-11T23:31:28.238+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:c138e0af:::obj576:head, version: 111'576, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c138e0af:::obj576:head@111'576, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.238+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c138e0af:::obj576:head, version: 111'576, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c138e0af:::obj576:head@111'576, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.238+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.238+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c138e0af:::obj576:head, version: 111'576, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c138e0af:::obj576:head@111'576, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.238+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 970 ==== pg_backfill(progress 4.0 e 119/119 lb 4:c0c49def:::obj22:head) ==== 1061+0+0 (unknown 0 0 0) 0x6177b80 con 0x507ac00 2026-02-11T23:31:28.238+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.238+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:c0c49def:::obj22:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.238+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c138e0af:::obj576:head, version: 111'576, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c138e0af:::obj576:head@111'576, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.238+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c0c49def:::obj22:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.238+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.238+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:c138e0af:::obj576:head, version: 111'576, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c138e0af:::obj576:head@111'576, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 119/117 [PushOp(4:c138e0af:::obj576:head, version: 111'576, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c138e0af:::obj576:head@111'576, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:c0848ef4:::obj545:head local-lis/les=117/118 n=464 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.238+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c0c49def:::obj22:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.238+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.238+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c0848ef4:::obj545:head local-lis/les=117/118 n=464 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:c138e0af:::obj576:head, version: 111'576, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c138e0af:::obj576:head@111'576, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.238+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c0848ef4:::obj545:head local-lis/les=117/118 n=464 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:c138e0af:::obj576:head@111'576, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.238+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c0848ef4:::obj545:head local-lis/les=117/118 n=464 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:c138e0af:::obj576:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.238+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c0848ef4:::obj545:head local-lis/les=117/118 n=464 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:c138e0af:::obj576:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.238+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c0848ef4:::obj545:head local-lis/les=117/118 n=464 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:c138e0af:::obj576:head 2026-02-11T23:31:28.238+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c0848ef4:::obj545:head local-lis/les=117/118 n=464 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:c138e0af:::obj576:head v 111'576 2026-02-11T23:31:28.238+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c0848ef4:::obj545:head local-lis/les=117/118 n=464 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.238+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.238+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:c138e0af:::obj576:head, version: 111'576, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c138e0af:::obj576:head@111'576, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:28.238+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.238+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.238+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c0c49def:::obj22:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.238+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:c0c49def:::obj22:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 4.0 e 119/119 lb 4:c0c49def:::obj22:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:c0848ef4:::obj545:head local-lis/les=117/118 n=464 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.238+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.238+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c0848ef4:::obj545:head local-lis/les=117/118 n=464 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:c0c49def:::obj22:head) v3 2026-02-11T23:31:28.238+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c0848ef4:::obj545:head local-lis/les=117/118 n=464 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:c0c49def:::obj22:head) 2026-02-11T23:31:28.238+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.238+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:c0c49def:::obj22:head) v3 finish latency 0.000017 2026-02-11T23:31:28.238+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c0c49def:::obj22:head local-lis/les=117/118 n=465 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.238+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.238+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.238+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c0c49def:::obj22:head local-lis/les=117/118 n=465 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.238+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.238+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.238+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.238+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x642d800 con 0x507ac00 2026-02-11T23:31:28.238+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:c138e0af:::obj576:head)]) -- 0x6738c00 con 0x507ac00 2026-02-11T23:31:28.238+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.239+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 971 ==== MOSDPGPush(4.0 119/117 [PushOp(4:c19e5add:::obj152:head, version: 111'152, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c19e5add:::obj152:head@111'152, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x554e680 con 0x507ac00 2026-02-11T23:31:28.239+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:c19e5add:::obj152:head, version: 111'152, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c19e5add:::obj152:head@111'152, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.239+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c19e5add:::obj152:head, version: 111'152, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c19e5add:::obj152:head@111'152, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.239+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.239+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 972 ==== pg_backfill(progress 4.0 e 119/119 lb 4:c138e0af:::obj576:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177600 con 0x507ac00 2026-02-11T23:31:28.239+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c19e5add:::obj152:head, version: 111'152, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c19e5add:::obj152:head@111'152, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.239+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:c138e0af:::obj576:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.239+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c138e0af:::obj576:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.239+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.239+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c19e5add:::obj152:head, version: 111'152, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c19e5add:::obj152:head@111'152, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.239+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:c19e5add:::obj152:head, version: 111'152, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c19e5add:::obj152:head@111'152, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 119/117 [PushOp(4:c19e5add:::obj152:head, version: 111'152, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c19e5add:::obj152:head@111'152, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:c0c49def:::obj22:head local-lis/les=117/118 n=465 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.239+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.239+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.239+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c138e0af:::obj576:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.239+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c0c49def:::obj22:head local-lis/les=117/118 n=465 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:c19e5add:::obj152:head, version: 111'152, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c19e5add:::obj152:head@111'152, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.239+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c0c49def:::obj22:head local-lis/les=117/118 n=465 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:c19e5add:::obj152:head@111'152, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.239+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c0c49def:::obj22:head local-lis/les=117/118 n=465 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:c19e5add:::obj152:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.239+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c0c49def:::obj22:head local-lis/les=117/118 n=465 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:c19e5add:::obj152:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.239+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c0c49def:::obj22:head local-lis/les=117/118 n=465 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:c19e5add:::obj152:head 2026-02-11T23:31:28.239+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c0c49def:::obj22:head local-lis/les=117/118 n=465 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:c19e5add:::obj152:head v 111'152 2026-02-11T23:31:28.239+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c0c49def:::obj22:head local-lis/les=117/118 n=465 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.239+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.239+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:c19e5add:::obj152:head, version: 111'152, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c19e5add:::obj152:head@111'152, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:28.239+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.239+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.239+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c138e0af:::obj576:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.239+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:c138e0af:::obj576:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 119/119 lb 4:c138e0af:::obj576:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:c0c49def:::obj22:head local-lis/les=117/118 n=465 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.239+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.239+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c0c49def:::obj22:head local-lis/les=117/118 n=465 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:c138e0af:::obj576:head) v3 2026-02-11T23:31:28.239+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c0c49def:::obj22:head local-lis/les=117/118 n=465 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:c138e0af:::obj576:head) 2026-02-11T23:31:28.239+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.239+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:c138e0af:::obj576:head) v3 finish latency 0.000017 2026-02-11T23:31:28.239+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c138e0af:::obj576:head local-lis/les=117/118 n=466 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.239+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.239+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.239+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c138e0af:::obj576:head local-lis/les=117/118 n=466 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.239+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.239+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.239+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.239+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66d8480 con 0x507ac00 2026-02-11T23:31:28.239+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:c19e5add:::obj152:head)]) -- 0x6738f00 con 0x507ac00 2026-02-11T23:31:28.239+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.239+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 973 ==== MOSDPGPush(4.0 119/117 [PushOp(4:c275c16a:::obj32:head, version: 111'32, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c275c16a:::obj32:head@111'32, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x3815520 con 0x507ac00 2026-02-11T23:31:28.239+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:c275c16a:::obj32:head, version: 111'32, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c275c16a:::obj32:head@111'32, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 119 2026-02-11T23:31:28.239+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c275c16a:::obj32:head, version: 111'32, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c275c16a:::obj32:head@111'32, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.239+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.239+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c275c16a:::obj32:head, version: 111'32, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c275c16a:::obj32:head@111'32, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.239+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 974 ==== pg_backfill(progress 4.0 e 119/119 lb 4:c19e5add:::obj152:head) ==== 1062+0+0 (unknown 0 0 0) 0x5985700 con 0x507ac00 2026-02-11T23:31:28.239+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.239+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:c19e5add:::obj152:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.239+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c275c16a:::obj32:head, version: 111'32, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c275c16a:::obj32:head@111'32, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.239+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c19e5add:::obj152:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.239+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:c275c16a:::obj32:head, version: 111'32, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c275c16a:::obj32:head@111'32, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 119/117 [PushOp(4:c275c16a:::obj32:head, version: 111'32, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c275c16a:::obj32:head@111'32, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:c138e0af:::obj576:head local-lis/les=117/118 n=466 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.239+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.239+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.239+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c19e5add:::obj152:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.239+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c138e0af:::obj576:head local-lis/les=117/118 n=466 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:c275c16a:::obj32:head, version: 111'32, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c275c16a:::obj32:head@111'32, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.239+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c138e0af:::obj576:head local-lis/les=117/118 n=466 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:c275c16a:::obj32:head@111'32, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.239+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c138e0af:::obj576:head local-lis/les=117/118 n=466 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:c275c16a:::obj32:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.239+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c138e0af:::obj576:head local-lis/les=117/118 n=466 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:c275c16a:::obj32:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.239+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c138e0af:::obj576:head local-lis/les=117/118 n=466 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:c275c16a:::obj32:head 2026-02-11T23:31:28.239+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c138e0af:::obj576:head local-lis/les=117/118 n=466 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:c275c16a:::obj32:head v 111'32 2026-02-11T23:31:28.239+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c138e0af:::obj576:head local-lis/les=117/118 n=466 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.239+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.239+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:c275c16a:::obj32:head, version: 111'32, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c275c16a:::obj32:head@111'32, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:28.239+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.239+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.239+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c19e5add:::obj152:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.239+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:c19e5add:::obj152:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 4.0 e 119/119 lb 4:c19e5add:::obj152:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:c138e0af:::obj576:head local-lis/les=117/118 n=466 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.239+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.239+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c138e0af:::obj576:head local-lis/les=117/118 n=466 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:c19e5add:::obj152:head) v3 2026-02-11T23:31:28.239+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c138e0af:::obj576:head local-lis/les=117/118 n=466 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:c19e5add:::obj152:head) 2026-02-11T23:31:28.239+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.239+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:c19e5add:::obj152:head) v3 finish latency 0.000014 2026-02-11T23:31:28.239+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c19e5add:::obj152:head local-lis/les=117/118 n=467 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.239+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.239+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.239+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c19e5add:::obj152:head local-lis/les=117/118 n=467 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.239+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.239+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.239+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.239+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x642d500 con 0x507ac00 2026-02-11T23:31:28.239+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:c275c16a:::obj32:head)]) -- 0x6739200 con 0x507ac00 2026-02-11T23:31:28.239+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.240+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 975 ==== MOSDPGPush(4.0 119/117 [PushOp(4:c290fff4:::obj320:head, version: 111'320, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c290fff4:::obj320:head@111'320, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56171e0 con 0x507ac00 2026-02-11T23:31:28.240+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:c290fff4:::obj320:head, version: 111'320, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c290fff4:::obj320:head@111'320, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.240+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c290fff4:::obj320:head, version: 111'320, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c290fff4:::obj320:head@111'320, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.240+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.240+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c290fff4:::obj320:head, version: 111'320, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c290fff4:::obj320:head@111'320, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.240+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 976 ==== pg_backfill(progress 4.0 e 119/119 lb 4:c275c16a:::obj32:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d9a680 con 0x507ac00 2026-02-11T23:31:28.240+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:c275c16a:::obj32:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.240+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.240+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c290fff4:::obj320:head, version: 111'320, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c290fff4:::obj320:head@111'320, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.240+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c275c16a:::obj32:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.240+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.240+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:c290fff4:::obj320:head, version: 111'320, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c290fff4:::obj320:head@111'320, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 119/117 [PushOp(4:c290fff4:::obj320:head, version: 111'320, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c290fff4:::obj320:head@111'320, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:c19e5add:::obj152:head local-lis/les=117/118 n=467 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.240+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.240+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c275c16a:::obj32:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.240+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c19e5add:::obj152:head local-lis/les=117/118 n=467 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:c290fff4:::obj320:head, version: 111'320, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c290fff4:::obj320:head@111'320, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.240+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c19e5add:::obj152:head local-lis/les=117/118 n=467 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:c290fff4:::obj320:head@111'320, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.240+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c19e5add:::obj152:head local-lis/les=117/118 n=467 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:c290fff4:::obj320:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.240+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c19e5add:::obj152:head local-lis/les=117/118 n=467 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:c290fff4:::obj320:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.240+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c19e5add:::obj152:head local-lis/les=117/118 n=467 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:c290fff4:::obj320:head 2026-02-11T23:31:28.240+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c19e5add:::obj152:head local-lis/les=117/118 n=467 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:c290fff4:::obj320:head v 111'320 2026-02-11T23:31:28.240+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c19e5add:::obj152:head local-lis/les=117/118 n=467 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.240+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.240+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:c290fff4:::obj320:head, version: 111'320, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c290fff4:::obj320:head@111'320, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:28.240+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.240+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.240+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c275c16a:::obj32:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.240+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:c275c16a:::obj32:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 4.0 e 119/119 lb 4:c275c16a:::obj32:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:c19e5add:::obj152:head local-lis/les=117/118 n=467 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.240+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.240+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c19e5add:::obj152:head local-lis/les=117/118 n=467 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:c275c16a:::obj32:head) v3 2026-02-11T23:31:28.240+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c19e5add:::obj152:head local-lis/les=117/118 n=467 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:c275c16a:::obj32:head) 2026-02-11T23:31:28.240+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.240+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:c275c16a:::obj32:head) v3 finish latency 0.000016 2026-02-11T23:31:28.240+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c275c16a:::obj32:head local-lis/les=117/118 n=468 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.240+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.240+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.240+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c275c16a:::obj32:head local-lis/les=117/118 n=468 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.240+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.240+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.240+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.240+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66d9200 con 0x507ac00 2026-02-11T23:31:28.240+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:c290fff4:::obj320:head)]) -- 0x6739380 con 0x507ac00 2026-02-11T23:31:28.240+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.240+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 977 ==== MOSDPGPush(4.0 119/117 [PushOp(4:c3033278:::obj208:head, version: 111'208, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3033278:::obj208:head@111'208, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56cad00 con 0x507ac00 2026-02-11T23:31:28.240+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:c3033278:::obj208:head, version: 111'208, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3033278:::obj208:head@111'208, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.240+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c3033278:::obj208:head, version: 111'208, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3033278:::obj208:head@111'208, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.240+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.240+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c3033278:::obj208:head, version: 111'208, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3033278:::obj208:head@111'208, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.240+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 978 ==== pg_backfill(progress 4.0 e 119/119 lb 4:c290fff4:::obj320:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176000 con 0x507ac00 2026-02-11T23:31:28.240+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.240+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:c290fff4:::obj320:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.240+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c3033278:::obj208:head, version: 111'208, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3033278:::obj208:head@111'208, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.240+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c290fff4:::obj320:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.240+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.240+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:c3033278:::obj208:head, version: 111'208, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3033278:::obj208:head@111'208, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 119/117 [PushOp(4:c3033278:::obj208:head, version: 111'208, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3033278:::obj208:head@111'208, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:c275c16a:::obj32:head local-lis/les=117/118 n=468 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.240+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c290fff4:::obj320:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.240+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.240+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c275c16a:::obj32:head local-lis/les=117/118 n=468 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:c3033278:::obj208:head, version: 111'208, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3033278:::obj208:head@111'208, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.240+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c275c16a:::obj32:head local-lis/les=117/118 n=468 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:c3033278:::obj208:head@111'208, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.240+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c275c16a:::obj32:head local-lis/les=117/118 n=468 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:c3033278:::obj208:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.240+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c275c16a:::obj32:head local-lis/les=117/118 n=468 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:c3033278:::obj208:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.240+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c275c16a:::obj32:head local-lis/les=117/118 n=468 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:c3033278:::obj208:head 2026-02-11T23:31:28.240+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c275c16a:::obj32:head local-lis/les=117/118 n=468 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:c3033278:::obj208:head v 111'208 2026-02-11T23:31:28.240+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c275c16a:::obj32:head local-lis/les=117/118 n=468 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.240+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.240+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:c3033278:::obj208:head, version: 111'208, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3033278:::obj208:head@111'208, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:28.240+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.240+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.240+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c290fff4:::obj320:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.240+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:c290fff4:::obj320:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 4.0 e 119/119 lb 4:c290fff4:::obj320:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:c275c16a:::obj32:head local-lis/les=117/118 n=468 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.240+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.240+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c275c16a:::obj32:head local-lis/les=117/118 n=468 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:c290fff4:::obj320:head) v3 2026-02-11T23:31:28.240+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c275c16a:::obj32:head local-lis/les=117/118 n=468 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:c290fff4:::obj320:head) 2026-02-11T23:31:28.240+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.240+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:c290fff4:::obj320:head) v3 finish latency 0.000014 2026-02-11T23:31:28.240+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c290fff4:::obj320:head local-lis/les=117/118 n=469 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.240+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.241+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.241+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c290fff4:::obj320:head local-lis/les=117/118 n=469 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.241+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.241+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.241+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.241+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x642de00 con 0x507ac00 2026-02-11T23:31:28.241+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:c3033278:::obj208:head)]) -- 0x6739680 con 0x507ac00 2026-02-11T23:31:28.241+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.241+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 979 ==== MOSDPGPush(4.0 119/117 [PushOp(4:c3ae426e:::obj537:head, version: 111'537, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3ae426e:::obj537:head@111'537, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db0b60 con 0x507ac00 2026-02-11T23:31:28.241+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:c3ae426e:::obj537:head, version: 111'537, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3ae426e:::obj537:head@111'537, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.241+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c3ae426e:::obj537:head, version: 111'537, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3ae426e:::obj537:head@111'537, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.241+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.241+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c3ae426e:::obj537:head, version: 111'537, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3ae426e:::obj537:head@111'537, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.241+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 980 ==== pg_backfill(progress 4.0 e 119/119 lb 4:c3033278:::obj208:head) ==== 1062+0+0 (unknown 0 0 0) 0x6179700 con 0x507ac00 2026-02-11T23:31:28.241+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:c3033278:::obj208:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.241+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.241+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c3ae426e:::obj537:head, version: 111'537, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3ae426e:::obj537:head@111'537, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.241+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c3033278:::obj208:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.241+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:c3ae426e:::obj537:head, version: 111'537, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3ae426e:::obj537:head@111'537, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 119/117 [PushOp(4:c3ae426e:::obj537:head, version: 111'537, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3ae426e:::obj537:head@111'537, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:c290fff4:::obj320:head local-lis/les=117/118 n=469 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.241+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.241+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.241+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c3033278:::obj208:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.241+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c290fff4:::obj320:head local-lis/les=117/118 n=469 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:c3ae426e:::obj537:head, version: 111'537, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3ae426e:::obj537:head@111'537, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.241+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c290fff4:::obj320:head local-lis/les=117/118 n=469 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:c3ae426e:::obj537:head@111'537, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.241+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c290fff4:::obj320:head local-lis/les=117/118 n=469 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:c3ae426e:::obj537:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.241+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c290fff4:::obj320:head local-lis/les=117/118 n=469 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:c3ae426e:::obj537:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.241+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c290fff4:::obj320:head local-lis/les=117/118 n=469 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:c3ae426e:::obj537:head 2026-02-11T23:31:28.241+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c290fff4:::obj320:head local-lis/les=117/118 n=469 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:c3ae426e:::obj537:head v 111'537 2026-02-11T23:31:28.241+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c290fff4:::obj320:head local-lis/les=117/118 n=469 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.241+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.241+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:c3ae426e:::obj537:head, version: 111'537, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3ae426e:::obj537:head@111'537, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:28.241+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.241+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.241+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c3033278:::obj208:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.241+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:c3033278:::obj208:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 4.0 e 119/119 lb 4:c3033278:::obj208:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:c290fff4:::obj320:head local-lis/les=117/118 n=469 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.241+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.241+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c290fff4:::obj320:head local-lis/les=117/118 n=469 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:c3033278:::obj208:head) v3 2026-02-11T23:31:28.241+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c290fff4:::obj320:head local-lis/les=117/118 n=469 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:c3033278:::obj208:head) 2026-02-11T23:31:28.241+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.241+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:c3033278:::obj208:head) v3 finish latency 0.000021 2026-02-11T23:31:28.241+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3033278:::obj208:head local-lis/les=117/118 n=470 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.241+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.241+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.241+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3033278:::obj208:head local-lis/les=117/118 n=470 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.241+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.241+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.241+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.241+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66d9500 con 0x507ac00 2026-02-11T23:31:28.241+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:c3ae426e:::obj537:head)]) -- 0x6739980 con 0x507ac00 2026-02-11T23:31:28.241+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.241+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 981 ==== MOSDPGPush(4.0 119/117 [PushOp(4:c3ba172d:::obj29:head, version: 111'29, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3ba172d:::obj29:head@111'29, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x56a2d00 con 0x507ac00 2026-02-11T23:31:28.241+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:c3ba172d:::obj29:head, version: 111'29, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3ba172d:::obj29:head@111'29, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.241+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c3ba172d:::obj29:head, version: 111'29, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3ba172d:::obj29:head@111'29, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.241+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.241+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c3ba172d:::obj29:head, version: 111'29, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3ba172d:::obj29:head@111'29, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.241+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.241+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c3ba172d:::obj29:head, version: 111'29, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3ba172d:::obj29:head@111'29, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.241+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 982 ==== pg_backfill(progress 4.0 e 119/119 lb 4:c3ae426e:::obj537:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2cb00 con 0x507ac00 2026-02-11T23:31:28.241+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:c3ae426e:::obj537:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 119 2026-02-11T23:31:28.241+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.241+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:c3ba172d:::obj29:head, version: 111'29, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3ba172d:::obj29:head@111'29, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 119/117 [PushOp(4:c3ba172d:::obj29:head, version: 111'29, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3ba172d:::obj29:head@111'29, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:c3033278:::obj208:head local-lis/les=117/118 n=470 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.241+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c3ae426e:::obj537:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.241+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.241+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3033278:::obj208:head local-lis/les=117/118 n=470 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:c3ba172d:::obj29:head, version: 111'29, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3ba172d:::obj29:head@111'29, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.241+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3033278:::obj208:head local-lis/les=117/118 n=470 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:c3ba172d:::obj29:head@111'29, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.241+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.241+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c3ae426e:::obj537:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.241+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3033278:::obj208:head local-lis/les=117/118 n=470 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:c3ba172d:::obj29:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.241+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3033278:::obj208:head local-lis/les=117/118 n=470 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:c3ba172d:::obj29:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.241+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3033278:::obj208:head local-lis/les=117/118 n=470 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:c3ba172d:::obj29:head 2026-02-11T23:31:28.241+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3033278:::obj208:head local-lis/les=117/118 n=470 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:c3ba172d:::obj29:head v 111'29 2026-02-11T23:31:28.241+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3033278:::obj208:head local-lis/les=117/118 n=470 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.241+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.241+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:c3ba172d:::obj29:head, version: 111'29, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3ba172d:::obj29:head@111'29, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:28.241+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.241+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.241+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c3ae426e:::obj537:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.241+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:c3ae426e:::obj537:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 4.0 e 119/119 lb 4:c3ae426e:::obj537:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:c3033278:::obj208:head local-lis/les=117/118 n=470 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.241+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.241+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3033278:::obj208:head local-lis/les=117/118 n=470 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:c3ae426e:::obj537:head) v3 2026-02-11T23:31:28.241+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3033278:::obj208:head local-lis/les=117/118 n=470 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:c3ae426e:::obj537:head) 2026-02-11T23:31:28.241+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.241+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:c3ae426e:::obj537:head) v3 finish latency 0.000015 2026-02-11T23:31:28.241+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3ae426e:::obj537:head local-lis/les=117/118 n=471 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.241+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.242+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.242+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3ae426e:::obj537:head local-lis/les=117/118 n=471 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.242+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.242+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.242+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.242+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x5d55200 con 0x507ac00 2026-02-11T23:31:28.242+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:c3ba172d:::obj29:head)]) -- 0x6739c80 con 0x507ac00 2026-02-11T23:31:28.242+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.242+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 983 ==== MOSDPGPush(4.0 119/117 [PushOp(4:c3bae056:::obj148:head, version: 111'148, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3bae056:::obj148:head@111'148, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f0ea0 con 0x507ac00 2026-02-11T23:31:28.242+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:c3bae056:::obj148:head, version: 111'148, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3bae056:::obj148:head@111'148, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.242+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c3bae056:::obj148:head, version: 111'148, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3bae056:::obj148:head@111'148, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.242+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.242+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c3bae056:::obj148:head, version: 111'148, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3bae056:::obj148:head@111'148, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.242+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 984 ==== pg_backfill(progress 4.0 e 119/119 lb 4:c3ba172d:::obj29:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d2c000 con 0x507ac00 2026-02-11T23:31:28.242+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.242+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:c3ba172d:::obj29:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.242+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c3bae056:::obj148:head, version: 111'148, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3bae056:::obj148:head@111'148, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.242+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c3ba172d:::obj29:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.242+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.242+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:c3bae056:::obj148:head, version: 111'148, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3bae056:::obj148:head@111'148, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 119/117 [PushOp(4:c3bae056:::obj148:head, version: 111'148, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3bae056:::obj148:head@111'148, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:c3ae426e:::obj537:head local-lis/les=117/118 n=471 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.242+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c3ba172d:::obj29:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.242+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.242+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3ae426e:::obj537:head local-lis/les=117/118 n=471 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:c3bae056:::obj148:head, version: 111'148, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3bae056:::obj148:head@111'148, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.242+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3ae426e:::obj537:head local-lis/les=117/118 n=471 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:c3bae056:::obj148:head@111'148, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.242+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3ae426e:::obj537:head local-lis/les=117/118 n=471 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:c3bae056:::obj148:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.242+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3ae426e:::obj537:head local-lis/les=117/118 n=471 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:c3bae056:::obj148:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.242+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3ae426e:::obj537:head local-lis/les=117/118 n=471 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:c3bae056:::obj148:head 2026-02-11T23:31:28.242+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3ae426e:::obj537:head local-lis/les=117/118 n=471 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:c3bae056:::obj148:head v 111'148 2026-02-11T23:31:28.242+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3ae426e:::obj537:head local-lis/les=117/118 n=471 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.242+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.242+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:c3bae056:::obj148:head, version: 111'148, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3bae056:::obj148:head@111'148, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:28.242+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.242+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.242+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c3ba172d:::obj29:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.242+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:c3ba172d:::obj29:head) v3 prio 15 cost 0 latency 0.000060 pg_backfill(progress 4.0 e 119/119 lb 4:c3ba172d:::obj29:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:c3ae426e:::obj537:head local-lis/les=117/118 n=471 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.242+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.242+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3ae426e:::obj537:head local-lis/les=117/118 n=471 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:c3ba172d:::obj29:head) v3 2026-02-11T23:31:28.242+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3ae426e:::obj537:head local-lis/les=117/118 n=471 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:c3ba172d:::obj29:head) 2026-02-11T23:31:28.242+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.242+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:c3ba172d:::obj29:head) v3 finish latency 0.000015 2026-02-11T23:31:28.242+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3ba172d:::obj29:head local-lis/les=117/118 n=472 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.242+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.242+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.242+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3ba172d:::obj29:head local-lis/les=117/118 n=472 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.242+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.242+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.242+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.242+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x671d080 con 0x507ac00 2026-02-11T23:31:28.242+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:c3bae056:::obj148:head)]) -- 0x6739e00 con 0x507ac00 2026-02-11T23:31:28.242+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.242+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 985 ==== MOSDPGPush(4.0 119/117 [PushOp(4:c3bcd33c:::obj192:head, version: 111'192, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3bcd33c:::obj192:head@111'192, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a3040 con 0x507ac00 2026-02-11T23:31:28.242+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:c3bcd33c:::obj192:head, version: 111'192, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3bcd33c:::obj192:head@111'192, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.242+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c3bcd33c:::obj192:head, version: 111'192, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3bcd33c:::obj192:head@111'192, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.242+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.242+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 986 ==== pg_backfill(progress 4.0 e 119/119 lb 4:c3bae056:::obj148:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e100 con 0x507ac00 2026-02-11T23:31:28.242+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c3bcd33c:::obj192:head, version: 111'192, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3bcd33c:::obj192:head@111'192, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.242+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:c3bae056:::obj148:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.242+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.242+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c3bae056:::obj148:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.242+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c3bcd33c:::obj192:head, version: 111'192, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3bcd33c:::obj192:head@111'192, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.242+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:c3bcd33c:::obj192:head, version: 111'192, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3bcd33c:::obj192:head@111'192, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 119/117 [PushOp(4:c3bcd33c:::obj192:head, version: 111'192, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3bcd33c:::obj192:head@111'192, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:c3ba172d:::obj29:head local-lis/les=117/118 n=472 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.242+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.242+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.242+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c3bae056:::obj148:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.242+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3ba172d:::obj29:head local-lis/les=117/118 n=472 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:c3bcd33c:::obj192:head, version: 111'192, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3bcd33c:::obj192:head@111'192, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.242+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3ba172d:::obj29:head local-lis/les=117/118 n=472 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:c3bcd33c:::obj192:head@111'192, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.242+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3ba172d:::obj29:head local-lis/les=117/118 n=472 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:c3bcd33c:::obj192:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.242+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3ba172d:::obj29:head local-lis/les=117/118 n=472 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:c3bcd33c:::obj192:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.242+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3ba172d:::obj29:head local-lis/les=117/118 n=472 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:c3bcd33c:::obj192:head 2026-02-11T23:31:28.242+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3ba172d:::obj29:head local-lis/les=117/118 n=472 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:c3bcd33c:::obj192:head v 111'192 2026-02-11T23:31:28.242+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3ba172d:::obj29:head local-lis/les=117/118 n=472 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.242+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.242+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:c3bcd33c:::obj192:head, version: 111'192, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3bcd33c:::obj192:head@111'192, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:28.242+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.242+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.242+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c3bae056:::obj148:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.242+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:c3bae056:::obj148:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 4.0 e 119/119 lb 4:c3bae056:::obj148:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:c3ba172d:::obj29:head local-lis/les=117/118 n=472 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.242+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.242+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3ba172d:::obj29:head local-lis/les=117/118 n=472 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:c3bae056:::obj148:head) v3 2026-02-11T23:31:28.242+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3ba172d:::obj29:head local-lis/les=117/118 n=472 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:c3bae056:::obj148:head) 2026-02-11T23:31:28.242+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.242+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:c3bae056:::obj148:head) v3 finish latency 0.000015 2026-02-11T23:31:28.242+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3bae056:::obj148:head local-lis/les=117/118 n=473 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.242+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.243+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.243+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3bae056:::obj148:head local-lis/les=117/118 n=473 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.243+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.243+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.243+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.243+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x640fb00 con 0x507ac00 2026-02-11T23:31:28.243+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:c3bcd33c:::obj192:head)]) -- 0x642d500 con 0x507ac00 2026-02-11T23:31:28.243+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.243+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 987 ==== MOSDPGPush(4.0 119/117 [PushOp(4:c41d6f6c:::obj298:head, version: 111'298, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c41d6f6c:::obj298:head@111'298, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3814d00 con 0x507ac00 2026-02-11T23:31:28.243+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:c41d6f6c:::obj298:head, version: 111'298, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c41d6f6c:::obj298:head@111'298, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.243+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c41d6f6c:::obj298:head, version: 111'298, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c41d6f6c:::obj298:head@111'298, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.243+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.243+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c41d6f6c:::obj298:head, version: 111'298, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c41d6f6c:::obj298:head@111'298, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.243+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 988 ==== pg_backfill(progress 4.0 e 119/119 lb 4:c3bcd33c:::obj192:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176580 con 0x507ac00 2026-02-11T23:31:28.243+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:c3bcd33c:::obj192:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.243+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.243+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c41d6f6c:::obj298:head, version: 111'298, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c41d6f6c:::obj298:head@111'298, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.243+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c3bcd33c:::obj192:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.243+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.243+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:c41d6f6c:::obj298:head, version: 111'298, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c41d6f6c:::obj298:head@111'298, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 119/117 [PushOp(4:c41d6f6c:::obj298:head, version: 111'298, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c41d6f6c:::obj298:head@111'298, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:c3bae056:::obj148:head local-lis/les=117/118 n=473 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.243+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c3bcd33c:::obj192:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.243+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.243+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3bae056:::obj148:head local-lis/les=117/118 n=473 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:c41d6f6c:::obj298:head, version: 111'298, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c41d6f6c:::obj298:head@111'298, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.243+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3bae056:::obj148:head local-lis/les=117/118 n=473 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:c41d6f6c:::obj298:head@111'298, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.243+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3bae056:::obj148:head local-lis/les=117/118 n=473 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:c41d6f6c:::obj298:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.243+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3bae056:::obj148:head local-lis/les=117/118 n=473 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:c41d6f6c:::obj298:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.243+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3bae056:::obj148:head local-lis/les=117/118 n=473 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:c41d6f6c:::obj298:head 2026-02-11T23:31:28.243+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3bae056:::obj148:head local-lis/les=117/118 n=473 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:c41d6f6c:::obj298:head v 111'298 2026-02-11T23:31:28.243+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3bae056:::obj148:head local-lis/les=117/118 n=473 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.243+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.243+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:c41d6f6c:::obj298:head, version: 111'298, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c41d6f6c:::obj298:head@111'298, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:28.243+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.243+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.243+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c3bcd33c:::obj192:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.243+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:c3bcd33c:::obj192:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 4.0 e 119/119 lb 4:c3bcd33c:::obj192:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:c3bae056:::obj148:head local-lis/les=117/118 n=473 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.243+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.243+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3bae056:::obj148:head local-lis/les=117/118 n=473 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:c3bcd33c:::obj192:head) v3 2026-02-11T23:31:28.243+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3bae056:::obj148:head local-lis/les=117/118 n=473 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:c3bcd33c:::obj192:head) 2026-02-11T23:31:28.243+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.243+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:c3bcd33c:::obj192:head) v3 finish latency 0.000014 2026-02-11T23:31:28.243+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3bcd33c:::obj192:head local-lis/les=117/118 n=474 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.243+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.243+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.243+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3bcd33c:::obj192:head local-lis/les=117/118 n=474 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.243+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.243+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.243+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.243+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x671dc80 con 0x507ac00 2026-02-11T23:31:28.243+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:c41d6f6c:::obj298:head)]) -- 0x66d8480 con 0x507ac00 2026-02-11T23:31:28.243+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.243+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 989 ==== MOSDPGPush(4.0 119/117 [PushOp(4:c4f18846:::obj81:head, version: 111'81, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c4f18846:::obj81:head@111'81, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x559c1a0 con 0x507ac00 2026-02-11T23:31:28.243+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:c4f18846:::obj81:head, version: 111'81, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c4f18846:::obj81:head@111'81, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.243+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c4f18846:::obj81:head, version: 111'81, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c4f18846:::obj81:head@111'81, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.243+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.243+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c4f18846:::obj81:head, version: 111'81, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c4f18846:::obj81:head@111'81, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.243+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 990 ==== pg_backfill(progress 4.0 e 119/119 lb 4:c41d6f6c:::obj298:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e680 con 0x507ac00 2026-02-11T23:31:28.243+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.243+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:c41d6f6c:::obj298:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.243+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c4f18846:::obj81:head, version: 111'81, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c4f18846:::obj81:head@111'81, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.243+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c41d6f6c:::obj298:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.243+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.243+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:c4f18846:::obj81:head, version: 111'81, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c4f18846:::obj81:head@111'81, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 119/117 [PushOp(4:c4f18846:::obj81:head, version: 111'81, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c4f18846:::obj81:head@111'81, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:c3bcd33c:::obj192:head local-lis/les=117/118 n=474 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.243+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c41d6f6c:::obj298:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.243+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.243+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3bcd33c:::obj192:head local-lis/les=117/118 n=474 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:c4f18846:::obj81:head, version: 111'81, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c4f18846:::obj81:head@111'81, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.243+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3bcd33c:::obj192:head local-lis/les=117/118 n=474 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:c4f18846:::obj81:head@111'81, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.243+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3bcd33c:::obj192:head local-lis/les=117/118 n=474 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:c4f18846:::obj81:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.243+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3bcd33c:::obj192:head local-lis/les=117/118 n=474 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:c4f18846:::obj81:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.243+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3bcd33c:::obj192:head local-lis/les=117/118 n=474 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:c4f18846:::obj81:head 2026-02-11T23:31:28.243+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3bcd33c:::obj192:head local-lis/les=117/118 n=474 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:c4f18846:::obj81:head v 111'81 2026-02-11T23:31:28.243+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3bcd33c:::obj192:head local-lis/les=117/118 n=474 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.243+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.243+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:c4f18846:::obj81:head, version: 111'81, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c4f18846:::obj81:head@111'81, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:28.243+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.243+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.243+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c41d6f6c:::obj298:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.243+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:c41d6f6c:::obj298:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 4.0 e 119/119 lb 4:c41d6f6c:::obj298:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:c3bcd33c:::obj192:head local-lis/les=117/118 n=474 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.243+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.243+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3bcd33c:::obj192:head local-lis/les=117/118 n=474 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:c41d6f6c:::obj298:head) v3 2026-02-11T23:31:28.243+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c3bcd33c:::obj192:head local-lis/les=117/118 n=474 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:c41d6f6c:::obj298:head) 2026-02-11T23:31:28.243+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.243+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:c41d6f6c:::obj298:head) v3 finish latency 0.000015 2026-02-11T23:31:28.243+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c41d6f6c:::obj298:head local-lis/les=117/118 n=475 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.243+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.244+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.244+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c41d6f6c:::obj298:head local-lis/les=117/118 n=475 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.244+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.244+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.244+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.244+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c9e00 con 0x507ac00 2026-02-11T23:31:28.244+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:c4f18846:::obj81:head)]) -- 0x642d800 con 0x507ac00 2026-02-11T23:31:28.244+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.244+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 991 ==== MOSDPGPush(4.0 119/117 [PushOp(4:c52c9666:::obj5:head, version: 111'5, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c52c9666:::obj5:head@111'5, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 4999+0+0 (unknown 0 0 0) 0x56a2d00 con 0x507ac00 2026-02-11T23:31:28.244+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:c52c9666:::obj5:head, version: 111'5, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c52c9666:::obj5:head@111'5, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.244+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c52c9666:::obj5:head, version: 111'5, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c52c9666:::obj5:head@111'5, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.244+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.244+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c52c9666:::obj5:head, version: 111'5, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c52c9666:::obj5:head@111'5, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.244+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 992 ==== pg_backfill(progress 4.0 e 119/119 lb 4:c4f18846:::obj81:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d2d080 con 0x507ac00 2026-02-11T23:31:28.244+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:c4f18846:::obj81:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.244+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.244+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c4f18846:::obj81:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.244+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c52c9666:::obj5:head, version: 111'5, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c52c9666:::obj5:head@111'5, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.244+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.244+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:c52c9666:::obj5:head, version: 111'5, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c52c9666:::obj5:head@111'5, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 119/117 [PushOp(4:c52c9666:::obj5:head, version: 111'5, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c52c9666:::obj5:head@111'5, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:c41d6f6c:::obj298:head local-lis/les=117/118 n=475 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.244+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.244+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c41d6f6c:::obj298:head local-lis/les=117/118 n=475 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:c52c9666:::obj5:head, version: 111'5, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c52c9666:::obj5:head@111'5, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.244+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c41d6f6c:::obj298:head local-lis/les=117/118 n=475 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:c52c9666:::obj5:head@111'5, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.244+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.244+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c4f18846:::obj81:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.244+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c41d6f6c:::obj298:head local-lis/les=117/118 n=475 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:c52c9666:::obj5:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.244+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c41d6f6c:::obj298:head local-lis/les=117/118 n=475 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:c52c9666:::obj5:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.244+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c41d6f6c:::obj298:head local-lis/les=117/118 n=475 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:c52c9666:::obj5:head 2026-02-11T23:31:28.244+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c41d6f6c:::obj298:head local-lis/les=117/118 n=475 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:c52c9666:::obj5:head v 111'5 2026-02-11T23:31:28.244+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c41d6f6c:::obj298:head local-lis/les=117/118 n=475 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.244+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.244+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:c52c9666:::obj5:head, version: 111'5, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c52c9666:::obj5:head@111'5, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:28.244+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.244+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.244+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c4f18846:::obj81:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.244+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:c4f18846:::obj81:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 4.0 e 119/119 lb 4:c4f18846:::obj81:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:c41d6f6c:::obj298:head local-lis/les=117/118 n=475 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.244+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.244+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c41d6f6c:::obj298:head local-lis/les=117/118 n=475 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:c4f18846:::obj81:head) v3 2026-02-11T23:31:28.244+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c41d6f6c:::obj298:head local-lis/les=117/118 n=475 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:c4f18846:::obj81:head) 2026-02-11T23:31:28.244+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.244+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:c4f18846:::obj81:head) v3 finish latency 0.000015 2026-02-11T23:31:28.244+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c4f18846:::obj81:head local-lis/les=117/118 n=476 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.244+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.244+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.244+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c4f18846:::obj81:head local-lis/les=117/118 n=476 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.244+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.244+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.244+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.244+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x671d980 con 0x507ac00 2026-02-11T23:31:28.244+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:c52c9666:::obj5:head)]) -- 0x6738a80 con 0x507ac00 2026-02-11T23:31:28.244+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.244+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 993 ==== MOSDPGPush(4.0 119/117 [PushOp(4:c57e020a:::obj395:head, version: 111'395, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c57e020a:::obj395:head@111'395, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db0b60 con 0x507ac00 2026-02-11T23:31:28.244+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:c57e020a:::obj395:head, version: 111'395, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c57e020a:::obj395:head@111'395, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.244+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c57e020a:::obj395:head, version: 111'395, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c57e020a:::obj395:head@111'395, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.244+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.244+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c57e020a:::obj395:head, version: 111'395, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c57e020a:::obj395:head@111'395, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.244+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 994 ==== pg_backfill(progress 4.0 e 119/119 lb 4:c52c9666:::obj5:head) ==== 1060+0+0 (unknown 0 0 0) 0x5d2f700 con 0x507ac00 2026-02-11T23:31:28.244+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:c52c9666:::obj5:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.244+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.244+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c57e020a:::obj395:head, version: 111'395, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c57e020a:::obj395:head@111'395, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.244+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c52c9666:::obj5:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.244+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:c57e020a:::obj395:head, version: 111'395, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c57e020a:::obj395:head@111'395, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 119/117 [PushOp(4:c57e020a:::obj395:head, version: 111'395, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c57e020a:::obj395:head@111'395, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:c4f18846:::obj81:head local-lis/les=117/118 n=476 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.244+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.244+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.244+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c4f18846:::obj81:head local-lis/les=117/118 n=476 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:c57e020a:::obj395:head, version: 111'395, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c57e020a:::obj395:head@111'395, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.244+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c52c9666:::obj5:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.244+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c4f18846:::obj81:head local-lis/les=117/118 n=476 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:c57e020a:::obj395:head@111'395, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.244+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c4f18846:::obj81:head local-lis/les=117/118 n=476 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:c57e020a:::obj395:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.244+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c4f18846:::obj81:head local-lis/les=117/118 n=476 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:c57e020a:::obj395:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.244+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c4f18846:::obj81:head local-lis/les=117/118 n=476 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:c57e020a:::obj395:head 2026-02-11T23:31:28.244+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c4f18846:::obj81:head local-lis/les=117/118 n=476 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:c57e020a:::obj395:head v 111'395 2026-02-11T23:31:28.244+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c4f18846:::obj81:head local-lis/les=117/118 n=476 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.244+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.244+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:c57e020a:::obj395:head, version: 111'395, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c57e020a:::obj395:head@111'395, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:28.244+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.244+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.244+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c52c9666:::obj5:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.244+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:c52c9666:::obj5:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 4.0 e 119/119 lb 4:c52c9666:::obj5:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:c4f18846:::obj81:head local-lis/les=117/118 n=476 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.244+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.244+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c4f18846:::obj81:head local-lis/les=117/118 n=476 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:c52c9666:::obj5:head) v3 2026-02-11T23:31:28.244+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c4f18846:::obj81:head local-lis/les=117/118 n=476 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:c52c9666:::obj5:head) 2026-02-11T23:31:28.244+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.244+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:c52c9666:::obj5:head) v3 finish latency 0.000017 2026-02-11T23:31:28.244+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c52c9666:::obj5:head local-lis/les=117/118 n=477 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.244+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.245+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.245+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c52c9666:::obj5:head local-lis/les=117/118 n=477 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.245+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.245+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.245+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.245+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x642cd80 con 0x507ac00 2026-02-11T23:31:28.245+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:c57e020a:::obj395:head)]) -- 0x671c780 con 0x507ac00 2026-02-11T23:31:28.245+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.245+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 995 ==== MOSDPGPush(4.0 119/117 [PushOp(4:c59209e1:::obj523:head, version: 111'523, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c59209e1:::obj523:head@111'523, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56cad00 con 0x507ac00 2026-02-11T23:31:28.245+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:c59209e1:::obj523:head, version: 111'523, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c59209e1:::obj523:head@111'523, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.245+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c59209e1:::obj523:head, version: 111'523, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c59209e1:::obj523:head@111'523, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.245+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.245+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c59209e1:::obj523:head, version: 111'523, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c59209e1:::obj523:head@111'523, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.245+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.245+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 996 ==== pg_backfill(progress 4.0 e 119/119 lb 4:c57e020a:::obj395:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f11600 con 0x507ac00 2026-02-11T23:31:28.245+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c59209e1:::obj523:head, version: 111'523, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c59209e1:::obj523:head@111'523, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.245+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:c57e020a:::obj395:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.245+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c57e020a:::obj395:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.245+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:c59209e1:::obj523:head, version: 111'523, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c59209e1:::obj523:head@111'523, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 119/117 [PushOp(4:c59209e1:::obj523:head, version: 111'523, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c59209e1:::obj523:head@111'523, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:c52c9666:::obj5:head local-lis/les=117/118 n=477 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.245+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.245+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c52c9666:::obj5:head local-lis/les=117/118 n=477 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:c59209e1:::obj523:head, version: 111'523, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c59209e1:::obj523:head@111'523, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.245+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.245+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c52c9666:::obj5:head local-lis/les=117/118 n=477 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:c59209e1:::obj523:head@111'523, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.245+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c57e020a:::obj395:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.245+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c52c9666:::obj5:head local-lis/les=117/118 n=477 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:c59209e1:::obj523:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.245+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c52c9666:::obj5:head local-lis/les=117/118 n=477 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:c59209e1:::obj523:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.245+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c52c9666:::obj5:head local-lis/les=117/118 n=477 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:c59209e1:::obj523:head 2026-02-11T23:31:28.245+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c52c9666:::obj5:head local-lis/les=117/118 n=477 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:c59209e1:::obj523:head v 111'523 2026-02-11T23:31:28.245+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c52c9666:::obj5:head local-lis/les=117/118 n=477 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.245+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.245+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:c59209e1:::obj523:head, version: 111'523, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c59209e1:::obj523:head@111'523, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:28.245+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.245+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.245+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c57e020a:::obj395:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.245+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:c57e020a:::obj395:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 4.0 e 119/119 lb 4:c57e020a:::obj395:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:c52c9666:::obj5:head local-lis/les=117/118 n=477 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.245+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.245+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c52c9666:::obj5:head local-lis/les=117/118 n=477 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:c57e020a:::obj395:head) v3 2026-02-11T23:31:28.245+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c52c9666:::obj5:head local-lis/les=117/118 n=477 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:c57e020a:::obj395:head) 2026-02-11T23:31:28.245+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.245+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:c57e020a:::obj395:head) v3 finish latency 0.000014 2026-02-11T23:31:28.245+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c57e020a:::obj395:head local-lis/les=117/118 n=478 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.245+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.245+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.245+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c57e020a:::obj395:head local-lis/les=117/118 n=478 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.245+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.245+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.245+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.245+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x671d680 con 0x507ac00 2026-02-11T23:31:28.245+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:c59209e1:::obj523:head)]) -- 0x640e180 con 0x507ac00 2026-02-11T23:31:28.245+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.245+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 997 ==== MOSDPGPush(4.0 119/117 [PushOp(4:c5df6474:::obj459:head, version: 111'459, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c5df6474:::obj459:head@111'459, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56171e0 con 0x507ac00 2026-02-11T23:31:28.245+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:c5df6474:::obj459:head, version: 111'459, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c5df6474:::obj459:head@111'459, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.245+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c5df6474:::obj459:head, version: 111'459, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c5df6474:::obj459:head@111'459, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.245+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.245+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c5df6474:::obj459:head, version: 111'459, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c5df6474:::obj459:head@111'459, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.245+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 998 ==== pg_backfill(progress 4.0 e 119/119 lb 4:c59209e1:::obj523:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f11080 con 0x507ac00 2026-02-11T23:31:28.245+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:c59209e1:::obj523:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.245+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.245+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c59209e1:::obj523:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.245+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c5df6474:::obj459:head, version: 111'459, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c5df6474:::obj459:head@111'459, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.245+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:c5df6474:::obj459:head, version: 111'459, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c5df6474:::obj459:head@111'459, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 119/117 [PushOp(4:c5df6474:::obj459:head, version: 111'459, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c5df6474:::obj459:head@111'459, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:c57e020a:::obj395:head local-lis/les=117/118 n=478 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.245+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.245+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.245+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c59209e1:::obj523:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.245+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c57e020a:::obj395:head local-lis/les=117/118 n=478 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:c5df6474:::obj459:head, version: 111'459, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c5df6474:::obj459:head@111'459, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.245+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c57e020a:::obj395:head local-lis/les=117/118 n=478 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:c5df6474:::obj459:head@111'459, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.245+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c57e020a:::obj395:head local-lis/les=117/118 n=478 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:c5df6474:::obj459:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.245+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c57e020a:::obj395:head local-lis/les=117/118 n=478 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:c5df6474:::obj459:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.245+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c57e020a:::obj395:head local-lis/les=117/118 n=478 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:c5df6474:::obj459:head 2026-02-11T23:31:28.245+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c57e020a:::obj395:head local-lis/les=117/118 n=478 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:c5df6474:::obj459:head v 111'459 2026-02-11T23:31:28.245+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c57e020a:::obj395:head local-lis/les=117/118 n=478 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.245+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.245+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:c5df6474:::obj459:head, version: 111'459, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c5df6474:::obj459:head@111'459, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:28.245+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.245+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.245+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c59209e1:::obj523:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.245+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:c59209e1:::obj523:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 4.0 e 119/119 lb 4:c59209e1:::obj523:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:c57e020a:::obj395:head local-lis/les=117/118 n=478 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.245+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.245+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c57e020a:::obj395:head local-lis/les=117/118 n=478 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:c59209e1:::obj523:head) v3 2026-02-11T23:31:28.245+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c57e020a:::obj395:head local-lis/les=117/118 n=478 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:c59209e1:::obj523:head) 2026-02-11T23:31:28.245+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.245+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:c59209e1:::obj523:head) v3 finish latency 0.000014 2026-02-11T23:31:28.245+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c59209e1:::obj523:head local-lis/les=117/118 n=479 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.245+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.246+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.246+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c59209e1:::obj523:head local-lis/les=117/118 n=479 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.246+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.246+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.246+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.246+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66d9080 con 0x507ac00 2026-02-11T23:31:28.246+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:c5df6474:::obj459:head)]) -- 0x671c480 con 0x507ac00 2026-02-11T23:31:28.246+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.246+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 999 ==== MOSDPGPush(4.0 119/117 [PushOp(4:c5e569a1:::obj444:head, version: 111'444, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c5e569a1:::obj444:head@111'444, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3815520 con 0x507ac00 2026-02-11T23:31:28.246+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:c5e569a1:::obj444:head, version: 111'444, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c5e569a1:::obj444:head@111'444, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.246+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c5e569a1:::obj444:head, version: 111'444, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c5e569a1:::obj444:head@111'444, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.246+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.246+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c5e569a1:::obj444:head, version: 111'444, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c5e569a1:::obj444:head@111'444, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.246+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1000 ==== pg_backfill(progress 4.0 e 119/119 lb 4:c5df6474:::obj459:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f13180 con 0x507ac00 2026-02-11T23:31:28.246+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.246+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:c5df6474:::obj459:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.246+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c5e569a1:::obj444:head, version: 111'444, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c5e569a1:::obj444:head@111'444, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.246+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c5df6474:::obj459:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.246+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:c5e569a1:::obj444:head, version: 111'444, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c5e569a1:::obj444:head@111'444, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 119/117 [PushOp(4:c5e569a1:::obj444:head, version: 111'444, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c5e569a1:::obj444:head@111'444, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:c59209e1:::obj523:head local-lis/les=117/118 n=479 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.246+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.246+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.246+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c5df6474:::obj459:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.246+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c59209e1:::obj523:head local-lis/les=117/118 n=479 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:c5e569a1:::obj444:head, version: 111'444, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c5e569a1:::obj444:head@111'444, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.246+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c59209e1:::obj523:head local-lis/les=117/118 n=479 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:c5e569a1:::obj444:head@111'444, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.246+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c59209e1:::obj523:head local-lis/les=117/118 n=479 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:c5e569a1:::obj444:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.246+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c59209e1:::obj523:head local-lis/les=117/118 n=479 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:c5e569a1:::obj444:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.246+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c59209e1:::obj523:head local-lis/les=117/118 n=479 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:c5e569a1:::obj444:head 2026-02-11T23:31:28.246+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c59209e1:::obj523:head local-lis/les=117/118 n=479 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:c5e569a1:::obj444:head v 111'444 2026-02-11T23:31:28.246+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c59209e1:::obj523:head local-lis/les=117/118 n=479 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.246+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.246+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:c5e569a1:::obj444:head, version: 111'444, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c5e569a1:::obj444:head@111'444, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:28.246+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.246+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.246+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c5df6474:::obj459:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.246+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:c5df6474:::obj459:head) v3 prio 15 cost 0 latency 0.000059 pg_backfill(progress 4.0 e 119/119 lb 4:c5df6474:::obj459:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:c59209e1:::obj523:head local-lis/les=117/118 n=479 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.246+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.246+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c59209e1:::obj523:head local-lis/les=117/118 n=479 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:c5df6474:::obj459:head) v3 2026-02-11T23:31:28.246+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c59209e1:::obj523:head local-lis/les=117/118 n=479 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:c5df6474:::obj459:head) 2026-02-11T23:31:28.246+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.246+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:c5df6474:::obj459:head) v3 finish latency 0.000014 2026-02-11T23:31:28.246+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c5df6474:::obj459:head local-lis/les=117/118 n=480 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.246+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.246+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.246+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c5df6474:::obj459:head local-lis/les=117/118 n=480 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.246+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.246+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.246+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.246+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x671d380 con 0x507ac00 2026-02-11T23:31:28.246+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:c5e569a1:::obj444:head)]) -- 0x642db00 con 0x507ac00 2026-02-11T23:31:28.246+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.246+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1001 ==== MOSDPGPush(4.0 119/117 [PushOp(4:c64b60bf:::obj563:head, version: 111'563, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c64b60bf:::obj563:head@111'563, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x554e680 con 0x507ac00 2026-02-11T23:31:28.246+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:c64b60bf:::obj563:head, version: 111'563, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c64b60bf:::obj563:head@111'563, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.246+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c64b60bf:::obj563:head, version: 111'563, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c64b60bf:::obj563:head@111'563, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.247+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.247+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c64b60bf:::obj563:head, version: 111'563, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c64b60bf:::obj563:head@111'563, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.247+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1002 ==== pg_backfill(progress 4.0 e 119/119 lb 4:c5e569a1:::obj444:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f10b00 con 0x507ac00 2026-02-11T23:31:28.247+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.247+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:c5e569a1:::obj444:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.247+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c64b60bf:::obj563:head, version: 111'563, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c64b60bf:::obj563:head@111'563, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.247+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c5e569a1:::obj444:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.247+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:c64b60bf:::obj563:head, version: 111'563, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c64b60bf:::obj563:head@111'563, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 119/117 [PushOp(4:c64b60bf:::obj563:head, version: 111'563, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c64b60bf:::obj563:head@111'563, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:c5df6474:::obj459:head local-lis/les=117/118 n=480 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.247+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.247+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.247+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c5df6474:::obj459:head local-lis/les=117/118 n=480 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:c64b60bf:::obj563:head, version: 111'563, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c64b60bf:::obj563:head@111'563, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.247+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c5e569a1:::obj444:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.247+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c5df6474:::obj459:head local-lis/les=117/118 n=480 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:c64b60bf:::obj563:head@111'563, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.247+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c5df6474:::obj459:head local-lis/les=117/118 n=480 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:c64b60bf:::obj563:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.247+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c5df6474:::obj459:head local-lis/les=117/118 n=480 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:c64b60bf:::obj563:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.247+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c5df6474:::obj459:head local-lis/les=117/118 n=480 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:c64b60bf:::obj563:head 2026-02-11T23:31:28.247+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c5df6474:::obj459:head local-lis/les=117/118 n=480 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:c64b60bf:::obj563:head v 111'563 2026-02-11T23:31:28.247+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c5df6474:::obj459:head local-lis/les=117/118 n=480 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.247+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.247+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:c64b60bf:::obj563:head, version: 111'563, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c64b60bf:::obj563:head@111'563, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:28.247+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.247+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.247+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c5e569a1:::obj444:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.247+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:c5e569a1:::obj444:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 4.0 e 119/119 lb 4:c5e569a1:::obj444:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:c5df6474:::obj459:head local-lis/les=117/118 n=480 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.247+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.247+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c5df6474:::obj459:head local-lis/les=117/118 n=480 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:c5e569a1:::obj444:head) v3 2026-02-11T23:31:28.247+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c5df6474:::obj459:head local-lis/les=117/118 n=480 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:c5e569a1:::obj444:head) 2026-02-11T23:31:28.247+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.247+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:c5e569a1:::obj444:head) v3 finish latency 0.000015 2026-02-11T23:31:28.247+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c5e569a1:::obj444:head local-lis/les=117/118 n=481 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.247+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.247+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.247+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c5e569a1:::obj444:head local-lis/les=117/118 n=481 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.247+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.247+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.247+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.247+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x640f680 con 0x507ac00 2026-02-11T23:31:28.247+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:c64b60bf:::obj563:head)]) -- 0x671c180 con 0x507ac00 2026-02-11T23:31:28.247+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.247+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1003 ==== MOSDPGPush(4.0 119/117 [PushOp(4:c68dea80:::obj235:head, version: 111'235, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c68dea80:::obj235:head@111'235, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x555d040 con 0x507ac00 2026-02-11T23:31:28.247+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:c68dea80:::obj235:head, version: 111'235, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c68dea80:::obj235:head@111'235, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.247+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c68dea80:::obj235:head, version: 111'235, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c68dea80:::obj235:head@111'235, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.247+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.247+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c68dea80:::obj235:head, version: 111'235, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c68dea80:::obj235:head@111'235, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.247+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1004 ==== pg_backfill(progress 4.0 e 119/119 lb 4:c64b60bf:::obj563:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f10580 con 0x507ac00 2026-02-11T23:31:28.247+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:c64b60bf:::obj563:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.247+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.247+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c68dea80:::obj235:head, version: 111'235, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c68dea80:::obj235:head@111'235, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.247+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c64b60bf:::obj563:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.247+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:c68dea80:::obj235:head, version: 111'235, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c68dea80:::obj235:head@111'235, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 119/117 [PushOp(4:c68dea80:::obj235:head, version: 111'235, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c68dea80:::obj235:head@111'235, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:c5e569a1:::obj444:head local-lis/les=117/118 n=481 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.247+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.247+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.247+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c64b60bf:::obj563:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.247+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c5e569a1:::obj444:head local-lis/les=117/118 n=481 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:c68dea80:::obj235:head, version: 111'235, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c68dea80:::obj235:head@111'235, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.247+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c5e569a1:::obj444:head local-lis/les=117/118 n=481 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:c68dea80:::obj235:head@111'235, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.247+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c5e569a1:::obj444:head local-lis/les=117/118 n=481 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:c68dea80:::obj235:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.247+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c5e569a1:::obj444:head local-lis/les=117/118 n=481 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:c68dea80:::obj235:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.247+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c5e569a1:::obj444:head local-lis/les=117/118 n=481 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:c68dea80:::obj235:head 2026-02-11T23:31:28.247+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c5e569a1:::obj444:head local-lis/les=117/118 n=481 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:c68dea80:::obj235:head v 111'235 2026-02-11T23:31:28.247+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c5e569a1:::obj444:head local-lis/les=117/118 n=481 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.247+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.247+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:c68dea80:::obj235:head, version: 111'235, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c68dea80:::obj235:head@111'235, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:28.247+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.247+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.247+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c64b60bf:::obj563:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.247+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:c64b60bf:::obj563:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 4.0 e 119/119 lb 4:c64b60bf:::obj563:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:c5e569a1:::obj444:head local-lis/les=117/118 n=481 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.247+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.247+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c5e569a1:::obj444:head local-lis/les=117/118 n=481 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:c64b60bf:::obj563:head) v3 2026-02-11T23:31:28.247+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c5e569a1:::obj444:head local-lis/les=117/118 n=481 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:c64b60bf:::obj563:head) 2026-02-11T23:31:28.247+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.247+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:c64b60bf:::obj563:head) v3 finish latency 0.000014 2026-02-11T23:31:28.247+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c64b60bf:::obj563:head local-lis/les=117/118 n=482 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.247+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.247+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.247+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c64b60bf:::obj563:head local-lis/les=117/118 n=482 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.247+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.247+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.247+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.247+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x674a000 con 0x507ac00 2026-02-11T23:31:28.247+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:c68dea80:::obj235:head)]) -- 0x66fa480 con 0x507ac00 2026-02-11T23:31:28.247+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.247+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1005 ==== MOSDPGPush(4.0 119/117 [PushOp(4:c71bdb5a:::obj349:head, version: 111'349, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c71bdb5a:::obj349:head@111'349, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56aa000 con 0x507ac00 2026-02-11T23:31:28.247+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:c71bdb5a:::obj349:head, version: 111'349, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c71bdb5a:::obj349:head@111'349, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.247+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c71bdb5a:::obj349:head, version: 111'349, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c71bdb5a:::obj349:head@111'349, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.248+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.248+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1006 ==== pg_backfill(progress 4.0 e 119/119 lb 4:c68dea80:::obj235:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f10000 con 0x507ac00 2026-02-11T23:31:28.248+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c71bdb5a:::obj349:head, version: 111'349, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c71bdb5a:::obj349:head@111'349, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.248+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.248+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c71bdb5a:::obj349:head, version: 111'349, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c71bdb5a:::obj349:head@111'349, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.248+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.248+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:c71bdb5a:::obj349:head, version: 111'349, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c71bdb5a:::obj349:head@111'349, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 119/117 [PushOp(4:c71bdb5a:::obj349:head, version: 111'349, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c71bdb5a:::obj349:head@111'349, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:c64b60bf:::obj563:head local-lis/les=117/118 n=482 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.248+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.248+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c64b60bf:::obj563:head local-lis/les=117/118 n=482 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:c71bdb5a:::obj349:head, version: 111'349, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c71bdb5a:::obj349:head@111'349, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.248+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c64b60bf:::obj563:head local-lis/les=117/118 n=482 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:c71bdb5a:::obj349:head@111'349, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.248+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c64b60bf:::obj563:head local-lis/les=117/118 n=482 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:c71bdb5a:::obj349:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.248+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c64b60bf:::obj563:head local-lis/les=117/118 n=482 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:c71bdb5a:::obj349:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.248+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c64b60bf:::obj563:head local-lis/les=117/118 n=482 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:c71bdb5a:::obj349:head 2026-02-11T23:31:28.248+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c64b60bf:::obj563:head local-lis/les=117/118 n=482 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:c71bdb5a:::obj349:head v 111'349 2026-02-11T23:31:28.248+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c64b60bf:::obj563:head local-lis/les=117/118 n=482 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.248+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.248+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:c68dea80:::obj235:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.248+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c68dea80:::obj235:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.248+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.248+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c68dea80:::obj235:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.248+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:c71bdb5a:::obj349:head, version: 111'349, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c71bdb5a:::obj349:head@111'349, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:28.248+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.248+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.248+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c68dea80:::obj235:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.248+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:c68dea80:::obj235:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 4.0 e 119/119 lb 4:c68dea80:::obj235:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:c64b60bf:::obj563:head local-lis/les=117/118 n=482 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.248+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.248+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c64b60bf:::obj563:head local-lis/les=117/118 n=482 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:c68dea80:::obj235:head) v3 2026-02-11T23:31:28.248+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c64b60bf:::obj563:head local-lis/les=117/118 n=482 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:c68dea80:::obj235:head) 2026-02-11T23:31:28.248+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.248+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:c68dea80:::obj235:head) v3 finish latency 0.000017 2026-02-11T23:31:28.248+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c68dea80:::obj235:head local-lis/les=117/118 n=483 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.248+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.248+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.248+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c68dea80:::obj235:head local-lis/les=117/118 n=483 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.248+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.248+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.248+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.248+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x674a180 con 0x507ac00 2026-02-11T23:31:28.248+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:c71bdb5a:::obj349:head)]) -- 0x66fb800 con 0x507ac00 2026-02-11T23:31:28.248+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.248+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1007 ==== MOSDPGPush(4.0 119/117 [PushOp(4:c7481937:::obj131:head, version: 111'131, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c7481937:::obj131:head@111'131, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2000 con 0x507ac00 2026-02-11T23:31:28.248+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:c7481937:::obj131:head, version: 111'131, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c7481937:::obj131:head@111'131, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.248+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c7481937:::obj131:head, version: 111'131, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c7481937:::obj131:head@111'131, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.248+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.248+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c7481937:::obj131:head, version: 111'131, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c7481937:::obj131:head@111'131, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.248+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1008 ==== pg_backfill(progress 4.0 e 119/119 lb 4:c71bdb5a:::obj349:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f13700 con 0x507ac00 2026-02-11T23:31:28.248+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:c71bdb5a:::obj349:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.248+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.248+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c7481937:::obj131:head, version: 111'131, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c7481937:::obj131:head@111'131, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.248+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c71bdb5a:::obj349:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.248+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:c7481937:::obj131:head, version: 111'131, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c7481937:::obj131:head@111'131, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 119/117 [PushOp(4:c7481937:::obj131:head, version: 111'131, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c7481937:::obj131:head@111'131, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:c68dea80:::obj235:head local-lis/les=117/118 n=483 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.248+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.248+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.248+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c71bdb5a:::obj349:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.248+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c68dea80:::obj235:head local-lis/les=117/118 n=483 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:c7481937:::obj131:head, version: 111'131, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c7481937:::obj131:head@111'131, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.248+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c68dea80:::obj235:head local-lis/les=117/118 n=483 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:c7481937:::obj131:head@111'131, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.248+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c68dea80:::obj235:head local-lis/les=117/118 n=483 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:c7481937:::obj131:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.248+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c68dea80:::obj235:head local-lis/les=117/118 n=483 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:c7481937:::obj131:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.248+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c68dea80:::obj235:head local-lis/les=117/118 n=483 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:c7481937:::obj131:head 2026-02-11T23:31:28.248+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c68dea80:::obj235:head local-lis/les=117/118 n=483 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:c7481937:::obj131:head v 111'131 2026-02-11T23:31:28.248+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c68dea80:::obj235:head local-lis/les=117/118 n=483 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.248+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.248+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:c7481937:::obj131:head, version: 111'131, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c7481937:::obj131:head@111'131, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:28.248+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.248+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.248+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c71bdb5a:::obj349:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.248+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:c71bdb5a:::obj349:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 4.0 e 119/119 lb 4:c71bdb5a:::obj349:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:c68dea80:::obj235:head local-lis/les=117/118 n=483 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.248+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.248+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c68dea80:::obj235:head local-lis/les=117/118 n=483 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:c71bdb5a:::obj349:head) v3 2026-02-11T23:31:28.248+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c68dea80:::obj235:head local-lis/les=117/118 n=483 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:c71bdb5a:::obj349:head) 2026-02-11T23:31:28.248+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.248+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:c71bdb5a:::obj349:head) v3 finish latency 0.000015 2026-02-11T23:31:28.248+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c71bdb5a:::obj349:head local-lis/les=117/118 n=484 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.248+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.248+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.248+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c71bdb5a:::obj349:head local-lis/les=117/118 n=484 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.248+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.248+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.248+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.248+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x674a300 con 0x507ac00 2026-02-11T23:31:28.248+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:c7481937:::obj131:head)]) -- 0x5f2dc80 con 0x507ac00 2026-02-11T23:31:28.248+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.249+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1009 ==== MOSDPGPush(4.0 119/117 [PushOp(4:c8495ae6:::obj149:head, version: 111'149, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c8495ae6:::obj149:head@111'149, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f1d40 con 0x507ac00 2026-02-11T23:31:28.249+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:c8495ae6:::obj149:head, version: 111'149, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c8495ae6:::obj149:head@111'149, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.249+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c8495ae6:::obj149:head, version: 111'149, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c8495ae6:::obj149:head@111'149, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.249+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.249+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c8495ae6:::obj149:head, version: 111'149, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c8495ae6:::obj149:head@111'149, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.249+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1010 ==== pg_backfill(progress 4.0 e 119/119 lb 4:c7481937:::obj131:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f12c00 con 0x507ac00 2026-02-11T23:31:28.249+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.249+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:c7481937:::obj131:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.249+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:c8495ae6:::obj149:head, version: 111'149, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c8495ae6:::obj149:head@111'149, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.249+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c7481937:::obj131:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.249+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.249+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:c8495ae6:::obj149:head, version: 111'149, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c8495ae6:::obj149:head@111'149, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 119/117 [PushOp(4:c8495ae6:::obj149:head, version: 111'149, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c8495ae6:::obj149:head@111'149, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:c71bdb5a:::obj349:head local-lis/les=117/118 n=484 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.249+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.249+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c7481937:::obj131:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.249+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c71bdb5a:::obj349:head local-lis/les=117/118 n=484 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:c8495ae6:::obj149:head, version: 111'149, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c8495ae6:::obj149:head@111'149, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.249+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c71bdb5a:::obj349:head local-lis/les=117/118 n=484 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:c8495ae6:::obj149:head@111'149, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.249+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c71bdb5a:::obj349:head local-lis/les=117/118 n=484 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:c8495ae6:::obj149:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.249+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c71bdb5a:::obj349:head local-lis/les=117/118 n=484 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:c8495ae6:::obj149:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.249+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c71bdb5a:::obj349:head local-lis/les=117/118 n=484 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:c8495ae6:::obj149:head 2026-02-11T23:31:28.249+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c71bdb5a:::obj349:head local-lis/les=117/118 n=484 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:c8495ae6:::obj149:head v 111'149 2026-02-11T23:31:28.249+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c71bdb5a:::obj349:head local-lis/les=117/118 n=484 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.249+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.249+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:c8495ae6:::obj149:head, version: 111'149, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c8495ae6:::obj149:head@111'149, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:28.249+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.249+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.249+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c7481937:::obj131:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.249+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:c7481937:::obj131:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 4.0 e 119/119 lb 4:c7481937:::obj131:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:c71bdb5a:::obj349:head local-lis/les=117/118 n=484 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.249+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.249+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c71bdb5a:::obj349:head local-lis/les=117/118 n=484 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:c7481937:::obj131:head) v3 2026-02-11T23:31:28.249+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c71bdb5a:::obj349:head local-lis/les=117/118 n=484 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:c7481937:::obj131:head) 2026-02-11T23:31:28.249+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.249+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:c7481937:::obj131:head) v3 finish latency 0.000014 2026-02-11T23:31:28.249+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c7481937:::obj131:head local-lis/les=117/118 n=485 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.249+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.249+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.249+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c7481937:::obj131:head local-lis/les=117/118 n=485 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.249+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.249+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.249+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.249+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x674a480 con 0x507ac00 2026-02-11T23:31:28.249+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:c8495ae6:::obj149:head)]) -- 0x66fbc80 con 0x507ac00 2026-02-11T23:31:28.249+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.249+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1011 ==== MOSDPGPush(4.0 119/117 [PushOp(4:ca82ed30:::obj135:head, version: 111'135, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ca82ed30:::obj135:head@111'135, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x554e4e0 con 0x507ac00 2026-02-11T23:31:28.249+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:ca82ed30:::obj135:head, version: 111'135, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ca82ed30:::obj135:head@111'135, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.249+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ca82ed30:::obj135:head, version: 111'135, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ca82ed30:::obj135:head@111'135, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.249+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.249+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ca82ed30:::obj135:head, version: 111'135, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ca82ed30:::obj135:head@111'135, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.249+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.249+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1012 ==== pg_backfill(progress 4.0 e 119/119 lb 4:c8495ae6:::obj149:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f12680 con 0x507ac00 2026-02-11T23:31:28.249+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ca82ed30:::obj135:head, version: 111'135, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ca82ed30:::obj135:head@111'135, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.249+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:c8495ae6:::obj149:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 119 2026-02-11T23:31:28.249+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c8495ae6:::obj149:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.249+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:ca82ed30:::obj135:head, version: 111'135, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ca82ed30:::obj135:head@111'135, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 119/117 [PushOp(4:ca82ed30:::obj135:head, version: 111'135, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ca82ed30:::obj135:head@111'135, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:c7481937:::obj131:head local-lis/les=117/118 n=485 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.249+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.249+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.249+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c8495ae6:::obj149:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.249+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c7481937:::obj131:head local-lis/les=117/118 n=485 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:ca82ed30:::obj135:head, version: 111'135, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ca82ed30:::obj135:head@111'135, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.249+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c7481937:::obj131:head local-lis/les=117/118 n=485 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:ca82ed30:::obj135:head@111'135, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.249+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c7481937:::obj131:head local-lis/les=117/118 n=485 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:ca82ed30:::obj135:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.249+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c7481937:::obj131:head local-lis/les=117/118 n=485 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:ca82ed30:::obj135:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.249+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c7481937:::obj131:head local-lis/les=117/118 n=485 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:ca82ed30:::obj135:head 2026-02-11T23:31:28.249+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c7481937:::obj131:head local-lis/les=117/118 n=485 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:ca82ed30:::obj135:head v 111'135 2026-02-11T23:31:28.249+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c7481937:::obj131:head local-lis/les=117/118 n=485 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.249+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.249+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:ca82ed30:::obj135:head, version: 111'135, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ca82ed30:::obj135:head@111'135, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:28.249+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.249+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.249+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:c8495ae6:::obj149:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.249+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:c8495ae6:::obj149:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 4.0 e 119/119 lb 4:c8495ae6:::obj149:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:c7481937:::obj131:head local-lis/les=117/118 n=485 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.249+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.249+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c7481937:::obj131:head local-lis/les=117/118 n=485 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:c8495ae6:::obj149:head) v3 2026-02-11T23:31:28.249+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c7481937:::obj131:head local-lis/les=117/118 n=485 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:c8495ae6:::obj149:head) 2026-02-11T23:31:28.249+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.249+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:c8495ae6:::obj149:head) v3 finish latency 0.000014 2026-02-11T23:31:28.249+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c8495ae6:::obj149:head local-lis/les=117/118 n=486 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.249+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.249+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.249+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c8495ae6:::obj149:head local-lis/les=117/118 n=486 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.249+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.249+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.249+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.249+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x674a600 con 0x507ac00 2026-02-11T23:31:28.249+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:ca82ed30:::obj135:head)]) -- 0x4f7aa80 con 0x507ac00 2026-02-11T23:31:28.249+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.250+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1013 ==== MOSDPGPush(4.0 119/117 [PushOp(4:cadaa6e3:::obj220:head, version: 111'220, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cadaa6e3:::obj220:head@111'220, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2820 con 0x507ac00 2026-02-11T23:31:28.250+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:cadaa6e3:::obj220:head, version: 111'220, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cadaa6e3:::obj220:head@111'220, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.250+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:cadaa6e3:::obj220:head, version: 111'220, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cadaa6e3:::obj220:head@111'220, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.250+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.250+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:cadaa6e3:::obj220:head, version: 111'220, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cadaa6e3:::obj220:head@111'220, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.250+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1014 ==== pg_backfill(progress 4.0 e 119/119 lb 4:ca82ed30:::obj135:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f12100 con 0x507ac00 2026-02-11T23:31:28.250+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.250+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:ca82ed30:::obj135:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.250+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:cadaa6e3:::obj220:head, version: 111'220, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cadaa6e3:::obj220:head@111'220, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.250+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:ca82ed30:::obj135:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.250+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.250+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:cadaa6e3:::obj220:head, version: 111'220, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cadaa6e3:::obj220:head@111'220, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 119/117 [PushOp(4:cadaa6e3:::obj220:head, version: 111'220, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cadaa6e3:::obj220:head@111'220, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:c8495ae6:::obj149:head local-lis/les=117/118 n=486 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.250+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.250+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:ca82ed30:::obj135:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.250+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c8495ae6:::obj149:head local-lis/les=117/118 n=486 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:cadaa6e3:::obj220:head, version: 111'220, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cadaa6e3:::obj220:head@111'220, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.250+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c8495ae6:::obj149:head local-lis/les=117/118 n=486 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:cadaa6e3:::obj220:head@111'220, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.250+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c8495ae6:::obj149:head local-lis/les=117/118 n=486 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:cadaa6e3:::obj220:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.250+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c8495ae6:::obj149:head local-lis/les=117/118 n=486 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:cadaa6e3:::obj220:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.250+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c8495ae6:::obj149:head local-lis/les=117/118 n=486 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:cadaa6e3:::obj220:head 2026-02-11T23:31:28.250+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c8495ae6:::obj149:head local-lis/les=117/118 n=486 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:cadaa6e3:::obj220:head v 111'220 2026-02-11T23:31:28.250+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c8495ae6:::obj149:head local-lis/les=117/118 n=486 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.250+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.250+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:cadaa6e3:::obj220:head, version: 111'220, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cadaa6e3:::obj220:head@111'220, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000062 2026-02-11T23:31:28.250+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.250+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.250+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:ca82ed30:::obj135:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.250+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:ca82ed30:::obj135:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 4.0 e 119/119 lb 4:ca82ed30:::obj135:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:c8495ae6:::obj149:head local-lis/les=117/118 n=486 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.250+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.250+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c8495ae6:::obj149:head local-lis/les=117/118 n=486 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:ca82ed30:::obj135:head) v3 2026-02-11T23:31:28.250+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:c8495ae6:::obj149:head local-lis/les=117/118 n=486 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:ca82ed30:::obj135:head) 2026-02-11T23:31:28.250+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.250+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:ca82ed30:::obj135:head) v3 finish latency 0.000014 2026-02-11T23:31:28.250+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ca82ed30:::obj135:head local-lis/les=117/118 n=487 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.250+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.250+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.250+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ca82ed30:::obj135:head local-lis/les=117/118 n=487 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.250+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.250+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.250+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.250+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x674a780 con 0x507ac00 2026-02-11T23:31:28.250+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:cadaa6e3:::obj220:head)]) -- 0x66fbb00 con 0x507ac00 2026-02-11T23:31:28.250+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.250+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1015 ==== MOSDPGPush(4.0 119/117 [PushOp(4:cb1a5635:::obj233:head, version: 111'233, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb1a5635:::obj233:head@111'233, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a3a00 con 0x507ac00 2026-02-11T23:31:28.250+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:cb1a5635:::obj233:head, version: 111'233, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb1a5635:::obj233:head@111'233, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.250+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:cb1a5635:::obj233:head, version: 111'233, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb1a5635:::obj233:head@111'233, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.250+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.250+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:cb1a5635:::obj233:head, version: 111'233, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb1a5635:::obj233:head@111'233, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.250+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1016 ==== pg_backfill(progress 4.0 e 119/119 lb 4:cadaa6e3:::obj220:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f11b80 con 0x507ac00 2026-02-11T23:31:28.250+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:cadaa6e3:::obj220:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.250+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.250+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:cadaa6e3:::obj220:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.250+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:cb1a5635:::obj233:head, version: 111'233, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb1a5635:::obj233:head@111'233, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.250+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.250+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:cb1a5635:::obj233:head, version: 111'233, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb1a5635:::obj233:head@111'233, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 119/117 [PushOp(4:cb1a5635:::obj233:head, version: 111'233, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb1a5635:::obj233:head@111'233, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:ca82ed30:::obj135:head local-lis/les=117/118 n=487 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.250+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.250+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ca82ed30:::obj135:head local-lis/les=117/118 n=487 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:cb1a5635:::obj233:head, version: 111'233, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb1a5635:::obj233:head@111'233, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.250+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ca82ed30:::obj135:head local-lis/les=117/118 n=487 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:cb1a5635:::obj233:head@111'233, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.250+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.250+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:cadaa6e3:::obj220:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.250+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ca82ed30:::obj135:head local-lis/les=117/118 n=487 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:cb1a5635:::obj233:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.250+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ca82ed30:::obj135:head local-lis/les=117/118 n=487 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:cb1a5635:::obj233:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.250+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ca82ed30:::obj135:head local-lis/les=117/118 n=487 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:cb1a5635:::obj233:head 2026-02-11T23:31:28.250+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ca82ed30:::obj135:head local-lis/les=117/118 n=487 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:cb1a5635:::obj233:head v 111'233 2026-02-11T23:31:28.250+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ca82ed30:::obj135:head local-lis/les=117/118 n=487 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.250+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.250+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:cb1a5635:::obj233:head, version: 111'233, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb1a5635:::obj233:head@111'233, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:28.250+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.250+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.250+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:cadaa6e3:::obj220:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.250+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:cadaa6e3:::obj220:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 4.0 e 119/119 lb 4:cadaa6e3:::obj220:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:ca82ed30:::obj135:head local-lis/les=117/118 n=487 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.250+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.250+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ca82ed30:::obj135:head local-lis/les=117/118 n=487 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:cadaa6e3:::obj220:head) v3 2026-02-11T23:31:28.250+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ca82ed30:::obj135:head local-lis/les=117/118 n=487 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:cadaa6e3:::obj220:head) 2026-02-11T23:31:28.250+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.250+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:cadaa6e3:::obj220:head) v3 finish latency 0.000013 2026-02-11T23:31:28.250+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cadaa6e3:::obj220:head local-lis/les=117/118 n=488 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.250+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.250+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.250+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cadaa6e3:::obj220:head local-lis/les=117/118 n=488 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.250+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.250+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.250+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.250+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x674a900 con 0x507ac00 2026-02-11T23:31:28.250+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:cb1a5635:::obj233:head)]) -- 0x63ff200 con 0x507ac00 2026-02-11T23:31:28.250+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.251+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1017 ==== MOSDPGPush(4.0 119/117 [PushOp(4:cb1be4ca:::obj406:head, version: 111'406, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb1be4ca:::obj406:head@111'406, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db0000 con 0x507ac00 2026-02-11T23:31:28.251+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:cb1be4ca:::obj406:head, version: 111'406, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb1be4ca:::obj406:head@111'406, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.251+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:cb1be4ca:::obj406:head, version: 111'406, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb1be4ca:::obj406:head@111'406, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.251+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.251+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1018 ==== pg_backfill(progress 4.0 e 119/119 lb 4:cb1a5635:::obj233:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f10b00 con 0x507ac00 2026-02-11T23:31:28.251+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:cb1a5635:::obj233:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.251+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:cb1be4ca:::obj406:head, version: 111'406, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb1be4ca:::obj406:head@111'406, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.251+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:cb1a5635:::obj233:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.251+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.251+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:cb1be4ca:::obj406:head, version: 111'406, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb1be4ca:::obj406:head@111'406, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.251+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:cb1be4ca:::obj406:head, version: 111'406, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb1be4ca:::obj406:head@111'406, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 119/117 [PushOp(4:cb1be4ca:::obj406:head, version: 111'406, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb1be4ca:::obj406:head@111'406, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:cadaa6e3:::obj220:head local-lis/les=117/118 n=488 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.251+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.251+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.251+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:cb1a5635:::obj233:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.251+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cadaa6e3:::obj220:head local-lis/les=117/118 n=488 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:cb1be4ca:::obj406:head, version: 111'406, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb1be4ca:::obj406:head@111'406, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.251+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cadaa6e3:::obj220:head local-lis/les=117/118 n=488 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:cb1be4ca:::obj406:head@111'406, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.251+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cadaa6e3:::obj220:head local-lis/les=117/118 n=488 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:cb1be4ca:::obj406:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.251+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cadaa6e3:::obj220:head local-lis/les=117/118 n=488 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:cb1be4ca:::obj406:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.251+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cadaa6e3:::obj220:head local-lis/les=117/118 n=488 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:cb1be4ca:::obj406:head 2026-02-11T23:31:28.251+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cadaa6e3:::obj220:head local-lis/les=117/118 n=488 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:cb1be4ca:::obj406:head v 111'406 2026-02-11T23:31:28.251+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cadaa6e3:::obj220:head local-lis/les=117/118 n=488 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.251+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.251+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:cb1be4ca:::obj406:head, version: 111'406, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb1be4ca:::obj406:head@111'406, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000078 2026-02-11T23:31:28.251+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.251+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.251+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:cb1a5635:::obj233:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.251+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:cb1a5635:::obj233:head) v3 prio 15 cost 0 latency 0.000096 pg_backfill(progress 4.0 e 119/119 lb 4:cb1a5635:::obj233:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:cadaa6e3:::obj220:head local-lis/les=117/118 n=488 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.251+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.251+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cadaa6e3:::obj220:head local-lis/les=117/118 n=488 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:cb1a5635:::obj233:head) v3 2026-02-11T23:31:28.251+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cadaa6e3:::obj220:head local-lis/les=117/118 n=488 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:cb1a5635:::obj233:head) 2026-02-11T23:31:28.251+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.251+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:cb1a5635:::obj233:head) v3 finish latency 0.000014 2026-02-11T23:31:28.251+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cb1a5635:::obj233:head local-lis/les=117/118 n=489 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.251+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.251+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.251+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cb1a5635:::obj233:head local-lis/les=117/118 n=489 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.251+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.251+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.251+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.251+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x674aa80 con 0x507ac00 2026-02-11T23:31:28.251+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:cb1be4ca:::obj406:head)]) -- 0x670b680 con 0x507ac00 2026-02-11T23:31:28.251+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.251+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1019 ==== MOSDPGPush(4.0 119/117 [PushOp(4:cb4c4ca4:::obj480:head, version: 111'480, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb4c4ca4:::obj480:head@111'480, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56caea0 con 0x507ac00 2026-02-11T23:31:28.251+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:cb4c4ca4:::obj480:head, version: 111'480, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb4c4ca4:::obj480:head@111'480, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.251+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:cb4c4ca4:::obj480:head, version: 111'480, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb4c4ca4:::obj480:head@111'480, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.251+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.251+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1020 ==== pg_backfill(progress 4.0 e 119/119 lb 4:cb1be4ca:::obj406:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f13180 con 0x507ac00 2026-02-11T23:31:28.251+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:cb1be4ca:::obj406:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.251+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:cb4c4ca4:::obj480:head, version: 111'480, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb4c4ca4:::obj480:head@111'480, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.251+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:cb1be4ca:::obj406:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.251+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.251+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:cb4c4ca4:::obj480:head, version: 111'480, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb4c4ca4:::obj480:head@111'480, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.251+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:cb4c4ca4:::obj480:head, version: 111'480, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb4c4ca4:::obj480:head@111'480, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 119/117 [PushOp(4:cb4c4ca4:::obj480:head, version: 111'480, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb4c4ca4:::obj480:head@111'480, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:cb1a5635:::obj233:head local-lis/les=117/118 n=489 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.251+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.251+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.251+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:cb1be4ca:::obj406:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.251+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cb1a5635:::obj233:head local-lis/les=117/118 n=489 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:cb4c4ca4:::obj480:head, version: 111'480, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb4c4ca4:::obj480:head@111'480, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.251+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cb1a5635:::obj233:head local-lis/les=117/118 n=489 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:cb4c4ca4:::obj480:head@111'480, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.251+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cb1a5635:::obj233:head local-lis/les=117/118 n=489 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:cb4c4ca4:::obj480:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.251+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cb1a5635:::obj233:head local-lis/les=117/118 n=489 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:cb4c4ca4:::obj480:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.251+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cb1a5635:::obj233:head local-lis/les=117/118 n=489 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:cb4c4ca4:::obj480:head 2026-02-11T23:31:28.251+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cb1a5635:::obj233:head local-lis/les=117/118 n=489 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:cb4c4ca4:::obj480:head v 111'480 2026-02-11T23:31:28.251+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cb1a5635:::obj233:head local-lis/les=117/118 n=489 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.251+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.251+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:cb4c4ca4:::obj480:head, version: 111'480, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb4c4ca4:::obj480:head@111'480, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000070 2026-02-11T23:31:28.251+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.251+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.251+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:cb1be4ca:::obj406:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.251+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:cb1be4ca:::obj406:head) v3 prio 15 cost 0 latency 0.000087 pg_backfill(progress 4.0 e 119/119 lb 4:cb1be4ca:::obj406:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:cb1a5635:::obj233:head local-lis/les=117/118 n=489 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.251+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.251+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cb1a5635:::obj233:head local-lis/les=117/118 n=489 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:cb1be4ca:::obj406:head) v3 2026-02-11T23:31:28.251+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cb1a5635:::obj233:head local-lis/les=117/118 n=489 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:cb1be4ca:::obj406:head) 2026-02-11T23:31:28.251+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.251+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:cb1be4ca:::obj406:head) v3 finish latency 0.000016 2026-02-11T23:31:28.251+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cb1be4ca:::obj406:head local-lis/les=117/118 n=490 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.251+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.252+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.252+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cb1be4ca:::obj406:head local-lis/les=117/118 n=490 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.252+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.252+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.252+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.252+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x674ac00 con 0x507ac00 2026-02-11T23:31:28.252+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:cb4c4ca4:::obj480:head)]) -- 0x674a180 con 0x507ac00 2026-02-11T23:31:28.252+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.252+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1021 ==== MOSDPGPush(4.0 119/117 [PushOp(4:cb843a8e:::obj34:head, version: 111'34, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb843a8e:::obj34:head@111'34, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x56b64e0 con 0x507ac00 2026-02-11T23:31:28.252+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:cb843a8e:::obj34:head, version: 111'34, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb843a8e:::obj34:head@111'34, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.252+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:cb843a8e:::obj34:head, version: 111'34, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb843a8e:::obj34:head@111'34, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.252+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.252+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1022 ==== pg_backfill(progress 4.0 e 119/119 lb 4:cb4c4ca4:::obj480:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f11080 con 0x507ac00 2026-02-11T23:31:28.252+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:cb843a8e:::obj34:head, version: 111'34, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb843a8e:::obj34:head@111'34, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.252+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:cb4c4ca4:::obj480:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.252+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.252+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:cb4c4ca4:::obj480:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.252+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:cb843a8e:::obj34:head, version: 111'34, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb843a8e:::obj34:head@111'34, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.252+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.252+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:cb843a8e:::obj34:head, version: 111'34, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb843a8e:::obj34:head@111'34, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 119/117 [PushOp(4:cb843a8e:::obj34:head, version: 111'34, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb843a8e:::obj34:head@111'34, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:cb1be4ca:::obj406:head local-lis/les=117/118 n=490 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.252+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.252+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cb1be4ca:::obj406:head local-lis/les=117/118 n=490 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:cb843a8e:::obj34:head, version: 111'34, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb843a8e:::obj34:head@111'34, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.252+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cb1be4ca:::obj406:head local-lis/les=117/118 n=490 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:cb843a8e:::obj34:head@111'34, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.252+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.252+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:cb4c4ca4:::obj480:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.252+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cb1be4ca:::obj406:head local-lis/les=117/118 n=490 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:cb843a8e:::obj34:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.252+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cb1be4ca:::obj406:head local-lis/les=117/118 n=490 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:cb843a8e:::obj34:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.252+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cb1be4ca:::obj406:head local-lis/les=117/118 n=490 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:cb843a8e:::obj34:head 2026-02-11T23:31:28.252+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cb1be4ca:::obj406:head local-lis/les=117/118 n=490 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:cb843a8e:::obj34:head v 111'34 2026-02-11T23:31:28.252+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cb1be4ca:::obj406:head local-lis/les=117/118 n=490 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.252+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.252+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:cb843a8e:::obj34:head, version: 111'34, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb843a8e:::obj34:head@111'34, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:28.252+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.252+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.252+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:cb4c4ca4:::obj480:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.252+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:cb4c4ca4:::obj480:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 4.0 e 119/119 lb 4:cb4c4ca4:::obj480:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:cb1be4ca:::obj406:head local-lis/les=117/118 n=490 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.252+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.252+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cb1be4ca:::obj406:head local-lis/les=117/118 n=490 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:cb4c4ca4:::obj480:head) v3 2026-02-11T23:31:28.252+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cb1be4ca:::obj406:head local-lis/les=117/118 n=490 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:cb4c4ca4:::obj480:head) 2026-02-11T23:31:28.252+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.252+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:cb4c4ca4:::obj480:head) v3 finish latency 0.000015 2026-02-11T23:31:28.252+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cb4c4ca4:::obj480:head local-lis/les=117/118 n=491 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.252+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.252+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.252+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cb4c4ca4:::obj480:head local-lis/les=117/118 n=491 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.252+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.252+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.252+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.252+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x674ad80 con 0x507ac00 2026-02-11T23:31:28.252+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:cb843a8e:::obj34:head)]) -- 0x674a000 con 0x507ac00 2026-02-11T23:31:28.252+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.253+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1023 ==== MOSDPGPush(4.0 119/117 [PushOp(4:cbce4a0b:::obj95:head, version: 111'95, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cbce4a0b:::obj95:head@111'95, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x55f0d00 con 0x507ac00 2026-02-11T23:31:28.253+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:cbce4a0b:::obj95:head, version: 111'95, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cbce4a0b:::obj95:head@111'95, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.253+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:cbce4a0b:::obj95:head, version: 111'95, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cbce4a0b:::obj95:head@111'95, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.253+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.253+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1024 ==== pg_backfill(progress 4.0 e 119/119 lb 4:cb843a8e:::obj34:head) ==== 1061+0+0 (unknown 0 0 0) 0x5f11600 con 0x507ac00 2026-02-11T23:31:28.253+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:cb843a8e:::obj34:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.253+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:cbce4a0b:::obj95:head, version: 111'95, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cbce4a0b:::obj95:head@111'95, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.253+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:cb843a8e:::obj34:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.253+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.253+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:cbce4a0b:::obj95:head, version: 111'95, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cbce4a0b:::obj95:head@111'95, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.253+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:cbce4a0b:::obj95:head, version: 111'95, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cbce4a0b:::obj95:head@111'95, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 119/117 [PushOp(4:cbce4a0b:::obj95:head, version: 111'95, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cbce4a0b:::obj95:head@111'95, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:cb4c4ca4:::obj480:head local-lis/les=117/118 n=491 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.253+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.253+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.253+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:cb843a8e:::obj34:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.253+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cb4c4ca4:::obj480:head local-lis/les=117/118 n=491 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:cbce4a0b:::obj95:head, version: 111'95, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cbce4a0b:::obj95:head@111'95, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.253+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cb4c4ca4:::obj480:head local-lis/les=117/118 n=491 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:cbce4a0b:::obj95:head@111'95, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.253+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cb4c4ca4:::obj480:head local-lis/les=117/118 n=491 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:cbce4a0b:::obj95:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.253+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cb4c4ca4:::obj480:head local-lis/les=117/118 n=491 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:cbce4a0b:::obj95:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.253+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cb4c4ca4:::obj480:head local-lis/les=117/118 n=491 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:cbce4a0b:::obj95:head 2026-02-11T23:31:28.253+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cb4c4ca4:::obj480:head local-lis/les=117/118 n=491 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:cbce4a0b:::obj95:head v 111'95 2026-02-11T23:31:28.253+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cb4c4ca4:::obj480:head local-lis/les=117/118 n=491 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.253+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.253+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:cbce4a0b:::obj95:head, version: 111'95, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cbce4a0b:::obj95:head@111'95, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000068 2026-02-11T23:31:28.253+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.253+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.253+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:cb843a8e:::obj34:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.253+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:cb843a8e:::obj34:head) v3 prio 15 cost 0 latency 0.000086 pg_backfill(progress 4.0 e 119/119 lb 4:cb843a8e:::obj34:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:cb4c4ca4:::obj480:head local-lis/les=117/118 n=491 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.253+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.253+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cb4c4ca4:::obj480:head local-lis/les=117/118 n=491 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:cb843a8e:::obj34:head) v3 2026-02-11T23:31:28.253+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cb4c4ca4:::obj480:head local-lis/les=117/118 n=491 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:cb843a8e:::obj34:head) 2026-02-11T23:31:28.253+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.253+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:cb843a8e:::obj34:head) v3 finish latency 0.000014 2026-02-11T23:31:28.253+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cb843a8e:::obj34:head local-lis/les=117/118 n=492 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.253+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.253+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.253+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cb843a8e:::obj34:head local-lis/les=117/118 n=492 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.253+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.253+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.253+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.253+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x674af00 con 0x507ac00 2026-02-11T23:31:28.253+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:cbce4a0b:::obj95:head)]) -- 0x671c180 con 0x507ac00 2026-02-11T23:31:28.253+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.253+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1025 ==== MOSDPGPush(4.0 119/117 [PushOp(4:cc461e5d:::obj45:head, version: 111'45, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cc461e5d:::obj45:head@111'45, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x559c4e0 con 0x507ac00 2026-02-11T23:31:28.253+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:cc461e5d:::obj45:head, version: 111'45, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cc461e5d:::obj45:head@111'45, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.253+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:cc461e5d:::obj45:head, version: 111'45, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cc461e5d:::obj45:head@111'45, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.253+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.253+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1026 ==== pg_backfill(progress 4.0 e 119/119 lb 4:cbce4a0b:::obj95:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d2f700 con 0x507ac00 2026-02-11T23:31:28.253+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:cbce4a0b:::obj95:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.253+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:cc461e5d:::obj45:head, version: 111'45, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cc461e5d:::obj45:head@111'45, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.253+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:cbce4a0b:::obj95:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.253+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.253+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:cc461e5d:::obj45:head, version: 111'45, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cc461e5d:::obj45:head@111'45, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.253+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.253+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:cc461e5d:::obj45:head, version: 111'45, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cc461e5d:::obj45:head@111'45, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(4.0 119/117 [PushOp(4:cc461e5d:::obj45:head, version: 111'45, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cc461e5d:::obj45:head@111'45, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:cb843a8e:::obj34:head local-lis/les=117/118 n=492 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.253+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.253+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:cbce4a0b:::obj95:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.253+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cb843a8e:::obj34:head local-lis/les=117/118 n=492 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:cc461e5d:::obj45:head, version: 111'45, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cc461e5d:::obj45:head@111'45, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.253+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cb843a8e:::obj34:head local-lis/les=117/118 n=492 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:cc461e5d:::obj45:head@111'45, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.253+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cb843a8e:::obj34:head local-lis/les=117/118 n=492 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:cc461e5d:::obj45:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.253+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cb843a8e:::obj34:head local-lis/les=117/118 n=492 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:cc461e5d:::obj45:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.253+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cb843a8e:::obj34:head local-lis/les=117/118 n=492 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:cc461e5d:::obj45:head 2026-02-11T23:31:28.253+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cb843a8e:::obj34:head local-lis/les=117/118 n=492 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:cc461e5d:::obj45:head v 111'45 2026-02-11T23:31:28.253+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cb843a8e:::obj34:head local-lis/les=117/118 n=492 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.253+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.253+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:cc461e5d:::obj45:head, version: 111'45, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cc461e5d:::obj45:head@111'45, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000069 2026-02-11T23:31:28.253+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cb843a8e:::obj34:head local-lis/les=117/118 n=492 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.253+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.253+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.253+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:cbce4a0b:::obj95:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.253+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:cbce4a0b:::obj95:head) v3 prio 15 cost 0 latency 0.000091 pg_backfill(progress 4.0 e 119/119 lb 4:cbce4a0b:::obj95:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:cb843a8e:::obj34:head local-lis/les=117/118 n=492 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.253+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.253+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cb843a8e:::obj34:head local-lis/les=117/118 n=492 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:cbce4a0b:::obj95:head) v3 2026-02-11T23:31:28.253+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cb843a8e:::obj34:head local-lis/les=117/118 n=492 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:cbce4a0b:::obj95:head) 2026-02-11T23:31:28.253+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.253+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:cbce4a0b:::obj95:head) v3 finish latency 0.000019 2026-02-11T23:31:28.253+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.253+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.253+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cbce4a0b:::obj95:head local-lis/les=117/118 n=493 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.253+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.253+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.253+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.253+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x674b200 con 0x507ac00 2026-02-11T23:31:28.253+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:cc461e5d:::obj45:head)]) -- 0x674b080 con 0x507ac00 2026-02-11T23:31:28.253+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.256+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1027 ==== MOSDPGPush(4.0 119/117 [PushOp(4:cdb46b38:::obj588:head, version: 111'588, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cdb46b38:::obj588:head@111'588, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3814ea0 con 0x507ac00 2026-02-11T23:31:28.256+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:cdb46b38:::obj588:head, version: 111'588, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cdb46b38:::obj588:head@111'588, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 119 2026-02-11T23:31:28.256+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:cdb46b38:::obj588:head, version: 111'588, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cdb46b38:::obj588:head@111'588, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.256+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.256+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:cdb46b38:::obj588:head, version: 111'588, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cdb46b38:::obj588:head@111'588, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.256+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1028 ==== pg_backfill(progress 4.0 e 119/119 lb 4:cc461e5d:::obj45:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d2d080 con 0x507ac00 2026-02-11T23:31:28.256+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:cc461e5d:::obj45:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.256+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.256+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:cdb46b38:::obj588:head, version: 111'588, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cdb46b38:::obj588:head@111'588, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.256+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:cc461e5d:::obj45:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.256+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:cdb46b38:::obj588:head, version: 111'588, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cdb46b38:::obj588:head@111'588, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 119/117 [PushOp(4:cdb46b38:::obj588:head, version: 111'588, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cdb46b38:::obj588:head@111'588, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:cbce4a0b:::obj95:head local-lis/les=117/118 n=493 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.256+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.256+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.256+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:cc461e5d:::obj45:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.256+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cbce4a0b:::obj95:head local-lis/les=117/118 n=493 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:cdb46b38:::obj588:head, version: 111'588, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cdb46b38:::obj588:head@111'588, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.256+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cbce4a0b:::obj95:head local-lis/les=117/118 n=493 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:cdb46b38:::obj588:head@111'588, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.256+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cbce4a0b:::obj95:head local-lis/les=117/118 n=493 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:cdb46b38:::obj588:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.256+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cbce4a0b:::obj95:head local-lis/les=117/118 n=493 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:cdb46b38:::obj588:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.256+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cbce4a0b:::obj95:head local-lis/les=117/118 n=493 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:cdb46b38:::obj588:head 2026-02-11T23:31:28.256+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cbce4a0b:::obj95:head local-lis/les=117/118 n=493 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:cdb46b38:::obj588:head v 111'588 2026-02-11T23:31:28.256+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cbce4a0b:::obj95:head local-lis/les=117/118 n=493 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.256+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.256+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:cdb46b38:::obj588:head, version: 111'588, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cdb46b38:::obj588:head@111'588, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:28.256+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.256+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.256+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:cc461e5d:::obj45:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.256+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:cc461e5d:::obj45:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 4.0 e 119/119 lb 4:cc461e5d:::obj45:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:cbce4a0b:::obj95:head local-lis/les=117/118 n=493 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.256+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.256+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cbce4a0b:::obj95:head local-lis/les=117/118 n=493 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:cc461e5d:::obj45:head) v3 2026-02-11T23:31:28.256+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cbce4a0b:::obj95:head local-lis/les=117/118 n=493 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:cc461e5d:::obj45:head) 2026-02-11T23:31:28.256+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.256+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:cc461e5d:::obj45:head) v3 finish latency 0.000016 2026-02-11T23:31:28.256+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cc461e5d:::obj45:head local-lis/les=117/118 n=494 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.256+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.256+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.256+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cc461e5d:::obj45:head local-lis/les=117/118 n=494 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.256+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.256+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.256+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.256+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x674b380 con 0x507ac00 2026-02-11T23:31:28.256+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:cdb46b38:::obj588:head)]) -- 0x640f680 con 0x507ac00 2026-02-11T23:31:28.256+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.256+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1029 ==== MOSDPGPush(4.0 119/117 [PushOp(4:cde49336:::obj263:head, version: 111'263, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cde49336:::obj263:head@111'263, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a3ba0 con 0x507ac00 2026-02-11T23:31:28.256+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:cde49336:::obj263:head, version: 111'263, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cde49336:::obj263:head@111'263, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.256+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:cde49336:::obj263:head, version: 111'263, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cde49336:::obj263:head@111'263, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.256+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.256+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1030 ==== pg_backfill(progress 4.0 e 119/119 lb 4:cdb46b38:::obj588:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e680 con 0x507ac00 2026-02-11T23:31:28.256+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:cde49336:::obj263:head, version: 111'263, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cde49336:::obj263:head@111'263, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.256+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:cdb46b38:::obj588:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.256+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:cdb46b38:::obj588:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.256+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.256+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:cde49336:::obj263:head, version: 111'263, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cde49336:::obj263:head@111'263, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.256+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:cde49336:::obj263:head, version: 111'263, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cde49336:::obj263:head@111'263, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 119/117 [PushOp(4:cde49336:::obj263:head, version: 111'263, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cde49336:::obj263:head@111'263, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:cc461e5d:::obj45:head local-lis/les=117/118 n=494 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.256+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.256+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.256+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cc461e5d:::obj45:head local-lis/les=117/118 n=494 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:cde49336:::obj263:head, version: 111'263, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cde49336:::obj263:head@111'263, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.256+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:cdb46b38:::obj588:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.256+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cc461e5d:::obj45:head local-lis/les=117/118 n=494 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:cde49336:::obj263:head@111'263, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.256+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cc461e5d:::obj45:head local-lis/les=117/118 n=494 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:cde49336:::obj263:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.256+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cc461e5d:::obj45:head local-lis/les=117/118 n=494 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:cde49336:::obj263:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.256+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cc461e5d:::obj45:head local-lis/les=117/118 n=494 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:cde49336:::obj263:head 2026-02-11T23:31:28.256+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cc461e5d:::obj45:head local-lis/les=117/118 n=494 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:cde49336:::obj263:head v 111'263 2026-02-11T23:31:28.256+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cc461e5d:::obj45:head local-lis/les=117/118 n=494 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.256+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.256+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:cde49336:::obj263:head, version: 111'263, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cde49336:::obj263:head@111'263, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:28.256+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.256+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.256+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:cdb46b38:::obj588:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.256+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:cdb46b38:::obj588:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 4.0 e 119/119 lb 4:cdb46b38:::obj588:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:cc461e5d:::obj45:head local-lis/les=117/118 n=494 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.256+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.256+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cc461e5d:::obj45:head local-lis/les=117/118 n=494 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:cdb46b38:::obj588:head) v3 2026-02-11T23:31:28.256+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cc461e5d:::obj45:head local-lis/les=117/118 n=494 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:cdb46b38:::obj588:head) 2026-02-11T23:31:28.256+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.256+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:cdb46b38:::obj588:head) v3 finish latency 0.000014 2026-02-11T23:31:28.256+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cdb46b38:::obj588:head local-lis/les=117/118 n=495 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.256+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.256+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.256+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cdb46b38:::obj588:head local-lis/les=117/118 n=495 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.256+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.256+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.256+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.256+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x674b500 con 0x507ac00 2026-02-11T23:31:28.256+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:cde49336:::obj263:head)]) -- 0x671d380 con 0x507ac00 2026-02-11T23:31:28.256+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.256+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1031 ==== MOSDPGPush(4.0 119/117 [PushOp(4:cf1e3413:::obj224:head, version: 111'224, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cf1e3413:::obj224:head@111'224, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x38156c0 con 0x507ac00 2026-02-11T23:31:28.256+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:cf1e3413:::obj224:head, version: 111'224, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cf1e3413:::obj224:head@111'224, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 119 2026-02-11T23:31:28.256+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:cf1e3413:::obj224:head, version: 111'224, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cf1e3413:::obj224:head@111'224, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.256+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.256+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1032 ==== pg_backfill(progress 4.0 e 119/119 lb 4:cde49336:::obj263:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176580 con 0x507ac00 2026-02-11T23:31:28.256+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:cf1e3413:::obj224:head, version: 111'224, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cf1e3413:::obj224:head@111'224, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.256+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:cde49336:::obj263:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.256+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:cde49336:::obj263:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.256+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.256+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:cf1e3413:::obj224:head, version: 111'224, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cf1e3413:::obj224:head@111'224, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.256+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:cf1e3413:::obj224:head, version: 111'224, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cf1e3413:::obj224:head@111'224, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 119/117 [PushOp(4:cf1e3413:::obj224:head, version: 111'224, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cf1e3413:::obj224:head@111'224, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:cdb46b38:::obj588:head local-lis/les=117/118 n=495 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.256+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.256+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.256+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:cde49336:::obj263:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.256+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cdb46b38:::obj588:head local-lis/les=117/118 n=495 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:cf1e3413:::obj224:head, version: 111'224, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cf1e3413:::obj224:head@111'224, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.256+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cdb46b38:::obj588:head local-lis/les=117/118 n=495 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:cf1e3413:::obj224:head@111'224, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.256+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cdb46b38:::obj588:head local-lis/les=117/118 n=495 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:cf1e3413:::obj224:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.256+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cdb46b38:::obj588:head local-lis/les=117/118 n=495 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:cf1e3413:::obj224:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.256+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cdb46b38:::obj588:head local-lis/les=117/118 n=495 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:cf1e3413:::obj224:head 2026-02-11T23:31:28.256+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cdb46b38:::obj588:head local-lis/les=117/118 n=495 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:cf1e3413:::obj224:head v 111'224 2026-02-11T23:31:28.256+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cdb46b38:::obj588:head local-lis/les=117/118 n=495 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.256+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.256+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:cf1e3413:::obj224:head, version: 111'224, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cf1e3413:::obj224:head@111'224, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:28.256+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.256+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.256+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:cde49336:::obj263:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.256+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:cde49336:::obj263:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 4.0 e 119/119 lb 4:cde49336:::obj263:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:cdb46b38:::obj588:head local-lis/les=117/118 n=495 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.256+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.256+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cdb46b38:::obj588:head local-lis/les=117/118 n=495 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:cde49336:::obj263:head) v3 2026-02-11T23:31:28.256+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cdb46b38:::obj588:head local-lis/les=117/118 n=495 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:cde49336:::obj263:head) 2026-02-11T23:31:28.256+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.256+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:cde49336:::obj263:head) v3 finish latency 0.000012 2026-02-11T23:31:28.256+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cde49336:::obj263:head local-lis/les=117/118 n=496 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.256+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.256+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.256+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cde49336:::obj263:head local-lis/les=117/118 n=496 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.256+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.256+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.256+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.256+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x674b680 con 0x507ac00 2026-02-11T23:31:28.256+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:cf1e3413:::obj224:head)]) -- 0x66d9080 con 0x507ac00 2026-02-11T23:31:28.256+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.257+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1033 ==== MOSDPGPush(4.0 119/117 [PushOp(4:d05ea685:::obj543:head, version: 111'543, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d05ea685:::obj543:head@111'543, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e26680 con 0x507ac00 2026-02-11T23:31:28.257+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:d05ea685:::obj543:head, version: 111'543, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d05ea685:::obj543:head@111'543, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.257+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d05ea685:::obj543:head, version: 111'543, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d05ea685:::obj543:head@111'543, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.257+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.257+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1034 ==== pg_backfill(progress 4.0 e 119/119 lb 4:cf1e3413:::obj224:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e100 con 0x507ac00 2026-02-11T23:31:28.257+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:cf1e3413:::obj224:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.257+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d05ea685:::obj543:head, version: 111'543, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d05ea685:::obj543:head@111'543, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.257+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:cf1e3413:::obj224:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.257+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.257+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d05ea685:::obj543:head, version: 111'543, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d05ea685:::obj543:head@111'543, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.257+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.257+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:cf1e3413:::obj224:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.257+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:d05ea685:::obj543:head, version: 111'543, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d05ea685:::obj543:head@111'543, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 119/117 [PushOp(4:d05ea685:::obj543:head, version: 111'543, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d05ea685:::obj543:head@111'543, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:cde49336:::obj263:head local-lis/les=117/118 n=496 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.257+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.257+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cde49336:::obj263:head local-lis/les=117/118 n=496 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:d05ea685:::obj543:head, version: 111'543, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d05ea685:::obj543:head@111'543, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.257+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cde49336:::obj263:head local-lis/les=117/118 n=496 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:d05ea685:::obj543:head@111'543, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.257+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cde49336:::obj263:head local-lis/les=117/118 n=496 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:d05ea685:::obj543:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.257+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cde49336:::obj263:head local-lis/les=117/118 n=496 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:d05ea685:::obj543:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.257+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cde49336:::obj263:head local-lis/les=117/118 n=496 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:d05ea685:::obj543:head 2026-02-11T23:31:28.257+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cde49336:::obj263:head local-lis/les=117/118 n=496 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:d05ea685:::obj543:head v 111'543 2026-02-11T23:31:28.257+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cde49336:::obj263:head local-lis/les=117/118 n=496 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.257+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.257+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:d05ea685:::obj543:head, version: 111'543, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d05ea685:::obj543:head@111'543, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000077 2026-02-11T23:31:28.257+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.257+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.257+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:cf1e3413:::obj224:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.257+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:cf1e3413:::obj224:head) v3 prio 15 cost 0 latency 0.000094 pg_backfill(progress 4.0 e 119/119 lb 4:cf1e3413:::obj224:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:cde49336:::obj263:head local-lis/les=117/118 n=496 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.257+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.257+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cde49336:::obj263:head local-lis/les=117/118 n=496 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:cf1e3413:::obj224:head) v3 2026-02-11T23:31:28.257+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cde49336:::obj263:head local-lis/les=117/118 n=496 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:cf1e3413:::obj224:head) 2026-02-11T23:31:28.257+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.257+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:cf1e3413:::obj224:head) v3 finish latency 0.000014 2026-02-11T23:31:28.257+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cf1e3413:::obj224:head local-lis/les=117/118 n=497 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.257+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.257+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.257+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cf1e3413:::obj224:head local-lis/les=117/118 n=497 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.257+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.257+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.257+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.257+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x674b800 con 0x507ac00 2026-02-11T23:31:28.257+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:d05ea685:::obj543:head)]) -- 0x671d680 con 0x507ac00 2026-02-11T23:31:28.257+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.257+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1035 ==== MOSDPGPush(4.0 119/117 [PushOp(4:d06c69cb:::obj532:head, version: 111'532, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d06c69cb:::obj532:head@111'532, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a3860 con 0x507ac00 2026-02-11T23:31:28.257+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:d06c69cb:::obj532:head, version: 111'532, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d06c69cb:::obj532:head@111'532, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.257+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d06c69cb:::obj532:head, version: 111'532, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d06c69cb:::obj532:head@111'532, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.257+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.257+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1036 ==== pg_backfill(progress 4.0 e 119/119 lb 4:d05ea685:::obj543:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2c000 con 0x507ac00 2026-02-11T23:31:28.257+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:d05ea685:::obj543:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.257+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d06c69cb:::obj532:head, version: 111'532, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d06c69cb:::obj532:head@111'532, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.257+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d05ea685:::obj543:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.257+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.257+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d06c69cb:::obj532:head, version: 111'532, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d06c69cb:::obj532:head@111'532, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.257+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.257+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d05ea685:::obj543:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.257+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:d06c69cb:::obj532:head, version: 111'532, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d06c69cb:::obj532:head@111'532, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 119/117 [PushOp(4:d06c69cb:::obj532:head, version: 111'532, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d06c69cb:::obj532:head@111'532, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:cf1e3413:::obj224:head local-lis/les=117/118 n=497 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.257+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.257+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cf1e3413:::obj224:head local-lis/les=117/118 n=497 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:d06c69cb:::obj532:head, version: 111'532, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d06c69cb:::obj532:head@111'532, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.257+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cf1e3413:::obj224:head local-lis/les=117/118 n=497 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:d06c69cb:::obj532:head@111'532, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.257+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cf1e3413:::obj224:head local-lis/les=117/118 n=497 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:d06c69cb:::obj532:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.257+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cf1e3413:::obj224:head local-lis/les=117/118 n=497 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:d06c69cb:::obj532:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.257+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cf1e3413:::obj224:head local-lis/les=117/118 n=497 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:d06c69cb:::obj532:head 2026-02-11T23:31:28.257+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cf1e3413:::obj224:head local-lis/les=117/118 n=497 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:d06c69cb:::obj532:head v 111'532 2026-02-11T23:31:28.257+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cf1e3413:::obj224:head local-lis/les=117/118 n=497 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.257+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.257+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:d06c69cb:::obj532:head, version: 111'532, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d06c69cb:::obj532:head@111'532, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000066 2026-02-11T23:31:28.257+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.257+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.257+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d05ea685:::obj543:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.257+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:d05ea685:::obj543:head) v3 prio 15 cost 0 latency 0.000084 pg_backfill(progress 4.0 e 119/119 lb 4:d05ea685:::obj543:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:cf1e3413:::obj224:head local-lis/les=117/118 n=497 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.257+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.257+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cf1e3413:::obj224:head local-lis/les=117/118 n=497 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:d05ea685:::obj543:head) v3 2026-02-11T23:31:28.257+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:cf1e3413:::obj224:head local-lis/les=117/118 n=497 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:d05ea685:::obj543:head) 2026-02-11T23:31:28.257+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.257+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:d05ea685:::obj543:head) v3 finish latency 0.000015 2026-02-11T23:31:28.257+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d05ea685:::obj543:head local-lis/les=117/118 n=498 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.257+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.257+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.257+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d05ea685:::obj543:head local-lis/les=117/118 n=498 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.257+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.257+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.257+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.257+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x674b980 con 0x507ac00 2026-02-11T23:31:28.257+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.257+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:d06c69cb:::obj532:head)]) -- 0x642cd80 con 0x507ac00 2026-02-11T23:31:28.257+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.258+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1037 ==== MOSDPGPush(4.0 119/117 [PushOp(4:d0e799f1:::obj299:head, version: 111'299, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d0e799f1:::obj299:head@111'299, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56cab60 con 0x507ac00 2026-02-11T23:31:28.258+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:d0e799f1:::obj299:head, version: 111'299, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d0e799f1:::obj299:head@111'299, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.258+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d0e799f1:::obj299:head, version: 111'299, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d0e799f1:::obj299:head@111'299, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.258+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.258+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d0e799f1:::obj299:head, version: 111'299, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d0e799f1:::obj299:head@111'299, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.258+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.258+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1038 ==== pg_backfill(progress 4.0 e 119/119 lb 4:d06c69cb:::obj532:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2cb00 con 0x507ac00 2026-02-11T23:31:28.258+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:d06c69cb:::obj532:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.258+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d0e799f1:::obj299:head, version: 111'299, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d0e799f1:::obj299:head@111'299, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.258+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d06c69cb:::obj532:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.258+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.258+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d06c69cb:::obj532:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.258+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:d0e799f1:::obj299:head, version: 111'299, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d0e799f1:::obj299:head@111'299, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(4.0 119/117 [PushOp(4:d0e799f1:::obj299:head, version: 111'299, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d0e799f1:::obj299:head@111'299, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:d05ea685:::obj543:head local-lis/les=117/118 n=498 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.258+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.258+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d05ea685:::obj543:head local-lis/les=117/118 n=498 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:d0e799f1:::obj299:head, version: 111'299, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d0e799f1:::obj299:head@111'299, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.258+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d05ea685:::obj543:head local-lis/les=117/118 n=498 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:d0e799f1:::obj299:head@111'299, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.258+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d05ea685:::obj543:head local-lis/les=117/118 n=498 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:d0e799f1:::obj299:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.258+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d05ea685:::obj543:head local-lis/les=117/118 n=498 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:d0e799f1:::obj299:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.258+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d05ea685:::obj543:head local-lis/les=117/118 n=498 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:d0e799f1:::obj299:head 2026-02-11T23:31:28.258+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d05ea685:::obj543:head local-lis/les=117/118 n=498 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:d0e799f1:::obj299:head v 111'299 2026-02-11T23:31:28.258+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d05ea685:::obj543:head local-lis/les=117/118 n=498 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.258+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.258+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:d0e799f1:::obj299:head, version: 111'299, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d0e799f1:::obj299:head@111'299, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000067 2026-02-11T23:31:28.258+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d05ea685:::obj543:head local-lis/les=117/118 n=498 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.258+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.258+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.258+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d06c69cb:::obj532:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.258+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:d06c69cb:::obj532:head) v3 prio 15 cost 0 latency 0.000085 pg_backfill(progress 4.0 e 119/119 lb 4:d06c69cb:::obj532:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:d05ea685:::obj543:head local-lis/les=117/118 n=498 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.258+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.258+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d05ea685:::obj543:head local-lis/les=117/118 n=498 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:d06c69cb:::obj532:head) v3 2026-02-11T23:31:28.258+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d05ea685:::obj543:head local-lis/les=117/118 n=498 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:d06c69cb:::obj532:head) 2026-02-11T23:31:28.258+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.258+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:d06c69cb:::obj532:head) v3 finish latency 0.000018 2026-02-11T23:31:28.258+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.258+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.258+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d06c69cb:::obj532:head local-lis/les=117/118 n=499 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.258+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.258+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.258+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.258+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x674be00 con 0x507ac00 2026-02-11T23:31:28.258+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:d0e799f1:::obj299:head)]) -- 0x674bb00 con 0x507ac00 2026-02-11T23:31:28.258+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.258+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1039 ==== MOSDPGPush(4.0 119/117 [PushOp(4:d0f049f1:::obj387:head, version: 111'387, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d0f049f1:::obj387:head@111'387, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2680 con 0x507ac00 2026-02-11T23:31:28.258+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:d0f049f1:::obj387:head, version: 111'387, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d0f049f1:::obj387:head@111'387, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 119 2026-02-11T23:31:28.258+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d0f049f1:::obj387:head, version: 111'387, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d0f049f1:::obj387:head@111'387, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.258+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.258+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d0f049f1:::obj387:head, version: 111'387, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d0f049f1:::obj387:head@111'387, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.258+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1040 ==== pg_backfill(progress 4.0 e 119/119 lb 4:d0e799f1:::obj299:head) ==== 1062+0+0 (unknown 0 0 0) 0x6179700 con 0x507ac00 2026-02-11T23:31:28.258+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:d0e799f1:::obj299:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 119 2026-02-11T23:31:28.258+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.258+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d0f049f1:::obj387:head, version: 111'387, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d0f049f1:::obj387:head@111'387, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.258+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d0e799f1:::obj299:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.258+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.258+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:d0f049f1:::obj387:head, version: 111'387, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d0f049f1:::obj387:head@111'387, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(4.0 119/117 [PushOp(4:d0f049f1:::obj387:head, version: 111'387, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d0f049f1:::obj387:head@111'387, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:d06c69cb:::obj532:head local-lis/les=117/118 n=499 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.258+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d0e799f1:::obj299:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.258+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.258+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d06c69cb:::obj532:head local-lis/les=117/118 n=499 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:d0f049f1:::obj387:head, version: 111'387, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d0f049f1:::obj387:head@111'387, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.258+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d06c69cb:::obj532:head local-lis/les=117/118 n=499 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:d0f049f1:::obj387:head@111'387, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.258+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d06c69cb:::obj532:head local-lis/les=117/118 n=499 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:d0f049f1:::obj387:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.258+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d06c69cb:::obj532:head local-lis/les=117/118 n=499 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:d0f049f1:::obj387:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.258+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d06c69cb:::obj532:head local-lis/les=117/118 n=499 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:d0f049f1:::obj387:head 2026-02-11T23:31:28.258+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d06c69cb:::obj532:head local-lis/les=117/118 n=499 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:d0f049f1:::obj387:head v 111'387 2026-02-11T23:31:28.258+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d06c69cb:::obj532:head local-lis/les=117/118 n=499 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.258+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.258+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:d0f049f1:::obj387:head, version: 111'387, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d0f049f1:::obj387:head@111'387, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:28.258+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.258+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.258+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d0e799f1:::obj299:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.258+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:d0e799f1:::obj299:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 4.0 e 119/119 lb 4:d0e799f1:::obj299:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:d06c69cb:::obj532:head local-lis/les=117/118 n=499 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.258+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.258+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d06c69cb:::obj532:head local-lis/les=117/118 n=499 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:d0e799f1:::obj299:head) v3 2026-02-11T23:31:28.259+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d06c69cb:::obj532:head local-lis/les=117/118 n=499 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:d0e799f1:::obj299:head) 2026-02-11T23:31:28.259+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.259+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:d0e799f1:::obj299:head) v3 finish latency 0.000020 2026-02-11T23:31:28.259+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d0e799f1:::obj299:head local-lis/les=117/118 n=500 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.259+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.259+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.259+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d0e799f1:::obj299:head local-lis/les=117/118 n=500 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.259+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.259+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.259+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.259+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x674bb00 con 0x507ac00 2026-02-11T23:31:28.259+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:d0f049f1:::obj387:head)]) -- 0x671d980 con 0x507ac00 2026-02-11T23:31:28.259+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.259+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1041 ==== MOSDPGPush(4.0 119/117 [PushOp(4:d11b9b34:::obj309:head, version: 111'309, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d11b9b34:::obj309:head@111'309, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3815380 con 0x507ac00 2026-02-11T23:31:28.259+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:d11b9b34:::obj309:head, version: 111'309, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d11b9b34:::obj309:head@111'309, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.259+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d11b9b34:::obj309:head, version: 111'309, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d11b9b34:::obj309:head@111'309, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.259+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.259+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1042 ==== pg_backfill(progress 4.0 e 119/119 lb 4:d0f049f1:::obj387:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176000 con 0x507ac00 2026-02-11T23:31:28.259+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:d0f049f1:::obj387:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.259+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d11b9b34:::obj309:head, version: 111'309, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d11b9b34:::obj309:head@111'309, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.259+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d0f049f1:::obj387:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.259+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.259+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d11b9b34:::obj309:head, version: 111'309, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d11b9b34:::obj309:head@111'309, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.259+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:d11b9b34:::obj309:head, version: 111'309, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d11b9b34:::obj309:head@111'309, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(4.0 119/117 [PushOp(4:d11b9b34:::obj309:head, version: 111'309, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d11b9b34:::obj309:head@111'309, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:d0e799f1:::obj299:head local-lis/les=117/118 n=500 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.259+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.259+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.259+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d0f049f1:::obj387:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.259+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d0e799f1:::obj299:head local-lis/les=117/118 n=500 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:d11b9b34:::obj309:head, version: 111'309, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d11b9b34:::obj309:head@111'309, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.259+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d0e799f1:::obj299:head local-lis/les=117/118 n=500 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:d11b9b34:::obj309:head@111'309, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.259+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d0e799f1:::obj299:head local-lis/les=117/118 n=500 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:d11b9b34:::obj309:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.259+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d0e799f1:::obj299:head local-lis/les=117/118 n=500 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:d11b9b34:::obj309:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.259+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d0e799f1:::obj299:head local-lis/les=117/118 n=500 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:d11b9b34:::obj309:head 2026-02-11T23:31:28.259+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d0e799f1:::obj299:head local-lis/les=117/118 n=500 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:d11b9b34:::obj309:head v 111'309 2026-02-11T23:31:28.259+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d0e799f1:::obj299:head local-lis/les=117/118 n=500 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.259+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.259+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:d11b9b34:::obj309:head, version: 111'309, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d11b9b34:::obj309:head@111'309, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000070 2026-02-11T23:31:28.259+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.259+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.259+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d0f049f1:::obj387:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.259+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:d0f049f1:::obj387:head) v3 prio 15 cost 0 latency 0.000088 pg_backfill(progress 4.0 e 119/119 lb 4:d0f049f1:::obj387:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:d0e799f1:::obj299:head local-lis/les=117/118 n=500 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.259+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.259+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d0e799f1:::obj299:head local-lis/les=117/118 n=500 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:d0f049f1:::obj387:head) v3 2026-02-11T23:31:28.259+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d0e799f1:::obj299:head local-lis/les=117/118 n=500 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:d0f049f1:::obj387:head) 2026-02-11T23:31:28.259+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.259+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:d0f049f1:::obj387:head) v3 finish latency 0.000016 2026-02-11T23:31:28.259+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d0f049f1:::obj387:head local-lis/les=117/118 n=501 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.259+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.259+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.259+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d0f049f1:::obj387:head local-lis/les=117/118 n=501 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.259+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.259+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.259+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.259+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x674be00 con 0x507ac00 2026-02-11T23:31:28.259+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:d11b9b34:::obj309:head)]) -- 0x63c9e00 con 0x507ac00 2026-02-11T23:31:28.259+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.260+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1043 ==== MOSDPGPush(4.0 119/117 [PushOp(4:d1337354:::obj10:head, version: 111'10, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d1337354:::obj10:head@111'10, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5e26b60 con 0x507ac00 2026-02-11T23:31:28.260+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:d1337354:::obj10:head, version: 111'10, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d1337354:::obj10:head@111'10, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.260+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d1337354:::obj10:head, version: 111'10, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d1337354:::obj10:head@111'10, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.260+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.260+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1044 ==== pg_backfill(progress 4.0 e 119/119 lb 4:d11b9b34:::obj309:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9a680 con 0x507ac00 2026-02-11T23:31:28.260+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:d11b9b34:::obj309:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.260+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d1337354:::obj10:head, version: 111'10, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d1337354:::obj10:head@111'10, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.260+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d11b9b34:::obj309:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.260+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.260+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d1337354:::obj10:head, version: 111'10, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d1337354:::obj10:head@111'10, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.260+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.260+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d11b9b34:::obj309:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.260+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:d1337354:::obj10:head, version: 111'10, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d1337354:::obj10:head@111'10, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 119/117 [PushOp(4:d1337354:::obj10:head, version: 111'10, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d1337354:::obj10:head@111'10, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:d0f049f1:::obj387:head local-lis/les=117/118 n=501 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.260+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.260+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d0f049f1:::obj387:head local-lis/les=117/118 n=501 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:d1337354:::obj10:head, version: 111'10, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d1337354:::obj10:head@111'10, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.260+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d0f049f1:::obj387:head local-lis/les=117/118 n=501 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:d1337354:::obj10:head@111'10, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.260+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d0f049f1:::obj387:head local-lis/les=117/118 n=501 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:d1337354:::obj10:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.260+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d0f049f1:::obj387:head local-lis/les=117/118 n=501 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:d1337354:::obj10:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.260+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d0f049f1:::obj387:head local-lis/les=117/118 n=501 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:d1337354:::obj10:head 2026-02-11T23:31:28.260+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d0f049f1:::obj387:head local-lis/les=117/118 n=501 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:d1337354:::obj10:head v 111'10 2026-02-11T23:31:28.260+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d0f049f1:::obj387:head local-lis/les=117/118 n=501 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.260+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.260+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:d1337354:::obj10:head, version: 111'10, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d1337354:::obj10:head@111'10, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000071 2026-02-11T23:31:28.260+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.260+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.260+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d11b9b34:::obj309:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.260+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:d11b9b34:::obj309:head) v3 prio 15 cost 0 latency 0.000089 pg_backfill(progress 4.0 e 119/119 lb 4:d11b9b34:::obj309:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:d0f049f1:::obj387:head local-lis/les=117/118 n=501 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.260+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.260+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d0f049f1:::obj387:head local-lis/les=117/118 n=501 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:d11b9b34:::obj309:head) v3 2026-02-11T23:31:28.260+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d0f049f1:::obj387:head local-lis/les=117/118 n=501 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:d11b9b34:::obj309:head) 2026-02-11T23:31:28.260+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.260+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:d11b9b34:::obj309:head) v3 finish latency 0.000014 2026-02-11T23:31:28.260+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d11b9b34:::obj309:head local-lis/les=117/118 n=502 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.260+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.260+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.260+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d11b9b34:::obj309:head local-lis/les=117/118 n=502 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.260+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.260+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.260+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.260+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x642cd80 con 0x507ac00 2026-02-11T23:31:28.260+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:d1337354:::obj10:head)]) -- 0x671dc80 con 0x507ac00 2026-02-11T23:31:28.260+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.260+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1045 ==== MOSDPGPush(4.0 119/117 [PushOp(4:d1c8135f:::obj67:head, version: 111'67, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d1c8135f:::obj67:head@111'67, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x56a29c0 con 0x507ac00 2026-02-11T23:31:28.260+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:d1c8135f:::obj67:head, version: 111'67, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d1c8135f:::obj67:head@111'67, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.260+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d1c8135f:::obj67:head, version: 111'67, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d1c8135f:::obj67:head@111'67, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.260+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.260+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1046 ==== pg_backfill(progress 4.0 e 119/119 lb 4:d1337354:::obj10:head) ==== 1061+0+0 (unknown 0 0 0) 0x5985700 con 0x507ac00 2026-02-11T23:31:28.260+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:d1337354:::obj10:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.260+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d1c8135f:::obj67:head, version: 111'67, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d1c8135f:::obj67:head@111'67, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.260+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d1337354:::obj10:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.260+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.260+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d1c8135f:::obj67:head, version: 111'67, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d1c8135f:::obj67:head@111'67, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.260+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.260+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d1337354:::obj10:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.260+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:d1c8135f:::obj67:head, version: 111'67, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d1c8135f:::obj67:head@111'67, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 119/117 [PushOp(4:d1c8135f:::obj67:head, version: 111'67, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d1c8135f:::obj67:head@111'67, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:d11b9b34:::obj309:head local-lis/les=117/118 n=502 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.260+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.260+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d11b9b34:::obj309:head local-lis/les=117/118 n=502 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:d1c8135f:::obj67:head, version: 111'67, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d1c8135f:::obj67:head@111'67, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.260+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d11b9b34:::obj309:head local-lis/les=117/118 n=502 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:d1c8135f:::obj67:head@111'67, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.260+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d11b9b34:::obj309:head local-lis/les=117/118 n=502 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:d1c8135f:::obj67:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.260+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d11b9b34:::obj309:head local-lis/les=117/118 n=502 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:d1c8135f:::obj67:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.260+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d11b9b34:::obj309:head local-lis/les=117/118 n=502 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:d1c8135f:::obj67:head 2026-02-11T23:31:28.260+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d11b9b34:::obj309:head local-lis/les=117/118 n=502 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:d1c8135f:::obj67:head v 111'67 2026-02-11T23:31:28.260+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d11b9b34:::obj309:head local-lis/les=117/118 n=502 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.260+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.260+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:d1c8135f:::obj67:head, version: 111'67, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d1c8135f:::obj67:head@111'67, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000068 2026-02-11T23:31:28.260+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.260+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.260+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d1337354:::obj10:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.260+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:d1337354:::obj10:head) v3 prio 15 cost 0 latency 0.000087 pg_backfill(progress 4.0 e 119/119 lb 4:d1337354:::obj10:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:d11b9b34:::obj309:head local-lis/les=117/118 n=502 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.260+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.260+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d11b9b34:::obj309:head local-lis/les=117/118 n=502 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:d1337354:::obj10:head) v3 2026-02-11T23:31:28.260+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d11b9b34:::obj309:head local-lis/les=117/118 n=502 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:d1337354:::obj10:head) 2026-02-11T23:31:28.260+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.260+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:d1337354:::obj10:head) v3 finish latency 0.000014 2026-02-11T23:31:28.260+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d1337354:::obj10:head local-lis/les=117/118 n=503 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.260+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.260+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.260+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d1337354:::obj10:head local-lis/les=117/118 n=503 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.260+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.260+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.260+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.260+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x674b980 con 0x507ac00 2026-02-11T23:31:28.260+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:d1c8135f:::obj67:head)]) -- 0x642d500 con 0x507ac00 2026-02-11T23:31:28.260+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.261+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1047 ==== MOSDPGPush(4.0 119/117 [PushOp(4:d33f403b:::obj434:head, version: 111'434, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d33f403b:::obj434:head@111'434, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x554e340 con 0x507ac00 2026-02-11T23:31:28.261+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:d33f403b:::obj434:head, version: 111'434, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d33f403b:::obj434:head@111'434, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.261+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d33f403b:::obj434:head, version: 111'434, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d33f403b:::obj434:head@111'434, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.261+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.261+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1048 ==== pg_backfill(progress 4.0 e 119/119 lb 4:d1c8135f:::obj67:head) ==== 1061+0+0 (unknown 0 0 0) 0x6177600 con 0x507ac00 2026-02-11T23:31:28.261+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d33f403b:::obj434:head, version: 111'434, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d33f403b:::obj434:head@111'434, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.261+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:d1c8135f:::obj67:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.261+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d1c8135f:::obj67:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.261+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.261+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d33f403b:::obj434:head, version: 111'434, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d33f403b:::obj434:head@111'434, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.261+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.261+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:d33f403b:::obj434:head, version: 111'434, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d33f403b:::obj434:head@111'434, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 119/117 [PushOp(4:d33f403b:::obj434:head, version: 111'434, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d33f403b:::obj434:head@111'434, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:d1337354:::obj10:head local-lis/les=117/118 n=503 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.261+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d1c8135f:::obj67:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.261+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.261+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d1337354:::obj10:head local-lis/les=117/118 n=503 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:d33f403b:::obj434:head, version: 111'434, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d33f403b:::obj434:head@111'434, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.261+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d1337354:::obj10:head local-lis/les=117/118 n=503 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:d33f403b:::obj434:head@111'434, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.261+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d1337354:::obj10:head local-lis/les=117/118 n=503 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:d33f403b:::obj434:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.261+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d1337354:::obj10:head local-lis/les=117/118 n=503 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:d33f403b:::obj434:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.261+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d1337354:::obj10:head local-lis/les=117/118 n=503 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:d33f403b:::obj434:head 2026-02-11T23:31:28.261+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d1337354:::obj10:head local-lis/les=117/118 n=503 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:d33f403b:::obj434:head v 111'434 2026-02-11T23:31:28.261+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d1337354:::obj10:head local-lis/les=117/118 n=503 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.261+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.261+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:d33f403b:::obj434:head, version: 111'434, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d33f403b:::obj434:head@111'434, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000069 2026-02-11T23:31:28.261+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d1337354:::obj10:head local-lis/les=117/118 n=503 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.261+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.261+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.261+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d1c8135f:::obj67:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.261+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:d1c8135f:::obj67:head) v3 prio 15 cost 0 latency 0.000091 pg_backfill(progress 4.0 e 119/119 lb 4:d1c8135f:::obj67:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:d1337354:::obj10:head local-lis/les=117/118 n=503 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.261+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.261+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d1337354:::obj10:head local-lis/les=117/118 n=503 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:d1c8135f:::obj67:head) v3 2026-02-11T23:31:28.261+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d1337354:::obj10:head local-lis/les=117/118 n=503 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:d1c8135f:::obj67:head) 2026-02-11T23:31:28.261+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.261+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:d1c8135f:::obj67:head) v3 finish latency 0.000018 2026-02-11T23:31:28.261+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.261+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.261+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d1c8135f:::obj67:head local-lis/les=117/118 n=504 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.261+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.261+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.261+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.261+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66d9080 con 0x507ac00 2026-02-11T23:31:28.261+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:d33f403b:::obj434:head)]) -- 0x671d680 con 0x507ac00 2026-02-11T23:31:28.261+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.261+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1049 ==== MOSDPGPush(4.0 119/117 [PushOp(4:d3f6948e:::obj218:head, version: 111'218, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d3f6948e:::obj218:head@111'218, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5646d00 con 0x507ac00 2026-02-11T23:31:28.261+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:d3f6948e:::obj218:head, version: 111'218, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d3f6948e:::obj218:head@111'218, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 119 2026-02-11T23:31:28.261+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d3f6948e:::obj218:head, version: 111'218, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d3f6948e:::obj218:head@111'218, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.261+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.261+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1050 ==== pg_backfill(progress 4.0 e 119/119 lb 4:d33f403b:::obj434:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177b80 con 0x507ac00 2026-02-11T23:31:28.261+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d3f6948e:::obj218:head, version: 111'218, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d3f6948e:::obj218:head@111'218, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.261+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:d33f403b:::obj434:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.261+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.261+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d33f403b:::obj434:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.261+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d3f6948e:::obj218:head, version: 111'218, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d3f6948e:::obj218:head@111'218, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.261+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.261+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:d3f6948e:::obj218:head, version: 111'218, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d3f6948e:::obj218:head@111'218, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 119/117 [PushOp(4:d3f6948e:::obj218:head, version: 111'218, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d3f6948e:::obj218:head@111'218, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:d1c8135f:::obj67:head local-lis/les=117/118 n=504 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.261+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.261+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d1c8135f:::obj67:head local-lis/les=117/118 n=504 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:d3f6948e:::obj218:head, version: 111'218, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d3f6948e:::obj218:head@111'218, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.261+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d1c8135f:::obj67:head local-lis/les=117/118 n=504 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:d3f6948e:::obj218:head@111'218, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.261+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.261+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d33f403b:::obj434:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.261+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d1c8135f:::obj67:head local-lis/les=117/118 n=504 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:d3f6948e:::obj218:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.261+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d1c8135f:::obj67:head local-lis/les=117/118 n=504 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:d3f6948e:::obj218:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.261+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d1c8135f:::obj67:head local-lis/les=117/118 n=504 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:d3f6948e:::obj218:head 2026-02-11T23:31:28.261+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d1c8135f:::obj67:head local-lis/les=117/118 n=504 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:d3f6948e:::obj218:head v 111'218 2026-02-11T23:31:28.261+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d1c8135f:::obj67:head local-lis/les=117/118 n=504 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.261+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.261+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:d3f6948e:::obj218:head, version: 111'218, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d3f6948e:::obj218:head@111'218, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:28.261+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.261+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.261+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d33f403b:::obj434:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.261+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:d33f403b:::obj434:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 4.0 e 119/119 lb 4:d33f403b:::obj434:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:d1c8135f:::obj67:head local-lis/les=117/118 n=504 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.261+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.261+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d1c8135f:::obj67:head local-lis/les=117/118 n=504 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:d33f403b:::obj434:head) v3 2026-02-11T23:31:28.261+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d1c8135f:::obj67:head local-lis/les=117/118 n=504 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:d33f403b:::obj434:head) 2026-02-11T23:31:28.261+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.261+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:d33f403b:::obj434:head) v3 finish latency 0.000017 2026-02-11T23:31:28.261+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d33f403b:::obj434:head local-lis/les=117/118 n=505 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.261+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.262+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.262+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d33f403b:::obj434:head local-lis/les=117/118 n=505 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.262+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.262+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.262+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.262+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x674b680 con 0x507ac00 2026-02-11T23:31:28.262+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:d3f6948e:::obj218:head)]) -- 0x640fb00 con 0x507ac00 2026-02-11T23:31:28.262+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.262+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1051 ==== MOSDPGPush(4.0 119/117 [PushOp(4:d4006106:::obj329:head, version: 111'329, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d4006106:::obj329:head@111'329, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56ca680 con 0x507ac00 2026-02-11T23:31:28.262+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:d4006106:::obj329:head, version: 111'329, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d4006106:::obj329:head@111'329, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.262+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d4006106:::obj329:head, version: 111'329, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d4006106:::obj329:head@111'329, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.262+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.262+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d4006106:::obj329:head, version: 111'329, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d4006106:::obj329:head@111'329, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.262+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1052 ==== pg_backfill(progress 4.0 e 119/119 lb 4:d3f6948e:::obj218:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2c580 con 0x507ac00 2026-02-11T23:31:28.262+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:d3f6948e:::obj218:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.262+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.262+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d4006106:::obj329:head, version: 111'329, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d4006106:::obj329:head@111'329, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.262+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d3f6948e:::obj218:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.262+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:d4006106:::obj329:head, version: 111'329, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d4006106:::obj329:head@111'329, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 119/117 [PushOp(4:d4006106:::obj329:head, version: 111'329, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d4006106:::obj329:head@111'329, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:d33f403b:::obj434:head local-lis/les=117/118 n=505 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.262+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.262+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.262+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d3f6948e:::obj218:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.262+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d33f403b:::obj434:head local-lis/les=117/118 n=505 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:d4006106:::obj329:head, version: 111'329, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d4006106:::obj329:head@111'329, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.262+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d33f403b:::obj434:head local-lis/les=117/118 n=505 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:d4006106:::obj329:head@111'329, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.262+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d33f403b:::obj434:head local-lis/les=117/118 n=505 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:d4006106:::obj329:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.262+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d33f403b:::obj434:head local-lis/les=117/118 n=505 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:d4006106:::obj329:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.262+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d33f403b:::obj434:head local-lis/les=117/118 n=505 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:d4006106:::obj329:head 2026-02-11T23:31:28.262+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d33f403b:::obj434:head local-lis/les=117/118 n=505 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:d4006106:::obj329:head v 111'329 2026-02-11T23:31:28.262+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d33f403b:::obj434:head local-lis/les=117/118 n=505 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.262+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.262+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:d4006106:::obj329:head, version: 111'329, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d4006106:::obj329:head@111'329, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:28.262+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.262+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.262+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d3f6948e:::obj218:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.262+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:d3f6948e:::obj218:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 4.0 e 119/119 lb 4:d3f6948e:::obj218:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:d33f403b:::obj434:head local-lis/les=117/118 n=505 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.262+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.262+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d33f403b:::obj434:head local-lis/les=117/118 n=505 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:d3f6948e:::obj218:head) v3 2026-02-11T23:31:28.262+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d33f403b:::obj434:head local-lis/les=117/118 n=505 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:d3f6948e:::obj218:head) 2026-02-11T23:31:28.262+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.262+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:d3f6948e:::obj218:head) v3 finish latency 0.000015 2026-02-11T23:31:28.262+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d3f6948e:::obj218:head local-lis/les=117/118 n=506 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.262+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.262+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.262+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d3f6948e:::obj218:head local-lis/les=117/118 n=506 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.262+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.262+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.262+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.262+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x671d380 con 0x507ac00 2026-02-11T23:31:28.262+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:d4006106:::obj329:head)]) -- 0x671d080 con 0x507ac00 2026-02-11T23:31:28.262+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.262+0000 7f41633eb640 10 osd.0 119 tick 2026-02-11T23:31:28.262+0000 7f41633eb640 20 osd.0 119 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:28.263+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1053 ==== MOSDPGPush(4.0 119/117 [PushOp(4:d43593c2:::obj365:head, version: 111'365, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d43593c2:::obj365:head@111'365, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f09c0 con 0x507ac00 2026-02-11T23:31:28.263+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:d43593c2:::obj365:head, version: 111'365, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d43593c2:::obj365:head@111'365, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.263+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d43593c2:::obj365:head, version: 111'365, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d43593c2:::obj365:head@111'365, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.263+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.263+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1054 ==== pg_backfill(progress 4.0 e 119/119 lb 4:d4006106:::obj329:head) ==== 1062+0+0 (unknown 0 0 0) 0x6178100 con 0x507ac00 2026-02-11T23:31:28.263+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d43593c2:::obj365:head, version: 111'365, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d43593c2:::obj365:head@111'365, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.263+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:d4006106:::obj329:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.263+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d4006106:::obj329:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.263+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.263+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d43593c2:::obj365:head, version: 111'365, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d43593c2:::obj365:head@111'365, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.263+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.263+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:d43593c2:::obj365:head, version: 111'365, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d43593c2:::obj365:head@111'365, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 119/117 [PushOp(4:d43593c2:::obj365:head, version: 111'365, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d43593c2:::obj365:head@111'365, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:d3f6948e:::obj218:head local-lis/les=117/118 n=506 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.263+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.263+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.263+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d3f6948e:::obj218:head local-lis/les=117/118 n=506 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:d43593c2:::obj365:head, version: 111'365, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d43593c2:::obj365:head@111'365, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.263+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d4006106:::obj329:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.263+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d3f6948e:::obj218:head local-lis/les=117/118 n=506 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:d43593c2:::obj365:head@111'365, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.263+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d3f6948e:::obj218:head local-lis/les=117/118 n=506 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:d43593c2:::obj365:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.263+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d3f6948e:::obj218:head local-lis/les=117/118 n=506 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:d43593c2:::obj365:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.263+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d3f6948e:::obj218:head local-lis/les=117/118 n=506 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:d43593c2:::obj365:head 2026-02-11T23:31:28.263+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d3f6948e:::obj218:head local-lis/les=117/118 n=506 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:d43593c2:::obj365:head v 111'365 2026-02-11T23:31:28.263+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d3f6948e:::obj218:head local-lis/les=117/118 n=506 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.263+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.263+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:d43593c2:::obj365:head, version: 111'365, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d43593c2:::obj365:head@111'365, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000066 2026-02-11T23:31:28.263+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.263+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.263+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d4006106:::obj329:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.263+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:d4006106:::obj329:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 4.0 e 119/119 lb 4:d4006106:::obj329:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:d3f6948e:::obj218:head local-lis/les=117/118 n=506 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.263+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.263+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d3f6948e:::obj218:head local-lis/les=117/118 n=506 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:d4006106:::obj329:head) v3 2026-02-11T23:31:28.263+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d3f6948e:::obj218:head local-lis/les=117/118 n=506 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:d4006106:::obj329:head) 2026-02-11T23:31:28.263+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.263+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:d4006106:::obj329:head) v3 finish latency 0.000014 2026-02-11T23:31:28.263+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d4006106:::obj329:head local-lis/les=117/118 n=507 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.263+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.263+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.263+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d4006106:::obj329:head local-lis/les=117/118 n=507 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.263+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.263+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.263+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.263+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x674b500 con 0x507ac00 2026-02-11T23:31:28.263+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:d43593c2:::obj365:head)]) -- 0x5d55200 con 0x507ac00 2026-02-11T23:31:28.263+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.263+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1055 ==== MOSDPGPush(4.0 119/117 [PushOp(4:d4c3bba7:::obj106:head, version: 111'106, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d4c3bba7:::obj106:head@111'106, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a29c0 con 0x507ac00 2026-02-11T23:31:28.263+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:d4c3bba7:::obj106:head, version: 111'106, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d4c3bba7:::obj106:head@111'106, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.263+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d4c3bba7:::obj106:head, version: 111'106, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d4c3bba7:::obj106:head@111'106, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.263+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.263+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1056 ==== pg_backfill(progress 4.0 e 119/119 lb 4:d43593c2:::obj365:head) ==== 1062+0+0 (unknown 0 0 0) 0x6179180 con 0x507ac00 2026-02-11T23:31:28.263+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:d43593c2:::obj365:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.263+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d4c3bba7:::obj106:head, version: 111'106, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d4c3bba7:::obj106:head@111'106, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.263+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d43593c2:::obj365:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.263+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.263+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d43593c2:::obj365:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.263+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.263+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d4c3bba7:::obj106:head, version: 111'106, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d4c3bba7:::obj106:head@111'106, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.263+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:d4c3bba7:::obj106:head, version: 111'106, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d4c3bba7:::obj106:head@111'106, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000046 MOSDPGPush(4.0 119/117 [PushOp(4:d4c3bba7:::obj106:head, version: 111'106, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d4c3bba7:::obj106:head@111'106, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:d4006106:::obj329:head local-lis/les=117/118 n=507 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.263+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.263+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d4006106:::obj329:head local-lis/les=117/118 n=507 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:d4c3bba7:::obj106:head, version: 111'106, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d4c3bba7:::obj106:head@111'106, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.263+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d4006106:::obj329:head local-lis/les=117/118 n=507 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:d4c3bba7:::obj106:head@111'106, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.263+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d4006106:::obj329:head local-lis/les=117/118 n=507 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:d4c3bba7:::obj106:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.263+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d4006106:::obj329:head local-lis/les=117/118 n=507 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:d4c3bba7:::obj106:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.263+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d4006106:::obj329:head local-lis/les=117/118 n=507 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:d4c3bba7:::obj106:head 2026-02-11T23:31:28.263+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d4006106:::obj329:head local-lis/les=117/118 n=507 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:d4c3bba7:::obj106:head v 111'106 2026-02-11T23:31:28.263+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d4006106:::obj329:head local-lis/les=117/118 n=507 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.263+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.263+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:d4c3bba7:::obj106:head, version: 111'106, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d4c3bba7:::obj106:head@111'106, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000073 2026-02-11T23:31:28.263+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.263+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.263+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d43593c2:::obj365:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.263+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:d43593c2:::obj365:head) v3 prio 15 cost 0 latency 0.000106 pg_backfill(progress 4.0 e 119/119 lb 4:d43593c2:::obj365:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:d4006106:::obj329:head local-lis/les=117/118 n=507 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.263+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.263+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d4006106:::obj329:head local-lis/les=117/118 n=507 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:d43593c2:::obj365:head) v3 2026-02-11T23:31:28.263+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d4006106:::obj329:head local-lis/les=117/118 n=507 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:d43593c2:::obj365:head) 2026-02-11T23:31:28.263+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.263+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:d43593c2:::obj365:head) v3 finish latency 0.000014 2026-02-11T23:31:28.263+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d43593c2:::obj365:head local-lis/les=117/118 n=508 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.263+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.263+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.263+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d43593c2:::obj365:head local-lis/les=117/118 n=508 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.263+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.263+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.263+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.263+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x640f680 con 0x507ac00 2026-02-11T23:31:28.263+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:d4c3bba7:::obj106:head)]) -- 0x66d9500 con 0x507ac00 2026-02-11T23:31:28.263+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.264+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1057 ==== MOSDPGPush(4.0 119/117 [PushOp(4:d5f69272:::obj407:head, version: 111'407, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d5f69272:::obj407:head@111'407, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e26b60 con 0x507ac00 2026-02-11T23:31:28.264+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:d5f69272:::obj407:head, version: 111'407, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d5f69272:::obj407:head@111'407, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.264+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d5f69272:::obj407:head, version: 111'407, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d5f69272:::obj407:head@111'407, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.264+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.264+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1058 ==== pg_backfill(progress 4.0 e 119/119 lb 4:d4c3bba7:::obj106:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9b180 con 0x507ac00 2026-02-11T23:31:28.264+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:d4c3bba7:::obj106:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.264+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d5f69272:::obj407:head, version: 111'407, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d5f69272:::obj407:head@111'407, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.264+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d4c3bba7:::obj106:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.264+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.264+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d5f69272:::obj407:head, version: 111'407, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d5f69272:::obj407:head@111'407, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.264+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:d5f69272:::obj407:head, version: 111'407, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d5f69272:::obj407:head@111'407, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 119/117 [PushOp(4:d5f69272:::obj407:head, version: 111'407, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d5f69272:::obj407:head@111'407, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:d43593c2:::obj365:head local-lis/les=117/118 n=508 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.264+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.264+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d4c3bba7:::obj106:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.264+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.264+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d43593c2:::obj365:head local-lis/les=117/118 n=508 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:d5f69272:::obj407:head, version: 111'407, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d5f69272:::obj407:head@111'407, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.264+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d43593c2:::obj365:head local-lis/les=117/118 n=508 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:d5f69272:::obj407:head@111'407, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.264+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d43593c2:::obj365:head local-lis/les=117/118 n=508 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:d5f69272:::obj407:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.264+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d43593c2:::obj365:head local-lis/les=117/118 n=508 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:d5f69272:::obj407:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.264+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d43593c2:::obj365:head local-lis/les=117/118 n=508 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:d5f69272:::obj407:head 2026-02-11T23:31:28.264+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d43593c2:::obj365:head local-lis/les=117/118 n=508 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:d5f69272:::obj407:head v 111'407 2026-02-11T23:31:28.264+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d43593c2:::obj365:head local-lis/les=117/118 n=508 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.264+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.264+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:d5f69272:::obj407:head, version: 111'407, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d5f69272:::obj407:head@111'407, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000070 2026-02-11T23:31:28.264+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.264+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.264+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d4c3bba7:::obj106:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.264+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:d4c3bba7:::obj106:head) v3 prio 15 cost 0 latency 0.000088 pg_backfill(progress 4.0 e 119/119 lb 4:d4c3bba7:::obj106:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:d43593c2:::obj365:head local-lis/les=117/118 n=508 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.264+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.264+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d43593c2:::obj365:head local-lis/les=117/118 n=508 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:d4c3bba7:::obj106:head) v3 2026-02-11T23:31:28.264+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d43593c2:::obj365:head local-lis/les=117/118 n=508 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:d4c3bba7:::obj106:head) 2026-02-11T23:31:28.264+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.264+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:d4c3bba7:::obj106:head) v3 finish latency 0.000019 2026-02-11T23:31:28.264+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d4c3bba7:::obj106:head local-lis/les=117/118 n=509 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.264+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.264+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.264+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d4c3bba7:::obj106:head local-lis/les=117/118 n=509 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.264+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.264+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.264+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.264+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x674b380 con 0x507ac00 2026-02-11T23:31:28.264+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:d5f69272:::obj407:head)]) -- 0x642de00 con 0x507ac00 2026-02-11T23:31:28.264+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.264+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1059 ==== MOSDPGPush(4.0 119/117 [PushOp(4:d6df9426:::obj469:head, version: 111'469, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d6df9426:::obj469:head@111'469, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3815380 con 0x507ac00 2026-02-11T23:31:28.264+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:d6df9426:::obj469:head, version: 111'469, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d6df9426:::obj469:head@111'469, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.264+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d6df9426:::obj469:head, version: 111'469, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d6df9426:::obj469:head@111'469, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.264+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.264+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1060 ==== pg_backfill(progress 4.0 e 119/119 lb 4:d5f69272:::obj407:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98000 con 0x507ac00 2026-02-11T23:31:28.264+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:d5f69272:::obj407:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.264+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d6df9426:::obj469:head, version: 111'469, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d6df9426:::obj469:head@111'469, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.264+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d5f69272:::obj407:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.264+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.264+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d6df9426:::obj469:head, version: 111'469, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d6df9426:::obj469:head@111'469, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.264+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:d6df9426:::obj469:head, version: 111'469, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d6df9426:::obj469:head@111'469, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 119/117 [PushOp(4:d6df9426:::obj469:head, version: 111'469, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d6df9426:::obj469:head@111'469, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:d4c3bba7:::obj106:head local-lis/les=117/118 n=509 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.264+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.264+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.264+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d5f69272:::obj407:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.264+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d4c3bba7:::obj106:head local-lis/les=117/118 n=509 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:d6df9426:::obj469:head, version: 111'469, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d6df9426:::obj469:head@111'469, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.264+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d4c3bba7:::obj106:head local-lis/les=117/118 n=509 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:d6df9426:::obj469:head@111'469, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.264+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d4c3bba7:::obj106:head local-lis/les=117/118 n=509 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:d6df9426:::obj469:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.264+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d4c3bba7:::obj106:head local-lis/les=117/118 n=509 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:d6df9426:::obj469:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.264+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d4c3bba7:::obj106:head local-lis/les=117/118 n=509 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:d6df9426:::obj469:head 2026-02-11T23:31:28.264+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d4c3bba7:::obj106:head local-lis/les=117/118 n=509 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:d6df9426:::obj469:head v 111'469 2026-02-11T23:31:28.264+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d4c3bba7:::obj106:head local-lis/les=117/118 n=509 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.264+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.265+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:d6df9426:::obj469:head, version: 111'469, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d6df9426:::obj469:head@111'469, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000094 2026-02-11T23:31:28.265+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.265+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.265+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d5f69272:::obj407:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.265+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:d5f69272:::obj407:head) v3 prio 15 cost 0 latency 0.000114 pg_backfill(progress 4.0 e 119/119 lb 4:d5f69272:::obj407:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:d4c3bba7:::obj106:head local-lis/les=117/118 n=509 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.265+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.265+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d4c3bba7:::obj106:head local-lis/les=117/118 n=509 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:d5f69272:::obj407:head) v3 2026-02-11T23:31:28.265+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d4c3bba7:::obj106:head local-lis/les=117/118 n=509 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:d5f69272:::obj407:head) 2026-02-11T23:31:28.265+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.265+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:d5f69272:::obj407:head) v3 finish latency 0.000015 2026-02-11T23:31:28.265+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d5f69272:::obj407:head local-lis/les=117/118 n=510 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.265+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.265+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.265+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d5f69272:::obj407:head local-lis/les=117/118 n=510 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.265+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.265+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.265+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.265+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x674b080 con 0x507ac00 2026-02-11T23:31:28.265+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:d6df9426:::obj469:head)]) -- 0x66d9200 con 0x507ac00 2026-02-11T23:31:28.265+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.266+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1061 ==== MOSDPGPush(4.0 119/117 [PushOp(4:d80bb902:::obj418:head, version: 111'418, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d80bb902:::obj418:head@111'418, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2680 con 0x507ac00 2026-02-11T23:31:28.266+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:d80bb902:::obj418:head, version: 111'418, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d80bb902:::obj418:head@111'418, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.266+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d80bb902:::obj418:head, version: 111'418, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d80bb902:::obj418:head@111'418, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.266+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.266+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1062 ==== pg_backfill(progress 4.0 e 119/119 lb 4:d6df9426:::obj469:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2ec00 con 0x507ac00 2026-02-11T23:31:28.266+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d80bb902:::obj418:head, version: 111'418, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d80bb902:::obj418:head@111'418, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.266+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:d6df9426:::obj469:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.266+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d6df9426:::obj469:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.266+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.266+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d80bb902:::obj418:head, version: 111'418, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d80bb902:::obj418:head@111'418, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.266+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.266+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:d80bb902:::obj418:head, version: 111'418, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d80bb902:::obj418:head@111'418, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(4.0 119/117 [PushOp(4:d80bb902:::obj418:head, version: 111'418, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d80bb902:::obj418:head@111'418, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:d5f69272:::obj407:head local-lis/les=117/118 n=510 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.266+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.266+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.266+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d5f69272:::obj407:head local-lis/les=117/118 n=510 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:d80bb902:::obj418:head, version: 111'418, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d80bb902:::obj418:head@111'418, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.266+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d6df9426:::obj469:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.266+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d5f69272:::obj407:head local-lis/les=117/118 n=510 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:d80bb902:::obj418:head@111'418, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.266+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d5f69272:::obj407:head local-lis/les=117/118 n=510 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:d80bb902:::obj418:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.266+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d5f69272:::obj407:head local-lis/les=117/118 n=510 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:d80bb902:::obj418:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.266+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d5f69272:::obj407:head local-lis/les=117/118 n=510 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:d80bb902:::obj418:head 2026-02-11T23:31:28.266+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d5f69272:::obj407:head local-lis/les=117/118 n=510 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:d80bb902:::obj418:head v 111'418 2026-02-11T23:31:28.266+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d5f69272:::obj407:head local-lis/les=117/118 n=510 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.266+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.266+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:d80bb902:::obj418:head, version: 111'418, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d80bb902:::obj418:head@111'418, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000067 2026-02-11T23:31:28.266+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d5f69272:::obj407:head local-lis/les=117/118 n=510 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.266+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.266+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.266+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d6df9426:::obj469:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.266+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:d6df9426:::obj469:head) v3 prio 15 cost 0 latency 0.000092 pg_backfill(progress 4.0 e 119/119 lb 4:d6df9426:::obj469:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:d5f69272:::obj407:head local-lis/les=117/118 n=510 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.266+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.266+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d5f69272:::obj407:head local-lis/les=117/118 n=510 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:d6df9426:::obj469:head) v3 2026-02-11T23:31:28.266+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d5f69272:::obj407:head local-lis/les=117/118 n=510 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:d6df9426:::obj469:head) 2026-02-11T23:31:28.266+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.266+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:d6df9426:::obj469:head) v3 finish latency 0.000028 2026-02-11T23:31:28.266+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.266+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.266+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d6df9426:::obj469:head local-lis/les=117/118 n=511 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.266+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.266+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.266+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.266+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x674af00 con 0x507ac00 2026-02-11T23:31:28.266+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:d80bb902:::obj418:head)]) -- 0x674b200 con 0x507ac00 2026-02-11T23:31:28.266+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.266+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1063 ==== MOSDPGPush(4.0 119/117 [PushOp(4:d83b1cbf:::obj272:head, version: 111'272, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d83b1cbf:::obj272:head@111'272, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56cab60 con 0x507ac00 2026-02-11T23:31:28.266+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:d83b1cbf:::obj272:head, version: 111'272, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d83b1cbf:::obj272:head@111'272, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 119 2026-02-11T23:31:28.266+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d83b1cbf:::obj272:head, version: 111'272, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d83b1cbf:::obj272:head@111'272, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.266+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.266+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1064 ==== pg_backfill(progress 4.0 e 119/119 lb 4:d80bb902:::obj418:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2d600 con 0x507ac00 2026-02-11T23:31:28.266+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:d80bb902:::obj418:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.266+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d80bb902:::obj418:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.266+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d83b1cbf:::obj272:head, version: 111'272, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d83b1cbf:::obj272:head@111'272, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.266+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.266+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d83b1cbf:::obj272:head, version: 111'272, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d83b1cbf:::obj272:head@111'272, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.266+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.266+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d80bb902:::obj418:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.266+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:d83b1cbf:::obj272:head, version: 111'272, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d83b1cbf:::obj272:head@111'272, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(4.0 119/117 [PushOp(4:d83b1cbf:::obj272:head, version: 111'272, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d83b1cbf:::obj272:head@111'272, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:d6df9426:::obj469:head local-lis/les=117/118 n=511 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.266+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.266+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d6df9426:::obj469:head local-lis/les=117/118 n=511 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:d83b1cbf:::obj272:head, version: 111'272, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d83b1cbf:::obj272:head@111'272, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.266+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d6df9426:::obj469:head local-lis/les=117/118 n=511 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:d83b1cbf:::obj272:head@111'272, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.266+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d6df9426:::obj469:head local-lis/les=117/118 n=511 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:d83b1cbf:::obj272:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.266+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d6df9426:::obj469:head local-lis/les=117/118 n=511 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:d83b1cbf:::obj272:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.266+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d6df9426:::obj469:head local-lis/les=117/118 n=511 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:d83b1cbf:::obj272:head 2026-02-11T23:31:28.266+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d6df9426:::obj469:head local-lis/les=117/118 n=511 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:d83b1cbf:::obj272:head v 111'272 2026-02-11T23:31:28.266+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d6df9426:::obj469:head local-lis/les=117/118 n=511 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.266+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.266+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:d83b1cbf:::obj272:head, version: 111'272, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d83b1cbf:::obj272:head@111'272, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000086 2026-02-11T23:31:28.266+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.266+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.266+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d80bb902:::obj418:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.266+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:d80bb902:::obj418:head) v3 prio 15 cost 0 latency 0.000116 pg_backfill(progress 4.0 e 119/119 lb 4:d80bb902:::obj418:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:d6df9426:::obj469:head local-lis/les=117/118 n=511 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.266+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.266+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d6df9426:::obj469:head local-lis/les=117/118 n=511 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:d80bb902:::obj418:head) v3 2026-02-11T23:31:28.266+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d6df9426:::obj469:head local-lis/les=117/118 n=511 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:d80bb902:::obj418:head) 2026-02-11T23:31:28.266+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.266+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:d80bb902:::obj418:head) v3 finish latency 0.000016 2026-02-11T23:31:28.266+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d80bb902:::obj418:head local-lis/les=117/118 n=512 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.266+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.266+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d80bb902:::obj418:head local-lis/les=117/118 n=512 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.266+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.266+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.266+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.266+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.266+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x674a000 con 0x507ac00 2026-02-11T23:31:28.266+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:d83b1cbf:::obj272:head)]) -- 0x6766180 con 0x507ac00 2026-02-11T23:31:28.266+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.267+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1065 ==== MOSDPGPush(4.0 119/117 [PushOp(4:d85a09a6:::obj441:head, version: 111'441, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d85a09a6:::obj441:head@111'441, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a3860 con 0x507ac00 2026-02-11T23:31:28.267+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:d85a09a6:::obj441:head, version: 111'441, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d85a09a6:::obj441:head@111'441, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.267+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d85a09a6:::obj441:head, version: 111'441, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d85a09a6:::obj441:head@111'441, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.267+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.267+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1066 ==== pg_backfill(progress 4.0 e 119/119 lb 4:d83b1cbf:::obj272:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99600 con 0x507ac00 2026-02-11T23:31:28.267+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d85a09a6:::obj441:head, version: 111'441, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d85a09a6:::obj441:head@111'441, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.267+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:d83b1cbf:::obj272:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.267+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d83b1cbf:::obj272:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.267+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.267+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d85a09a6:::obj441:head, version: 111'441, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d85a09a6:::obj441:head@111'441, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.267+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:d85a09a6:::obj441:head, version: 111'441, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d85a09a6:::obj441:head@111'441, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 119/117 [PushOp(4:d85a09a6:::obj441:head, version: 111'441, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d85a09a6:::obj441:head@111'441, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:d80bb902:::obj418:head local-lis/les=117/118 n=512 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.267+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.267+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.267+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d83b1cbf:::obj272:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.267+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d80bb902:::obj418:head local-lis/les=117/118 n=512 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:d85a09a6:::obj441:head, version: 111'441, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d85a09a6:::obj441:head@111'441, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.267+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d80bb902:::obj418:head local-lis/les=117/118 n=512 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:d85a09a6:::obj441:head@111'441, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.267+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d80bb902:::obj418:head local-lis/les=117/118 n=512 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:d85a09a6:::obj441:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.267+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d80bb902:::obj418:head local-lis/les=117/118 n=512 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:d85a09a6:::obj441:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.267+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d80bb902:::obj418:head local-lis/les=117/118 n=512 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:d85a09a6:::obj441:head 2026-02-11T23:31:28.267+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d80bb902:::obj418:head local-lis/les=117/118 n=512 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:d85a09a6:::obj441:head v 111'441 2026-02-11T23:31:28.267+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d80bb902:::obj418:head local-lis/les=117/118 n=512 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.267+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.267+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:d85a09a6:::obj441:head, version: 111'441, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d85a09a6:::obj441:head@111'441, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000069 2026-02-11T23:31:28.267+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d80bb902:::obj418:head local-lis/les=117/118 n=512 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.267+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.267+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.267+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d83b1cbf:::obj272:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.267+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:d83b1cbf:::obj272:head) v3 prio 15 cost 0 latency 0.000093 pg_backfill(progress 4.0 e 119/119 lb 4:d83b1cbf:::obj272:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:d80bb902:::obj418:head local-lis/les=117/118 n=512 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.267+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.267+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d80bb902:::obj418:head local-lis/les=117/118 n=512 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:d83b1cbf:::obj272:head) v3 2026-02-11T23:31:28.267+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d80bb902:::obj418:head local-lis/les=117/118 n=512 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:d83b1cbf:::obj272:head) 2026-02-11T23:31:28.267+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.267+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:d83b1cbf:::obj272:head) v3 finish latency 0.000025 2026-02-11T23:31:28.267+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.267+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d83b1cbf:::obj272:head local-lis/les=117/118 n=513 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.267+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.267+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.267+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.267+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.267+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x674ac00 con 0x507ac00 2026-02-11T23:31:28.267+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:d85a09a6:::obj441:head)]) -- 0x674ad80 con 0x507ac00 2026-02-11T23:31:28.267+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.267+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1067 ==== MOSDPGPush(4.0 119/117 [PushOp(4:d881fc7d:::obj242:head, version: 111'242, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d881fc7d:::obj242:head@111'242, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e26680 con 0x507ac00 2026-02-11T23:31:28.267+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:d881fc7d:::obj242:head, version: 111'242, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d881fc7d:::obj242:head@111'242, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 119 2026-02-11T23:31:28.267+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d881fc7d:::obj242:head, version: 111'242, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d881fc7d:::obj242:head@111'242, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.267+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.267+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1068 ==== pg_backfill(progress 4.0 e 119/119 lb 4:d85a09a6:::obj441:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176b00 con 0x507ac00 2026-02-11T23:31:28.267+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:d85a09a6:::obj441:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.267+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d881fc7d:::obj242:head, version: 111'242, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d881fc7d:::obj242:head@111'242, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.267+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d85a09a6:::obj441:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.267+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.267+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d881fc7d:::obj242:head, version: 111'242, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d881fc7d:::obj242:head@111'242, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.267+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.267+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d85a09a6:::obj441:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.267+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:d881fc7d:::obj242:head, version: 111'242, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d881fc7d:::obj242:head@111'242, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(4.0 119/117 [PushOp(4:d881fc7d:::obj242:head, version: 111'242, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d881fc7d:::obj242:head@111'242, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:d83b1cbf:::obj272:head local-lis/les=117/118 n=513 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.267+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.267+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d83b1cbf:::obj272:head local-lis/les=117/118 n=513 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:d881fc7d:::obj242:head, version: 111'242, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d881fc7d:::obj242:head@111'242, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.267+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d83b1cbf:::obj272:head local-lis/les=117/118 n=513 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:d881fc7d:::obj242:head@111'242, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.267+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d83b1cbf:::obj272:head local-lis/les=117/118 n=513 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:d881fc7d:::obj242:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.267+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d83b1cbf:::obj272:head local-lis/les=117/118 n=513 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:d881fc7d:::obj242:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.267+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d83b1cbf:::obj272:head local-lis/les=117/118 n=513 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:d881fc7d:::obj242:head 2026-02-11T23:31:28.267+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d83b1cbf:::obj272:head local-lis/les=117/118 n=513 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:d881fc7d:::obj242:head v 111'242 2026-02-11T23:31:28.267+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d83b1cbf:::obj272:head local-lis/les=117/118 n=513 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.267+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.267+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:d881fc7d:::obj242:head, version: 111'242, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d881fc7d:::obj242:head@111'242, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000069 2026-02-11T23:31:28.267+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d83b1cbf:::obj272:head local-lis/les=117/118 n=513 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.267+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.267+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.267+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d85a09a6:::obj441:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.267+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:d85a09a6:::obj441:head) v3 prio 15 cost 0 latency 0.000094 pg_backfill(progress 4.0 e 119/119 lb 4:d85a09a6:::obj441:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:d83b1cbf:::obj272:head local-lis/les=117/118 n=513 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.267+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.267+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d83b1cbf:::obj272:head local-lis/les=117/118 n=513 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:d85a09a6:::obj441:head) v3 2026-02-11T23:31:28.267+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d83b1cbf:::obj272:head local-lis/les=117/118 n=513 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:d85a09a6:::obj441:head) 2026-02-11T23:31:28.267+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.267+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:d85a09a6:::obj441:head) v3 finish latency 0.000021 2026-02-11T23:31:28.267+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.268+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d85a09a6:::obj441:head local-lis/les=117/118 n=514 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.268+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.268+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.268+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.268+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.268+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63ff200 con 0x507ac00 2026-02-11T23:31:28.268+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:d881fc7d:::obj242:head)]) -- 0x670b680 con 0x507ac00 2026-02-11T23:31:28.268+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.268+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1069 ==== MOSDPGPush(4.0 119/117 [PushOp(4:d9239af3:::obj409:head, version: 111'409, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9239af3:::obj409:head@111'409, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x38156c0 con 0x507ac00 2026-02-11T23:31:28.268+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:d9239af3:::obj409:head, version: 111'409, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9239af3:::obj409:head@111'409, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 119 2026-02-11T23:31:28.268+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d9239af3:::obj409:head, version: 111'409, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9239af3:::obj409:head@111'409, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.268+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.268+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1070 ==== pg_backfill(progress 4.0 e 119/119 lb 4:d881fc7d:::obj242:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99b80 con 0x507ac00 2026-02-11T23:31:28.268+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:d881fc7d:::obj242:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.268+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d9239af3:::obj409:head, version: 111'409, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9239af3:::obj409:head@111'409, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.268+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d881fc7d:::obj242:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.268+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.268+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d9239af3:::obj409:head, version: 111'409, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9239af3:::obj409:head@111'409, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.268+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.268+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d881fc7d:::obj242:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.268+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:d9239af3:::obj409:head, version: 111'409, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9239af3:::obj409:head@111'409, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 119/117 [PushOp(4:d9239af3:::obj409:head, version: 111'409, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9239af3:::obj409:head@111'409, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:d85a09a6:::obj441:head local-lis/les=117/118 n=514 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.268+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.268+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d85a09a6:::obj441:head local-lis/les=117/118 n=514 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:d9239af3:::obj409:head, version: 111'409, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9239af3:::obj409:head@111'409, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.268+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d85a09a6:::obj441:head local-lis/les=117/118 n=514 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:d9239af3:::obj409:head@111'409, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.268+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d85a09a6:::obj441:head local-lis/les=117/118 n=514 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:d9239af3:::obj409:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.268+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d85a09a6:::obj441:head local-lis/les=117/118 n=514 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:d9239af3:::obj409:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.268+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d85a09a6:::obj441:head local-lis/les=117/118 n=514 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:d9239af3:::obj409:head 2026-02-11T23:31:28.268+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d85a09a6:::obj441:head local-lis/les=117/118 n=514 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:d9239af3:::obj409:head v 111'409 2026-02-11T23:31:28.268+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d85a09a6:::obj441:head local-lis/les=117/118 n=514 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.268+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.268+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:d9239af3:::obj409:head, version: 111'409, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9239af3:::obj409:head@111'409, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000067 2026-02-11T23:31:28.268+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.268+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.268+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d881fc7d:::obj242:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.268+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:d881fc7d:::obj242:head) v3 prio 15 cost 0 latency 0.000087 pg_backfill(progress 4.0 e 119/119 lb 4:d881fc7d:::obj242:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:d85a09a6:::obj441:head local-lis/les=117/118 n=514 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.268+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.268+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d85a09a6:::obj441:head local-lis/les=117/118 n=514 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:d881fc7d:::obj242:head) v3 2026-02-11T23:31:28.268+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d85a09a6:::obj441:head local-lis/les=117/118 n=514 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:d881fc7d:::obj242:head) 2026-02-11T23:31:28.268+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.268+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:d881fc7d:::obj242:head) v3 finish latency 0.000016 2026-02-11T23:31:28.268+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d881fc7d:::obj242:head local-lis/les=117/118 n=515 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.268+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.268+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.268+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d881fc7d:::obj242:head local-lis/les=117/118 n=515 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.268+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.268+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.268+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.268+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x674a900 con 0x507ac00 2026-02-11T23:31:28.268+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:d9239af3:::obj409:head)]) -- 0x6766480 con 0x507ac00 2026-02-11T23:31:28.268+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.269+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1071 ==== MOSDPGPush(4.0 119/117 [PushOp(4:d99b7c45:::obj564:head, version: 111'564, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d99b7c45:::obj564:head@111'564, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a3ba0 con 0x507ac00 2026-02-11T23:31:28.269+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:d99b7c45:::obj564:head, version: 111'564, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d99b7c45:::obj564:head@111'564, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.269+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d99b7c45:::obj564:head, version: 111'564, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d99b7c45:::obj564:head@111'564, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.269+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.269+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1072 ==== pg_backfill(progress 4.0 e 119/119 lb 4:d9239af3:::obj409:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9b700 con 0x507ac00 2026-02-11T23:31:28.269+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d99b7c45:::obj564:head, version: 111'564, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d99b7c45:::obj564:head@111'564, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.269+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:d9239af3:::obj409:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.269+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d9239af3:::obj409:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.269+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.269+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d99b7c45:::obj564:head, version: 111'564, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d99b7c45:::obj564:head@111'564, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.269+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:d99b7c45:::obj564:head, version: 111'564, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d99b7c45:::obj564:head@111'564, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 119/117 [PushOp(4:d99b7c45:::obj564:head, version: 111'564, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d99b7c45:::obj564:head@111'564, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:d881fc7d:::obj242:head local-lis/les=117/118 n=515 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.269+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.269+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.269+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d9239af3:::obj409:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.269+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d881fc7d:::obj242:head local-lis/les=117/118 n=515 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:d99b7c45:::obj564:head, version: 111'564, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d99b7c45:::obj564:head@111'564, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.269+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d881fc7d:::obj242:head local-lis/les=117/118 n=515 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:d99b7c45:::obj564:head@111'564, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.269+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d881fc7d:::obj242:head local-lis/les=117/118 n=515 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:d99b7c45:::obj564:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.269+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d881fc7d:::obj242:head local-lis/les=117/118 n=515 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:d99b7c45:::obj564:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.269+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d881fc7d:::obj242:head local-lis/les=117/118 n=515 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:d99b7c45:::obj564:head 2026-02-11T23:31:28.269+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d881fc7d:::obj242:head local-lis/les=117/118 n=515 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:d99b7c45:::obj564:head v 111'564 2026-02-11T23:31:28.269+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d881fc7d:::obj242:head local-lis/les=117/118 n=515 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.269+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.269+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:d99b7c45:::obj564:head, version: 111'564, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d99b7c45:::obj564:head@111'564, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000066 2026-02-11T23:31:28.269+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.269+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.269+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d9239af3:::obj409:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.269+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:d9239af3:::obj409:head) v3 prio 15 cost 0 latency 0.000084 pg_backfill(progress 4.0 e 119/119 lb 4:d9239af3:::obj409:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:d881fc7d:::obj242:head local-lis/les=117/118 n=515 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.269+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.269+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d881fc7d:::obj242:head local-lis/les=117/118 n=515 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:d9239af3:::obj409:head) v3 2026-02-11T23:31:28.269+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d881fc7d:::obj242:head local-lis/les=117/118 n=515 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:d9239af3:::obj409:head) 2026-02-11T23:31:28.269+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.269+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:d9239af3:::obj409:head) v3 finish latency 0.000015 2026-02-11T23:31:28.269+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d9239af3:::obj409:head local-lis/les=117/118 n=516 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.269+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.269+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.269+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d9239af3:::obj409:head local-lis/les=117/118 n=516 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.269+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.269+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.269+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.269+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66fbb00 con 0x507ac00 2026-02-11T23:31:28.269+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:d99b7c45:::obj564:head)]) -- 0x6766780 con 0x507ac00 2026-02-11T23:31:28.269+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.269+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1073 ==== MOSDPGPush(4.0 119/117 [PushOp(4:d9c98c7a:::obj262:head, version: 111'262, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9c98c7a:::obj262:head@111'262, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3814ea0 con 0x507ac00 2026-02-11T23:31:28.269+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:d9c98c7a:::obj262:head, version: 111'262, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9c98c7a:::obj262:head@111'262, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.269+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d9c98c7a:::obj262:head, version: 111'262, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9c98c7a:::obj262:head@111'262, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.269+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.269+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1074 ==== pg_backfill(progress 4.0 e 119/119 lb 4:d99b7c45:::obj564:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98580 con 0x507ac00 2026-02-11T23:31:28.269+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:d99b7c45:::obj564:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 119 2026-02-11T23:31:28.269+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d9c98c7a:::obj262:head, version: 111'262, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9c98c7a:::obj262:head@111'262, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.269+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d99b7c45:::obj564:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.269+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.269+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d9c98c7a:::obj262:head, version: 111'262, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9c98c7a:::obj262:head@111'262, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.269+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:d9c98c7a:::obj262:head, version: 111'262, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9c98c7a:::obj262:head@111'262, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 119/117 [PushOp(4:d9c98c7a:::obj262:head, version: 111'262, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9c98c7a:::obj262:head@111'262, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:d9239af3:::obj409:head local-lis/les=117/118 n=516 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.269+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.269+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.269+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d99b7c45:::obj564:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.269+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d9239af3:::obj409:head local-lis/les=117/118 n=516 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:d9c98c7a:::obj262:head, version: 111'262, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9c98c7a:::obj262:head@111'262, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.269+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d9239af3:::obj409:head local-lis/les=117/118 n=516 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:d9c98c7a:::obj262:head@111'262, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.269+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d9239af3:::obj409:head local-lis/les=117/118 n=516 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:d9c98c7a:::obj262:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.269+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d9239af3:::obj409:head local-lis/les=117/118 n=516 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:d9c98c7a:::obj262:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.269+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d9239af3:::obj409:head local-lis/les=117/118 n=516 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:d9c98c7a:::obj262:head 2026-02-11T23:31:28.269+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d9239af3:::obj409:head local-lis/les=117/118 n=516 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:d9c98c7a:::obj262:head v 111'262 2026-02-11T23:31:28.269+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d9239af3:::obj409:head local-lis/les=117/118 n=516 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.269+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.269+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:d9c98c7a:::obj262:head, version: 111'262, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9c98c7a:::obj262:head@111'262, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000064 2026-02-11T23:31:28.269+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.269+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.269+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d99b7c45:::obj564:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.269+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:d99b7c45:::obj564:head) v3 prio 15 cost 0 latency 0.000084 pg_backfill(progress 4.0 e 119/119 lb 4:d99b7c45:::obj564:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:d9239af3:::obj409:head local-lis/les=117/118 n=516 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.269+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.269+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d9239af3:::obj409:head local-lis/les=117/118 n=516 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:d99b7c45:::obj564:head) v3 2026-02-11T23:31:28.269+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d9239af3:::obj409:head local-lis/les=117/118 n=516 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:d99b7c45:::obj564:head) 2026-02-11T23:31:28.269+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.269+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:d99b7c45:::obj564:head) v3 finish latency 0.000015 2026-02-11T23:31:28.269+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d99b7c45:::obj564:head local-lis/les=117/118 n=517 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.269+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.269+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.269+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d99b7c45:::obj564:head local-lis/les=117/118 n=517 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.269+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.269+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.269+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.269+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x674a780 con 0x507ac00 2026-02-11T23:31:28.269+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:d9c98c7a:::obj262:head)]) -- 0x6766a80 con 0x507ac00 2026-02-11T23:31:28.269+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.270+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1075 ==== MOSDPGPush(4.0 119/117 [PushOp(4:d9da6b63:::obj302:head, version: 111'302, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9da6b63:::obj302:head@111'302, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c4e0 con 0x507ac00 2026-02-11T23:31:28.270+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:d9da6b63:::obj302:head, version: 111'302, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9da6b63:::obj302:head@111'302, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 119 2026-02-11T23:31:28.270+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d9da6b63:::obj302:head, version: 111'302, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9da6b63:::obj302:head@111'302, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.270+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.270+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1076 ==== pg_backfill(progress 4.0 e 119/119 lb 4:d9c98c7a:::obj262:head) ==== 1062+0+0 (unknown 0 0 0) 0x4a2c000 con 0x507ac00 2026-02-11T23:31:28.270+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d9da6b63:::obj302:head, version: 111'302, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9da6b63:::obj302:head@111'302, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.270+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:d9c98c7a:::obj262:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.270+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d9c98c7a:::obj262:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.270+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.270+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:d9da6b63:::obj302:head, version: 111'302, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9da6b63:::obj302:head@111'302, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.270+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:d9da6b63:::obj302:head, version: 111'302, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9da6b63:::obj302:head@111'302, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(4.0 119/117 [PushOp(4:d9da6b63:::obj302:head, version: 111'302, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9da6b63:::obj302:head@111'302, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:d99b7c45:::obj564:head local-lis/les=117/118 n=517 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.270+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.270+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.270+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d9c98c7a:::obj262:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.270+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d99b7c45:::obj564:head local-lis/les=117/118 n=517 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:d9da6b63:::obj302:head, version: 111'302, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9da6b63:::obj302:head@111'302, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.270+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d99b7c45:::obj564:head local-lis/les=117/118 n=517 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:d9da6b63:::obj302:head@111'302, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.270+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d99b7c45:::obj564:head local-lis/les=117/118 n=517 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:d9da6b63:::obj302:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.270+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d99b7c45:::obj564:head local-lis/les=117/118 n=517 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:d9da6b63:::obj302:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.270+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d99b7c45:::obj564:head local-lis/les=117/118 n=517 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:d9da6b63:::obj302:head 2026-02-11T23:31:28.270+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d99b7c45:::obj564:head local-lis/les=117/118 n=517 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:d9da6b63:::obj302:head v 111'302 2026-02-11T23:31:28.270+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d99b7c45:::obj564:head local-lis/les=117/118 n=517 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.270+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.270+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:d9da6b63:::obj302:head, version: 111'302, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9da6b63:::obj302:head@111'302, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000069 2026-02-11T23:31:28.270+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.270+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.270+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d9c98c7a:::obj262:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.270+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:d9c98c7a:::obj262:head) v3 prio 15 cost 0 latency 0.000088 pg_backfill(progress 4.0 e 119/119 lb 4:d9c98c7a:::obj262:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:d99b7c45:::obj564:head local-lis/les=117/118 n=517 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.270+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.270+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d99b7c45:::obj564:head local-lis/les=117/118 n=517 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:d9c98c7a:::obj262:head) v3 2026-02-11T23:31:28.270+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d99b7c45:::obj564:head local-lis/les=117/118 n=517 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:d9c98c7a:::obj262:head) 2026-02-11T23:31:28.270+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.270+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:d9c98c7a:::obj262:head) v3 finish latency 0.000018 2026-02-11T23:31:28.270+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d9c98c7a:::obj262:head local-lis/les=117/118 n=518 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.270+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.270+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.270+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d9c98c7a:::obj262:head local-lis/les=117/118 n=518 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.270+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.270+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.270+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.270+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x4f7aa80 con 0x507ac00 2026-02-11T23:31:28.270+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:d9da6b63:::obj302:head)]) -- 0x6766d80 con 0x507ac00 2026-02-11T23:31:28.270+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.270+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1077 ==== MOSDPGPush(4.0 119/117 [PushOp(4:da0e2ac9:::obj69:head, version: 111'69, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da0e2ac9:::obj69:head@111'69, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x56a3860 con 0x507ac00 2026-02-11T23:31:28.270+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:da0e2ac9:::obj69:head, version: 111'69, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da0e2ac9:::obj69:head@111'69, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.270+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:da0e2ac9:::obj69:head, version: 111'69, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da0e2ac9:::obj69:head@111'69, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.270+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.270+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1078 ==== pg_backfill(progress 4.0 e 119/119 lb 4:d9da6b63:::obj302:head) ==== 1062+0+0 (unknown 0 0 0) 0x4a2cb00 con 0x507ac00 2026-02-11T23:31:28.270+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:da0e2ac9:::obj69:head, version: 111'69, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da0e2ac9:::obj69:head@111'69, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.270+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:d9da6b63:::obj302:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.270+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d9da6b63:::obj302:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.270+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.270+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:da0e2ac9:::obj69:head, version: 111'69, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da0e2ac9:::obj69:head@111'69, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.270+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:da0e2ac9:::obj69:head, version: 111'69, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da0e2ac9:::obj69:head@111'69, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 119/117 [PushOp(4:da0e2ac9:::obj69:head, version: 111'69, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da0e2ac9:::obj69:head@111'69, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:d9c98c7a:::obj262:head local-lis/les=117/118 n=518 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.270+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.270+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.270+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d9da6b63:::obj302:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.270+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d9c98c7a:::obj262:head local-lis/les=117/118 n=518 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:da0e2ac9:::obj69:head, version: 111'69, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da0e2ac9:::obj69:head@111'69, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.270+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d9c98c7a:::obj262:head local-lis/les=117/118 n=518 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:da0e2ac9:::obj69:head@111'69, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.271+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d9c98c7a:::obj262:head local-lis/les=117/118 n=518 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:da0e2ac9:::obj69:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.271+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d9c98c7a:::obj262:head local-lis/les=117/118 n=518 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:da0e2ac9:::obj69:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.271+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d9c98c7a:::obj262:head local-lis/les=117/118 n=518 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:da0e2ac9:::obj69:head 2026-02-11T23:31:28.271+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d9c98c7a:::obj262:head local-lis/les=117/118 n=518 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:da0e2ac9:::obj69:head v 111'69 2026-02-11T23:31:28.271+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d9c98c7a:::obj262:head local-lis/les=117/118 n=518 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.271+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.271+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:da0e2ac9:::obj69:head, version: 111'69, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da0e2ac9:::obj69:head@111'69, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000065 2026-02-11T23:31:28.271+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.271+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.271+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:d9da6b63:::obj302:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.271+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:d9da6b63:::obj302:head) v3 prio 15 cost 0 latency 0.000082 pg_backfill(progress 4.0 e 119/119 lb 4:d9da6b63:::obj302:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:d9c98c7a:::obj262:head local-lis/les=117/118 n=518 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.271+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.271+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d9c98c7a:::obj262:head local-lis/les=117/118 n=518 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:d9da6b63:::obj302:head) v3 2026-02-11T23:31:28.271+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d9c98c7a:::obj262:head local-lis/les=117/118 n=518 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:d9da6b63:::obj302:head) 2026-02-11T23:31:28.271+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.271+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:d9da6b63:::obj302:head) v3 finish latency 0.000014 2026-02-11T23:31:28.271+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d9da6b63:::obj302:head local-lis/les=117/118 n=519 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.271+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.271+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.271+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d9da6b63:::obj302:head local-lis/les=117/118 n=519 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.271+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.271+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.271+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.271+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x674a600 con 0x507ac00 2026-02-11T23:31:28.271+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:da0e2ac9:::obj69:head)]) -- 0x6767080 con 0x507ac00 2026-02-11T23:31:28.271+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.271+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1079 ==== MOSDPGPush(4.0 119/117 [PushOp(4:da920364:::obj185:head, version: 111'185, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da920364:::obj185:head@111'185, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56cab60 con 0x507ac00 2026-02-11T23:31:28.271+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:da920364:::obj185:head, version: 111'185, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da920364:::obj185:head@111'185, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.271+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:da920364:::obj185:head, version: 111'185, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da920364:::obj185:head@111'185, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.271+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.271+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:da920364:::obj185:head, version: 111'185, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da920364:::obj185:head@111'185, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.271+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1080 ==== pg_backfill(progress 4.0 e 119/119 lb 4:da0e2ac9:::obj69:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d9ac00 con 0x507ac00 2026-02-11T23:31:28.271+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.271+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:da0e2ac9:::obj69:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.271+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:da920364:::obj185:head, version: 111'185, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da920364:::obj185:head@111'185, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.271+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:da0e2ac9:::obj69:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.271+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:da920364:::obj185:head, version: 111'185, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da920364:::obj185:head@111'185, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(4.0 119/117 [PushOp(4:da920364:::obj185:head, version: 111'185, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da920364:::obj185:head@111'185, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:d9da6b63:::obj302:head local-lis/les=117/118 n=519 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.271+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.271+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.271+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d9da6b63:::obj302:head local-lis/les=117/118 n=519 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:da920364:::obj185:head, version: 111'185, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da920364:::obj185:head@111'185, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.271+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:da0e2ac9:::obj69:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.271+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d9da6b63:::obj302:head local-lis/les=117/118 n=519 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:da920364:::obj185:head@111'185, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.271+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d9da6b63:::obj302:head local-lis/les=117/118 n=519 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:da920364:::obj185:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.271+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d9da6b63:::obj302:head local-lis/les=117/118 n=519 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:da920364:::obj185:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.271+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d9da6b63:::obj302:head local-lis/les=117/118 n=519 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:da920364:::obj185:head 2026-02-11T23:31:28.271+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d9da6b63:::obj302:head local-lis/les=117/118 n=519 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:da920364:::obj185:head v 111'185 2026-02-11T23:31:28.271+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d9da6b63:::obj302:head local-lis/les=117/118 n=519 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.271+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.271+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:da920364:::obj185:head, version: 111'185, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da920364:::obj185:head@111'185, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:28.271+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.271+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.271+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:da0e2ac9:::obj69:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.271+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:da0e2ac9:::obj69:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 4.0 e 119/119 lb 4:da0e2ac9:::obj69:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:d9da6b63:::obj302:head local-lis/les=117/118 n=519 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.271+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.271+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d9da6b63:::obj302:head local-lis/les=117/118 n=519 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:da0e2ac9:::obj69:head) v3 2026-02-11T23:31:28.271+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:d9da6b63:::obj302:head local-lis/les=117/118 n=519 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:da0e2ac9:::obj69:head) 2026-02-11T23:31:28.271+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.271+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:da0e2ac9:::obj69:head) v3 finish latency 0.000019 2026-02-11T23:31:28.271+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:da0e2ac9:::obj69:head local-lis/les=117/118 n=520 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.271+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.271+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.271+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:da0e2ac9:::obj69:head local-lis/les=117/118 n=520 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.271+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.271+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.271+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.271+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66fbc80 con 0x507ac00 2026-02-11T23:31:28.271+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:da920364:::obj185:head)]) -- 0x6767200 con 0x507ac00 2026-02-11T23:31:28.271+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.272+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1081 ==== MOSDPGPush(4.0 119/117 [PushOp(4:da932e1e:::obj333:head, version: 111'333, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da932e1e:::obj333:head@111'333, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2680 con 0x507ac00 2026-02-11T23:31:28.272+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:da932e1e:::obj333:head, version: 111'333, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da932e1e:::obj333:head@111'333, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.272+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:da932e1e:::obj333:head, version: 111'333, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da932e1e:::obj333:head@111'333, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.272+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.272+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:da932e1e:::obj333:head, version: 111'333, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da932e1e:::obj333:head@111'333, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.272+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1082 ==== pg_backfill(progress 4.0 e 119/119 lb 4:da920364:::obj185:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176b00 con 0x507ac00 2026-02-11T23:31:28.272+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.272+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:da920364:::obj185:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.272+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:da932e1e:::obj333:head, version: 111'333, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da932e1e:::obj333:head@111'333, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.272+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:da920364:::obj185:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.272+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.272+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:da932e1e:::obj333:head, version: 111'333, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da932e1e:::obj333:head@111'333, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 119/117 [PushOp(4:da932e1e:::obj333:head, version: 111'333, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da932e1e:::obj333:head@111'333, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:da0e2ac9:::obj69:head local-lis/les=117/118 n=520 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.272+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:da920364:::obj185:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.272+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.272+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:da0e2ac9:::obj69:head local-lis/les=117/118 n=520 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:da932e1e:::obj333:head, version: 111'333, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da932e1e:::obj333:head@111'333, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.272+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:da0e2ac9:::obj69:head local-lis/les=117/118 n=520 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:da932e1e:::obj333:head@111'333, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.272+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:da0e2ac9:::obj69:head local-lis/les=117/118 n=520 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:da932e1e:::obj333:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.272+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:da0e2ac9:::obj69:head local-lis/les=117/118 n=520 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:da932e1e:::obj333:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.272+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:da0e2ac9:::obj69:head local-lis/les=117/118 n=520 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:da932e1e:::obj333:head 2026-02-11T23:31:28.272+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:da0e2ac9:::obj69:head local-lis/les=117/118 n=520 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:da932e1e:::obj333:head v 111'333 2026-02-11T23:31:28.272+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:da0e2ac9:::obj69:head local-lis/les=117/118 n=520 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.272+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.272+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:da932e1e:::obj333:head, version: 111'333, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da932e1e:::obj333:head@111'333, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:28.272+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.272+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.272+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:da920364:::obj185:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.272+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:da920364:::obj185:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 119/119 lb 4:da920364:::obj185:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:da0e2ac9:::obj69:head local-lis/les=117/118 n=520 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.272+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.272+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:da0e2ac9:::obj69:head local-lis/les=117/118 n=520 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:da920364:::obj185:head) v3 2026-02-11T23:31:28.272+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:da0e2ac9:::obj69:head local-lis/les=117/118 n=520 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:da920364:::obj185:head) 2026-02-11T23:31:28.272+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.272+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:da920364:::obj185:head) v3 finish latency 0.000013 2026-02-11T23:31:28.272+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:da920364:::obj185:head local-lis/les=117/118 n=521 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.272+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.272+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.272+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:da920364:::obj185:head local-lis/les=117/118 n=521 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.272+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.272+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.272+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.272+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x674a480 con 0x507ac00 2026-02-11T23:31:28.272+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:da932e1e:::obj333:head)]) -- 0x6767500 con 0x507ac00 2026-02-11T23:31:28.272+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.272+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1083 ==== MOSDPGPush(4.0 119/117 [PushOp(4:dbca42a3:::obj382:head, version: 111'382, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dbca42a3:::obj382:head@111'382, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db0000 con 0x507ac00 2026-02-11T23:31:28.272+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:dbca42a3:::obj382:head, version: 111'382, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dbca42a3:::obj382:head@111'382, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.272+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:dbca42a3:::obj382:head, version: 111'382, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dbca42a3:::obj382:head@111'382, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.272+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.272+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1084 ==== pg_backfill(progress 4.0 e 119/119 lb 4:da932e1e:::obj333:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99600 con 0x507ac00 2026-02-11T23:31:28.272+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:dbca42a3:::obj382:head, version: 111'382, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dbca42a3:::obj382:head@111'382, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.272+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:da932e1e:::obj333:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.272+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:da932e1e:::obj333:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.272+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.272+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:dbca42a3:::obj382:head, version: 111'382, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dbca42a3:::obj382:head@111'382, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.272+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.272+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:dbca42a3:::obj382:head, version: 111'382, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dbca42a3:::obj382:head@111'382, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 119/117 [PushOp(4:dbca42a3:::obj382:head, version: 111'382, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dbca42a3:::obj382:head@111'382, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:da920364:::obj185:head local-lis/les=117/118 n=521 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.272+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.272+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:da920364:::obj185:head local-lis/les=117/118 n=521 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:dbca42a3:::obj382:head, version: 111'382, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dbca42a3:::obj382:head@111'382, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.272+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.272+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:da932e1e:::obj333:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.272+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:da920364:::obj185:head local-lis/les=117/118 n=521 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:dbca42a3:::obj382:head@111'382, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.272+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:da920364:::obj185:head local-lis/les=117/118 n=521 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:dbca42a3:::obj382:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.272+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:da920364:::obj185:head local-lis/les=117/118 n=521 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:dbca42a3:::obj382:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.272+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:da920364:::obj185:head local-lis/les=117/118 n=521 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:dbca42a3:::obj382:head 2026-02-11T23:31:28.272+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:da920364:::obj185:head local-lis/les=117/118 n=521 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:dbca42a3:::obj382:head v 111'382 2026-02-11T23:31:28.272+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:da920364:::obj185:head local-lis/les=117/118 n=521 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.272+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.272+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:dbca42a3:::obj382:head, version: 111'382, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dbca42a3:::obj382:head@111'382, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000077 2026-02-11T23:31:28.272+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:da920364:::obj185:head local-lis/les=117/118 n=521 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.272+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.272+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.272+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:da932e1e:::obj333:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.272+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:da932e1e:::obj333:head) v3 prio 15 cost 0 latency 0.000105 pg_backfill(progress 4.0 e 119/119 lb 4:da932e1e:::obj333:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:da920364:::obj185:head local-lis/les=117/118 n=521 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.272+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.272+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:da920364:::obj185:head local-lis/les=117/118 n=521 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:da932e1e:::obj333:head) v3 2026-02-11T23:31:28.272+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:da920364:::obj185:head local-lis/les=117/118 n=521 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:da932e1e:::obj333:head) 2026-02-11T23:31:28.272+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.272+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:da932e1e:::obj333:head) v3 finish latency 0.000019 2026-02-11T23:31:28.272+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.273+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:da932e1e:::obj333:head local-lis/les=117/118 n=522 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.273+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.273+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.273+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.273+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.273+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x6766480 con 0x507ac00 2026-02-11T23:31:28.273+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:dbca42a3:::obj382:head)]) -- 0x5f2dc80 con 0x507ac00 2026-02-11T23:31:28.273+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.273+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.273+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.273+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1085 ==== MOSDPGPush(4.0 119/117 [PushOp(4:dc49c339:::obj378:head, version: 111'378, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc49c339:::obj378:head@111'378, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a3a00 con 0x507ac00 2026-02-11T23:31:28.273+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:dc49c339:::obj378:head, version: 111'378, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc49c339:::obj378:head@111'378, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.273+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:dc49c339:::obj378:head, version: 111'378, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc49c339:::obj378:head@111'378, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.273+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.273+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1086 ==== pg_backfill(progress 4.0 e 119/119 lb 4:dbca42a3:::obj382:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99080 con 0x507ac00 2026-02-11T23:31:28.273+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:dc49c339:::obj378:head, version: 111'378, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc49c339:::obj378:head@111'378, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.273+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:dbca42a3:::obj382:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.273+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:dbca42a3:::obj382:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.273+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.273+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:dc49c339:::obj378:head, version: 111'378, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc49c339:::obj378:head@111'378, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.273+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.273+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:dc49c339:::obj378:head, version: 111'378, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc49c339:::obj378:head@111'378, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 119/117 [PushOp(4:dc49c339:::obj378:head, version: 111'378, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc49c339:::obj378:head@111'378, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:da932e1e:::obj333:head local-lis/les=117/118 n=522 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.273+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.273+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.273+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:dbca42a3:::obj382:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.273+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:da932e1e:::obj333:head local-lis/les=117/118 n=522 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:dc49c339:::obj378:head, version: 111'378, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc49c339:::obj378:head@111'378, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.273+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:da932e1e:::obj333:head local-lis/les=117/118 n=522 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:dc49c339:::obj378:head@111'378, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.273+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:da932e1e:::obj333:head local-lis/les=117/118 n=522 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:dc49c339:::obj378:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.273+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:da932e1e:::obj333:head local-lis/les=117/118 n=522 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:dc49c339:::obj378:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.273+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:da932e1e:::obj333:head local-lis/les=117/118 n=522 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:dc49c339:::obj378:head 2026-02-11T23:31:28.273+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:da932e1e:::obj333:head local-lis/les=117/118 n=522 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:dc49c339:::obj378:head v 111'378 2026-02-11T23:31:28.273+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:da932e1e:::obj333:head local-lis/les=117/118 n=522 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.273+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.273+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:dc49c339:::obj378:head, version: 111'378, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc49c339:::obj378:head@111'378, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:28.273+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:da932e1e:::obj333:head local-lis/les=117/118 n=522 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.273+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.273+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.273+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:dbca42a3:::obj382:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.273+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:dbca42a3:::obj382:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 4.0 e 119/119 lb 4:dbca42a3:::obj382:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:da932e1e:::obj333:head local-lis/les=117/118 n=522 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.273+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.273+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:da932e1e:::obj333:head local-lis/les=117/118 n=522 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:dbca42a3:::obj382:head) v3 2026-02-11T23:31:28.273+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:da932e1e:::obj333:head local-lis/les=117/118 n=522 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:dbca42a3:::obj382:head) 2026-02-11T23:31:28.273+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.273+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:dbca42a3:::obj382:head) v3 finish latency 0.000015 2026-02-11T23:31:28.273+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.273+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.273+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dbca42a3:::obj382:head local-lis/les=117/118 n=523 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.273+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.273+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.273+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.273+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63ff200 con 0x507ac00 2026-02-11T23:31:28.273+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:dc49c339:::obj378:head)]) -- 0x674a900 con 0x507ac00 2026-02-11T23:31:28.273+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.273+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1087 ==== MOSDPGPush(4.0 119/117 [PushOp(4:dc6ed3c7:::obj113:head, version: 111'113, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc6ed3c7:::obj113:head@111'113, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2820 con 0x507ac00 2026-02-11T23:31:28.273+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:dc6ed3c7:::obj113:head, version: 111'113, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc6ed3c7:::obj113:head@111'113, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.273+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:dc6ed3c7:::obj113:head, version: 111'113, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc6ed3c7:::obj113:head@111'113, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.274+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.274+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:dc6ed3c7:::obj113:head, version: 111'113, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc6ed3c7:::obj113:head@111'113, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.274+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.274+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:dc6ed3c7:::obj113:head, version: 111'113, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc6ed3c7:::obj113:head@111'113, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.274+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.274+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:dc6ed3c7:::obj113:head, version: 111'113, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc6ed3c7:::obj113:head@111'113, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000041 MOSDPGPush(4.0 119/117 [PushOp(4:dc6ed3c7:::obj113:head, version: 111'113, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc6ed3c7:::obj113:head@111'113, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:dbca42a3:::obj382:head local-lis/les=117/118 n=523 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.274+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.274+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dbca42a3:::obj382:head local-lis/les=117/118 n=523 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:dc6ed3c7:::obj113:head, version: 111'113, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc6ed3c7:::obj113:head@111'113, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.274+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dbca42a3:::obj382:head local-lis/les=117/118 n=523 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:dc6ed3c7:::obj113:head@111'113, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.274+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1088 ==== pg_backfill(progress 4.0 e 119/119 lb 4:dc49c339:::obj378:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2ec00 con 0x507ac00 2026-02-11T23:31:28.274+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dbca42a3:::obj382:head local-lis/les=117/118 n=523 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:dc6ed3c7:::obj113:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.274+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:dc49c339:::obj378:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 119 2026-02-11T23:31:28.274+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dbca42a3:::obj382:head local-lis/les=117/118 n=523 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:dc6ed3c7:::obj113:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.274+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:dc49c339:::obj378:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.274+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dbca42a3:::obj382:head local-lis/les=117/118 n=523 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:dc6ed3c7:::obj113:head 2026-02-11T23:31:28.274+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dbca42a3:::obj382:head local-lis/les=117/118 n=523 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:dc6ed3c7:::obj113:head v 111'113 2026-02-11T23:31:28.274+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.274+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dbca42a3:::obj382:head local-lis/les=117/118 n=523 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.274+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:dc49c339:::obj378:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.274+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.274+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:dc6ed3c7:::obj113:head, version: 111'113, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc6ed3c7:::obj113:head@111'113, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000070 2026-02-11T23:31:28.274+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.274+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.274+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:dc49c339:::obj378:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.274+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:dc49c339:::obj378:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 4.0 e 119/119 lb 4:dc49c339:::obj378:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:dbca42a3:::obj382:head local-lis/les=117/118 n=523 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.274+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.274+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dbca42a3:::obj382:head local-lis/les=117/118 n=523 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:dc49c339:::obj378:head) v3 2026-02-11T23:31:28.274+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dbca42a3:::obj382:head local-lis/les=117/118 n=523 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:dc49c339:::obj378:head) 2026-02-11T23:31:28.274+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.274+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:dc49c339:::obj378:head) v3 finish latency 0.000013 2026-02-11T23:31:28.274+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc49c339:::obj378:head local-lis/les=117/118 n=524 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.274+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.274+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc49c339:::obj378:head local-lis/les=117/118 n=524 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.274+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.274+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.274+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.274+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.274+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x674ad80 con 0x507ac00 2026-02-11T23:31:28.274+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:dc6ed3c7:::obj113:head)]) -- 0x6767800 con 0x507ac00 2026-02-11T23:31:28.274+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.274+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1089 ==== MOSDPGPush(4.0 119/117 [PushOp(4:dc6fad89:::obj337:head, version: 111'337, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc6fad89:::obj337:head@111'337, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f1860 con 0x507ac00 2026-02-11T23:31:28.274+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:dc6fad89:::obj337:head, version: 111'337, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc6fad89:::obj337:head@111'337, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.274+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:dc6fad89:::obj337:head, version: 111'337, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc6fad89:::obj337:head@111'337, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.274+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.274+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1090 ==== pg_backfill(progress 4.0 e 119/119 lb 4:dc6ed3c7:::obj113:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9b180 con 0x507ac00 2026-02-11T23:31:28.274+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:dc6fad89:::obj337:head, version: 111'337, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc6fad89:::obj337:head@111'337, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.274+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:dc6ed3c7:::obj113:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.274+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:dc6ed3c7:::obj113:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.274+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.274+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:dc6fad89:::obj337:head, version: 111'337, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc6fad89:::obj337:head@111'337, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.274+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.274+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:dc6fad89:::obj337:head, version: 111'337, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc6fad89:::obj337:head@111'337, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 119/117 [PushOp(4:dc6fad89:::obj337:head, version: 111'337, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc6fad89:::obj337:head@111'337, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:dc49c339:::obj378:head local-lis/les=117/118 n=524 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.274+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.274+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.274+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:dc6ed3c7:::obj113:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.274+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc49c339:::obj378:head local-lis/les=117/118 n=524 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:dc6fad89:::obj337:head, version: 111'337, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc6fad89:::obj337:head@111'337, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.274+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc49c339:::obj378:head local-lis/les=117/118 n=524 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:dc6fad89:::obj337:head@111'337, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.274+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc49c339:::obj378:head local-lis/les=117/118 n=524 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:dc6fad89:::obj337:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.274+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc49c339:::obj378:head local-lis/les=117/118 n=524 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:dc6fad89:::obj337:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.274+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc49c339:::obj378:head local-lis/les=117/118 n=524 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:dc6fad89:::obj337:head 2026-02-11T23:31:28.274+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc49c339:::obj378:head local-lis/les=117/118 n=524 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:dc6fad89:::obj337:head v 111'337 2026-02-11T23:31:28.274+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc49c339:::obj378:head local-lis/les=117/118 n=524 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.274+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.274+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:dc6fad89:::obj337:head, version: 111'337, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc6fad89:::obj337:head@111'337, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:28.274+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.274+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.274+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:dc6ed3c7:::obj113:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.274+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:dc6ed3c7:::obj113:head) v3 prio 15 cost 0 latency 0.000081 pg_backfill(progress 4.0 e 119/119 lb 4:dc6ed3c7:::obj113:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:dc49c339:::obj378:head local-lis/les=117/118 n=524 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.274+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.274+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc49c339:::obj378:head local-lis/les=117/118 n=524 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:dc6ed3c7:::obj113:head) v3 2026-02-11T23:31:28.274+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc49c339:::obj378:head local-lis/les=117/118 n=524 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:dc6ed3c7:::obj113:head) 2026-02-11T23:31:28.274+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.274+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:dc6ed3c7:::obj113:head) v3 finish latency 0.000013 2026-02-11T23:31:28.274+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc6ed3c7:::obj113:head local-lis/les=117/118 n=525 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.274+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.274+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.274+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc6ed3c7:::obj113:head local-lis/les=117/118 n=525 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.274+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.274+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.274+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.274+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x674ac00 con 0x507ac00 2026-02-11T23:31:28.274+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:dc6fad89:::obj337:head)]) -- 0x6767b00 con 0x507ac00 2026-02-11T23:31:28.274+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.275+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1091 ==== MOSDPGPush(4.0 119/117 [PushOp(4:dc7d10b2:::obj533:head, version: 111'533, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc7d10b2:::obj533:head@111'533, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559d520 con 0x507ac00 2026-02-11T23:31:28.275+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:dc7d10b2:::obj533:head, version: 111'533, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc7d10b2:::obj533:head@111'533, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.275+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:dc7d10b2:::obj533:head, version: 111'533, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc7d10b2:::obj533:head@111'533, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.275+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.275+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1092 ==== pg_backfill(progress 4.0 e 119/119 lb 4:dc6fad89:::obj337:head) ==== 1062+0+0 (unknown 0 0 0) 0x6179180 con 0x507ac00 2026-02-11T23:31:28.275+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:dc7d10b2:::obj533:head, version: 111'533, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc7d10b2:::obj533:head@111'533, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.275+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:dc6fad89:::obj337:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.275+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:dc6fad89:::obj337:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.275+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.275+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:dc7d10b2:::obj533:head, version: 111'533, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc7d10b2:::obj533:head@111'533, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.275+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.275+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:dc7d10b2:::obj533:head, version: 111'533, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc7d10b2:::obj533:head@111'533, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 119/117 [PushOp(4:dc7d10b2:::obj533:head, version: 111'533, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc7d10b2:::obj533:head@111'533, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:dc6ed3c7:::obj113:head local-lis/les=117/118 n=525 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.275+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.275+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.275+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:dc6fad89:::obj337:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.275+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc6ed3c7:::obj113:head local-lis/les=117/118 n=525 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:dc7d10b2:::obj533:head, version: 111'533, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc7d10b2:::obj533:head@111'533, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.275+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc6ed3c7:::obj113:head local-lis/les=117/118 n=525 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:dc7d10b2:::obj533:head@111'533, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.275+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc6ed3c7:::obj113:head local-lis/les=117/118 n=525 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:dc7d10b2:::obj533:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.275+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc6ed3c7:::obj113:head local-lis/les=117/118 n=525 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:dc7d10b2:::obj533:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.275+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc6ed3c7:::obj113:head local-lis/les=117/118 n=525 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:dc7d10b2:::obj533:head 2026-02-11T23:31:28.275+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc6ed3c7:::obj113:head local-lis/les=117/118 n=525 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:dc7d10b2:::obj533:head v 111'533 2026-02-11T23:31:28.275+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc6ed3c7:::obj113:head local-lis/les=117/118 n=525 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.275+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.275+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:dc7d10b2:::obj533:head, version: 111'533, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc7d10b2:::obj533:head@111'533, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000064 2026-02-11T23:31:28.275+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc6ed3c7:::obj113:head local-lis/les=117/118 n=525 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.275+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.275+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.275+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:dc6fad89:::obj337:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.275+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:dc6fad89:::obj337:head) v3 prio 15 cost 0 latency 0.000088 pg_backfill(progress 4.0 e 119/119 lb 4:dc6fad89:::obj337:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:dc6ed3c7:::obj113:head local-lis/les=117/118 n=525 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.275+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.275+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc6ed3c7:::obj113:head local-lis/les=117/118 n=525 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:dc6fad89:::obj337:head) v3 2026-02-11T23:31:28.275+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc6ed3c7:::obj113:head local-lis/les=117/118 n=525 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:dc6fad89:::obj337:head) 2026-02-11T23:31:28.275+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.275+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:dc6fad89:::obj337:head) v3 finish latency 0.000013 2026-02-11T23:31:28.275+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.275+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc6fad89:::obj337:head local-lis/les=117/118 n=526 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.275+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.275+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.275+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.275+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.275+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x674b200 con 0x507ac00 2026-02-11T23:31:28.275+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:dc7d10b2:::obj533:head)]) -- 0x6766180 con 0x507ac00 2026-02-11T23:31:28.275+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.275+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1093 ==== MOSDPGPush(4.0 119/117 [PushOp(4:dc9e4e8c:::obj567:head, version: 111'567, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc9e4e8c:::obj567:head@111'567, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56ca9c0 con 0x507ac00 2026-02-11T23:31:28.275+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:dc9e4e8c:::obj567:head, version: 111'567, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc9e4e8c:::obj567:head@111'567, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 119 2026-02-11T23:31:28.275+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:dc9e4e8c:::obj567:head, version: 111'567, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc9e4e8c:::obj567:head@111'567, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.275+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.275+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1094 ==== pg_backfill(progress 4.0 e 119/119 lb 4:dc7d10b2:::obj533:head) ==== 1062+0+0 (unknown 0 0 0) 0x6178100 con 0x507ac00 2026-02-11T23:31:28.275+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:dc9e4e8c:::obj567:head, version: 111'567, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc9e4e8c:::obj567:head@111'567, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.275+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:dc7d10b2:::obj533:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.275+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:dc7d10b2:::obj533:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.275+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.275+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:dc9e4e8c:::obj567:head, version: 111'567, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc9e4e8c:::obj567:head@111'567, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.275+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:dc9e4e8c:::obj567:head, version: 111'567, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc9e4e8c:::obj567:head@111'567, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(4.0 119/117 [PushOp(4:dc9e4e8c:::obj567:head, version: 111'567, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc9e4e8c:::obj567:head@111'567, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:dc6fad89:::obj337:head local-lis/les=117/118 n=526 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.275+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.275+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.275+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:dc7d10b2:::obj533:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.275+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc6fad89:::obj337:head local-lis/les=117/118 n=526 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:dc9e4e8c:::obj567:head, version: 111'567, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc9e4e8c:::obj567:head@111'567, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.275+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc6fad89:::obj337:head local-lis/les=117/118 n=526 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:dc9e4e8c:::obj567:head@111'567, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.275+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc6fad89:::obj337:head local-lis/les=117/118 n=526 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:dc9e4e8c:::obj567:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.275+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc6fad89:::obj337:head local-lis/les=117/118 n=526 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:dc9e4e8c:::obj567:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.275+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc6fad89:::obj337:head local-lis/les=117/118 n=526 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:dc9e4e8c:::obj567:head 2026-02-11T23:31:28.275+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc6fad89:::obj337:head local-lis/les=117/118 n=526 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:dc9e4e8c:::obj567:head v 111'567 2026-02-11T23:31:28.275+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc6fad89:::obj337:head local-lis/les=117/118 n=526 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.275+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.275+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:dc9e4e8c:::obj567:head, version: 111'567, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc9e4e8c:::obj567:head@111'567, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:28.275+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc6fad89:::obj337:head local-lis/les=117/118 n=526 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.275+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.275+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.275+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:dc7d10b2:::obj533:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.275+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:dc7d10b2:::obj533:head) v3 prio 15 cost 0 latency 0.000081 pg_backfill(progress 4.0 e 119/119 lb 4:dc7d10b2:::obj533:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:dc6fad89:::obj337:head local-lis/les=117/118 n=526 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.275+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.275+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc6fad89:::obj337:head local-lis/les=117/118 n=526 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:dc7d10b2:::obj533:head) v3 2026-02-11T23:31:28.275+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc6fad89:::obj337:head local-lis/les=117/118 n=526 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:dc7d10b2:::obj533:head) 2026-02-11T23:31:28.275+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.275+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:dc7d10b2:::obj533:head) v3 finish latency 0.000012 2026-02-11T23:31:28.275+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.276+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.276+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc7d10b2:::obj533:head local-lis/les=117/118 n=527 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.276+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.276+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.276+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.276+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x674b080 con 0x507ac00 2026-02-11T23:31:28.276+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:dc9e4e8c:::obj567:head)]) -- 0x674af00 con 0x507ac00 2026-02-11T23:31:28.276+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.276+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1095 ==== MOSDPGPush(4.0 119/117 [PushOp(4:dcf61aca:::obj332:head, version: 111'332, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dcf61aca:::obj332:head@111'332, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c000 con 0x507ac00 2026-02-11T23:31:28.276+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:dcf61aca:::obj332:head, version: 111'332, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dcf61aca:::obj332:head@111'332, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 119 2026-02-11T23:31:28.276+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:dcf61aca:::obj332:head, version: 111'332, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dcf61aca:::obj332:head@111'332, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.276+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.276+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1096 ==== pg_backfill(progress 4.0 e 119/119 lb 4:dc9e4e8c:::obj567:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2c580 con 0x507ac00 2026-02-11T23:31:28.276+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:dcf61aca:::obj332:head, version: 111'332, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dcf61aca:::obj332:head@111'332, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.276+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:dc9e4e8c:::obj567:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.276+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:dc9e4e8c:::obj567:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.276+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.276+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:dcf61aca:::obj332:head, version: 111'332, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dcf61aca:::obj332:head@111'332, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.276+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.276+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:dcf61aca:::obj332:head, version: 111'332, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dcf61aca:::obj332:head@111'332, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 119/117 [PushOp(4:dcf61aca:::obj332:head, version: 111'332, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dcf61aca:::obj332:head@111'332, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:dc7d10b2:::obj533:head local-lis/les=117/118 n=527 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.276+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.276+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.276+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:dc9e4e8c:::obj567:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.276+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc7d10b2:::obj533:head local-lis/les=117/118 n=527 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:dcf61aca:::obj332:head, version: 111'332, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dcf61aca:::obj332:head@111'332, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.276+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc7d10b2:::obj533:head local-lis/les=117/118 n=527 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:dcf61aca:::obj332:head@111'332, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.276+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc7d10b2:::obj533:head local-lis/les=117/118 n=527 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:dcf61aca:::obj332:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.276+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc7d10b2:::obj533:head local-lis/les=117/118 n=527 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:dcf61aca:::obj332:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.276+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc7d10b2:::obj533:head local-lis/les=117/118 n=527 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:dcf61aca:::obj332:head 2026-02-11T23:31:28.276+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc7d10b2:::obj533:head local-lis/les=117/118 n=527 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:dcf61aca:::obj332:head v 111'332 2026-02-11T23:31:28.276+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc7d10b2:::obj533:head local-lis/les=117/118 n=527 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.276+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.276+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:dcf61aca:::obj332:head, version: 111'332, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dcf61aca:::obj332:head@111'332, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:28.276+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.276+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.276+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:dc9e4e8c:::obj567:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.276+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:dc9e4e8c:::obj567:head) v3 prio 15 cost 0 latency 0.000086 pg_backfill(progress 4.0 e 119/119 lb 4:dc9e4e8c:::obj567:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:dc7d10b2:::obj533:head local-lis/les=117/118 n=527 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.276+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.276+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc7d10b2:::obj533:head local-lis/les=117/118 n=527 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:dc9e4e8c:::obj567:head) v3 2026-02-11T23:31:28.276+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc7d10b2:::obj533:head local-lis/les=117/118 n=527 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:dc9e4e8c:::obj567:head) 2026-02-11T23:31:28.276+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.276+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:dc9e4e8c:::obj567:head) v3 finish latency 0.000015 2026-02-11T23:31:28.276+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc9e4e8c:::obj567:head local-lis/les=117/118 n=528 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.276+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.276+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.276+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc9e4e8c:::obj567:head local-lis/les=117/118 n=528 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.276+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.276+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.276+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.276+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x642de00 con 0x507ac00 2026-02-11T23:31:28.276+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:dcf61aca:::obj332:head)]) -- 0x6767e00 con 0x507ac00 2026-02-11T23:31:28.276+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.277+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1097 ==== MOSDPGPush(4.0 119/117 [PushOp(4:ddce93c9:::obj164:head, version: 111'164, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ddce93c9:::obj164:head@111'164, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2b60 con 0x507ac00 2026-02-11T23:31:28.277+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:ddce93c9:::obj164:head, version: 111'164, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ddce93c9:::obj164:head@111'164, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 119 2026-02-11T23:31:28.277+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ddce93c9:::obj164:head, version: 111'164, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ddce93c9:::obj164:head@111'164, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.277+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.277+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1098 ==== pg_backfill(progress 4.0 e 119/119 lb 4:dcf61aca:::obj332:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177b80 con 0x507ac00 2026-02-11T23:31:28.277+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ddce93c9:::obj164:head, version: 111'164, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ddce93c9:::obj164:head@111'164, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.277+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:dcf61aca:::obj332:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.277+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:dcf61aca:::obj332:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.277+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.277+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ddce93c9:::obj164:head, version: 111'164, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ddce93c9:::obj164:head@111'164, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.277+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.277+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:ddce93c9:::obj164:head, version: 111'164, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ddce93c9:::obj164:head@111'164, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(4.0 119/117 [PushOp(4:ddce93c9:::obj164:head, version: 111'164, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ddce93c9:::obj164:head@111'164, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:dc9e4e8c:::obj567:head local-lis/les=117/118 n=528 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.277+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.277+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.277+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:dcf61aca:::obj332:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.277+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc9e4e8c:::obj567:head local-lis/les=117/118 n=528 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:ddce93c9:::obj164:head, version: 111'164, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ddce93c9:::obj164:head@111'164, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.277+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc9e4e8c:::obj567:head local-lis/les=117/118 n=528 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:ddce93c9:::obj164:head@111'164, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.277+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc9e4e8c:::obj567:head local-lis/les=117/118 n=528 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:ddce93c9:::obj164:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.277+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc9e4e8c:::obj567:head local-lis/les=117/118 n=528 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:ddce93c9:::obj164:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.277+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc9e4e8c:::obj567:head local-lis/les=117/118 n=528 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:ddce93c9:::obj164:head 2026-02-11T23:31:28.277+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc9e4e8c:::obj567:head local-lis/les=117/118 n=528 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:ddce93c9:::obj164:head v 111'164 2026-02-11T23:31:28.277+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc9e4e8c:::obj567:head local-lis/les=117/118 n=528 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.277+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.277+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:ddce93c9:::obj164:head, version: 111'164, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ddce93c9:::obj164:head@111'164, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000062 2026-02-11T23:31:28.277+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.277+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.277+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:dcf61aca:::obj332:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.277+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:dcf61aca:::obj332:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 4.0 e 119/119 lb 4:dcf61aca:::obj332:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:dc9e4e8c:::obj567:head local-lis/les=117/118 n=528 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.277+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.277+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc9e4e8c:::obj567:head local-lis/les=117/118 n=528 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:dcf61aca:::obj332:head) v3 2026-02-11T23:31:28.277+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dc9e4e8c:::obj567:head local-lis/les=117/118 n=528 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:dcf61aca:::obj332:head) 2026-02-11T23:31:28.277+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.277+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:dcf61aca:::obj332:head) v3 finish latency 0.000014 2026-02-11T23:31:28.277+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dcf61aca:::obj332:head local-lis/les=117/118 n=529 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.277+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.277+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.277+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dcf61aca:::obj332:head local-lis/les=117/118 n=529 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.277+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.277+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.277+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.277+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x674b380 con 0x507ac00 2026-02-11T23:31:28.277+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:ddce93c9:::obj164:head)]) -- 0x6776180 con 0x507ac00 2026-02-11T23:31:28.277+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.277+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1099 ==== MOSDPGPush(4.0 119/117 [PushOp(4:de0491ba:::obj374:head, version: 111'374, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:de0491ba:::obj374:head@111'374, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e26b60 con 0x507ac00 2026-02-11T23:31:28.277+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:de0491ba:::obj374:head, version: 111'374, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:de0491ba:::obj374:head@111'374, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 119 2026-02-11T23:31:28.277+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:de0491ba:::obj374:head, version: 111'374, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:de0491ba:::obj374:head@111'374, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.277+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.277+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1100 ==== pg_backfill(progress 4.0 e 119/119 lb 4:ddce93c9:::obj164:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177600 con 0x507ac00 2026-02-11T23:31:28.277+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:de0491ba:::obj374:head, version: 111'374, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:de0491ba:::obj374:head@111'374, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.277+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:ddce93c9:::obj164:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.277+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:ddce93c9:::obj164:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.277+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.277+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:de0491ba:::obj374:head, version: 111'374, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:de0491ba:::obj374:head@111'374, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.277+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:de0491ba:::obj374:head, version: 111'374, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:de0491ba:::obj374:head@111'374, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(4.0 119/117 [PushOp(4:de0491ba:::obj374:head, version: 111'374, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:de0491ba:::obj374:head@111'374, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:dcf61aca:::obj332:head local-lis/les=117/118 n=529 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.277+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.277+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.277+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:ddce93c9:::obj164:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.277+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dcf61aca:::obj332:head local-lis/les=117/118 n=529 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:de0491ba:::obj374:head, version: 111'374, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:de0491ba:::obj374:head@111'374, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.277+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dcf61aca:::obj332:head local-lis/les=117/118 n=529 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:de0491ba:::obj374:head@111'374, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.277+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dcf61aca:::obj332:head local-lis/les=117/118 n=529 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:de0491ba:::obj374:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.277+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dcf61aca:::obj332:head local-lis/les=117/118 n=529 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:de0491ba:::obj374:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.277+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dcf61aca:::obj332:head local-lis/les=117/118 n=529 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:de0491ba:::obj374:head 2026-02-11T23:31:28.277+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dcf61aca:::obj332:head local-lis/les=117/118 n=529 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:de0491ba:::obj374:head v 111'374 2026-02-11T23:31:28.277+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dcf61aca:::obj332:head local-lis/les=117/118 n=529 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.277+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.277+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:de0491ba:::obj374:head, version: 111'374, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:de0491ba:::obj374:head@111'374, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:28.277+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.277+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.277+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:ddce93c9:::obj164:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.277+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:ddce93c9:::obj164:head) v3 prio 15 cost 0 latency 0.000079 pg_backfill(progress 4.0 e 119/119 lb 4:ddce93c9:::obj164:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:dcf61aca:::obj332:head local-lis/les=117/118 n=529 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.277+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.277+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dcf61aca:::obj332:head local-lis/les=117/118 n=529 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:ddce93c9:::obj164:head) v3 2026-02-11T23:31:28.277+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dcf61aca:::obj332:head local-lis/les=117/118 n=529 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:ddce93c9:::obj164:head) 2026-02-11T23:31:28.277+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.277+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:ddce93c9:::obj164:head) v3 finish latency 0.000012 2026-02-11T23:31:28.277+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ddce93c9:::obj164:head local-lis/les=117/118 n=530 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.277+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.277+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.277+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ddce93c9:::obj164:head local-lis/les=117/118 n=530 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.277+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.277+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.277+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.277+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66d9500 con 0x507ac00 2026-02-11T23:31:28.277+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:de0491ba:::obj374:head)]) -- 0x6776480 con 0x507ac00 2026-02-11T23:31:28.277+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.278+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1101 ==== MOSDPGPush(4.0 119/117 [PushOp(4:deffd1ec:::obj134:head, version: 111'134, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:deffd1ec:::obj134:head@111'134, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3815380 con 0x507ac00 2026-02-11T23:31:28.278+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:deffd1ec:::obj134:head, version: 111'134, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:deffd1ec:::obj134:head@111'134, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 119 2026-02-11T23:31:28.278+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:deffd1ec:::obj134:head, version: 111'134, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:deffd1ec:::obj134:head@111'134, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.278+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.278+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1102 ==== pg_backfill(progress 4.0 e 119/119 lb 4:de0491ba:::obj374:head) ==== 1062+0+0 (unknown 0 0 0) 0x5985700 con 0x507ac00 2026-02-11T23:31:28.278+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:deffd1ec:::obj134:head, version: 111'134, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:deffd1ec:::obj134:head@111'134, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.278+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:de0491ba:::obj374:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.278+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.278+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:de0491ba:::obj374:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.278+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:deffd1ec:::obj134:head, version: 111'134, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:deffd1ec:::obj134:head@111'134, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.278+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.278+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:deffd1ec:::obj134:head, version: 111'134, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:deffd1ec:::obj134:head@111'134, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 119/117 [PushOp(4:deffd1ec:::obj134:head, version: 111'134, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:deffd1ec:::obj134:head@111'134, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:ddce93c9:::obj164:head local-lis/les=117/118 n=530 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.278+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.278+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ddce93c9:::obj164:head local-lis/les=117/118 n=530 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:deffd1ec:::obj134:head, version: 111'134, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:deffd1ec:::obj134:head@111'134, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.278+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.278+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ddce93c9:::obj164:head local-lis/les=117/118 n=530 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:deffd1ec:::obj134:head@111'134, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.278+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:de0491ba:::obj374:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.278+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ddce93c9:::obj164:head local-lis/les=117/118 n=530 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:deffd1ec:::obj134:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.278+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ddce93c9:::obj164:head local-lis/les=117/118 n=530 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:deffd1ec:::obj134:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.278+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ddce93c9:::obj164:head local-lis/les=117/118 n=530 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:deffd1ec:::obj134:head 2026-02-11T23:31:28.278+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ddce93c9:::obj164:head local-lis/les=117/118 n=530 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:deffd1ec:::obj134:head v 111'134 2026-02-11T23:31:28.278+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ddce93c9:::obj164:head local-lis/les=117/118 n=530 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.278+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.278+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:deffd1ec:::obj134:head, version: 111'134, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:deffd1ec:::obj134:head@111'134, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:28.278+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.278+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.278+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:de0491ba:::obj374:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.278+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:de0491ba:::obj374:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 4.0 e 119/119 lb 4:de0491ba:::obj374:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:ddce93c9:::obj164:head local-lis/les=117/118 n=530 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.278+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.278+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ddce93c9:::obj164:head local-lis/les=117/118 n=530 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:de0491ba:::obj374:head) v3 2026-02-11T23:31:28.278+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ddce93c9:::obj164:head local-lis/les=117/118 n=530 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:de0491ba:::obj374:head) 2026-02-11T23:31:28.278+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.278+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:de0491ba:::obj374:head) v3 finish latency 0.000015 2026-02-11T23:31:28.278+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:de0491ba:::obj374:head local-lis/les=117/118 n=531 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.278+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.278+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.278+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:de0491ba:::obj374:head local-lis/les=117/118 n=531 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.278+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.278+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.278+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.278+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x640f680 con 0x507ac00 2026-02-11T23:31:28.278+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:deffd1ec:::obj134:head)]) -- 0x6776780 con 0x507ac00 2026-02-11T23:31:28.278+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.278+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1103 ==== MOSDPGPush(4.0 119/117 [PushOp(4:dfa74a43:::obj27:head, version: 111'27, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dfa74a43:::obj27:head@111'27, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x3815a00 con 0x507ac00 2026-02-11T23:31:28.278+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:dfa74a43:::obj27:head, version: 111'27, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dfa74a43:::obj27:head@111'27, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 119 2026-02-11T23:31:28.278+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:dfa74a43:::obj27:head, version: 111'27, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dfa74a43:::obj27:head@111'27, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.278+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.278+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:dfa74a43:::obj27:head, version: 111'27, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dfa74a43:::obj27:head@111'27, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.278+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1104 ==== pg_backfill(progress 4.0 e 119/119 lb 4:deffd1ec:::obj134:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9a680 con 0x507ac00 2026-02-11T23:31:28.278+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:deffd1ec:::obj134:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.278+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.278+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:deffd1ec:::obj134:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.278+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:dfa74a43:::obj27:head, version: 111'27, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dfa74a43:::obj27:head@111'27, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.278+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.278+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:dfa74a43:::obj27:head, version: 111'27, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dfa74a43:::obj27:head@111'27, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(4.0 119/117 [PushOp(4:dfa74a43:::obj27:head, version: 111'27, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dfa74a43:::obj27:head@111'27, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:de0491ba:::obj374:head local-lis/les=117/118 n=531 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.278+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.278+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.278+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:de0491ba:::obj374:head local-lis/les=117/118 n=531 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:dfa74a43:::obj27:head, version: 111'27, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dfa74a43:::obj27:head@111'27, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.278+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:deffd1ec:::obj134:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.278+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:de0491ba:::obj374:head local-lis/les=117/118 n=531 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:dfa74a43:::obj27:head@111'27, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.278+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:de0491ba:::obj374:head local-lis/les=117/118 n=531 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:dfa74a43:::obj27:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.278+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:de0491ba:::obj374:head local-lis/les=117/118 n=531 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:dfa74a43:::obj27:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.278+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:de0491ba:::obj374:head local-lis/les=117/118 n=531 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:dfa74a43:::obj27:head 2026-02-11T23:31:28.278+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:de0491ba:::obj374:head local-lis/les=117/118 n=531 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:dfa74a43:::obj27:head v 111'27 2026-02-11T23:31:28.278+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:de0491ba:::obj374:head local-lis/les=117/118 n=531 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.278+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.278+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:dfa74a43:::obj27:head, version: 111'27, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dfa74a43:::obj27:head@111'27, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:28.278+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.278+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.278+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:deffd1ec:::obj134:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.278+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:deffd1ec:::obj134:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 4.0 e 119/119 lb 4:deffd1ec:::obj134:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:de0491ba:::obj374:head local-lis/les=117/118 n=531 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.278+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.278+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:de0491ba:::obj374:head local-lis/les=117/118 n=531 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:deffd1ec:::obj134:head) v3 2026-02-11T23:31:28.278+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:de0491ba:::obj374:head local-lis/les=117/118 n=531 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:deffd1ec:::obj134:head) 2026-02-11T23:31:28.278+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.278+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:deffd1ec:::obj134:head) v3 finish latency 0.000015 2026-02-11T23:31:28.278+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:deffd1ec:::obj134:head local-lis/les=117/118 n=532 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.278+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.278+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:deffd1ec:::obj134:head local-lis/les=117/118 n=532 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.278+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.278+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.278+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.278+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.278+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x5d55200 con 0x507ac00 2026-02-11T23:31:28.278+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:dfa74a43:::obj27:head)]) -- 0x6776a80 con 0x507ac00 2026-02-11T23:31:28.278+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.279+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1105 ==== MOSDPGPush(4.0 119/117 [PushOp(4:e03f6950:::obj575:head, version: 111'575, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e03f6950:::obj575:head@111'575, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56cad00 con 0x507ac00 2026-02-11T23:31:28.279+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:e03f6950:::obj575:head, version: 111'575, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e03f6950:::obj575:head@111'575, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.279+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e03f6950:::obj575:head, version: 111'575, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e03f6950:::obj575:head@111'575, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.279+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.279+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1106 ==== pg_backfill(progress 4.0 e 119/119 lb 4:dfa74a43:::obj27:head) ==== 1061+0+0 (unknown 0 0 0) 0x6176000 con 0x507ac00 2026-02-11T23:31:28.279+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:dfa74a43:::obj27:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.279+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e03f6950:::obj575:head, version: 111'575, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e03f6950:::obj575:head@111'575, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.279+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:dfa74a43:::obj27:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.279+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.279+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e03f6950:::obj575:head, version: 111'575, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e03f6950:::obj575:head@111'575, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.279+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:e03f6950:::obj575:head, version: 111'575, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e03f6950:::obj575:head@111'575, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 119/117 [PushOp(4:e03f6950:::obj575:head, version: 111'575, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e03f6950:::obj575:head@111'575, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:deffd1ec:::obj134:head local-lis/les=117/118 n=532 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.279+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.279+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:deffd1ec:::obj134:head local-lis/les=117/118 n=532 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:e03f6950:::obj575:head, version: 111'575, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e03f6950:::obj575:head@111'575, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.279+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.279+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:deffd1ec:::obj134:head local-lis/les=117/118 n=532 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:e03f6950:::obj575:head@111'575, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.279+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:dfa74a43:::obj27:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.279+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:deffd1ec:::obj134:head local-lis/les=117/118 n=532 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:e03f6950:::obj575:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.279+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:deffd1ec:::obj134:head local-lis/les=117/118 n=532 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:e03f6950:::obj575:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.279+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:deffd1ec:::obj134:head local-lis/les=117/118 n=532 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:e03f6950:::obj575:head 2026-02-11T23:31:28.279+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:deffd1ec:::obj134:head local-lis/les=117/118 n=532 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:e03f6950:::obj575:head v 111'575 2026-02-11T23:31:28.279+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:deffd1ec:::obj134:head local-lis/les=117/118 n=532 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.279+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.279+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:e03f6950:::obj575:head, version: 111'575, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e03f6950:::obj575:head@111'575, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:28.279+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.279+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.279+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:dfa74a43:::obj27:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.279+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:dfa74a43:::obj27:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 4.0 e 119/119 lb 4:dfa74a43:::obj27:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:deffd1ec:::obj134:head local-lis/les=117/118 n=532 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.279+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.279+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:deffd1ec:::obj134:head local-lis/les=117/118 n=532 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:dfa74a43:::obj27:head) v3 2026-02-11T23:31:28.279+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:deffd1ec:::obj134:head local-lis/les=117/118 n=532 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:dfa74a43:::obj27:head) 2026-02-11T23:31:28.279+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.279+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:dfa74a43:::obj27:head) v3 finish latency 0.000012 2026-02-11T23:31:28.279+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dfa74a43:::obj27:head local-lis/les=117/118 n=533 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.279+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.279+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.279+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dfa74a43:::obj27:head local-lis/les=117/118 n=533 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.279+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.279+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.279+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.279+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x674b500 con 0x507ac00 2026-02-11T23:31:28.279+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:e03f6950:::obj575:head)]) -- 0x6776c00 con 0x507ac00 2026-02-11T23:31:28.279+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.279+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1107 ==== MOSDPGPush(4.0 119/117 [PushOp(4:e05d79f1:::obj206:head, version: 111'206, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e05d79f1:::obj206:head@111'206, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x555cea0 con 0x507ac00 2026-02-11T23:31:28.279+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:e05d79f1:::obj206:head, version: 111'206, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e05d79f1:::obj206:head@111'206, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.279+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e05d79f1:::obj206:head, version: 111'206, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e05d79f1:::obj206:head@111'206, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.279+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.279+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1108 ==== pg_backfill(progress 4.0 e 119/119 lb 4:e03f6950:::obj575:head) ==== 1062+0+0 (unknown 0 0 0) 0x6179700 con 0x507ac00 2026-02-11T23:31:28.279+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e05d79f1:::obj206:head, version: 111'206, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e05d79f1:::obj206:head@111'206, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.279+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:e03f6950:::obj575:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.279+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e03f6950:::obj575:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.279+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.279+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e05d79f1:::obj206:head, version: 111'206, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e05d79f1:::obj206:head@111'206, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.279+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:e05d79f1:::obj206:head, version: 111'206, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e05d79f1:::obj206:head@111'206, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 119/117 [PushOp(4:e05d79f1:::obj206:head, version: 111'206, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e05d79f1:::obj206:head@111'206, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:dfa74a43:::obj27:head local-lis/les=117/118 n=533 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.279+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.279+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.279+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dfa74a43:::obj27:head local-lis/les=117/118 n=533 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:e05d79f1:::obj206:head, version: 111'206, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e05d79f1:::obj206:head@111'206, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.279+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e03f6950:::obj575:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.279+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dfa74a43:::obj27:head local-lis/les=117/118 n=533 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:e05d79f1:::obj206:head@111'206, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.279+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dfa74a43:::obj27:head local-lis/les=117/118 n=533 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:e05d79f1:::obj206:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.279+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dfa74a43:::obj27:head local-lis/les=117/118 n=533 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:e05d79f1:::obj206:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.279+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dfa74a43:::obj27:head local-lis/les=117/118 n=533 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:e05d79f1:::obj206:head 2026-02-11T23:31:28.279+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dfa74a43:::obj27:head local-lis/les=117/118 n=533 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:e05d79f1:::obj206:head v 111'206 2026-02-11T23:31:28.279+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dfa74a43:::obj27:head local-lis/les=117/118 n=533 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.279+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.279+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:e05d79f1:::obj206:head, version: 111'206, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e05d79f1:::obj206:head@111'206, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:28.279+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.279+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.279+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e03f6950:::obj575:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.279+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:e03f6950:::obj575:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 4.0 e 119/119 lb 4:e03f6950:::obj575:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:dfa74a43:::obj27:head local-lis/les=117/118 n=533 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.279+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.279+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dfa74a43:::obj27:head local-lis/les=117/118 n=533 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:e03f6950:::obj575:head) v3 2026-02-11T23:31:28.279+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:dfa74a43:::obj27:head local-lis/les=117/118 n=533 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:e03f6950:::obj575:head) 2026-02-11T23:31:28.279+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.279+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:e03f6950:::obj575:head) v3 finish latency 0.000017 2026-02-11T23:31:28.279+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e03f6950:::obj575:head local-lis/les=117/118 n=534 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.279+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.280+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.280+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e03f6950:::obj575:head local-lis/les=117/118 n=534 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.280+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.280+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.280+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.280+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x671d080 con 0x507ac00 2026-02-11T23:31:28.280+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:e05d79f1:::obj206:head)]) -- 0x6776f00 con 0x507ac00 2026-02-11T23:31:28.280+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.280+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1109 ==== MOSDPGPush(4.0 119/117 [PushOp(4:e0a44829:::obj14:head, version: 111'14, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e0a44829:::obj14:head@111'14, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x55f1520 con 0x507ac00 2026-02-11T23:31:28.280+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:e0a44829:::obj14:head, version: 111'14, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e0a44829:::obj14:head@111'14, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 119 2026-02-11T23:31:28.280+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e0a44829:::obj14:head, version: 111'14, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e0a44829:::obj14:head@111'14, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.280+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.280+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1110 ==== pg_backfill(progress 4.0 e 119/119 lb 4:e05d79f1:::obj206:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2cb00 con 0x507ac00 2026-02-11T23:31:28.280+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:e05d79f1:::obj206:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.280+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e05d79f1:::obj206:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.280+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e0a44829:::obj14:head, version: 111'14, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e0a44829:::obj14:head@111'14, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.280+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.280+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e0a44829:::obj14:head, version: 111'14, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e0a44829:::obj14:head@111'14, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.280+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:e0a44829:::obj14:head, version: 111'14, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e0a44829:::obj14:head@111'14, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 119/117 [PushOp(4:e0a44829:::obj14:head, version: 111'14, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e0a44829:::obj14:head@111'14, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:e03f6950:::obj575:head local-lis/les=117/118 n=534 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.280+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.280+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.280+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e05d79f1:::obj206:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.280+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e03f6950:::obj575:head local-lis/les=117/118 n=534 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:e0a44829:::obj14:head, version: 111'14, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e0a44829:::obj14:head@111'14, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.280+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e03f6950:::obj575:head local-lis/les=117/118 n=534 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:e0a44829:::obj14:head@111'14, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.280+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e03f6950:::obj575:head local-lis/les=117/118 n=534 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:e0a44829:::obj14:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.280+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e03f6950:::obj575:head local-lis/les=117/118 n=534 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:e0a44829:::obj14:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.280+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e03f6950:::obj575:head local-lis/les=117/118 n=534 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:e0a44829:::obj14:head 2026-02-11T23:31:28.280+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e03f6950:::obj575:head local-lis/les=117/118 n=534 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:e0a44829:::obj14:head v 111'14 2026-02-11T23:31:28.280+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e03f6950:::obj575:head local-lis/les=117/118 n=534 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.280+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.280+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:e0a44829:::obj14:head, version: 111'14, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e0a44829:::obj14:head@111'14, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:28.280+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.280+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.280+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e05d79f1:::obj206:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.280+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:e05d79f1:::obj206:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 4.0 e 119/119 lb 4:e05d79f1:::obj206:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:e03f6950:::obj575:head local-lis/les=117/118 n=534 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.280+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.280+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e03f6950:::obj575:head local-lis/les=117/118 n=534 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:e05d79f1:::obj206:head) v3 2026-02-11T23:31:28.280+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e03f6950:::obj575:head local-lis/les=117/118 n=534 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:e05d79f1:::obj206:head) 2026-02-11T23:31:28.280+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.280+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:e05d79f1:::obj206:head) v3 finish latency 0.000013 2026-02-11T23:31:28.280+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e05d79f1:::obj206:head local-lis/les=117/118 n=535 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.280+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.280+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.280+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e05d79f1:::obj206:head local-lis/les=117/118 n=535 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.280+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.280+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.280+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.280+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x671d380 con 0x507ac00 2026-02-11T23:31:28.280+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:e0a44829:::obj14:head)]) -- 0x6777200 con 0x507ac00 2026-02-11T23:31:28.280+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.280+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1111 ==== MOSDPGPush(4.0 119/117 [PushOp(4:e10533c6:::obj450:head, version: 111'450, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e10533c6:::obj450:head@111'450, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f0b60 con 0x507ac00 2026-02-11T23:31:28.280+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:e10533c6:::obj450:head, version: 111'450, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e10533c6:::obj450:head@111'450, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.280+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e10533c6:::obj450:head, version: 111'450, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e10533c6:::obj450:head@111'450, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.281+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.281+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e10533c6:::obj450:head, version: 111'450, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e10533c6:::obj450:head@111'450, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.281+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.281+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1112 ==== pg_backfill(progress 4.0 e 119/119 lb 4:e0a44829:::obj14:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d2c000 con 0x507ac00 2026-02-11T23:31:28.281+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:e0a44829:::obj14:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.281+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e0a44829:::obj14:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.281+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e10533c6:::obj450:head, version: 111'450, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e10533c6:::obj450:head@111'450, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.281+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.281+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:e10533c6:::obj450:head, version: 111'450, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e10533c6:::obj450:head@111'450, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 119/117 [PushOp(4:e10533c6:::obj450:head, version: 111'450, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e10533c6:::obj450:head@111'450, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:e05d79f1:::obj206:head local-lis/les=117/118 n=535 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.281+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.281+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e05d79f1:::obj206:head local-lis/les=117/118 n=535 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:e10533c6:::obj450:head, version: 111'450, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e10533c6:::obj450:head@111'450, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.281+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e05d79f1:::obj206:head local-lis/les=117/118 n=535 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:e10533c6:::obj450:head@111'450, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.281+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.281+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e0a44829:::obj14:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.281+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e05d79f1:::obj206:head local-lis/les=117/118 n=535 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:e10533c6:::obj450:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.281+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e05d79f1:::obj206:head local-lis/les=117/118 n=535 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:e10533c6:::obj450:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.281+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e05d79f1:::obj206:head local-lis/les=117/118 n=535 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:e10533c6:::obj450:head 2026-02-11T23:31:28.281+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e05d79f1:::obj206:head local-lis/les=117/118 n=535 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:e10533c6:::obj450:head v 111'450 2026-02-11T23:31:28.281+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e05d79f1:::obj206:head local-lis/les=117/118 n=535 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.281+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.281+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:e10533c6:::obj450:head, version: 111'450, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e10533c6:::obj450:head@111'450, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:28.281+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.281+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.281+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e0a44829:::obj14:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.281+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:e0a44829:::obj14:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 4.0 e 119/119 lb 4:e0a44829:::obj14:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:e05d79f1:::obj206:head local-lis/les=117/118 n=535 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.281+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.281+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e05d79f1:::obj206:head local-lis/les=117/118 n=535 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:e0a44829:::obj14:head) v3 2026-02-11T23:31:28.281+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e05d79f1:::obj206:head local-lis/les=117/118 n=535 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:e0a44829:::obj14:head) 2026-02-11T23:31:28.281+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.281+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:e0a44829:::obj14:head) v3 finish latency 0.000012 2026-02-11T23:31:28.281+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e0a44829:::obj14:head local-lis/les=117/118 n=536 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.281+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.281+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.281+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e0a44829:::obj14:head local-lis/les=117/118 n=536 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.281+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.281+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.281+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.281+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x640fb00 con 0x507ac00 2026-02-11T23:31:28.281+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:e10533c6:::obj450:head)]) -- 0x6777380 con 0x507ac00 2026-02-11T23:31:28.281+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.281+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1113 ==== MOSDPGPush(4.0 119/117 [PushOp(4:e196274b:::obj304:head, version: 111'304, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e196274b:::obj304:head@111'304, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5617380 con 0x507ac00 2026-02-11T23:31:28.281+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:e196274b:::obj304:head, version: 111'304, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e196274b:::obj304:head@111'304, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 119 2026-02-11T23:31:28.281+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e196274b:::obj304:head, version: 111'304, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e196274b:::obj304:head@111'304, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.281+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.281+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1114 ==== pg_backfill(progress 4.0 e 119/119 lb 4:e10533c6:::obj450:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e100 con 0x507ac00 2026-02-11T23:31:28.281+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e196274b:::obj304:head, version: 111'304, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e196274b:::obj304:head@111'304, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.281+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:e10533c6:::obj450:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.281+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e10533c6:::obj450:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.281+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.281+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e196274b:::obj304:head, version: 111'304, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e196274b:::obj304:head@111'304, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.281+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:e196274b:::obj304:head, version: 111'304, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e196274b:::obj304:head@111'304, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(4.0 119/117 [PushOp(4:e196274b:::obj304:head, version: 111'304, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e196274b:::obj304:head@111'304, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:e0a44829:::obj14:head local-lis/les=117/118 n=536 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.281+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.281+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.281+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e0a44829:::obj14:head local-lis/les=117/118 n=536 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:e196274b:::obj304:head, version: 111'304, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e196274b:::obj304:head@111'304, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.281+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e10533c6:::obj450:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.281+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e0a44829:::obj14:head local-lis/les=117/118 n=536 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:e196274b:::obj304:head@111'304, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.281+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e0a44829:::obj14:head local-lis/les=117/118 n=536 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:e196274b:::obj304:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.281+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e0a44829:::obj14:head local-lis/les=117/118 n=536 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:e196274b:::obj304:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.281+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e0a44829:::obj14:head local-lis/les=117/118 n=536 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:e196274b:::obj304:head 2026-02-11T23:31:28.281+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e0a44829:::obj14:head local-lis/les=117/118 n=536 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:e196274b:::obj304:head v 111'304 2026-02-11T23:31:28.281+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e0a44829:::obj14:head local-lis/les=117/118 n=536 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.281+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.281+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:e196274b:::obj304:head, version: 111'304, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e196274b:::obj304:head@111'304, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:28.281+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.281+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.281+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e10533c6:::obj450:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.281+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:e10533c6:::obj450:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 4.0 e 119/119 lb 4:e10533c6:::obj450:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:e0a44829:::obj14:head local-lis/les=117/118 n=536 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.281+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.281+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e0a44829:::obj14:head local-lis/les=117/118 n=536 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:e10533c6:::obj450:head) v3 2026-02-11T23:31:28.281+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e0a44829:::obj14:head local-lis/les=117/118 n=536 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:e10533c6:::obj450:head) 2026-02-11T23:31:28.281+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.281+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:e10533c6:::obj450:head) v3 finish latency 0.000018 2026-02-11T23:31:28.281+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e10533c6:::obj450:head local-lis/les=117/118 n=537 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.281+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.281+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.281+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e10533c6:::obj450:head local-lis/les=117/118 n=537 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.281+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.281+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.281+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.281+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x674b680 con 0x507ac00 2026-02-11T23:31:28.281+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:e196274b:::obj304:head)]) -- 0x6777680 con 0x507ac00 2026-02-11T23:31:28.281+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.282+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1115 ==== MOSDPGPush(4.0 119/117 [PushOp(4:e1d56785:::obj141:head, version: 111'141, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e1d56785:::obj141:head@111'141, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5697ba0 con 0x507ac00 2026-02-11T23:31:28.282+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:e1d56785:::obj141:head, version: 111'141, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e1d56785:::obj141:head@111'141, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.282+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e1d56785:::obj141:head, version: 111'141, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e1d56785:::obj141:head@111'141, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.282+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.282+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1116 ==== pg_backfill(progress 4.0 e 119/119 lb 4:e196274b:::obj304:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176580 con 0x507ac00 2026-02-11T23:31:28.282+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e1d56785:::obj141:head, version: 111'141, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e1d56785:::obj141:head@111'141, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.282+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:e196274b:::obj304:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.282+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e196274b:::obj304:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.282+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.282+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e1d56785:::obj141:head, version: 111'141, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e1d56785:::obj141:head@111'141, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.282+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:e1d56785:::obj141:head, version: 111'141, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e1d56785:::obj141:head@111'141, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 119/117 [PushOp(4:e1d56785:::obj141:head, version: 111'141, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e1d56785:::obj141:head@111'141, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:e10533c6:::obj450:head local-lis/les=117/118 n=537 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.282+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.282+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e10533c6:::obj450:head local-lis/les=117/118 n=537 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:e1d56785:::obj141:head, version: 111'141, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e1d56785:::obj141:head@111'141, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.282+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.282+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e196274b:::obj304:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.282+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e10533c6:::obj450:head local-lis/les=117/118 n=537 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:e1d56785:::obj141:head@111'141, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.282+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e10533c6:::obj450:head local-lis/les=117/118 n=537 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:e1d56785:::obj141:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.282+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e10533c6:::obj450:head local-lis/les=117/118 n=537 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:e1d56785:::obj141:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.282+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e10533c6:::obj450:head local-lis/les=117/118 n=537 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:e1d56785:::obj141:head 2026-02-11T23:31:28.282+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e10533c6:::obj450:head local-lis/les=117/118 n=537 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:e1d56785:::obj141:head v 111'141 2026-02-11T23:31:28.282+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e10533c6:::obj450:head local-lis/les=117/118 n=537 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.282+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.282+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:e1d56785:::obj141:head, version: 111'141, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e1d56785:::obj141:head@111'141, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:28.282+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.282+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.282+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e196274b:::obj304:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.282+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:e196274b:::obj304:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 4.0 e 119/119 lb 4:e196274b:::obj304:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:e10533c6:::obj450:head local-lis/les=117/118 n=537 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.282+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.282+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e10533c6:::obj450:head local-lis/les=117/118 n=537 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:e196274b:::obj304:head) v3 2026-02-11T23:31:28.282+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e10533c6:::obj450:head local-lis/les=117/118 n=537 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:e196274b:::obj304:head) 2026-02-11T23:31:28.282+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.282+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:e196274b:::obj304:head) v3 finish latency 0.000018 2026-02-11T23:31:28.282+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e196274b:::obj304:head local-lis/les=117/118 n=538 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.282+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.282+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.282+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e196274b:::obj304:head local-lis/les=117/118 n=538 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.282+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.282+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.282+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.282+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x671d680 con 0x507ac00 2026-02-11T23:31:28.282+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:e1d56785:::obj141:head)]) -- 0x6777980 con 0x507ac00 2026-02-11T23:31:28.282+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.282+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1117 ==== MOSDPGPush(4.0 119/117 [PushOp(4:e25e0189:::obj579:head, version: 111'579, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e25e0189:::obj579:head@111'579, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5647380 con 0x507ac00 2026-02-11T23:31:28.282+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:e25e0189:::obj579:head, version: 111'579, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e25e0189:::obj579:head@111'579, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 119 2026-02-11T23:31:28.282+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e25e0189:::obj579:head, version: 111'579, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e25e0189:::obj579:head@111'579, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.282+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.282+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1118 ==== pg_backfill(progress 4.0 e 119/119 lb 4:e1d56785:::obj141:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e680 con 0x507ac00 2026-02-11T23:31:28.282+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e25e0189:::obj579:head, version: 111'579, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e25e0189:::obj579:head@111'579, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.282+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:e1d56785:::obj141:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.282+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.282+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e1d56785:::obj141:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.282+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e25e0189:::obj579:head, version: 111'579, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e25e0189:::obj579:head@111'579, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.282+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:e25e0189:::obj579:head, version: 111'579, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e25e0189:::obj579:head@111'579, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(4.0 119/117 [PushOp(4:e25e0189:::obj579:head, version: 111'579, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e25e0189:::obj579:head@111'579, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:e196274b:::obj304:head local-lis/les=117/118 n=538 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.282+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.282+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.282+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e196274b:::obj304:head local-lis/les=117/118 n=538 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:e25e0189:::obj579:head, version: 111'579, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e25e0189:::obj579:head@111'579, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.282+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e196274b:::obj304:head local-lis/les=117/118 n=538 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:e25e0189:::obj579:head@111'579, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.282+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.282+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e1d56785:::obj141:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.282+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e196274b:::obj304:head local-lis/les=117/118 n=538 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:e25e0189:::obj579:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.282+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e196274b:::obj304:head local-lis/les=117/118 n=538 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:e25e0189:::obj579:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.282+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e196274b:::obj304:head local-lis/les=117/118 n=538 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:e25e0189:::obj579:head 2026-02-11T23:31:28.282+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e196274b:::obj304:head local-lis/les=117/118 n=538 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:e25e0189:::obj579:head v 111'579 2026-02-11T23:31:28.282+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e196274b:::obj304:head local-lis/les=117/118 n=538 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.282+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.282+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:e25e0189:::obj579:head, version: 111'579, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e25e0189:::obj579:head@111'579, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:28.282+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.282+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.282+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e1d56785:::obj141:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.282+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:e1d56785:::obj141:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 4.0 e 119/119 lb 4:e1d56785:::obj141:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:e196274b:::obj304:head local-lis/les=117/118 n=538 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.282+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.282+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e196274b:::obj304:head local-lis/les=117/118 n=538 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:e1d56785:::obj141:head) v3 2026-02-11T23:31:28.282+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e196274b:::obj304:head local-lis/les=117/118 n=538 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:e1d56785:::obj141:head) 2026-02-11T23:31:28.282+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.282+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:e1d56785:::obj141:head) v3 finish latency 0.000018 2026-02-11T23:31:28.282+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e1d56785:::obj141:head local-lis/les=117/118 n=539 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.282+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.282+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.282+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e1d56785:::obj141:head local-lis/les=117/118 n=539 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.282+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.282+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.282+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.282+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66d9080 con 0x507ac00 2026-02-11T23:31:28.282+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:e25e0189:::obj579:head)]) -- 0x6777c80 con 0x507ac00 2026-02-11T23:31:28.282+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.283+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1119 ==== MOSDPGPush(4.0 119/117 [PushOp(4:e26151b8:::obj492:head, version: 111'492, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e26151b8:::obj492:head@111'492, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f0d00 con 0x507ac00 2026-02-11T23:31:28.283+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:e26151b8:::obj492:head, version: 111'492, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e26151b8:::obj492:head@111'492, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 119 2026-02-11T23:31:28.283+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e26151b8:::obj492:head, version: 111'492, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e26151b8:::obj492:head@111'492, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.283+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.283+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1120 ==== pg_backfill(progress 4.0 e 119/119 lb 4:e25e0189:::obj579:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2d080 con 0x507ac00 2026-02-11T23:31:28.283+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e26151b8:::obj492:head, version: 111'492, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e26151b8:::obj492:head@111'492, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.283+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:e25e0189:::obj579:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 119 2026-02-11T23:31:28.283+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e25e0189:::obj579:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.283+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.283+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e26151b8:::obj492:head, version: 111'492, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e26151b8:::obj492:head@111'492, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.283+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:e26151b8:::obj492:head, version: 111'492, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e26151b8:::obj492:head@111'492, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 119/117 [PushOp(4:e26151b8:::obj492:head, version: 111'492, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e26151b8:::obj492:head@111'492, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:e1d56785:::obj141:head local-lis/les=117/118 n=539 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.283+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.283+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.283+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e25e0189:::obj579:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.283+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e1d56785:::obj141:head local-lis/les=117/118 n=539 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:e26151b8:::obj492:head, version: 111'492, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e26151b8:::obj492:head@111'492, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.283+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e1d56785:::obj141:head local-lis/les=117/118 n=539 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:e26151b8:::obj492:head@111'492, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.283+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e1d56785:::obj141:head local-lis/les=117/118 n=539 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:e26151b8:::obj492:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.283+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e1d56785:::obj141:head local-lis/les=117/118 n=539 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:e26151b8:::obj492:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.283+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e1d56785:::obj141:head local-lis/les=117/118 n=539 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:e26151b8:::obj492:head 2026-02-11T23:31:28.283+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e1d56785:::obj141:head local-lis/les=117/118 n=539 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:e26151b8:::obj492:head v 111'492 2026-02-11T23:31:28.283+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e1d56785:::obj141:head local-lis/les=117/118 n=539 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.283+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.283+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:e26151b8:::obj492:head, version: 111'492, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e26151b8:::obj492:head@111'492, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000072 2026-02-11T23:31:28.283+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.283+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.283+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e25e0189:::obj579:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.283+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:e25e0189:::obj579:head) v3 prio 15 cost 0 latency 0.000089 pg_backfill(progress 4.0 e 119/119 lb 4:e25e0189:::obj579:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:e1d56785:::obj141:head local-lis/les=117/118 n=539 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.283+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.283+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e1d56785:::obj141:head local-lis/les=117/118 n=539 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:e25e0189:::obj579:head) v3 2026-02-11T23:31:28.283+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e1d56785:::obj141:head local-lis/les=117/118 n=539 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:e25e0189:::obj579:head) 2026-02-11T23:31:28.283+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.283+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:e25e0189:::obj579:head) v3 finish latency 0.000013 2026-02-11T23:31:28.283+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e25e0189:::obj579:head local-lis/les=117/118 n=540 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.283+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.283+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.283+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e25e0189:::obj579:head local-lis/les=117/118 n=540 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.283+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.283+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.283+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.283+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x642d500 con 0x507ac00 2026-02-11T23:31:28.283+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:e26151b8:::obj492:head)]) -- 0x6776780 con 0x507ac00 2026-02-11T23:31:28.283+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.283+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1121 ==== MOSDPGPush(4.0 119/117 [PushOp(4:e3cd6c93:::obj259:head, version: 111'259, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e3cd6c93:::obj259:head@111'259, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56b64e0 con 0x507ac00 2026-02-11T23:31:28.283+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:e3cd6c93:::obj259:head, version: 111'259, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e3cd6c93:::obj259:head@111'259, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 119 2026-02-11T23:31:28.283+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e3cd6c93:::obj259:head, version: 111'259, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e3cd6c93:::obj259:head@111'259, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.283+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.283+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1122 ==== pg_backfill(progress 4.0 e 119/119 lb 4:e26151b8:::obj492:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2f700 con 0x507ac00 2026-02-11T23:31:28.283+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e3cd6c93:::obj259:head, version: 111'259, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e3cd6c93:::obj259:head@111'259, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.283+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:e26151b8:::obj492:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.283+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e26151b8:::obj492:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.283+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.283+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e3cd6c93:::obj259:head, version: 111'259, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e3cd6c93:::obj259:head@111'259, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.283+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:e3cd6c93:::obj259:head, version: 111'259, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e3cd6c93:::obj259:head@111'259, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 119/117 [PushOp(4:e3cd6c93:::obj259:head, version: 111'259, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e3cd6c93:::obj259:head@111'259, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:e25e0189:::obj579:head local-lis/les=117/118 n=540 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.283+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.283+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.283+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e25e0189:::obj579:head local-lis/les=117/118 n=540 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:e3cd6c93:::obj259:head, version: 111'259, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e3cd6c93:::obj259:head@111'259, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.283+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e25e0189:::obj579:head local-lis/les=117/118 n=540 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:e3cd6c93:::obj259:head@111'259, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.283+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.283+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e26151b8:::obj492:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.283+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e25e0189:::obj579:head local-lis/les=117/118 n=540 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:e3cd6c93:::obj259:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.283+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e25e0189:::obj579:head local-lis/les=117/118 n=540 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:e3cd6c93:::obj259:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.283+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e25e0189:::obj579:head local-lis/les=117/118 n=540 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:e3cd6c93:::obj259:head 2026-02-11T23:31:28.283+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e25e0189:::obj579:head local-lis/les=117/118 n=540 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:e3cd6c93:::obj259:head v 111'259 2026-02-11T23:31:28.283+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e25e0189:::obj579:head local-lis/les=117/118 n=540 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.283+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.283+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:e3cd6c93:::obj259:head, version: 111'259, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e3cd6c93:::obj259:head@111'259, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:28.283+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.283+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.283+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e26151b8:::obj492:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.283+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:e26151b8:::obj492:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 4.0 e 119/119 lb 4:e26151b8:::obj492:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:e25e0189:::obj579:head local-lis/les=117/118 n=540 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.283+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.283+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e25e0189:::obj579:head local-lis/les=117/118 n=540 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:e26151b8:::obj492:head) v3 2026-02-11T23:31:28.283+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e25e0189:::obj579:head local-lis/les=117/118 n=540 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:e26151b8:::obj492:head) 2026-02-11T23:31:28.283+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.283+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:e26151b8:::obj492:head) v3 finish latency 0.000018 2026-02-11T23:31:28.283+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e26151b8:::obj492:head local-lis/les=117/118 n=541 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.283+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.284+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.284+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e26151b8:::obj492:head local-lis/les=117/118 n=541 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.284+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.284+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.284+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.284+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x674b980 con 0x507ac00 2026-02-11T23:31:28.284+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:e3cd6c93:::obj259:head)]) -- 0x6776480 con 0x507ac00 2026-02-11T23:31:28.284+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.284+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1123 ==== MOSDPGPush(4.0 119/117 [PushOp(4:e4109927:::obj482:head, version: 111'482, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e4109927:::obj482:head@111'482, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56caea0 con 0x507ac00 2026-02-11T23:31:28.284+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:e4109927:::obj482:head, version: 111'482, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e4109927:::obj482:head@111'482, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 119 2026-02-11T23:31:28.284+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e4109927:::obj482:head, version: 111'482, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e4109927:::obj482:head@111'482, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.284+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.284+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1124 ==== pg_backfill(progress 4.0 e 119/119 lb 4:e3cd6c93:::obj259:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f11600 con 0x507ac00 2026-02-11T23:31:28.284+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e4109927:::obj482:head, version: 111'482, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e4109927:::obj482:head@111'482, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.284+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:e3cd6c93:::obj259:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.284+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e3cd6c93:::obj259:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.284+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.284+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e4109927:::obj482:head, version: 111'482, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e4109927:::obj482:head@111'482, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.284+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:e4109927:::obj482:head, version: 111'482, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e4109927:::obj482:head@111'482, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(4.0 119/117 [PushOp(4:e4109927:::obj482:head, version: 111'482, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e4109927:::obj482:head@111'482, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:e26151b8:::obj492:head local-lis/les=117/118 n=541 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.284+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.284+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e26151b8:::obj492:head local-lis/les=117/118 n=541 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:e4109927:::obj482:head, version: 111'482, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e4109927:::obj482:head@111'482, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.284+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.284+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e26151b8:::obj492:head local-lis/les=117/118 n=541 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:e4109927:::obj482:head@111'482, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.284+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e3cd6c93:::obj259:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.284+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e26151b8:::obj492:head local-lis/les=117/118 n=541 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:e4109927:::obj482:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.284+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e26151b8:::obj492:head local-lis/les=117/118 n=541 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:e4109927:::obj482:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.284+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e26151b8:::obj492:head local-lis/les=117/118 n=541 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:e4109927:::obj482:head 2026-02-11T23:31:28.284+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e26151b8:::obj492:head local-lis/les=117/118 n=541 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:e4109927:::obj482:head v 111'482 2026-02-11T23:31:28.284+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e26151b8:::obj492:head local-lis/les=117/118 n=541 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.284+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.284+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:e4109927:::obj482:head, version: 111'482, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e4109927:::obj482:head@111'482, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:28.284+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.284+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.284+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e3cd6c93:::obj259:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.284+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:e3cd6c93:::obj259:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 119/119 lb 4:e3cd6c93:::obj259:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:e26151b8:::obj492:head local-lis/les=117/118 n=541 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.284+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.284+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e26151b8:::obj492:head local-lis/les=117/118 n=541 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:e3cd6c93:::obj259:head) v3 2026-02-11T23:31:28.284+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e26151b8:::obj492:head local-lis/les=117/118 n=541 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:e3cd6c93:::obj259:head) 2026-02-11T23:31:28.284+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.284+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:e3cd6c93:::obj259:head) v3 finish latency 0.000018 2026-02-11T23:31:28.284+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e3cd6c93:::obj259:head local-lis/les=117/118 n=542 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.284+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.284+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.284+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e3cd6c93:::obj259:head local-lis/les=117/118 n=542 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.284+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.284+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.284+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.284+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x671dc80 con 0x507ac00 2026-02-11T23:31:28.284+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:e4109927:::obj482:head)]) -- 0x6776180 con 0x507ac00 2026-02-11T23:31:28.284+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.284+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1125 ==== MOSDPGPush(4.0 119/117 [PushOp(4:e43ffde6:::obj285:head, version: 111'285, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e43ffde6:::obj285:head@111'285, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c340 con 0x507ac00 2026-02-11T23:31:28.284+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:e43ffde6:::obj285:head, version: 111'285, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e43ffde6:::obj285:head@111'285, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 119 2026-02-11T23:31:28.284+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e43ffde6:::obj285:head, version: 111'285, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e43ffde6:::obj285:head@111'285, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.284+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.284+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1126 ==== pg_backfill(progress 4.0 e 119/119 lb 4:e4109927:::obj482:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f11080 con 0x507ac00 2026-02-11T23:31:28.284+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e43ffde6:::obj285:head, version: 111'285, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e43ffde6:::obj285:head@111'285, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.284+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:e4109927:::obj482:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.284+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e4109927:::obj482:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.284+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.284+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e43ffde6:::obj285:head, version: 111'285, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e43ffde6:::obj285:head@111'285, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.284+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:e43ffde6:::obj285:head, version: 111'285, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e43ffde6:::obj285:head@111'285, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(4.0 119/117 [PushOp(4:e43ffde6:::obj285:head, version: 111'285, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e43ffde6:::obj285:head@111'285, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:e3cd6c93:::obj259:head local-lis/les=117/118 n=542 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.284+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.284+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e3cd6c93:::obj259:head local-lis/les=117/118 n=542 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:e43ffde6:::obj285:head, version: 111'285, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e43ffde6:::obj285:head@111'285, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.284+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.284+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e3cd6c93:::obj259:head local-lis/les=117/118 n=542 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:e43ffde6:::obj285:head@111'285, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.284+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e4109927:::obj482:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.284+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e3cd6c93:::obj259:head local-lis/les=117/118 n=542 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:e43ffde6:::obj285:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.284+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e3cd6c93:::obj259:head local-lis/les=117/118 n=542 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:e43ffde6:::obj285:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.284+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e3cd6c93:::obj259:head local-lis/les=117/118 n=542 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:e43ffde6:::obj285:head 2026-02-11T23:31:28.284+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e3cd6c93:::obj259:head local-lis/les=117/118 n=542 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:e43ffde6:::obj285:head v 111'285 2026-02-11T23:31:28.284+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e3cd6c93:::obj259:head local-lis/les=117/118 n=542 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.284+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.284+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:e43ffde6:::obj285:head, version: 111'285, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e43ffde6:::obj285:head@111'285, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:28.284+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.284+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.284+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e4109927:::obj482:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.284+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:e4109927:::obj482:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 4.0 e 119/119 lb 4:e4109927:::obj482:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:e3cd6c93:::obj259:head local-lis/les=117/118 n=542 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.284+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.284+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e3cd6c93:::obj259:head local-lis/les=117/118 n=542 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:e4109927:::obj482:head) v3 2026-02-11T23:31:28.284+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e3cd6c93:::obj259:head local-lis/les=117/118 n=542 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:e4109927:::obj482:head) 2026-02-11T23:31:28.285+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.285+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:e4109927:::obj482:head) v3 finish latency 0.000022 2026-02-11T23:31:28.285+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e4109927:::obj482:head local-lis/les=117/118 n=543 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.285+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.285+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.285+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e4109927:::obj482:head local-lis/les=117/118 n=543 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.285+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.285+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.285+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.285+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x642cd80 con 0x507ac00 2026-02-11T23:31:28.285+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:e43ffde6:::obj285:head)]) -- 0x6767e00 con 0x507ac00 2026-02-11T23:31:28.285+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.285+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1127 ==== MOSDPGPush(4.0 119/117 [PushOp(4:e46d208d:::obj89:head, version: 111'89, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e46d208d:::obj89:head@111'89, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x3815520 con 0x507ac00 2026-02-11T23:31:28.285+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:e46d208d:::obj89:head, version: 111'89, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e46d208d:::obj89:head@111'89, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 119 2026-02-11T23:31:28.285+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e46d208d:::obj89:head, version: 111'89, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e46d208d:::obj89:head@111'89, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.285+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.285+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1128 ==== pg_backfill(progress 4.0 e 119/119 lb 4:e43ffde6:::obj285:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f13180 con 0x507ac00 2026-02-11T23:31:28.285+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:e43ffde6:::obj285:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.285+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e43ffde6:::obj285:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.285+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e46d208d:::obj89:head, version: 111'89, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e46d208d:::obj89:head@111'89, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.285+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.285+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e46d208d:::obj89:head, version: 111'89, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e46d208d:::obj89:head@111'89, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.285+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.285+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e43ffde6:::obj285:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.285+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:e46d208d:::obj89:head, version: 111'89, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e46d208d:::obj89:head@111'89, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 119/117 [PushOp(4:e46d208d:::obj89:head, version: 111'89, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e46d208d:::obj89:head@111'89, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:e4109927:::obj482:head local-lis/les=117/118 n=543 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.285+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.285+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e4109927:::obj482:head local-lis/les=117/118 n=543 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:e46d208d:::obj89:head, version: 111'89, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e46d208d:::obj89:head@111'89, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.285+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e4109927:::obj482:head local-lis/les=117/118 n=543 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:e46d208d:::obj89:head@111'89, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.285+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e4109927:::obj482:head local-lis/les=117/118 n=543 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:e46d208d:::obj89:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.285+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e4109927:::obj482:head local-lis/les=117/118 n=543 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:e46d208d:::obj89:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.285+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e4109927:::obj482:head local-lis/les=117/118 n=543 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:e46d208d:::obj89:head 2026-02-11T23:31:28.285+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e4109927:::obj482:head local-lis/les=117/118 n=543 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:e46d208d:::obj89:head v 111'89 2026-02-11T23:31:28.285+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e4109927:::obj482:head local-lis/les=117/118 n=543 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.285+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.285+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:e46d208d:::obj89:head, version: 111'89, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e46d208d:::obj89:head@111'89, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:28.285+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.285+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.285+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e43ffde6:::obj285:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.285+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:e43ffde6:::obj285:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 4.0 e 119/119 lb 4:e43ffde6:::obj285:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:e4109927:::obj482:head local-lis/les=117/118 n=543 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.285+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.285+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e4109927:::obj482:head local-lis/les=117/118 n=543 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:e43ffde6:::obj285:head) v3 2026-02-11T23:31:28.285+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e4109927:::obj482:head local-lis/les=117/118 n=543 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:e43ffde6:::obj285:head) 2026-02-11T23:31:28.285+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.285+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:e43ffde6:::obj285:head) v3 finish latency 0.000010 2026-02-11T23:31:28.285+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e43ffde6:::obj285:head local-lis/les=117/118 n=544 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.285+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.285+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.285+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e43ffde6:::obj285:head local-lis/les=117/118 n=544 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.285+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.285+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.285+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.285+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c9e00 con 0x507ac00 2026-02-11T23:31:28.285+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:e46d208d:::obj89:head)]) -- 0x674af00 con 0x507ac00 2026-02-11T23:31:28.285+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.285+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1129 ==== MOSDPGPush(4.0 119/117 [PushOp(4:e48b6d56:::obj389:head, version: 111'389, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e48b6d56:::obj389:head@111'389, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x554e680 con 0x507ac00 2026-02-11T23:31:28.285+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:e48b6d56:::obj389:head, version: 111'389, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e48b6d56:::obj389:head@111'389, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 119 2026-02-11T23:31:28.285+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e48b6d56:::obj389:head, version: 111'389, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e48b6d56:::obj389:head@111'389, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.285+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.285+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1130 ==== pg_backfill(progress 4.0 e 119/119 lb 4:e46d208d:::obj89:head) ==== 1061+0+0 (unknown 0 0 0) 0x5f10b00 con 0x507ac00 2026-02-11T23:31:28.285+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e48b6d56:::obj389:head, version: 111'389, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e48b6d56:::obj389:head@111'389, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.285+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:e46d208d:::obj89:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.285+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e46d208d:::obj89:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.285+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.285+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e48b6d56:::obj389:head, version: 111'389, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e48b6d56:::obj389:head@111'389, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.285+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:e48b6d56:::obj389:head, version: 111'389, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e48b6d56:::obj389:head@111'389, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000022 MOSDPGPush(4.0 119/117 [PushOp(4:e48b6d56:::obj389:head, version: 111'389, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e48b6d56:::obj389:head@111'389, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:e43ffde6:::obj285:head local-lis/les=117/118 n=544 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.285+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.285+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.285+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e43ffde6:::obj285:head local-lis/les=117/118 n=544 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:e48b6d56:::obj389:head, version: 111'389, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e48b6d56:::obj389:head@111'389, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.285+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e46d208d:::obj89:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.285+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e43ffde6:::obj285:head local-lis/les=117/118 n=544 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:e48b6d56:::obj389:head@111'389, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.285+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e43ffde6:::obj285:head local-lis/les=117/118 n=544 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:e48b6d56:::obj389:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.285+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e43ffde6:::obj285:head local-lis/les=117/118 n=544 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:e48b6d56:::obj389:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.285+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e43ffde6:::obj285:head local-lis/les=117/118 n=544 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:e48b6d56:::obj389:head 2026-02-11T23:31:28.285+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e43ffde6:::obj285:head local-lis/les=117/118 n=544 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:e48b6d56:::obj389:head v 111'389 2026-02-11T23:31:28.285+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e43ffde6:::obj285:head local-lis/les=117/118 n=544 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.285+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.286+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:e48b6d56:::obj389:head, version: 111'389, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e48b6d56:::obj389:head@111'389, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:28.286+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.286+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.286+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e46d208d:::obj89:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.286+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:e46d208d:::obj89:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 4.0 e 119/119 lb 4:e46d208d:::obj89:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:e43ffde6:::obj285:head local-lis/les=117/118 n=544 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.286+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.286+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e43ffde6:::obj285:head local-lis/les=117/118 n=544 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:e46d208d:::obj89:head) v3 2026-02-11T23:31:28.286+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e43ffde6:::obj285:head local-lis/les=117/118 n=544 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:e46d208d:::obj89:head) 2026-02-11T23:31:28.286+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.286+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:e46d208d:::obj89:head) v3 finish latency 0.000013 2026-02-11T23:31:28.286+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e46d208d:::obj89:head local-lis/les=117/118 n=545 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.286+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.286+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.286+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e46d208d:::obj89:head local-lis/les=117/118 n=545 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.286+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.286+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.286+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.286+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x674be00 con 0x507ac00 2026-02-11T23:31:28.286+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:e48b6d56:::obj389:head)]) -- 0x674b080 con 0x507ac00 2026-02-11T23:31:28.286+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.286+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1131 ==== MOSDPGPush(4.0 119/117 [PushOp(4:e5124c98:::obj133:head, version: 111'133, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e5124c98:::obj133:head@111'133, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x554e4e0 con 0x507ac00 2026-02-11T23:31:28.286+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:e5124c98:::obj133:head, version: 111'133, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e5124c98:::obj133:head@111'133, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 119 2026-02-11T23:31:28.286+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e5124c98:::obj133:head, version: 111'133, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e5124c98:::obj133:head@111'133, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.286+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.286+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1132 ==== pg_backfill(progress 4.0 e 119/119 lb 4:e48b6d56:::obj389:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f11b80 con 0x507ac00 2026-02-11T23:31:28.286+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:e48b6d56:::obj389:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.286+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e5124c98:::obj133:head, version: 111'133, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e5124c98:::obj133:head@111'133, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.286+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e48b6d56:::obj389:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.286+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.286+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e5124c98:::obj133:head, version: 111'133, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e5124c98:::obj133:head@111'133, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.286+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.286+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e48b6d56:::obj389:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.286+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:e5124c98:::obj133:head, version: 111'133, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e5124c98:::obj133:head@111'133, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 119/117 [PushOp(4:e5124c98:::obj133:head, version: 111'133, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e5124c98:::obj133:head@111'133, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:e46d208d:::obj89:head local-lis/les=117/118 n=545 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.286+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.286+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e46d208d:::obj89:head local-lis/les=117/118 n=545 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:e5124c98:::obj133:head, version: 111'133, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e5124c98:::obj133:head@111'133, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.286+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e46d208d:::obj89:head local-lis/les=117/118 n=545 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:e5124c98:::obj133:head@111'133, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.286+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e46d208d:::obj89:head local-lis/les=117/118 n=545 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:e5124c98:::obj133:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.286+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e46d208d:::obj89:head local-lis/les=117/118 n=545 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:e5124c98:::obj133:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.286+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e46d208d:::obj89:head local-lis/les=117/118 n=545 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:e5124c98:::obj133:head 2026-02-11T23:31:28.286+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e46d208d:::obj89:head local-lis/les=117/118 n=545 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:e5124c98:::obj133:head v 111'133 2026-02-11T23:31:28.286+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e46d208d:::obj89:head local-lis/les=117/118 n=545 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.286+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.286+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:e5124c98:::obj133:head, version: 111'133, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e5124c98:::obj133:head@111'133, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000064 2026-02-11T23:31:28.286+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e46d208d:::obj89:head local-lis/les=117/118 n=545 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.286+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.286+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.286+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e48b6d56:::obj389:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.286+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:e48b6d56:::obj389:head) v3 prio 15 cost 0 latency 0.000082 pg_backfill(progress 4.0 e 119/119 lb 4:e48b6d56:::obj389:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:e46d208d:::obj89:head local-lis/les=117/118 n=545 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.286+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.286+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e46d208d:::obj89:head local-lis/les=117/118 n=545 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:e48b6d56:::obj389:head) v3 2026-02-11T23:31:28.286+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e46d208d:::obj89:head local-lis/les=117/118 n=545 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:e48b6d56:::obj389:head) 2026-02-11T23:31:28.286+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.286+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:e48b6d56:::obj389:head) v3 finish latency 0.000100 2026-02-11T23:31:28.286+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.286+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e48b6d56:::obj389:head local-lis/les=117/118 n=546 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.286+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.286+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.286+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.286+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x6788000 con 0x507ac00 2026-02-11T23:31:28.286+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:e5124c98:::obj133:head)]) -- 0x671d980 con 0x507ac00 2026-02-11T23:31:28.286+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.286+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.286+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.286+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1133 ==== MOSDPGPush(4.0 119/117 [PushOp(4:e6508d05:::obj595:head, version: 111'595, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e6508d05:::obj595:head@111'595, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f1d40 con 0x507ac00 2026-02-11T23:31:28.287+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:e6508d05:::obj595:head, version: 111'595, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e6508d05:::obj595:head@111'595, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 119 2026-02-11T23:31:28.287+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e6508d05:::obj595:head, version: 111'595, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e6508d05:::obj595:head@111'595, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.287+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.287+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1134 ==== pg_backfill(progress 4.0 e 119/119 lb 4:e5124c98:::obj133:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f12100 con 0x507ac00 2026-02-11T23:31:28.287+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:e5124c98:::obj133:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.287+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e6508d05:::obj595:head, version: 111'595, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e6508d05:::obj595:head@111'595, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.287+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e5124c98:::obj133:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.287+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.287+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e6508d05:::obj595:head, version: 111'595, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e6508d05:::obj595:head@111'595, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.287+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.287+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e5124c98:::obj133:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.287+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:e6508d05:::obj595:head, version: 111'595, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e6508d05:::obj595:head@111'595, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 119/117 [PushOp(4:e6508d05:::obj595:head, version: 111'595, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e6508d05:::obj595:head@111'595, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:e48b6d56:::obj389:head local-lis/les=117/118 n=546 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.287+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.287+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e48b6d56:::obj389:head local-lis/les=117/118 n=546 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:e6508d05:::obj595:head, version: 111'595, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e6508d05:::obj595:head@111'595, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.287+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e48b6d56:::obj389:head local-lis/les=117/118 n=546 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:e6508d05:::obj595:head@111'595, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.287+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e48b6d56:::obj389:head local-lis/les=117/118 n=546 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:e6508d05:::obj595:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.287+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e48b6d56:::obj389:head local-lis/les=117/118 n=546 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:e6508d05:::obj595:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.287+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e48b6d56:::obj389:head local-lis/les=117/118 n=546 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:e6508d05:::obj595:head 2026-02-11T23:31:28.287+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e48b6d56:::obj389:head local-lis/les=117/118 n=546 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:e6508d05:::obj595:head v 111'595 2026-02-11T23:31:28.287+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e48b6d56:::obj389:head local-lis/les=117/118 n=546 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.287+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.287+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:e6508d05:::obj595:head, version: 111'595, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e6508d05:::obj595:head@111'595, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000065 2026-02-11T23:31:28.287+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e48b6d56:::obj389:head local-lis/les=117/118 n=546 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.287+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.287+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.287+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e5124c98:::obj133:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.287+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:e5124c98:::obj133:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 4.0 e 119/119 lb 4:e5124c98:::obj133:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:e48b6d56:::obj389:head local-lis/les=117/118 n=546 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.287+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.287+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e48b6d56:::obj389:head local-lis/les=117/118 n=546 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:e5124c98:::obj133:head) v3 2026-02-11T23:31:28.287+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e48b6d56:::obj389:head local-lis/les=117/118 n=546 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:e5124c98:::obj133:head) 2026-02-11T23:31:28.287+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.287+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:e5124c98:::obj133:head) v3 finish latency 0.000012 2026-02-11T23:31:28.287+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.287+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.287+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e5124c98:::obj133:head local-lis/les=117/118 n=547 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.287+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.287+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.287+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.287+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x6788480 con 0x507ac00 2026-02-11T23:31:28.287+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:e6508d05:::obj595:head)]) -- 0x6788180 con 0x507ac00 2026-02-11T23:31:28.287+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.287+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1135 ==== MOSDPGPush(4.0 119/117 [PushOp(4:e6e28dcd:::obj87:head, version: 111'87, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e6e28dcd:::obj87:head@111'87, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x56a2000 con 0x507ac00 2026-02-11T23:31:28.287+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:e6e28dcd:::obj87:head, version: 111'87, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e6e28dcd:::obj87:head@111'87, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 119 2026-02-11T23:31:28.287+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e6e28dcd:::obj87:head, version: 111'87, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e6e28dcd:::obj87:head@111'87, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.287+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.287+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1136 ==== pg_backfill(progress 4.0 e 119/119 lb 4:e6508d05:::obj595:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f12680 con 0x507ac00 2026-02-11T23:31:28.287+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:e6508d05:::obj595:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.287+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e6e28dcd:::obj87:head, version: 111'87, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e6e28dcd:::obj87:head@111'87, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.287+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e6508d05:::obj595:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.287+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.287+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e6e28dcd:::obj87:head, version: 111'87, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e6e28dcd:::obj87:head@111'87, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.287+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:e6e28dcd:::obj87:head, version: 111'87, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e6e28dcd:::obj87:head@111'87, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 119/117 [PushOp(4:e6e28dcd:::obj87:head, version: 111'87, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e6e28dcd:::obj87:head@111'87, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:e5124c98:::obj133:head local-lis/les=117/118 n=547 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.287+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.287+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e5124c98:::obj133:head local-lis/les=117/118 n=547 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:e6e28dcd:::obj87:head, version: 111'87, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e6e28dcd:::obj87:head@111'87, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.287+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.287+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e5124c98:::obj133:head local-lis/les=117/118 n=547 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:e6e28dcd:::obj87:head@111'87, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.287+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e6508d05:::obj595:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.287+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e5124c98:::obj133:head local-lis/les=117/118 n=547 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:e6e28dcd:::obj87:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.287+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e5124c98:::obj133:head local-lis/les=117/118 n=547 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:e6e28dcd:::obj87:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.287+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e5124c98:::obj133:head local-lis/les=117/118 n=547 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:e6e28dcd:::obj87:head 2026-02-11T23:31:28.287+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e5124c98:::obj133:head local-lis/les=117/118 n=547 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:e6e28dcd:::obj87:head v 111'87 2026-02-11T23:31:28.287+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e5124c98:::obj133:head local-lis/les=117/118 n=547 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.287+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.287+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:e6e28dcd:::obj87:head, version: 111'87, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e6e28dcd:::obj87:head@111'87, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:28.287+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.287+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.287+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e6508d05:::obj595:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.287+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:e6508d05:::obj595:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 4.0 e 119/119 lb 4:e6508d05:::obj595:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:e5124c98:::obj133:head local-lis/les=117/118 n=547 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.287+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.287+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e5124c98:::obj133:head local-lis/les=117/118 n=547 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:e6508d05:::obj595:head) v3 2026-02-11T23:31:28.287+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e5124c98:::obj133:head local-lis/les=117/118 n=547 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:e6508d05:::obj595:head) 2026-02-11T23:31:28.287+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.287+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:e6508d05:::obj595:head) v3 finish latency 0.000020 2026-02-11T23:31:28.287+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e6508d05:::obj595:head local-lis/les=117/118 n=548 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.287+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.287+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.287+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e6508d05:::obj595:head local-lis/les=117/118 n=548 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.287+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.287+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.287+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.287+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x6788600 con 0x507ac00 2026-02-11T23:31:28.287+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:e6e28dcd:::obj87:head)]) -- 0x674b200 con 0x507ac00 2026-02-11T23:31:28.287+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.288+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1137 ==== MOSDPGPush(4.0 119/117 [PushOp(4:e74a4ee3:::obj396:head, version: 111'396, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e74a4ee3:::obj396:head@111'396, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56aa000 con 0x507ac00 2026-02-11T23:31:28.288+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:e74a4ee3:::obj396:head, version: 111'396, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e74a4ee3:::obj396:head@111'396, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 119 2026-02-11T23:31:28.288+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e74a4ee3:::obj396:head, version: 111'396, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e74a4ee3:::obj396:head@111'396, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.288+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.288+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1138 ==== pg_backfill(progress 4.0 e 119/119 lb 4:e6e28dcd:::obj87:head) ==== 1061+0+0 (unknown 0 0 0) 0x5f12c00 con 0x507ac00 2026-02-11T23:31:28.288+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:e6e28dcd:::obj87:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.288+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e74a4ee3:::obj396:head, version: 111'396, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e74a4ee3:::obj396:head@111'396, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.288+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e6e28dcd:::obj87:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.288+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.288+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e74a4ee3:::obj396:head, version: 111'396, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e74a4ee3:::obj396:head@111'396, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.288+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:e74a4ee3:::obj396:head, version: 111'396, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e74a4ee3:::obj396:head@111'396, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 119/117 [PushOp(4:e74a4ee3:::obj396:head, version: 111'396, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e74a4ee3:::obj396:head@111'396, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:e6508d05:::obj595:head local-lis/les=117/118 n=548 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.288+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.288+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.288+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e6e28dcd:::obj87:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.288+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e6508d05:::obj595:head local-lis/les=117/118 n=548 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:e74a4ee3:::obj396:head, version: 111'396, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e74a4ee3:::obj396:head@111'396, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.288+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e6508d05:::obj595:head local-lis/les=117/118 n=548 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:e74a4ee3:::obj396:head@111'396, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.288+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e6508d05:::obj595:head local-lis/les=117/118 n=548 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:e74a4ee3:::obj396:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.288+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e6508d05:::obj595:head local-lis/les=117/118 n=548 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:e74a4ee3:::obj396:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.288+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e6508d05:::obj595:head local-lis/les=117/118 n=548 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:e74a4ee3:::obj396:head 2026-02-11T23:31:28.288+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e6508d05:::obj595:head local-lis/les=117/118 n=548 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:e74a4ee3:::obj396:head v 111'396 2026-02-11T23:31:28.288+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e6508d05:::obj595:head local-lis/les=117/118 n=548 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.288+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.288+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:e74a4ee3:::obj396:head, version: 111'396, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e74a4ee3:::obj396:head@111'396, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:28.288+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e6508d05:::obj595:head local-lis/les=117/118 n=548 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.288+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.288+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e6e28dcd:::obj87:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.288+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.288+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:e6e28dcd:::obj87:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 4.0 e 119/119 lb 4:e6e28dcd:::obj87:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:e6508d05:::obj595:head local-lis/les=117/118 n=548 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.288+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.288+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e6508d05:::obj595:head local-lis/les=117/118 n=548 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:e6e28dcd:::obj87:head) v3 2026-02-11T23:31:28.288+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e6508d05:::obj595:head local-lis/les=117/118 n=548 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:e6e28dcd:::obj87:head) 2026-02-11T23:31:28.288+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.288+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:e6e28dcd:::obj87:head) v3 finish latency 0.000020 2026-02-11T23:31:28.288+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.288+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.288+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e6e28dcd:::obj87:head local-lis/les=117/118 n=549 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.288+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.288+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.288+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.288+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x6788a80 con 0x507ac00 2026-02-11T23:31:28.288+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:e74a4ee3:::obj396:head)]) -- 0x6788780 con 0x507ac00 2026-02-11T23:31:28.288+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.288+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1139 ==== MOSDPGPush(4.0 119/117 [PushOp(4:e7b9c028:::obj510:head, version: 111'510, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e7b9c028:::obj510:head@111'510, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x555d040 con 0x507ac00 2026-02-11T23:31:28.288+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:e7b9c028:::obj510:head, version: 111'510, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e7b9c028:::obj510:head@111'510, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 119 2026-02-11T23:31:28.288+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e7b9c028:::obj510:head, version: 111'510, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e7b9c028:::obj510:head@111'510, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.288+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.288+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1140 ==== pg_backfill(progress 4.0 e 119/119 lb 4:e74a4ee3:::obj396:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f13700 con 0x507ac00 2026-02-11T23:31:28.288+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:e74a4ee3:::obj396:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 119 2026-02-11T23:31:28.288+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e7b9c028:::obj510:head, version: 111'510, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e7b9c028:::obj510:head@111'510, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.288+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e74a4ee3:::obj396:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.288+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.288+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e7b9c028:::obj510:head, version: 111'510, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e7b9c028:::obj510:head@111'510, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.288+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.288+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e74a4ee3:::obj396:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.288+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:e7b9c028:::obj510:head, version: 111'510, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e7b9c028:::obj510:head@111'510, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 119/117 [PushOp(4:e7b9c028:::obj510:head, version: 111'510, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e7b9c028:::obj510:head@111'510, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:e6e28dcd:::obj87:head local-lis/les=117/118 n=549 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.288+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.288+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e6e28dcd:::obj87:head local-lis/les=117/118 n=549 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:e7b9c028:::obj510:head, version: 111'510, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e7b9c028:::obj510:head@111'510, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.288+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e6e28dcd:::obj87:head local-lis/les=117/118 n=549 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:e7b9c028:::obj510:head@111'510, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.288+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e6e28dcd:::obj87:head local-lis/les=117/118 n=549 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:e7b9c028:::obj510:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.288+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e6e28dcd:::obj87:head local-lis/les=117/118 n=549 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:e7b9c028:::obj510:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.288+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e6e28dcd:::obj87:head local-lis/les=117/118 n=549 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:e7b9c028:::obj510:head 2026-02-11T23:31:28.288+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e6e28dcd:::obj87:head local-lis/les=117/118 n=549 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:e7b9c028:::obj510:head v 111'510 2026-02-11T23:31:28.288+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e6e28dcd:::obj87:head local-lis/les=117/118 n=549 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.288+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.288+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:e7b9c028:::obj510:head, version: 111'510, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e7b9c028:::obj510:head@111'510, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:28.288+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e6e28dcd:::obj87:head local-lis/les=117/118 n=549 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.288+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.288+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.288+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e74a4ee3:::obj396:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.288+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:e74a4ee3:::obj396:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 4.0 e 119/119 lb 4:e74a4ee3:::obj396:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:e6e28dcd:::obj87:head local-lis/les=117/118 n=549 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.288+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.288+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e6e28dcd:::obj87:head local-lis/les=117/118 n=549 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:e74a4ee3:::obj396:head) v3 2026-02-11T23:31:28.288+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e6e28dcd:::obj87:head local-lis/les=117/118 n=549 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:e74a4ee3:::obj396:head) 2026-02-11T23:31:28.288+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.288+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:e74a4ee3:::obj396:head) v3 finish latency 0.000012 2026-02-11T23:31:28.288+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.288+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.288+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e74a4ee3:::obj396:head local-lis/les=117/118 n=550 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.288+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.288+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.288+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.288+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x6788f00 con 0x507ac00 2026-02-11T23:31:28.288+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:e7b9c028:::obj510:head)]) -- 0x6788c00 con 0x507ac00 2026-02-11T23:31:28.288+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.289+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1141 ==== MOSDPGPush(4.0 119/117 [PushOp(4:e7f87738:::obj356:head, version: 111'356, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e7f87738:::obj356:head@111'356, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x554e680 con 0x507ac00 2026-02-11T23:31:28.289+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:e7f87738:::obj356:head, version: 111'356, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e7f87738:::obj356:head@111'356, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 119 2026-02-11T23:31:28.289+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e7f87738:::obj356:head, version: 111'356, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e7f87738:::obj356:head@111'356, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.289+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.289+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1142 ==== pg_backfill(progress 4.0 e 119/119 lb 4:e7b9c028:::obj510:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f10000 con 0x507ac00 2026-02-11T23:31:28.289+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:e7b9c028:::obj510:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.289+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e7f87738:::obj356:head, version: 111'356, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e7f87738:::obj356:head@111'356, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.289+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e7b9c028:::obj510:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.289+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.289+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e7f87738:::obj356:head, version: 111'356, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e7f87738:::obj356:head@111'356, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.289+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:e7f87738:::obj356:head, version: 111'356, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e7f87738:::obj356:head@111'356, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(4.0 119/117 [PushOp(4:e7f87738:::obj356:head, version: 111'356, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e7f87738:::obj356:head@111'356, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:e74a4ee3:::obj396:head local-lis/les=117/118 n=550 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.289+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.289+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.289+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e74a4ee3:::obj396:head local-lis/les=117/118 n=550 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:e7f87738:::obj356:head, version: 111'356, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e7f87738:::obj356:head@111'356, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.289+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e7b9c028:::obj510:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.289+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e74a4ee3:::obj396:head local-lis/les=117/118 n=550 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:e7f87738:::obj356:head@111'356, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.289+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e74a4ee3:::obj396:head local-lis/les=117/118 n=550 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:e7f87738:::obj356:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.289+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e74a4ee3:::obj396:head local-lis/les=117/118 n=550 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:e7f87738:::obj356:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.289+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e74a4ee3:::obj396:head local-lis/les=117/118 n=550 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:e7f87738:::obj356:head 2026-02-11T23:31:28.289+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e74a4ee3:::obj396:head local-lis/les=117/118 n=550 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:e7f87738:::obj356:head v 111'356 2026-02-11T23:31:28.289+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e74a4ee3:::obj396:head local-lis/les=117/118 n=550 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.289+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.289+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:e7f87738:::obj356:head, version: 111'356, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e7f87738:::obj356:head@111'356, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:28.289+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e74a4ee3:::obj396:head local-lis/les=117/118 n=550 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.289+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.289+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.289+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e7b9c028:::obj510:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.289+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:e7b9c028:::obj510:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 4.0 e 119/119 lb 4:e7b9c028:::obj510:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:e74a4ee3:::obj396:head local-lis/les=117/118 n=550 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.289+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.289+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e74a4ee3:::obj396:head local-lis/les=117/118 n=550 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:e7b9c028:::obj510:head) v3 2026-02-11T23:31:28.289+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e74a4ee3:::obj396:head local-lis/les=117/118 n=550 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:e7b9c028:::obj510:head) 2026-02-11T23:31:28.289+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.289+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:e7b9c028:::obj510:head) v3 finish latency 0.000015 2026-02-11T23:31:28.289+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.289+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.289+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e7b9c028:::obj510:head local-lis/les=117/118 n=551 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.289+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.289+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.289+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.289+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x6789380 con 0x507ac00 2026-02-11T23:31:28.289+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:e7f87738:::obj356:head)]) -- 0x6789080 con 0x507ac00 2026-02-11T23:31:28.289+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.289+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1143 ==== MOSDPGPush(4.0 119/117 [PushOp(4:e8366861:::obj250:head, version: 111'250, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e8366861:::obj250:head@111'250, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3815520 con 0x507ac00 2026-02-11T23:31:28.289+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:e8366861:::obj250:head, version: 111'250, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e8366861:::obj250:head@111'250, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 119 2026-02-11T23:31:28.289+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e8366861:::obj250:head, version: 111'250, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e8366861:::obj250:head@111'250, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.289+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.289+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1144 ==== pg_backfill(progress 4.0 e 119/119 lb 4:e7f87738:::obj356:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f10580 con 0x507ac00 2026-02-11T23:31:28.289+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:e7f87738:::obj356:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.289+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e8366861:::obj250:head, version: 111'250, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e8366861:::obj250:head@111'250, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.289+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e7f87738:::obj356:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.289+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.289+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e8366861:::obj250:head, version: 111'250, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e8366861:::obj250:head@111'250, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.289+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.289+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e7f87738:::obj356:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.289+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:e8366861:::obj250:head, version: 111'250, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e8366861:::obj250:head@111'250, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 119/117 [PushOp(4:e8366861:::obj250:head, version: 111'250, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e8366861:::obj250:head@111'250, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:e7b9c028:::obj510:head local-lis/les=117/118 n=551 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.289+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.289+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e7b9c028:::obj510:head local-lis/les=117/118 n=551 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:e8366861:::obj250:head, version: 111'250, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e8366861:::obj250:head@111'250, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.289+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e7b9c028:::obj510:head local-lis/les=117/118 n=551 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:e8366861:::obj250:head@111'250, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.289+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e7b9c028:::obj510:head local-lis/les=117/118 n=551 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:e8366861:::obj250:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.289+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e7b9c028:::obj510:head local-lis/les=117/118 n=551 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:e8366861:::obj250:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.289+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e7b9c028:::obj510:head local-lis/les=117/118 n=551 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:e8366861:::obj250:head 2026-02-11T23:31:28.289+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e7b9c028:::obj510:head local-lis/les=117/118 n=551 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:e8366861:::obj250:head v 111'250 2026-02-11T23:31:28.289+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e7b9c028:::obj510:head local-lis/les=117/118 n=551 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.289+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.289+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:e8366861:::obj250:head, version: 111'250, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e8366861:::obj250:head@111'250, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:28.289+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.289+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.289+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e7f87738:::obj356:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.289+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:e7f87738:::obj356:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 4.0 e 119/119 lb 4:e7f87738:::obj356:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:e7b9c028:::obj510:head local-lis/les=117/118 n=551 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.289+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.289+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e7b9c028:::obj510:head local-lis/les=117/118 n=551 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:e7f87738:::obj356:head) v3 2026-02-11T23:31:28.289+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e7b9c028:::obj510:head local-lis/les=117/118 n=551 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:e7f87738:::obj356:head) 2026-02-11T23:31:28.289+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.289+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:e7f87738:::obj356:head) v3 finish latency 0.000013 2026-02-11T23:31:28.289+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e7f87738:::obj356:head local-lis/les=117/118 n=552 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.289+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.289+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.289+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e7f87738:::obj356:head local-lis/les=117/118 n=552 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.289+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.289+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.289+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.289+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x6789500 con 0x507ac00 2026-02-11T23:31:28.289+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:e8366861:::obj250:head)]) -- 0x6767b00 con 0x507ac00 2026-02-11T23:31:28.289+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.290+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1145 ==== MOSDPGPush(4.0 119/117 [PushOp(4:e9ac1484:::obj417:head, version: 111'417, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e9ac1484:::obj417:head@111'417, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c340 con 0x507ac00 2026-02-11T23:31:28.290+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:e9ac1484:::obj417:head, version: 111'417, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e9ac1484:::obj417:head@111'417, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 119 2026-02-11T23:31:28.290+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e9ac1484:::obj417:head, version: 111'417, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e9ac1484:::obj417:head@111'417, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.290+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.290+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1146 ==== pg_backfill(progress 4.0 e 119/119 lb 4:e8366861:::obj250:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f12100 con 0x507ac00 2026-02-11T23:31:28.290+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e9ac1484:::obj417:head, version: 111'417, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e9ac1484:::obj417:head@111'417, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.290+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:e8366861:::obj250:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.290+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e8366861:::obj250:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.290+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.290+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:e9ac1484:::obj417:head, version: 111'417, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e9ac1484:::obj417:head@111'417, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.290+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:e9ac1484:::obj417:head, version: 111'417, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e9ac1484:::obj417:head@111'417, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 119/117 [PushOp(4:e9ac1484:::obj417:head, version: 111'417, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e9ac1484:::obj417:head@111'417, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:e7f87738:::obj356:head local-lis/les=117/118 n=552 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.290+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.290+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.290+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e8366861:::obj250:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.290+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e7f87738:::obj356:head local-lis/les=117/118 n=552 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:e9ac1484:::obj417:head, version: 111'417, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e9ac1484:::obj417:head@111'417, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.290+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e7f87738:::obj356:head local-lis/les=117/118 n=552 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:e9ac1484:::obj417:head@111'417, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.290+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e7f87738:::obj356:head local-lis/les=117/118 n=552 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:e9ac1484:::obj417:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.290+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e7f87738:::obj356:head local-lis/les=117/118 n=552 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:e9ac1484:::obj417:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.290+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e7f87738:::obj356:head local-lis/les=117/118 n=552 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:e9ac1484:::obj417:head 2026-02-11T23:31:28.290+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e7f87738:::obj356:head local-lis/les=117/118 n=552 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:e9ac1484:::obj417:head v 111'417 2026-02-11T23:31:28.290+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e7f87738:::obj356:head local-lis/les=117/118 n=552 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.290+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.290+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:e9ac1484:::obj417:head, version: 111'417, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e9ac1484:::obj417:head@111'417, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:28.290+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.290+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.290+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e8366861:::obj250:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.290+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:e8366861:::obj250:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 4.0 e 119/119 lb 4:e8366861:::obj250:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:e7f87738:::obj356:head local-lis/les=117/118 n=552 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.290+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.290+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e7f87738:::obj356:head local-lis/les=117/118 n=552 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:e8366861:::obj250:head) v3 2026-02-11T23:31:28.290+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e7f87738:::obj356:head local-lis/les=117/118 n=552 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:e8366861:::obj250:head) 2026-02-11T23:31:28.290+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.290+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:e8366861:::obj250:head) v3 finish latency 0.000013 2026-02-11T23:31:28.290+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e8366861:::obj250:head local-lis/les=117/118 n=553 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.290+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.290+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.290+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e8366861:::obj250:head local-lis/les=117/118 n=553 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.290+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.290+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.290+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.290+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x6789680 con 0x507ac00 2026-02-11T23:31:28.290+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:e9ac1484:::obj417:head)]) -- 0x6767800 con 0x507ac00 2026-02-11T23:31:28.290+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.290+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1147 ==== MOSDPGPush(4.0 119/117 [PushOp(4:ea485012:::obj312:head, version: 111'312, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ea485012:::obj312:head@111'312, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56caea0 con 0x507ac00 2026-02-11T23:31:28.290+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:ea485012:::obj312:head, version: 111'312, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ea485012:::obj312:head@111'312, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 119 2026-02-11T23:31:28.290+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ea485012:::obj312:head, version: 111'312, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ea485012:::obj312:head@111'312, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.290+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.290+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1148 ==== pg_backfill(progress 4.0 e 119/119 lb 4:e9ac1484:::obj417:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f10b00 con 0x507ac00 2026-02-11T23:31:28.290+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:e9ac1484:::obj417:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.290+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ea485012:::obj312:head, version: 111'312, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ea485012:::obj312:head@111'312, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.290+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e9ac1484:::obj417:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.290+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.290+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ea485012:::obj312:head, version: 111'312, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ea485012:::obj312:head@111'312, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.290+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:ea485012:::obj312:head, version: 111'312, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ea485012:::obj312:head@111'312, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 119/117 [PushOp(4:ea485012:::obj312:head, version: 111'312, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ea485012:::obj312:head@111'312, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:e8366861:::obj250:head local-lis/les=117/118 n=553 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.290+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.290+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.290+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e9ac1484:::obj417:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.290+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e8366861:::obj250:head local-lis/les=117/118 n=553 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:ea485012:::obj312:head, version: 111'312, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ea485012:::obj312:head@111'312, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.290+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e8366861:::obj250:head local-lis/les=117/118 n=553 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:ea485012:::obj312:head@111'312, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.290+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e8366861:::obj250:head local-lis/les=117/118 n=553 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:ea485012:::obj312:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.290+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e8366861:::obj250:head local-lis/les=117/118 n=553 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:ea485012:::obj312:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.290+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e8366861:::obj250:head local-lis/les=117/118 n=553 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:ea485012:::obj312:head 2026-02-11T23:31:28.290+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e8366861:::obj250:head local-lis/les=117/118 n=553 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:ea485012:::obj312:head v 111'312 2026-02-11T23:31:28.290+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e8366861:::obj250:head local-lis/les=117/118 n=553 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.290+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.290+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:ea485012:::obj312:head, version: 111'312, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ea485012:::obj312:head@111'312, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:28.290+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.290+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.290+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:e9ac1484:::obj417:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.290+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:e9ac1484:::obj417:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 4.0 e 119/119 lb 4:e9ac1484:::obj417:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:e8366861:::obj250:head local-lis/les=117/118 n=553 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.290+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.290+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e8366861:::obj250:head local-lis/les=117/118 n=553 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:e9ac1484:::obj417:head) v3 2026-02-11T23:31:28.290+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e8366861:::obj250:head local-lis/les=117/118 n=553 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:e9ac1484:::obj417:head) 2026-02-11T23:31:28.290+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.290+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:e9ac1484:::obj417:head) v3 finish latency 0.000013 2026-02-11T23:31:28.290+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e9ac1484:::obj417:head local-lis/les=117/118 n=554 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.290+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.291+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e9ac1484:::obj417:head local-lis/les=117/118 n=554 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.291+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.291+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.291+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.291+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x6789800 con 0x507ac00 2026-02-11T23:31:28.291+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.291+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:ea485012:::obj312:head)]) -- 0x674a900 con 0x507ac00 2026-02-11T23:31:28.291+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.291+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1149 ==== MOSDPGPush(4.0 119/117 [PushOp(4:ea54a43e:::obj502:head, version: 111'502, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ea54a43e:::obj502:head@111'502, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56b64e0 con 0x507ac00 2026-02-11T23:31:28.291+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:ea54a43e:::obj502:head, version: 111'502, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ea54a43e:::obj502:head@111'502, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.291+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ea54a43e:::obj502:head, version: 111'502, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ea54a43e:::obj502:head@111'502, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.291+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.291+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1150 ==== pg_backfill(progress 4.0 e 119/119 lb 4:ea485012:::obj312:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f13180 con 0x507ac00 2026-02-11T23:31:28.291+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:ea485012:::obj312:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.291+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ea54a43e:::obj502:head, version: 111'502, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ea54a43e:::obj502:head@111'502, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.291+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:ea485012:::obj312:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.291+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.291+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ea54a43e:::obj502:head, version: 111'502, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ea54a43e:::obj502:head@111'502, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.291+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:ea54a43e:::obj502:head, version: 111'502, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ea54a43e:::obj502:head@111'502, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 119/117 [PushOp(4:ea54a43e:::obj502:head, version: 111'502, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ea54a43e:::obj502:head@111'502, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:e9ac1484:::obj417:head local-lis/les=117/118 n=554 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.291+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.291+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.291+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e9ac1484:::obj417:head local-lis/les=117/118 n=554 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:ea54a43e:::obj502:head, version: 111'502, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ea54a43e:::obj502:head@111'502, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.291+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:ea485012:::obj312:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.291+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e9ac1484:::obj417:head local-lis/les=117/118 n=554 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:ea54a43e:::obj502:head@111'502, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.291+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e9ac1484:::obj417:head local-lis/les=117/118 n=554 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:ea54a43e:::obj502:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.291+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e9ac1484:::obj417:head local-lis/les=117/118 n=554 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:ea54a43e:::obj502:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.291+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e9ac1484:::obj417:head local-lis/les=117/118 n=554 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:ea54a43e:::obj502:head 2026-02-11T23:31:28.291+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e9ac1484:::obj417:head local-lis/les=117/118 n=554 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:ea54a43e:::obj502:head v 111'502 2026-02-11T23:31:28.291+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e9ac1484:::obj417:head local-lis/les=117/118 n=554 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.291+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.291+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:ea54a43e:::obj502:head, version: 111'502, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ea54a43e:::obj502:head@111'502, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:28.291+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.291+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.291+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:ea485012:::obj312:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.291+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:ea485012:::obj312:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 4.0 e 119/119 lb 4:ea485012:::obj312:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:e9ac1484:::obj417:head local-lis/les=117/118 n=554 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.291+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.291+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e9ac1484:::obj417:head local-lis/les=117/118 n=554 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:ea485012:::obj312:head) v3 2026-02-11T23:31:28.291+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:e9ac1484:::obj417:head local-lis/les=117/118 n=554 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:ea485012:::obj312:head) 2026-02-11T23:31:28.291+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.291+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:ea485012:::obj312:head) v3 finish latency 0.000018 2026-02-11T23:31:28.291+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ea485012:::obj312:head local-lis/les=117/118 n=555 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.291+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.291+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.291+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ea485012:::obj312:head local-lis/les=117/118 n=555 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.291+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.291+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.291+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.291+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x6789980 con 0x507ac00 2026-02-11T23:31:28.291+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:ea54a43e:::obj502:head)]) -- 0x5f2dc80 con 0x507ac00 2026-02-11T23:31:28.291+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.291+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1151 ==== MOSDPGPush(4.0 119/117 [PushOp(4:eaf65c1c:::obj380:head, version: 111'380, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eaf65c1c:::obj380:head@111'380, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f0d00 con 0x507ac00 2026-02-11T23:31:28.291+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:eaf65c1c:::obj380:head, version: 111'380, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eaf65c1c:::obj380:head@111'380, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 119 2026-02-11T23:31:28.291+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:eaf65c1c:::obj380:head, version: 111'380, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eaf65c1c:::obj380:head@111'380, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.291+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.291+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1152 ==== pg_backfill(progress 4.0 e 119/119 lb 4:ea54a43e:::obj502:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f11080 con 0x507ac00 2026-02-11T23:31:28.291+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:ea54a43e:::obj502:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.291+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:eaf65c1c:::obj380:head, version: 111'380, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eaf65c1c:::obj380:head@111'380, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.291+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:ea54a43e:::obj502:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.291+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.291+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:eaf65c1c:::obj380:head, version: 111'380, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eaf65c1c:::obj380:head@111'380, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.291+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.291+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:eaf65c1c:::obj380:head, version: 111'380, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eaf65c1c:::obj380:head@111'380, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 119/117 [PushOp(4:eaf65c1c:::obj380:head, version: 111'380, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eaf65c1c:::obj380:head@111'380, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:ea485012:::obj312:head local-lis/les=117/118 n=555 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.291+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.291+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:ea54a43e:::obj502:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.291+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ea485012:::obj312:head local-lis/les=117/118 n=555 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:eaf65c1c:::obj380:head, version: 111'380, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eaf65c1c:::obj380:head@111'380, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.291+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ea485012:::obj312:head local-lis/les=117/118 n=555 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:eaf65c1c:::obj380:head@111'380, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.291+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ea485012:::obj312:head local-lis/les=117/118 n=555 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:eaf65c1c:::obj380:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.291+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ea485012:::obj312:head local-lis/les=117/118 n=555 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:eaf65c1c:::obj380:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.291+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ea485012:::obj312:head local-lis/les=117/118 n=555 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:eaf65c1c:::obj380:head 2026-02-11T23:31:28.291+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ea485012:::obj312:head local-lis/les=117/118 n=555 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:eaf65c1c:::obj380:head v 111'380 2026-02-11T23:31:28.291+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ea485012:::obj312:head local-lis/les=117/118 n=555 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.291+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.291+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:eaf65c1c:::obj380:head, version: 111'380, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eaf65c1c:::obj380:head@111'380, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:28.292+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.292+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.292+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:ea54a43e:::obj502:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.292+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:ea54a43e:::obj502:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 4.0 e 119/119 lb 4:ea54a43e:::obj502:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:ea485012:::obj312:head local-lis/les=117/118 n=555 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.292+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.292+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ea485012:::obj312:head local-lis/les=117/118 n=555 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:ea54a43e:::obj502:head) v3 2026-02-11T23:31:28.292+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ea485012:::obj312:head local-lis/les=117/118 n=555 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:ea54a43e:::obj502:head) 2026-02-11T23:31:28.292+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.292+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:ea54a43e:::obj502:head) v3 finish latency 0.000012 2026-02-11T23:31:28.292+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ea54a43e:::obj502:head local-lis/les=117/118 n=556 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.292+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.292+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.292+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ea54a43e:::obj502:head local-lis/les=117/118 n=556 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.292+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.292+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.292+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.292+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x6789b00 con 0x507ac00 2026-02-11T23:31:28.292+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:eaf65c1c:::obj380:head)]) -- 0x6767500 con 0x507ac00 2026-02-11T23:31:28.292+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.292+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1153 ==== MOSDPGPush(4.0 119/117 [PushOp(4:eb0c167c:::obj188:head, version: 111'188, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb0c167c:::obj188:head@111'188, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5647380 con 0x507ac00 2026-02-11T23:31:28.292+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:eb0c167c:::obj188:head, version: 111'188, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb0c167c:::obj188:head@111'188, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 119 2026-02-11T23:31:28.292+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:eb0c167c:::obj188:head, version: 111'188, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb0c167c:::obj188:head@111'188, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.292+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.292+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1154 ==== pg_backfill(progress 4.0 e 119/119 lb 4:eaf65c1c:::obj380:head) ==== 1062+0+0 (unknown 0 0 0) 0x5f11600 con 0x507ac00 2026-02-11T23:31:28.292+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:eaf65c1c:::obj380:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.292+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:eb0c167c:::obj188:head, version: 111'188, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb0c167c:::obj188:head@111'188, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.292+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:eaf65c1c:::obj380:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.292+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.292+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:eb0c167c:::obj188:head, version: 111'188, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb0c167c:::obj188:head@111'188, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.292+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:eb0c167c:::obj188:head, version: 111'188, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb0c167c:::obj188:head@111'188, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 119/117 [PushOp(4:eb0c167c:::obj188:head, version: 111'188, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb0c167c:::obj188:head@111'188, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:ea54a43e:::obj502:head local-lis/les=117/118 n=556 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.292+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.292+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.292+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:eaf65c1c:::obj380:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.292+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ea54a43e:::obj502:head local-lis/les=117/118 n=556 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:eb0c167c:::obj188:head, version: 111'188, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb0c167c:::obj188:head@111'188, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.292+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ea54a43e:::obj502:head local-lis/les=117/118 n=556 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:eb0c167c:::obj188:head@111'188, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.292+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ea54a43e:::obj502:head local-lis/les=117/118 n=556 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:eb0c167c:::obj188:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.292+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ea54a43e:::obj502:head local-lis/les=117/118 n=556 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:eb0c167c:::obj188:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.292+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ea54a43e:::obj502:head local-lis/les=117/118 n=556 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:eb0c167c:::obj188:head 2026-02-11T23:31:28.292+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ea54a43e:::obj502:head local-lis/les=117/118 n=556 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:eb0c167c:::obj188:head v 111'188 2026-02-11T23:31:28.292+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ea54a43e:::obj502:head local-lis/les=117/118 n=556 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.292+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.292+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:eb0c167c:::obj188:head, version: 111'188, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb0c167c:::obj188:head@111'188, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:28.292+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.292+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.292+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:eaf65c1c:::obj380:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.292+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:eaf65c1c:::obj380:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 4.0 e 119/119 lb 4:eaf65c1c:::obj380:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:ea54a43e:::obj502:head local-lis/les=117/118 n=556 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.292+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.292+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ea54a43e:::obj502:head local-lis/les=117/118 n=556 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:eaf65c1c:::obj380:head) v3 2026-02-11T23:31:28.292+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ea54a43e:::obj502:head local-lis/les=117/118 n=556 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:eaf65c1c:::obj380:head) 2026-02-11T23:31:28.292+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.292+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:eaf65c1c:::obj380:head) v3 finish latency 0.000019 2026-02-11T23:31:28.292+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eaf65c1c:::obj380:head local-lis/les=117/118 n=557 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.292+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.292+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.292+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eaf65c1c:::obj380:head local-lis/les=117/118 n=557 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.292+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.292+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.292+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.292+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x6789c80 con 0x507ac00 2026-02-11T23:31:28.292+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:eb0c167c:::obj188:head)]) -- 0x6767200 con 0x507ac00 2026-02-11T23:31:28.292+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.292+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1155 ==== MOSDPGPush(4.0 119/117 [PushOp(4:eb29d295:::obj554:head, version: 111'554, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb29d295:::obj554:head@111'554, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5697ba0 con 0x507ac00 2026-02-11T23:31:28.292+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:eb29d295:::obj554:head, version: 111'554, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb29d295:::obj554:head@111'554, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 119 2026-02-11T23:31:28.292+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:eb29d295:::obj554:head, version: 111'554, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb29d295:::obj554:head@111'554, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.293+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.293+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:eb29d295:::obj554:head, version: 111'554, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb29d295:::obj554:head@111'554, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.293+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.293+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:eb29d295:::obj554:head, version: 111'554, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb29d295:::obj554:head@111'554, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.293+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:eb29d295:::obj554:head, version: 111'554, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb29d295:::obj554:head@111'554, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 119/117 [PushOp(4:eb29d295:::obj554:head, version: 111'554, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb29d295:::obj554:head@111'554, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:eaf65c1c:::obj380:head local-lis/les=117/118 n=557 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.293+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.293+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.293+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1156 ==== pg_backfill(progress 4.0 e 119/119 lb 4:eb0c167c:::obj188:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2f700 con 0x507ac00 2026-02-11T23:31:28.293+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eaf65c1c:::obj380:head local-lis/les=117/118 n=557 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:eb29d295:::obj554:head, version: 111'554, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb29d295:::obj554:head@111'554, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.293+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:eb0c167c:::obj188:head) v3 prio 15 type 95 cost 0 latency 0.000010 epoch 119 2026-02-11T23:31:28.293+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eaf65c1c:::obj380:head local-lis/les=117/118 n=557 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:eb29d295:::obj554:head@111'554, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.293+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:eb0c167c:::obj188:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.293+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eaf65c1c:::obj380:head local-lis/les=117/118 n=557 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:eb29d295:::obj554:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.293+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.293+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:eb0c167c:::obj188:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.293+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eaf65c1c:::obj380:head local-lis/les=117/118 n=557 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:eb29d295:::obj554:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.293+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eaf65c1c:::obj380:head local-lis/les=117/118 n=557 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:eb29d295:::obj554:head 2026-02-11T23:31:28.293+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eaf65c1c:::obj380:head local-lis/les=117/118 n=557 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:eb29d295:::obj554:head v 111'554 2026-02-11T23:31:28.293+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eaf65c1c:::obj380:head local-lis/les=117/118 n=557 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.293+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.293+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:eb29d295:::obj554:head, version: 111'554, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb29d295:::obj554:head@111'554, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:28.293+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.293+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.293+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:eb0c167c:::obj188:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.293+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:eb0c167c:::obj188:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 4.0 e 119/119 lb 4:eb0c167c:::obj188:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:eaf65c1c:::obj380:head local-lis/les=117/118 n=557 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.293+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.293+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eaf65c1c:::obj380:head local-lis/les=117/118 n=557 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:eb0c167c:::obj188:head) v3 2026-02-11T23:31:28.293+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eaf65c1c:::obj380:head local-lis/les=117/118 n=557 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:eb0c167c:::obj188:head) 2026-02-11T23:31:28.293+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.293+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:eb0c167c:::obj188:head) v3 finish latency 0.000016 2026-02-11T23:31:28.293+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eb0c167c:::obj188:head local-lis/les=117/118 n=558 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.293+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.293+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.293+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eb0c167c:::obj188:head local-lis/les=117/118 n=558 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.293+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.293+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.293+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.293+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x6789e00 con 0x507ac00 2026-02-11T23:31:28.293+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:eb29d295:::obj554:head)]) -- 0x6767080 con 0x507ac00 2026-02-11T23:31:28.293+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.293+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1157 ==== MOSDPGPush(4.0 119/117 [PushOp(4:eb806c57:::obj373:head, version: 111'373, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb806c57:::obj373:head@111'373, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5617380 con 0x507ac00 2026-02-11T23:31:28.293+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:eb806c57:::obj373:head, version: 111'373, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb806c57:::obj373:head@111'373, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 119 2026-02-11T23:31:28.293+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:eb806c57:::obj373:head, version: 111'373, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb806c57:::obj373:head@111'373, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.293+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.293+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1158 ==== pg_backfill(progress 4.0 e 119/119 lb 4:eb29d295:::obj554:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2d080 con 0x507ac00 2026-02-11T23:31:28.293+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:eb806c57:::obj373:head, version: 111'373, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb806c57:::obj373:head@111'373, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.293+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:eb29d295:::obj554:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.293+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:eb29d295:::obj554:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.293+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.293+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:eb806c57:::obj373:head, version: 111'373, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb806c57:::obj373:head@111'373, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.293+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:eb806c57:::obj373:head, version: 111'373, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb806c57:::obj373:head@111'373, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000022 MOSDPGPush(4.0 119/117 [PushOp(4:eb806c57:::obj373:head, version: 111'373, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb806c57:::obj373:head@111'373, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:eb0c167c:::obj188:head local-lis/les=117/118 n=558 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.293+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.293+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.293+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eb0c167c:::obj188:head local-lis/les=117/118 n=558 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:eb806c57:::obj373:head, version: 111'373, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb806c57:::obj373:head@111'373, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.293+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:eb29d295:::obj554:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.293+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eb0c167c:::obj188:head local-lis/les=117/118 n=558 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:eb806c57:::obj373:head@111'373, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.293+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eb0c167c:::obj188:head local-lis/les=117/118 n=558 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:eb806c57:::obj373:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.293+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eb0c167c:::obj188:head local-lis/les=117/118 n=558 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:eb806c57:::obj373:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.293+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eb0c167c:::obj188:head local-lis/les=117/118 n=558 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:eb806c57:::obj373:head 2026-02-11T23:31:28.293+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eb0c167c:::obj188:head local-lis/les=117/118 n=558 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:eb806c57:::obj373:head v 111'373 2026-02-11T23:31:28.293+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eb0c167c:::obj188:head local-lis/les=117/118 n=558 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.293+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.293+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:eb806c57:::obj373:head, version: 111'373, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb806c57:::obj373:head@111'373, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:28.293+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.293+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.293+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:eb29d295:::obj554:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.293+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:eb29d295:::obj554:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 4.0 e 119/119 lb 4:eb29d295:::obj554:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:eb0c167c:::obj188:head local-lis/les=117/118 n=558 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.293+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.293+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eb0c167c:::obj188:head local-lis/les=117/118 n=558 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:eb29d295:::obj554:head) v3 2026-02-11T23:31:28.293+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eb0c167c:::obj188:head local-lis/les=117/118 n=558 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:eb29d295:::obj554:head) 2026-02-11T23:31:28.293+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.293+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:eb29d295:::obj554:head) v3 finish latency 0.000020 2026-02-11T23:31:28.293+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eb29d295:::obj554:head local-lis/les=117/118 n=559 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.293+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.293+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.293+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eb29d295:::obj554:head local-lis/les=117/118 n=559 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.293+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.293+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.293+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.293+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x6788180 con 0x507ac00 2026-02-11T23:31:28.293+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:eb806c57:::obj373:head)]) -- 0x6766d80 con 0x507ac00 2026-02-11T23:31:28.293+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.294+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1159 ==== MOSDPGPush(4.0 119/117 [PushOp(4:ecbe4ab7:::obj125:head, version: 111'125, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ecbe4ab7:::obj125:head@111'125, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f0b60 con 0x507ac00 2026-02-11T23:31:28.294+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:ecbe4ab7:::obj125:head, version: 111'125, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ecbe4ab7:::obj125:head@111'125, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 119 2026-02-11T23:31:28.294+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ecbe4ab7:::obj125:head, version: 111'125, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ecbe4ab7:::obj125:head@111'125, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.294+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.294+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1160 ==== pg_backfill(progress 4.0 e 119/119 lb 4:eb806c57:::obj373:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e680 con 0x507ac00 2026-02-11T23:31:28.294+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ecbe4ab7:::obj125:head, version: 111'125, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ecbe4ab7:::obj125:head@111'125, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.294+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:eb806c57:::obj373:head) v3 prio 15 type 95 cost 0 latency 0.000002 epoch 119 2026-02-11T23:31:28.294+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:eb806c57:::obj373:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.294+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.294+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ecbe4ab7:::obj125:head, version: 111'125, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ecbe4ab7:::obj125:head@111'125, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.294+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:ecbe4ab7:::obj125:head, version: 111'125, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ecbe4ab7:::obj125:head@111'125, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(4.0 119/117 [PushOp(4:ecbe4ab7:::obj125:head, version: 111'125, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ecbe4ab7:::obj125:head@111'125, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:eb29d295:::obj554:head local-lis/les=117/118 n=559 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.294+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.294+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.294+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:eb806c57:::obj373:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.294+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eb29d295:::obj554:head local-lis/les=117/118 n=559 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:ecbe4ab7:::obj125:head, version: 111'125, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ecbe4ab7:::obj125:head@111'125, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.294+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eb29d295:::obj554:head local-lis/les=117/118 n=559 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:ecbe4ab7:::obj125:head@111'125, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.294+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eb29d295:::obj554:head local-lis/les=117/118 n=559 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:ecbe4ab7:::obj125:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.294+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eb29d295:::obj554:head local-lis/les=117/118 n=559 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:ecbe4ab7:::obj125:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.294+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eb29d295:::obj554:head local-lis/les=117/118 n=559 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:ecbe4ab7:::obj125:head 2026-02-11T23:31:28.294+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eb29d295:::obj554:head local-lis/les=117/118 n=559 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:ecbe4ab7:::obj125:head v 111'125 2026-02-11T23:31:28.294+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eb29d295:::obj554:head local-lis/les=117/118 n=559 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.294+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.294+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:ecbe4ab7:::obj125:head, version: 111'125, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ecbe4ab7:::obj125:head@111'125, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:28.294+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.294+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.294+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:eb806c57:::obj373:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.294+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:eb806c57:::obj373:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 4.0 e 119/119 lb 4:eb806c57:::obj373:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:eb29d295:::obj554:head local-lis/les=117/118 n=559 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.294+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.294+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eb29d295:::obj554:head local-lis/les=117/118 n=559 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:eb806c57:::obj373:head) v3 2026-02-11T23:31:28.294+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eb29d295:::obj554:head local-lis/les=117/118 n=559 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:eb806c57:::obj373:head) 2026-02-11T23:31:28.294+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.294+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:eb806c57:::obj373:head) v3 finish latency 0.000019 2026-02-11T23:31:28.294+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eb806c57:::obj373:head local-lis/les=117/118 n=560 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.294+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.294+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.294+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eb806c57:::obj373:head local-lis/les=117/118 n=560 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.294+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.294+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.294+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.294+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x6788480 con 0x507ac00 2026-02-11T23:31:28.294+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:ecbe4ab7:::obj125:head)]) -- 0x6766a80 con 0x507ac00 2026-02-11T23:31:28.294+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.294+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1161 ==== MOSDPGPush(4.0 119/117 [PushOp(4:ed234181:::obj472:head, version: 111'472, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ed234181:::obj472:head@111'472, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f1520 con 0x507ac00 2026-02-11T23:31:28.294+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:ed234181:::obj472:head, version: 111'472, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ed234181:::obj472:head@111'472, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 119 2026-02-11T23:31:28.294+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ed234181:::obj472:head, version: 111'472, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ed234181:::obj472:head@111'472, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.294+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.294+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1162 ==== pg_backfill(progress 4.0 e 119/119 lb 4:ecbe4ab7:::obj125:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176580 con 0x507ac00 2026-02-11T23:31:28.294+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:ecbe4ab7:::obj125:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.294+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:ecbe4ab7:::obj125:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.294+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ed234181:::obj472:head, version: 111'472, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ed234181:::obj472:head@111'472, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.294+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.294+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ed234181:::obj472:head, version: 111'472, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ed234181:::obj472:head@111'472, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.294+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.294+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:ed234181:::obj472:head, version: 111'472, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ed234181:::obj472:head@111'472, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 119/117 [PushOp(4:ed234181:::obj472:head, version: 111'472, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ed234181:::obj472:head@111'472, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:eb806c57:::obj373:head local-lis/les=117/118 n=560 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.294+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:ecbe4ab7:::obj125:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.294+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.294+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eb806c57:::obj373:head local-lis/les=117/118 n=560 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:ed234181:::obj472:head, version: 111'472, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ed234181:::obj472:head@111'472, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.294+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eb806c57:::obj373:head local-lis/les=117/118 n=560 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:ed234181:::obj472:head@111'472, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.294+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eb806c57:::obj373:head local-lis/les=117/118 n=560 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:ed234181:::obj472:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.294+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eb806c57:::obj373:head local-lis/les=117/118 n=560 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:ed234181:::obj472:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.294+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eb806c57:::obj373:head local-lis/les=117/118 n=560 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:ed234181:::obj472:head 2026-02-11T23:31:28.294+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eb806c57:::obj373:head local-lis/les=117/118 n=560 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:ed234181:::obj472:head v 111'472 2026-02-11T23:31:28.294+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eb806c57:::obj373:head local-lis/les=117/118 n=560 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.294+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.294+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:ed234181:::obj472:head, version: 111'472, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ed234181:::obj472:head@111'472, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:28.294+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.294+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.294+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:ecbe4ab7:::obj125:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.294+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:ecbe4ab7:::obj125:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 4.0 e 119/119 lb 4:ecbe4ab7:::obj125:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:eb806c57:::obj373:head local-lis/les=117/118 n=560 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.294+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.294+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eb806c57:::obj373:head local-lis/les=117/118 n=560 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:ecbe4ab7:::obj125:head) v3 2026-02-11T23:31:28.294+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eb806c57:::obj373:head local-lis/les=117/118 n=560 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:ecbe4ab7:::obj125:head) 2026-02-11T23:31:28.294+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.294+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:ecbe4ab7:::obj125:head) v3 finish latency 0.000013 2026-02-11T23:31:28.294+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ecbe4ab7:::obj125:head local-lis/les=117/118 n=561 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.294+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.294+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.294+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ecbe4ab7:::obj125:head local-lis/les=117/118 n=561 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.294+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.294+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.294+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.294+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x671d980 con 0x507ac00 2026-02-11T23:31:28.294+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:ed234181:::obj472:head)]) -- 0x6766780 con 0x507ac00 2026-02-11T23:31:28.294+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.295+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1163 ==== MOSDPGPush(4.0 119/117 [PushOp(4:ee44f25c:::obj334:head, version: 111'334, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ee44f25c:::obj334:head@111'334, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56b64e0 con 0x507ac00 2026-02-11T23:31:28.295+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:ee44f25c:::obj334:head, version: 111'334, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ee44f25c:::obj334:head@111'334, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 119 2026-02-11T23:31:28.295+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ee44f25c:::obj334:head, version: 111'334, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ee44f25c:::obj334:head@111'334, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.295+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.295+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1164 ==== pg_backfill(progress 4.0 e 119/119 lb 4:ed234181:::obj472:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2e100 con 0x507ac00 2026-02-11T23:31:28.295+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:ed234181:::obj472:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.295+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ee44f25c:::obj334:head, version: 111'334, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ee44f25c:::obj334:head@111'334, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.295+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:ed234181:::obj472:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.295+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.295+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:ed234181:::obj472:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.295+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.295+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ee44f25c:::obj334:head, version: 111'334, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ee44f25c:::obj334:head@111'334, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.295+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:ee44f25c:::obj334:head, version: 111'334, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ee44f25c:::obj334:head@111'334, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 119/117 [PushOp(4:ee44f25c:::obj334:head, version: 111'334, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ee44f25c:::obj334:head@111'334, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:ecbe4ab7:::obj125:head local-lis/les=117/118 n=561 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.295+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.295+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ecbe4ab7:::obj125:head local-lis/les=117/118 n=561 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:ee44f25c:::obj334:head, version: 111'334, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ee44f25c:::obj334:head@111'334, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.295+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ecbe4ab7:::obj125:head local-lis/les=117/118 n=561 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:ee44f25c:::obj334:head@111'334, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.295+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ecbe4ab7:::obj125:head local-lis/les=117/118 n=561 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:ee44f25c:::obj334:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.295+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ecbe4ab7:::obj125:head local-lis/les=117/118 n=561 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:ee44f25c:::obj334:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.295+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ecbe4ab7:::obj125:head local-lis/les=117/118 n=561 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:ee44f25c:::obj334:head 2026-02-11T23:31:28.295+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ecbe4ab7:::obj125:head local-lis/les=117/118 n=561 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:ee44f25c:::obj334:head v 111'334 2026-02-11T23:31:28.295+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ecbe4ab7:::obj125:head local-lis/les=117/118 n=561 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.295+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.295+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:ee44f25c:::obj334:head, version: 111'334, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ee44f25c:::obj334:head@111'334, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:28.295+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.295+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.295+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:ed234181:::obj472:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.295+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:ed234181:::obj472:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 4.0 e 119/119 lb 4:ed234181:::obj472:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:ecbe4ab7:::obj125:head local-lis/les=117/118 n=561 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.295+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.295+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ecbe4ab7:::obj125:head local-lis/les=117/118 n=561 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:ed234181:::obj472:head) v3 2026-02-11T23:31:28.295+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ecbe4ab7:::obj125:head local-lis/les=117/118 n=561 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:ed234181:::obj472:head) 2026-02-11T23:31:28.295+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.295+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:ed234181:::obj472:head) v3 finish latency 0.000010 2026-02-11T23:31:28.295+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ed234181:::obj472:head local-lis/les=117/118 n=562 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.295+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.295+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.295+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ed234181:::obj472:head local-lis/les=117/118 n=562 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.295+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.295+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.295+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.295+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x6788000 con 0x507ac00 2026-02-11T23:31:28.295+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:ee44f25c:::obj334:head)]) -- 0x6794000 con 0x507ac00 2026-02-11T23:31:28.295+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.295+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1165 ==== MOSDPGPush(4.0 119/117 [PushOp(4:eeae2a94:::obj11:head, version: 111'11, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eeae2a94:::obj11:head@111'11, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x56caea0 con 0x507ac00 2026-02-11T23:31:28.295+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:eeae2a94:::obj11:head, version: 111'11, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eeae2a94:::obj11:head@111'11, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 119 2026-02-11T23:31:28.295+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:eeae2a94:::obj11:head, version: 111'11, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eeae2a94:::obj11:head@111'11, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.295+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.295+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1166 ==== pg_backfill(progress 4.0 e 119/119 lb 4:ee44f25c:::obj334:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2c000 con 0x507ac00 2026-02-11T23:31:28.295+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:ee44f25c:::obj334:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.295+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:eeae2a94:::obj11:head, version: 111'11, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eeae2a94:::obj11:head@111'11, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.295+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:ee44f25c:::obj334:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.295+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.295+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:eeae2a94:::obj11:head, version: 111'11, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eeae2a94:::obj11:head@111'11, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.295+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.295+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:ee44f25c:::obj334:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.295+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:eeae2a94:::obj11:head, version: 111'11, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eeae2a94:::obj11:head@111'11, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 119/117 [PushOp(4:eeae2a94:::obj11:head, version: 111'11, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eeae2a94:::obj11:head@111'11, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:ed234181:::obj472:head local-lis/les=117/118 n=562 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.295+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.295+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ed234181:::obj472:head local-lis/les=117/118 n=562 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:eeae2a94:::obj11:head, version: 111'11, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eeae2a94:::obj11:head@111'11, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.295+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ed234181:::obj472:head local-lis/les=117/118 n=562 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:eeae2a94:::obj11:head@111'11, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.295+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ed234181:::obj472:head local-lis/les=117/118 n=562 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:eeae2a94:::obj11:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.295+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ed234181:::obj472:head local-lis/les=117/118 n=562 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:eeae2a94:::obj11:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.295+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ed234181:::obj472:head local-lis/les=117/118 n=562 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:eeae2a94:::obj11:head 2026-02-11T23:31:28.295+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ed234181:::obj472:head local-lis/les=117/118 n=562 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:eeae2a94:::obj11:head v 111'11 2026-02-11T23:31:28.295+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ed234181:::obj472:head local-lis/les=117/118 n=562 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.295+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.295+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:eeae2a94:::obj11:head, version: 111'11, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eeae2a94:::obj11:head@111'11, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:28.295+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.295+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.295+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:ee44f25c:::obj334:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.295+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:ee44f25c:::obj334:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 4.0 e 119/119 lb 4:ee44f25c:::obj334:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:ed234181:::obj472:head local-lis/les=117/118 n=562 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.295+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.295+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ed234181:::obj472:head local-lis/les=117/118 n=562 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:ee44f25c:::obj334:head) v3 2026-02-11T23:31:28.295+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ed234181:::obj472:head local-lis/les=117/118 n=562 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:ee44f25c:::obj334:head) 2026-02-11T23:31:28.295+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.295+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:ee44f25c:::obj334:head) v3 finish latency 0.000013 2026-02-11T23:31:28.295+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ee44f25c:::obj334:head local-lis/les=117/118 n=563 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.295+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.296+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ee44f25c:::obj334:head local-lis/les=117/118 n=563 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.296+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.296+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.296+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.296+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.296+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x674b080 con 0x507ac00 2026-02-11T23:31:28.296+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:eeae2a94:::obj11:head)]) -- 0x6794300 con 0x507ac00 2026-02-11T23:31:28.296+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.296+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1167 ==== MOSDPGPush(4.0 119/117 [PushOp(4:eff0a739:::obj127:head, version: 111'127, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eff0a739:::obj127:head@111'127, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c340 con 0x507ac00 2026-02-11T23:31:28.296+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:eff0a739:::obj127:head, version: 111'127, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eff0a739:::obj127:head@111'127, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 119 2026-02-11T23:31:28.296+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:eff0a739:::obj127:head, version: 111'127, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eff0a739:::obj127:head@111'127, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.296+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.296+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1168 ==== pg_backfill(progress 4.0 e 119/119 lb 4:eeae2a94:::obj11:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d2cb00 con 0x507ac00 2026-02-11T23:31:28.296+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:eeae2a94:::obj11:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.296+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:eff0a739:::obj127:head, version: 111'127, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eff0a739:::obj127:head@111'127, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.296+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:eeae2a94:::obj11:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.296+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.296+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:eff0a739:::obj127:head, version: 111'127, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eff0a739:::obj127:head@111'127, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.296+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:eff0a739:::obj127:head, version: 111'127, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eff0a739:::obj127:head@111'127, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 119/117 [PushOp(4:eff0a739:::obj127:head, version: 111'127, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eff0a739:::obj127:head@111'127, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:ee44f25c:::obj334:head local-lis/les=117/118 n=563 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.296+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.296+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.296+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ee44f25c:::obj334:head local-lis/les=117/118 n=563 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:eff0a739:::obj127:head, version: 111'127, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eff0a739:::obj127:head@111'127, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.296+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:eeae2a94:::obj11:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.296+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ee44f25c:::obj334:head local-lis/les=117/118 n=563 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:eff0a739:::obj127:head@111'127, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.296+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ee44f25c:::obj334:head local-lis/les=117/118 n=563 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:eff0a739:::obj127:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.296+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ee44f25c:::obj334:head local-lis/les=117/118 n=563 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:eff0a739:::obj127:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.296+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ee44f25c:::obj334:head local-lis/les=117/118 n=563 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:eff0a739:::obj127:head 2026-02-11T23:31:28.296+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ee44f25c:::obj334:head local-lis/les=117/118 n=563 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:eff0a739:::obj127:head v 111'127 2026-02-11T23:31:28.296+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ee44f25c:::obj334:head local-lis/les=117/118 n=563 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.296+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.296+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:eff0a739:::obj127:head, version: 111'127, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eff0a739:::obj127:head@111'127, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:28.296+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.296+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.296+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:eeae2a94:::obj11:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.296+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:eeae2a94:::obj11:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 4.0 e 119/119 lb 4:eeae2a94:::obj11:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:ee44f25c:::obj334:head local-lis/les=117/118 n=563 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.296+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.296+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ee44f25c:::obj334:head local-lis/les=117/118 n=563 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:eeae2a94:::obj11:head) v3 2026-02-11T23:31:28.296+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ee44f25c:::obj334:head local-lis/les=117/118 n=563 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:eeae2a94:::obj11:head) 2026-02-11T23:31:28.296+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.296+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:eeae2a94:::obj11:head) v3 finish latency 0.000018 2026-02-11T23:31:28.296+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eeae2a94:::obj11:head local-lis/les=117/118 n=564 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.296+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.296+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.296+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eeae2a94:::obj11:head local-lis/les=117/118 n=564 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.296+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.296+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.296+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.296+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x674be00 con 0x507ac00 2026-02-11T23:31:28.296+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:eff0a739:::obj127:head)]) -- 0x6794480 con 0x507ac00 2026-02-11T23:31:28.296+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.296+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1169 ==== MOSDPGPush(4.0 119/117 [PushOp(4:f0a12295:::obj487:head, version: 111'487, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f0a12295:::obj487:head@111'487, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3815520 con 0x507ac00 2026-02-11T23:31:28.296+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:f0a12295:::obj487:head, version: 111'487, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f0a12295:::obj487:head@111'487, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 119 2026-02-11T23:31:28.296+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f0a12295:::obj487:head, version: 111'487, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f0a12295:::obj487:head@111'487, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.296+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.296+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1170 ==== pg_backfill(progress 4.0 e 119/119 lb 4:eff0a739:::obj127:head) ==== 1062+0+0 (unknown 0 0 0) 0x6179700 con 0x507ac00 2026-02-11T23:31:28.296+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:eff0a739:::obj127:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.296+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f0a12295:::obj487:head, version: 111'487, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f0a12295:::obj487:head@111'487, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.296+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:eff0a739:::obj127:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.296+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.296+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f0a12295:::obj487:head, version: 111'487, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f0a12295:::obj487:head@111'487, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.296+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:f0a12295:::obj487:head, version: 111'487, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f0a12295:::obj487:head@111'487, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 119/117 [PushOp(4:f0a12295:::obj487:head, version: 111'487, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f0a12295:::obj487:head@111'487, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:eeae2a94:::obj11:head local-lis/les=117/118 n=564 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.296+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.296+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.296+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:eff0a739:::obj127:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.296+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eeae2a94:::obj11:head local-lis/les=117/118 n=564 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:f0a12295:::obj487:head, version: 111'487, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f0a12295:::obj487:head@111'487, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.296+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eeae2a94:::obj11:head local-lis/les=117/118 n=564 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:f0a12295:::obj487:head@111'487, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.296+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eeae2a94:::obj11:head local-lis/les=117/118 n=564 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:f0a12295:::obj487:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.297+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eeae2a94:::obj11:head local-lis/les=117/118 n=564 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:f0a12295:::obj487:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.297+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eeae2a94:::obj11:head local-lis/les=117/118 n=564 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:f0a12295:::obj487:head 2026-02-11T23:31:28.297+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eeae2a94:::obj11:head local-lis/les=117/118 n=564 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:f0a12295:::obj487:head v 111'487 2026-02-11T23:31:28.297+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eeae2a94:::obj11:head local-lis/les=117/118 n=564 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.297+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.297+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:f0a12295:::obj487:head, version: 111'487, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f0a12295:::obj487:head@111'487, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000068 2026-02-11T23:31:28.297+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.297+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.297+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:eff0a739:::obj127:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.297+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:eff0a739:::obj127:head) v3 prio 15 cost 0 latency 0.000084 pg_backfill(progress 4.0 e 119/119 lb 4:eff0a739:::obj127:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:eeae2a94:::obj11:head local-lis/les=117/118 n=564 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.297+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.297+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eeae2a94:::obj11:head local-lis/les=117/118 n=564 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:eff0a739:::obj127:head) v3 2026-02-11T23:31:28.297+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eeae2a94:::obj11:head local-lis/les=117/118 n=564 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:eff0a739:::obj127:head) 2026-02-11T23:31:28.297+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.297+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:eff0a739:::obj127:head) v3 finish latency 0.000012 2026-02-11T23:31:28.297+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eff0a739:::obj127:head local-lis/les=117/118 n=565 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.297+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.297+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eff0a739:::obj127:head local-lis/les=117/118 n=565 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.297+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.297+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.297+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.297+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.297+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x674af00 con 0x507ac00 2026-02-11T23:31:28.297+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:f0a12295:::obj487:head)]) -- 0x6794780 con 0x507ac00 2026-02-11T23:31:28.297+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.297+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1171 ==== MOSDPGPush(4.0 119/117 [PushOp(4:f1182822:::obj193:head, version: 111'193, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f1182822:::obj193:head@111'193, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x554e680 con 0x507ac00 2026-02-11T23:31:28.297+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:f1182822:::obj193:head, version: 111'193, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f1182822:::obj193:head@111'193, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 119 2026-02-11T23:31:28.297+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f1182822:::obj193:head, version: 111'193, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f1182822:::obj193:head@111'193, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.297+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.297+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1172 ==== pg_backfill(progress 4.0 e 119/119 lb 4:f0a12295:::obj487:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176000 con 0x507ac00 2026-02-11T23:31:28.297+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:f0a12295:::obj487:head) v3 prio 15 type 95 cost 0 latency 0.000002 epoch 119 2026-02-11T23:31:28.297+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f1182822:::obj193:head, version: 111'193, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f1182822:::obj193:head@111'193, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.297+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f0a12295:::obj487:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.297+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.297+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f0a12295:::obj487:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.297+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.297+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f1182822:::obj193:head, version: 111'193, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f1182822:::obj193:head@111'193, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.297+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:f1182822:::obj193:head, version: 111'193, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f1182822:::obj193:head@111'193, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 119/117 [PushOp(4:f1182822:::obj193:head, version: 111'193, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f1182822:::obj193:head@111'193, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:eff0a739:::obj127:head local-lis/les=117/118 n=565 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.297+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.297+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eff0a739:::obj127:head local-lis/les=117/118 n=565 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:f1182822:::obj193:head, version: 111'193, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f1182822:::obj193:head@111'193, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.297+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eff0a739:::obj127:head local-lis/les=117/118 n=565 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:f1182822:::obj193:head@111'193, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.297+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eff0a739:::obj127:head local-lis/les=117/118 n=565 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:f1182822:::obj193:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.297+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eff0a739:::obj127:head local-lis/les=117/118 n=565 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:f1182822:::obj193:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.297+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eff0a739:::obj127:head local-lis/les=117/118 n=565 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:f1182822:::obj193:head 2026-02-11T23:31:28.297+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eff0a739:::obj127:head local-lis/les=117/118 n=565 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:f1182822:::obj193:head v 111'193 2026-02-11T23:31:28.297+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eff0a739:::obj127:head local-lis/les=117/118 n=565 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.297+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.297+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:f1182822:::obj193:head, version: 111'193, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f1182822:::obj193:head@111'193, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:28.297+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.297+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.297+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f0a12295:::obj487:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.297+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:f0a12295:::obj487:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 4.0 e 119/119 lb 4:f0a12295:::obj487:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:eff0a739:::obj127:head local-lis/les=117/118 n=565 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.297+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.297+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eff0a739:::obj127:head local-lis/les=117/118 n=565 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:f0a12295:::obj487:head) v3 2026-02-11T23:31:28.297+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:eff0a739:::obj127:head local-lis/les=117/118 n=565 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:f0a12295:::obj487:head) 2026-02-11T23:31:28.297+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.297+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:f0a12295:::obj487:head) v3 finish latency 0.000014 2026-02-11T23:31:28.297+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f0a12295:::obj487:head local-lis/les=117/118 n=566 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.297+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.297+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.297+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f0a12295:::obj487:head local-lis/les=117/118 n=566 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.297+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.297+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.297+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.297+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x63c9e00 con 0x507ac00 2026-02-11T23:31:28.297+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:f1182822:::obj193:head)]) -- 0x6794a80 con 0x507ac00 2026-02-11T23:31:28.297+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.298+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1173 ==== MOSDPGPush(4.0 119/117 [PushOp(4:f11aef48:::obj58:head, version: 111'58, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f11aef48:::obj58:head@111'58, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x555d040 con 0x507ac00 2026-02-11T23:31:28.298+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:f11aef48:::obj58:head, version: 111'58, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f11aef48:::obj58:head@111'58, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000015 epoch 119 2026-02-11T23:31:28.298+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f11aef48:::obj58:head, version: 111'58, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f11aef48:::obj58:head@111'58, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.298+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.298+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1174 ==== pg_backfill(progress 4.0 e 119/119 lb 4:f1182822:::obj193:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9a680 con 0x507ac00 2026-02-11T23:31:28.298+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f11aef48:::obj58:head, version: 111'58, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f11aef48:::obj58:head@111'58, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.298+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:f1182822:::obj193:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.298+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.298+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f1182822:::obj193:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.298+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f11aef48:::obj58:head, version: 111'58, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f11aef48:::obj58:head@111'58, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.298+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.298+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:f11aef48:::obj58:head, version: 111'58, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f11aef48:::obj58:head@111'58, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(4.0 119/117 [PushOp(4:f11aef48:::obj58:head, version: 111'58, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f11aef48:::obj58:head@111'58, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:f0a12295:::obj487:head local-lis/les=117/118 n=566 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.298+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.298+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f0a12295:::obj487:head local-lis/les=117/118 n=566 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:f11aef48:::obj58:head, version: 111'58, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f11aef48:::obj58:head@111'58, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.298+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f0a12295:::obj487:head local-lis/les=117/118 n=566 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:f11aef48:::obj58:head@111'58, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.298+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.298+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f1182822:::obj193:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.298+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f0a12295:::obj487:head local-lis/les=117/118 n=566 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:f11aef48:::obj58:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.298+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f0a12295:::obj487:head local-lis/les=117/118 n=566 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:f11aef48:::obj58:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.298+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f0a12295:::obj487:head local-lis/les=117/118 n=566 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:f11aef48:::obj58:head 2026-02-11T23:31:28.298+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f0a12295:::obj487:head local-lis/les=117/118 n=566 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:f11aef48:::obj58:head v 111'58 2026-02-11T23:31:28.298+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f0a12295:::obj487:head local-lis/les=117/118 n=566 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.298+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.298+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:f11aef48:::obj58:head, version: 111'58, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f11aef48:::obj58:head@111'58, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000064 2026-02-11T23:31:28.298+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.298+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.298+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f1182822:::obj193:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.298+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:f1182822:::obj193:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 4.0 e 119/119 lb 4:f1182822:::obj193:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:f0a12295:::obj487:head local-lis/les=117/118 n=566 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.298+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.298+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f0a12295:::obj487:head local-lis/les=117/118 n=566 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:f1182822:::obj193:head) v3 2026-02-11T23:31:28.298+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f0a12295:::obj487:head local-lis/les=117/118 n=566 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:f1182822:::obj193:head) 2026-02-11T23:31:28.298+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.298+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:f1182822:::obj193:head) v3 finish latency 0.000015 2026-02-11T23:31:28.298+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f1182822:::obj193:head local-lis/les=117/118 n=567 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.298+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.298+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.298+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f1182822:::obj193:head local-lis/les=117/118 n=567 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.298+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.298+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.298+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.298+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x6767e00 con 0x507ac00 2026-02-11T23:31:28.298+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:f11aef48:::obj58:head)]) -- 0x6794d80 con 0x507ac00 2026-02-11T23:31:28.298+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.298+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1175 ==== MOSDPGPush(4.0 119/117 [PushOp(4:f13c731e:::obj449:head, version: 111'449, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f13c731e:::obj449:head@111'449, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56aa000 con 0x507ac00 2026-02-11T23:31:28.298+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:f13c731e:::obj449:head, version: 111'449, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f13c731e:::obj449:head@111'449, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 119 2026-02-11T23:31:28.298+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f13c731e:::obj449:head, version: 111'449, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f13c731e:::obj449:head@111'449, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.298+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.298+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1176 ==== pg_backfill(progress 4.0 e 119/119 lb 4:f11aef48:::obj58:head) ==== 1061+0+0 (unknown 0 0 0) 0x5985700 con 0x507ac00 2026-02-11T23:31:28.298+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:f11aef48:::obj58:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.298+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f13c731e:::obj449:head, version: 111'449, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f13c731e:::obj449:head@111'449, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.298+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f11aef48:::obj58:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.298+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.298+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f13c731e:::obj449:head, version: 111'449, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f13c731e:::obj449:head@111'449, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.298+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:f13c731e:::obj449:head, version: 111'449, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f13c731e:::obj449:head@111'449, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 119/117 [PushOp(4:f13c731e:::obj449:head, version: 111'449, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f13c731e:::obj449:head@111'449, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:f1182822:::obj193:head local-lis/les=117/118 n=567 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.298+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.298+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.298+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f11aef48:::obj58:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.298+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f1182822:::obj193:head local-lis/les=117/118 n=567 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:f13c731e:::obj449:head, version: 111'449, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f13c731e:::obj449:head@111'449, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.298+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f1182822:::obj193:head local-lis/les=117/118 n=567 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:f13c731e:::obj449:head@111'449, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.298+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f1182822:::obj193:head local-lis/les=117/118 n=567 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:f13c731e:::obj449:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.298+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f1182822:::obj193:head local-lis/les=117/118 n=567 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:f13c731e:::obj449:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.298+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f1182822:::obj193:head local-lis/les=117/118 n=567 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:f13c731e:::obj449:head 2026-02-11T23:31:28.298+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f1182822:::obj193:head local-lis/les=117/118 n=567 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:f13c731e:::obj449:head v 111'449 2026-02-11T23:31:28.298+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f1182822:::obj193:head local-lis/les=117/118 n=567 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.298+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.298+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:f13c731e:::obj449:head, version: 111'449, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f13c731e:::obj449:head@111'449, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:28.298+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.298+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.298+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f11aef48:::obj58:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.298+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:f11aef48:::obj58:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 4.0 e 119/119 lb 4:f11aef48:::obj58:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:f1182822:::obj193:head local-lis/les=117/118 n=567 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.298+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.298+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f1182822:::obj193:head local-lis/les=117/118 n=567 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:f11aef48:::obj58:head) v3 2026-02-11T23:31:28.298+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f1182822:::obj193:head local-lis/les=117/118 n=567 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:f11aef48:::obj58:head) 2026-02-11T23:31:28.298+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.298+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:f11aef48:::obj58:head) v3 finish latency 0.000012 2026-02-11T23:31:28.298+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f11aef48:::obj58:head local-lis/les=117/118 n=568 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.298+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.298+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.298+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f11aef48:::obj58:head local-lis/les=117/118 n=568 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.298+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.298+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.298+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.298+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x642cd80 con 0x507ac00 2026-02-11T23:31:28.298+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:f13c731e:::obj449:head)]) -- 0x6794f00 con 0x507ac00 2026-02-11T23:31:28.298+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.299+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1177 ==== MOSDPGPush(4.0 119/117 [PushOp(4:f22dcb20:::obj569:head, version: 111'569, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f22dcb20:::obj569:head@111'569, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2000 con 0x507ac00 2026-02-11T23:31:28.299+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:f22dcb20:::obj569:head, version: 111'569, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f22dcb20:::obj569:head@111'569, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 119 2026-02-11T23:31:28.299+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f22dcb20:::obj569:head, version: 111'569, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f22dcb20:::obj569:head@111'569, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.299+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.299+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1178 ==== pg_backfill(progress 4.0 e 119/119 lb 4:f13c731e:::obj449:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177600 con 0x507ac00 2026-02-11T23:31:28.299+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f22dcb20:::obj569:head, version: 111'569, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f22dcb20:::obj569:head@111'569, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.299+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:f13c731e:::obj449:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.299+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f13c731e:::obj449:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.299+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.299+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f22dcb20:::obj569:head, version: 111'569, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f22dcb20:::obj569:head@111'569, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.299+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:f22dcb20:::obj569:head, version: 111'569, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f22dcb20:::obj569:head@111'569, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 119/117 [PushOp(4:f22dcb20:::obj569:head, version: 111'569, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f22dcb20:::obj569:head@111'569, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:f11aef48:::obj58:head local-lis/les=117/118 n=568 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.299+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.299+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f11aef48:::obj58:head local-lis/les=117/118 n=568 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:f22dcb20:::obj569:head, version: 111'569, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f22dcb20:::obj569:head@111'569, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.299+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.299+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f11aef48:::obj58:head local-lis/les=117/118 n=568 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:f22dcb20:::obj569:head@111'569, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.299+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f13c731e:::obj449:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.299+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f11aef48:::obj58:head local-lis/les=117/118 n=568 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:f22dcb20:::obj569:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.299+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f11aef48:::obj58:head local-lis/les=117/118 n=568 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:f22dcb20:::obj569:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.299+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f11aef48:::obj58:head local-lis/les=117/118 n=568 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:f22dcb20:::obj569:head 2026-02-11T23:31:28.299+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f11aef48:::obj58:head local-lis/les=117/118 n=568 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:f22dcb20:::obj569:head v 111'569 2026-02-11T23:31:28.299+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f11aef48:::obj58:head local-lis/les=117/118 n=568 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.299+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.299+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:f22dcb20:::obj569:head, version: 111'569, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f22dcb20:::obj569:head@111'569, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:28.299+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.299+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.299+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f13c731e:::obj449:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.299+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:f13c731e:::obj449:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 4.0 e 119/119 lb 4:f13c731e:::obj449:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:f11aef48:::obj58:head local-lis/les=117/118 n=568 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.299+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.299+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f11aef48:::obj58:head local-lis/les=117/118 n=568 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:f13c731e:::obj449:head) v3 2026-02-11T23:31:28.299+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f11aef48:::obj58:head local-lis/les=117/118 n=568 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:f13c731e:::obj449:head) 2026-02-11T23:31:28.299+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.299+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:f13c731e:::obj449:head) v3 finish latency 0.000017 2026-02-11T23:31:28.299+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f13c731e:::obj449:head local-lis/les=117/118 n=569 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.299+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.299+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.299+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f13c731e:::obj449:head local-lis/les=117/118 n=569 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.299+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.299+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.299+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.299+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x6776180 con 0x507ac00 2026-02-11T23:31:28.299+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:f22dcb20:::obj569:head)]) -- 0x6795200 con 0x507ac00 2026-02-11T23:31:28.299+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.299+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1179 ==== MOSDPGPush(4.0 119/117 [PushOp(4:f2a1ba99:::obj364:head, version: 111'364, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2a1ba99:::obj364:head@111'364, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f1d40 con 0x507ac00 2026-02-11T23:31:28.299+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:f2a1ba99:::obj364:head, version: 111'364, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2a1ba99:::obj364:head@111'364, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 119 2026-02-11T23:31:28.299+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f2a1ba99:::obj364:head, version: 111'364, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2a1ba99:::obj364:head@111'364, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.299+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.299+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1180 ==== pg_backfill(progress 4.0 e 119/119 lb 4:f22dcb20:::obj569:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177b80 con 0x507ac00 2026-02-11T23:31:28.299+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:f22dcb20:::obj569:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.299+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f2a1ba99:::obj364:head, version: 111'364, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2a1ba99:::obj364:head@111'364, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.299+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f22dcb20:::obj569:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.299+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.299+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f2a1ba99:::obj364:head, version: 111'364, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2a1ba99:::obj364:head@111'364, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.299+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:f2a1ba99:::obj364:head, version: 111'364, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2a1ba99:::obj364:head@111'364, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 119/117 [PushOp(4:f2a1ba99:::obj364:head, version: 111'364, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2a1ba99:::obj364:head@111'364, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:f13c731e:::obj449:head local-lis/les=117/118 n=569 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.299+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.299+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.299+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f22dcb20:::obj569:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.299+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f13c731e:::obj449:head local-lis/les=117/118 n=569 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:f2a1ba99:::obj364:head, version: 111'364, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2a1ba99:::obj364:head@111'364, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.299+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f13c731e:::obj449:head local-lis/les=117/118 n=569 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:f2a1ba99:::obj364:head@111'364, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.299+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f13c731e:::obj449:head local-lis/les=117/118 n=569 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:f2a1ba99:::obj364:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.299+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f13c731e:::obj449:head local-lis/les=117/118 n=569 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:f2a1ba99:::obj364:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.299+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f13c731e:::obj449:head local-lis/les=117/118 n=569 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:f2a1ba99:::obj364:head 2026-02-11T23:31:28.299+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f13c731e:::obj449:head local-lis/les=117/118 n=569 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:f2a1ba99:::obj364:head v 111'364 2026-02-11T23:31:28.299+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f13c731e:::obj449:head local-lis/les=117/118 n=569 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.299+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.299+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:f2a1ba99:::obj364:head, version: 111'364, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2a1ba99:::obj364:head@111'364, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:28.299+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.299+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.299+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f22dcb20:::obj569:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.299+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:f22dcb20:::obj569:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 4.0 e 119/119 lb 4:f22dcb20:::obj569:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:f13c731e:::obj449:head local-lis/les=117/118 n=569 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.299+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.299+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f13c731e:::obj449:head local-lis/les=117/118 n=569 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:f22dcb20:::obj569:head) v3 2026-02-11T23:31:28.299+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f13c731e:::obj449:head local-lis/les=117/118 n=569 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:f22dcb20:::obj569:head) 2026-02-11T23:31:28.299+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.299+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:f22dcb20:::obj569:head) v3 finish latency 0.000012 2026-02-11T23:31:28.299+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f22dcb20:::obj569:head local-lis/les=117/118 n=570 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.299+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.299+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.300+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f22dcb20:::obj569:head local-lis/les=117/118 n=570 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.300+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.300+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.300+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.300+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x671dc80 con 0x507ac00 2026-02-11T23:31:28.300+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:f2a1ba99:::obj364:head)]) -- 0x6795500 con 0x507ac00 2026-02-11T23:31:28.300+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.300+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1181 ==== MOSDPGPush(4.0 119/117 [PushOp(4:f2bc7f0e:::obj94:head, version: 111'94, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2bc7f0e:::obj94:head@111'94, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x554e340 con 0x507ac00 2026-02-11T23:31:28.300+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:f2bc7f0e:::obj94:head, version: 111'94, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2bc7f0e:::obj94:head@111'94, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.300+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f2bc7f0e:::obj94:head, version: 111'94, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2bc7f0e:::obj94:head@111'94, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.300+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.300+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f2bc7f0e:::obj94:head, version: 111'94, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2bc7f0e:::obj94:head@111'94, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.300+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.300+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1182 ==== pg_backfill(progress 4.0 e 119/119 lb 4:f2a1ba99:::obj364:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2c580 con 0x507ac00 2026-02-11T23:31:28.300+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:f2a1ba99:::obj364:head) v3 prio 15 type 95 cost 0 latency 0.000010 epoch 119 2026-02-11T23:31:28.300+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f2bc7f0e:::obj94:head, version: 111'94, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2bc7f0e:::obj94:head@111'94, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.300+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f2a1ba99:::obj364:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.300+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.300+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f2a1ba99:::obj364:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.300+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:f2bc7f0e:::obj94:head, version: 111'94, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2bc7f0e:::obj94:head@111'94, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 119/117 [PushOp(4:f2bc7f0e:::obj94:head, version: 111'94, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2bc7f0e:::obj94:head@111'94, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:f22dcb20:::obj569:head local-lis/les=117/118 n=570 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.300+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.300+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f22dcb20:::obj569:head local-lis/les=117/118 n=570 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:f2bc7f0e:::obj94:head, version: 111'94, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2bc7f0e:::obj94:head@111'94, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.300+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f22dcb20:::obj569:head local-lis/les=117/118 n=570 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:f2bc7f0e:::obj94:head@111'94, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.300+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f22dcb20:::obj569:head local-lis/les=117/118 n=570 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:f2bc7f0e:::obj94:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.300+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f22dcb20:::obj569:head local-lis/les=117/118 n=570 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:f2bc7f0e:::obj94:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.300+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f22dcb20:::obj569:head local-lis/les=117/118 n=570 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:f2bc7f0e:::obj94:head 2026-02-11T23:31:28.300+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f22dcb20:::obj569:head local-lis/les=117/118 n=570 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:f2bc7f0e:::obj94:head v 111'94 2026-02-11T23:31:28.300+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f22dcb20:::obj569:head local-lis/les=117/118 n=570 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.300+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.300+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:f2bc7f0e:::obj94:head, version: 111'94, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2bc7f0e:::obj94:head@111'94, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000079 2026-02-11T23:31:28.300+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f22dcb20:::obj569:head local-lis/les=117/118 n=570 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.300+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.300+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.300+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f2a1ba99:::obj364:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.300+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:f2a1ba99:::obj364:head) v3 prio 15 cost 0 latency 0.000102 pg_backfill(progress 4.0 e 119/119 lb 4:f2a1ba99:::obj364:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:f22dcb20:::obj569:head local-lis/les=117/118 n=570 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.300+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.300+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f22dcb20:::obj569:head local-lis/les=117/118 n=570 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:f2a1ba99:::obj364:head) v3 2026-02-11T23:31:28.300+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f22dcb20:::obj569:head local-lis/les=117/118 n=570 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:f2a1ba99:::obj364:head) 2026-02-11T23:31:28.300+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.300+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:f2a1ba99:::obj364:head) v3 finish latency 0.000019 2026-02-11T23:31:28.300+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.300+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.300+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f2a1ba99:::obj364:head local-lis/les=117/118 n=571 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.300+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.300+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.300+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.300+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x674b980 con 0x507ac00 2026-02-11T23:31:28.300+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:f2bc7f0e:::obj94:head)]) -- 0x6776480 con 0x507ac00 2026-02-11T23:31:28.300+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.300+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1183 ==== MOSDPGPush(4.0 119/117 [PushOp(4:f2f9dd5a:::obj121:head, version: 111'121, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2f9dd5a:::obj121:head@111'121, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2820 con 0x507ac00 2026-02-11T23:31:28.300+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:f2f9dd5a:::obj121:head, version: 111'121, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2f9dd5a:::obj121:head@111'121, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.300+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f2f9dd5a:::obj121:head, version: 111'121, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2f9dd5a:::obj121:head@111'121, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.300+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.300+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1184 ==== pg_backfill(progress 4.0 e 119/119 lb 4:f2bc7f0e:::obj94:head) ==== 1061+0+0 (unknown 0 0 0) 0x6178100 con 0x507ac00 2026-02-11T23:31:28.300+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:f2bc7f0e:::obj94:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.300+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f2f9dd5a:::obj121:head, version: 111'121, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2f9dd5a:::obj121:head@111'121, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.300+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f2bc7f0e:::obj94:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.300+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.300+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f2f9dd5a:::obj121:head, version: 111'121, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2f9dd5a:::obj121:head@111'121, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.300+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.300+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f2bc7f0e:::obj94:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.300+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:f2f9dd5a:::obj121:head, version: 111'121, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2f9dd5a:::obj121:head@111'121, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(4.0 119/117 [PushOp(4:f2f9dd5a:::obj121:head, version: 111'121, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2f9dd5a:::obj121:head@111'121, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:f2a1ba99:::obj364:head local-lis/les=117/118 n=571 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.300+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.300+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f2a1ba99:::obj364:head local-lis/les=117/118 n=571 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:f2f9dd5a:::obj121:head, version: 111'121, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2f9dd5a:::obj121:head@111'121, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.300+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f2a1ba99:::obj364:head local-lis/les=117/118 n=571 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:f2f9dd5a:::obj121:head@111'121, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.300+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f2a1ba99:::obj364:head local-lis/les=117/118 n=571 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:f2f9dd5a:::obj121:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.300+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f2a1ba99:::obj364:head local-lis/les=117/118 n=571 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:f2f9dd5a:::obj121:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.300+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f2a1ba99:::obj364:head local-lis/les=117/118 n=571 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:f2f9dd5a:::obj121:head 2026-02-11T23:31:28.300+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f2a1ba99:::obj364:head local-lis/les=117/118 n=571 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:f2f9dd5a:::obj121:head v 111'121 2026-02-11T23:31:28.300+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f2a1ba99:::obj364:head local-lis/les=117/118 n=571 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.300+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.300+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:f2f9dd5a:::obj121:head, version: 111'121, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2f9dd5a:::obj121:head@111'121, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:28.300+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.300+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.300+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f2bc7f0e:::obj94:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.300+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:f2bc7f0e:::obj94:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 4.0 e 119/119 lb 4:f2bc7f0e:::obj94:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:f2a1ba99:::obj364:head local-lis/les=117/118 n=571 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.300+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.300+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f2a1ba99:::obj364:head local-lis/les=117/118 n=571 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:f2bc7f0e:::obj94:head) v3 2026-02-11T23:31:28.300+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f2a1ba99:::obj364:head local-lis/les=117/118 n=571 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:f2bc7f0e:::obj94:head) 2026-02-11T23:31:28.300+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.301+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:f2bc7f0e:::obj94:head) v3 finish latency 0.000025 2026-02-11T23:31:28.301+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f2bc7f0e:::obj94:head local-lis/les=117/118 n=572 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.301+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.301+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.301+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f2bc7f0e:::obj94:head local-lis/les=117/118 n=572 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.301+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.301+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.301+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.301+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x6776780 con 0x507ac00 2026-02-11T23:31:28.301+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:f2f9dd5a:::obj121:head)]) -- 0x6795800 con 0x507ac00 2026-02-11T23:31:28.301+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.301+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1185 ==== MOSDPGPush(4.0 119/117 [PushOp(4:f2fc2104:::obj429:head, version: 111'429, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2fc2104:::obj429:head@111'429, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a3a00 con 0x507ac00 2026-02-11T23:31:28.301+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:f2fc2104:::obj429:head, version: 111'429, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2fc2104:::obj429:head@111'429, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 119 2026-02-11T23:31:28.301+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f2fc2104:::obj429:head, version: 111'429, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2fc2104:::obj429:head@111'429, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.301+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.301+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1186 ==== pg_backfill(progress 4.0 e 119/119 lb 4:f2f9dd5a:::obj121:head) ==== 1062+0+0 (unknown 0 0 0) 0x6179180 con 0x507ac00 2026-02-11T23:31:28.301+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:f2f9dd5a:::obj121:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.301+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f2fc2104:::obj429:head, version: 111'429, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2fc2104:::obj429:head@111'429, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.301+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f2f9dd5a:::obj121:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.301+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.301+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f2fc2104:::obj429:head, version: 111'429, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2fc2104:::obj429:head@111'429, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.301+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.301+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f2f9dd5a:::obj121:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.301+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:f2fc2104:::obj429:head, version: 111'429, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2fc2104:::obj429:head@111'429, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 119/117 [PushOp(4:f2fc2104:::obj429:head, version: 111'429, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2fc2104:::obj429:head@111'429, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:f2bc7f0e:::obj94:head local-lis/les=117/118 n=572 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.301+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.301+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f2bc7f0e:::obj94:head local-lis/les=117/118 n=572 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:f2fc2104:::obj429:head, version: 111'429, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2fc2104:::obj429:head@111'429, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.301+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f2bc7f0e:::obj94:head local-lis/les=117/118 n=572 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:f2fc2104:::obj429:head@111'429, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.301+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f2bc7f0e:::obj94:head local-lis/les=117/118 n=572 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:f2fc2104:::obj429:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.301+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f2bc7f0e:::obj94:head local-lis/les=117/118 n=572 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:f2fc2104:::obj429:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.301+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f2bc7f0e:::obj94:head local-lis/les=117/118 n=572 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:f2fc2104:::obj429:head 2026-02-11T23:31:28.301+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f2bc7f0e:::obj94:head local-lis/les=117/118 n=572 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:f2fc2104:::obj429:head v 111'429 2026-02-11T23:31:28.301+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f2bc7f0e:::obj94:head local-lis/les=117/118 n=572 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.301+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.301+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:f2fc2104:::obj429:head, version: 111'429, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2fc2104:::obj429:head@111'429, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:28.301+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.301+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.301+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f2f9dd5a:::obj121:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.301+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:f2f9dd5a:::obj121:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 4.0 e 119/119 lb 4:f2f9dd5a:::obj121:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:f2bc7f0e:::obj94:head local-lis/les=117/118 n=572 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.301+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.301+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f2bc7f0e:::obj94:head local-lis/les=117/118 n=572 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:f2f9dd5a:::obj121:head) v3 2026-02-11T23:31:28.301+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f2bc7f0e:::obj94:head local-lis/les=117/118 n=572 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:f2f9dd5a:::obj121:head) 2026-02-11T23:31:28.301+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.301+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:f2f9dd5a:::obj121:head) v3 finish latency 0.000014 2026-02-11T23:31:28.301+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f2f9dd5a:::obj121:head local-lis/les=117/118 n=573 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.301+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.301+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.301+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f2f9dd5a:::obj121:head local-lis/les=117/118 n=573 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.301+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.301+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.301+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.301+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x642d500 con 0x507ac00 2026-02-11T23:31:28.301+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:f2fc2104:::obj429:head)]) -- 0x6795b00 con 0x507ac00 2026-02-11T23:31:28.301+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.301+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1187 ==== MOSDPGPush(4.0 119/117 [PushOp(4:f31cde5e:::obj446:head, version: 111'446, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f31cde5e:::obj446:head@111'446, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5db0000 con 0x507ac00 2026-02-11T23:31:28.301+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:f31cde5e:::obj446:head, version: 111'446, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f31cde5e:::obj446:head@111'446, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 119 2026-02-11T23:31:28.301+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f31cde5e:::obj446:head, version: 111'446, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f31cde5e:::obj446:head@111'446, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.301+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.301+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1188 ==== pg_backfill(progress 4.0 e 119/119 lb 4:f2fc2104:::obj429:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9b180 con 0x507ac00 2026-02-11T23:31:28.301+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:f2fc2104:::obj429:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.301+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f31cde5e:::obj446:head, version: 111'446, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f31cde5e:::obj446:head@111'446, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.301+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f2fc2104:::obj429:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.301+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.301+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f31cde5e:::obj446:head, version: 111'446, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f31cde5e:::obj446:head@111'446, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.301+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.301+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:f31cde5e:::obj446:head, version: 111'446, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f31cde5e:::obj446:head@111'446, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 119/117 [PushOp(4:f31cde5e:::obj446:head, version: 111'446, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f31cde5e:::obj446:head@111'446, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:f2f9dd5a:::obj121:head local-lis/les=117/118 n=573 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.301+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f2fc2104:::obj429:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.301+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.301+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f2f9dd5a:::obj121:head local-lis/les=117/118 n=573 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:f31cde5e:::obj446:head, version: 111'446, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f31cde5e:::obj446:head@111'446, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.301+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f2f9dd5a:::obj121:head local-lis/les=117/118 n=573 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:f31cde5e:::obj446:head@111'446, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.302+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f2f9dd5a:::obj121:head local-lis/les=117/118 n=573 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:f31cde5e:::obj446:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.302+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f2f9dd5a:::obj121:head local-lis/les=117/118 n=573 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:f31cde5e:::obj446:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.302+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f2f9dd5a:::obj121:head local-lis/les=117/118 n=573 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:f31cde5e:::obj446:head 2026-02-11T23:31:28.302+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f2f9dd5a:::obj121:head local-lis/les=117/118 n=573 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:f31cde5e:::obj446:head v 111'446 2026-02-11T23:31:28.302+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f2f9dd5a:::obj121:head local-lis/les=117/118 n=573 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.302+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.302+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:f31cde5e:::obj446:head, version: 111'446, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f31cde5e:::obj446:head@111'446, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:28.302+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.302+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.302+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f2fc2104:::obj429:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.302+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:f2fc2104:::obj429:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 4.0 e 119/119 lb 4:f2fc2104:::obj429:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:f2f9dd5a:::obj121:head local-lis/les=117/118 n=573 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.302+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.302+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f2f9dd5a:::obj121:head local-lis/les=117/118 n=573 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:f2fc2104:::obj429:head) v3 2026-02-11T23:31:28.302+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f2f9dd5a:::obj121:head local-lis/les=117/118 n=573 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:f2fc2104:::obj429:head) 2026-02-11T23:31:28.302+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.302+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:f2fc2104:::obj429:head) v3 finish latency 0.000013 2026-02-11T23:31:28.302+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f2fc2104:::obj429:head local-lis/les=117/118 n=574 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.302+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.302+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.302+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f2fc2104:::obj429:head local-lis/les=117/118 n=574 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.302+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.302+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.302+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.302+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x6777c80 con 0x507ac00 2026-02-11T23:31:28.302+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:f31cde5e:::obj446:head)]) -- 0x6795e00 con 0x507ac00 2026-02-11T23:31:28.302+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.302+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1189 ==== MOSDPGPush(4.0 119/117 [PushOp(4:f33597d0:::obj489:head, version: 111'489, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f33597d0:::obj489:head@111'489, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2680 con 0x507ac00 2026-02-11T23:31:28.302+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:f33597d0:::obj489:head, version: 111'489, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f33597d0:::obj489:head@111'489, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 119 2026-02-11T23:31:28.302+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f33597d0:::obj489:head, version: 111'489, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f33597d0:::obj489:head@111'489, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.302+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.302+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f33597d0:::obj489:head, version: 111'489, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f33597d0:::obj489:head@111'489, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.302+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1190 ==== pg_backfill(progress 4.0 e 119/119 lb 4:f31cde5e:::obj446:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2ec00 con 0x507ac00 2026-02-11T23:31:28.302+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:f31cde5e:::obj446:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.302+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.302+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f31cde5e:::obj446:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.302+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f33597d0:::obj489:head, version: 111'489, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f33597d0:::obj489:head@111'489, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.302+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.302+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:f33597d0:::obj489:head, version: 111'489, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f33597d0:::obj489:head@111'489, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 119/117 [PushOp(4:f33597d0:::obj489:head, version: 111'489, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f33597d0:::obj489:head@111'489, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:f2fc2104:::obj429:head local-lis/les=117/118 n=574 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.302+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.302+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f2fc2104:::obj429:head local-lis/les=117/118 n=574 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:f33597d0:::obj489:head, version: 111'489, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f33597d0:::obj489:head@111'489, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.302+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.302+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f2fc2104:::obj429:head local-lis/les=117/118 n=574 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:f33597d0:::obj489:head@111'489, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.302+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f31cde5e:::obj446:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.302+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f2fc2104:::obj429:head local-lis/les=117/118 n=574 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:f33597d0:::obj489:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.302+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f2fc2104:::obj429:head local-lis/les=117/118 n=574 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:f33597d0:::obj489:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.302+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f2fc2104:::obj429:head local-lis/les=117/118 n=574 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:f33597d0:::obj489:head 2026-02-11T23:31:28.302+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f2fc2104:::obj429:head local-lis/les=117/118 n=574 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:f33597d0:::obj489:head v 111'489 2026-02-11T23:31:28.302+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f2fc2104:::obj429:head local-lis/les=117/118 n=574 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.302+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.302+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:f33597d0:::obj489:head, version: 111'489, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f33597d0:::obj489:head@111'489, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:28.302+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.302+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.302+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f31cde5e:::obj446:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.302+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:f31cde5e:::obj446:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 4.0 e 119/119 lb 4:f31cde5e:::obj446:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:f2fc2104:::obj429:head local-lis/les=117/118 n=574 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.302+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.302+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f2fc2104:::obj429:head local-lis/les=117/118 n=574 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:f31cde5e:::obj446:head) v3 2026-02-11T23:31:28.302+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f2fc2104:::obj429:head local-lis/les=117/118 n=574 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:f31cde5e:::obj446:head) 2026-02-11T23:31:28.302+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.302+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:f31cde5e:::obj446:head) v3 finish latency 0.000017 2026-02-11T23:31:28.302+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f31cde5e:::obj446:head local-lis/les=117/118 n=575 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.302+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.302+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.302+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f31cde5e:::obj446:head local-lis/les=117/118 n=575 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.302+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.302+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.302+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.302+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x66d9080 con 0x507ac00 2026-02-11T23:31:28.302+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:f33597d0:::obj489:head)]) -- 0x674b080 con 0x507ac00 2026-02-11T23:31:28.302+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.303+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1191 ==== MOSDPGPush(4.0 119/117 [PushOp(4:f3768836:::obj189:head, version: 111'189, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f3768836:::obj189:head@111'189, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56cab60 con 0x507ac00 2026-02-11T23:31:28.303+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:f3768836:::obj189:head, version: 111'189, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f3768836:::obj189:head@111'189, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 119 2026-02-11T23:31:28.303+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f3768836:::obj189:head, version: 111'189, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f3768836:::obj189:head@111'189, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.303+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.303+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1192 ==== pg_backfill(progress 4.0 e 119/119 lb 4:f33597d0:::obj489:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99080 con 0x507ac00 2026-02-11T23:31:28.303+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:f33597d0:::obj489:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.303+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f3768836:::obj189:head, version: 111'189, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f3768836:::obj189:head@111'189, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.303+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f33597d0:::obj489:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.303+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.303+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f3768836:::obj189:head, version: 111'189, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f3768836:::obj189:head@111'189, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.303+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.303+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f33597d0:::obj489:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.303+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:f3768836:::obj189:head, version: 111'189, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f3768836:::obj189:head@111'189, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 119/117 [PushOp(4:f3768836:::obj189:head, version: 111'189, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f3768836:::obj189:head@111'189, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:f31cde5e:::obj446:head local-lis/les=117/118 n=575 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.303+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.303+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f31cde5e:::obj446:head local-lis/les=117/118 n=575 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:f3768836:::obj189:head, version: 111'189, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f3768836:::obj189:head@111'189, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.303+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f31cde5e:::obj446:head local-lis/les=117/118 n=575 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:f3768836:::obj189:head@111'189, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.303+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f31cde5e:::obj446:head local-lis/les=117/118 n=575 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:f3768836:::obj189:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.303+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f31cde5e:::obj446:head local-lis/les=117/118 n=575 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:f3768836:::obj189:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.303+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f31cde5e:::obj446:head local-lis/les=117/118 n=575 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:f3768836:::obj189:head 2026-02-11T23:31:28.303+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f31cde5e:::obj446:head local-lis/les=117/118 n=575 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:f3768836:::obj189:head v 111'189 2026-02-11T23:31:28.303+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f31cde5e:::obj446:head local-lis/les=117/118 n=575 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.303+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.303+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:f3768836:::obj189:head, version: 111'189, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f3768836:::obj189:head@111'189, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:28.303+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.303+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.303+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f33597d0:::obj489:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.303+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:f33597d0:::obj489:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 4.0 e 119/119 lb 4:f33597d0:::obj489:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:f31cde5e:::obj446:head local-lis/les=117/118 n=575 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.303+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.303+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f31cde5e:::obj446:head local-lis/les=117/118 n=575 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:f33597d0:::obj489:head) v3 2026-02-11T23:31:28.303+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f31cde5e:::obj446:head local-lis/les=117/118 n=575 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:f33597d0:::obj489:head) 2026-02-11T23:31:28.303+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.303+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:f33597d0:::obj489:head) v3 finish latency 0.000013 2026-02-11T23:31:28.303+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f33597d0:::obj489:head local-lis/les=117/118 n=576 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.303+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.303+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.303+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f33597d0:::obj489:head local-lis/les=117/118 n=576 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.303+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.303+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.303+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.303+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x6777980 con 0x507ac00 2026-02-11T23:31:28.303+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:f3768836:::obj189:head)]) -- 0x6788000 con 0x507ac00 2026-02-11T23:31:28.303+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.303+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1193 ==== MOSDPGPush(4.0 119/117 [PushOp(4:f3c2593a:::obj26:head, version: 111'26, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f3c2593a:::obj26:head@111'26, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x56a3860 con 0x507ac00 2026-02-11T23:31:28.303+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:f3c2593a:::obj26:head, version: 111'26, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f3c2593a:::obj26:head@111'26, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.303+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f3c2593a:::obj26:head, version: 111'26, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f3c2593a:::obj26:head@111'26, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.303+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.303+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1194 ==== pg_backfill(progress 4.0 e 119/119 lb 4:f3768836:::obj189:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99600 con 0x507ac00 2026-02-11T23:31:28.303+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f3c2593a:::obj26:head, version: 111'26, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f3c2593a:::obj26:head@111'26, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.303+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:f3768836:::obj189:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.303+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f3768836:::obj189:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.303+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.303+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f3c2593a:::obj26:head, version: 111'26, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f3c2593a:::obj26:head@111'26, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.303+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:f3c2593a:::obj26:head, version: 111'26, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f3c2593a:::obj26:head@111'26, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 119/117 [PushOp(4:f3c2593a:::obj26:head, version: 111'26, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f3c2593a:::obj26:head@111'26, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:f33597d0:::obj489:head local-lis/les=117/118 n=576 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.303+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.303+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.303+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f33597d0:::obj489:head local-lis/les=117/118 n=576 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:f3c2593a:::obj26:head, version: 111'26, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f3c2593a:::obj26:head@111'26, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.303+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f3768836:::obj189:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.303+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f33597d0:::obj489:head local-lis/les=117/118 n=576 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:f3c2593a:::obj26:head@111'26, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.303+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f33597d0:::obj489:head local-lis/les=117/118 n=576 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:f3c2593a:::obj26:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.303+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f33597d0:::obj489:head local-lis/les=117/118 n=576 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:f3c2593a:::obj26:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.303+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f33597d0:::obj489:head local-lis/les=117/118 n=576 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:f3c2593a:::obj26:head 2026-02-11T23:31:28.303+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f33597d0:::obj489:head local-lis/les=117/118 n=576 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:f3c2593a:::obj26:head v 111'26 2026-02-11T23:31:28.303+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f33597d0:::obj489:head local-lis/les=117/118 n=576 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.303+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.303+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:f3c2593a:::obj26:head, version: 111'26, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f3c2593a:::obj26:head@111'26, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:28.303+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.303+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.303+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f3768836:::obj189:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.303+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:f3768836:::obj189:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 4.0 e 119/119 lb 4:f3768836:::obj189:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:f33597d0:::obj489:head local-lis/les=117/118 n=576 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.303+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.303+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f33597d0:::obj489:head local-lis/les=117/118 n=576 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:f3768836:::obj189:head) v3 2026-02-11T23:31:28.303+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f33597d0:::obj489:head local-lis/les=117/118 n=576 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:f3768836:::obj189:head) 2026-02-11T23:31:28.303+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.303+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:f3768836:::obj189:head) v3 finish latency 0.000023 2026-02-11T23:31:28.303+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f3768836:::obj189:head local-lis/les=117/118 n=577 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.303+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.303+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.303+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f3768836:::obj189:head local-lis/les=117/118 n=577 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.303+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.303+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.303+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.303+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x671d680 con 0x507ac00 2026-02-11T23:31:28.303+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:f3c2593a:::obj26:head)]) -- 0x671d980 con 0x507ac00 2026-02-11T23:31:28.303+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.304+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1195 ==== MOSDPGPush(4.0 119/117 [PushOp(4:f42cbfe2:::obj212:head, version: 111'212, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f42cbfe2:::obj212:head@111'212, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x559c1a0 con 0x507ac00 2026-02-11T23:31:28.304+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:f42cbfe2:::obj212:head, version: 111'212, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f42cbfe2:::obj212:head@111'212, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.304+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f42cbfe2:::obj212:head, version: 111'212, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f42cbfe2:::obj212:head@111'212, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.304+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.304+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1196 ==== pg_backfill(progress 4.0 e 119/119 lb 4:f3c2593a:::obj26:head) ==== 1061+0+0 (unknown 0 0 0) 0x6176b00 con 0x507ac00 2026-02-11T23:31:28.304+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f42cbfe2:::obj212:head, version: 111'212, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f42cbfe2:::obj212:head@111'212, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.304+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:f3c2593a:::obj26:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 119 2026-02-11T23:31:28.304+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f3c2593a:::obj26:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.304+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.304+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f42cbfe2:::obj212:head, version: 111'212, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f42cbfe2:::obj212:head@111'212, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.304+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:f42cbfe2:::obj212:head, version: 111'212, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f42cbfe2:::obj212:head@111'212, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 119/117 [PushOp(4:f42cbfe2:::obj212:head, version: 111'212, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f42cbfe2:::obj212:head@111'212, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:f3768836:::obj189:head local-lis/les=117/118 n=577 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.304+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.304+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.304+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f3c2593a:::obj26:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.304+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f3768836:::obj189:head local-lis/les=117/118 n=577 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:f42cbfe2:::obj212:head, version: 111'212, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f42cbfe2:::obj212:head@111'212, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.304+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f3768836:::obj189:head local-lis/les=117/118 n=577 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:f42cbfe2:::obj212:head@111'212, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.304+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f3768836:::obj189:head local-lis/les=117/118 n=577 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:f42cbfe2:::obj212:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.304+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f3768836:::obj189:head local-lis/les=117/118 n=577 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:f42cbfe2:::obj212:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.304+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f3768836:::obj189:head local-lis/les=117/118 n=577 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:f42cbfe2:::obj212:head 2026-02-11T23:31:28.304+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f3768836:::obj189:head local-lis/les=117/118 n=577 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:f42cbfe2:::obj212:head v 111'212 2026-02-11T23:31:28.304+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f3768836:::obj189:head local-lis/les=117/118 n=577 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.304+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.304+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:f42cbfe2:::obj212:head, version: 111'212, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f42cbfe2:::obj212:head@111'212, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:28.304+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.304+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.304+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f3c2593a:::obj26:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.304+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:f3c2593a:::obj26:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 4.0 e 119/119 lb 4:f3c2593a:::obj26:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:f3768836:::obj189:head local-lis/les=117/118 n=577 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.304+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.304+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f3768836:::obj189:head local-lis/les=117/118 n=577 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:f3c2593a:::obj26:head) v3 2026-02-11T23:31:28.304+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f3768836:::obj189:head local-lis/les=117/118 n=577 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:f3c2593a:::obj26:head) 2026-02-11T23:31:28.304+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.304+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:f3c2593a:::obj26:head) v3 finish latency 0.000014 2026-02-11T23:31:28.304+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f3c2593a:::obj26:head local-lis/les=117/118 n=578 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.304+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.304+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.304+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f3c2593a:::obj26:head local-lis/les=117/118 n=578 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.304+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.304+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.304+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.304+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x6777680 con 0x507ac00 2026-02-11T23:31:28.304+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:f42cbfe2:::obj212:head)]) -- 0x6766a80 con 0x507ac00 2026-02-11T23:31:28.304+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.304+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1197 ==== MOSDPGPush(4.0 119/117 [PushOp(4:f4af07f1:::obj515:head, version: 111'515, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f4af07f1:::obj515:head@111'515, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3814d00 con 0x507ac00 2026-02-11T23:31:28.304+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:f4af07f1:::obj515:head, version: 111'515, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f4af07f1:::obj515:head@111'515, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.304+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f4af07f1:::obj515:head, version: 111'515, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f4af07f1:::obj515:head@111'515, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.304+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.304+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1198 ==== pg_backfill(progress 4.0 e 119/119 lb 4:f42cbfe2:::obj212:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9ac00 con 0x507ac00 2026-02-11T23:31:28.304+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:f42cbfe2:::obj212:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.304+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f4af07f1:::obj515:head, version: 111'515, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f4af07f1:::obj515:head@111'515, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.304+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f42cbfe2:::obj212:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.304+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.304+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f4af07f1:::obj515:head, version: 111'515, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f4af07f1:::obj515:head@111'515, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.304+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:f4af07f1:::obj515:head, version: 111'515, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f4af07f1:::obj515:head@111'515, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 119/117 [PushOp(4:f4af07f1:::obj515:head, version: 111'515, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f4af07f1:::obj515:head@111'515, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:f3c2593a:::obj26:head local-lis/les=117/118 n=578 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.304+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.304+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.304+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f3c2593a:::obj26:head local-lis/les=117/118 n=578 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:f4af07f1:::obj515:head, version: 111'515, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f4af07f1:::obj515:head@111'515, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.304+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f42cbfe2:::obj212:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.304+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f3c2593a:::obj26:head local-lis/les=117/118 n=578 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:f4af07f1:::obj515:head@111'515, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.305+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f3c2593a:::obj26:head local-lis/les=117/118 n=578 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:f4af07f1:::obj515:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.305+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f3c2593a:::obj26:head local-lis/les=117/118 n=578 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:f4af07f1:::obj515:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.305+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f3c2593a:::obj26:head local-lis/les=117/118 n=578 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:f4af07f1:::obj515:head 2026-02-11T23:31:28.305+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f3c2593a:::obj26:head local-lis/les=117/118 n=578 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:f4af07f1:::obj515:head v 111'515 2026-02-11T23:31:28.305+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f3c2593a:::obj26:head local-lis/les=117/118 n=578 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.305+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.305+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:f4af07f1:::obj515:head, version: 111'515, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f4af07f1:::obj515:head@111'515, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:28.305+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.305+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.305+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f42cbfe2:::obj212:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.305+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:f42cbfe2:::obj212:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 4.0 e 119/119 lb 4:f42cbfe2:::obj212:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:f3c2593a:::obj26:head local-lis/les=117/118 n=578 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.305+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.305+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f3c2593a:::obj26:head local-lis/les=117/118 n=578 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:f42cbfe2:::obj212:head) v3 2026-02-11T23:31:28.305+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f3c2593a:::obj26:head local-lis/les=117/118 n=578 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:f42cbfe2:::obj212:head) 2026-02-11T23:31:28.305+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.305+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:f42cbfe2:::obj212:head) v3 finish latency 0.000018 2026-02-11T23:31:28.305+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f42cbfe2:::obj212:head local-lis/les=117/118 n=579 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.305+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.305+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.305+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f42cbfe2:::obj212:head local-lis/les=117/118 n=579 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.305+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.305+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.305+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.305+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x674b680 con 0x507ac00 2026-02-11T23:31:28.305+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:f4af07f1:::obj515:head)]) -- 0x6766d80 con 0x507ac00 2026-02-11T23:31:28.305+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.305+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1199 ==== MOSDPGPush(4.0 119/117 [PushOp(4:f4b32ae1:::obj130:head, version: 111'130, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f4b32ae1:::obj130:head@111'130, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a3040 con 0x507ac00 2026-02-11T23:31:28.305+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:f4b32ae1:::obj130:head, version: 111'130, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f4b32ae1:::obj130:head@111'130, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.305+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f4b32ae1:::obj130:head, version: 111'130, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f4b32ae1:::obj130:head@111'130, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.305+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.305+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1200 ==== pg_backfill(progress 4.0 e 119/119 lb 4:f4af07f1:::obj515:head) ==== 1062+0+0 (unknown 0 0 0) 0x4a2cb00 con 0x507ac00 2026-02-11T23:31:28.305+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:f4af07f1:::obj515:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.305+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f4af07f1:::obj515:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.305+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f4b32ae1:::obj130:head, version: 111'130, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f4b32ae1:::obj130:head@111'130, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.305+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.305+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f4b32ae1:::obj130:head, version: 111'130, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f4b32ae1:::obj130:head@111'130, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.305+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:f4b32ae1:::obj130:head, version: 111'130, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f4b32ae1:::obj130:head@111'130, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 119/117 [PushOp(4:f4b32ae1:::obj130:head, version: 111'130, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f4b32ae1:::obj130:head@111'130, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:f42cbfe2:::obj212:head local-lis/les=117/118 n=579 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.305+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.305+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.305+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f4af07f1:::obj515:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.305+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f42cbfe2:::obj212:head local-lis/les=117/118 n=579 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:f4b32ae1:::obj130:head, version: 111'130, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f4b32ae1:::obj130:head@111'130, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.305+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f42cbfe2:::obj212:head local-lis/les=117/118 n=579 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:f4b32ae1:::obj130:head@111'130, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.305+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f42cbfe2:::obj212:head local-lis/les=117/118 n=579 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:f4b32ae1:::obj130:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.305+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f42cbfe2:::obj212:head local-lis/les=117/118 n=579 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:f4b32ae1:::obj130:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.305+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f42cbfe2:::obj212:head local-lis/les=117/118 n=579 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:f4b32ae1:::obj130:head 2026-02-11T23:31:28.305+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f42cbfe2:::obj212:head local-lis/les=117/118 n=579 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:f4b32ae1:::obj130:head v 111'130 2026-02-11T23:31:28.305+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f42cbfe2:::obj212:head local-lis/les=117/118 n=579 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.305+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.305+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:f4b32ae1:::obj130:head, version: 111'130, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f4b32ae1:::obj130:head@111'130, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:28.305+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.305+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.305+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f4af07f1:::obj515:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.305+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:f4af07f1:::obj515:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 4.0 e 119/119 lb 4:f4af07f1:::obj515:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:f42cbfe2:::obj212:head local-lis/les=117/118 n=579 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.305+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.305+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f42cbfe2:::obj212:head local-lis/les=117/118 n=579 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:f4af07f1:::obj515:head) v3 2026-02-11T23:31:28.305+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f42cbfe2:::obj212:head local-lis/les=117/118 n=579 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:f4af07f1:::obj515:head) 2026-02-11T23:31:28.305+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.305+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:f4af07f1:::obj515:head) v3 finish latency 0.000013 2026-02-11T23:31:28.305+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f4af07f1:::obj515:head local-lis/les=117/118 n=580 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.305+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.305+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.305+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f4af07f1:::obj515:head local-lis/les=117/118 n=580 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.305+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.305+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.305+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.305+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x6777380 con 0x507ac00 2026-02-11T23:31:28.305+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:f4b32ae1:::obj130:head)]) -- 0x6767080 con 0x507ac00 2026-02-11T23:31:28.305+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.306+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1201 ==== MOSDPGPush(4.0 119/117 [PushOp(4:f677cfaa:::obj144:head, version: 111'144, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f677cfaa:::obj144:head@111'144, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f0ea0 con 0x507ac00 2026-02-11T23:31:28.306+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:f677cfaa:::obj144:head, version: 111'144, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f677cfaa:::obj144:head@111'144, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 119 2026-02-11T23:31:28.306+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f677cfaa:::obj144:head, version: 111'144, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f677cfaa:::obj144:head@111'144, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.306+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.306+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1202 ==== pg_backfill(progress 4.0 e 119/119 lb 4:f4b32ae1:::obj130:head) ==== 1062+0+0 (unknown 0 0 0) 0x4a2c000 con 0x507ac00 2026-02-11T23:31:28.306+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f677cfaa:::obj144:head, version: 111'144, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f677cfaa:::obj144:head@111'144, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.306+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:f4b32ae1:::obj130:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 119 2026-02-11T23:31:28.306+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f4b32ae1:::obj130:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.306+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.306+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f677cfaa:::obj144:head, version: 111'144, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f677cfaa:::obj144:head@111'144, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.306+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:f677cfaa:::obj144:head, version: 111'144, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f677cfaa:::obj144:head@111'144, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(4.0 119/117 [PushOp(4:f677cfaa:::obj144:head, version: 111'144, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f677cfaa:::obj144:head@111'144, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:f4af07f1:::obj515:head local-lis/les=117/118 n=580 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.306+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.306+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.306+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f4b32ae1:::obj130:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.306+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f4af07f1:::obj515:head local-lis/les=117/118 n=580 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:f677cfaa:::obj144:head, version: 111'144, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f677cfaa:::obj144:head@111'144, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.306+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f4af07f1:::obj515:head local-lis/les=117/118 n=580 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:f677cfaa:::obj144:head@111'144, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.306+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f4af07f1:::obj515:head local-lis/les=117/118 n=580 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:f677cfaa:::obj144:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.306+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f4af07f1:::obj515:head local-lis/les=117/118 n=580 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:f677cfaa:::obj144:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.306+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f4af07f1:::obj515:head local-lis/les=117/118 n=580 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:f677cfaa:::obj144:head 2026-02-11T23:31:28.306+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f4af07f1:::obj515:head local-lis/les=117/118 n=580 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:f677cfaa:::obj144:head v 111'144 2026-02-11T23:31:28.306+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f4af07f1:::obj515:head local-lis/les=117/118 n=580 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.306+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.306+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:f677cfaa:::obj144:head, version: 111'144, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f677cfaa:::obj144:head@111'144, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000069 2026-02-11T23:31:28.306+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f4af07f1:::obj515:head local-lis/les=117/118 n=580 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.306+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.306+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.306+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f4b32ae1:::obj130:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.306+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:f4b32ae1:::obj130:head) v3 prio 15 cost 0 latency 0.000098 pg_backfill(progress 4.0 e 119/119 lb 4:f4b32ae1:::obj130:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:f4af07f1:::obj515:head local-lis/les=117/118 n=580 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.306+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.306+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f4af07f1:::obj515:head local-lis/les=117/118 n=580 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:f4b32ae1:::obj130:head) v3 2026-02-11T23:31:28.306+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f4af07f1:::obj515:head local-lis/les=117/118 n=580 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:f4b32ae1:::obj130:head) 2026-02-11T23:31:28.306+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.306+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:f4b32ae1:::obj130:head) v3 finish latency 0.000019 2026-02-11T23:31:28.306+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.306+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.306+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f4b32ae1:::obj130:head local-lis/les=117/118 n=581 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.306+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.306+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.306+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.306+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x671d380 con 0x507ac00 2026-02-11T23:31:28.306+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:f677cfaa:::obj144:head)]) -- 0x640fb00 con 0x507ac00 2026-02-11T23:31:28.306+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.306+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1203 ==== MOSDPGPush(4.0 119/117 [PushOp(4:f76d37c8:::obj371:head, version: 111'371, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f76d37c8:::obj371:head@111'371, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2d00 con 0x507ac00 2026-02-11T23:31:28.306+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:f76d37c8:::obj371:head, version: 111'371, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f76d37c8:::obj371:head@111'371, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.306+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f76d37c8:::obj371:head, version: 111'371, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f76d37c8:::obj371:head@111'371, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.306+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.306+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1204 ==== pg_backfill(progress 4.0 e 119/119 lb 4:f677cfaa:::obj144:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d98580 con 0x507ac00 2026-02-11T23:31:28.306+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f76d37c8:::obj371:head, version: 111'371, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f76d37c8:::obj371:head@111'371, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.306+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:f677cfaa:::obj144:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 119 2026-02-11T23:31:28.306+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f677cfaa:::obj144:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.306+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.306+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f76d37c8:::obj371:head, version: 111'371, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f76d37c8:::obj371:head@111'371, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.306+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:f76d37c8:::obj371:head, version: 111'371, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f76d37c8:::obj371:head@111'371, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(4.0 119/117 [PushOp(4:f76d37c8:::obj371:head, version: 111'371, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f76d37c8:::obj371:head@111'371, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:f4b32ae1:::obj130:head local-lis/les=117/118 n=581 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.306+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.306+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.306+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f677cfaa:::obj144:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.306+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f4b32ae1:::obj130:head local-lis/les=117/118 n=581 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:f76d37c8:::obj371:head, version: 111'371, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f76d37c8:::obj371:head@111'371, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.306+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f4b32ae1:::obj130:head local-lis/les=117/118 n=581 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:f76d37c8:::obj371:head@111'371, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.306+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f4b32ae1:::obj130:head local-lis/les=117/118 n=581 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:f76d37c8:::obj371:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.306+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f4b32ae1:::obj130:head local-lis/les=117/118 n=581 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:f76d37c8:::obj371:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.306+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f4b32ae1:::obj130:head local-lis/les=117/118 n=581 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:f76d37c8:::obj371:head 2026-02-11T23:31:28.306+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f4b32ae1:::obj130:head local-lis/les=117/118 n=581 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:f76d37c8:::obj371:head v 111'371 2026-02-11T23:31:28.306+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f4b32ae1:::obj130:head local-lis/les=117/118 n=581 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.306+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.306+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:f76d37c8:::obj371:head, version: 111'371, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f76d37c8:::obj371:head@111'371, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:28.306+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f4b32ae1:::obj130:head local-lis/les=117/118 n=581 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.306+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.306+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.306+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f677cfaa:::obj144:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.306+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:f677cfaa:::obj144:head) v3 prio 15 cost 0 latency 0.000092 pg_backfill(progress 4.0 e 119/119 lb 4:f677cfaa:::obj144:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:f4b32ae1:::obj130:head local-lis/les=117/118 n=581 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.306+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.306+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f4b32ae1:::obj130:head local-lis/les=117/118 n=581 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:f677cfaa:::obj144:head) v3 2026-02-11T23:31:28.306+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f4b32ae1:::obj130:head local-lis/les=117/118 n=581 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:f677cfaa:::obj144:head) 2026-02-11T23:31:28.306+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.306+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:f677cfaa:::obj144:head) v3 finish latency 0.000022 2026-02-11T23:31:28.306+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.306+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.306+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f677cfaa:::obj144:head local-lis/les=117/118 n=582 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.306+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.306+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.306+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.306+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x6776c00 con 0x507ac00 2026-02-11T23:31:28.306+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:f76d37c8:::obj371:head)]) -- 0x6776f00 con 0x507ac00 2026-02-11T23:31:28.306+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.306+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.306+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.307+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1205 ==== MOSDPGPush(4.0 119/117 [PushOp(4:f77ac86f:::obj198:head, version: 111'198, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f77ac86f:::obj198:head@111'198, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x555cea0 con 0x507ac00 2026-02-11T23:31:28.307+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:f77ac86f:::obj198:head, version: 111'198, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f77ac86f:::obj198:head@111'198, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 119 2026-02-11T23:31:28.307+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f77ac86f:::obj198:head, version: 111'198, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f77ac86f:::obj198:head@111'198, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.307+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.307+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1206 ==== pg_backfill(progress 4.0 e 119/119 lb 4:f76d37c8:::obj371:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9b700 con 0x507ac00 2026-02-11T23:31:28.307+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f77ac86f:::obj198:head, version: 111'198, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f77ac86f:::obj198:head@111'198, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.307+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:f76d37c8:::obj371:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 119 2026-02-11T23:31:28.307+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f76d37c8:::obj371:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.307+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.307+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f77ac86f:::obj198:head, version: 111'198, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f77ac86f:::obj198:head@111'198, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.307+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:f77ac86f:::obj198:head, version: 111'198, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f77ac86f:::obj198:head@111'198, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(4.0 119/117 [PushOp(4:f77ac86f:::obj198:head, version: 111'198, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f77ac86f:::obj198:head@111'198, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:f677cfaa:::obj144:head local-lis/les=117/118 n=582 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.307+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.307+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.307+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f76d37c8:::obj371:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.307+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f677cfaa:::obj144:head local-lis/les=117/118 n=582 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:f77ac86f:::obj198:head, version: 111'198, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f77ac86f:::obj198:head@111'198, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.307+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f677cfaa:::obj144:head local-lis/les=117/118 n=582 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:f77ac86f:::obj198:head@111'198, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.307+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f677cfaa:::obj144:head local-lis/les=117/118 n=582 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:f77ac86f:::obj198:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.307+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f677cfaa:::obj144:head local-lis/les=117/118 n=582 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:f77ac86f:::obj198:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.307+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f677cfaa:::obj144:head local-lis/les=117/118 n=582 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:f77ac86f:::obj198:head 2026-02-11T23:31:28.307+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f677cfaa:::obj144:head local-lis/les=117/118 n=582 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:f77ac86f:::obj198:head v 111'198 2026-02-11T23:31:28.307+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f677cfaa:::obj144:head local-lis/les=117/118 n=582 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.307+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.307+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:f77ac86f:::obj198:head, version: 111'198, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f77ac86f:::obj198:head@111'198, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:28.307+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.307+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.307+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f76d37c8:::obj371:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.307+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:f76d37c8:::obj371:head) v3 prio 15 cost 0 latency 0.000087 pg_backfill(progress 4.0 e 119/119 lb 4:f76d37c8:::obj371:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:f677cfaa:::obj144:head local-lis/les=117/118 n=582 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.307+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.307+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f677cfaa:::obj144:head local-lis/les=117/118 n=582 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:f76d37c8:::obj371:head) v3 2026-02-11T23:31:28.307+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f677cfaa:::obj144:head local-lis/les=117/118 n=582 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:f76d37c8:::obj371:head) 2026-02-11T23:31:28.307+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.307+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:f76d37c8:::obj371:head) v3 finish latency 0.000014 2026-02-11T23:31:28.307+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f76d37c8:::obj371:head local-lis/les=117/118 n=583 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.307+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.307+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.307+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f76d37c8:::obj371:head local-lis/les=117/118 n=583 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.307+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.307+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.307+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.307+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x674b500 con 0x507ac00 2026-02-11T23:31:28.307+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:f77ac86f:::obj198:head)]) -- 0x6767200 con 0x507ac00 2026-02-11T23:31:28.307+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.307+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1207 ==== MOSDPGPush(4.0 119/117 [PushOp(4:f7c5b032:::obj82:head, version: 111'82, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f7c5b032:::obj82:head@111'82, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x56cad00 con 0x507ac00 2026-02-11T23:31:28.307+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:f7c5b032:::obj82:head, version: 111'82, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f7c5b032:::obj82:head@111'82, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 119 2026-02-11T23:31:28.307+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f7c5b032:::obj82:head, version: 111'82, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f7c5b032:::obj82:head@111'82, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.307+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.307+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f7c5b032:::obj82:head, version: 111'82, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f7c5b032:::obj82:head@111'82, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.307+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1208 ==== pg_backfill(progress 4.0 e 119/119 lb 4:f77ac86f:::obj198:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99b80 con 0x507ac00 2026-02-11T23:31:28.307+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:f77ac86f:::obj198:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 119 2026-02-11T23:31:28.307+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f77ac86f:::obj198:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.307+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.307+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f7c5b032:::obj82:head, version: 111'82, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f7c5b032:::obj82:head@111'82, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.307+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.307+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:f7c5b032:::obj82:head, version: 111'82, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f7c5b032:::obj82:head@111'82, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 119/117 [PushOp(4:f7c5b032:::obj82:head, version: 111'82, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f7c5b032:::obj82:head@111'82, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:f76d37c8:::obj371:head local-lis/les=117/118 n=583 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.307+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.307+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.307+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f77ac86f:::obj198:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.307+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f76d37c8:::obj371:head local-lis/les=117/118 n=583 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:f7c5b032:::obj82:head, version: 111'82, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f7c5b032:::obj82:head@111'82, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.307+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f76d37c8:::obj371:head local-lis/les=117/118 n=583 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:f7c5b032:::obj82:head@111'82, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.307+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f76d37c8:::obj371:head local-lis/les=117/118 n=583 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:f7c5b032:::obj82:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.307+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f76d37c8:::obj371:head local-lis/les=117/118 n=583 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:f7c5b032:::obj82:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.307+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f76d37c8:::obj371:head local-lis/les=117/118 n=583 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:f7c5b032:::obj82:head 2026-02-11T23:31:28.307+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f76d37c8:::obj371:head local-lis/les=117/118 n=583 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:f7c5b032:::obj82:head v 111'82 2026-02-11T23:31:28.307+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f76d37c8:::obj371:head local-lis/les=117/118 n=583 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.307+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.307+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:f7c5b032:::obj82:head, version: 111'82, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f7c5b032:::obj82:head@111'82, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:28.307+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.307+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.307+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f77ac86f:::obj198:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.308+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:f77ac86f:::obj198:head) v3 prio 15 cost 0 latency 0.000086 pg_backfill(progress 4.0 e 119/119 lb 4:f77ac86f:::obj198:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:f76d37c8:::obj371:head local-lis/les=117/118 n=583 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.308+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.308+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f76d37c8:::obj371:head local-lis/les=117/118 n=583 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:f77ac86f:::obj198:head) v3 2026-02-11T23:31:28.308+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f76d37c8:::obj371:head local-lis/les=117/118 n=583 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:f77ac86f:::obj198:head) 2026-02-11T23:31:28.308+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.308+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:f77ac86f:::obj198:head) v3 finish latency 0.000018 2026-02-11T23:31:28.308+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f77ac86f:::obj198:head local-lis/les=117/118 n=584 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.308+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.308+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.308+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f77ac86f:::obj198:head local-lis/les=117/118 n=584 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.308+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.308+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.308+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.308+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x6776a80 con 0x507ac00 2026-02-11T23:31:28.308+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:f7c5b032:::obj82:head)]) -- 0x6767500 con 0x507ac00 2026-02-11T23:31:28.308+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.308+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1209 ==== MOSDPGPush(4.0 119/117 [PushOp(4:f7eaa9b8:::obj104:head, version: 111'104, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f7eaa9b8:::obj104:head@111'104, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3815a00 con 0x507ac00 2026-02-11T23:31:28.308+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:f7eaa9b8:::obj104:head, version: 111'104, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f7eaa9b8:::obj104:head@111'104, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 119 2026-02-11T23:31:28.308+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f7eaa9b8:::obj104:head, version: 111'104, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f7eaa9b8:::obj104:head@111'104, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.308+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.308+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1210 ==== pg_backfill(progress 4.0 e 119/119 lb 4:f7c5b032:::obj82:head) ==== 1061+0+0 (unknown 0 0 0) 0x5d9ac00 con 0x507ac00 2026-02-11T23:31:28.308+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f7eaa9b8:::obj104:head, version: 111'104, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f7eaa9b8:::obj104:head@111'104, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.308+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:f7c5b032:::obj82:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 119 2026-02-11T23:31:28.308+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f7c5b032:::obj82:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.308+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.308+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f7eaa9b8:::obj104:head, version: 111'104, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f7eaa9b8:::obj104:head@111'104, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.308+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:f7eaa9b8:::obj104:head, version: 111'104, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f7eaa9b8:::obj104:head@111'104, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(4.0 119/117 [PushOp(4:f7eaa9b8:::obj104:head, version: 111'104, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f7eaa9b8:::obj104:head@111'104, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:f77ac86f:::obj198:head local-lis/les=117/118 n=584 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.308+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.308+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.308+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f7c5b032:::obj82:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.308+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f77ac86f:::obj198:head local-lis/les=117/118 n=584 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:f7eaa9b8:::obj104:head, version: 111'104, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f7eaa9b8:::obj104:head@111'104, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.308+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f77ac86f:::obj198:head local-lis/les=117/118 n=584 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:f7eaa9b8:::obj104:head@111'104, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.308+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f77ac86f:::obj198:head local-lis/les=117/118 n=584 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:f7eaa9b8:::obj104:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.308+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f77ac86f:::obj198:head local-lis/les=117/118 n=584 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:f7eaa9b8:::obj104:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.308+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f77ac86f:::obj198:head local-lis/les=117/118 n=584 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:f7eaa9b8:::obj104:head 2026-02-11T23:31:28.308+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f77ac86f:::obj198:head local-lis/les=117/118 n=584 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:f7eaa9b8:::obj104:head v 111'104 2026-02-11T23:31:28.308+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f77ac86f:::obj198:head local-lis/les=117/118 n=584 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.308+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.308+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:f7eaa9b8:::obj104:head, version: 111'104, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f7eaa9b8:::obj104:head@111'104, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:28.308+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f77ac86f:::obj198:head local-lis/les=117/118 n=584 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.308+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.308+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.308+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f7c5b032:::obj82:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.308+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:f7c5b032:::obj82:head) v3 prio 15 cost 0 latency 0.000085 pg_backfill(progress 4.0 e 119/119 lb 4:f7c5b032:::obj82:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:f77ac86f:::obj198:head local-lis/les=117/118 n=584 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.308+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.308+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f77ac86f:::obj198:head local-lis/les=117/118 n=584 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:f7c5b032:::obj82:head) v3 2026-02-11T23:31:28.308+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f77ac86f:::obj198:head local-lis/les=117/118 n=584 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:f7c5b032:::obj82:head) 2026-02-11T23:31:28.308+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.308+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:f7c5b032:::obj82:head) v3 finish latency 0.000013 2026-02-11T23:31:28.308+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.308+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.308+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f7c5b032:::obj82:head local-lis/les=117/118 n=585 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.308+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.308+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.308+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.308+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x674b680 con 0x507ac00 2026-02-11T23:31:28.308+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:f7eaa9b8:::obj104:head)]) -- 0x5d55200 con 0x507ac00 2026-02-11T23:31:28.308+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.308+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1211 ==== MOSDPGPush(4.0 119/117 [PushOp(4:f948044a:::obj145:head, version: 111'145, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f948044a:::obj145:head@111'145, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3815380 con 0x507ac00 2026-02-11T23:31:28.308+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:f948044a:::obj145:head, version: 111'145, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f948044a:::obj145:head@111'145, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.308+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f948044a:::obj145:head, version: 111'145, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f948044a:::obj145:head@111'145, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.309+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.309+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1212 ==== pg_backfill(progress 4.0 e 119/119 lb 4:f7eaa9b8:::obj104:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176b00 con 0x507ac00 2026-02-11T23:31:28.309+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f948044a:::obj145:head, version: 111'145, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f948044a:::obj145:head@111'145, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.309+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:f7eaa9b8:::obj104:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.309+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f7eaa9b8:::obj104:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.309+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.309+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:f948044a:::obj145:head, version: 111'145, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f948044a:::obj145:head@111'145, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.309+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:f948044a:::obj145:head, version: 111'145, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f948044a:::obj145:head@111'145, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(4.0 119/117 [PushOp(4:f948044a:::obj145:head, version: 111'145, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f948044a:::obj145:head@111'145, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:f7c5b032:::obj82:head local-lis/les=117/118 n=585 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.309+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.309+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.309+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f7eaa9b8:::obj104:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.309+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f7c5b032:::obj82:head local-lis/les=117/118 n=585 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:f948044a:::obj145:head, version: 111'145, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f948044a:::obj145:head@111'145, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.309+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f7c5b032:::obj82:head local-lis/les=117/118 n=585 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:f948044a:::obj145:head@111'145, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.309+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f7c5b032:::obj82:head local-lis/les=117/118 n=585 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:f948044a:::obj145:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.309+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f7c5b032:::obj82:head local-lis/les=117/118 n=585 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:f948044a:::obj145:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.309+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f7c5b032:::obj82:head local-lis/les=117/118 n=585 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:f948044a:::obj145:head 2026-02-11T23:31:28.309+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f7c5b032:::obj82:head local-lis/les=117/118 n=585 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:f948044a:::obj145:head v 111'145 2026-02-11T23:31:28.309+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f7c5b032:::obj82:head local-lis/les=117/118 n=585 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.309+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.309+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:f948044a:::obj145:head, version: 111'145, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f948044a:::obj145:head@111'145, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:28.309+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.309+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.309+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f7eaa9b8:::obj104:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.309+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:f7eaa9b8:::obj104:head) v3 prio 15 cost 0 latency 0.000079 pg_backfill(progress 4.0 e 119/119 lb 4:f7eaa9b8:::obj104:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:f7c5b032:::obj82:head local-lis/les=117/118 n=585 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.309+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.309+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f7c5b032:::obj82:head local-lis/les=117/118 n=585 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:f7eaa9b8:::obj104:head) v3 2026-02-11T23:31:28.309+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f7c5b032:::obj82:head local-lis/les=117/118 n=585 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:f7eaa9b8:::obj104:head) 2026-02-11T23:31:28.309+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.309+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:f7eaa9b8:::obj104:head) v3 finish latency 0.000013 2026-02-11T23:31:28.309+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f7eaa9b8:::obj104:head local-lis/les=117/118 n=586 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.309+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.309+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.309+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f7eaa9b8:::obj104:head local-lis/les=117/118 n=586 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.309+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.309+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.309+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.309+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x6766a80 con 0x507ac00 2026-02-11T23:31:28.309+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:f948044a:::obj145:head)]) -- 0x6789b00 con 0x507ac00 2026-02-11T23:31:28.309+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.309+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1213 ==== MOSDPGPush(4.0 119/117 [PushOp(4:fa02d04b:::obj266:head, version: 111'266, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fa02d04b:::obj266:head@111'266, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e26b60 con 0x507ac00 2026-02-11T23:31:28.309+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:fa02d04b:::obj266:head, version: 111'266, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fa02d04b:::obj266:head@111'266, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 119 2026-02-11T23:31:28.309+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:fa02d04b:::obj266:head, version: 111'266, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fa02d04b:::obj266:head@111'266, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.309+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.309+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1214 ==== pg_backfill(progress 4.0 e 119/119 lb 4:f948044a:::obj145:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99600 con 0x507ac00 2026-02-11T23:31:28.309+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:f948044a:::obj145:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.309+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f948044a:::obj145:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.309+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:fa02d04b:::obj266:head, version: 111'266, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fa02d04b:::obj266:head@111'266, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.309+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.309+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:fa02d04b:::obj266:head, version: 111'266, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fa02d04b:::obj266:head@111'266, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.309+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:fa02d04b:::obj266:head, version: 111'266, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fa02d04b:::obj266:head@111'266, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(4.0 119/117 [PushOp(4:fa02d04b:::obj266:head, version: 111'266, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fa02d04b:::obj266:head@111'266, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:f7eaa9b8:::obj104:head local-lis/les=117/118 n=586 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.309+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.309+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.309+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f948044a:::obj145:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.309+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f7eaa9b8:::obj104:head local-lis/les=117/118 n=586 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:fa02d04b:::obj266:head, version: 111'266, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fa02d04b:::obj266:head@111'266, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.309+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f7eaa9b8:::obj104:head local-lis/les=117/118 n=586 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:fa02d04b:::obj266:head@111'266, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.309+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f7eaa9b8:::obj104:head local-lis/les=117/118 n=586 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:fa02d04b:::obj266:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.309+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f7eaa9b8:::obj104:head local-lis/les=117/118 n=586 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:fa02d04b:::obj266:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.309+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f7eaa9b8:::obj104:head local-lis/les=117/118 n=586 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:fa02d04b:::obj266:head 2026-02-11T23:31:28.309+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f7eaa9b8:::obj104:head local-lis/les=117/118 n=586 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:fa02d04b:::obj266:head v 111'266 2026-02-11T23:31:28.309+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f7eaa9b8:::obj104:head local-lis/les=117/118 n=586 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.309+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.309+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:fa02d04b:::obj266:head, version: 111'266, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fa02d04b:::obj266:head@111'266, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:28.309+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f7eaa9b8:::obj104:head local-lis/les=117/118 n=586 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.309+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.309+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.309+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:f948044a:::obj145:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.309+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:f948044a:::obj145:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 4.0 e 119/119 lb 4:f948044a:::obj145:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:f7eaa9b8:::obj104:head local-lis/les=117/118 n=586 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.309+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.309+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f7eaa9b8:::obj104:head local-lis/les=117/118 n=586 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:f948044a:::obj145:head) v3 2026-02-11T23:31:28.309+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f7eaa9b8:::obj104:head local-lis/les=117/118 n=586 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:f948044a:::obj145:head) 2026-02-11T23:31:28.309+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.309+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:f948044a:::obj145:head) v3 finish latency 0.000012 2026-02-11T23:31:28.309+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.309+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.309+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f948044a:::obj145:head local-lis/les=117/118 n=587 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.309+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.309+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.309+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.309+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x671d680 con 0x507ac00 2026-02-11T23:31:28.309+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:fa02d04b:::obj266:head)]) -- 0x6777680 con 0x507ac00 2026-02-11T23:31:28.309+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.310+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1215 ==== MOSDPGPush(4.0 119/117 [PushOp(4:fa6c2b5c:::obj438:head, version: 111'438, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fa6c2b5c:::obj438:head@111'438, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2b60 con 0x507ac00 2026-02-11T23:31:28.310+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:fa6c2b5c:::obj438:head, version: 111'438, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fa6c2b5c:::obj438:head@111'438, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 119 2026-02-11T23:31:28.310+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:fa6c2b5c:::obj438:head, version: 111'438, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fa6c2b5c:::obj438:head@111'438, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.310+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.310+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1216 ==== pg_backfill(progress 4.0 e 119/119 lb 4:fa02d04b:::obj266:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d99080 con 0x507ac00 2026-02-11T23:31:28.310+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:fa6c2b5c:::obj438:head, version: 111'438, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fa6c2b5c:::obj438:head@111'438, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.310+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:fa02d04b:::obj266:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 119 2026-02-11T23:31:28.310+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:fa02d04b:::obj266:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.310+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.310+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:fa6c2b5c:::obj438:head, version: 111'438, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fa6c2b5c:::obj438:head@111'438, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.310+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:fa6c2b5c:::obj438:head, version: 111'438, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fa6c2b5c:::obj438:head@111'438, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 119/117 [PushOp(4:fa6c2b5c:::obj438:head, version: 111'438, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fa6c2b5c:::obj438:head@111'438, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:f948044a:::obj145:head local-lis/les=117/118 n=587 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.310+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.310+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.310+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:fa02d04b:::obj266:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.310+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f948044a:::obj145:head local-lis/les=117/118 n=587 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:fa6c2b5c:::obj438:head, version: 111'438, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fa6c2b5c:::obj438:head@111'438, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.310+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f948044a:::obj145:head local-lis/les=117/118 n=587 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:fa6c2b5c:::obj438:head@111'438, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.310+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f948044a:::obj145:head local-lis/les=117/118 n=587 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:fa6c2b5c:::obj438:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.310+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f948044a:::obj145:head local-lis/les=117/118 n=587 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:fa6c2b5c:::obj438:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.310+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f948044a:::obj145:head local-lis/les=117/118 n=587 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:fa6c2b5c:::obj438:head 2026-02-11T23:31:28.310+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f948044a:::obj145:head local-lis/les=117/118 n=587 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:fa6c2b5c:::obj438:head v 111'438 2026-02-11T23:31:28.310+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f948044a:::obj145:head local-lis/les=117/118 n=587 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.310+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.310+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:fa6c2b5c:::obj438:head, version: 111'438, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fa6c2b5c:::obj438:head@111'438, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:28.310+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f948044a:::obj145:head local-lis/les=117/118 n=587 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.310+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.310+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.310+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:fa02d04b:::obj266:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.310+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:fa02d04b:::obj266:head) v3 prio 15 cost 0 latency 0.000086 pg_backfill(progress 4.0 e 119/119 lb 4:fa02d04b:::obj266:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:f948044a:::obj145:head local-lis/les=117/118 n=587 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.310+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.310+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f948044a:::obj145:head local-lis/les=117/118 n=587 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:fa02d04b:::obj266:head) v3 2026-02-11T23:31:28.310+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:f948044a:::obj145:head local-lis/les=117/118 n=587 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:fa02d04b:::obj266:head) 2026-02-11T23:31:28.310+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.310+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:fa02d04b:::obj266:head) v3 finish latency 0.000012 2026-02-11T23:31:28.310+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.310+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.310+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fa02d04b:::obj266:head local-lis/les=117/118 n=588 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.310+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.310+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.310+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.310+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x674b080 con 0x507ac00 2026-02-11T23:31:28.310+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:fa6c2b5c:::obj438:head)]) -- 0x6788000 con 0x507ac00 2026-02-11T23:31:28.310+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.310+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1217 ==== MOSDPGPush(4.0 119/117 [PushOp(4:fb333d49:::obj180:head, version: 111'180, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb333d49:::obj180:head@111'180, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2340 con 0x507ac00 2026-02-11T23:31:28.310+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:fb333d49:::obj180:head, version: 111'180, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb333d49:::obj180:head@111'180, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 119 2026-02-11T23:31:28.310+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:fb333d49:::obj180:head, version: 111'180, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb333d49:::obj180:head@111'180, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.310+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.310+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1218 ==== pg_backfill(progress 4.0 e 119/119 lb 4:fa6c2b5c:::obj438:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2ec00 con 0x507ac00 2026-02-11T23:31:28.310+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:fa6c2b5c:::obj438:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 119 2026-02-11T23:31:28.310+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:fb333d49:::obj180:head, version: 111'180, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb333d49:::obj180:head@111'180, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.310+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:fa6c2b5c:::obj438:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.310+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.310+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:fb333d49:::obj180:head, version: 111'180, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb333d49:::obj180:head@111'180, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.310+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.310+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:fa6c2b5c:::obj438:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.310+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:fb333d49:::obj180:head, version: 111'180, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb333d49:::obj180:head@111'180, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(4.0 119/117 [PushOp(4:fb333d49:::obj180:head, version: 111'180, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb333d49:::obj180:head@111'180, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:fa02d04b:::obj266:head local-lis/les=117/118 n=588 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.310+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.310+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fa02d04b:::obj266:head local-lis/les=117/118 n=588 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:fb333d49:::obj180:head, version: 111'180, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb333d49:::obj180:head@111'180, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.310+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fa02d04b:::obj266:head local-lis/les=117/118 n=588 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:fb333d49:::obj180:head@111'180, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.310+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fa02d04b:::obj266:head local-lis/les=117/118 n=588 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:fb333d49:::obj180:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.310+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fa02d04b:::obj266:head local-lis/les=117/118 n=588 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:fb333d49:::obj180:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.310+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fa02d04b:::obj266:head local-lis/les=117/118 n=588 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:fb333d49:::obj180:head 2026-02-11T23:31:28.310+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fa02d04b:::obj266:head local-lis/les=117/118 n=588 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:fb333d49:::obj180:head v 111'180 2026-02-11T23:31:28.310+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fa02d04b:::obj266:head local-lis/les=117/118 n=588 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.310+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.310+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:fb333d49:::obj180:head, version: 111'180, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb333d49:::obj180:head@111'180, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:28.310+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fa02d04b:::obj266:head local-lis/les=117/118 n=588 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.310+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.310+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.310+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:fa6c2b5c:::obj438:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.310+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:fa6c2b5c:::obj438:head) v3 prio 15 cost 0 latency 0.000080 pg_backfill(progress 4.0 e 119/119 lb 4:fa6c2b5c:::obj438:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:fa02d04b:::obj266:head local-lis/les=117/118 n=588 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.310+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.310+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fa02d04b:::obj266:head local-lis/les=117/118 n=588 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:fa6c2b5c:::obj438:head) v3 2026-02-11T23:31:28.310+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fa02d04b:::obj266:head local-lis/les=117/118 n=588 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:fa6c2b5c:::obj438:head) 2026-02-11T23:31:28.310+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.310+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:fa6c2b5c:::obj438:head) v3 finish latency 0.000012 2026-02-11T23:31:28.310+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.310+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.310+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fa6c2b5c:::obj438:head local-lis/les=117/118 n=589 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.310+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.310+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.310+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.310+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x6777c80 con 0x507ac00 2026-02-11T23:31:28.310+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:fb333d49:::obj180:head)]) -- 0x66d9080 con 0x507ac00 2026-02-11T23:31:28.310+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.311+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1219 ==== MOSDPGPush(4.0 119/117 [PushOp(4:fb44f156:::obj279:head, version: 111'279, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb44f156:::obj279:head@111'279, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5646d00 con 0x507ac00 2026-02-11T23:31:28.311+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:fb44f156:::obj279:head, version: 111'279, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb44f156:::obj279:head@111'279, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.311+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:fb44f156:::obj279:head, version: 111'279, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb44f156:::obj279:head@111'279, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.311+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.311+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1220 ==== pg_backfill(progress 4.0 e 119/119 lb 4:fb333d49:::obj180:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9b180 con 0x507ac00 2026-02-11T23:31:28.311+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:fb44f156:::obj279:head, version: 111'279, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb44f156:::obj279:head@111'279, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.311+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:fb333d49:::obj180:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 119 2026-02-11T23:31:28.311+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:fb333d49:::obj180:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.311+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.311+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:fb44f156:::obj279:head, version: 111'279, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb44f156:::obj279:head@111'279, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.311+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:fb44f156:::obj279:head, version: 111'279, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb44f156:::obj279:head@111'279, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 119/117 [PushOp(4:fb44f156:::obj279:head, version: 111'279, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb44f156:::obj279:head@111'279, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:fa6c2b5c:::obj438:head local-lis/les=117/118 n=589 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.311+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.311+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.311+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:fb333d49:::obj180:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.311+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fa6c2b5c:::obj438:head local-lis/les=117/118 n=589 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:fb44f156:::obj279:head, version: 111'279, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb44f156:::obj279:head@111'279, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.311+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fa6c2b5c:::obj438:head local-lis/les=117/118 n=589 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:fb44f156:::obj279:head@111'279, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.311+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fa6c2b5c:::obj438:head local-lis/les=117/118 n=589 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:fb44f156:::obj279:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.311+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fa6c2b5c:::obj438:head local-lis/les=117/118 n=589 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:fb44f156:::obj279:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.311+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fa6c2b5c:::obj438:head local-lis/les=117/118 n=589 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:fb44f156:::obj279:head 2026-02-11T23:31:28.311+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fa6c2b5c:::obj438:head local-lis/les=117/118 n=589 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:fb44f156:::obj279:head v 111'279 2026-02-11T23:31:28.311+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fa6c2b5c:::obj438:head local-lis/les=117/118 n=589 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.311+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.311+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:fb44f156:::obj279:head, version: 111'279, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb44f156:::obj279:head@111'279, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:28.311+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fa6c2b5c:::obj438:head local-lis/les=117/118 n=589 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.311+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.311+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.311+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:fb333d49:::obj180:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.311+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:fb333d49:::obj180:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 4.0 e 119/119 lb 4:fb333d49:::obj180:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:fa6c2b5c:::obj438:head local-lis/les=117/118 n=589 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.311+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.311+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fa6c2b5c:::obj438:head local-lis/les=117/118 n=589 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:fb333d49:::obj180:head) v3 2026-02-11T23:31:28.311+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fa6c2b5c:::obj438:head local-lis/les=117/118 n=589 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:fb333d49:::obj180:head) 2026-02-11T23:31:28.311+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.311+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:fb333d49:::obj180:head) v3 finish latency 0.000010 2026-02-11T23:31:28.311+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.311+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.311+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fb333d49:::obj180:head local-lis/les=117/118 n=590 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.311+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.311+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.311+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.311+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x6795800 con 0x507ac00 2026-02-11T23:31:28.311+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:fb44f156:::obj279:head)]) -- 0x6795b00 con 0x507ac00 2026-02-11T23:31:28.311+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.311+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1221 ==== MOSDPGPush(4.0 119/117 [PushOp(4:fb8f5c57:::obj248:head, version: 111'248, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb8f5c57:::obj248:head@111'248, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56ca680 con 0x507ac00 2026-02-11T23:31:28.311+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:fb8f5c57:::obj248:head, version: 111'248, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb8f5c57:::obj248:head@111'248, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.311+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:fb8f5c57:::obj248:head, version: 111'248, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb8f5c57:::obj248:head@111'248, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.311+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.311+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1222 ==== pg_backfill(progress 4.0 e 119/119 lb 4:fb44f156:::obj279:head) ==== 1062+0+0 (unknown 0 0 0) 0x6179180 con 0x507ac00 2026-02-11T23:31:28.311+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:fb44f156:::obj279:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.311+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:fb8f5c57:::obj248:head, version: 111'248, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb8f5c57:::obj248:head@111'248, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.311+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:fb44f156:::obj279:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.311+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.311+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:fb8f5c57:::obj248:head, version: 111'248, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb8f5c57:::obj248:head@111'248, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.311+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.311+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:fb44f156:::obj279:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.311+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:fb8f5c57:::obj248:head, version: 111'248, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb8f5c57:::obj248:head@111'248, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 119/117 [PushOp(4:fb8f5c57:::obj248:head, version: 111'248, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb8f5c57:::obj248:head@111'248, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:fb333d49:::obj180:head local-lis/les=117/118 n=590 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.311+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.311+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fb333d49:::obj180:head local-lis/les=117/118 n=590 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:fb8f5c57:::obj248:head, version: 111'248, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb8f5c57:::obj248:head@111'248, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.311+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fb333d49:::obj180:head local-lis/les=117/118 n=590 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:fb8f5c57:::obj248:head@111'248, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.311+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fb333d49:::obj180:head local-lis/les=117/118 n=590 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:fb8f5c57:::obj248:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.311+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fb333d49:::obj180:head local-lis/les=117/118 n=590 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:fb8f5c57:::obj248:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.311+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fb333d49:::obj180:head local-lis/les=117/118 n=590 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:fb8f5c57:::obj248:head 2026-02-11T23:31:28.311+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fb333d49:::obj180:head local-lis/les=117/118 n=590 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:fb8f5c57:::obj248:head v 111'248 2026-02-11T23:31:28.311+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fb333d49:::obj180:head local-lis/les=117/118 n=590 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.311+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.311+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:fb8f5c57:::obj248:head, version: 111'248, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb8f5c57:::obj248:head@111'248, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:28.311+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.311+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.311+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:fb44f156:::obj279:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.311+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:fb44f156:::obj279:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 119/119 lb 4:fb44f156:::obj279:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:fb333d49:::obj180:head local-lis/les=117/118 n=590 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.311+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.311+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fb333d49:::obj180:head local-lis/les=117/118 n=590 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:fb44f156:::obj279:head) v3 2026-02-11T23:31:28.311+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fb333d49:::obj180:head local-lis/les=117/118 n=590 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:fb44f156:::obj279:head) 2026-02-11T23:31:28.311+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.311+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:fb44f156:::obj279:head) v3 finish latency 0.000010 2026-02-11T23:31:28.311+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fb44f156:::obj279:head local-lis/les=117/118 n=591 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.311+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.311+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.311+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fb44f156:::obj279:head local-lis/les=117/118 n=591 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.311+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.311+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.311+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.311+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x6776780 con 0x507ac00 2026-02-11T23:31:28.311+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:fb8f5c57:::obj248:head)]) -- 0x6789980 con 0x507ac00 2026-02-11T23:31:28.311+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.312+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1223 ==== MOSDPGPush(4.0 119/117 [PushOp(4:fcc34b3b:::obj191:head, version: 111'191, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fcc34b3b:::obj191:head@111'191, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x55f09c0 con 0x507ac00 2026-02-11T23:31:28.312+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:fcc34b3b:::obj191:head, version: 111'191, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fcc34b3b:::obj191:head@111'191, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.312+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:fcc34b3b:::obj191:head, version: 111'191, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fcc34b3b:::obj191:head@111'191, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.312+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.312+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1224 ==== pg_backfill(progress 4.0 e 119/119 lb 4:fb8f5c57:::obj248:head) ==== 1062+0+0 (unknown 0 0 0) 0x6178100 con 0x507ac00 2026-02-11T23:31:28.312+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:fcc34b3b:::obj191:head, version: 111'191, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fcc34b3b:::obj191:head@111'191, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.312+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:fb8f5c57:::obj248:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.312+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:fb8f5c57:::obj248:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.312+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.312+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:fcc34b3b:::obj191:head, version: 111'191, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fcc34b3b:::obj191:head@111'191, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.312+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:fcc34b3b:::obj191:head, version: 111'191, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fcc34b3b:::obj191:head@111'191, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 119/117 [PushOp(4:fcc34b3b:::obj191:head, version: 111'191, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fcc34b3b:::obj191:head@111'191, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:fb44f156:::obj279:head local-lis/les=117/118 n=591 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.312+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.312+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.312+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:fb8f5c57:::obj248:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.312+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fb44f156:::obj279:head local-lis/les=117/118 n=591 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:fcc34b3b:::obj191:head, version: 111'191, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fcc34b3b:::obj191:head@111'191, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.312+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fb44f156:::obj279:head local-lis/les=117/118 n=591 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:fcc34b3b:::obj191:head@111'191, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.312+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fb44f156:::obj279:head local-lis/les=117/118 n=591 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:fcc34b3b:::obj191:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.312+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fb44f156:::obj279:head local-lis/les=117/118 n=591 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:fcc34b3b:::obj191:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.312+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fb44f156:::obj279:head local-lis/les=117/118 n=591 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:fcc34b3b:::obj191:head 2026-02-11T23:31:28.312+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fb44f156:::obj279:head local-lis/les=117/118 n=591 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:fcc34b3b:::obj191:head v 111'191 2026-02-11T23:31:28.312+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fb44f156:::obj279:head local-lis/les=117/118 n=591 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.312+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.312+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:fcc34b3b:::obj191:head, version: 111'191, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fcc34b3b:::obj191:head@111'191, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:28.312+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.312+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.312+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:fb8f5c57:::obj248:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.312+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:fb8f5c57:::obj248:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 4.0 e 119/119 lb 4:fb8f5c57:::obj248:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:fb44f156:::obj279:head local-lis/les=117/118 n=591 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.312+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.312+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fb44f156:::obj279:head local-lis/les=117/118 n=591 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:fb8f5c57:::obj248:head) v3 2026-02-11T23:31:28.312+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fb44f156:::obj279:head local-lis/les=117/118 n=591 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:fb8f5c57:::obj248:head) 2026-02-11T23:31:28.312+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.312+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:fb8f5c57:::obj248:head) v3 finish latency 0.000010 2026-02-11T23:31:28.312+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fb8f5c57:::obj248:head local-lis/les=117/118 n=592 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.312+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.312+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.312+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fb8f5c57:::obj248:head local-lis/les=117/118 n=592 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.312+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.312+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.312+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.312+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x6776480 con 0x507ac00 2026-02-11T23:31:28.312+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:fcc34b3b:::obj191:head)]) -- 0x6789800 con 0x507ac00 2026-02-11T23:31:28.312+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.312+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1225 ==== MOSDPGPush(4.0 119/117 [PushOp(4:fcdeb4b1:::obj379:head, version: 111'379, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fcdeb4b1:::obj379:head@111'379, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a29c0 con 0x507ac00 2026-02-11T23:31:28.312+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:fcdeb4b1:::obj379:head, version: 111'379, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fcdeb4b1:::obj379:head@111'379, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 119 2026-02-11T23:31:28.312+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:fcdeb4b1:::obj379:head, version: 111'379, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fcdeb4b1:::obj379:head@111'379, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.312+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.312+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1226 ==== pg_backfill(progress 4.0 e 119/119 lb 4:fcc34b3b:::obj191:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d2c580 con 0x507ac00 2026-02-11T23:31:28.312+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:fcc34b3b:::obj191:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.312+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:fcdeb4b1:::obj379:head, version: 111'379, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fcdeb4b1:::obj379:head@111'379, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.312+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:fcc34b3b:::obj191:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.312+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.312+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:fcdeb4b1:::obj379:head, version: 111'379, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fcdeb4b1:::obj379:head@111'379, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.312+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.312+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:fcc34b3b:::obj191:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.312+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:fcdeb4b1:::obj379:head, version: 111'379, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fcdeb4b1:::obj379:head@111'379, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 119/117 [PushOp(4:fcdeb4b1:::obj379:head, version: 111'379, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fcdeb4b1:::obj379:head@111'379, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:fb8f5c57:::obj248:head local-lis/les=117/118 n=592 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.312+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.312+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fb8f5c57:::obj248:head local-lis/les=117/118 n=592 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:fcdeb4b1:::obj379:head, version: 111'379, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fcdeb4b1:::obj379:head@111'379, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.312+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fb8f5c57:::obj248:head local-lis/les=117/118 n=592 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:fcdeb4b1:::obj379:head@111'379, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.312+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fb8f5c57:::obj248:head local-lis/les=117/118 n=592 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:fcdeb4b1:::obj379:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.312+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fb8f5c57:::obj248:head local-lis/les=117/118 n=592 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:fcdeb4b1:::obj379:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.312+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fb8f5c57:::obj248:head local-lis/les=117/118 n=592 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:fcdeb4b1:::obj379:head 2026-02-11T23:31:28.312+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fb8f5c57:::obj248:head local-lis/les=117/118 n=592 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:fcdeb4b1:::obj379:head v 111'379 2026-02-11T23:31:28.312+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fb8f5c57:::obj248:head local-lis/les=117/118 n=592 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.312+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.312+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:fcdeb4b1:::obj379:head, version: 111'379, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fcdeb4b1:::obj379:head@111'379, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:28.312+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fb8f5c57:::obj248:head local-lis/les=117/118 n=592 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.312+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.312+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.312+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:fcc34b3b:::obj191:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.312+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:fcc34b3b:::obj191:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 4.0 e 119/119 lb 4:fcc34b3b:::obj191:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:fb8f5c57:::obj248:head local-lis/les=117/118 n=592 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.312+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.312+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fb8f5c57:::obj248:head local-lis/les=117/118 n=592 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:fcc34b3b:::obj191:head) v3 2026-02-11T23:31:28.312+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fb8f5c57:::obj248:head local-lis/les=117/118 n=592 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:fcc34b3b:::obj191:head) 2026-02-11T23:31:28.312+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.312+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:fcc34b3b:::obj191:head) v3 finish latency 0.000010 2026-02-11T23:31:28.312+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.312+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.312+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fcc34b3b:::obj191:head local-lis/les=117/118 n=593 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.313+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.313+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.313+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.313+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x671dc80 con 0x507ac00 2026-02-11T23:31:28.313+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:fcdeb4b1:::obj379:head)]) -- 0x674b980 con 0x507ac00 2026-02-11T23:31:28.313+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.313+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1227 ==== MOSDPGPush(4.0 119/117 [PushOp(4:fd91bcb6:::obj528:head, version: 111'528, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fd91bcb6:::obj528:head@111'528, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5e26b60 con 0x507ac00 2026-02-11T23:31:28.313+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:fd91bcb6:::obj528:head, version: 111'528, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fd91bcb6:::obj528:head@111'528, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.313+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:fd91bcb6:::obj528:head, version: 111'528, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fd91bcb6:::obj528:head@111'528, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.313+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.313+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1228 ==== pg_backfill(progress 4.0 e 119/119 lb 4:fcdeb4b1:::obj379:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177b80 con 0x507ac00 2026-02-11T23:31:28.313+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:fd91bcb6:::obj528:head, version: 111'528, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fd91bcb6:::obj528:head@111'528, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.313+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:fcdeb4b1:::obj379:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 119 2026-02-11T23:31:28.313+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:fcdeb4b1:::obj379:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.313+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.313+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:fd91bcb6:::obj528:head, version: 111'528, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fd91bcb6:::obj528:head@111'528, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.313+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.313+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:fd91bcb6:::obj528:head, version: 111'528, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fd91bcb6:::obj528:head@111'528, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 119/117 [PushOp(4:fd91bcb6:::obj528:head, version: 111'528, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fd91bcb6:::obj528:head@111'528, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:fcc34b3b:::obj191:head local-lis/les=117/118 n=593 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.313+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.313+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.313+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:fcdeb4b1:::obj379:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.313+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fcc34b3b:::obj191:head local-lis/les=117/118 n=593 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:fd91bcb6:::obj528:head, version: 111'528, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fd91bcb6:::obj528:head@111'528, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.313+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fcc34b3b:::obj191:head local-lis/les=117/118 n=593 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:fd91bcb6:::obj528:head@111'528, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.313+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fcc34b3b:::obj191:head local-lis/les=117/118 n=593 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:fd91bcb6:::obj528:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.313+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fcc34b3b:::obj191:head local-lis/les=117/118 n=593 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:fd91bcb6:::obj528:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.313+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fcc34b3b:::obj191:head local-lis/les=117/118 n=593 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:fd91bcb6:::obj528:head 2026-02-11T23:31:28.313+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fcc34b3b:::obj191:head local-lis/les=117/118 n=593 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:fd91bcb6:::obj528:head v 111'528 2026-02-11T23:31:28.313+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fcc34b3b:::obj191:head local-lis/les=117/118 n=593 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.313+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.313+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:fd91bcb6:::obj528:head, version: 111'528, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fd91bcb6:::obj528:head@111'528, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:28.313+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.313+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.313+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:fcdeb4b1:::obj379:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.313+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:fcdeb4b1:::obj379:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 4.0 e 119/119 lb 4:fcdeb4b1:::obj379:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:fcc34b3b:::obj191:head local-lis/les=117/118 n=593 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.313+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.313+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fcc34b3b:::obj191:head local-lis/les=117/118 n=593 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:fcdeb4b1:::obj379:head) v3 2026-02-11T23:31:28.313+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fcc34b3b:::obj191:head local-lis/les=117/118 n=593 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:fcdeb4b1:::obj379:head) 2026-02-11T23:31:28.313+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.313+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:fcdeb4b1:::obj379:head) v3 finish latency 0.000010 2026-02-11T23:31:28.313+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fcdeb4b1:::obj379:head local-lis/les=117/118 n=594 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.313+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.313+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fcdeb4b1:::obj379:head local-lis/les=117/118 n=594 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.313+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.313+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.313+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.313+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.313+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x6795200 con 0x507ac00 2026-02-11T23:31:28.313+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:fd91bcb6:::obj528:head)]) -- 0x6789680 con 0x507ac00 2026-02-11T23:31:28.313+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.313+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1229 ==== MOSDPGPush(4.0 119/117 [PushOp(4:fe351e27:::obj9:head, version: 111'9, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fe351e27:::obj9:head@111'9, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 4999+0+0 (unknown 0 0 0) 0x3815380 con 0x507ac00 2026-02-11T23:31:28.313+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:fe351e27:::obj9:head, version: 111'9, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fe351e27:::obj9:head@111'9, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 119 2026-02-11T23:31:28.313+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:fe351e27:::obj9:head, version: 111'9, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fe351e27:::obj9:head@111'9, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.313+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.313+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:fe351e27:::obj9:head, version: 111'9, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fe351e27:::obj9:head@111'9, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.313+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.313+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:fe351e27:::obj9:head, version: 111'9, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fe351e27:::obj9:head@111'9, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.313+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.313+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:fe351e27:::obj9:head, version: 111'9, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fe351e27:::obj9:head@111'9, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 119/117 [PushOp(4:fe351e27:::obj9:head, version: 111'9, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fe351e27:::obj9:head@111'9, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:fcdeb4b1:::obj379:head local-lis/les=117/118 n=594 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.313+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.313+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1230 ==== pg_backfill(progress 4.0 e 119/119 lb 4:fd91bcb6:::obj528:head) ==== 1062+0+0 (unknown 0 0 0) 0x6177600 con 0x507ac00 2026-02-11T23:31:28.313+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:fd91bcb6:::obj528:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 119 2026-02-11T23:31:28.313+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fcdeb4b1:::obj379:head local-lis/les=117/118 n=594 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:fe351e27:::obj9:head, version: 111'9, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fe351e27:::obj9:head@111'9, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.313+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:fd91bcb6:::obj528:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.313+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fcdeb4b1:::obj379:head local-lis/les=117/118 n=594 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:fe351e27:::obj9:head@111'9, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.313+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.313+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:fd91bcb6:::obj528:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.313+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fcdeb4b1:::obj379:head local-lis/les=117/118 n=594 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:fe351e27:::obj9:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.313+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fcdeb4b1:::obj379:head local-lis/les=117/118 n=594 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:fe351e27:::obj9:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.313+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fcdeb4b1:::obj379:head local-lis/les=117/118 n=594 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:fe351e27:::obj9:head 2026-02-11T23:31:28.313+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fcdeb4b1:::obj379:head local-lis/les=117/118 n=594 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:fe351e27:::obj9:head v 111'9 2026-02-11T23:31:28.313+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fcdeb4b1:::obj379:head local-lis/les=117/118 n=594 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.313+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.313+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:fe351e27:::obj9:head, version: 111'9, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fe351e27:::obj9:head@111'9, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:28.313+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.313+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.313+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:fd91bcb6:::obj528:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.313+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:fd91bcb6:::obj528:head) v3 prio 15 cost 0 latency 0.000051 pg_backfill(progress 4.0 e 119/119 lb 4:fd91bcb6:::obj528:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:fcdeb4b1:::obj379:head local-lis/les=117/118 n=594 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.313+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.313+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fcdeb4b1:::obj379:head local-lis/les=117/118 n=594 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:fd91bcb6:::obj528:head) v3 2026-02-11T23:31:28.313+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fcdeb4b1:::obj379:head local-lis/les=117/118 n=594 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:fd91bcb6:::obj528:head) 2026-02-11T23:31:28.313+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.313+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:fd91bcb6:::obj528:head) v3 finish latency 0.000010 2026-02-11T23:31:28.314+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fd91bcb6:::obj528:head local-lis/les=117/118 n=595 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.314+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.314+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fd91bcb6:::obj528:head local-lis/les=117/118 n=595 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.314+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.314+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.314+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.314+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.314+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x6776180 con 0x507ac00 2026-02-11T23:31:28.314+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:fe351e27:::obj9:head)]) -- 0x6789500 con 0x507ac00 2026-02-11T23:31:28.314+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.314+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1231 ==== MOSDPGPush(4.0 119/117 [PushOp(4:fedf1386:::obj565:head, version: 111'565, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fedf1386:::obj565:head@111'565, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3815a00 con 0x507ac00 2026-02-11T23:31:28.314+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:fedf1386:::obj565:head, version: 111'565, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fedf1386:::obj565:head@111'565, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 119 2026-02-11T23:31:28.314+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:fedf1386:::obj565:head, version: 111'565, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fedf1386:::obj565:head@111'565, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.314+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.314+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1232 ==== pg_backfill(progress 4.0 e 119/119 lb 4:fe351e27:::obj9:head) ==== 1060+0+0 (unknown 0 0 0) 0x5985700 con 0x507ac00 2026-02-11T23:31:28.314+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:fedf1386:::obj565:head, version: 111'565, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fedf1386:::obj565:head@111'565, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.314+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:fe351e27:::obj9:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 119 2026-02-11T23:31:28.314+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:fe351e27:::obj9:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.314+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.314+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:fedf1386:::obj565:head, version: 111'565, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fedf1386:::obj565:head@111'565, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.314+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.314+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:fedf1386:::obj565:head, version: 111'565, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fedf1386:::obj565:head@111'565, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(4.0 119/117 [PushOp(4:fedf1386:::obj565:head, version: 111'565, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fedf1386:::obj565:head@111'565, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:fd91bcb6:::obj528:head local-lis/les=117/118 n=595 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.314+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.314+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.314+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:fe351e27:::obj9:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.314+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fd91bcb6:::obj528:head local-lis/les=117/118 n=595 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:fedf1386:::obj565:head, version: 111'565, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fedf1386:::obj565:head@111'565, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.314+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fd91bcb6:::obj528:head local-lis/les=117/118 n=595 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:fedf1386:::obj565:head@111'565, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.314+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fd91bcb6:::obj528:head local-lis/les=117/118 n=595 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:fedf1386:::obj565:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.314+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fd91bcb6:::obj528:head local-lis/les=117/118 n=595 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:fedf1386:::obj565:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.314+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fd91bcb6:::obj528:head local-lis/les=117/118 n=595 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:fedf1386:::obj565:head 2026-02-11T23:31:28.314+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fd91bcb6:::obj528:head local-lis/les=117/118 n=595 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:fedf1386:::obj565:head v 111'565 2026-02-11T23:31:28.314+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fd91bcb6:::obj528:head local-lis/les=117/118 n=595 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.314+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.314+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:fedf1386:::obj565:head, version: 111'565, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fedf1386:::obj565:head@111'565, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:28.314+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.314+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.314+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:fe351e27:::obj9:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.314+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:fe351e27:::obj9:head) v3 prio 15 cost 0 latency 0.000084 pg_backfill(progress 4.0 e 119/119 lb 4:fe351e27:::obj9:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:fd91bcb6:::obj528:head local-lis/les=117/118 n=595 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.314+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.314+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fd91bcb6:::obj528:head local-lis/les=117/118 n=595 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:fe351e27:::obj9:head) v3 2026-02-11T23:31:28.314+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fd91bcb6:::obj528:head local-lis/les=117/118 n=595 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:fe351e27:::obj9:head) 2026-02-11T23:31:28.314+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.314+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:fe351e27:::obj9:head) v3 finish latency 0.000012 2026-02-11T23:31:28.314+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fe351e27:::obj9:head local-lis/les=117/118 n=596 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.314+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.314+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fe351e27:::obj9:head local-lis/les=117/118 n=596 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.314+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.314+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.314+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.314+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.314+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x6794f00 con 0x507ac00 2026-02-11T23:31:28.314+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:fedf1386:::obj565:head)]) -- 0x6789080 con 0x507ac00 2026-02-11T23:31:28.314+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.315+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1233 ==== MOSDPGPush(4.0 119/117 [PushOp(4:ff38ecdd:::obj155:head, version: 111'155, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ff38ecdd:::obj155:head@111'155, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56cad00 con 0x507ac00 2026-02-11T23:31:28.315+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:ff38ecdd:::obj155:head, version: 111'155, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ff38ecdd:::obj155:head@111'155, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 119 2026-02-11T23:31:28.315+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ff38ecdd:::obj155:head, version: 111'155, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ff38ecdd:::obj155:head@111'155, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.315+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.315+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1234 ==== pg_backfill(progress 4.0 e 119/119 lb 4:fedf1386:::obj565:head) ==== 1062+0+0 (unknown 0 0 0) 0x5d9a680 con 0x507ac00 2026-02-11T23:31:28.315+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ff38ecdd:::obj155:head, version: 111'155, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ff38ecdd:::obj155:head@111'155, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.315+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:fedf1386:::obj565:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 119 2026-02-11T23:31:28.315+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:fedf1386:::obj565:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.315+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.315+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ff38ecdd:::obj155:head, version: 111'155, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ff38ecdd:::obj155:head@111'155, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.315+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:ff38ecdd:::obj155:head, version: 111'155, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ff38ecdd:::obj155:head@111'155, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 119/117 [PushOp(4:ff38ecdd:::obj155:head, version: 111'155, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ff38ecdd:::obj155:head@111'155, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:fe351e27:::obj9:head local-lis/les=117/118 n=596 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.315+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.315+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.315+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:fedf1386:::obj565:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.315+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fe351e27:::obj9:head local-lis/les=117/118 n=596 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:ff38ecdd:::obj155:head, version: 111'155, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ff38ecdd:::obj155:head@111'155, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.315+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fe351e27:::obj9:head local-lis/les=117/118 n=596 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:ff38ecdd:::obj155:head@111'155, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.315+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fe351e27:::obj9:head local-lis/les=117/118 n=596 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:ff38ecdd:::obj155:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.315+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fe351e27:::obj9:head local-lis/les=117/118 n=596 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:ff38ecdd:::obj155:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.315+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fe351e27:::obj9:head local-lis/les=117/118 n=596 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:ff38ecdd:::obj155:head 2026-02-11T23:31:28.315+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fe351e27:::obj9:head local-lis/les=117/118 n=596 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:ff38ecdd:::obj155:head v 111'155 2026-02-11T23:31:28.315+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fe351e27:::obj9:head local-lis/les=117/118 n=596 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.315+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.315+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:ff38ecdd:::obj155:head, version: 111'155, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ff38ecdd:::obj155:head@111'155, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:28.315+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.315+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.315+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:fedf1386:::obj565:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.315+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:fedf1386:::obj565:head) v3 prio 15 cost 0 latency 0.000084 pg_backfill(progress 4.0 e 119/119 lb 4:fedf1386:::obj565:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:fe351e27:::obj9:head local-lis/les=117/118 n=596 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.315+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.315+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fe351e27:::obj9:head local-lis/les=117/118 n=596 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:fedf1386:::obj565:head) v3 2026-02-11T23:31:28.315+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fe351e27:::obj9:head local-lis/les=117/118 n=596 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:fedf1386:::obj565:head) 2026-02-11T23:31:28.315+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.315+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:fedf1386:::obj565:head) v3 finish latency 0.000014 2026-02-11T23:31:28.315+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fedf1386:::obj565:head local-lis/les=117/118 n=597 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.315+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.315+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.315+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fedf1386:::obj565:head local-lis/les=117/118 n=597 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.315+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.315+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.315+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.315+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x642cd80 con 0x507ac00 2026-02-11T23:31:28.315+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:ff38ecdd:::obj155:head)]) -- 0x6788c00 con 0x507ac00 2026-02-11T23:31:28.315+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.315+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1235 ==== MOSDPGPush(4.0 119/117 [PushOp(4:ff7b1f36:::obj1:head, version: 111'1, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ff7b1f36:::obj1:head@111'1, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 4999+0+0 (unknown 0 0 0) 0x555cea0 con 0x507ac00 2026-02-11T23:31:28.315+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:ff7b1f36:::obj1:head, version: 111'1, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ff7b1f36:::obj1:head@111'1, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 119 2026-02-11T23:31:28.315+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ff7b1f36:::obj1:head, version: 111'1, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ff7b1f36:::obj1:head@111'1, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.315+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.315+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1236 ==== pg_backfill(progress 4.0 e 119/119 lb 4:ff38ecdd:::obj155:head) ==== 1062+0+0 (unknown 0 0 0) 0x6176000 con 0x507ac00 2026-02-11T23:31:28.315+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ff7b1f36:::obj1:head, version: 111'1, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ff7b1f36:::obj1:head@111'1, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.315+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:ff38ecdd:::obj155:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 119 2026-02-11T23:31:28.315+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:ff38ecdd:::obj155:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.315+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.315+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ff7b1f36:::obj1:head, version: 111'1, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ff7b1f36:::obj1:head@111'1, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.315+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:ff7b1f36:::obj1:head, version: 111'1, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ff7b1f36:::obj1:head@111'1, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 119/117 [PushOp(4:ff7b1f36:::obj1:head, version: 111'1, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ff7b1f36:::obj1:head@111'1, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:fedf1386:::obj565:head local-lis/les=117/118 n=597 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.315+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.315+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.315+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:ff38ecdd:::obj155:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.315+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fedf1386:::obj565:head local-lis/les=117/118 n=597 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:ff7b1f36:::obj1:head, version: 111'1, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ff7b1f36:::obj1:head@111'1, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.315+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fedf1386:::obj565:head local-lis/les=117/118 n=597 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:ff7b1f36:::obj1:head@111'1, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.315+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fedf1386:::obj565:head local-lis/les=117/118 n=597 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:ff7b1f36:::obj1:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.315+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fedf1386:::obj565:head local-lis/les=117/118 n=597 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:ff7b1f36:::obj1:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.315+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fedf1386:::obj565:head local-lis/les=117/118 n=597 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:ff7b1f36:::obj1:head 2026-02-11T23:31:28.315+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fedf1386:::obj565:head local-lis/les=117/118 n=597 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:ff7b1f36:::obj1:head v 111'1 2026-02-11T23:31:28.315+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fedf1386:::obj565:head local-lis/les=117/118 n=597 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.315+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.315+0000 7f414f993640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:ff7b1f36:::obj1:head, version: 111'1, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ff7b1f36:::obj1:head@111'1, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:28.315+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fedf1386:::obj565:head local-lis/les=117/118 n=597 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.315+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.315+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.315+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:ff38ecdd:::obj155:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.315+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:ff38ecdd:::obj155:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 4.0 e 119/119 lb 4:ff38ecdd:::obj155:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:fedf1386:::obj565:head local-lis/les=117/118 n=597 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.315+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.315+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fedf1386:::obj565:head local-lis/les=117/118 n=597 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:ff38ecdd:::obj155:head) v3 2026-02-11T23:31:28.315+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:fedf1386:::obj565:head local-lis/les=117/118 n=597 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:ff38ecdd:::obj155:head) 2026-02-11T23:31:28.315+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.315+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:ff38ecdd:::obj155:head) v3 finish latency 0.000012 2026-02-11T23:31:28.315+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.315+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.315+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ff38ecdd:::obj155:head local-lis/les=117/118 n=598 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.315+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.315+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.315+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.315+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x6767e00 con 0x507ac00 2026-02-11T23:31:28.315+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:ff7b1f36:::obj1:head)]) -- 0x6794d80 con 0x507ac00 2026-02-11T23:31:28.315+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.316+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1237 ==== MOSDPGPush(4.0 119/117 [PushOp(4:ffd8ed3d:::obj316:head, version: 111'316, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ffd8ed3d:::obj316:head@111'316, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x56a2d00 con 0x507ac00 2026-02-11T23:31:28.316+0000 7f416abfa640 15 osd.0 119 enqueue_op MOSDPGPush(4.0 119/117 [PushOp(4:ffd8ed3d:::obj316:head, version: 111'316, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ffd8ed3d:::obj316:head@111'316, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 119 2026-02-11T23:31:28.316+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ffd8ed3d:::obj316:head, version: 111'316, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ffd8ed3d:::obj316:head@111'316, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e119) 2026-02-11T23:31:28.316+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.316+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1238 ==== pg_backfill(progress 4.0 e 119/119 lb 4:ff7b1f36:::obj1:head) ==== 1060+0+0 (unknown 0 0 0) 0x6179700 con 0x507ac00 2026-02-11T23:31:28.316+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ffd8ed3d:::obj316:head, version: 111'316, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ffd8ed3d:::obj316:head@111'316, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) queued 2026-02-11T23:31:28.316+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(progress 4.0 e 119/119 lb 4:ff7b1f36:::obj1:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 119 2026-02-11T23:31:28.316+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.316+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:ff7b1f36:::obj1:head) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.316+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 119/117 [PushOp(4:ffd8ed3d:::obj316:head, version: 111'316, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ffd8ed3d:::obj316:head@111'316, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 19865 cost 5096 e119) pg 0x5978000 2026-02-11T23:31:28.316+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.316+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:ffd8ed3d:::obj316:head, version: 111'316, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ffd8ed3d:::obj316:head@111'316, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 119/117 [PushOp(4:ffd8ed3d:::obj316:head, version: 111'316, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ffd8ed3d:::obj316:head@111'316, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:ff38ecdd:::obj155:head local-lis/les=117/118 n=598 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.316+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.316+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ff38ecdd:::obj155:head local-lis/les=117/118 n=598 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: MOSDPGPush(4.0 119/117 [PushOp(4:ffd8ed3d:::obj316:head, version: 111'316, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ffd8ed3d:::obj316:head@111'316, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:28.316+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.316+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ff38ecdd:::obj155:head local-lis/les=117/118 n=598 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_push ObjectRecoveryInfo(4:ffd8ed3d:::obj316:head@111'316, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:28.316+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:ff7b1f36:::obj1:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.316+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ff38ecdd:::obj155:head local-lis/les=117/118 n=598 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data 4:ffd8ed3d:::obj316:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:28.316+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ff38ecdd:::obj155:head local-lis/les=117/118 n=598 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] submit_push_data recovering object 4:ffd8ed3d:::obj316:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:28.316+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ff38ecdd:::obj155:head local-lis/les=117/118 n=598 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] on_local_recover: 4:ffd8ed3d:::obj316:head 2026-02-11T23:31:28.316+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ff38ecdd:::obj155:head local-lis/les=117/118 n=598 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got got missing 4:ffd8ed3d:::obj316:head v 111'316 2026-02-11T23:31:28.316+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ff38ecdd:::obj155:head local-lis/les=117/118 n=598 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recover_got last_complete now 111'600 log.complete_to at end 2026-02-11T23:31:28.316+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.316+0000 7f414b98b640 10 osd.0 119 dequeue_op MOSDPGPush(4.0 119/117 [PushOp(4:ffd8ed3d:::obj316:head, version: 111'316, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ffd8ed3d:::obj316:head@111'316, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:28.316+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.316+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.316+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 119/119 lb 4:ff7b1f36:::obj1:head) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.316+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:ff7b1f36:::obj1:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 4.0 e 119/119 lb 4:ff7b1f36:::obj1:head) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:ff38ecdd:::obj155:head local-lis/les=117/118 n=598 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.316+0000 7f414f993640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.316+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ff38ecdd:::obj155:head local-lis/les=117/118 n=598 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 119/119 lb 4:ff7b1f36:::obj1:head) v3 2026-02-11T23:31:28.316+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ff38ecdd:::obj155:head local-lis/les=117/118 n=598 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 119/119 lb 4:ff7b1f36:::obj1:head) 2026-02-11T23:31:28.316+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.316+0000 7f414f993640 10 osd.0 119 dequeue_op pg_backfill(progress 4.0 e 119/119 lb 4:ff7b1f36:::obj1:head) v3 finish latency 0.000017 2026-02-11T23:31:28.316+0000 7f414f993640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ff7b1f36:::obj1:head local-lis/les=117/118 n=599 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:28.316+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.316+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ff7b1f36:::obj1:head local-lis/les=117/118 n=599 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::recovery_committed_to version 111'600 now ondisk 2026-02-11T23:31:28.316+0000 7f414f993640 20 osd.0 119 get_con_osd_cluster to osd.4 from_epoch 119 2026-02-11T23:31:28.316+0000 7f414f993640 20 osd.0 119 get_nextmap_reserved map_reservations: {119=1} 2026-02-11T23:31:28.316+0000 7f414f993640 20 osd.0 119 release_map epoch: 119 2026-02-11T23:31:28.316+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.316+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_trim(4.0 111'600 e119/119) -- 0x6794a80 con 0x507ac00 2026-02-11T23:31:28.316+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- MOSDPGPushReply(4.0 119/117 [PushReplyOp(4:ffd8ed3d:::obj316:head)]) -- 0x6788780 con 0x507ac00 2026-02-11T23:31:28.316+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.316+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1239 ==== pg_backfill(finish 4.0 e 119/119 lb MAX) ==== 1098+0+0 (unknown 0 0 0) 0x5d2cb00 con 0x507ac00 2026-02-11T23:31:28.316+0000 7f416abfa640 15 osd.0 119 enqueue_op pg_backfill(finish 4.0 e 119/119 lb MAX) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 119 2026-02-11T23:31:28.316+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(finish 4.0 e 119/119 lb MAX) v3) class_id 0 prio 15 cost 0 e119) 2026-02-11T23:31:28.316+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.316+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(finish 4.0 e 119/119 lb MAX) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) queued 2026-02-11T23:31:28.316+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.316+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(finish 4.0 e 119/119 lb MAX) v3) class_id 0background_recovery prio 15 qos_cost 19865 cost 0 e119) pg 0x5978000 2026-02-11T23:31:28.316+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.316+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(finish 4.0 e 119/119 lb MAX) v3 prio 15 cost 0 latency 0.000018 pg_backfill(finish 4.0 e 119/119 lb MAX) v3 pg pg[4.0( v 111'600 (111'590,111'600] lb 4:ff7b1f36:::obj1:head local-lis/les=117/118 n=599 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] 2026-02-11T23:31:28.316+0000 7f414b98b640 10 osd.0 119 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 119 is not newer than session's projected_epoch of 119 2026-02-11T23:31:28.316+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ff7b1f36:::obj1:head local-lis/les=117/118 n=599 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] _handle_message: pg_backfill(finish 4.0 e 119/119 lb MAX) v3 2026-02-11T23:31:28.316+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] lb 4:ff7b1f36:::obj1:head local-lis/les=117/118 n=599 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_backfill pg_backfill(finish 4.0 e 119/119 lb MAX) 2026-02-11T23:31:28.316+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_backfill(finish_ack 4.0 e 119/119 lb MIN) -- 0x5f84580 con 0x507ac00 2026-02-11T23:31:28.316+0000 7f414b98b640 15 osd.0 119 enqueue_peering_evt 4.0 epoch_sent: 119 epoch_requested: 119 RecoveryDone 2026-02-11T23:31:28.316+0000 7f414b98b640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 119 epoch_requested: 119 RecoveryDone) class_id 2 prio 255 cost 10 e119) 2026-02-11T23:31:28.316+0000 7f414b98b640 20 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=599 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::update_backfill_progress final 2457600 replaces local 2457600 2026-02-11T23:31:28.316+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.316+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.316+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 119 epoch_requested: 119 RecoveryDone) class_id 2immediate prio 255 cost 10 e119) queued 2026-02-11T23:31:28.316+0000 7f414b98b640 10 osd.0 119 dequeue_op pg_backfill(finish 4.0 e 119/119 lb MAX) v3 finish latency 0.000027 2026-02-11T23:31:28.316+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.316+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.316+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 119 epoch_requested: 119 RecoveryDone) class_id 2immediate prio 255 cost 10 e119) pg 0x5978000 2026-02-11T23:31:28.316+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_peering_event: epoch_sent: 119 epoch_requested: 119 RecoveryDone 2026-02-11T23:31:28.316+0000 7f414f993640 5 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] exit Started/ReplicaActive/RepRecovering 0.382716 4 0.000018 2026-02-11T23:31:28.316+0000 7f414f993640 5 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] enter Started/ReplicaActive/RepNotRecovering 2026-02-11T23:31:28.316+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.316+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.316+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.316+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.316+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1240 ==== MBackfillReserve(4.0 RELEASE e119/119) ==== 46+0+0 (unknown 0 0 0) 0x6788780 con 0x507ac00 2026-02-11T23:31:28.316+0000 7f416abfa640 15 osd.0 119 enqueue_peering_evt 4.0 epoch_sent: 119 epoch_requested: 119 RemoteReservationCanceled 2026-02-11T23:31:28.316+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 119 epoch_requested: 119 RemoteReservationCanceled) class_id 2 prio 255 cost 10 e119) 2026-02-11T23:31:28.316+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.316+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 119 epoch_requested: 119 RemoteReservationCanceled) class_id 2immediate prio 255 cost 10 e119) queued 2026-02-11T23:31:28.316+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.316+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 119 epoch_requested: 119 RemoteReservationCanceled) class_id 2immediate prio 255 cost 10 e119) pg 0x5978000 2026-02-11T23:31:28.316+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_peering_event: epoch_sent: 119 epoch_requested: 119 RemoteReservationCanceled 2026-02-11T23:31:28.316+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.316+0000 7f414b98b640 5 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] exit Started/ReplicaActive/RepNotRecovering 0.000096 1 0.000009 2026-02-11T23:31:28.316+0000 7f414b98b640 5 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] enter Started/ReplicaActive/RepNotRecovering 2026-02-11T23:31:28.316+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.316+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.316+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1241 ==== pg_info2(4.0 4.0 4.0( v 111'600 (111'500,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) e119/119) ==== 1122+0+0 (unknown 0 0 0) 0x5e1d500 con 0x507ac00 2026-02-11T23:31:28.316+0000 7f416abfa640 15 osd.0 119 enqueue_peering_evt 4.0 epoch_sent: 119 epoch_requested: 119 MInfoRec from 4 info: 4.0( v 111'600 (111'500,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) pg_lease(ru 76.773750305s ub 76.773750305s int 16.000000000s) 2026-02-11T23:31:28.316+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 119 epoch_requested: 119 MInfoRec from 4 info: 4.0( v 111'600 (111'500,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) pg_lease(ru 76.773750305s ub 76.773750305s int 16.000000000s)) class_id 2 prio 255 cost 10 e119) 2026-02-11T23:31:28.316+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.316+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 119 epoch_requested: 119 MInfoRec from 4 info: 4.0( v 111'600 (111'500,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) pg_lease(ru 76.773750305s ub 76.773750305s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e119) queued 2026-02-11T23:31:28.316+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.316+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 119 epoch_requested: 119 MInfoRec from 4 info: 4.0( v 111'600 (111'500,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) pg_lease(ru 76.773750305s ub 76.773750305s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e119) pg 0x5978000 2026-02-11T23:31:28.316+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.316+0000 7f414b98b640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] do_peering_event: epoch_sent: 119 epoch_requested: 119 MInfoRec from 4 info: 4.0( v 111'600 (111'500,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) pg_lease(ru 76.773750305s ub 76.773750305s int 16.000000000s) 2026-02-11T23:31:28.316+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.316+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.330+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:28.330+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:28.330+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:58.330796+0000) 2026-02-11T23:31:28.337+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1243 ==== pg_query2(4.0 4.0 query(info 0'0 epoch_sent 120) e120/120) ==== 158+0+0 (unknown 0 0 0) 0x6418780 con 0x507ac00 2026-02-11T23:31:28.337+0000 7f415aefe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1242 ==== osd_map(120..120 src has 1..120) ==== 317+0+0 (unknown 0 0 0) 0x5955340 con 0x507ac00 2026-02-11T23:31:28.337+0000 7f416abfa640 15 osd.0 119 enqueue_peering_evt 4.0 epoch_sent: 120 epoch_requested: 120 MQuery 4.0 from 4 query_epoch 120 query: query(info 0'0 epoch_sent 120) 2026-02-11T23:31:28.337+0000 7f415aefe640 20 osd.0 119 OSD::ms_dispatch: osd_map(120..120 src has 1..120) 2026-02-11T23:31:28.337+0000 7f415aefe640 20 osd.0 119 _dispatch 0x5955340 osd_map(120..120 src has 1..120) 2026-02-11T23:31:28.337+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 120 epoch_requested: 120 MQuery 4.0 from 4 query_epoch 120 query: query(info 0'0 epoch_sent 120)) class_id 2 prio 255 cost 10 e120) 2026-02-11T23:31:28.337+0000 7f415aefe640 3 osd.0 119 handle_osd_map epochs [120,120], i have 119, src has [1,120] 2026-02-11T23:31:28.337+0000 7f415aefe640 10 osd.0 119 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:31:28.337+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.337+0000 7f415aefe640 10 osd.0 119 handle_osd_map got inc map for epoch 120 2026-02-11T23:31:28.337+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 120 epoch_requested: 120 MQuery 4.0 from 4 query_epoch 120 query: query(info 0'0 epoch_sent 120)) class_id 2immediate prio 255 cost 10 e120) queued 2026-02-11T23:31:28.337+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.337+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 120 epoch_requested: 120 MQuery 4.0 from 4 query_epoch 120 query: query(info 0'0 epoch_sent 120)) class_id 2immediate prio 255 cost 10 e120) pg 0x5978000 2026-02-11T23:31:28.337+0000 7f414f993640 20 osd.0 op_wq _add_slot_waiter 4.0 peering, item epoch is 120, will wait on OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 120 epoch_requested: 120 MQuery 4.0 from 4 query_epoch 120 query: query(info 0'0 epoch_sent 120)) class_id 2immediate prio 255 cost 10 e120) 2026-02-11T23:31:28.337+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.337+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.337+0000 7f415aefe640 20 osd.0 119 got_full_map 120, nothing requested 2026-02-11T23:31:28.337+0000 7f415aefe640 20 osd.0 119 trim_maps: min=1 oldest_map=1 2026-02-11T23:31:28.337+0000 7f415aefe640 20 osd.0 119 handle_osd_map pg_num_history pg_num_history(e120 pg_nums {1={27=1},2={34=1},3={42=1},4={49=1},5={56=1},6={64=1},7={70=1},8={77=1},9={85=1},10={92=1}} deleted_pools 99,2,100,3,101,5,102,6,103,7,104,8,105,9,106,10) 2026-02-11T23:31:28.337+0000 7f415aefe640 10 snap_mapper record_purged_snaps purged_snaps {120={}} 2026-02-11T23:31:28.337+0000 7f415aefe640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:31:28.337+0000 7f415aefe640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e120 maps [1~120] lci=[5,120] tlb=1) 2026-02-11T23:31:28.337+0000 7f41591a6640 10 osd.0 119 _committed_osd_maps 120..120 2026-02-11T23:31:28.337+0000 7f41591a6640 10 osd.0 119 advance to epoch 120 (<= last 120 <= newest_map 120) 2026-02-11T23:31:28.337+0000 7f41591a6640 20 osd.0 120 consume_map version 120 2026-02-11T23:31:28.337+0000 7f41591a6640 20 osd.0 119 await_reserved_maps epoch:120 2026-02-11T23:31:28.337+0000 7f41591a6640 20 osd.0 119 await_reserved_maps done 0 2026-02-11T23:31:28.337+0000 7f41591a6640 20 osd.0 120 consume_map 120 -- publish done 2026-02-11T23:31:28.337+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:31:28.337+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x5efde40 2026-02-11T23:31:28.337+0000 7f41591a6640 20 osd.0 120 identify_splits_and_merges 4.0 e119 to e120 2026-02-11T23:31:28.337+0000 7f41591a6640 20 osd.0 120 identify_splits_and_merges 4.0 e119 to e120 pg_nums {49=1} 2026-02-11T23:31:28.337+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3701340 2026-02-11T23:31:28.337+0000 7f41591a6640 20 osd.0 120 identify_splits_and_merges 1.0 e119 to e120 2026-02-11T23:31:28.337+0000 7f41591a6640 20 osd.0 120 identify_splits_and_merges 1.0 e119 to e120 pg_nums {27=1} 2026-02-11T23:31:28.337+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:28.337+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:28.337+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:28.337+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:28.337+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:28.337+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:28.337+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:28.337+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:28.337+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:28.337+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:28.337+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:28.337+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:28.337+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:28.337+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:28.337+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:28.337+0000 7f41591a6640 20 osd.0 120 prune_pg_created 2026-02-11T23:31:28.337+0000 7f41591a6640 10 osd.0:0.consume_map 120 (was 119) 2026-02-11T23:31:28.337+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:31:28.337+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 pending_peering first epoch 120 <= 120, requeueing 2026-02-11T23:31:28.337+0000 7f41591a6640 20 osd.0:0._wake_pg_slot _wake_pg_slot 4.0 to_process <> waiting <> waiting_peering {120=} 2026-02-11T23:31:28.337+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:31:28.337+0000 7f41591a6640 10 osd.0:1.consume_map 120 (was 119) 2026-02-11T23:31:28.337+0000 7f41591a6640 10 osd.0:2.consume_map 120 (was 119) 2026-02-11T23:31:28.337+0000 7f41591a6640 10 osd.0:3.consume_map 120 (was 119) 2026-02-11T23:31:28.337+0000 7f41591a6640 10 osd.0:4.consume_map 120 (was 119) 2026-02-11T23:31:28.337+0000 7f41591a6640 10 osd.0:5.consume_map 120 (was 119) 2026-02-11T23:31:28.337+0000 7f41591a6640 10 osd.0:6.consume_map 120 (was 119) 2026-02-11T23:31:28.337+0000 7f41591a6640 10 osd.0:7.consume_map 120 (was 119) 2026-02-11T23:31:28.337+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.337+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 120 epoch_requested: 120 MQuery 4.0 from 4 query_epoch 120 query: query(info 0'0 epoch_sent 120)) class_id 2immediate prio 255 cost 10 e120) queued 2026-02-11T23:31:28.337+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.337+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 120 epoch_requested: 120 MQuery 4.0 from 4 query_epoch 120 query: query(info 0'0 epoch_sent 120)) class_id 2immediate prio 255 cost 10 e120) pg 0x5978000 2026-02-11T23:31:28.337+0000 7f41591a6640 10 osd.0 120 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:31:28.337+0000 7f41591a6640 15 osd.0 120 enqueue_peering_evt 4.0 epoch_sent: 120 epoch_requested: 120 NullEvt 2026-02-11T23:31:28.337+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 120 epoch_requested: 120 NullEvt) class_id 2 prio 255 cost 10 e120) 2026-02-11T23:31:28.337+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] handle_advance_map: 120 2026-02-11T23:31:28.338+0000 7f414f993640 20 osd.0:0.update_pg_epoch 4.0 119 -> 120 2026-02-11T23:31:28.338+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:31:28.338+0000 7f41591a6640 15 osd.0 120 enqueue_peering_evt 1.0 epoch_sent: 120 epoch_requested: 120 NullEvt 2026-02-11T23:31:28.338+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 120 epoch_requested: 120 NullEvt) class_id 2 prio 255 cost 10 e120) 2026-02-11T23:31:28.338+0000 7f414f993640 10 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] state: Started advmap 2026-02-11T23:31:28.338+0000 7f41591a6640 7 osd.0 120 activate_map version 120 2026-02-11T23:31:28.338+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.338+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 120 epoch_requested: 120 NullEvt) class_id 2immediate prio 255 cost 10 e120) queued 2026-02-11T23:31:28.338+0000 7f414f993640 20 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::should_restart_peering new interval newup [4,0] newacting [4,0] 2026-02-11T23:31:28.338+0000 7f414f993640 10 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] state: should_restart_peering, transitioning to Reset 2026-02-11T23:31:28.338+0000 7f4150194640 20 osd.0 120 agent_entry empty queue 2026-02-11T23:31:28.338+0000 7f414f993640 5 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] exit Started/ReplicaActive/RepNotRecovering 0.021251 2 0.000005 2026-02-11T23:31:28.338+0000 7f414f993640 5 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] exit Started/ReplicaActive 0.405568 0 0.000000 2026-02-11T23:31:28.338+0000 7f414f993640 5 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] exit Started 0.405713 0 0.000000 2026-02-11T23:31:28.338+0000 7f414f993640 5 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] enter Reset 2026-02-11T23:31:28.338+0000 7f414f993640 20 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=117 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::set_last_peering_reset set_last_peering_reset 120 2026-02-11T23:31:28.338+0000 7f414f993640 10 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=120 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::set_last_peering_reset Clearing blocked outgoing recovery messages 2026-02-11T23:31:28.338+0000 7f414f993640 10 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=120 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::set_last_peering_reset Not blocking outgoing recovery messages 2026-02-11T23:31:28.338+0000 7f414f993640 10 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=120 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] state: Reset advmap 2026-02-11T23:31:28.338+0000 7f414f993640 20 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=120 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::should_restart_peering new interval newup [4,0] newacting [4,0] 2026-02-11T23:31:28.338+0000 7f414f993640 10 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=120 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] state: should restart peering, calling start_peering_interval again 2026-02-11T23:31:28.338+0000 7f414f993640 20 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0]/[4] r=-1 lpr=120 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::set_last_peering_reset set_last_peering_reset 120 2026-02-11T23:31:28.338+0000 7f414f993640 15 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0] r=-1 lpr=120 pi=[109,117)/1 pct=0'0 crt=111'600 active mbc={}] clear_stats 2026-02-11T23:31:28.338+0000 7f414f993640 10 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0] r=1 lpr=120 pi=[109,120)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::start_peering_interval : check_new_interval output: check_new_interval interval(117-119 up [4, 0](4) acting [4](4)) up_thru 117 up_from 23 last_epoch_clean 111 interval(117-119 up [4, 0](4) acting [4](4) maybe_went_rw) : primary up 23-117 includes interval 2026-02-11T23:31:28.338+0000 7f414f993640 10 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=117) [4,0] r=1 lpr=120 pi=[109,120)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::start_peering_interval noting past ([109,119] all_participants={0, 4} intervals=[([117,119] acting={4})]) 2026-02-11T23:31:28.338+0000 7f414f993640 20 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::on_new_interval 2026-02-11T23:31:28.338+0000 7f414f993640 20 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::on_new_interval upacting_features 0x3f0ffffffffdffff from [4,0]+[4,0] 2026-02-11T23:31:28.338+0000 7f414f993640 20 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::on_new_interval checking missing set deletes flag. missing = missing(0 may_include_deletes = 1) 2026-02-11T23:31:28.338+0000 7f414f993640 10 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::init_hb_stamps now [0x56859d0] 2026-02-11T23:31:28.338+0000 7f414f993640 10 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::on_new_interval prior_readable_until_ub 0.000000000s (mnow 71.014762878s + 0.000000000s) 2026-02-11T23:31:28.338+0000 7f414f993640 10 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 pct=0'0 crt=111'600 active mbc={}] cancel_recovery 2026-02-11T23:31:28.338+0000 7f414f993640 10 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 pct=0'0 crt=111'600 active mbc={}] clear_recovery_state 2026-02-11T23:31:28.338+0000 7f414f993640 15 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 pct=0'0 crt=111'600 active mbc={}] _clear_recovery_state 2026-02-11T23:31:28.338+0000 7f414f993640 10 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 pct=0'0 crt=111'600 active mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:31:28.338+0000 7f414f993640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:31:28.338+0000 7f414f993640 20 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 pct=0'0 crt=111'600 active mbc={}] scrubber: FSM: ReplicaActive::exit() 2026-02-11T23:31:28.338+0000 7f414f993640 10 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 pct=0'0 crt=111'600 active mbc={}] scrubber: FSM: ReplicaActive::clear_remote_reservation(): pending_reservation_nonce 0, reservation_granted false 2026-02-11T23:31:28.338+0000 7f414f993640 10 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 pct=0'0 crt=111'600 active mbc={}] scrubber: FSM: -- state -->> NotActive 2026-02-11T23:31:28.338+0000 7f414f993640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:31:28.338+0000 7f414f993640 1 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 pct=0'0 crt=111'600 active mbc={}] PeeringState::start_peering_interval up [4,0] -> [4,0], acting [4] -> [4,0], acting_primary 4 -> 4, up_primary 4 -> 4, role -1 -> 1, features acting 4544132024016699391 upacting 4544132024016699391 2026-02-11T23:31:28.338+0000 7f414f993640 10 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 unknown mbc={}] PeeringState::clear_primary_state clear_primary_state 2026-02-11T23:31:28.338+0000 7f414f993640 20 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 unknown mbc={}] clear_primary_state 2026-02-11T23:31:28.338+0000 7f414f993640 10 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 unknown mbc={}] release_backoffs [4:00000000::::0,MAX) 2026-02-11T23:31:28.338+0000 7f414f993640 10 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 unknown mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:31:28.338+0000 7f414f993640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:31:28.338+0000 7f414f993640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:31:28.338+0000 7f414f993640 20 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 unknown mbc={}] agent_stop 2026-02-11T23:31:28.338+0000 7f414f993640 10 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 unknown mbc={}] on_change 2026-02-11T23:31:28.338+0000 7f414f993640 10 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 unknown mbc={}] cancel_copy_ops 2026-02-11T23:31:28.338+0000 7f414f993640 10 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 unknown mbc={}] cancel_flush_ops 2026-02-11T23:31:28.338+0000 7f414f993640 10 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 unknown mbc={}] cancel_proxy_ops 2026-02-11T23:31:28.338+0000 7f414f993640 10 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 unknown mbc={}] cancel_manifest_ops 2026-02-11T23:31:28.338+0000 7f414f993640 10 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 unknown mbc={}] cancel_cls_gather_ops 2026-02-11T23:31:28.338+0000 7f414f993640 10 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 unknown mbc={}] on_change_cleanup 2026-02-11T23:31:28.338+0000 7f414f993640 10 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 unknown mbc={}] on_change 2026-02-11T23:31:28.338+0000 7f414f993640 20 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 unknown mbc={}] exit NotTrimming 2026-02-11T23:31:28.338+0000 7f414f993640 20 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 unknown mbc={}] enter NotTrimming 2026-02-11T23:31:28.338+0000 7f414f993640 10 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 unknown NOTIFY mbc={}] plpg_on_role_change 2026-02-11T23:31:28.338+0000 7f414f993640 10 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 unknown NOTIFY mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:31:28.338+0000 7f414f993640 20 osd.0 120 advance_pg new pool opts old pool opts 2026-02-11T23:31:28.338+0000 7f414f993640 10 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 unknown NOTIFY mbc={}] handle_activate_map: epoch range: 120..120 2026-02-11T23:31:28.338+0000 7f414f993640 10 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 unknown NOTIFY mbc={}] PeeringState::activate_map 2026-02-11T23:31:28.338+0000 7f414f993640 5 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 unknown NOTIFY mbc={}] exit Reset 0.000073 1 0.000077 2026-02-11T23:31:28.338+0000 7f414f993640 5 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 unknown NOTIFY mbc={}] enter Started 2026-02-11T23:31:28.338+0000 7f414f993640 5 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 unknown NOTIFY mbc={}] enter Start 2026-02-11T23:31:28.338+0000 7f414f993640 1 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 unknown NOTIFY mbc={}] state: transitioning to Stray 2026-02-11T23:31:28.338+0000 7f414f993640 5 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 unknown NOTIFY mbc={}] exit Start 0.000002 0 0.000000 2026-02-11T23:31:28.338+0000 7f414f993640 5 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 unknown NOTIFY mbc={}] enter Started/Stray 2026-02-11T23:31:28.338+0000 7f414f993640 20 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 unknown NOTIFY mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 119 while current is 120 2026-02-11T23:31:28.338+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.338+0000 7f414f993640 10 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 unknown NOTIFY mbc={}] do_peering_event: epoch_sent: 120 epoch_requested: 120 MQuery 4.0 from 4 query_epoch 120 query: query(info 0'0 epoch_sent 120) 2026-02-11T23:31:28.338+0000 7f414f993640 10 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 unknown NOTIFY mbc={}] PeeringState::fulfill_info sending info 2026-02-11T23:31:28.338+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.338+0000 7f414f993640 20 osd.0 120 get_con_osd_cluster to osd.4 from_epoch 120 2026-02-11T23:31:28.338+0000 7f414f993640 20 osd.0 120 get_nextmap_reserved map_reservations: {120=1} 2026-02-11T23:31:28.338+0000 7f414f993640 20 osd.0 120 release_map epoch: 120 2026-02-11T23:31:28.338+0000 7f414f993640 10 osd.0 120 maybe_share_map: con v2:127.0.0.1:6836/3331151400 map epoch 119 -> 120 (shared) 2026-02-11T23:31:28.338+0000 7f414f993640 10 osd.0 120 send_incremental_map epoch range: (120, 120] to v2:127.0.0.1:6836/3331151400 2026-02-11T23:31:28.338+0000 7f414f993640 10 osd.0 120 add_map_inc_bl 120 273 bytes 2026-02-11T23:31:28.338+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- osd_map(120..120 src has 1..120) -- 0x4a64fc0 con 0x507ac00 2026-02-11T23:31:28.338+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_notify2(4.0 4.0 (query:120 sent:120 4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) ([109,119] all_participants={0, 4} intervals=[([117,119] acting={4})])) e120/120) -- 0x6496000 con 0x507ac00 2026-02-11T23:31:28.338+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_notify2(4.0 4.0 (query:120 sent:120 4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) ([109,119] all_participants={0, 4} intervals=[([117,119] acting={4})])) e120/120) -- 0x6496700 con 0x507ac00 2026-02-11T23:31:28.338+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.338+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 120 epoch_requested: 120 NullEvt) class_id 2immediate prio 255 cost 10 e120) queued 2026-02-11T23:31:28.338+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.338+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 120 epoch_requested: 120 NullEvt) class_id 2immediate prio 255 cost 10 e120) pg 0x4ef0000 2026-02-11T23:31:28.338+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.338+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 120 epoch_requested: 120 NullEvt) class_id 2immediate prio 255 cost 10 e120) pg 0x5978000 2026-02-11T23:31:28.338+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/0 sis=118) [1,0] r=1 lpr=118 crt=111'600 active mbc={}] handle_advance_map: 120 2026-02-11T23:31:28.338+0000 7f414f993640 20 osd.0:0.update_pg_epoch 1.0 119 -> 120 2026-02-11T23:31:28.338+0000 7f414b98b640 10 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 unknown NOTIFY mbc={}] do_peering_event: epoch_sent: 120 epoch_requested: 120 NullEvt 2026-02-11T23:31:28.338+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.338+0000 7f414f993640 10 osd.0 pg_epoch: 119 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/0 sis=118) [1,0] r=1 lpr=118 crt=111'600 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:31:28.338+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.338+0000 7f414f993640 10 osd.0 pg_epoch: 120 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/0 sis=118) [1,0] r=1 lpr=118 crt=111'600 active mbc={}] state: Started advmap 2026-02-11T23:31:28.338+0000 7f414f993640 10 osd.0 pg_epoch: 120 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/0 sis=118) [1,0] r=1 lpr=118 crt=111'600 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:31:28.338+0000 7f414f993640 20 osd.0 120 advance_pg new pool opts old pool opts 2026-02-11T23:31:28.338+0000 7f414f993640 10 osd.0 pg_epoch: 120 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/0 sis=118) [1,0] r=1 lpr=118 crt=111'600 active mbc={}] handle_activate_map: epoch range: 120..120 2026-02-11T23:31:28.338+0000 7f414f993640 10 osd.0 pg_epoch: 120 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/0 sis=118) [1,0] r=1 lpr=118 crt=111'600 active mbc={}] PeeringState::activate_map 2026-02-11T23:31:28.338+0000 7f414f993640 20 osd.0 pg_epoch: 120 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/0 sis=118) [1,0] r=1 lpr=118 crt=111'600 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 119 while current is 120 2026-02-11T23:31:28.338+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.338+0000 7f414f993640 10 osd.0 pg_epoch: 120 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/0 sis=118) [1,0] r=1 lpr=118 crt=111'600 active mbc={}] do_peering_event: epoch_sent: 120 epoch_requested: 120 NullEvt 2026-02-11T23:31:28.338+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.338+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.338+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.338+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.436+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 55 ==== osd_map(120..121 src has 1..121) ==== 577+0+0 (crc 0 0 0) 0x5954fc0 con 0x4acbc00 2026-02-11T23:31:28.436+0000 7f415cf02640 20 osd.0 120 OSD::ms_dispatch: osd_map(120..121 src has 1..121) 2026-02-11T23:31:28.436+0000 7f415cf02640 20 osd.0 120 _dispatch 0x5954fc0 osd_map(120..121 src has 1..121) 2026-02-11T23:31:28.436+0000 7f415cf02640 3 osd.0 120 handle_osd_map epochs [120,121], i have 120, src has [1,121] 2026-02-11T23:31:28.436+0000 7f415cf02640 10 osd.0 120 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:31:28.436+0000 7f415cf02640 10 osd.0 120 handle_osd_map got inc map for epoch 121 2026-02-11T23:31:28.436+0000 7f415cf02640 20 osd.0 120 got_full_map 121, nothing requested 2026-02-11T23:31:28.436+0000 7f415cf02640 20 osd.0 120 trim_maps: min=1 oldest_map=1 2026-02-11T23:31:28.436+0000 7f415cf02640 20 osd.0 120 handle_osd_map pg_num_history pg_num_history(e121 pg_nums {1={27=1},2={34=1},3={42=1},4={49=1},5={56=1},6={64=1},7={70=1},8={77=1},9={85=1},10={92=1}} deleted_pools 99,2,100,3,101,5,102,6,103,7,104,8,105,9,106,10) 2026-02-11T23:31:28.436+0000 7f415cf02640 10 snap_mapper record_purged_snaps purged_snaps {121={}} 2026-02-11T23:31:28.436+0000 7f415cf02640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:31:28.436+0000 7f415cf02640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e121 maps [1~121] lci=[5,121] tlb=1) 2026-02-11T23:31:28.436+0000 7f41591a6640 10 osd.0 120 _committed_osd_maps 121..121 2026-02-11T23:31:28.436+0000 7f41591a6640 10 osd.0 120 advance to epoch 121 (<= last 121 <= newest_map 121) 2026-02-11T23:31:28.436+0000 7f41591a6640 20 osd.0 121 consume_map version 121 2026-02-11T23:31:28.436+0000 7f41591a6640 20 osd.0 120 await_reserved_maps epoch:121 2026-02-11T23:31:28.436+0000 7f41591a6640 20 osd.0 120 await_reserved_maps done 0 2026-02-11T23:31:28.436+0000 7f41591a6640 20 osd.0 121 consume_map 121 -- publish done 2026-02-11T23:31:28.436+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:31:28.436+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x5efde40 2026-02-11T23:31:28.436+0000 7f41591a6640 20 osd.0 121 identify_splits_and_merges 4.0 e120 to e121 2026-02-11T23:31:28.436+0000 7f41591a6640 20 osd.0 121 identify_splits_and_merges 4.0 e120 to e121 pg_nums {49=1} 2026-02-11T23:31:28.436+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3701340 2026-02-11T23:31:28.436+0000 7f41591a6640 20 osd.0 121 identify_splits_and_merges 1.0 e120 to e121 2026-02-11T23:31:28.436+0000 7f41591a6640 20 osd.0 121 identify_splits_and_merges 1.0 e120 to e121 pg_nums {27=1} 2026-02-11T23:31:28.436+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:28.436+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:28.436+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:28.436+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:28.436+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:28.436+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:28.436+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:28.436+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:28.436+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:28.436+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:28.436+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:28.436+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:28.436+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:28.436+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:28.436+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:28.436+0000 7f41591a6640 20 osd.0 121 prune_pg_created 2026-02-11T23:31:28.436+0000 7f41591a6640 10 osd.0:0.consume_map 121 (was 120) 2026-02-11T23:31:28.436+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:31:28.436+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:31:28.436+0000 7f41591a6640 10 osd.0:1.consume_map 121 (was 120) 2026-02-11T23:31:28.436+0000 7f41591a6640 10 osd.0:2.consume_map 121 (was 120) 2026-02-11T23:31:28.436+0000 7f41591a6640 10 osd.0:3.consume_map 121 (was 120) 2026-02-11T23:31:28.436+0000 7f41591a6640 10 osd.0:4.consume_map 121 (was 120) 2026-02-11T23:31:28.436+0000 7f41591a6640 10 osd.0:5.consume_map 121 (was 120) 2026-02-11T23:31:28.436+0000 7f41591a6640 10 osd.0:6.consume_map 121 (was 120) 2026-02-11T23:31:28.436+0000 7f41591a6640 10 osd.0:7.consume_map 121 (was 120) 2026-02-11T23:31:28.436+0000 7f41591a6640 10 osd.0 121 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:31:28.436+0000 7f41591a6640 15 osd.0 121 enqueue_peering_evt 4.0 epoch_sent: 121 epoch_requested: 121 NullEvt 2026-02-11T23:31:28.436+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 121 epoch_requested: 121 NullEvt) class_id 2 prio 255 cost 10 e121) 2026-02-11T23:31:28.436+0000 7f41591a6640 15 osd.0 121 enqueue_peering_evt 1.0 epoch_sent: 121 epoch_requested: 121 NullEvt 2026-02-11T23:31:28.436+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 121 epoch_requested: 121 NullEvt) class_id 2 prio 255 cost 10 e121) 2026-02-11T23:31:28.436+0000 7f41591a6640 7 osd.0 121 activate_map version 121 2026-02-11T23:31:28.436+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.436+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 121 epoch_requested: 121 NullEvt) class_id 2immediate prio 255 cost 10 e121) queued 2026-02-11T23:31:28.436+0000 7f4150194640 20 osd.0 121 agent_entry empty queue 2026-02-11T23:31:28.436+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.436+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 121 epoch_requested: 121 NullEvt) class_id 2immediate prio 255 cost 10 e121) pg 0x5978000 2026-02-11T23:31:28.436+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.436+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 121 epoch_requested: 121 NullEvt) class_id 2immediate prio 255 cost 10 e121) queued 2026-02-11T23:31:28.436+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.436+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 121 epoch_requested: 121 NullEvt) class_id 2immediate prio 255 cost 10 e121) pg 0x4ef0000 2026-02-11T23:31:28.436+0000 7f414f993640 10 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 unknown NOTIFY mbc={}] handle_advance_map: 121 2026-02-11T23:31:28.436+0000 7f414f993640 20 osd.0:0.update_pg_epoch 4.0 120 -> 121 2026-02-11T23:31:28.436+0000 7f414f993640 10 osd.0 pg_epoch: 120 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 unknown NOTIFY mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:31:28.436+0000 7f414b98b640 10 osd.0 pg_epoch: 120 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/0 sis=118) [1,0] r=1 lpr=118 crt=111'600 active mbc={}] handle_advance_map: 121 2026-02-11T23:31:28.436+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 1.0 120 -> 121 2026-02-11T23:31:28.436+0000 7f414f993640 10 osd.0 pg_epoch: 121 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 unknown NOTIFY mbc={}] state: Started advmap 2026-02-11T23:31:28.436+0000 7f414b98b640 10 osd.0 pg_epoch: 120 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/0 sis=118) [1,0] r=1 lpr=118 crt=111'600 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:31:28.436+0000 7f414f993640 10 osd.0 pg_epoch: 121 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 unknown NOTIFY mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:31:28.436+0000 7f414f993640 20 osd.0 121 advance_pg new pool opts old pool opts 2026-02-11T23:31:28.436+0000 7f414b98b640 10 osd.0 pg_epoch: 121 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/0 sis=118) [1,0] r=1 lpr=118 crt=111'600 active mbc={}] state: Started advmap 2026-02-11T23:31:28.436+0000 7f414f993640 10 osd.0 pg_epoch: 121 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 unknown NOTIFY mbc={}] handle_activate_map: epoch range: 121..121 2026-02-11T23:31:28.436+0000 7f414b98b640 10 osd.0 pg_epoch: 121 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/0 sis=118) [1,0] r=1 lpr=118 crt=111'600 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:31:28.436+0000 7f414f993640 10 osd.0 pg_epoch: 121 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 unknown NOTIFY mbc={}] PeeringState::activate_map 2026-02-11T23:31:28.436+0000 7f414b98b640 20 osd.0 121 advance_pg new pool opts old pool opts 2026-02-11T23:31:28.436+0000 7f414b98b640 10 osd.0 pg_epoch: 121 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/0 sis=118) [1,0] r=1 lpr=118 crt=111'600 active mbc={}] handle_activate_map: epoch range: 121..121 2026-02-11T23:31:28.436+0000 7f414f993640 20 osd.0 pg_epoch: 121 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 unknown NOTIFY mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 120 while current is 121 2026-02-11T23:31:28.436+0000 7f414b98b640 10 osd.0 pg_epoch: 121 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/0 sis=118) [1,0] r=1 lpr=118 crt=111'600 active mbc={}] PeeringState::activate_map 2026-02-11T23:31:28.436+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.436+0000 7f414f993640 10 osd.0 pg_epoch: 121 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 unknown NOTIFY mbc={}] do_peering_event: epoch_sent: 121 epoch_requested: 121 NullEvt 2026-02-11T23:31:28.436+0000 7f414b98b640 20 osd.0 pg_epoch: 121 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/0 sis=118) [1,0] r=1 lpr=118 crt=111'600 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 119 while current is 121 2026-02-11T23:31:28.436+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.436+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.436+0000 7f414f993640 20 osd.0 121 get_con_osd_cluster to osd.4 from_epoch 121 2026-02-11T23:31:28.436+0000 7f414f993640 20 osd.0 121 get_nextmap_reserved map_reservations: {121=1} 2026-02-11T23:31:28.436+0000 7f414f993640 20 osd.0 121 release_map epoch: 121 2026-02-11T23:31:28.436+0000 7f414b98b640 10 osd.0 pg_epoch: 121 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/0 sis=118) [1,0] r=1 lpr=118 crt=111'600 active mbc={}] do_peering_event: epoch_sent: 121 epoch_requested: 121 NullEvt 2026-02-11T23:31:28.436+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.436+0000 7f414f993640 10 osd.0 121 maybe_share_map: con v2:127.0.0.1:6836/3331151400 map epoch 120 -> 121 (shared) 2026-02-11T23:31:28.436+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.436+0000 7f414f993640 10 osd.0 121 send_incremental_map epoch range: (121, 121] to v2:127.0.0.1:6836/3331151400 2026-02-11T23:31:28.436+0000 7f414f993640 10 osd.0 121 add_map_inc_bl 121 252 bytes 2026-02-11T23:31:28.436+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- osd_map(121..121 src has 1..121) -- 0x4a64000 con 0x507ac00 2026-02-11T23:31:28.436+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_notify2(4.0 4.0 (query:121 sent:121 4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) ([109,119] all_participants={0, 4} intervals=[([117,119] acting={4})])) e121/121) -- 0x6496e00 con 0x507ac00 2026-02-11T23:31:28.436+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.437+0000 7f415aefe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1244 ==== osd_map(121..121 src has 1..121) ==== 296+0+0 (unknown 0 0 0) 0x4a64000 con 0x507ac00 2026-02-11T23:31:28.437+0000 7f415aefe640 20 osd.0 121 OSD::ms_dispatch: osd_map(121..121 src has 1..121) 2026-02-11T23:31:28.437+0000 7f415aefe640 20 osd.0 121 _dispatch 0x4a64000 osd_map(121..121 src has 1..121) 2026-02-11T23:31:28.437+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1245 ==== pg_info2(4.0 4.0 4.0( v 111'600 (111'500,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) e121/121) ==== 1085+0+0 (unknown 0 0 0) 0x6496e00 con 0x507ac00 2026-02-11T23:31:28.437+0000 7f415aefe640 3 osd.0 121 handle_osd_map epochs [121,121], i have 121, src has [1,121] 2026-02-11T23:31:28.437+0000 7f415aefe640 10 osd.0 121 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:31:28.437+0000 7f415aefe640 10 osd.0 121 no new maps here, dropping 2026-02-11T23:31:28.437+0000 7f416abfa640 15 osd.0 121 enqueue_peering_evt 4.0 epoch_sent: 121 epoch_requested: 121 MInfoRec from 4 info: 4.0( v 111'600 (111'500,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) pg_lease(ru 0.000000000s ub 77.277175903s int 16.000000000s) 2026-02-11T23:31:28.437+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 121 epoch_requested: 121 MInfoRec from 4 info: 4.0( v 111'600 (111'500,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) pg_lease(ru 0.000000000s ub 77.277175903s int 16.000000000s)) class_id 2 prio 255 cost 10 e121) 2026-02-11T23:31:28.437+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.437+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 121 epoch_requested: 121 MInfoRec from 4 info: 4.0( v 111'600 (111'500,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) pg_lease(ru 0.000000000s ub 77.277175903s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e121) queued 2026-02-11T23:31:28.437+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.437+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 121 epoch_requested: 121 MInfoRec from 4 info: 4.0( v 111'600 (111'500,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) pg_lease(ru 0.000000000s ub 77.277175903s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e121) pg 0x5978000 2026-02-11T23:31:28.437+0000 7f414f993640 10 osd.0 pg_epoch: 121 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 unknown NOTIFY mbc={}] do_peering_event: epoch_sent: 121 epoch_requested: 121 MInfoRec from 4 info: 4.0( v 111'600 (111'500,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) pg_lease(ru 0.000000000s ub 77.277175903s int 16.000000000s) 2026-02-11T23:31:28.437+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.437+0000 7f414f993640 10 osd.0 pg_epoch: 121 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 unknown NOTIFY mbc={}] state: got info from osd.4 4.0( v 111'600 (111'500,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) 2026-02-11T23:31:28.437+0000 7f414f993640 10 osd.0 pg_epoch: 121 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 unknown NOTIFY mbc={}] PeeringState::proc_lease pg_lease(ru 0.000000000s ub 77.277175903s int 16.000000000s) 2026-02-11T23:31:28.437+0000 7f414f993640 20 osd.0 pg_epoch: 121 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 unknown NOTIFY mbc={}] PeeringState::proc_lease peer_clock_delta_lb -9.836852074s -> ruub 87.114028931s 2026-02-11T23:31:28.437+0000 7f414f993640 20 osd.0 pg_epoch: 121 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 unknown NOTIFY mbc={}] PeeringState::proc_lease readable_until_ub now 87.114028931s 2026-02-11T23:31:28.437+0000 7f414f993640 5 osd.0 pg_epoch: 121 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 unknown NOTIFY mbc={}] exit Started/Stray 0.099190 6 0.000020 2026-02-11T23:31:28.437+0000 7f414f993640 5 osd.0 pg_epoch: 121 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 unknown NOTIFY mbc={}] enter Started/ReplicaActive 2026-02-11T23:31:28.437+0000 7f414f993640 5 osd.0 pg_epoch: 121 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 unknown NOTIFY mbc={}] enter Started/ReplicaActive/RepNotRecovering 2026-02-11T23:31:28.437+0000 7f414f993640 10 osd.0 pg_epoch: 121 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=117/118 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 unknown NOTIFY mbc={}] state: In ReplicaActive, about to call activate 2026-02-11T23:31:28.437+0000 7f414f993640 10 osd.0 pg_epoch: 121 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 unknown mbc={}] PeeringState::activate activate - no missing, moving last_complete 111'600 -> 111'600 2026-02-11T23:31:28.437+0000 7f414f993640 10 osd.0 pg_epoch: 121 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 unknown mbc={}] scrubber: on_replica_activate 2026-02-11T23:31:28.437+0000 7f414f993640 20 scrubberFSM event: --vvvv---- ReplicaActivate 2026-02-11T23:31:28.437+0000 7f414f993640 10 osd.0 pg_epoch: 121 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 unknown mbc={}] scrubber: FSM: -- state -->> ReplicaActive 2026-02-11T23:31:28.437+0000 7f414f993640 10 osd.0 pg_epoch: 121 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 unknown mbc={}] scrubber: FSM: -- state -->> ReplicaActive/ReplicaIdle 2026-02-11T23:31:28.437+0000 7f414f993640 20 scrubberFSM event: --^^^^---- ReplicaActivate 2026-02-11T23:31:28.437+0000 7f414f993640 10 osd.0 pg_epoch: 121 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 unknown mbc={}] state: Activate Finished 2026-02-11T23:31:28.437+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.437+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.437+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.437+0000 7f414f993640 15 osd.0 121 enqueue_peering_evt 4.0 epoch_sent: 121 epoch_requested: 121 ActivateCommitted from 121 processed at 121 2026-02-11T23:31:28.437+0000 7f414f993640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 121 epoch_requested: 121 ActivateCommitted from 121 processed at 121) class_id 2 prio 255 cost 10 e121) 2026-02-11T23:31:28.437+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.437+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.437+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 121 epoch_requested: 121 ActivateCommitted from 121 processed at 121) class_id 2immediate prio 255 cost 10 e121) queued 2026-02-11T23:31:28.437+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.437+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 121 epoch_requested: 121 ActivateCommitted from 121 processed at 121) class_id 2immediate prio 255 cost 10 e121) pg 0x5978000 2026-02-11T23:31:28.437+0000 7f414f993640 10 osd.0 pg_epoch: 121 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 unknown mbc={}] do_peering_event: epoch_sent: 121 epoch_requested: 121 ActivateCommitted from 121 processed at 121 2026-02-11T23:31:28.437+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.437+0000 7f414f993640 10 osd.0 pg_epoch: 121 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 unknown mbc={}] state: react 121 telling primary 2026-02-11T23:31:28.437+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.437+0000 7f414f993640 20 osd.0 121 get_con_osd_cluster to osd.4 from_epoch 121 2026-02-11T23:31:28.437+0000 7f414f993640 20 osd.0 121 get_nextmap_reserved map_reservations: {121=1} 2026-02-11T23:31:28.437+0000 7f414f993640 20 osd.0 121 release_map epoch: 121 2026-02-11T23:31:28.437+0000 7f414f993640 10 osd.0 121 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 121 is not newer than session's projected_epoch of 121 2026-02-11T23:31:28.437+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_info2(4.0 4.0 4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/109 les/c/f=121/111/0 sis=120) e121/121) -- 0x6497500 con 0x507ac00 2026-02-11T23:31:28.437+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.437+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1246 ==== pg_lease(4.0 pg_lease(ru 77.277175903s ub 77.277549744s int 16.000000000s) e121/121) ==== 58+0+0 (unknown 0 0 0) 0x6794a80 con 0x507ac00 2026-02-11T23:31:28.437+0000 7f416abfa640 15 osd.0 121 enqueue_peering_evt 4.0 epoch_sent: 121 epoch_requested: 121 MLease epoch 121 from osd.4 pg_lease(ru 77.277175903s ub 77.277549744s int 16.000000000s) 2026-02-11T23:31:28.437+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 121 epoch_requested: 121 MLease epoch 121 from osd.4 pg_lease(ru 77.277175903s ub 77.277549744s int 16.000000000s)) class_id 2 prio 255 cost 10 e121) 2026-02-11T23:31:28.437+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.437+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 121 epoch_requested: 121 MLease epoch 121 from osd.4 pg_lease(ru 77.277175903s ub 77.277549744s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e121) queued 2026-02-11T23:31:28.437+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.437+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 121 epoch_requested: 121 MLease epoch 121 from osd.4 pg_lease(ru 77.277175903s ub 77.277549744s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e121) pg 0x5978000 2026-02-11T23:31:28.437+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.437+0000 7f414f993640 10 osd.0 pg_epoch: 121 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 active mbc={}] do_peering_event: epoch_sent: 121 epoch_requested: 121 MLease epoch 121 from osd.4 pg_lease(ru 77.277175903s ub 77.277549744s int 16.000000000s) 2026-02-11T23:31:28.437+0000 7f414f993640 10 osd.0 pg_epoch: 121 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 active mbc={}] PeeringState::proc_lease pg_lease(ru 77.277175903s ub 77.277549744s int 16.000000000s) 2026-02-11T23:31:28.437+0000 7f414f993640 20 osd.0 pg_epoch: 121 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 active mbc={}] PeeringState::proc_lease peer_clock_delta_ub -9.836483955s -> ru 87.113662720s 2026-02-11T23:31:28.437+0000 7f414f993640 20 osd.0 pg_epoch: 121 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 active mbc={}] PeeringState::proc_lease readable_until now 87.113662720s 2026-02-11T23:31:28.437+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1247 ==== pg_info2(4.0 4.0 4.0( v 111'600 (111'500,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/109 les/c/f=121/111/0 sis=120) e121/121) ==== 1137+0+0 (unknown 0 0 0) 0x6496000 con 0x507ac00 2026-02-11T23:31:28.437+0000 7f414f993640 20 osd.0 pg_epoch: 121 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 active mbc={}] PeeringState::proc_lease peer_clock_delta_lb -9.836852074s -> ruub 87.114410400s 2026-02-11T23:31:28.437+0000 7f414f993640 20 osd.0 pg_epoch: 121 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 active mbc={}] PeeringState::proc_lease readable_until_ub now 87.114410400s 2026-02-11T23:31:28.437+0000 7f414f993640 20 osd.0 121 get_con_osd_cluster to osd.4 from_epoch 121 2026-02-11T23:31:28.437+0000 7f414f993640 20 osd.0 121 get_nextmap_reserved map_reservations: {121=1} 2026-02-11T23:31:28.437+0000 7f414f993640 20 osd.0 121 release_map epoch: 121 2026-02-11T23:31:28.437+0000 7f416abfa640 15 osd.0 121 enqueue_peering_evt 4.0 epoch_sent: 121 epoch_requested: 121 MInfoRec from 4 info: 4.0( v 111'600 (111'500,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/109 les/c/f=121/111/0 sis=120) pg_lease(ru 77.277175903s ub 77.277549744s int 16.000000000s) 2026-02-11T23:31:28.437+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 121 epoch_requested: 121 MInfoRec from 4 info: 4.0( v 111'600 (111'500,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/109 les/c/f=121/111/0 sis=120) pg_lease(ru 77.277175903s ub 77.277549744s int 16.000000000s)) class_id 2 prio 255 cost 10 e121) 2026-02-11T23:31:28.437+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_lease_ack(4.0 pg_lease_ack(ruub 77.277549744s) e121/121) -- 0x6427e40 con 0x507ac00 2026-02-11T23:31:28.437+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:28.437+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.437+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 121 epoch_requested: 121 MInfoRec from 4 info: 4.0( v 111'600 (111'500,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/109 les/c/f=121/111/0 sis=120) pg_lease(ru 77.277175903s ub 77.277549744s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e121) queued 2026-02-11T23:31:28.437+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.437+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 121 epoch_requested: 121 MInfoRec from 4 info: 4.0( v 111'600 (111'500,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/109 les/c/f=121/111/0 sis=120) pg_lease(ru 77.277175903s ub 77.277549744s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e121) pg 0x5978000 2026-02-11T23:31:28.437+0000 7f414b98b640 10 osd.0 pg_epoch: 121 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=117/109 les/c/f=118/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 active mbc={}] do_peering_event: epoch_sent: 121 epoch_requested: 121 MInfoRec from 4 info: 4.0( v 111'600 (111'500,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/109 les/c/f=121/111/0 sis=120) pg_lease(ru 77.277175903s ub 77.277549744s int 16.000000000s) 2026-02-11T23:31:28.437+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.437+0000 7f414b98b640 20 osd.0 pg_epoch: 121 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/109 les/c/f=121/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 active mbc={}] PeeringState::update_history advanced history from ec=49/49 lis/c=120/109 les/c/f=121/111/0 sis=120 2026-02-11T23:31:28.437+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.437+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.437+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1248 ==== pg_info2(4.0 4.0 4.0( v 111'600 (111'500,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/120 les/c/f=121/121/0 sis=120) e121/121) ==== 1137+0+0 (unknown 0 0 0) 0x6496000 con 0x507ac00 2026-02-11T23:31:28.437+0000 7f416abfa640 15 osd.0 121 enqueue_peering_evt 4.0 epoch_sent: 121 epoch_requested: 121 MInfoRec from 4 info: 4.0( v 111'600 (111'500,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/120 les/c/f=121/121/0 sis=120) pg_lease(ru 77.277175903s ub 77.277549744s int 16.000000000s) 2026-02-11T23:31:28.437+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 121 epoch_requested: 121 MInfoRec from 4 info: 4.0( v 111'600 (111'500,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/120 les/c/f=121/121/0 sis=120) pg_lease(ru 77.277175903s ub 77.277549744s int 16.000000000s)) class_id 2 prio 255 cost 10 e121) 2026-02-11T23:31:28.437+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:28.437+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 121 epoch_requested: 121 MInfoRec from 4 info: 4.0( v 111'600 (111'500,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/120 les/c/f=121/121/0 sis=120) pg_lease(ru 77.277175903s ub 77.277549744s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e121) queued 2026-02-11T23:31:28.437+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:28.437+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 121 epoch_requested: 121 MInfoRec from 4 info: 4.0( v 111'600 (111'500,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/120 les/c/f=121/121/0 sis=120) pg_lease(ru 77.277175903s ub 77.277549744s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e121) pg 0x5978000 2026-02-11T23:31:28.437+0000 7f414b98b640 10 osd.0 pg_epoch: 121 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/109 les/c/f=121/111/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 active mbc={}] do_peering_event: epoch_sent: 121 epoch_requested: 121 MInfoRec from 4 info: 4.0( v 111'600 (111'500,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/120 les/c/f=121/121/0 sis=120) pg_lease(ru 77.277175903s ub 77.277549744s int 16.000000000s) 2026-02-11T23:31:28.437+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.437+0000 7f414b98b640 20 osd.0 pg_epoch: 121 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/120 les/c/f=121/121/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 active mbc={}] PeeringState::update_history advanced history from ec=49/49 lis/c=120/120 les/c/f=121/121/0 sis=120 2026-02-11T23:31:28.437+0000 7f414b98b640 20 osd.0 pg_epoch: 121 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/120 les/c/f=121/121/0 sis=120) [4,0] r=1 lpr=120 pi=[109,120)/1 crt=111'600 active mbc={}] PeeringState::update_history clearing past_intervals 2026-02-11T23:31:28.437+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:28.437+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.437+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.437+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.437+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.437+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:28.507+0000 7f4162bea640 10 osd.0 121 tick_without_osd_lock 2026-02-11T23:31:28.507+0000 7f4162bea640 0 osd.0 121 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:28.507+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:28.507+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.221 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:28.507+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:28.507, recovery is active?:false restrictions:<....> 2026-02-11T23:31:28.507+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:28.507+0000 7f4162bea640 10 osd.0 121 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:28.507+0000 7f4162bea640 20 osd.0 121 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:28.507+0000 7f4162bea640 10 osd.0 121 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:28.507+0000 7f4162bea640 20 osd.0 121 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:28.896+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.2 127.0.0.1:0/2365263801 20 ==== osd_ping(ping e115 up_from 14 ping_stamp 2026-02-11T23:31:28.897899+0000/67.006500244s send_stamp 67.006500244s delta_ub -4.566675663s) ==== 2033+0+0 (crc 0 0 0) 0x4ec2180 con 0x4641000 2026-02-11T23:31:28.896+0000 7f416cbfe640 20 osd.0 121 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566896915s,-4.566675663s]) 2026-02-11T23:31:28.896+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e121 up_from 5 ping_stamp 2026-02-11T23:31:28.897899+0000/67.006500244s send_stamp 71.573387146s delta_ub 4.566896915s) -- 0x4ecc300 con 0x4641000 2026-02-11T23:31:28.896+0000 7f416cbfe640 20 osd.0 121 get_con_osd_cluster to osd.2 from_epoch 121 2026-02-11T23:31:28.896+0000 7f416cbfe640 20 osd.0 121 get_nextmap_reserved map_reservations: {121=1} 2026-02-11T23:31:28.896+0000 7f416cbfe640 20 osd.0 121 release_map epoch: 121 2026-02-11T23:31:28.896+0000 7f416cbfe640 10 osd.0 121 maybe_share_map: con v2:127.0.0.1:6820/2365263801 updating session's projected_epoch from 113 to ping map epoch of 115 2026-02-11T23:31:28.896+0000 7f416cbfe640 10 osd.0 121 maybe_share_map: con v2:127.0.0.1:6820/2365263801 map epoch 115 -> 121 (shared) 2026-02-11T23:31:28.896+0000 7f416cbfe640 10 osd.0 121 send_incremental_map epoch range: (116, 121] to v2:127.0.0.1:6820/2365263801 2026-02-11T23:31:28.896+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.2 127.0.0.1:0/2365263801 20 ==== osd_ping(ping e115 up_from 14 ping_stamp 2026-02-11T23:31:28.897899+0000/67.006500244s send_stamp 67.006500244s delta_ub -4.566675663s) ==== 2033+0+0 (crc 0 0 0) 0x4ecca80 con 0x48cc000 2026-02-11T23:31:28.896+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6820/2365263801,v1:127.0.0.1:6821/2365263801] -- osd_map(116..121 src has 1..121) -- 0x5954540 con 0x507b800 2026-02-11T23:31:28.896+0000 7f416c3fd640 20 osd.0 121 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566943169s,-4.566675663s]) 2026-02-11T23:31:28.896+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e121 up_from 5 ping_stamp 2026-02-11T23:31:28.897899+0000/67.006500244s send_stamp 71.573440552s delta_ub 4.566943169s) -- 0x4ecc780 con 0x48cc000 2026-02-11T23:31:28.896+0000 7f416c3fd640 20 osd.0 121 get_con_osd_cluster to osd.2 from_epoch 121 2026-02-11T23:31:28.896+0000 7f416c3fd640 20 osd.0 121 get_nextmap_reserved map_reservations: {121=1} 2026-02-11T23:31:28.896+0000 7f416c3fd640 20 osd.0 121 release_map epoch: 121 2026-02-11T23:31:28.896+0000 7f416c3fd640 10 osd.0 121 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 121 is not newer than session's projected_epoch of 121 2026-02-11T23:31:29.042+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.1 127.0.0.1:0/3043114013 23 ==== osd_ping(ping e120 up_from 10 ping_stamp 2026-02-11T23:31:29.043049+0000/69.706794739s send_stamp 69.706794739s delta_ub -2.011524916s) ==== 2033+0+0 (crc 0 0 0) 0x4ecc300 con 0x4aca000 2026-02-11T23:31:29.042+0000 7f416cbfe640 20 osd.0 121 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011720896s,-2.011524916s]) 2026-02-11T23:31:29.042+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e121 up_from 5 ping_stamp 2026-02-11T23:31:29.043049+0000/69.706794739s send_stamp 71.718521118s delta_ub 2.011720896s) -- 0x4ec2180 con 0x4aca000 2026-02-11T23:31:29.042+0000 7f416cbfe640 20 osd.0 121 get_con_osd_cluster to osd.1 from_epoch 121 2026-02-11T23:31:29.042+0000 7f416cbfe640 20 osd.0 121 get_nextmap_reserved map_reservations: {121=1} 2026-02-11T23:31:29.042+0000 7f416cbfe640 20 osd.0 121 release_map epoch: 121 2026-02-11T23:31:29.042+0000 7f416cbfe640 10 osd.0 121 maybe_share_map: con v2:127.0.0.1:6812/3043114013 updating session's projected_epoch from 119 to ping map epoch of 120 2026-02-11T23:31:29.042+0000 7f416cbfe640 10 osd.0 121 maybe_share_map: con v2:127.0.0.1:6812/3043114013 map epoch 120 -> 121 (shared) 2026-02-11T23:31:29.042+0000 7f416cbfe640 10 osd.0 121 send_incremental_map epoch range: (121, 121] to v2:127.0.0.1:6812/3043114013 2026-02-11T23:31:29.042+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- osd_map(121..121 src has 1..121) -- 0x5954540 con 0x4acb000 2026-02-11T23:31:29.042+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.1 127.0.0.1:0/3043114013 23 ==== osd_ping(ping e120 up_from 10 ping_stamp 2026-02-11T23:31:29.043049+0000/69.706794739s send_stamp 69.706794739s delta_ub -2.011524916s) ==== 2033+0+0 (crc 0 0 0) 0x4ecc780 con 0x4aca400 2026-02-11T23:31:29.042+0000 7f416c3fd640 20 osd.0 121 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011756659s,-2.011524916s]) 2026-02-11T23:31:29.042+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e121 up_from 5 ping_stamp 2026-02-11T23:31:29.043049+0000/69.706794739s send_stamp 71.718551636s delta_ub 2.011756659s) -- 0x4ecca80 con 0x4aca400 2026-02-11T23:31:29.042+0000 7f416c3fd640 20 osd.0 121 get_con_osd_cluster to osd.1 from_epoch 121 2026-02-11T23:31:29.042+0000 7f416c3fd640 20 osd.0 121 get_nextmap_reserved map_reservations: {121=1} 2026-02-11T23:31:29.042+0000 7f416c3fd640 20 osd.0 121 release_map epoch: 121 2026-02-11T23:31:29.042+0000 7f416c3fd640 10 osd.0 121 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 121 is not newer than session's projected_epoch of 121 2026-02-11T23:31:29.313+0000 7f41633eb640 10 osd.0 121 tick 2026-02-11T23:31:29.313+0000 7f41633eb640 20 osd.0 121 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:29.330+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:29.330+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:29.330+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:59.330840+0000) 2026-02-11T23:31:29.500+0000 7f4162bea640 10 osd.0 121 tick_without_osd_lock 2026-02-11T23:31:29.500+0000 7f4162bea640 0 osd.0 121 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:29.500+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:29.500+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.221 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:29.500+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:29.500, recovery is active?:false restrictions:<....> 2026-02-11T23:31:29.500+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:29.500+0000 7f4162bea640 10 osd.0 121 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:29.500+0000 7f4162bea640 20 osd.0 121 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:29.500+0000 7f4162bea640 10 osd.0 121 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:29.500+0000 7f4162bea640 20 osd.0 121 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:30.330+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:30.330+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:30.330+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:00.330875+0000) 2026-02-11T23:31:30.330+0000 7f415bf00640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-11T23:31:30.358+0000 7f41633eb640 10 osd.0 121 tick 2026-02-11T23:31:30.358+0000 7f41633eb640 20 osd.0 121 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:30.507+0000 7f4162bea640 10 osd.0 121 tick_without_osd_lock 2026-02-11T23:31:30.507+0000 7f4162bea640 0 osd.0 121 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:30.507+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:30.507+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:30.507+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:30.507+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.221 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:30.507+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:30.507, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:30.507+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:30.507+0000 7f4162bea640 10 osd.0 121 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:30.507+0000 7f4162bea640 20 osd.0 121 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:30.507+0000 7f4162bea640 10 osd.0 121 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:30.507+0000 7f4162bea640 20 osd.0 121 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:31.266+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.4 127.0.0.1:0/3331151400 19 ==== osd_ping(ping e121 up_from 23 ping_stamp 2026-02-11T23:31:31.267376+0000/64.106140137s send_stamp 64.106140137s delta_ub -9.836492538s) ==== 2033+0+0 (crc 0 0 0) 0x4ecca80 con 0x507a800 2026-02-11T23:31:31.266+0000 7f416c3fd640 20 osd.0 121 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836721420s,-9.836492538s]) 2026-02-11T23:31:31.266+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e121 up_from 5 ping_stamp 2026-02-11T23:31:31.267376+0000/64.106140137s send_stamp 73.942871094s delta_ub 9.836721420s) -- 0x4ecc780 con 0x507a800 2026-02-11T23:31:31.266+0000 7f416c3fd640 20 osd.0 121 get_con_osd_cluster to osd.4 from_epoch 121 2026-02-11T23:31:31.266+0000 7f416c3fd640 20 osd.0 121 get_nextmap_reserved map_reservations: {121=1} 2026-02-11T23:31:31.266+0000 7f416c3fd640 20 osd.0 121 release_map epoch: 121 2026-02-11T23:31:31.266+0000 7f416c3fd640 10 osd.0 121 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 121 is not newer than session's projected_epoch of 121 2026-02-11T23:31:31.266+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.4 127.0.0.1:0/3331151400 19 ==== osd_ping(ping e121 up_from 23 ping_stamp 2026-02-11T23:31:31.267376+0000/64.106140137s send_stamp 64.106140137s delta_ub -9.836492538s) ==== 2033+0+0 (crc 0 0 0) 0x6794a80 con 0x507a400 2026-02-11T23:31:31.266+0000 7f416abfa640 20 osd.0 121 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836743355s,-9.836492538s]) 2026-02-11T23:31:31.266+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e121 up_from 5 ping_stamp 2026-02-11T23:31:31.267376+0000/64.106140137s send_stamp 73.942886353s delta_ub 9.836743355s) -- 0x6794d80 con 0x507a400 2026-02-11T23:31:31.266+0000 7f416abfa640 20 osd.0 121 get_con_osd_cluster to osd.4 from_epoch 121 2026-02-11T23:31:31.266+0000 7f416abfa640 20 osd.0 121 get_nextmap_reserved map_reservations: {121=1} 2026-02-11T23:31:31.266+0000 7f416abfa640 20 osd.0 121 release_map epoch: 121 2026-02-11T23:31:31.266+0000 7f416abfa640 10 osd.0 121 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 121 is not newer than session's projected_epoch of 121 2026-02-11T23:31:31.330+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:31.330+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:31.330+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:01.330935+0000) 2026-02-11T23:31:31.332+0000 7f41633eb640 10 osd.0 121 tick 2026-02-11T23:31:31.332+0000 7f41633eb640 20 osd.0 121 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:31.531+0000 7f4162bea640 10 osd.0 121 tick_without_osd_lock 2026-02-11T23:31:31.531+0000 7f4162bea640 0 osd.0 121 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:31.531+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:31.531+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.221 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:31.531+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:31.531, recovery is active?:false restrictions:<....> 2026-02-11T23:31:31.531+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:31.531+0000 7f4162bea640 10 osd.0 121 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:31.531+0000 7f4162bea640 20 osd.0 121 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:31.531+0000 7f4162bea640 10 osd.0 121 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:31.531+0000 7f4162bea640 20 osd.0 121 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:31.931+0000 7f4147983640 5 osd.0 121 heartbeat osd_stat(store_statfs(0x0/0x0/0x384000, data 0x5391f8/0x5e4000, compress 0x0/0x0/0x0, omap 0xa403, meta 0x1a45bfd), peers [1,2,3,4] op hist []) 2026-02-11T23:31:31.931+0000 7f4147983640 20 osd.0 121 check_full_status cur ratio 1, physical ratio 1, new state failsafe 2026-02-11T23:31:31.931+0000 7f4147983640 10 osd.0 121 check_full_status none -> failsafe 2026-02-11T23:31:31.931+0000 7f4147983640 -1 log_channel(cluster) log [ERR] : full status failsafe engaged, dropping updates, now 100% full 2026-02-11T23:31:31.931+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6816/3043114013,v1:127.0.0.1:6817/3043114013] -- osd_ping(ping e121 up_from 5 ping_stamp 2026-02-11T23:31:31.931846+0000/74.607254028s send_stamp 74.607254028s delta_ub 2.011756659s) -- 0x4eccc00 con 0x48c4400 2026-02-11T23:31:31.931+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6814/3043114013,v1:127.0.0.1:6815/3043114013] -- osd_ping(ping e121 up_from 5 ping_stamp 2026-02-11T23:31:31.931846+0000/74.607254028s send_stamp 74.607254028s delta_ub 2.011756659s) -- 0x4eccd80 con 0x4640c00 2026-02-11T23:31:31.931+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6824/2365263801,v1:127.0.0.1:6825/2365263801] -- osd_ping(ping e121 up_from 5 ping_stamp 2026-02-11T23:31:31.931846+0000/74.607254028s send_stamp 74.607254028s delta_ub 4.566943169s) -- 0x4eccf00 con 0x48c4000 2026-02-11T23:31:31.931+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6822/2365263801,v1:127.0.0.1:6823/2365263801] -- osd_ping(ping e121 up_from 5 ping_stamp 2026-02-11T23:31:31.931846+0000/74.607254028s send_stamp 74.607254028s delta_ub 4.566943169s) -- 0x4ecd080 con 0x48cd000 2026-02-11T23:31:31.931+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6832/2699377081,v1:127.0.0.1:6833/2699377081] -- osd_ping(ping e121 up_from 5 ping_stamp 2026-02-11T23:31:31.931846+0000/74.607254028s send_stamp 74.607254028s delta_ub 7.219604492s) -- 0x4ecd200 con 0x48cd400 2026-02-11T23:31:31.931+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6830/2699377081,v1:127.0.0.1:6831/2699377081] -- osd_ping(ping e121 up_from 5 ping_stamp 2026-02-11T23:31:31.931846+0000/74.607254028s send_stamp 74.607254028s delta_ub 7.219604492s) -- 0x4ecd380 con 0x48cd800 2026-02-11T23:31:31.931+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6840/3331151400,v1:127.0.0.1:6841/3331151400] -- osd_ping(ping e121 up_from 5 ping_stamp 2026-02-11T23:31:31.931846+0000/74.607254028s send_stamp 74.607254028s delta_ub 9.836743355s) -- 0x4ecd500 con 0x507bc00 2026-02-11T23:31:31.931+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6838/3331151400,v1:127.0.0.1:6839/3331151400] -- osd_ping(ping e121 up_from 5 ping_stamp 2026-02-11T23:31:31.931846+0000/74.607254028s send_stamp 74.607254028s delta_ub 9.836743355s) -- 0x4ecd680 con 0x5282000 2026-02-11T23:31:31.931+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6816/3043114013 22 ==== osd_ping(ping_reply e121 up_from 10 ping_stamp 2026-02-11T23:31:31.931846+0000/74.607254028s send_stamp 72.595642090s delta_ub -2.011603594s) ==== 2033+0+0 (crc 0 0 0) 0x4ecd500 con 0x48c4400 2026-02-11T23:31:31.931+0000 7f416abfa640 20 osd.0 121 get_con_osd_cluster to osd.1 from_epoch 121 2026-02-11T23:31:31.931+0000 7f416abfa640 20 osd.0 121 get_nextmap_reserved map_reservations: {121=1} 2026-02-11T23:31:31.931+0000 7f416abfa640 20 osd.0 121 release_map epoch: 121 2026-02-11T23:31:31.931+0000 7f416abfa640 10 osd.0 121 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 121 is not newer than session's projected_epoch of 121 2026-02-11T23:31:31.931+0000 7f416abfa640 20 osd.0 121 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011710644s,-2.011603594s]) 2026-02-11T23:31:31.931+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6814/3043114013 22 ==== osd_ping(ping_reply e121 up_from 10 ping_stamp 2026-02-11T23:31:31.931846+0000/74.607254028s send_stamp 72.595657349s delta_ub -2.011587858s) ==== 2033+0+0 (crc 0 0 0) 0x4ecd680 con 0x4640c00 2026-02-11T23:31:31.931+0000 7f416c3fd640 20 osd.0 121 get_con_osd_cluster to osd.1 from_epoch 121 2026-02-11T23:31:31.931+0000 7f416c3fd640 20 osd.0 121 get_nextmap_reserved map_reservations: {121=1} 2026-02-11T23:31:31.931+0000 7f416c3fd640 20 osd.0 121 release_map epoch: 121 2026-02-11T23:31:31.931+0000 7f416c3fd640 10 osd.0 121 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 121 is not newer than session's projected_epoch of 121 2026-02-11T23:31:31.931+0000 7f416c3fd640 20 osd.0 121 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011705875s,-2.011587858s]) 2026-02-11T23:31:31.931+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6822/2365263801 21 ==== osd_ping(ping_reply e121 up_from 14 ping_stamp 2026-02-11T23:31:31.931846+0000/74.607254028s send_stamp 70.040534973s delta_ub -4.566718578s) ==== 2033+0+0 (crc 0 0 0) 0x4ecd500 con 0x48cd000 2026-02-11T23:31:31.931+0000 7f416abfa640 20 osd.0 121 get_con_osd_cluster to osd.2 from_epoch 121 2026-02-11T23:31:31.931+0000 7f416abfa640 20 osd.0 121 get_nextmap_reserved map_reservations: {121=1} 2026-02-11T23:31:31.931+0000 7f416abfa640 20 osd.0 121 release_map epoch: 121 2026-02-11T23:31:31.931+0000 7f416abfa640 10 osd.0 121 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 121 is not newer than session's projected_epoch of 121 2026-02-11T23:31:31.931+0000 7f416abfa640 20 osd.0 121 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566853046s,-4.566718578s]) 2026-02-11T23:31:31.931+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6840/3331151400 20 ==== osd_ping(ping_reply e121 up_from 23 ping_stamp 2026-02-11T23:31:31.931846+0000/74.607254028s send_stamp 64.770721436s delta_ub -9.836530685s) ==== 2033+0+0 (crc 0 0 0) 0x4ecd500 con 0x507bc00 2026-02-11T23:31:31.931+0000 7f416abfa640 20 osd.0 121 get_con_osd_cluster to osd.4 from_epoch 121 2026-02-11T23:31:31.931+0000 7f416abfa640 20 osd.0 121 get_nextmap_reserved map_reservations: {121=1} 2026-02-11T23:31:31.931+0000 7f416abfa640 20 osd.0 121 release_map epoch: 121 2026-02-11T23:31:31.931+0000 7f416abfa640 10 osd.0 121 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 121 is not newer than session's projected_epoch of 121 2026-02-11T23:31:31.931+0000 7f416abfa640 20 osd.0 121 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836682320s,-9.836530685s]) 2026-02-11T23:31:31.931+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6830/2699377081 21 ==== osd_ping(ping_reply e121 up_from 19 ping_stamp 2026-02-11T23:31:31.931846+0000/74.607254028s send_stamp 67.387893677s delta_ub -7.219361782s) ==== 2033+0+0 (crc 0 0 0) 0x4ecd680 con 0x48cd800 2026-02-11T23:31:31.931+0000 7f416c3fd640 20 osd.0 121 get_con_osd_cluster to osd.3 from_epoch 121 2026-02-11T23:31:31.931+0000 7f416c3fd640 20 osd.0 121 get_nextmap_reserved map_reservations: {121=1} 2026-02-11T23:31:31.931+0000 7f416c3fd640 20 osd.0 121 release_map epoch: 121 2026-02-11T23:31:31.931+0000 7f416c3fd640 10 osd.0 121 maybe_share_map: con v2:127.0.0.1:6828/2699377081 updating session's projected_epoch from 119 to ping map epoch of 121 2026-02-11T23:31:31.931+0000 7f416cbfe640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6824/2365263801 21 ==== osd_ping(ping_reply e121 up_from 14 ping_stamp 2026-02-11T23:31:31.931846+0000/74.607254028s send_stamp 70.040550232s delta_ub -4.566699505s) ==== 2033+0+0 (crc 0 0 0) 0x4ec2600 con 0x48c4000 2026-02-11T23:31:31.931+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6832/2699377081 21 ==== osd_ping(ping_reply e121 up_from 19 ping_stamp 2026-02-11T23:31:31.931846+0000/74.607254028s send_stamp 67.387908936s delta_ub -7.219342709s) ==== 2033+0+0 (crc 0 0 0) 0x4ecd500 con 0x48cd400 2026-02-11T23:31:31.931+0000 7f416c3fd640 10 osd.0 121 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 121 is not newer than session's projected_epoch of 121 2026-02-11T23:31:31.931+0000 7f416c3fd640 20 osd.0 121 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219520092s,-7.219361782s]) 2026-02-11T23:31:31.931+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6838/3331151400 20 ==== osd_ping(ping_reply e121 up_from 23 ping_stamp 2026-02-11T23:31:31.931846+0000/74.607254028s send_stamp 64.770736694s delta_ub -9.836517334s) ==== 2033+0+0 (crc 0 0 0) 0x4ecd680 con 0x5282000 2026-02-11T23:31:31.931+0000 7f416cbfe640 20 osd.0 121 get_con_osd_cluster to osd.2 from_epoch 121 2026-02-11T23:31:31.931+0000 7f416cbfe640 20 osd.0 121 get_nextmap_reserved map_reservations: {121=1} 2026-02-11T23:31:31.931+0000 7f416cbfe640 20 osd.0 121 release_map epoch: 121 2026-02-11T23:31:31.931+0000 7f416cbfe640 10 osd.0 121 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 121 is not newer than session's projected_epoch of 121 2026-02-11T23:31:31.931+0000 7f416cbfe640 20 osd.0 121 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566879272s,-4.566699505s]) 2026-02-11T23:31:31.931+0000 7f416abfa640 20 osd.0 121 get_con_osd_cluster to osd.3 from_epoch 121 2026-02-11T23:31:31.931+0000 7f416abfa640 20 osd.0 121 get_nextmap_reserved map_reservations: {121=1} 2026-02-11T23:31:31.931+0000 7f416abfa640 20 osd.0 121 release_map epoch: 121 2026-02-11T23:31:31.931+0000 7f416abfa640 10 osd.0 121 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 121 is not newer than session's projected_epoch of 121 2026-02-11T23:31:31.931+0000 7f416abfa640 20 osd.0 121 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219546318s,-7.219342709s]) 2026-02-11T23:31:31.931+0000 7f416c3fd640 20 osd.0 121 get_con_osd_cluster to osd.4 from_epoch 121 2026-02-11T23:31:31.931+0000 7f416c3fd640 20 osd.0 121 get_nextmap_reserved map_reservations: {121=1} 2026-02-11T23:31:31.931+0000 7f416c3fd640 20 osd.0 121 release_map epoch: 121 2026-02-11T23:31:31.931+0000 7f416c3fd640 10 osd.0 121 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 121 is not newer than session's projected_epoch of 121 2026-02-11T23:31:31.931+0000 7f416c3fd640 20 osd.0 121 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836734772s,-9.836517334s]) 2026-02-11T23:31:32.330+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:32.330+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:32.330+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:02.330997+0000) 2026-02-11T23:31:32.330+0000 7f415bf00640 10 log_client log_queue is 1 last_log 3 sent 2 num 1 unsent 1 sending 1 2026-02-11T23:31:32.330+0000 7f415bf00640 10 log_client will send 2026-02-11T23:31:31.931843+0000 osd.0 (osd.0) 3 : cluster [ERR] full status failsafe engaged, dropping updates, now 100% full 2026-02-11T23:31:32.330+0000 7f415bf00640 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:31:32.330+0000 7f415bf00640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> v2:127.0.0.1:7180/0 -- log(1 entries from seq 3 at 2026-02-11T23:31:31.931843+0000) -- 0x5954000 con 0x4acbc00 2026-02-11T23:31:32.355+0000 7f41633eb640 10 osd.0 121 tick 2026-02-11T23:31:32.355+0000 7f41633eb640 20 osd.0 121 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:32.380+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 56 ==== log(last 3) ==== 28+0+0 (crc 0 0 0) 0x4ec2600 con 0x4acbc00 2026-02-11T23:31:32.380+0000 7f415cf02640 10 log_client handle_log_ack log(last 3) 2026-02-11T23:31:32.380+0000 7f415cf02640 10 log_client logged 2026-02-11T23:31:31.931843+0000 osd.0 (osd.0) 3 : cluster [ERR] full status failsafe engaged, dropping updates, now 100% full 2026-02-11T23:31:32.534+0000 7f4162bea640 10 osd.0 121 tick_without_osd_lock 2026-02-11T23:31:32.534+0000 7f4162bea640 0 osd.0 121 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:32.534+0000 7f4162bea640 10 osd.0 121 send_full_update want state full 2026-02-11T23:31:32.534+0000 7f4162bea640 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:31:32.534+0000 7f4162bea640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> v2:127.0.0.1:7180/0 -- osd_full(e121 full v121) -- 0x672de40 con 0x4acbc00 2026-02-11T23:31:32.534+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:32.534+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.221 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:32.534+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:32.534, recovery is active?:false restrictions:<....> 2026-02-11T23:31:32.534+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:32.534+0000 7f4162bea640 10 osd.0 121 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:32.534+0000 7f4162bea640 20 osd.0 121 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:32.534+0000 7f4162bea640 10 osd.0 121 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:32.534+0000 7f4162bea640 20 osd.0 121 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:32.585+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== mon.0 v2:127.0.0.1:7180/0 57 ==== osd_map(121..122 src has 1..122) ==== 1216+0+0 (crc 0 0 0) 0x5954000 con 0x4acbc00 2026-02-11T23:31:32.585+0000 7f415cf02640 20 osd.0 121 OSD::ms_dispatch: osd_map(121..122 src has 1..122) 2026-02-11T23:31:32.585+0000 7f415cf02640 20 osd.0 121 _dispatch 0x5954000 osd_map(121..122 src has 1..122) 2026-02-11T23:31:32.585+0000 7f415cf02640 10 monclient: _renew_subs 2026-02-11T23:31:32.585+0000 7f415cf02640 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:31:32.585+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> v2:127.0.0.1:7180/0 -- mon_subscribe({osdmap=123}) -- 0x56461a0 con 0x4acbc00 2026-02-11T23:31:32.585+0000 7f415cf02640 3 osd.0 121 handle_osd_map epochs [121,122], i have 121, src has [1,122] 2026-02-11T23:31:32.585+0000 7f415cf02640 10 osd.0 121 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:31:32.585+0000 7f415cf02640 10 osd.0 121 handle_osd_map got inc map for epoch 122 2026-02-11T23:31:32.585+0000 7f415cf02640 20 osd.0 121 got_full_map 122, nothing requested 2026-02-11T23:31:32.585+0000 7f415cf02640 20 osd.0 121 trim_maps: min=1 oldest_map=1 2026-02-11T23:31:32.585+0000 7f415cf02640 20 osd.0 121 handle_osd_map pg_num_history pg_num_history(e122 pg_nums {1={27=1},2={34=1},3={42=1},4={49=1},5={56=1},6={64=1},7={70=1},8={77=1},9={85=1},10={92=1}} deleted_pools 99,2,100,3,101,5,102,6,103,7,104,8,105,9,106,10) 2026-02-11T23:31:32.585+0000 7f415cf02640 10 snap_mapper record_purged_snaps purged_snaps {122={}} 2026-02-11T23:31:32.585+0000 7f415cf02640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:31:32.585+0000 7f415cf02640 10 write_superblock sb(cb0a9481-482d-477a-a81c-1446b6498370 osd.0 01217804-d5cf-416e-93e4-5f1242b1c391 e122 maps [1~122] lci=[5,122] tlb=1) 2026-02-11T23:31:32.585+0000 7f41591a6640 10 osd.0 121 _committed_osd_maps 122..122 2026-02-11T23:31:32.585+0000 7f41591a6640 10 osd.0 121 advance to epoch 122 (<= last 122 <= newest_map 122) 2026-02-11T23:31:32.585+0000 7f41591a6640 20 osd.0 122 consume_map version 122 2026-02-11T23:31:32.585+0000 7f41591a6640 20 osd.0 121 await_reserved_maps epoch:122 2026-02-11T23:31:32.585+0000 7f41591a6640 20 osd.0 121 await_reserved_maps done 0 2026-02-11T23:31:32.585+0000 7f41591a6640 20 osd.0 122 consume_map 122 -- publish done 2026-02-11T23:31:32.585+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:31:32.585+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x5efde40 2026-02-11T23:31:32.585+0000 7f41591a6640 20 osd.0 122 identify_splits_and_merges 4.0 e121 to e122 2026-02-11T23:31:32.585+0000 7f41591a6640 20 osd.0 122 identify_splits_and_merges 4.0 e121 to e122 pg_nums {49=1} 2026-02-11T23:31:32.585+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x3701340 2026-02-11T23:31:32.585+0000 7f41591a6640 20 osd.0 122 identify_splits_and_merges 1.0 e121 to e122 2026-02-11T23:31:32.585+0000 7f41591a6640 20 osd.0 122 identify_splits_and_merges 1.0 e121 to e122 pg_nums {27=1} 2026-02-11T23:31:32.585+0000 7f41591a6640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:32.585+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:32.585+0000 7f41591a6640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:32.585+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:32.585+0000 7f41591a6640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:32.585+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:32.585+0000 7f41591a6640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:32.585+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:32.585+0000 7f41591a6640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:32.585+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:32.585+0000 7f41591a6640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:32.585+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:32.585+0000 7f41591a6640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:32.585+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:32.585+0000 7f41591a6640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:32.585+0000 7f41591a6640 20 osd.0 122 prune_pg_created 2026-02-11T23:31:32.585+0000 7f41591a6640 10 osd.0:0.consume_map 122 (was 121) 2026-02-11T23:31:32.585+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:31:32.585+0000 7f41591a6640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:31:32.585+0000 7f41591a6640 10 osd.0:1.consume_map 122 (was 121) 2026-02-11T23:31:32.585+0000 7f41591a6640 10 osd.0:2.consume_map 122 (was 121) 2026-02-11T23:31:32.585+0000 7f41591a6640 10 osd.0:3.consume_map 122 (was 121) 2026-02-11T23:31:32.585+0000 7f41591a6640 10 osd.0:4.consume_map 122 (was 121) 2026-02-11T23:31:32.585+0000 7f41591a6640 10 osd.0:5.consume_map 122 (was 121) 2026-02-11T23:31:32.585+0000 7f41591a6640 10 osd.0:6.consume_map 122 (was 121) 2026-02-11T23:31:32.585+0000 7f41591a6640 10 osd.0:7.consume_map 122 (was 121) 2026-02-11T23:31:32.585+0000 7f41591a6640 10 osd.0 122 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:31:32.585+0000 7f41591a6640 15 osd.0 122 enqueue_peering_evt 4.0 epoch_sent: 122 epoch_requested: 122 NullEvt 2026-02-11T23:31:32.585+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 122 epoch_requested: 122 NullEvt) class_id 2 prio 255 cost 10 e122) 2026-02-11T23:31:32.585+0000 7f41591a6640 15 osd.0 122 enqueue_peering_evt 1.0 epoch_sent: 122 epoch_requested: 122 NullEvt 2026-02-11T23:31:32.585+0000 7f41591a6640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 122 epoch_requested: 122 NullEvt) class_id 2 prio 255 cost 10 e122) 2026-02-11T23:31:32.585+0000 7f41591a6640 7 osd.0 122 activate_map version 122 2026-02-11T23:31:32.585+0000 7f4150194640 20 osd.0 122 agent_entry empty queue 2026-02-11T23:31:32.585+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:32.585+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 122 epoch_requested: 122 NullEvt) class_id 2immediate prio 255 cost 10 e122) queued 2026-02-11T23:31:32.585+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:32.585+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 122 epoch_requested: 122 NullEvt) class_id 2immediate prio 255 cost 10 e122) pg 0x5978000 2026-02-11T23:31:32.585+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:32.585+0000 7f414f993640 10 osd.0 pg_epoch: 121 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/120 les/c/f=121/121/0 sis=120) [4,0] r=1 lpr=120 crt=111'600 active mbc={}] handle_advance_map: 122 2026-02-11T23:31:32.585+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 122 epoch_requested: 122 NullEvt) class_id 2immediate prio 255 cost 10 e122) queued 2026-02-11T23:31:32.585+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:32.585+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 122 epoch_requested: 122 NullEvt) class_id 2immediate prio 255 cost 10 e122) pg 0x4ef0000 2026-02-11T23:31:32.585+0000 7f414f993640 20 osd.0:0.update_pg_epoch 4.0 121 -> 122 2026-02-11T23:31:32.585+0000 7f414f993640 10 osd.0 pg_epoch: 121 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/120 les/c/f=121/121/0 sis=120) [4,0] r=1 lpr=120 crt=111'600 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:31:32.585+0000 7f414b98b640 10 osd.0 pg_epoch: 121 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/0 sis=118) [1,0] r=1 lpr=118 crt=111'600 active mbc={}] handle_advance_map: 122 2026-02-11T23:31:32.585+0000 7f414b98b640 20 osd.0:0.update_pg_epoch 1.0 121 -> 122 2026-02-11T23:31:32.585+0000 7f414f993640 10 osd.0 pg_epoch: 122 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/120 les/c/f=121/121/0 sis=120) [4,0] r=1 lpr=120 crt=111'600 active mbc={}] state: Started advmap 2026-02-11T23:31:32.585+0000 7f414b98b640 10 osd.0 pg_epoch: 121 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/0 sis=118) [1,0] r=1 lpr=118 crt=111'600 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:31:32.585+0000 7f414f993640 10 osd.0 pg_epoch: 122 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/120 les/c/f=121/121/0 sis=120) [4,0] r=1 lpr=120 crt=111'600 active mbc={}] PeeringState::check_full_transition pool was marked full in 122 2026-02-11T23:31:32.585+0000 7f414b98b640 10 osd.0 pg_epoch: 122 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/0 sis=118) [1,0] r=1 lpr=118 crt=111'600 active mbc={}] state: Started advmap 2026-02-11T23:31:32.585+0000 7f414f993640 10 osd.0 pg_epoch: 122 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/120 les/c/f=121/121/122 sis=120) [4,0] r=1 lpr=120 crt=111'600 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:31:32.585+0000 7f414b98b640 10 osd.0 pg_epoch: 122 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/0 sis=118) [1,0] r=1 lpr=118 crt=111'600 active mbc={}] PeeringState::check_full_transition pool was marked full in 122 2026-02-11T23:31:32.585+0000 7f414f993640 10 osd.0 pg_epoch: 122 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/120 les/c/f=121/121/122 sis=120) [4,0] r=1 lpr=120 crt=111'600 active mbc={}] plpg_on_pool_change 2026-02-11T23:31:32.585+0000 7f414b98b640 10 osd.0 pg_epoch: 122 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/122 sis=118) [1,0] r=1 lpr=118 crt=111'600 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:31:32.585+0000 7f414f993640 20 osd.0 pg_epoch: 122 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/120 les/c/f=121/121/122 sis=120) [4,0] r=1 lpr=120 crt=111'600 active mbc={}] hit_set_clear 2026-02-11T23:31:32.585+0000 7f414b98b640 10 osd.0 pg_epoch: 122 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/122 sis=118) [1,0] r=1 lpr=118 crt=111'600 active mbc={}] plpg_on_pool_change 2026-02-11T23:31:32.585+0000 7f414b98b640 20 osd.0 pg_epoch: 122 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/122 sis=118) [1,0] r=1 lpr=118 crt=111'600 active mbc={}] hit_set_clear 2026-02-11T23:31:32.585+0000 7f414f993640 20 osd.0 pg_epoch: 122 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/120 les/c/f=121/121/122 sis=120) [4,0] r=1 lpr=120 crt=111'600 active mbc={}] agent_stop 2026-02-11T23:31:32.585+0000 7f414f993640 20 osd.0 122 advance_pg new pool opts old pool opts 2026-02-11T23:31:32.585+0000 7f414f993640 10 osd.0 pg_epoch: 122 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/120 les/c/f=121/121/122 sis=120) [4,0] r=1 lpr=120 crt=111'600 active mbc={}] handle_activate_map: epoch range: 122..122 2026-02-11T23:31:32.585+0000 7f414f993640 10 osd.0 pg_epoch: 122 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/120 les/c/f=121/121/122 sis=120) [4,0] r=1 lpr=120 crt=111'600 active mbc={}] PeeringState::activate_map 2026-02-11T23:31:32.585+0000 7f414b98b640 20 osd.0 pg_epoch: 122 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/122 sis=118) [1,0] r=1 lpr=118 crt=111'600 active mbc={}] agent_stop 2026-02-11T23:31:32.585+0000 7f414b98b640 20 osd.0 122 advance_pg new pool opts old pool opts 2026-02-11T23:31:32.585+0000 7f414f993640 20 osd.0 pg_epoch: 122 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/120 les/c/f=121/121/122 sis=120) [4,0] r=1 lpr=120 crt=111'600 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 121 while current is 122 2026-02-11T23:31:32.585+0000 7f414b98b640 10 osd.0 pg_epoch: 122 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/122 sis=118) [1,0] r=1 lpr=118 crt=111'600 active mbc={}] handle_activate_map: epoch range: 122..122 2026-02-11T23:31:32.585+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:32.585+0000 7f414b98b640 10 osd.0 pg_epoch: 122 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/122 sis=118) [1,0] r=1 lpr=118 crt=111'600 active mbc={}] PeeringState::activate_map 2026-02-11T23:31:32.585+0000 7f414b98b640 20 osd.0 pg_epoch: 122 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/122 sis=118) [1,0] r=1 lpr=118 crt=111'600 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 119 while current is 122 2026-02-11T23:31:32.585+0000 7f414f993640 10 osd.0 pg_epoch: 122 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/120 les/c/f=121/121/122 sis=120) [4,0] r=1 lpr=120 crt=111'600 active mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:31:32.585+0000 7f414f993640 10 osd.0 pg_epoch: 122 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/120 les/c/f=121/121/122 sis=120) [4,0] r=1 lpr=120 crt=111'600 active mbc={}] do_peering_event: epoch_sent: 122 epoch_requested: 122 NullEvt 2026-02-11T23:31:32.585+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:32.585+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:32.585+0000 7f414b98b640 10 osd.0 pg_epoch: 122 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/122 sis=118) [1,0] r=1 lpr=118 crt=111'600 active mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:31:32.585+0000 7f414b98b640 10 osd.0 pg_epoch: 122 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/122 sis=118) [1,0] r=1 lpr=118 crt=111'600 active mbc={}] do_peering_event: epoch_sent: 122 epoch_requested: 122 NullEvt 2026-02-11T23:31:32.585+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:32.585+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:32.585+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:32.585+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:32.585+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:32.585+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:32.585+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:32.849+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.3 127.0.0.1:0/2699377081 19 ==== osd_ping(ping e121 up_from 19 ping_stamp 2026-02-11T23:31:32.850439+0000/68.306381226s send_stamp 68.306381226s delta_ub -7.219342709s) ==== 2033+0+0 (crc 0 0 0) 0x4ecd680 con 0x48cc400 2026-02-11T23:31:32.849+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219535828s,-7.219342709s]) 2026-02-11T23:31:32.849+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.3 127.0.0.1:0/2699377081 19 ==== osd_ping(ping e121 up_from 19 ping_stamp 2026-02-11T23:31:32.850439+0000/68.306381226s send_stamp 68.306381226s delta_ub -7.219342709s) ==== 2033+0+0 (crc 0 0 0) 0x4a89200 con 0x48cc800 2026-02-11T23:31:32.849+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:32.850439+0000/68.306381226s send_stamp 75.525917053s delta_ub 7.219535828s) -- 0x4ecc180 con 0x48cc400 2026-02-11T23:31:32.849+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.3 from_epoch 122 2026-02-11T23:31:32.849+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:32.849+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:32.849+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6828/2699377081 map epoch 121 -> 122 (shared) 2026-02-11T23:31:32.849+0000 7f416c3fd640 10 osd.0 122 send_incremental_map epoch range: (122, 122] to v2:127.0.0.1:6828/2699377081 2026-02-11T23:31:32.849+0000 7f416c3fd640 10 osd.0 122 add_map_inc_bl 122 912 bytes 2026-02-11T23:31:32.849+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6828/2699377081,v1:127.0.0.1:6829/2699377081] -- osd_map(122..122 src has 1..122) -- 0x5954700 con 0x48ccc00 2026-02-11T23:31:32.849+0000 7f416cbfe640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219691753s,-7.219342709s]) 2026-02-11T23:31:32.849+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:32.850439+0000/68.306381226s send_stamp 75.526069641s delta_ub 7.219691753s) -- 0x4c9a480 con 0x48cc800 2026-02-11T23:31:32.849+0000 7f416cbfe640 20 osd.0 122 get_con_osd_cluster to osd.3 from_epoch 122 2026-02-11T23:31:32.849+0000 7f416cbfe640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:32.849+0000 7f416cbfe640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:32.849+0000 7f416cbfe640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:32.997+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.2 127.0.0.1:0/2365263801 21 ==== osd_ping(ping e121 up_from 14 ping_stamp 2026-02-11T23:31:32.998104+0000/71.106697083s send_stamp 71.106697083s delta_ub -4.566699505s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a480 con 0x4641000 2026-02-11T23:31:32.997+0000 7f416cbfe640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566873074s,-4.566699505s]) 2026-02-11T23:31:32.997+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.2 127.0.0.1:0/2365263801 21 ==== osd_ping(ping e121 up_from 14 ping_stamp 2026-02-11T23:31:32.998104+0000/71.106697083s send_stamp 71.106697083s delta_ub -4.566699505s) ==== 2033+0+0 (crc 0 0 0) 0x4ecc180 con 0x48cc000 2026-02-11T23:31:32.997+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:32.998104+0000/71.106697083s send_stamp 75.673576355s delta_ub 4.566873074s) -- 0x4a89200 con 0x4641000 2026-02-11T23:31:32.997+0000 7f416cbfe640 20 osd.0 122 get_con_osd_cluster to osd.2 from_epoch 122 2026-02-11T23:31:32.997+0000 7f416cbfe640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:32.997+0000 7f416cbfe640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:32.997+0000 7f416cbfe640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6820/2365263801 map epoch 121 -> 122 (shared) 2026-02-11T23:31:32.997+0000 7f416cbfe640 10 osd.0 122 send_incremental_map epoch range: (122, 122] to v2:127.0.0.1:6820/2365263801 2026-02-11T23:31:32.997+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6820/2365263801,v1:127.0.0.1:6821/2365263801] -- osd_map(122..122 src has 1..122) -- 0x5954700 con 0x507b800 2026-02-11T23:31:32.997+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566897869s,-4.566699505s]) 2026-02-11T23:31:32.997+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:32.998104+0000/71.106697083s send_stamp 75.673599243s delta_ub 4.566897869s) -- 0x4ecd680 con 0x48cc000 2026-02-11T23:31:32.997+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.2 from_epoch 122 2026-02-11T23:31:32.997+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:32.997+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:32.997+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:33.206+0000 7f415b6ff640 20 osd.0 122 reports for 0 queries 2026-02-11T23:31:33.206+0000 7f415b6ff640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] -- mgrreport(unknown.0 +0-0 packed 3294 daemon_metrics=2) -- 0x4f9e700 con 0x4aca800 2026-02-11T23:31:33.206+0000 7f415b6ff640 15 osd.0 122 collect_pg_stats 2026-02-11T23:31:33.207+0000 7f415b6ff640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] -- pg_stats(0 pgs seq 21474836500 v 0) -- 0x4aa1500 con 0x4aca800 2026-02-11T23:31:33.330+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:33.330+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:33.330+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:03.331037+0000) 2026-02-11T23:31:33.387+0000 7f41633eb640 10 osd.0 122 tick 2026-02-11T23:31:33.387+0000 7f41633eb640 20 osd.0 122 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:33.549+0000 7f4162bea640 10 osd.0 122 tick_without_osd_lock 2026-02-11T23:31:33.549+0000 7f4162bea640 0 osd.0 122 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:33.549+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:33.549+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:33.549+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:33.549+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.203 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:33.549+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:33.550, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:33.549+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:33.549+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:33.549+0000 7f4162bea640 20 osd.0 122 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:33.549+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:33.549+0000 7f4162bea640 20 osd.0 122 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:34.097+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.2 127.0.0.1:0/2365263801 22 ==== osd_ping(ping e122 up_from 14 ping_stamp 2026-02-11T23:31:34.098239+0000/72.206832886s send_stamp 72.206832886s delta_ub -4.566755295s) ==== 2033+0+0 (crc 0 0 0) 0x4a89200 con 0x4641000 2026-02-11T23:31:34.097+0000 7f416cbfe640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566860199s,-4.566755295s]) 2026-02-11T23:31:34.097+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:34.098239+0000/72.206832886s send_stamp 76.773696899s delta_ub 4.566860199s) -- 0x4c9a480 con 0x4641000 2026-02-11T23:31:34.097+0000 7f416cbfe640 20 osd.0 122 get_con_osd_cluster to osd.2 from_epoch 122 2026-02-11T23:31:34.097+0000 7f416cbfe640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:34.097+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.2 127.0.0.1:0/2365263801 22 ==== osd_ping(ping e122 up_from 14 ping_stamp 2026-02-11T23:31:34.098239+0000/72.206832886s send_stamp 72.206832886s delta_ub -4.566755295s) ==== 2033+0+0 (crc 0 0 0) 0x4ecd680 con 0x48cc000 2026-02-11T23:31:34.097+0000 7f416cbfe640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:34.097+0000 7f416cbfe640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:34.097+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566901207s,-4.566755295s]) 2026-02-11T23:31:34.097+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:34.098239+0000/72.206832886s send_stamp 76.773735046s delta_ub 4.566901207s) -- 0x4ecc180 con 0x48cc000 2026-02-11T23:31:34.097+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.2 from_epoch 122 2026-02-11T23:31:34.097+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:34.097+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:34.097+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:34.231+0000 7f4147983640 5 osd.0 122 heartbeat osd_stat(store_statfs(0x0/0x0/0x384000, data 0x53a3e5/0x5e6000, compress 0x0/0x0/0x0, omap 0xa403, meta 0x1a45bfd), peers [1,2,3,4] op hist []) 2026-02-11T23:31:34.231+0000 7f4147983640 20 osd.0 122 check_full_status cur ratio 1, physical ratio 1, new state failsafe 2026-02-11T23:31:34.231+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6816/3043114013,v1:127.0.0.1:6817/3043114013] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:34.232022+0000/76.907432556s send_stamp 76.907432556s delta_ub 2.011705875s) -- 0x4ecd800 con 0x48c4400 2026-02-11T23:31:34.231+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6814/3043114013,v1:127.0.0.1:6815/3043114013] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:34.232022+0000/76.907432556s send_stamp 76.907432556s delta_ub 2.011705875s) -- 0x4ecd980 con 0x4640c00 2026-02-11T23:31:34.231+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6824/2365263801,v1:127.0.0.1:6825/2365263801] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:34.232022+0000/76.907432556s send_stamp 76.907432556s delta_ub 4.566901207s) -- 0x4ecdb00 con 0x48c4000 2026-02-11T23:31:34.231+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6822/2365263801,v1:127.0.0.1:6823/2365263801] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:34.232022+0000/76.907432556s send_stamp 76.907432556s delta_ub 4.566901207s) -- 0x6788780 con 0x48cd000 2026-02-11T23:31:34.231+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6832/2699377081,v1:127.0.0.1:6833/2699377081] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:34.232022+0000/76.907432556s send_stamp 76.907432556s delta_ub 7.219691753s) -- 0x63c5380 con 0x48cd400 2026-02-11T23:31:34.231+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6830/2699377081,v1:127.0.0.1:6831/2699377081] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:34.232022+0000/76.907432556s send_stamp 76.907432556s delta_ub 7.219691753s) -- 0x4eccf00 con 0x48cd800 2026-02-11T23:31:34.231+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6840/3331151400,v1:127.0.0.1:6841/3331151400] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:34.232022+0000/76.907432556s send_stamp 76.907432556s delta_ub 9.836734772s) -- 0x4ecc300 con 0x507bc00 2026-02-11T23:31:34.231+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6838/3331151400,v1:127.0.0.1:6839/3331151400] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:34.232022+0000/76.907432556s send_stamp 76.907432556s delta_ub 9.836734772s) -- 0x4ecd380 con 0x5282000 2026-02-11T23:31:34.231+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6816/3043114013 23 ==== osd_ping(ping_reply e122 up_from 10 ping_stamp 2026-02-11T23:31:34.232022+0000/76.907432556s send_stamp 74.895835876s delta_ub -2.011594772s) ==== 2033+0+0 (crc 0 0 0) 0x4ecc300 con 0x48c4400 2026-02-11T23:31:34.231+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.1 from_epoch 122 2026-02-11T23:31:34.231+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:34.231+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:34.231+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6814/3043114013 23 ==== osd_ping(ping_reply e122 up_from 10 ping_stamp 2026-02-11T23:31:34.232022+0000/76.907432556s send_stamp 74.895851135s delta_ub -2.011577129s) ==== 2033+0+0 (crc 0 0 0) 0x4ecd380 con 0x4640c00 2026-02-11T23:31:34.231+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6812/3043114013 updating session's projected_epoch from 121 to ping map epoch of 122 2026-02-11T23:31:34.231+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:34.231+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011711597s,-2.011594772s]) 2026-02-11T23:31:34.231+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.1 from_epoch 122 2026-02-11T23:31:34.231+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:34.231+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:34.231+0000 7f416cbfe640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6824/2365263801 22 ==== osd_ping(ping_reply e122 up_from 14 ping_stamp 2026-02-11T23:31:34.232022+0000/76.907432556s send_stamp 72.340721130s delta_ub -4.566709042s) ==== 2033+0+0 (crc 0 0 0) 0x4ecdb00 con 0x48c4000 2026-02-11T23:31:34.231+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6822/2365263801 22 ==== osd_ping(ping_reply e122 up_from 14 ping_stamp 2026-02-11T23:31:34.232022+0000/76.907432556s send_stamp 72.340698242s delta_ub -4.566722870s) ==== 2033+0+0 (crc 0 0 0) 0x4ecc300 con 0x48cd000 2026-02-11T23:31:34.231+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:34.231+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011718988s,-2.011577129s]) 2026-02-11T23:31:34.231+0000 7f416cbfe640 20 osd.0 122 get_con_osd_cluster to osd.2 from_epoch 122 2026-02-11T23:31:34.231+0000 7f416cbfe640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:34.231+0000 7f416cbfe640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:34.231+0000 7f416cbfe640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:34.231+0000 7f416cbfe640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566882133s,-4.566709042s]) 2026-02-11T23:31:34.231+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6830/2699377081 22 ==== osd_ping(ping_reply e122 up_from 19 ping_stamp 2026-02-11T23:31:34.232022+0000/76.907432556s send_stamp 69.688079834s delta_ub -7.219346046s) ==== 2033+0+0 (crc 0 0 0) 0x4ecd380 con 0x48cd800 2026-02-11T23:31:34.231+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.3 from_epoch 122 2026-02-11T23:31:34.231+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:34.231+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:34.231+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:34.231+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219529152s,-7.219346046s]) 2026-02-11T23:31:34.231+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.2 from_epoch 122 2026-02-11T23:31:34.231+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:34.231+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6838/3331151400 21 ==== osd_ping(ping_reply e122 up_from 23 ping_stamp 2026-02-11T23:31:34.232022+0000/76.907432556s send_stamp 67.070930481s delta_ub -9.836495399s) ==== 2033+0+0 (crc 0 0 0) 0x4ecd380 con 0x5282000 2026-02-11T23:31:34.231+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:34.231+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:34.231+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566921711s,-4.566722870s]) 2026-02-11T23:31:34.231+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.4 from_epoch 122 2026-02-11T23:31:34.231+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:34.231+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:34.231+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6836/3331151400 updating session's projected_epoch from 121 to ping map epoch of 122 2026-02-11T23:31:34.231+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:34.231+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836709976s,-9.836495399s]) 2026-02-11T23:31:34.231+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6832/2699377081 22 ==== osd_ping(ping_reply e122 up_from 19 ping_stamp 2026-02-11T23:31:34.232022+0000/76.907432556s send_stamp 69.688064575s delta_ub -7.219363689s) ==== 2033+0+0 (crc 0 0 0) 0x4ecc300 con 0x48cd400 2026-02-11T23:31:34.231+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.3 from_epoch 122 2026-02-11T23:31:34.231+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:34.231+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:34.231+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:34.231+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219585419s,-7.219363689s]) 2026-02-11T23:31:34.231+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6840/3331151400 21 ==== osd_ping(ping_reply e122 up_from 23 ping_stamp 2026-02-11T23:31:34.232022+0000/76.907432556s send_stamp 67.070907593s delta_ub -9.836519241s) ==== 2033+0+0 (crc 0 0 0) 0x4ecc300 con 0x507bc00 2026-02-11T23:31:34.231+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.4 from_epoch 122 2026-02-11T23:31:34.231+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:34.231+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:34.231+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:34.231+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836761475s,-9.836519241s]) 2026-02-11T23:31:34.330+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:34.330+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:34.330+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:04.331066+0000) 2026-02-11T23:31:34.342+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.1 127.0.0.1:0/3043114013 24 ==== osd_ping(ping e122 up_from 10 ping_stamp 2026-02-11T23:31:34.343241+0000/75.006980896s send_stamp 75.006980896s delta_ub -2.011577129s) ==== 2033+0+0 (crc 0 0 0) 0x4ecdb00 con 0x4aca000 2026-02-11T23:31:34.342+0000 7f416cbfe640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011704206s,-2.011577129s]) 2026-02-11T23:31:34.342+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:34.343241+0000/75.006980896s send_stamp 77.018692017s delta_ub 2.011704206s) -- 0x4c9a480 con 0x4aca000 2026-02-11T23:31:34.342+0000 7f416cbfe640 20 osd.0 122 get_con_osd_cluster to osd.1 from_epoch 122 2026-02-11T23:31:34.342+0000 7f416cbfe640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:34.342+0000 7f416cbfe640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:34.342+0000 7f416cbfe640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:34.342+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.1 127.0.0.1:0/3043114013 24 ==== osd_ping(ping e122 up_from 10 ping_stamp 2026-02-11T23:31:34.343241+0000/75.006980896s send_stamp 75.006980896s delta_ub -2.011577129s) ==== 2033+0+0 (crc 0 0 0) 0x4ecd380 con 0x4aca400 2026-02-11T23:31:34.342+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011731863s,-2.011577129s]) 2026-02-11T23:31:34.342+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:34.343241+0000/75.006980896s send_stamp 77.018714905s delta_ub 2.011731863s) -- 0x4eccf00 con 0x4aca400 2026-02-11T23:31:34.342+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.1 from_epoch 122 2026-02-11T23:31:34.342+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:34.342+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:34.342+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:34.382+0000 7f41633eb640 10 osd.0 122 tick 2026-02-11T23:31:34.382+0000 7f41633eb640 20 osd.0 122 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:34.525+0000 7f4162bea640 10 osd.0 122 tick_without_osd_lock 2026-02-11T23:31:34.525+0000 7f4162bea640 0 osd.0 122 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:34.525+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:34.525+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:34.525+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:34.525+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.203 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:34.525+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:34.526, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:34.525+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:34.525+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:34.525+0000 7f4162bea640 20 osd.0 122 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:34.525+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:34.525+0000 7f4162bea640 20 osd.0 122 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:35.330+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:35.330+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:35.330+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:05.331128+0000) 2026-02-11T23:31:35.331+0000 7f4147983640 5 osd.0 122 heartbeat osd_stat(store_statfs(0x0/0x0/0x384000, data 0x53a3e5/0x5e6000, compress 0x0/0x0/0x0, omap 0xa403, meta 0x1a45bfd), peers [1,2,3,4] op hist []) 2026-02-11T23:31:35.331+0000 7f4147983640 20 osd.0 122 check_full_status cur ratio 1, physical ratio 1, new state failsafe 2026-02-11T23:31:35.331+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6816/3043114013,v1:127.0.0.1:6817/3043114013] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:35.332207+0000/78.007614136s send_stamp 78.007614136s delta_ub 2.011731863s) -- 0x4eccd80 con 0x48c4400 2026-02-11T23:31:35.331+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6814/3043114013,v1:127.0.0.1:6815/3043114013] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:35.332207+0000/78.007614136s send_stamp 78.007614136s delta_ub 2.011731863s) -- 0x4ecc780 con 0x4640c00 2026-02-11T23:31:35.331+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6824/2365263801,v1:127.0.0.1:6825/2365263801] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:35.332207+0000/78.007614136s send_stamp 78.007614136s delta_ub 4.566921711s) -- 0x4ecca80 con 0x48c4000 2026-02-11T23:31:35.331+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6822/2365263801,v1:127.0.0.1:6823/2365263801] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:35.332207+0000/78.007614136s send_stamp 78.007614136s delta_ub 4.566921711s) -- 0x4ecd680 con 0x48cd000 2026-02-11T23:31:35.331+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6832/2699377081,v1:127.0.0.1:6833/2699377081] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:35.332207+0000/78.007614136s send_stamp 78.007614136s delta_ub 7.219585419s) -- 0x4ecc180 con 0x48cd400 2026-02-11T23:31:35.331+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6830/2699377081,v1:127.0.0.1:6831/2699377081] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:35.332207+0000/78.007614136s send_stamp 78.007614136s delta_ub 7.219585419s) -- 0x4ecd980 con 0x48cd800 2026-02-11T23:31:35.331+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6840/3331151400,v1:127.0.0.1:6841/3331151400] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:35.332207+0000/78.007614136s send_stamp 78.007614136s delta_ub 9.836761475s) -- 0x4ec2180 con 0x507bc00 2026-02-11T23:31:35.331+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6838/3331151400,v1:127.0.0.1:6839/3331151400] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:35.332207+0000/78.007614136s send_stamp 78.007614136s delta_ub 9.836761475s) -- 0x4ec3e00 con 0x5282000 2026-02-11T23:31:35.331+0000 7f416cbfe640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6824/2365263801 23 ==== osd_ping(ping_reply e122 up_from 14 ping_stamp 2026-02-11T23:31:35.332207+0000/78.007614136s send_stamp 73.440895081s delta_ub -4.566718102s) ==== 2033+0+0 (crc 0 0 0) 0x4ecca80 con 0x48c4000 2026-02-11T23:31:35.331+0000 7f416cbfe640 20 osd.0 122 get_con_osd_cluster to osd.2 from_epoch 122 2026-02-11T23:31:35.331+0000 7f416cbfe640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:35.331+0000 7f416cbfe640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:35.331+0000 7f416cbfe640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:35.331+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6816/3043114013 24 ==== osd_ping(ping_reply e122 up_from 10 ping_stamp 2026-02-11T23:31:35.332207+0000/78.007614136s send_stamp 75.996002197s delta_ub -2.011605024s) ==== 2033+0+0 (crc 0 0 0) 0x4ec2180 con 0x48c4400 2026-02-11T23:31:35.331+0000 7f416cbfe640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566864491s,-4.566718102s]) 2026-02-11T23:31:35.331+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6814/3043114013 24 ==== osd_ping(ping_reply e122 up_from 10 ping_stamp 2026-02-11T23:31:35.332207+0000/78.007614136s send_stamp 75.996040344s delta_ub -2.011578560s) ==== 2033+0+0 (crc 0 0 0) 0x4ec3e00 con 0x4640c00 2026-02-11T23:31:35.331+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.1 from_epoch 122 2026-02-11T23:31:35.331+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:35.331+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:35.331+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:35.331+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011785269s,-2.011605024s]) 2026-02-11T23:31:35.331+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.1 from_epoch 122 2026-02-11T23:31:35.331+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:35.331+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:35.331+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:35.331+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011777878s,-2.011578560s]) 2026-02-11T23:31:35.331+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6830/2699377081 23 ==== osd_ping(ping_reply e122 up_from 19 ping_stamp 2026-02-11T23:31:35.332207+0000/78.007614136s send_stamp 70.788253784s delta_ub -7.219359398s) ==== 2033+0+0 (crc 0 0 0) 0x4ec3e00 con 0x48cd800 2026-02-11T23:31:35.331+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.3 from_epoch 122 2026-02-11T23:31:35.331+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6822/2365263801 23 ==== osd_ping(ping_reply e122 up_from 14 ping_stamp 2026-02-11T23:31:35.332207+0000/78.007614136s send_stamp 73.440879822s delta_ub -4.566733360s) ==== 2033+0+0 (crc 0 0 0) 0x4ec2180 con 0x48cd000 2026-02-11T23:31:35.331+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:35.331+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:35.331+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:35.331+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219592571s,-7.219359398s]) 2026-02-11T23:31:35.331+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.2 from_epoch 122 2026-02-11T23:31:35.331+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:35.331+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:35.331+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:35.331+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566985130s,-4.566733360s]) 2026-02-11T23:31:35.331+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6840/3331151400 22 ==== osd_ping(ping_reply e122 up_from 23 ping_stamp 2026-02-11T23:31:35.332207+0000/78.007614136s send_stamp 68.171112061s delta_ub -9.836505890s) ==== 2033+0+0 (crc 0 0 0) 0x4ec2180 con 0x507bc00 2026-02-11T23:31:35.331+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.4 from_epoch 122 2026-02-11T23:31:35.331+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:35.331+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:35.331+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6838/3331151400 22 ==== osd_ping(ping_reply e122 up_from 23 ping_stamp 2026-02-11T23:31:35.332207+0000/78.007614136s send_stamp 68.171142578s delta_ub -9.836465836s) ==== 2033+0+0 (crc 0 0 0) 0x4ec3e00 con 0x5282000 2026-02-11T23:31:35.331+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:35.331+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836784363s,-9.836505890s]) 2026-02-11T23:31:35.331+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.4 from_epoch 122 2026-02-11T23:31:35.331+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:35.331+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:35.331+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:35.331+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6832/2699377081 23 ==== osd_ping(ping_reply e122 up_from 19 ping_stamp 2026-02-11T23:31:35.332207+0000/78.007614136s send_stamp 70.788276672s delta_ub -7.219339848s) ==== 2033+0+0 (crc 0 0 0) 0x4ec2180 con 0x48cd400 2026-02-11T23:31:35.331+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836752892s,-9.836465836s]) 2026-02-11T23:31:35.331+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.3 from_epoch 122 2026-02-11T23:31:35.331+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:35.331+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:35.331+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:35.331+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219635487s,-7.219339848s]) 2026-02-11T23:31:35.343+0000 7f41633eb640 10 osd.0 122 tick 2026-02-11T23:31:35.343+0000 7f41633eb640 20 osd.0 122 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:35.570+0000 7f4162bea640 10 osd.0 122 tick_without_osd_lock 2026-02-11T23:31:35.570+0000 7f4162bea640 0 osd.0 122 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:35.570+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:35.570+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.203 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:35.570+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:35.571, recovery is active?:false restrictions:<....> 2026-02-11T23:31:35.570+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:35.570+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:35.570+0000 7f4162bea640 20 osd.0 122 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:35.570+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:35.570+0000 7f4162bea640 20 osd.0 122 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:36.034+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1246 ==== pg_lease(1.0 pg_lease(ru 84.698448181s ub 92.698547363s int 16.000000000s) e122/122) ==== 58+0+0 (unknown 0 0 0) 0x4ec2180 con 0x4acb000 2026-02-11T23:31:36.034+0000 7f416abfa640 15 osd.0 122 enqueue_peering_evt 1.0 epoch_sent: 122 epoch_requested: 122 MLease epoch 122 from osd.1 pg_lease(ru 84.698448181s ub 92.698547363s int 16.000000000s) 2026-02-11T23:31:36.034+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 122 epoch_requested: 122 MLease epoch 122 from osd.1 pg_lease(ru 84.698448181s ub 92.698547363s int 16.000000000s)) class_id 2 prio 255 cost 10 e122) 2026-02-11T23:31:36.034+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:36.034+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 122 epoch_requested: 122 MLease epoch 122 from osd.1 pg_lease(ru 84.698448181s ub 92.698547363s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e122) queued 2026-02-11T23:31:36.034+0000 7f414b98b640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:36.034+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 122 epoch_requested: 122 MLease epoch 122 from osd.1 pg_lease(ru 84.698448181s ub 92.698547363s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e122) pg 0x4ef0000 2026-02-11T23:31:36.034+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:36.034+0000 7f414b98b640 10 osd.0 pg_epoch: 122 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/122 sis=118) [1,0] r=1 lpr=118 crt=111'600 active mbc={}] do_peering_event: epoch_sent: 122 epoch_requested: 122 MLease epoch 122 from osd.1 pg_lease(ru 84.698448181s ub 92.698547363s int 16.000000000s) 2026-02-11T23:31:36.034+0000 7f414b98b640 10 osd.0 pg_epoch: 122 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/122 sis=118) [1,0] r=1 lpr=118 crt=111'600 active mbc={}] PeeringState::proc_lease pg_lease(ru 84.698448181s ub 92.698547363s int 16.000000000s) 2026-02-11T23:31:36.034+0000 7f414b98b640 20 osd.0 pg_epoch: 122 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/122 sis=118) [1,0] r=1 lpr=118 crt=111'600 active mbc={}] PeeringState::proc_lease peer_clock_delta_ub -2.011578560s -> ru 86.710029602s 2026-02-11T23:31:36.034+0000 7f414b98b640 20 osd.0 pg_epoch: 122 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/122 sis=118) [1,0] r=1 lpr=118 crt=111'600 active mbc={}] PeeringState::proc_lease readable_until now 86.710029602s 2026-02-11T23:31:36.034+0000 7f414b98b640 20 osd.0 pg_epoch: 122 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/122 sis=118) [1,0] r=1 lpr=118 crt=111'600 active mbc={}] PeeringState::proc_lease peer_clock_delta_lb -2.011777878s -> ruub 94.710334778s 2026-02-11T23:31:36.034+0000 7f414b98b640 20 osd.0 pg_epoch: 122 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/122 sis=118) [1,0] r=1 lpr=118 crt=111'600 active mbc={}] PeeringState::proc_lease readable_until_ub now 94.710334778s 2026-02-11T23:31:36.034+0000 7f414b98b640 20 osd.0 122 get_con_osd_cluster to osd.1 from_epoch 122 2026-02-11T23:31:36.034+0000 7f414b98b640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:36.034+0000 7f414b98b640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:36.034+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_lease_ack(1.0 pg_lease_ack(ruub 92.698547363s) e122/122) -- 0x672dce0 con 0x4acb000 2026-02-11T23:31:36.034+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:36.034+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:36.330+0000 7f41633eb640 10 osd.0 122 tick 2026-02-11T23:31:36.330+0000 7f41633eb640 20 osd.0 122 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:36.330+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:36.330+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:36.330+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:06.331158+0000) 2026-02-11T23:31:36.350+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.3 127.0.0.1:0/2699377081 20 ==== osd_ping(ping e122 up_from 19 ping_stamp 2026-02-11T23:31:36.350634+0000/71.806571960s send_stamp 71.806571960s delta_ub -7.219339848s) ==== 2033+0+0 (crc 0 0 0) 0x4ec3e00 con 0x48cc400 2026-02-11T23:31:36.350+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219533443s,-7.219339848s]) 2026-02-11T23:31:36.350+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:36.350634+0000/71.806571960s send_stamp 79.026107788s delta_ub 7.219533443s) -- 0x4ecd980 con 0x48cc400 2026-02-11T23:31:36.350+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.3 from_epoch 122 2026-02-11T23:31:36.350+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:36.350+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:36.350+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.3 127.0.0.1:0/2699377081 20 ==== osd_ping(ping e122 up_from 19 ping_stamp 2026-02-11T23:31:36.350634+0000/71.806571960s send_stamp 71.806571960s delta_ub -7.219339848s) ==== 2033+0+0 (crc 0 0 0) 0x4ecca80 con 0x48cc800 2026-02-11T23:31:36.350+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:36.350+0000 7f416cbfe640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219547749s,-7.219339848s]) 2026-02-11T23:31:36.350+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:36.350634+0000/71.806571960s send_stamp 79.026115417s delta_ub 7.219547749s) -- 0x4c9a480 con 0x48cc800 2026-02-11T23:31:36.350+0000 7f416cbfe640 20 osd.0 122 get_con_osd_cluster to osd.3 from_epoch 122 2026-02-11T23:31:36.350+0000 7f416cbfe640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:36.350+0000 7f416cbfe640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:36.350+0000 7f416cbfe640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:36.438+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1249 ==== pg_lease(4.0 pg_lease(ru 77.277549744s ub 85.277610779s int 16.000000000s) e122/122) ==== 58+0+0 (unknown 0 0 0) 0x4ec2180 con 0x507ac00 2026-02-11T23:31:36.438+0000 7f416abfa640 15 osd.0 122 enqueue_peering_evt 4.0 epoch_sent: 122 epoch_requested: 122 MLease epoch 122 from osd.4 pg_lease(ru 77.277549744s ub 85.277610779s int 16.000000000s) 2026-02-11T23:31:36.438+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 122 epoch_requested: 122 MLease epoch 122 from osd.4 pg_lease(ru 77.277549744s ub 85.277610779s int 16.000000000s)) class_id 2 prio 255 cost 10 e122) 2026-02-11T23:31:36.438+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:36.438+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 122 epoch_requested: 122 MLease epoch 122 from osd.4 pg_lease(ru 77.277549744s ub 85.277610779s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e122) queued 2026-02-11T23:31:36.438+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:36.438+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 122 epoch_requested: 122 MLease epoch 122 from osd.4 pg_lease(ru 77.277549744s ub 85.277610779s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e122) pg 0x5978000 2026-02-11T23:31:36.438+0000 7f414f993640 10 osd.0 pg_epoch: 122 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/120 les/c/f=121/121/122 sis=120) [4,0] r=1 lpr=120 crt=111'600 active mbc={}] do_peering_event: epoch_sent: 122 epoch_requested: 122 MLease epoch 122 from osd.4 pg_lease(ru 77.277549744s ub 85.277610779s int 16.000000000s) 2026-02-11T23:31:36.438+0000 7f414f993640 10 osd.0 pg_epoch: 122 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/120 les/c/f=121/121/122 sis=120) [4,0] r=1 lpr=120 crt=111'600 active mbc={}] PeeringState::proc_lease pg_lease(ru 77.277549744s ub 85.277610779s int 16.000000000s) 2026-02-11T23:31:36.438+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:36.438+0000 7f414f993640 20 osd.0 pg_epoch: 122 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/120 les/c/f=121/121/122 sis=120) [4,0] r=1 lpr=120 crt=111'600 active mbc={}] PeeringState::proc_lease peer_clock_delta_ub -9.836465836s -> ru 87.114021301s 2026-02-11T23:31:36.438+0000 7f414f993640 20 osd.0 pg_epoch: 122 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/120 les/c/f=121/121/122 sis=120) [4,0] r=1 lpr=120 crt=111'600 active mbc={}] PeeringState::proc_lease readable_until now 87.114021301s 2026-02-11T23:31:36.438+0000 7f414f993640 20 osd.0 pg_epoch: 122 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/120 les/c/f=121/121/122 sis=120) [4,0] r=1 lpr=120 crt=111'600 active mbc={}] PeeringState::proc_lease peer_clock_delta_lb -9.836752892s -> ruub 95.114372253s 2026-02-11T23:31:36.438+0000 7f414f993640 20 osd.0 pg_epoch: 122 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/120 les/c/f=121/121/122 sis=120) [4,0] r=1 lpr=120 crt=111'600 active mbc={}] PeeringState::proc_lease readable_until_ub now 95.114372253s 2026-02-11T23:31:36.438+0000 7f414f993640 20 osd.0 122 get_con_osd_cluster to osd.4 from_epoch 122 2026-02-11T23:31:36.438+0000 7f414f993640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:36.438+0000 7f414f993640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:36.438+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_lease_ack(4.0 pg_lease_ack(ruub 85.277610779s) e122/122) -- 0x68aa000 con 0x507ac00 2026-02-11T23:31:36.438+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:36.438+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:36.567+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.4 127.0.0.1:0/3331151400 20 ==== osd_ping(ping e122 up_from 23 ping_stamp 2026-02-11T23:31:36.567597+0000/69.406364441s send_stamp 69.406364441s delta_ub -9.836465836s) ==== 2033+0+0 (crc 0 0 0) 0x4ecd980 con 0x507a800 2026-02-11T23:31:36.567+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836680412s,-9.836465836s]) 2026-02-11T23:31:36.567+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:36.567597+0000/69.406364441s send_stamp 79.243041992s delta_ub 9.836680412s) -- 0x4ec3e00 con 0x507a800 2026-02-11T23:31:36.567+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.4 from_epoch 122 2026-02-11T23:31:36.567+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:36.567+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:36.567+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:36.567+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.4 127.0.0.1:0/3331151400 20 ==== osd_ping(ping e122 up_from 23 ping_stamp 2026-02-11T23:31:36.567597+0000/69.406364441s send_stamp 69.406364441s delta_ub -9.836465836s) ==== 2033+0+0 (crc 0 0 0) 0x4ec2180 con 0x507a400 2026-02-11T23:31:36.567+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836708069s,-9.836465836s]) 2026-02-11T23:31:36.567+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:36.567597+0000/69.406364441s send_stamp 79.243072510s delta_ub 9.836708069s) -- 0x4ecc180 con 0x507a400 2026-02-11T23:31:36.567+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.4 from_epoch 122 2026-02-11T23:31:36.567+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:36.567+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:36.567+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:36.592+0000 7f4162bea640 10 osd.0 122 tick_without_osd_lock 2026-02-11T23:31:36.592+0000 7f4162bea640 0 osd.0 122 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:36.592+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:36.592+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.203 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:36.592+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:36.592, recovery is active?:false restrictions:<....> 2026-02-11T23:31:36.592+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:36.592+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:36.592+0000 7f4162bea640 20 osd.0 122 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:36.592+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:36.592+0000 7f4162bea640 20 osd.0 122 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:37.304+0000 7f41633eb640 10 osd.0 122 tick 2026-02-11T23:31:37.304+0000 7f41633eb640 20 osd.0 122 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:37.330+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:37.330+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:37.330+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:07.331190+0000) 2026-02-11T23:31:37.556+0000 7f4162bea640 10 osd.0 122 tick_without_osd_lock 2026-02-11T23:31:37.556+0000 7f4162bea640 0 osd.0 122 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:37.556+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:37.556+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:37.556+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:37.556+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.203 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:37.556+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:37.557, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:37.556+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:37.556+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:37.556+0000 7f4162bea640 20 osd.0 122 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:37.556+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:37.556+0000 7f4162bea640 20 osd.0 122 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:38.208+0000 7f415b6ff640 20 osd.0 122 reports for 0 queries 2026-02-11T23:31:38.208+0000 7f415b6ff640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] -- mgrreport(unknown.0 +0-0 packed 3294 daemon_metrics=2) -- 0x68ac000 con 0x4aca800 2026-02-11T23:31:38.208+0000 7f415b6ff640 15 osd.0 122 collect_pg_stats 2026-02-11T23:31:38.208+0000 7f415b6ff640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] -- pg_stats(0 pgs seq 21474836501 v 0) -- 0x68ae000 con 0x4aca800 2026-02-11T23:31:38.331+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:38.331+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:38.331+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:08.331216+0000) 2026-02-11T23:31:38.332+0000 7f41633eb640 10 osd.0 122 tick 2026-02-11T23:31:38.332+0000 7f41633eb640 20 osd.0 122 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:38.599+0000 7f4162bea640 10 osd.0 122 tick_without_osd_lock 2026-02-11T23:31:38.599+0000 7f4162bea640 0 osd.0 122 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:38.599+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:38.599+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:38.599+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:38.599+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.187 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:38.599+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:38.600, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:38.599+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:38.599+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:38.599+0000 7f4162bea640 20 osd.0 122 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:38.599+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:38.599+0000 7f4162bea640 20 osd.0 122 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:39.295+0000 7f41633eb640 10 osd.0 122 tick 2026-02-11T23:31:39.295+0000 7f41633eb640 10 osd.0 122 maybe_update_heartbeat_peers forcing update after 20.087199 seconds 2026-02-11T23:31:39.295+0000 7f41633eb640 10 osd.0 122 reset_heartbeat_peers 2026-02-11T23:31:39.295+0000 7f41633eb640 10 osd.0 122 maybe_update_heartbeat_peers updating 2026-02-11T23:31:39.295+0000 7f41633eb640 10 osd.0 122 adding neighbor peer osd.1 2026-02-11T23:31:39.295+0000 7f41633eb640 10 osd.0 122 adding neighbor peer osd.4 2026-02-11T23:31:39.295+0000 7f41633eb640 10 osd.0 122 maybe_update_heartbeat_peers 4 peers, extras 1,2,3,4 2026-02-11T23:31:39.295+0000 7f41633eb640 20 osd.0 122 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:39.331+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:39.331+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:39.331+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:09.331247+0000) 2026-02-11T23:31:39.432+0000 7f4147983640 5 osd.0 122 heartbeat osd_stat(store_statfs(0x0/0x0/0x384000, data 0x53a3e5/0x5e6000, compress 0x0/0x0/0x0, omap 0xa403, meta 0x1a45bfd), peers [1,2,3,4] op hist []) 2026-02-11T23:31:39.432+0000 7f4147983640 20 osd.0 122 check_full_status cur ratio 1, physical ratio 1, new state failsafe 2026-02-11T23:31:39.432+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6816/3043114013,v1:127.0.0.1:6817/3043114013] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:39.432382+0000/82.107788086s send_stamp 82.107788086s delta_ub 2.011777878s) -- 0x4ec3b00 con 0x48c4400 2026-02-11T23:31:39.432+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6814/3043114013,v1:127.0.0.1:6815/3043114013] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:39.432382+0000/82.107788086s send_stamp 82.107788086s delta_ub 2.011777878s) -- 0x4ec2300 con 0x4640c00 2026-02-11T23:31:39.432+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6824/2365263801,v1:127.0.0.1:6825/2365263801] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:39.432382+0000/82.107788086s send_stamp 82.107788086s delta_ub 4.566985130s) -- 0x4ec3800 con 0x48c4000 2026-02-11T23:31:39.432+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6822/2365263801,v1:127.0.0.1:6823/2365263801] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:39.432382+0000/82.107788086s send_stamp 82.107788086s delta_ub 4.566985130s) -- 0x4ec3500 con 0x48cd000 2026-02-11T23:31:39.432+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6832/2699377081,v1:127.0.0.1:6833/2699377081] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:39.432382+0000/82.107788086s send_stamp 82.107788086s delta_ub 7.219547749s) -- 0x4ec3200 con 0x48cd400 2026-02-11T23:31:39.432+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6830/2699377081,v1:127.0.0.1:6831/2699377081] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:39.432382+0000/82.107788086s send_stamp 82.107788086s delta_ub 7.219547749s) -- 0x4ec2c00 con 0x48cd800 2026-02-11T23:31:39.432+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6840/3331151400,v1:127.0.0.1:6841/3331151400] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:39.432382+0000/82.107788086s send_stamp 82.107788086s delta_ub 9.836708069s) -- 0x4ec2900 con 0x507bc00 2026-02-11T23:31:39.432+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6838/3331151400,v1:127.0.0.1:6839/3331151400] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:39.432382+0000/82.107788086s send_stamp 82.107788086s delta_ub 9.836708069s) -- 0x4a89200 con 0x5282000 2026-02-11T23:31:39.432+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6816/3043114013 25 ==== osd_ping(ping_reply e122 up_from 10 ping_stamp 2026-02-11T23:31:39.432382+0000/82.107788086s send_stamp 80.096183777s delta_ub -2.011605024s) ==== 2033+0+0 (crc 0 0 0) 0x4ec2900 con 0x48c4400 2026-02-11T23:31:39.432+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.1 from_epoch 122 2026-02-11T23:31:39.432+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:39.432+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:39.432+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:39.432+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011715651s,-2.011605024s]) 2026-02-11T23:31:39.432+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6822/2365263801 24 ==== osd_ping(ping_reply e122 up_from 14 ping_stamp 2026-02-11T23:31:39.432382+0000/82.107788086s send_stamp 77.541053772s delta_ub -4.566734791s) ==== 2033+0+0 (crc 0 0 0) 0x4ec2900 con 0x48cd000 2026-02-11T23:31:39.432+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.2 from_epoch 122 2026-02-11T23:31:39.432+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:39.432+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:39.432+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:39.432+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566864967s,-4.566734791s]) 2026-02-11T23:31:39.432+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6814/3043114013 25 ==== osd_ping(ping_reply e122 up_from 10 ping_stamp 2026-02-11T23:31:39.432382+0000/82.107788086s send_stamp 80.096206665s delta_ub -2.011584282s) ==== 2033+0+0 (crc 0 0 0) 0x4a89200 con 0x4640c00 2026-02-11T23:31:39.432+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6832/2699377081 24 ==== osd_ping(ping_reply e122 up_from 19 ping_stamp 2026-02-11T23:31:39.432382+0000/82.107788086s send_stamp 74.888404846s delta_ub -7.219379902s) ==== 2033+0+0 (crc 0 0 0) 0x4ec2900 con 0x48cd400 2026-02-11T23:31:39.432+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.1 from_epoch 122 2026-02-11T23:31:39.432+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:39.432+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:39.432+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:39.432+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011725426s,-2.011584282s]) 2026-02-11T23:31:39.432+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.3 from_epoch 122 2026-02-11T23:31:39.432+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:39.432+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:39.432+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:39.432+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219533920s,-7.219379902s]) 2026-02-11T23:31:39.432+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6830/2699377081 24 ==== osd_ping(ping_reply e122 up_from 19 ping_stamp 2026-02-11T23:31:39.432382+0000/82.107788086s send_stamp 74.888427734s delta_ub -7.219357014s) ==== 2033+0+0 (crc 0 0 0) 0x4a89200 con 0x48cd800 2026-02-11T23:31:39.432+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.3 from_epoch 122 2026-02-11T23:31:39.432+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:39.432+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:39.432+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:39.432+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219520569s,-7.219357014s]) 2026-02-11T23:31:39.432+0000 7f416cbfe640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6824/2365263801 24 ==== osd_ping(ping_reply e122 up_from 14 ping_stamp 2026-02-11T23:31:39.432382+0000/82.107788086s send_stamp 77.541076660s delta_ub -4.566714287s) ==== 2033+0+0 (crc 0 0 0) 0x4ec3800 con 0x48c4000 2026-02-11T23:31:39.432+0000 7f416cbfe640 20 osd.0 122 get_con_osd_cluster to osd.2 from_epoch 122 2026-02-11T23:31:39.432+0000 7f416cbfe640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:39.432+0000 7f416cbfe640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:39.432+0000 7f416cbfe640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:39.432+0000 7f416cbfe640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566923141s,-4.566714287s]) 2026-02-11T23:31:39.432+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6840/3331151400 23 ==== osd_ping(ping_reply e122 up_from 23 ping_stamp 2026-02-11T23:31:39.432382+0000/82.107788086s send_stamp 72.271255493s delta_ub -9.836527824s) ==== 2033+0+0 (crc 0 0 0) 0x4ec2900 con 0x507bc00 2026-02-11T23:31:39.432+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.4 from_epoch 122 2026-02-11T23:31:39.432+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6838/3331151400 23 ==== osd_ping(ping_reply e122 up_from 23 ping_stamp 2026-02-11T23:31:39.432382+0000/82.107788086s send_stamp 72.271331787s delta_ub -9.836458206s) ==== 2033+0+0 (crc 0 0 0) 0x4a89200 con 0x5282000 2026-02-11T23:31:39.432+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:39.432+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:39.432+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:39.432+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836750031s,-9.836527824s]) 2026-02-11T23:31:39.432+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.4 from_epoch 122 2026-02-11T23:31:39.432+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:39.432+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:39.432+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:39.432+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836688995s,-9.836458206s]) 2026-02-11T23:31:39.467+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.4 127.0.0.1:0/3331151400 21 ==== osd_ping(ping e122 up_from 23 ping_stamp 2026-02-11T23:31:39.467794+0000/72.306564331s send_stamp 72.306564331s delta_ub -9.836458206s) ==== 2033+0+0 (crc 0 0 0) 0x4a89200 con 0x507a800 2026-02-11T23:31:39.467+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836686134s,-9.836458206s]) 2026-02-11T23:31:39.467+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:39.467794+0000/72.306564331s send_stamp 82.143249512s delta_ub 9.836686134s) -- 0x4ec2c00 con 0x507a800 2026-02-11T23:31:39.467+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.4 from_epoch 122 2026-02-11T23:31:39.467+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:39.467+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:39.467+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:39.467+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.4 127.0.0.1:0/3331151400 21 ==== osd_ping(ping e122 up_from 23 ping_stamp 2026-02-11T23:31:39.467794+0000/72.306564331s send_stamp 72.306564331s delta_ub -9.836458206s) ==== 2033+0+0 (crc 0 0 0) 0x4ec2900 con 0x507a400 2026-02-11T23:31:39.467+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836706161s,-9.836458206s]) 2026-02-11T23:31:39.467+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:39.467794+0000/72.306564331s send_stamp 82.143264771s delta_ub 9.836706161s) -- 0x4ec3200 con 0x507a400 2026-02-11T23:31:39.467+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.4 from_epoch 122 2026-02-11T23:31:39.467+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:39.467+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:39.467+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:39.564+0000 7f4162bea640 10 osd.0 122 tick_without_osd_lock 2026-02-11T23:31:39.564+0000 7f4162bea640 0 osd.0 122 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:39.564+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:39.564+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:39.564+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:39.564+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.187 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:39.564+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:39.564, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:39.564+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:39.564+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:39.564+0000 7f4162bea640 20 osd.0 122 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:39.564+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:39.564+0000 7f4162bea640 20 osd.0 122 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:39.643+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.1 127.0.0.1:0/3043114013 25 ==== osd_ping(ping e122 up_from 10 ping_stamp 2026-02-11T23:31:39.643424+0000/80.307167053s send_stamp 80.307167053s delta_ub -2.011584282s) ==== 2033+0+0 (crc 0 0 0) 0x4ec2c00 con 0x4aca400 2026-02-11T23:31:39.643+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011713743s,-2.011584282s]) 2026-02-11T23:31:39.643+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:39.643424+0000/80.307167053s send_stamp 82.318885803s delta_ub 2.011713743s) -- 0x4a89200 con 0x4aca400 2026-02-11T23:31:39.643+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.1 from_epoch 122 2026-02-11T23:31:39.643+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:39.643+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:39.643+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.1 127.0.0.1:0/3043114013 25 ==== osd_ping(ping e122 up_from 10 ping_stamp 2026-02-11T23:31:39.643424+0000/80.307167053s send_stamp 80.307167053s delta_ub -2.011584282s) ==== 2033+0+0 (crc 0 0 0) 0x4ec3800 con 0x4aca000 2026-02-11T23:31:39.643+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:39.643+0000 7f416cbfe640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011729002s,-2.011584282s]) 2026-02-11T23:31:39.643+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:39.643424+0000/80.307167053s send_stamp 82.318901062s delta_ub 2.011729002s) -- 0x4c9be00 con 0x4aca000 2026-02-11T23:31:39.643+0000 7f416cbfe640 20 osd.0 122 get_con_osd_cluster to osd.1 from_epoch 122 2026-02-11T23:31:39.643+0000 7f416cbfe640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:39.643+0000 7f416cbfe640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:39.643+0000 7f416cbfe640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:39.998+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.2 127.0.0.1:0/2365263801 23 ==== osd_ping(ping e122 up_from 14 ping_stamp 2026-02-11T23:31:39.998436+0000/78.107032776s send_stamp 78.107032776s delta_ub -4.566714287s) ==== 2033+0+0 (crc 0 0 0) 0x4a89200 con 0x48cc000 2026-02-11T23:31:39.998+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566854000s,-4.566714287s]) 2026-02-11T23:31:39.998+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.2 127.0.0.1:0/2365263801 23 ==== osd_ping(ping e122 up_from 14 ping_stamp 2026-02-11T23:31:39.998436+0000/78.107032776s send_stamp 78.107032776s delta_ub -4.566714287s) ==== 2033+0+0 (crc 0 0 0) 0x4c9be00 con 0x4641000 2026-02-11T23:31:39.998+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:39.998436+0000/78.107032776s send_stamp 82.673881531s delta_ub 4.566854000s) -- 0x4ec2c00 con 0x48cc000 2026-02-11T23:31:39.998+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.2 from_epoch 122 2026-02-11T23:31:39.998+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:39.998+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:39.998+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:39.998+0000 7f416cbfe640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566867352s,-4.566714287s]) 2026-02-11T23:31:39.998+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:39.998436+0000/78.107032776s send_stamp 82.673904419s delta_ub 4.566867352s) -- 0x4ec3800 con 0x4641000 2026-02-11T23:31:39.998+0000 7f416cbfe640 20 osd.0 122 get_con_osd_cluster to osd.2 from_epoch 122 2026-02-11T23:31:39.998+0000 7f416cbfe640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:39.998+0000 7f416cbfe640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:39.998+0000 7f416cbfe640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:40.331+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:40.331+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:40.331+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:10.331282+0000) 2026-02-11T23:31:40.331+0000 7f415bf00640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-11T23:31:40.331+0000 7f41633eb640 10 osd.0 122 tick 2026-02-11T23:31:40.331+0000 7f41633eb640 20 osd.0 122 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:40.549+0000 7f4162bea640 10 osd.0 122 tick_without_osd_lock 2026-02-11T23:31:40.549+0000 7f4162bea640 0 osd.0 122 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:40.549+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:40.549+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:40.549+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:40.549+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.187 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:40.549+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:40.549, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:40.549+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:40.549+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:40.549+0000 7f4162bea640 20 osd.0 122 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:40.549+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:40.549+0000 7f4162bea640 20 osd.0 122 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:41.050+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.3 127.0.0.1:0/2699377081 21 ==== osd_ping(ping e122 up_from 19 ping_stamp 2026-02-11T23:31:41.050869+0000/76.506805420s send_stamp 76.506805420s delta_ub -7.219357014s) ==== 2033+0+0 (crc 0 0 0) 0x4ec2c00 con 0x48cc400 2026-02-11T23:31:41.050+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219510078s,-7.219357014s]) 2026-02-11T23:31:41.050+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:41.050869+0000/76.506805420s send_stamp 83.726318359s delta_ub 7.219510078s) -- 0x4a89200 con 0x48cc400 2026-02-11T23:31:41.050+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.3 from_epoch 122 2026-02-11T23:31:41.050+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:41.050+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:41.050+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:41.050+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.3 127.0.0.1:0/2699377081 21 ==== osd_ping(ping e122 up_from 19 ping_stamp 2026-02-11T23:31:41.050869+0000/76.506805420s send_stamp 76.506805420s delta_ub -7.219357014s) ==== 2033+0+0 (crc 0 0 0) 0x4ec3800 con 0x48cc800 2026-02-11T23:31:41.050+0000 7f416cbfe640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219527245s,-7.219357014s]) 2026-02-11T23:31:41.050+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:41.050869+0000/76.506805420s send_stamp 83.726333618s delta_ub 7.219527245s) -- 0x4c9be00 con 0x48cc800 2026-02-11T23:31:41.050+0000 7f416cbfe640 20 osd.0 122 get_con_osd_cluster to osd.3 from_epoch 122 2026-02-11T23:31:41.050+0000 7f416cbfe640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:41.050+0000 7f416cbfe640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:41.050+0000 7f416cbfe640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:41.331+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:41.331+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:41.331+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:11.331317+0000) 2026-02-11T23:31:41.360+0000 7f41633eb640 10 osd.0 122 tick 2026-02-11T23:31:41.360+0000 7f41633eb640 20 osd.0 122 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:41.587+0000 7f4162bea640 10 osd.0 122 tick_without_osd_lock 2026-02-11T23:31:41.587+0000 7f4162bea640 0 osd.0 122 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:41.587+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:41.587+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:41.587+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:41.587+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.187 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:41.587+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:41.587, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:41.587+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:41.587+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:41.587+0000 7f4162bea640 20 osd.0 122 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:41.587+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:41.587+0000 7f4162bea640 20 osd.0 122 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:42.331+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:42.331+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:42.331+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:12.331349+0000) 2026-02-11T23:31:42.365+0000 7f41633eb640 10 osd.0 122 tick 2026-02-11T23:31:42.365+0000 7f41633eb640 20 osd.0 122 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:42.367+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.4 127.0.0.1:0/3331151400 22 ==== osd_ping(ping e122 up_from 23 ping_stamp 2026-02-11T23:31:42.368030+0000/75.206794739s send_stamp 75.206794739s delta_ub -9.836541176s) ==== 2033+0+0 (crc 0 0 0) 0x4a89200 con 0x507a800 2026-02-11T23:31:42.367+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836697578s,-9.836541176s]) 2026-02-11T23:31:42.367+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:42.368030+0000/75.206794739s send_stamp 85.043495178s delta_ub 9.836697578s) -- 0x4ec2c00 con 0x507a800 2026-02-11T23:31:42.367+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.4 from_epoch 122 2026-02-11T23:31:42.367+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:42.367+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.4 127.0.0.1:0/3331151400 22 ==== osd_ping(ping e122 up_from 23 ping_stamp 2026-02-11T23:31:42.368030+0000/75.206794739s send_stamp 75.206794739s delta_ub -9.836541176s) ==== 2033+0+0 (crc 0 0 0) 0x4ec3200 con 0x507a400 2026-02-11T23:31:42.367+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:42.367+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:42.367+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836711884s,-9.836541176s]) 2026-02-11T23:31:42.367+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:42.368030+0000/75.206794739s send_stamp 85.043510437s delta_ub 9.836711884s) -- 0x4ec2900 con 0x507a400 2026-02-11T23:31:42.367+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.4 from_epoch 122 2026-02-11T23:31:42.367+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:42.367+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:42.367+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:42.618+0000 7f4162bea640 10 osd.0 122 tick_without_osd_lock 2026-02-11T23:31:42.618+0000 7f4162bea640 0 osd.0 122 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:42.618+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:42.618+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.172 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:42.618+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:42.618, recovery is active?:false restrictions:<....> 2026-02-11T23:31:42.618+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:42.618+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:42.618+0000 7f4162bea640 20 osd.0 122 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:42.618+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:42.618+0000 7f4162bea640 20 osd.0 122 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:43.208+0000 7f415b6ff640 20 osd.0 122 reports for 0 queries 2026-02-11T23:31:43.208+0000 7f415b6ff640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] -- mgrreport(unknown.0 +0-0 packed 3294 daemon_metrics=2) -- 0x68ac380 con 0x4aca800 2026-02-11T23:31:43.208+0000 7f415b6ff640 15 osd.0 122 collect_pg_stats 2026-02-11T23:31:43.208+0000 7f415b6ff640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] -- pg_stats(0 pgs seq 21474836502 v 0) -- 0x68ae300 con 0x4aca800 2026-02-11T23:31:43.331+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:43.331+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:43.331+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:13.331392+0000) 2026-02-11T23:31:43.356+0000 7f41633eb640 10 osd.0 122 tick 2026-02-11T23:31:43.356+0000 7f41633eb640 20 osd.0 122 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:43.653+0000 7f4162bea640 10 osd.0 122 tick_without_osd_lock 2026-02-11T23:31:43.653+0000 7f4162bea640 0 osd.0 122 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:43.653+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:43.653+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:43.653+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:43.653+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.172 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:43.653+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:43.654, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:43.653+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:43.653+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:43.653+0000 7f4162bea640 20 osd.0 122 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:43.653+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:43.653+0000 7f4162bea640 20 osd.0 122 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:43.949+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.3 127.0.0.1:0/2699377081 22 ==== osd_ping(ping e122 up_from 19 ping_stamp 2026-02-11T23:31:43.951014+0000/79.406959534s send_stamp 79.406959534s delta_ub -7.219409466s) ==== 2033+0+0 (crc 0 0 0) 0x4ec2c00 con 0x48cc400 2026-02-11T23:31:43.949+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219520569s,-7.219409466s]) 2026-02-11T23:31:43.949+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:43.951014+0000/79.406959534s send_stamp 86.626472473s delta_ub 7.219520569s) -- 0x4a89200 con 0x48cc400 2026-02-11T23:31:43.949+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.3 from_epoch 122 2026-02-11T23:31:43.949+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:43.949+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:43.949+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:43.949+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.3 127.0.0.1:0/2699377081 22 ==== osd_ping(ping e122 up_from 19 ping_stamp 2026-02-11T23:31:43.951014+0000/79.406959534s send_stamp 79.406959534s delta_ub -7.219409466s) ==== 2033+0+0 (crc 0 0 0) 0x4c9be00 con 0x48cc800 2026-02-11T23:31:43.949+0000 7f416cbfe640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219534874s,-7.219409466s]) 2026-02-11T23:31:43.949+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:43.951014+0000/79.406959534s send_stamp 86.626495361s delta_ub 7.219534874s) -- 0x4ec3800 con 0x48cc800 2026-02-11T23:31:43.949+0000 7f416cbfe640 20 osd.0 122 get_con_osd_cluster to osd.3 from_epoch 122 2026-02-11T23:31:43.949+0000 7f416cbfe640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:43.949+0000 7f416cbfe640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:43.949+0000 7f416cbfe640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:44.033+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1247 ==== pg_lease(1.0 pg_lease(ru 92.698547363s ub 100.698654175s int 16.000000000s) e122/122) ==== 58+0+0 (unknown 0 0 0) 0x4ec2900 con 0x4acb000 2026-02-11T23:31:44.033+0000 7f416abfa640 15 osd.0 122 enqueue_peering_evt 1.0 epoch_sent: 122 epoch_requested: 122 MLease epoch 122 from osd.1 pg_lease(ru 92.698547363s ub 100.698654175s int 16.000000000s) 2026-02-11T23:31:44.033+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 122 epoch_requested: 122 MLease epoch 122 from osd.1 pg_lease(ru 92.698547363s ub 100.698654175s int 16.000000000s)) class_id 2 prio 255 cost 10 e122) 2026-02-11T23:31:44.033+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:44.033+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 122 epoch_requested: 122 MLease epoch 122 from osd.1 pg_lease(ru 92.698547363s ub 100.698654175s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e122) queued 2026-02-11T23:31:44.033+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:44.033+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 122 epoch_requested: 122 MLease epoch 122 from osd.1 pg_lease(ru 92.698547363s ub 100.698654175s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e122) pg 0x4ef0000 2026-02-11T23:31:44.033+0000 7f414f993640 10 osd.0 pg_epoch: 122 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/122 sis=118) [1,0] r=1 lpr=118 crt=111'600 active mbc={}] do_peering_event: epoch_sent: 122 epoch_requested: 122 MLease epoch 122 from osd.1 pg_lease(ru 92.698547363s ub 100.698654175s int 16.000000000s) 2026-02-11T23:31:44.033+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:44.033+0000 7f414f993640 10 osd.0 pg_epoch: 122 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/122 sis=118) [1,0] r=1 lpr=118 crt=111'600 active mbc={}] PeeringState::proc_lease pg_lease(ru 92.698547363s ub 100.698654175s int 16.000000000s) 2026-02-11T23:31:44.033+0000 7f414f993640 20 osd.0 pg_epoch: 122 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/122 sis=118) [1,0] r=1 lpr=118 crt=111'600 active mbc={}] PeeringState::proc_lease peer_clock_delta_ub -2.011584282s -> ru 94.710136414s 2026-02-11T23:31:44.033+0000 7f414f993640 20 osd.0 pg_epoch: 122 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/122 sis=118) [1,0] r=1 lpr=118 crt=111'600 active mbc={}] PeeringState::proc_lease readable_until now 94.710136414s 2026-02-11T23:31:44.033+0000 7f414f993640 20 osd.0 pg_epoch: 122 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/122 sis=118) [1,0] r=1 lpr=118 crt=111'600 active mbc={}] PeeringState::proc_lease peer_clock_delta_lb -2.011729002s -> ruub 102.710388184s 2026-02-11T23:31:44.033+0000 7f414f993640 20 osd.0 pg_epoch: 122 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/122 sis=118) [1,0] r=1 lpr=118 crt=111'600 active mbc={}] PeeringState::proc_lease readable_until_ub now 102.710388184s 2026-02-11T23:31:44.033+0000 7f414f993640 20 osd.0 122 get_con_osd_cluster to osd.1 from_epoch 122 2026-02-11T23:31:44.033+0000 7f414f993640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:44.033+0000 7f414f993640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:44.033+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_lease_ack(1.0 pg_lease_ack(ruub 100.698654175s) e122/122) -- 0x68aa160 con 0x4acb000 2026-02-11T23:31:44.033+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:44.033+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:44.330+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:44.330+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:44.330+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:14.331427+0000) 2026-02-11T23:31:44.395+0000 7f41633eb640 10 osd.0 122 tick 2026-02-11T23:31:44.395+0000 7f41633eb640 20 osd.0 122 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:44.437+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1250 ==== pg_lease(4.0 pg_lease(ru 85.277610779s ub 93.277740479s int 16.000000000s) e122/122) ==== 58+0+0 (unknown 0 0 0) 0x4ec2900 con 0x507ac00 2026-02-11T23:31:44.437+0000 7f416abfa640 15 osd.0 122 enqueue_peering_evt 4.0 epoch_sent: 122 epoch_requested: 122 MLease epoch 122 from osd.4 pg_lease(ru 85.277610779s ub 93.277740479s int 16.000000000s) 2026-02-11T23:31:44.437+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 122 epoch_requested: 122 MLease epoch 122 from osd.4 pg_lease(ru 85.277610779s ub 93.277740479s int 16.000000000s)) class_id 2 prio 255 cost 10 e122) 2026-02-11T23:31:44.437+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:44.437+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 122 epoch_requested: 122 MLease epoch 122 from osd.4 pg_lease(ru 85.277610779s ub 93.277740479s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e122) queued 2026-02-11T23:31:44.437+0000 7f414f993640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:44.437+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 122 epoch_requested: 122 MLease epoch 122 from osd.4 pg_lease(ru 85.277610779s ub 93.277740479s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e122) pg 0x5978000 2026-02-11T23:31:44.437+0000 7f414f993640 10 osd.0 pg_epoch: 122 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/120 les/c/f=121/121/122 sis=120) [4,0] r=1 lpr=120 crt=111'600 active mbc={}] do_peering_event: epoch_sent: 122 epoch_requested: 122 MLease epoch 122 from osd.4 pg_lease(ru 85.277610779s ub 93.277740479s int 16.000000000s) 2026-02-11T23:31:44.437+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:44.437+0000 7f414f993640 10 osd.0 pg_epoch: 122 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/120 les/c/f=121/121/122 sis=120) [4,0] r=1 lpr=120 crt=111'600 active mbc={}] PeeringState::proc_lease pg_lease(ru 85.277610779s ub 93.277740479s int 16.000000000s) 2026-02-11T23:31:44.437+0000 7f414f993640 20 osd.0 pg_epoch: 122 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/120 les/c/f=121/121/122 sis=120) [4,0] r=1 lpr=120 crt=111'600 active mbc={}] PeeringState::proc_lease peer_clock_delta_ub -9.836541176s -> ru 95.114158630s 2026-02-11T23:31:44.437+0000 7f414f993640 20 osd.0 pg_epoch: 122 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/120 les/c/f=121/121/122 sis=120) [4,0] r=1 lpr=120 crt=111'600 active mbc={}] PeeringState::proc_lease readable_until now 95.114158630s 2026-02-11T23:31:44.437+0000 7f414f993640 20 osd.0 pg_epoch: 122 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/120 les/c/f=121/121/122 sis=120) [4,0] r=1 lpr=120 crt=111'600 active mbc={}] PeeringState::proc_lease peer_clock_delta_lb -9.836711884s -> ruub 103.114448547s 2026-02-11T23:31:44.437+0000 7f414f993640 20 osd.0 pg_epoch: 122 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/120 les/c/f=121/121/122 sis=120) [4,0] r=1 lpr=120 crt=111'600 active mbc={}] PeeringState::proc_lease readable_until_ub now 103.114448547s 2026-02-11T23:31:44.437+0000 7f414f993640 20 osd.0 122 get_con_osd_cluster to osd.4 from_epoch 122 2026-02-11T23:31:44.437+0000 7f414f993640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:44.437+0000 7f414f993640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:44.437+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_lease_ack(4.0 pg_lease_ack(ruub 93.277740479s) e122/122) -- 0x68aa2c0 con 0x507ac00 2026-02-11T23:31:44.437+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:44.437+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:44.610+0000 7f4162bea640 10 osd.0 122 tick_without_osd_lock 2026-02-11T23:31:44.610+0000 7f4162bea640 0 osd.0 122 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:44.610+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:44.610+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.172 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:44.610+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:44.611, recovery is active?:false restrictions:<....> 2026-02-11T23:31:44.610+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:44.610+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:44.610+0000 7f4162bea640 20 osd.0 122 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:44.610+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:44.610+0000 7f4162bea640 20 osd.0 122 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:44.667+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.4 127.0.0.1:0/3331151400 23 ==== osd_ping(ping e122 up_from 23 ping_stamp 2026-02-11T23:31:44.668214+0000/77.506988525s send_stamp 77.506988525s delta_ub -9.836580276s) ==== 2033+0+0 (crc 0 0 0) 0x4a89200 con 0x507a800 2026-02-11T23:31:44.667+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836692810s,-9.836580276s]) 2026-02-11T23:31:44.667+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.4 127.0.0.1:0/3331151400 23 ==== osd_ping(ping e122 up_from 23 ping_stamp 2026-02-11T23:31:44.668214+0000/77.506988525s send_stamp 77.506988525s delta_ub -9.836580276s) ==== 2033+0+0 (crc 0 0 0) 0x4ec2900 con 0x507a400 2026-02-11T23:31:44.667+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:44.668214+0000/77.506988525s send_stamp 87.343673706s delta_ub 9.836692810s) -- 0x4ec2c00 con 0x507a800 2026-02-11T23:31:44.667+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.4 from_epoch 122 2026-02-11T23:31:44.667+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:44.667+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:44.667+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:44.667+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836709023s,-9.836580276s]) 2026-02-11T23:31:44.667+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:44.668214+0000/77.506988525s send_stamp 87.343696594s delta_ub 9.836709023s) -- 0x4ec3200 con 0x507a400 2026-02-11T23:31:44.667+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.4 from_epoch 122 2026-02-11T23:31:44.667+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:44.667+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:44.667+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:44.697+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.2 127.0.0.1:0/2365263801 24 ==== osd_ping(ping e122 up_from 14 ping_stamp 2026-02-11T23:31:44.698568+0000/82.807159424s send_stamp 82.807159424s delta_ub -4.566760540s) ==== 2033+0+0 (crc 0 0 0) 0x4ec3800 con 0x4641000 2026-02-11T23:31:44.697+0000 7f416cbfe640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566851616s,-4.566760540s]) 2026-02-11T23:31:44.697+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:44.698568+0000/82.807159424s send_stamp 87.374015808s delta_ub 4.566851616s) -- 0x4c9be00 con 0x4641000 2026-02-11T23:31:44.697+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.2 127.0.0.1:0/2365263801 24 ==== osd_ping(ping e122 up_from 14 ping_stamp 2026-02-11T23:31:44.698568+0000/82.807159424s send_stamp 82.807159424s delta_ub -4.566760540s) ==== 2033+0+0 (crc 0 0 0) 0x4ec2c00 con 0x48cc000 2026-02-11T23:31:44.697+0000 7f416cbfe640 20 osd.0 122 get_con_osd_cluster to osd.2 from_epoch 122 2026-02-11T23:31:44.697+0000 7f416cbfe640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:44.697+0000 7f416cbfe640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:44.697+0000 7f416cbfe640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:44.697+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566864014s,-4.566760540s]) 2026-02-11T23:31:44.697+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:44.698568+0000/82.807159424s send_stamp 87.374031067s delta_ub 4.566864014s) -- 0x4a89200 con 0x48cc000 2026-02-11T23:31:44.697+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.2 from_epoch 122 2026-02-11T23:31:44.697+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:44.697+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:44.697+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:44.731+0000 7f4147983640 5 osd.0 122 heartbeat osd_stat(store_statfs(0x0/0x0/0x384000, data 0x53a3e5/0x5e6000, compress 0x0/0x0/0x0, omap 0xa403, meta 0x1a45bfd), peers [1,2,3,4] op hist []) 2026-02-11T23:31:44.731+0000 7f4147983640 20 osd.0 122 check_full_status cur ratio 1, physical ratio 1, new state failsafe 2026-02-11T23:31:44.731+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6816/3043114013,v1:127.0.0.1:6817/3043114013] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:44.732554+0000/87.407958984s send_stamp 87.407958984s delta_ub 2.011729002s) -- 0x68b0000 con 0x48c4400 2026-02-11T23:31:44.731+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6814/3043114013,v1:127.0.0.1:6815/3043114013] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:44.732554+0000/87.407958984s send_stamp 87.407958984s delta_ub 2.011729002s) -- 0x68b0180 con 0x4640c00 2026-02-11T23:31:44.731+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6824/2365263801,v1:127.0.0.1:6825/2365263801] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:44.732554+0000/87.407958984s send_stamp 87.407958984s delta_ub 4.566864014s) -- 0x68b0300 con 0x48c4000 2026-02-11T23:31:44.731+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6822/2365263801,v1:127.0.0.1:6823/2365263801] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:44.732554+0000/87.407958984s send_stamp 87.407958984s delta_ub 4.566864014s) -- 0x68b0480 con 0x48cd000 2026-02-11T23:31:44.731+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6832/2699377081,v1:127.0.0.1:6833/2699377081] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:44.732554+0000/87.407958984s send_stamp 87.407958984s delta_ub 7.219534874s) -- 0x68b0600 con 0x48cd400 2026-02-11T23:31:44.731+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6830/2699377081,v1:127.0.0.1:6831/2699377081] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:44.732554+0000/87.407958984s send_stamp 87.407958984s delta_ub 7.219534874s) -- 0x68b0780 con 0x48cd800 2026-02-11T23:31:44.731+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6840/3331151400,v1:127.0.0.1:6841/3331151400] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:44.732554+0000/87.407958984s send_stamp 87.407958984s delta_ub 9.836709023s) -- 0x68b0900 con 0x507bc00 2026-02-11T23:31:44.731+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6838/3331151400,v1:127.0.0.1:6839/3331151400] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:44.732554+0000/87.407958984s send_stamp 87.407958984s delta_ub 9.836709023s) -- 0x68b0a80 con 0x5282000 2026-02-11T23:31:44.731+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6822/2365263801 25 ==== osd_ping(ping_reply e122 up_from 14 ping_stamp 2026-02-11T23:31:44.732554+0000/87.407958984s send_stamp 82.841209412s delta_ub -4.566756248s) ==== 2033+0+0 (crc 0 0 0) 0x68b0900 con 0x48cd000 2026-02-11T23:31:44.731+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.2 from_epoch 122 2026-02-11T23:31:44.731+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:44.731+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:44.731+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:44.731+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566845894s,-4.566756248s]) 2026-02-11T23:31:44.731+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6816/3043114013 26 ==== osd_ping(ping_reply e122 up_from 10 ping_stamp 2026-02-11T23:31:44.732554+0000/87.407958984s send_stamp 85.396347046s delta_ub -2.011613846s) ==== 2033+0+0 (crc 0 0 0) 0x68b0900 con 0x48c4400 2026-02-11T23:31:44.731+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.1 from_epoch 122 2026-02-11T23:31:44.731+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:44.731+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:44.731+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:44.731+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011714220s,-2.011613846s]) 2026-02-11T23:31:44.731+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6830/2699377081 25 ==== osd_ping(ping_reply e122 up_from 19 ping_stamp 2026-02-11T23:31:44.732554+0000/87.407958984s send_stamp 80.188552856s delta_ub -7.219403744s) ==== 2033+0+0 (crc 0 0 0) 0x68b0a80 con 0x48cd800 2026-02-11T23:31:44.731+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.3 from_epoch 122 2026-02-11T23:31:44.731+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:44.731+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:44.731+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:44.731+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219510555s,-7.219403744s]) 2026-02-11T23:31:44.731+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6832/2699377081 25 ==== osd_ping(ping_reply e122 up_from 19 ping_stamp 2026-02-11T23:31:44.732554+0000/87.407958984s send_stamp 80.188568115s delta_ub -7.219385624s) ==== 2033+0+0 (crc 0 0 0) 0x68b0900 con 0x48cd400 2026-02-11T23:31:44.731+0000 7f416cbfe640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6824/2365263801 25 ==== osd_ping(ping_reply e122 up_from 14 ping_stamp 2026-02-11T23:31:44.732554+0000/87.407958984s send_stamp 82.841217041s delta_ub -4.566742897s) ==== 2033+0+0 (crc 0 0 0) 0x68b0300 con 0x48c4000 2026-02-11T23:31:44.731+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.3 from_epoch 122 2026-02-11T23:31:44.731+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:44.731+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:44.731+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:44.731+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219500542s,-7.219385624s]) 2026-02-11T23:31:44.731+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6814/3043114013 26 ==== osd_ping(ping_reply e122 up_from 10 ping_stamp 2026-02-11T23:31:44.732554+0000/87.407958984s send_stamp 85.396362305s delta_ub -2.011593580s) ==== 2033+0+0 (crc 0 0 0) 0x68b0a80 con 0x4640c00 2026-02-11T23:31:44.731+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.1 from_epoch 122 2026-02-11T23:31:44.731+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:44.731+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:44.731+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:44.731+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011716366s,-2.011593580s]) 2026-02-11T23:31:44.731+0000 7f416cbfe640 20 osd.0 122 get_con_osd_cluster to osd.2 from_epoch 122 2026-02-11T23:31:44.731+0000 7f416cbfe640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:44.731+0000 7f416cbfe640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:44.731+0000 7f416cbfe640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:44.731+0000 7f416cbfe640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566874504s,-4.566742897s]) 2026-02-11T23:31:44.731+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6838/3331151400 24 ==== osd_ping(ping_reply e122 up_from 23 ping_stamp 2026-02-11T23:31:44.732554+0000/87.407958984s send_stamp 77.571418762s delta_ub -9.836544991s) ==== 2033+0+0 (crc 0 0 0) 0x68b0a80 con 0x5282000 2026-02-11T23:31:44.731+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.4 from_epoch 122 2026-02-11T23:31:44.731+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:44.731+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:44.731+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:44.731+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836685181s,-9.836544991s]) 2026-02-11T23:31:44.731+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6840/3331151400 24 ==== osd_ping(ping_reply e122 up_from 23 ping_stamp 2026-02-11T23:31:44.732554+0000/87.407958984s send_stamp 77.571441650s delta_ub -9.836522102s) ==== 2033+0+0 (crc 0 0 0) 0x68b0900 con 0x507bc00 2026-02-11T23:31:44.731+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.4 from_epoch 122 2026-02-11T23:31:44.731+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:44.731+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:44.731+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:44.731+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836680412s,-9.836522102s]) 2026-02-11T23:31:45.330+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:45.330+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:45.330+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:15.331450+0000) 2026-02-11T23:31:45.417+0000 7f41633eb640 10 osd.0 122 tick 2026-02-11T23:31:45.417+0000 7f41633eb640 20 osd.0 122 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:45.542+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.1 127.0.0.1:0/3043114013 26 ==== osd_ping(ping e122 up_from 10 ping_stamp 2026-02-11T23:31:45.543590+0000/86.207328796s send_stamp 86.207328796s delta_ub -2.011593580s) ==== 2033+0+0 (crc 0 0 0) 0x68b0300 con 0x4aca000 2026-02-11T23:31:45.542+0000 7f416cbfe640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011715651s,-2.011593580s]) 2026-02-11T23:31:45.542+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:45.543590+0000/86.207328796s send_stamp 88.219047546s delta_ub 2.011715651s) -- 0x4c9be00 con 0x4aca000 2026-02-11T23:31:45.542+0000 7f416cbfe640 20 osd.0 122 get_con_osd_cluster to osd.1 from_epoch 122 2026-02-11T23:31:45.542+0000 7f416cbfe640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:45.542+0000 7f416cbfe640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:45.542+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.1 127.0.0.1:0/3043114013 26 ==== osd_ping(ping e122 up_from 10 ping_stamp 2026-02-11T23:31:45.543590+0000/86.207328796s send_stamp 86.207328796s delta_ub -2.011593580s) ==== 2033+0+0 (crc 0 0 0) 0x68b0a80 con 0x4aca400 2026-02-11T23:31:45.542+0000 7f416cbfe640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:45.542+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011733770s,-2.011593580s]) 2026-02-11T23:31:45.542+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:45.543590+0000/86.207328796s send_stamp 88.219062805s delta_ub 2.011733770s) -- 0x68b0780 con 0x4aca400 2026-02-11T23:31:45.542+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.1 from_epoch 122 2026-02-11T23:31:45.542+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:45.542+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:45.542+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:45.656+0000 7f4162bea640 10 osd.0 122 tick_without_osd_lock 2026-02-11T23:31:45.656+0000 7f4162bea640 0 osd.0 122 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:45.656+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:45.656+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:45.656+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:45.656+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.172 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:45.656+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:45.656, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:45.656+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:45.656+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:45.656+0000 7f4162bea640 20 osd.0 122 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:45.656+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:45.656+0000 7f4162bea640 20 osd.0 122 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:46.249+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.3 127.0.0.1:0/2699377081 23 ==== osd_ping(ping e122 up_from 19 ping_stamp 2026-02-11T23:31:46.251136+0000/81.707077026s send_stamp 81.707077026s delta_ub -7.219385624s) ==== 2033+0+0 (crc 0 0 0) 0x68b0780 con 0x48cc400 2026-02-11T23:31:46.249+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219506264s,-7.219385624s]) 2026-02-11T23:31:46.249+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:46.251136+0000/81.707077026s send_stamp 88.926582336s delta_ub 7.219506264s) -- 0x68b0a80 con 0x48cc400 2026-02-11T23:31:46.249+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.3 from_epoch 122 2026-02-11T23:31:46.249+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:46.249+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:46.249+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:46.249+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.3 127.0.0.1:0/2699377081 23 ==== osd_ping(ping e122 up_from 19 ping_stamp 2026-02-11T23:31:46.251136+0000/81.707077026s send_stamp 81.707077026s delta_ub -7.219385624s) ==== 2033+0+0 (crc 0 0 0) 0x4c9be00 con 0x48cc800 2026-02-11T23:31:46.249+0000 7f416cbfe640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219523430s,-7.219385624s]) 2026-02-11T23:31:46.249+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:46.251136+0000/81.707077026s send_stamp 88.926597595s delta_ub 7.219523430s) -- 0x68b0300 con 0x48cc800 2026-02-11T23:31:46.249+0000 7f416cbfe640 20 osd.0 122 get_con_osd_cluster to osd.3 from_epoch 122 2026-02-11T23:31:46.249+0000 7f416cbfe640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:46.249+0000 7f416cbfe640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:46.249+0000 7f416cbfe640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:46.330+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:46.330+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:46.330+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:16.331484+0000) 2026-02-11T23:31:46.380+0000 7f41633eb640 10 osd.0 122 tick 2026-02-11T23:31:46.380+0000 7f41633eb640 20 osd.0 122 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:46.701+0000 7f4162bea640 10 osd.0 122 tick_without_osd_lock 2026-02-11T23:31:46.701+0000 7f4162bea640 0 osd.0 122 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:46.701+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:46.701+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:46.701+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:46.701+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.172 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:46.701+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:46.702, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:46.701+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:46.701+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:46.701+0000 7f4162bea640 20 osd.0 122 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:46.701+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:46.701+0000 7f4162bea640 20 osd.0 122 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:47.330+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:47.330+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:47.330+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:17.331544+0000) 2026-02-11T23:31:47.428+0000 7f41633eb640 10 osd.0 122 tick 2026-02-11T23:31:47.428+0000 7f41633eb640 20 osd.0 122 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:47.659+0000 7f4162bea640 10 osd.0 122 tick_without_osd_lock 2026-02-11T23:31:47.659+0000 7f4162bea640 0 osd.0 122 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:47.659+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:47.659+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.158 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:47.659+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:47.659, recovery is active?:false restrictions:<....> 2026-02-11T23:31:47.659+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:47.659+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:47.659+0000 7f4162bea640 20 osd.0 122 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:47.659+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:47.659+0000 7f4162bea640 20 osd.0 122 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:47.842+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.1 127.0.0.1:0/3043114013 27 ==== osd_ping(ping e122 up_from 10 ping_stamp 2026-02-11T23:31:47.843802+0000/88.507545471s send_stamp 88.507545471s delta_ub -2.011582613s) ==== 2033+0+0 (crc 0 0 0) 0x68b0300 con 0x4aca000 2026-02-11T23:31:47.842+0000 7f416cbfe640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011710405s,-2.011582613s]) 2026-02-11T23:31:47.842+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.1 127.0.0.1:0/3043114013 27 ==== osd_ping(ping e122 up_from 10 ping_stamp 2026-02-11T23:31:47.843802+0000/88.507545471s send_stamp 88.507545471s delta_ub -2.011582613s) ==== 2033+0+0 (crc 0 0 0) 0x68b0a80 con 0x4aca400 2026-02-11T23:31:47.842+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:47.843802+0000/88.507545471s send_stamp 90.519256592s delta_ub 2.011710405s) -- 0x4c9be00 con 0x4aca000 2026-02-11T23:31:47.842+0000 7f416cbfe640 20 osd.0 122 get_con_osd_cluster to osd.1 from_epoch 122 2026-02-11T23:31:47.842+0000 7f416cbfe640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:47.842+0000 7f416cbfe640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:47.842+0000 7f416cbfe640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:47.842+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011725187s,-2.011582613s]) 2026-02-11T23:31:47.842+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:47.843802+0000/88.507545471s send_stamp 90.519271851s delta_ub 2.011725187s) -- 0x68b0780 con 0x4aca400 2026-02-11T23:31:47.842+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.1 from_epoch 122 2026-02-11T23:31:47.842+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:47.842+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:47.842+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:48.167+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.4 127.0.0.1:0/3331151400 24 ==== osd_ping(ping e122 up_from 23 ping_stamp 2026-02-11T23:31:48.168380+0000/81.007148743s send_stamp 81.007148743s delta_ub -9.836522102s) ==== 2033+0+0 (crc 0 0 0) 0x68b0780 con 0x507a800 2026-02-11T23:31:48.167+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836695671s,-9.836522102s]) 2026-02-11T23:31:48.167+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:48.168380+0000/81.007148743s send_stamp 90.843841553s delta_ub 9.836695671s) -- 0x68b0a80 con 0x507a800 2026-02-11T23:31:48.167+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.4 from_epoch 122 2026-02-11T23:31:48.167+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.4 127.0.0.1:0/3331151400 24 ==== osd_ping(ping e122 up_from 23 ping_stamp 2026-02-11T23:31:48.168380+0000/81.007148743s send_stamp 81.007148743s delta_ub -9.836522102s) ==== 2033+0+0 (crc 0 0 0) 0x68b0900 con 0x507a400 2026-02-11T23:31:48.167+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:48.167+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:48.167+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:48.167+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836713791s,-9.836522102s]) 2026-02-11T23:31:48.167+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:48.168380+0000/81.007148743s send_stamp 90.843864441s delta_ub 9.836713791s) -- 0x68b0600 con 0x507a400 2026-02-11T23:31:48.167+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.4 from_epoch 122 2026-02-11T23:31:48.167+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:48.167+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:48.167+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:48.197+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.2 127.0.0.1:0/2365263801 25 ==== osd_ping(ping e122 up_from 14 ping_stamp 2026-02-11T23:31:48.198709+0000/86.307304382s send_stamp 86.307304382s delta_ub -4.566742897s) ==== 2033+0+0 (crc 0 0 0) 0x4c9be00 con 0x4641000 2026-02-11T23:31:48.197+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.2 127.0.0.1:0/2365263801 25 ==== osd_ping(ping e122 up_from 14 ping_stamp 2026-02-11T23:31:48.198709+0000/86.307304382s send_stamp 86.307304382s delta_ub -4.566742897s) ==== 2033+0+0 (crc 0 0 0) 0x68b0a80 con 0x48cc000 2026-02-11T23:31:48.197+0000 7f416cbfe640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566851139s,-4.566742897s]) 2026-02-11T23:31:48.197+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:48.198709+0000/86.307304382s send_stamp 90.874160767s delta_ub 4.566851139s) -- 0x68b0300 con 0x4641000 2026-02-11T23:31:48.197+0000 7f416cbfe640 20 osd.0 122 get_con_osd_cluster to osd.2 from_epoch 122 2026-02-11T23:31:48.197+0000 7f416cbfe640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:48.197+0000 7f416cbfe640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:48.197+0000 7f416cbfe640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:48.197+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566866875s,-4.566742897s]) 2026-02-11T23:31:48.197+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:48.198709+0000/86.307304382s send_stamp 90.874176025s delta_ub 4.566866875s) -- 0x68b0780 con 0x48cc000 2026-02-11T23:31:48.197+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.2 from_epoch 122 2026-02-11T23:31:48.197+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:48.197+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:48.197+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:48.208+0000 7f415b6ff640 20 osd.0 122 reports for 0 queries 2026-02-11T23:31:48.208+0000 7f415b6ff640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] -- mgrreport(unknown.0 +0-0 packed 3294 daemon_metrics=2) -- 0x68ac700 con 0x4aca800 2026-02-11T23:31:48.208+0000 7f415b6ff640 15 osd.0 122 collect_pg_stats 2026-02-11T23:31:48.208+0000 7f415b6ff640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] -- pg_stats(0 pgs seq 21474836503 v 0) -- 0x68ae600 con 0x4aca800 2026-02-11T23:31:48.330+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:48.330+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:48.330+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:18.331602+0000) 2026-02-11T23:31:48.384+0000 7f41633eb640 10 osd.0 122 tick 2026-02-11T23:31:48.384+0000 7f41633eb640 20 osd.0 122 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:48.666+0000 7f4162bea640 10 osd.0 122 tick_without_osd_lock 2026-02-11T23:31:48.666+0000 7f4162bea640 0 osd.0 122 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:48.666+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:48.666+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:48.666+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:48.666+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.158 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:48.666+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:48.667, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:48.666+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:48.666+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:48.666+0000 7f4162bea640 20 osd.0 122 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:48.666+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:48.666+0000 7f4162bea640 20 osd.0 122 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:49.330+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:49.330+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:49.330+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:19.331662+0000) 2026-02-11T23:31:49.396+0000 7f41633eb640 10 osd.0 122 tick 2026-02-11T23:31:49.396+0000 7f41633eb640 20 osd.0 122 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:49.431+0000 7f4147983640 5 osd.0 122 heartbeat osd_stat(store_statfs(0x0/0x0/0x384000, data 0x53a3e5/0x5e6000, compress 0x0/0x0/0x0, omap 0xa403, meta 0x1a45bfd), peers [1,2,3,4] op hist []) 2026-02-11T23:31:49.431+0000 7f4147983640 20 osd.0 122 check_full_status cur ratio 1, physical ratio 1, new state failsafe 2026-02-11T23:31:49.431+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6816/3043114013,v1:127.0.0.1:6817/3043114013] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:49.432705+0000/92.108108521s send_stamp 92.108108521s delta_ub 2.011725187s) -- 0x68b0c00 con 0x48c4400 2026-02-11T23:31:49.431+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6814/3043114013,v1:127.0.0.1:6815/3043114013] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:49.432705+0000/92.108108521s send_stamp 92.108108521s delta_ub 2.011725187s) -- 0x68b0d80 con 0x4640c00 2026-02-11T23:31:49.431+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6824/2365263801,v1:127.0.0.1:6825/2365263801] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:49.432705+0000/92.108108521s send_stamp 92.108108521s delta_ub 4.566866875s) -- 0x68b0f00 con 0x48c4000 2026-02-11T23:31:49.431+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6822/2365263801,v1:127.0.0.1:6823/2365263801] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:49.432705+0000/92.108108521s send_stamp 92.108108521s delta_ub 4.566866875s) -- 0x68b1080 con 0x48cd000 2026-02-11T23:31:49.431+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6832/2699377081,v1:127.0.0.1:6833/2699377081] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:49.432705+0000/92.108108521s send_stamp 92.108108521s delta_ub 7.219523430s) -- 0x68b1200 con 0x48cd400 2026-02-11T23:31:49.431+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6830/2699377081,v1:127.0.0.1:6831/2699377081] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:49.432705+0000/92.108108521s send_stamp 92.108108521s delta_ub 7.219523430s) -- 0x68b1380 con 0x48cd800 2026-02-11T23:31:49.431+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6840/3331151400,v1:127.0.0.1:6841/3331151400] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:49.432705+0000/92.108108521s send_stamp 92.108108521s delta_ub 9.836713791s) -- 0x68b1500 con 0x507bc00 2026-02-11T23:31:49.431+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6838/3331151400,v1:127.0.0.1:6839/3331151400] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:49.432705+0000/92.108108521s send_stamp 92.108108521s delta_ub 9.836713791s) -- 0x68b1680 con 0x5282000 2026-02-11T23:31:49.431+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6816/3043114013 27 ==== osd_ping(ping_reply e122 up_from 10 ping_stamp 2026-02-11T23:31:49.432705+0000/92.108108521s send_stamp 90.096519470s delta_ub -2.011595249s) ==== 2033+0+0 (crc 0 0 0) 0x68b1500 con 0x48c4400 2026-02-11T23:31:49.431+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.1 from_epoch 122 2026-02-11T23:31:49.431+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:49.431+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:49.431+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:49.431+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011713266s,-2.011595249s]) 2026-02-11T23:31:49.431+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6814/3043114013 27 ==== osd_ping(ping_reply e122 up_from 10 ping_stamp 2026-02-11T23:31:49.432705+0000/92.108108521s send_stamp 90.096534729s delta_ub -2.011575937s) ==== 2033+0+0 (crc 0 0 0) 0x68b1680 con 0x4640c00 2026-02-11T23:31:49.431+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.1 from_epoch 122 2026-02-11T23:31:49.431+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:49.431+0000 7f416cbfe640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6824/2365263801 26 ==== osd_ping(ping_reply e122 up_from 14 ping_stamp 2026-02-11T23:31:49.432705+0000/92.108108521s send_stamp 87.541374207s delta_ub -4.566737175s) ==== 2033+0+0 (crc 0 0 0) 0x68b0f00 con 0x48c4000 2026-02-11T23:31:49.431+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:49.431+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:49.431+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011703968s,-2.011575937s]) 2026-02-11T23:31:49.431+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6822/2365263801 26 ==== osd_ping(ping_reply e122 up_from 14 ping_stamp 2026-02-11T23:31:49.432705+0000/92.108108521s send_stamp 87.541389465s delta_ub -4.566719532s) ==== 2033+0+0 (crc 0 0 0) 0x68b1500 con 0x48cd000 2026-02-11T23:31:49.431+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.2 from_epoch 122 2026-02-11T23:31:49.431+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:49.431+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:49.431+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:49.431+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566857815s,-4.566719532s]) 2026-02-11T23:31:49.431+0000 7f416cbfe640 20 osd.0 122 get_con_osd_cluster to osd.2 from_epoch 122 2026-02-11T23:31:49.431+0000 7f416cbfe640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:49.431+0000 7f416cbfe640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:49.431+0000 7f416cbfe640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:49.431+0000 7f416cbfe640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566884041s,-4.566737175s]) 2026-02-11T23:31:49.431+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6830/2699377081 26 ==== osd_ping(ping_reply e122 up_from 19 ping_stamp 2026-02-11T23:31:49.432705+0000/92.108108521s send_stamp 84.888763428s delta_ub -7.219347477s) ==== 2033+0+0 (crc 0 0 0) 0x68b1680 con 0x48cd800 2026-02-11T23:31:49.431+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.3 from_epoch 122 2026-02-11T23:31:49.431+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:49.431+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:49.431+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6832/2699377081 26 ==== osd_ping(ping_reply e122 up_from 19 ping_stamp 2026-02-11T23:31:49.432705+0000/92.108108521s send_stamp 84.888748169s delta_ub -7.219365597s) ==== 2033+0+0 (crc 0 0 0) 0x68b1500 con 0x48cd400 2026-02-11T23:31:49.431+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:49.431+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219501019s,-7.219347477s]) 2026-02-11T23:31:49.431+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.3 from_epoch 122 2026-02-11T23:31:49.431+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:49.431+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:49.431+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6838/3331151400 25 ==== osd_ping(ping_reply e122 up_from 23 ping_stamp 2026-02-11T23:31:49.432705+0000/92.108108521s send_stamp 82.271606445s delta_ub -9.836504936s) ==== 2033+0+0 (crc 0 0 0) 0x68b1680 con 0x5282000 2026-02-11T23:31:49.431+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:49.431+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219528198s,-7.219365597s]) 2026-02-11T23:31:49.431+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.4 from_epoch 122 2026-02-11T23:31:49.431+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:49.431+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:49.431+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:49.431+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836677551s,-9.836504936s]) 2026-02-11T23:31:49.431+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6840/3331151400 25 ==== osd_ping(ping_reply e122 up_from 23 ping_stamp 2026-02-11T23:31:49.432705+0000/92.108108521s send_stamp 82.271591187s delta_ub -9.836517334s) ==== 2033+0+0 (crc 0 0 0) 0x68b1500 con 0x507bc00 2026-02-11T23:31:49.431+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.4 from_epoch 122 2026-02-11T23:31:49.431+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:49.431+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:49.431+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:49.431+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836695671s,-9.836517334s]) 2026-02-11T23:31:49.714+0000 7f4162bea640 10 osd.0 122 tick_without_osd_lock 2026-02-11T23:31:49.714+0000 7f4162bea640 0 osd.0 122 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:49.714+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:49.714+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.158 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:49.714+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:49.715, recovery is active?:false restrictions:<....> 2026-02-11T23:31:49.714+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:49.714+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:49.714+0000 7f4162bea640 20 osd.0 122 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:49.714+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:49.714+0000 7f4162bea640 20 osd.0 122 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:49.750+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.3 127.0.0.1:0/2699377081 24 ==== osd_ping(ping e122 up_from 19 ping_stamp 2026-02-11T23:31:49.751278+0000/85.207221985s send_stamp 85.207221985s delta_ub -7.219347477s) ==== 2033+0+0 (crc 0 0 0) 0x68b0f00 con 0x48cc800 2026-02-11T23:31:49.750+0000 7f416cbfe640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219521523s,-7.219347477s]) 2026-02-11T23:31:49.750+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:49.751278+0000/85.207221985s send_stamp 92.426742554s delta_ub 7.219521523s) -- 0x68b0300 con 0x48cc800 2026-02-11T23:31:49.750+0000 7f416cbfe640 20 osd.0 122 get_con_osd_cluster to osd.3 from_epoch 122 2026-02-11T23:31:49.750+0000 7f416cbfe640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:49.750+0000 7f416cbfe640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:49.750+0000 7f416cbfe640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:49.750+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.3 127.0.0.1:0/2699377081 24 ==== osd_ping(ping e122 up_from 19 ping_stamp 2026-02-11T23:31:49.751278+0000/85.207221985s send_stamp 85.207221985s delta_ub -7.219347477s) ==== 2033+0+0 (crc 0 0 0) 0x68b1680 con 0x48cc400 2026-02-11T23:31:49.750+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219598770s,-7.219347477s]) 2026-02-11T23:31:49.750+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:49.751278+0000/85.207221985s send_stamp 92.426811218s delta_ub 7.219598770s) -- 0x68b1380 con 0x48cc400 2026-02-11T23:31:49.750+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.3 from_epoch 122 2026-02-11T23:31:49.750+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:49.750+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:49.750+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:50.330+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:50.330+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:50.330+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:20.331723+0000) 2026-02-11T23:31:50.330+0000 7f415bf00640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-11T23:31:50.357+0000 7f41633eb640 10 osd.0 122 tick 2026-02-11T23:31:50.357+0000 7f41633eb640 20 osd.0 122 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:50.734+0000 7f4162bea640 10 osd.0 122 tick_without_osd_lock 2026-02-11T23:31:50.734+0000 7f4162bea640 0 osd.0 122 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:50.734+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:50.734+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.158 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:50.734+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:50.735, recovery is active?:false restrictions:<....> 2026-02-11T23:31:50.734+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:50.734+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:50.734+0000 7f4162bea640 20 osd.0 122 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:50.734+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:50.734+0000 7f4162bea640 20 osd.0 122 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:51.327+0000 7f41633eb640 10 osd.0 122 tick 2026-02-11T23:31:51.327+0000 7f41633eb640 20 osd.0 122 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:51.330+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:51.330+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:51.330+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:21.331793+0000) 2026-02-11T23:31:51.697+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.2 127.0.0.1:0/2365263801 26 ==== osd_ping(ping e122 up_from 14 ping_stamp 2026-02-11T23:31:51.698855+0000/89.807456970s send_stamp 89.807456970s delta_ub -4.566719532s) ==== 2033+0+0 (crc 0 0 0) 0x68b0300 con 0x4641000 2026-02-11T23:31:51.697+0000 7f416cbfe640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566870689s,-4.566719532s]) 2026-02-11T23:31:51.697+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.2 127.0.0.1:0/2365263801 26 ==== osd_ping(ping e122 up_from 14 ping_stamp 2026-02-11T23:31:51.698855+0000/89.807456970s send_stamp 89.807456970s delta_ub -4.566719532s) ==== 2033+0+0 (crc 0 0 0) 0x68b1380 con 0x48cc000 2026-02-11T23:31:51.697+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:51.698855+0000/89.807456970s send_stamp 94.374320984s delta_ub 4.566870689s) -- 0x68b0f00 con 0x4641000 2026-02-11T23:31:51.697+0000 7f416cbfe640 20 osd.0 122 get_con_osd_cluster to osd.2 from_epoch 122 2026-02-11T23:31:51.697+0000 7f416cbfe640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:51.697+0000 7f416cbfe640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:51.697+0000 7f416cbfe640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:51.697+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566888332s,-4.566719532s]) 2026-02-11T23:31:51.697+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:51.698855+0000/89.807456970s send_stamp 94.374336243s delta_ub 4.566888332s) -- 0x68b1680 con 0x48cc000 2026-02-11T23:31:51.697+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.2 from_epoch 122 2026-02-11T23:31:51.697+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:51.697+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:51.697+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:51.749+0000 7f4162bea640 10 osd.0 122 tick_without_osd_lock 2026-02-11T23:31:51.749+0000 7f4162bea640 0 osd.0 122 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:51.749+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:51.749+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.158 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:51.749+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:51.750, recovery is active?:false restrictions:<....> 2026-02-11T23:31:51.749+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:51.749+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:51.749+0000 7f4162bea640 20 osd.0 122 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:51.749+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:51.749+0000 7f4162bea640 20 osd.0 122 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:52.034+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1248 ==== pg_lease(1.0 pg_lease(ru 100.698654175s ub 108.698791504s int 16.000000000s) e122/122) ==== 58+0+0 (unknown 0 0 0) 0x68b1500 con 0x4acb000 2026-02-11T23:31:52.034+0000 7f416abfa640 15 osd.0 122 enqueue_peering_evt 1.0 epoch_sent: 122 epoch_requested: 122 MLease epoch 122 from osd.1 pg_lease(ru 100.698654175s ub 108.698791504s int 16.000000000s) 2026-02-11T23:31:52.034+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 122 epoch_requested: 122 MLease epoch 122 from osd.1 pg_lease(ru 100.698654175s ub 108.698791504s int 16.000000000s)) class_id 2 prio 255 cost 10 e122) 2026-02-11T23:31:52.034+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:52.034+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 122 epoch_requested: 122 MLease epoch 122 from osd.1 pg_lease(ru 100.698654175s ub 108.698791504s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e122) queued 2026-02-11T23:31:52.034+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:52.034+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 122 epoch_requested: 122 MLease epoch 122 from osd.1 pg_lease(ru 100.698654175s ub 108.698791504s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e122) pg 0x4ef0000 2026-02-11T23:31:52.034+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:52.034+0000 7f414f993640 10 osd.0 pg_epoch: 122 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/122 sis=118) [1,0] r=1 lpr=118 crt=111'600 active mbc={}] do_peering_event: epoch_sent: 122 epoch_requested: 122 MLease epoch 122 from osd.1 pg_lease(ru 100.698654175s ub 108.698791504s int 16.000000000s) 2026-02-11T23:31:52.034+0000 7f414f993640 10 osd.0 pg_epoch: 122 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/122 sis=118) [1,0] r=1 lpr=118 crt=111'600 active mbc={}] PeeringState::proc_lease pg_lease(ru 100.698654175s ub 108.698791504s int 16.000000000s) 2026-02-11T23:31:52.034+0000 7f414f993640 20 osd.0 pg_epoch: 122 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/122 sis=118) [1,0] r=1 lpr=118 crt=111'600 active mbc={}] PeeringState::proc_lease peer_clock_delta_ub -2.011575937s -> ru 102.710227966s 2026-02-11T23:31:52.034+0000 7f414f993640 20 osd.0 pg_epoch: 122 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/122 sis=118) [1,0] r=1 lpr=118 crt=111'600 active mbc={}] PeeringState::proc_lease readable_until now 102.710227966s 2026-02-11T23:31:52.034+0000 7f414f993640 20 osd.0 pg_epoch: 122 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/122 sis=118) [1,0] r=1 lpr=118 crt=111'600 active mbc={}] PeeringState::proc_lease peer_clock_delta_lb -2.011703968s -> ruub 110.710494995s 2026-02-11T23:31:52.034+0000 7f414f993640 20 osd.0 pg_epoch: 122 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/122 sis=118) [1,0] r=1 lpr=118 crt=111'600 active mbc={}] PeeringState::proc_lease readable_until_ub now 110.710494995s 2026-02-11T23:31:52.034+0000 7f414f993640 20 osd.0 122 get_con_osd_cluster to osd.1 from_epoch 122 2026-02-11T23:31:52.034+0000 7f414f993640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:52.034+0000 7f414f993640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:52.034+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_lease_ack(1.0 pg_lease_ack(ruub 108.698791504s) e122/122) -- 0x68aa420 con 0x4acb000 2026-02-11T23:31:52.034+0000 7f414f993640 10 log is not dirty 2026-02-11T23:31:52.034+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:52.325+0000 7f41633eb640 10 osd.0 122 tick 2026-02-11T23:31:52.325+0000 7f41633eb640 20 osd.0 122 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:52.330+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:52.330+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:52.330+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:22.331855+0000) 2026-02-11T23:31:52.438+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1251 ==== pg_lease(4.0 pg_lease(ru 93.277740479s ub 101.277824402s int 16.000000000s) e122/122) ==== 58+0+0 (unknown 0 0 0) 0x68b1500 con 0x507ac00 2026-02-11T23:31:52.438+0000 7f416abfa640 15 osd.0 122 enqueue_peering_evt 4.0 epoch_sent: 122 epoch_requested: 122 MLease epoch 122 from osd.4 pg_lease(ru 93.277740479s ub 101.277824402s int 16.000000000s) 2026-02-11T23:31:52.438+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 122 epoch_requested: 122 MLease epoch 122 from osd.4 pg_lease(ru 93.277740479s ub 101.277824402s int 16.000000000s)) class_id 2 prio 255 cost 10 e122) 2026-02-11T23:31:52.438+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:52.438+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 122 epoch_requested: 122 MLease epoch 122 from osd.4 pg_lease(ru 93.277740479s ub 101.277824402s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e122) queued 2026-02-11T23:31:52.438+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:52.438+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 122 epoch_requested: 122 MLease epoch 122 from osd.4 pg_lease(ru 93.277740479s ub 101.277824402s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e122) pg 0x5978000 2026-02-11T23:31:52.438+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:52.438+0000 7f414b98b640 10 osd.0 pg_epoch: 122 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/120 les/c/f=121/121/122 sis=120) [4,0] r=1 lpr=120 crt=111'600 active mbc={}] do_peering_event: epoch_sent: 122 epoch_requested: 122 MLease epoch 122 from osd.4 pg_lease(ru 93.277740479s ub 101.277824402s int 16.000000000s) 2026-02-11T23:31:52.438+0000 7f414b98b640 10 osd.0 pg_epoch: 122 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/120 les/c/f=121/121/122 sis=120) [4,0] r=1 lpr=120 crt=111'600 active mbc={}] PeeringState::proc_lease pg_lease(ru 93.277740479s ub 101.277824402s int 16.000000000s) 2026-02-11T23:31:52.438+0000 7f414b98b640 20 osd.0 pg_epoch: 122 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/120 les/c/f=121/121/122 sis=120) [4,0] r=1 lpr=120 crt=111'600 active mbc={}] PeeringState::proc_lease peer_clock_delta_ub -9.836517334s -> ru 103.114257812s 2026-02-11T23:31:52.438+0000 7f414b98b640 20 osd.0 pg_epoch: 122 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/120 les/c/f=121/121/122 sis=120) [4,0] r=1 lpr=120 crt=111'600 active mbc={}] PeeringState::proc_lease readable_until now 103.114257812s 2026-02-11T23:31:52.438+0000 7f414b98b640 20 osd.0 pg_epoch: 122 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/120 les/c/f=121/121/122 sis=120) [4,0] r=1 lpr=120 crt=111'600 active mbc={}] PeeringState::proc_lease peer_clock_delta_lb -9.836695671s -> ruub 111.114524841s 2026-02-11T23:31:52.438+0000 7f414b98b640 20 osd.0 pg_epoch: 122 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/120 les/c/f=121/121/122 sis=120) [4,0] r=1 lpr=120 crt=111'600 active mbc={}] PeeringState::proc_lease readable_until_ub now 111.114524841s 2026-02-11T23:31:52.438+0000 7f414b98b640 20 osd.0 122 get_con_osd_cluster to osd.4 from_epoch 122 2026-02-11T23:31:52.438+0000 7f414b98b640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:52.438+0000 7f414b98b640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:52.438+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_lease_ack(4.0 pg_lease_ack(ruub 101.277824402s) e122/122) -- 0x68aa000 con 0x507ac00 2026-02-11T23:31:52.438+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:31:52.438+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:52.787+0000 7f4162bea640 10 osd.0 122 tick_without_osd_lock 2026-02-11T23:31:52.787+0000 7f4162bea640 0 osd.0 122 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:52.787+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:52.787+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:52.787+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:52.787+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.145 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:52.787+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:52.788, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:52.787+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:52.787+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:52.787+0000 7f4162bea640 20 osd.0 122 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:52.787+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:52.787+0000 7f4162bea640 20 osd.0 122 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:53.142+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.1 127.0.0.1:0/3043114013 28 ==== osd_ping(ping e122 up_from 10 ping_stamp 2026-02-11T23:31:53.143976+0000/93.807723999s send_stamp 93.807723999s delta_ub -2.011575937s) ==== 2033+0+0 (crc 0 0 0) 0x68b1680 con 0x4aca400 2026-02-11T23:31:53.142+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011723995s,-2.011575937s]) 2026-02-11T23:31:53.142+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.1 127.0.0.1:0/3043114013 28 ==== osd_ping(ping e122 up_from 10 ping_stamp 2026-02-11T23:31:53.143976+0000/93.807723999s send_stamp 93.807723999s delta_ub -2.011575937s) ==== 2033+0+0 (crc 0 0 0) 0x68b0f00 con 0x4aca000 2026-02-11T23:31:53.142+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:53.143976+0000/93.807723999s send_stamp 95.819450378s delta_ub 2.011723995s) -- 0x68b1380 con 0x4aca400 2026-02-11T23:31:53.142+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.1 from_epoch 122 2026-02-11T23:31:53.142+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:53.142+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:53.142+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:53.142+0000 7f416cbfe640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011741161s,-2.011575937s]) 2026-02-11T23:31:53.142+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:53.143976+0000/93.807723999s send_stamp 95.819465637s delta_ub 2.011741161s) -- 0x68b0300 con 0x4aca000 2026-02-11T23:31:53.142+0000 7f416cbfe640 20 osd.0 122 get_con_osd_cluster to osd.1 from_epoch 122 2026-02-11T23:31:53.142+0000 7f416cbfe640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:53.142+0000 7f416cbfe640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:53.142+0000 7f416cbfe640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:53.208+0000 7f415b6ff640 20 osd.0 122 reports for 0 queries 2026-02-11T23:31:53.208+0000 7f415b6ff640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] -- mgrreport(unknown.0 +0-0 packed 3294 daemon_metrics=2) -- 0x68aca80 con 0x4aca800 2026-02-11T23:31:53.208+0000 7f415b6ff640 15 osd.0 122 collect_pg_stats 2026-02-11T23:31:53.208+0000 7f415b6ff640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] -- pg_stats(0 pgs seq 21474836504 v 0) -- 0x68ae900 con 0x4aca800 2026-02-11T23:31:53.276+0000 7f41633eb640 10 osd.0 122 tick 2026-02-11T23:31:53.276+0000 7f41633eb640 20 osd.0 122 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:53.330+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:53.330+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:53.330+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:23.331917+0000) 2026-02-11T23:31:53.398+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.2 127.0.0.1:0/2365263801 27 ==== osd_ping(ping e122 up_from 14 ping_stamp 2026-02-11T23:31:53.399005+0000/91.507598877s send_stamp 91.507598877s delta_ub -4.566763401s) ==== 2033+0+0 (crc 0 0 0) 0x68b0300 con 0x4641000 2026-02-11T23:31:53.398+0000 7f416cbfe640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566854477s,-4.566763401s]) 2026-02-11T23:31:53.398+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.2 127.0.0.1:0/2365263801 27 ==== osd_ping(ping e122 up_from 14 ping_stamp 2026-02-11T23:31:53.399005+0000/91.507598877s send_stamp 91.507598877s delta_ub -4.566763401s) ==== 2033+0+0 (crc 0 0 0) 0x68b1380 con 0x48cc000 2026-02-11T23:31:53.398+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:53.399005+0000/91.507598877s send_stamp 96.074455261s delta_ub 4.566854477s) -- 0x68b0f00 con 0x4641000 2026-02-11T23:31:53.398+0000 7f416cbfe640 20 osd.0 122 get_con_osd_cluster to osd.2 from_epoch 122 2026-02-11T23:31:53.398+0000 7f416cbfe640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:53.398+0000 7f416cbfe640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:53.398+0000 7f416cbfe640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:53.398+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566880703s,-4.566763401s]) 2026-02-11T23:31:53.398+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:53.399005+0000/91.507598877s send_stamp 96.074478149s delta_ub 4.566880703s) -- 0x68b1680 con 0x48cc000 2026-02-11T23:31:53.398+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.2 from_epoch 122 2026-02-11T23:31:53.398+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:53.398+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:53.398+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:53.467+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.4 127.0.0.1:0/3331151400 25 ==== osd_ping(ping e122 up_from 23 ping_stamp 2026-02-11T23:31:53.468526+0000/86.307289124s send_stamp 86.307289124s delta_ub -9.836504936s) ==== 2033+0+0 (crc 0 0 0) 0x68b1680 con 0x507a800 2026-02-11T23:31:53.467+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836693764s,-9.836504936s]) 2026-02-11T23:31:53.467+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.4 127.0.0.1:0/3331151400 25 ==== osd_ping(ping e122 up_from 23 ping_stamp 2026-02-11T23:31:53.468526+0000/86.307289124s send_stamp 86.307289124s delta_ub -9.836504936s) ==== 2033+0+0 (crc 0 0 0) 0x68b1500 con 0x507a400 2026-02-11T23:31:53.467+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:53.468526+0000/86.307289124s send_stamp 96.143989563s delta_ub 9.836693764s) -- 0x68b1380 con 0x507a800 2026-02-11T23:31:53.467+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.4 from_epoch 122 2026-02-11T23:31:53.467+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:53.467+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:53.467+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:53.467+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836709023s,-9.836504936s]) 2026-02-11T23:31:53.467+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:53.468526+0000/86.307289124s send_stamp 96.143997192s delta_ub 9.836709023s) -- 0x68b1200 con 0x507a400 2026-02-11T23:31:53.467+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.4 from_epoch 122 2026-02-11T23:31:53.467+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:53.467+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:53.467+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:53.783+0000 7f4162bea640 10 osd.0 122 tick_without_osd_lock 2026-02-11T23:31:53.783+0000 7f4162bea640 0 osd.0 122 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:53.783+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:53.783+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.145 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:53.783+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:53.784, recovery is active?:false restrictions:<....> 2026-02-11T23:31:53.783+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:53.783+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:53.783+0000 7f4162bea640 20 osd.0 122 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:53.783+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:53.783+0000 7f4162bea640 20 osd.0 122 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:54.132+0000 7f4147983640 5 osd.0 122 heartbeat osd_stat(store_statfs(0x0/0x0/0x384000, data 0x53a3e5/0x5e6000, compress 0x0/0x0/0x0, omap 0xa403, meta 0x1a45bfd), peers [1,2,3,4] op hist []) 2026-02-11T23:31:54.132+0000 7f4147983640 20 osd.0 122 check_full_status cur ratio 1, physical ratio 1, new state failsafe 2026-02-11T23:31:54.132+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6816/3043114013,v1:127.0.0.1:6817/3043114013] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:54.132864+0000/96.808273315s send_stamp 96.808273315s delta_ub 2.011741161s) -- 0x68b1800 con 0x48c4400 2026-02-11T23:31:54.132+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6814/3043114013,v1:127.0.0.1:6815/3043114013] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:54.132864+0000/96.808273315s send_stamp 96.808273315s delta_ub 2.011741161s) -- 0x68b1980 con 0x4640c00 2026-02-11T23:31:54.132+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6824/2365263801,v1:127.0.0.1:6825/2365263801] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:54.132864+0000/96.808273315s send_stamp 96.808273315s delta_ub 4.566880703s) -- 0x68b1b00 con 0x48c4000 2026-02-11T23:31:54.132+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6822/2365263801,v1:127.0.0.1:6823/2365263801] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:54.132864+0000/96.808273315s send_stamp 96.808273315s delta_ub 4.566880703s) -- 0x68b1c80 con 0x48cd000 2026-02-11T23:31:54.132+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6832/2699377081,v1:127.0.0.1:6833/2699377081] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:54.132864+0000/96.808273315s send_stamp 96.808273315s delta_ub 7.219598770s) -- 0x68b1e00 con 0x48cd400 2026-02-11T23:31:54.132+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6830/2699377081,v1:127.0.0.1:6831/2699377081] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:54.132864+0000/96.808273315s send_stamp 96.808273315s delta_ub 7.219598770s) -- 0x68bc000 con 0x48cd800 2026-02-11T23:31:54.132+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6840/3331151400,v1:127.0.0.1:6841/3331151400] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:54.132864+0000/96.808273315s send_stamp 96.808273315s delta_ub 9.836709023s) -- 0x68bc180 con 0x507bc00 2026-02-11T23:31:54.132+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6838/3331151400,v1:127.0.0.1:6839/3331151400] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:54.132864+0000/96.808273315s send_stamp 96.808273315s delta_ub 9.836709023s) -- 0x68bc300 con 0x5282000 2026-02-11T23:31:54.132+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6816/3043114013 28 ==== osd_ping(ping_reply e122 up_from 10 ping_stamp 2026-02-11T23:31:54.132864+0000/96.808273315s send_stamp 94.796653748s delta_ub -2.011613607s) ==== 2033+0+0 (crc 0 0 0) 0x68bc180 con 0x48c4400 2026-02-11T23:31:54.132+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.1 from_epoch 122 2026-02-11T23:31:54.132+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:54.132+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:54.132+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6814/3043114013 28 ==== osd_ping(ping_reply e122 up_from 10 ping_stamp 2026-02-11T23:31:54.132864+0000/96.808273315s send_stamp 94.796669006s delta_ub -2.011602879s) ==== 2033+0+0 (crc 0 0 0) 0x68bc300 con 0x4640c00 2026-02-11T23:31:54.132+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:54.132+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011706591s,-2.011613607s]) 2026-02-11T23:31:54.132+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.1 from_epoch 122 2026-02-11T23:31:54.132+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:54.132+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:54.132+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:54.132+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011708498s,-2.011602879s]) 2026-02-11T23:31:54.132+0000 7f416cbfe640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6824/2365263801 27 ==== osd_ping(ping_reply e122 up_from 14 ping_stamp 2026-02-11T23:31:54.132864+0000/96.808273315s send_stamp 92.241523743s delta_ub -4.566741943s) ==== 2033+0+0 (crc 0 0 0) 0x68b0300 con 0x48c4000 2026-02-11T23:31:54.132+0000 7f416cbfe640 20 osd.0 122 get_con_osd_cluster to osd.2 from_epoch 122 2026-02-11T23:31:54.132+0000 7f416cbfe640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:54.132+0000 7f416cbfe640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:54.132+0000 7f416cbfe640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:54.132+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6832/2699377081 27 ==== osd_ping(ping_reply e122 up_from 19 ping_stamp 2026-02-11T23:31:54.132864+0000/96.808273315s send_stamp 89.588882446s delta_ub -7.219388008s) ==== 2033+0+0 (crc 0 0 0) 0x68bc180 con 0x48cd400 2026-02-11T23:31:54.132+0000 7f416cbfe640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566859722s,-4.566741943s]) 2026-02-11T23:31:54.132+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6838/3331151400 26 ==== osd_ping(ping_reply e122 up_from 23 ping_stamp 2026-02-11T23:31:54.132864+0000/96.808273315s send_stamp 86.971725464s delta_ub -9.836544991s) ==== 2033+0+0 (crc 0 0 0) 0x68bc300 con 0x5282000 2026-02-11T23:31:54.132+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.3 from_epoch 122 2026-02-11T23:31:54.132+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:54.132+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:54.132+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:54.132+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219518661s,-7.219388008s]) 2026-02-11T23:31:54.132+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.4 from_epoch 122 2026-02-11T23:31:54.132+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:54.132+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:54.132+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:54.132+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836683273s,-9.836544991s]) 2026-02-11T23:31:54.132+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6822/2365263801 27 ==== osd_ping(ping_reply e122 up_from 14 ping_stamp 2026-02-11T23:31:54.132864+0000/96.808273315s send_stamp 92.241546631s delta_ub -4.566727161s) ==== 2033+0+0 (crc 0 0 0) 0x68bc180 con 0x48cd000 2026-02-11T23:31:54.132+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.2 from_epoch 122 2026-02-11T23:31:54.132+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:54.132+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:54.132+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:54.132+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566872597s,-4.566727161s]) 2026-02-11T23:31:54.132+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6830/2699377081 27 ==== osd_ping(ping_reply e122 up_from 19 ping_stamp 2026-02-11T23:31:54.132864+0000/96.808273315s send_stamp 89.588897705s delta_ub -7.219368935s) ==== 2033+0+0 (crc 0 0 0) 0x68bc300 con 0x48cd800 2026-02-11T23:31:54.132+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.3 from_epoch 122 2026-02-11T23:31:54.132+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:54.132+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:54.132+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:54.132+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219521999s,-7.219368935s]) 2026-02-11T23:31:54.132+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6840/3331151400 26 ==== osd_ping(ping_reply e122 up_from 23 ping_stamp 2026-02-11T23:31:54.132864+0000/96.808273315s send_stamp 86.971740723s delta_ub -9.836526871s) ==== 2033+0+0 (crc 0 0 0) 0x68bc180 con 0x507bc00 2026-02-11T23:31:54.132+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.4 from_epoch 122 2026-02-11T23:31:54.132+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:54.132+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:54.132+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:54.132+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836686134s,-9.836526871s]) 2026-02-11T23:31:54.277+0000 7f41633eb640 10 osd.0 122 tick 2026-02-11T23:31:54.277+0000 7f41633eb640 20 osd.0 122 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:54.330+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:54.330+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:54.330+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:24.331980+0000) 2026-02-11T23:31:54.450+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.3 127.0.0.1:0/2699377081 25 ==== osd_ping(ping e122 up_from 19 ping_stamp 2026-02-11T23:31:54.451448+0000/89.907386780s send_stamp 89.907386780s delta_ub -7.219368935s) ==== 2033+0+0 (crc 0 0 0) 0x68bc300 con 0x48cc400 2026-02-11T23:31:54.450+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219518661s,-7.219368935s]) 2026-02-11T23:31:54.450+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.3 127.0.0.1:0/2699377081 25 ==== osd_ping(ping e122 up_from 19 ping_stamp 2026-02-11T23:31:54.451448+0000/89.907386780s send_stamp 89.907386780s delta_ub -7.219368935s) ==== 2033+0+0 (crc 0 0 0) 0x68b0300 con 0x48cc800 2026-02-11T23:31:54.450+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:54.451448+0000/89.907386780s send_stamp 97.126907349s delta_ub 7.219518661s) -- 0x68bc000 con 0x48cc400 2026-02-11T23:31:54.450+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.3 from_epoch 122 2026-02-11T23:31:54.450+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:54.450+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:54.450+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:54.450+0000 7f416cbfe640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219533920s,-7.219368935s]) 2026-02-11T23:31:54.450+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:54.451448+0000/89.907386780s send_stamp 97.126922607s delta_ub 7.219533920s) -- 0x68b0f00 con 0x48cc800 2026-02-11T23:31:54.450+0000 7f416cbfe640 20 osd.0 122 get_con_osd_cluster to osd.3 from_epoch 122 2026-02-11T23:31:54.450+0000 7f416cbfe640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:54.450+0000 7f416cbfe640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:54.450+0000 7f416cbfe640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:54.744+0000 7f4162bea640 10 osd.0 122 tick_without_osd_lock 2026-02-11T23:31:54.744+0000 7f4162bea640 0 osd.0 122 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:54.744+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:54.744+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:54.744+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:54.744+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.145 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:54.744+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:54.744, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:54.744+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:54.744+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:54.744+0000 7f4162bea640 20 osd.0 122 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:54.744+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:54.744+0000 7f4162bea640 20 osd.0 122 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:55.168+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.4 127.0.0.1:0/3331151400 26 ==== osd_ping(ping e122 up_from 23 ping_stamp 2026-02-11T23:31:55.168691+0000/88.007461548s send_stamp 88.007461548s delta_ub -9.836526871s) ==== 2033+0+0 (crc 0 0 0) 0x68bc000 con 0x507a800 2026-02-11T23:31:55.168+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836690903s,-9.836526871s]) 2026-02-11T23:31:55.168+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.4 127.0.0.1:0/3331151400 26 ==== osd_ping(ping e122 up_from 23 ping_stamp 2026-02-11T23:31:55.168691+0000/88.007461548s send_stamp 88.007461548s delta_ub -9.836526871s) ==== 2033+0+0 (crc 0 0 0) 0x68bc180 con 0x507a400 2026-02-11T23:31:55.168+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:55.168691+0000/88.007461548s send_stamp 97.844146729s delta_ub 9.836690903s) -- 0x68bc300 con 0x507a800 2026-02-11T23:31:55.168+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.4 from_epoch 122 2026-02-11T23:31:55.168+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:55.168+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:55.168+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:55.168+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836703300s,-9.836526871s]) 2026-02-11T23:31:55.168+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:55.168691+0000/88.007461548s send_stamp 97.844169617s delta_ub 9.836703300s) -- 0x68b1e00 con 0x507a400 2026-02-11T23:31:55.168+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.4 from_epoch 122 2026-02-11T23:31:55.168+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:55.168+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:55.168+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:55.231+0000 7f4147983640 5 osd.0 122 heartbeat osd_stat(store_statfs(0x0/0x0/0x384000, data 0x53a3e5/0x5e6000, compress 0x0/0x0/0x0, omap 0xa403, meta 0x1a45bfd), peers [1,2,3,4] op hist []) 2026-02-11T23:31:55.231+0000 7f4147983640 20 osd.0 122 check_full_status cur ratio 1, physical ratio 1, new state failsafe 2026-02-11T23:31:55.231+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6816/3043114013,v1:127.0.0.1:6817/3043114013] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:55.232996+0000/97.908401489s send_stamp 97.908401489s delta_ub 2.011708498s) -- 0x68bc480 con 0x48c4400 2026-02-11T23:31:55.231+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6814/3043114013,v1:127.0.0.1:6815/3043114013] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:55.232996+0000/97.908401489s send_stamp 97.908401489s delta_ub 2.011708498s) -- 0x68bc600 con 0x4640c00 2026-02-11T23:31:55.231+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6824/2365263801,v1:127.0.0.1:6825/2365263801] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:55.232996+0000/97.908401489s send_stamp 97.908401489s delta_ub 4.566872597s) -- 0x68bc780 con 0x48c4000 2026-02-11T23:31:55.231+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6822/2365263801,v1:127.0.0.1:6823/2365263801] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:55.232996+0000/97.908401489s send_stamp 97.908401489s delta_ub 4.566872597s) -- 0x68bc900 con 0x48cd000 2026-02-11T23:31:55.231+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6832/2699377081,v1:127.0.0.1:6833/2699377081] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:55.232996+0000/97.908401489s send_stamp 97.908401489s delta_ub 7.219533920s) -- 0x68bca80 con 0x48cd400 2026-02-11T23:31:55.231+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6830/2699377081,v1:127.0.0.1:6831/2699377081] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:55.232996+0000/97.908401489s send_stamp 97.908401489s delta_ub 7.219533920s) -- 0x68bcc00 con 0x48cd800 2026-02-11T23:31:55.231+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6840/3331151400,v1:127.0.0.1:6841/3331151400] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:55.232996+0000/97.908401489s send_stamp 97.908401489s delta_ub 9.836703300s) -- 0x68bcd80 con 0x507bc00 2026-02-11T23:31:55.231+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6838/3331151400,v1:127.0.0.1:6839/3331151400] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:55.232996+0000/97.908401489s send_stamp 97.908401489s delta_ub 9.836703300s) -- 0x68bcf00 con 0x5282000 2026-02-11T23:31:55.231+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6816/3043114013 29 ==== osd_ping(ping_reply e122 up_from 10 ping_stamp 2026-02-11T23:31:55.232996+0000/97.908401489s send_stamp 95.896789551s delta_ub -2.011615276s) ==== 2033+0+0 (crc 0 0 0) 0x68bca80 con 0x48c4400 2026-02-11T23:31:55.231+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.1 from_epoch 122 2026-02-11T23:31:55.231+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:55.231+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:55.231+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:55.231+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6814/3043114013 29 ==== osd_ping(ping_reply e122 up_from 10 ping_stamp 2026-02-11T23:31:55.232996+0000/97.908401489s send_stamp 95.896797180s delta_ub -2.011604071s) ==== 2033+0+0 (crc 0 0 0) 0x68bc000 con 0x4640c00 2026-02-11T23:31:55.231+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011696339s,-2.011615276s]) 2026-02-11T23:31:55.231+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.1 from_epoch 122 2026-02-11T23:31:55.231+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:55.231+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:55.231+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:55.231+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011691570s,-2.011604071s]) 2026-02-11T23:31:55.231+0000 7f416cbfe640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6824/2365263801 28 ==== osd_ping(ping_reply e122 up_from 14 ping_stamp 2026-02-11T23:31:55.232996+0000/97.908401489s send_stamp 93.341651917s delta_ub -4.566748142s) ==== 2033+0+0 (crc 0 0 0) 0x68bc780 con 0x48c4000 2026-02-11T23:31:55.231+0000 7f416cbfe640 20 osd.0 122 get_con_osd_cluster to osd.2 from_epoch 122 2026-02-11T23:31:55.231+0000 7f416cbfe640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:55.231+0000 7f416cbfe640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:55.231+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6822/2365263801 28 ==== osd_ping(ping_reply e122 up_from 14 ping_stamp 2026-02-11T23:31:55.232996+0000/97.908401489s send_stamp 93.341674805s delta_ub -4.566733837s) ==== 2033+0+0 (crc 0 0 0) 0x68bcd80 con 0x48cd000 2026-02-11T23:31:55.231+0000 7f416cbfe640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:55.231+0000 7f416cbfe640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566853046s,-4.566748142s]) 2026-02-11T23:31:55.231+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.2 from_epoch 122 2026-02-11T23:31:55.231+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:55.231+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:55.231+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:55.231+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566849232s,-4.566733837s]) 2026-02-11T23:31:55.231+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6832/2699377081 28 ==== osd_ping(ping_reply e122 up_from 19 ping_stamp 2026-02-11T23:31:55.232996+0000/97.908401489s send_stamp 90.689010620s delta_ub -7.219386578s) ==== 2033+0+0 (crc 0 0 0) 0x68bcd80 con 0x48cd400 2026-02-11T23:31:55.231+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.3 from_epoch 122 2026-02-11T23:31:55.231+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:55.231+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:55.231+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:55.231+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6830/2699377081 28 ==== osd_ping(ping_reply e122 up_from 19 ping_stamp 2026-02-11T23:31:55.232996+0000/97.908401489s send_stamp 90.689033508s delta_ub -7.219366074s) ==== 2033+0+0 (crc 0 0 0) 0x68bcf00 con 0x48cd800 2026-02-11T23:31:55.231+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219520569s,-7.219386578s]) 2026-02-11T23:31:55.231+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.3 from_epoch 122 2026-02-11T23:31:55.231+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:55.231+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:55.231+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:55.231+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219508171s,-7.219366074s]) 2026-02-11T23:31:55.231+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6838/3331151400 27 ==== osd_ping(ping_reply e122 up_from 23 ping_stamp 2026-02-11T23:31:55.232996+0000/97.908401489s send_stamp 88.071891785s delta_ub -9.836516380s) ==== 2033+0+0 (crc 0 0 0) 0x68bcf00 con 0x5282000 2026-02-11T23:31:55.231+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.4 from_epoch 122 2026-02-11T23:31:55.231+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:55.231+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:55.231+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:55.231+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6840/3331151400 27 ==== osd_ping(ping_reply e122 up_from 23 ping_stamp 2026-02-11T23:31:55.232996+0000/97.908401489s send_stamp 88.071868896s delta_ub -9.836525917s) ==== 2033+0+0 (crc 0 0 0) 0x68bcd80 con 0x507bc00 2026-02-11T23:31:55.231+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836673737s,-9.836516380s]) 2026-02-11T23:31:55.231+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.4 from_epoch 122 2026-02-11T23:31:55.231+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:55.231+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:55.231+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:55.231+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836706161s,-9.836525917s]) 2026-02-11T23:31:55.277+0000 7f41633eb640 10 osd.0 122 tick 2026-02-11T23:31:55.277+0000 7f41633eb640 20 osd.0 122 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:55.331+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:55.331+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:55.331+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:25.332042+0000) 2026-02-11T23:31:55.700+0000 7f4162bea640 10 osd.0 122 tick_without_osd_lock 2026-02-11T23:31:55.700+0000 7f4162bea640 0 osd.0 122 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:55.700+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:55.700+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.145 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:55.700+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:55.700, recovery is active?:false restrictions:<....> 2026-02-11T23:31:55.700+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:55.700+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:55.700+0000 7f4162bea640 20 osd.0 122 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:55.700+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:55.700+0000 7f4162bea640 20 osd.0 122 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:56.264+0000 7f41633eb640 10 osd.0 122 tick 2026-02-11T23:31:56.264+0000 7f41633eb640 20 osd.0 122 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:56.267+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.4 127.0.0.1:0/3331151400 27 ==== osd_ping(ping e122 up_from 23 ping_stamp 2026-02-11T23:31:56.268799+0000/89.107574463s send_stamp 89.107574463s delta_ub -9.836516380s) ==== 2033+0+0 (crc 0 0 0) 0x68bcf00 con 0x507a800 2026-02-11T23:31:56.267+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.4 127.0.0.1:0/3331151400 27 ==== osd_ping(ping e122 up_from 23 ping_stamp 2026-02-11T23:31:56.268799+0000/89.107574463s send_stamp 89.107574463s delta_ub -9.836516380s) ==== 2033+0+0 (crc 0 0 0) 0x68bcd80 con 0x507a400 2026-02-11T23:31:56.267+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836688995s,-9.836516380s]) 2026-02-11T23:31:56.267+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:56.268799+0000/89.107574463s send_stamp 98.944252014s delta_ub 9.836688995s) -- 0x68bc000 con 0x507a800 2026-02-11T23:31:56.267+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.4 from_epoch 122 2026-02-11T23:31:56.267+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:56.267+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:56.267+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:56.267+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836707115s,-9.836516380s]) 2026-02-11T23:31:56.267+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:56.268799+0000/89.107574463s send_stamp 98.944267273s delta_ub 9.836707115s) -- 0x68bca80 con 0x507a400 2026-02-11T23:31:56.267+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.4 from_epoch 122 2026-02-11T23:31:56.267+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:56.267+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:56.267+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:56.298+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.2 127.0.0.1:0/2365263801 28 ==== osd_ping(ping e122 up_from 14 ping_stamp 2026-02-11T23:31:56.299139+0000/94.407737732s send_stamp 94.407737732s delta_ub -4.566733837s) ==== 2033+0+0 (crc 0 0 0) 0x68bc780 con 0x4641000 2026-02-11T23:31:56.298+0000 7f416cbfe640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566848278s,-4.566733837s]) 2026-02-11T23:31:56.298+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:56.299139+0000/94.407737732s send_stamp 98.974578857s delta_ub 4.566848278s) -- 0x68b0f00 con 0x4641000 2026-02-11T23:31:56.298+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.2 127.0.0.1:0/2365263801 28 ==== osd_ping(ping e122 up_from 14 ping_stamp 2026-02-11T23:31:56.299139+0000/94.407737732s send_stamp 94.407737732s delta_ub -4.566733837s) ==== 2033+0+0 (crc 0 0 0) 0x68bc000 con 0x48cc000 2026-02-11T23:31:56.298+0000 7f416cbfe640 20 osd.0 122 get_con_osd_cluster to osd.2 from_epoch 122 2026-02-11T23:31:56.298+0000 7f416cbfe640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:56.298+0000 7f416cbfe640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:56.298+0000 7f416cbfe640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:56.298+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566861153s,-4.566733837s]) 2026-02-11T23:31:56.298+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:56.299139+0000/94.407737732s send_stamp 98.974594116s delta_ub 4.566861153s) -- 0x68bcf00 con 0x48cc000 2026-02-11T23:31:56.298+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.2 from_epoch 122 2026-02-11T23:31:56.298+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:56.298+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:56.298+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:56.331+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:56.331+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:56.331+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:26.332104+0000) 2026-02-11T23:31:56.690+0000 7f4162bea640 10 osd.0 122 tick_without_osd_lock 2026-02-11T23:31:56.690+0000 7f4162bea640 0 osd.0 122 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:56.690+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:56.690+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.145 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:56.690+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:56.691, recovery is active?:false restrictions:<....> 2026-02-11T23:31:56.690+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:56.690+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:56.690+0000 7f4162bea640 20 osd.0 122 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:56.690+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:56.690+0000 7f4162bea640 20 osd.0 122 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:57.300+0000 7f41633eb640 10 osd.0 122 tick 2026-02-11T23:31:57.300+0000 7f41633eb640 20 osd.0 122 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:57.331+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:57.331+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:57.331+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:27.332166+0000) 2026-02-11T23:31:57.665+0000 7f4162bea640 10 osd.0 122 tick_without_osd_lock 2026-02-11T23:31:57.665+0000 7f4162bea640 0 osd.0 122 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:57.665+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:57.665+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.137 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:57.665+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:57.666, recovery is active?:false restrictions:<....> 2026-02-11T23:31:57.665+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:57.665+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:57.665+0000 7f4162bea640 20 osd.0 122 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:57.665+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:57.665+0000 7f4162bea640 20 osd.0 122 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:58.208+0000 7f415b6ff640 20 osd.0 122 reports for 0 queries 2026-02-11T23:31:58.208+0000 7f415b6ff640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] -- mgrreport(unknown.0 +0-0 packed 3294 daemon_metrics=2) -- 0x68ace00 con 0x4aca800 2026-02-11T23:31:58.208+0000 7f415b6ff640 15 osd.0 122 collect_pg_stats 2026-02-11T23:31:58.208+0000 7f415b6ff640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] -- pg_stats(0 pgs seq 21474836505 v 0) -- 0x68aec00 con 0x4aca800 2026-02-11T23:31:58.296+0000 7f416c3fd640 1 --2- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] >> conn(0x5957400 0x527fb80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).accept 2026-02-11T23:31:58.296+0000 7f416cbfe640 1 --2- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] >> conn(0x5957400 0x527fb80 unknown :-1 s=BANNER_ACCEPTING pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:31:58.296+0000 7f416cbfe640 10 osd.0 122 ms_handle_fast_authentication new session 0x5954700 con 0x5957400 entity client.admin addr 2026-02-11T23:31:58.296+0000 7f416cbfe640 1 --2- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] >> 127.0.0.1:0/1903321818 conn(0x5957400 0x527fb80 crc :-1 s=READY pgs=3 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=client.7931 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-11T23:31:58.296+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== client.7931 127.0.0.1:0/1903321818 1 ==== command(tid 1: {"prefix": "get_command_descriptions"}) ==== 62+0+0 (crc 0 0 0) 0x672de40 con 0x5957400 2026-02-11T23:31:58.296+0000 7f415cf02640 20 osd.0 122 OSD::ms_dispatch: command(tid 1: {"prefix": "get_command_descriptions"}) 2026-02-11T23:31:58.296+0000 7f415cf02640 20 osd.0 122 _dispatch 0x672de40 command(tid 1: {"prefix": "get_command_descriptions"}) 2026-02-11T23:31:58.296+0000 7f416bbfc640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> 127.0.0.1:0/1903321818 -- command_reply(tid 1: 0 ) -- 0x37002c0 con 0x5957400 2026-02-11T23:31:58.304+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== client.7931 127.0.0.1:0/1903321818 2 ==== command(tid 2: {"prefix": "flush_pg_stats"}) ==== 52+0+0 (crc 0 0 0) 0x37002c0 con 0x5957400 2026-02-11T23:31:58.304+0000 7f415cf02640 20 osd.0 122 OSD::ms_dispatch: command(tid 2: {"prefix": "flush_pg_stats"}) 2026-02-11T23:31:58.304+0000 7f415cf02640 20 osd.0 122 _dispatch 0x37002c0 command(tid 2: {"prefix": "flush_pg_stats"}) 2026-02-11T23:31:58.304+0000 7f416bbfc640 15 osd.0 122 collect_pg_stats 2026-02-11T23:31:58.304+0000 7f416bbfc640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] -- pg_stats(0 pgs seq 21474836506 v 0) -- 0x4aa0300 con 0x4aca800 2026-02-11T23:31:58.304+0000 7f416bbfc640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> 127.0.0.1:0/1903321818 -- command_reply(tid 2: 0 ) -- 0x672de40 con 0x5957400 2026-02-11T23:31:58.308+0000 7f41633eb640 10 osd.0 122 tick 2026-02-11T23:31:58.308+0000 7f41633eb640 20 osd.0 122 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:58.310+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== client.7931 127.0.0.1:0/1903321818 3 ==== command(tid 3: {"prefix": "flush_pg_stats"}) ==== 52+0+0 (crc 0 0 0) 0x672de40 con 0x5957400 2026-02-11T23:31:58.310+0000 7f415cf02640 20 osd.0 122 OSD::ms_dispatch: command(tid 3: {"prefix": "flush_pg_stats"}) 2026-02-11T23:31:58.310+0000 7f415cf02640 20 osd.0 122 _dispatch 0x672de40 command(tid 3: {"prefix": "flush_pg_stats"}) 2026-02-11T23:31:58.310+0000 7f416bbfc640 15 osd.0 122 collect_pg_stats 2026-02-11T23:31:58.310+0000 7f416bbfc640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] -- pg_stats(0 pgs seq 21474836507 v 0) -- 0x4aa0600 con 0x4aca800 2026-02-11T23:31:58.310+0000 7f416bbfc640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> 127.0.0.1:0/1903321818 -- command_reply(tid 3: 0 ) -- 0x37002c0 con 0x5957400 2026-02-11T23:31:58.311+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] >> 127.0.0.1:0/1903321818 conn(0x5957400 msgr2=0x527fb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 80 2026-02-11T23:31:58.311+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] >> 127.0.0.1:0/1903321818 conn(0x5957400 msgr2=0x527fb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-11T23:31:58.311+0000 7f416cbfe640 1 --2- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] >> 127.0.0.1:0/1903321818 conn(0x5957400 0x527fb80 crc :-1 s=READY pgs=3 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-11T23:31:58.311+0000 7f416cbfe640 1 --2- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] >> 127.0.0.1:0/1903321818 conn(0x5957400 0x527fb80 crc :-1 s=READY pgs=3 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-11T23:31:58.311+0000 7f415cf02640 2 osd.0 122 ms_handle_reset con 0x5957400 session 0x5954700 2026-02-11T23:31:58.330+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:58.330+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:58.330+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:28.332224+0000) 2026-02-11T23:31:58.625+0000 7f4162bea640 10 osd.0 122 tick_without_osd_lock 2026-02-11T23:31:58.625+0000 7f4162bea640 0 osd.0 122 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:58.625+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:58.625+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:58.625+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:58.625+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.137 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:58.625+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:58.627, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:58.625+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:58.625+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:58.625+0000 7f4162bea640 20 osd.0 122 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:58.625+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:58.625+0000 7f4162bea640 20 osd.0 122 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:58.731+0000 7f4147983640 5 osd.0 122 heartbeat osd_stat(store_statfs(0x0/0x0/0x384000, data 0x53a3e5/0x5e6000, compress 0x0/0x0/0x0, omap 0xa403, meta 0x1a45bfd), peers [1,2,3,4] op hist []) 2026-02-11T23:31:58.731+0000 7f4147983640 20 osd.0 122 check_full_status cur ratio 1, physical ratio 1, new state failsafe 2026-02-11T23:31:58.731+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6816/3043114013,v1:127.0.0.1:6817/3043114013] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:58.733175+0000/101.408576965s send_stamp 101.408576965s delta_ub 2.011691570s) -- 0x68bd080 con 0x48c4400 2026-02-11T23:31:58.731+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6814/3043114013,v1:127.0.0.1:6815/3043114013] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:58.733175+0000/101.408576965s send_stamp 101.408576965s delta_ub 2.011691570s) -- 0x68bd200 con 0x4640c00 2026-02-11T23:31:58.731+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6824/2365263801,v1:127.0.0.1:6825/2365263801] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:58.733175+0000/101.408576965s send_stamp 101.408576965s delta_ub 4.566861153s) -- 0x68bd380 con 0x48c4000 2026-02-11T23:31:58.731+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6822/2365263801,v1:127.0.0.1:6823/2365263801] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:58.733175+0000/101.408576965s send_stamp 101.408576965s delta_ub 4.566861153s) -- 0x68bd500 con 0x48cd000 2026-02-11T23:31:58.731+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6832/2699377081,v1:127.0.0.1:6833/2699377081] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:58.733175+0000/101.408576965s send_stamp 101.408576965s delta_ub 7.219508171s) -- 0x68bd680 con 0x48cd400 2026-02-11T23:31:58.731+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6830/2699377081,v1:127.0.0.1:6831/2699377081] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:58.733175+0000/101.408576965s send_stamp 101.408576965s delta_ub 7.219508171s) -- 0x68bd800 con 0x48cd800 2026-02-11T23:31:58.731+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6840/3331151400,v1:127.0.0.1:6841/3331151400] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:58.733175+0000/101.408576965s send_stamp 101.408576965s delta_ub 9.836707115s) -- 0x68bd980 con 0x507bc00 2026-02-11T23:31:58.731+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6838/3331151400,v1:127.0.0.1:6839/3331151400] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:31:58.733175+0000/101.408576965s send_stamp 101.408576965s delta_ub 9.836707115s) -- 0x68bdb00 con 0x5282000 2026-02-11T23:31:58.732+0000 7f416cbfe640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6824/2365263801 29 ==== osd_ping(ping_reply e122 up_from 14 ping_stamp 2026-02-11T23:31:58.733175+0000/101.408576965s send_stamp 96.841880798s delta_ub -4.566699028s) ==== 2033+0+0 (crc 0 0 0) 0x68b1b00 con 0x48c4000 2026-02-11T23:31:58.732+0000 7f416cbfe640 20 osd.0 122 get_con_osd_cluster to osd.2 from_epoch 122 2026-02-11T23:31:58.732+0000 7f416cbfe640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:58.732+0000 7f416cbfe640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:58.732+0000 7f416cbfe640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:58.732+0000 7f416cbfe640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566889286s,-4.566699028s]) 2026-02-11T23:31:58.732+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6814/3043114013 30 ==== osd_ping(ping_reply e122 up_from 10 ping_stamp 2026-02-11T23:31:58.733175+0000/101.408576965s send_stamp 99.397041321s delta_ub -2.011536837s) ==== 2033+0+0 (crc 0 0 0) 0x68bdb00 con 0x4640c00 2026-02-11T23:31:58.732+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.1 from_epoch 122 2026-02-11T23:31:58.732+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:58.732+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:58.732+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:58.732+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011742830s,-2.011536837s]) 2026-02-11T23:31:58.732+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6816/3043114013 30 ==== osd_ping(ping_reply e122 up_from 10 ping_stamp 2026-02-11T23:31:58.733175+0000/101.408576965s send_stamp 99.397018433s delta_ub -2.011564016s) ==== 2033+0+0 (crc 0 0 0) 0x68bd980 con 0x48c4400 2026-02-11T23:31:58.732+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6830/2699377081 29 ==== osd_ping(ping_reply e122 up_from 19 ping_stamp 2026-02-11T23:31:58.733175+0000/101.408576965s send_stamp 94.189254761s delta_ub -7.219324589s) ==== 2033+0+0 (crc 0 0 0) 0x68bdb00 con 0x48cd800 2026-02-11T23:31:58.732+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.1 from_epoch 122 2026-02-11T23:31:58.732+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:58.732+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:58.732+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:58.732+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011782885s,-2.011564016s]) 2026-02-11T23:31:58.732+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.3 from_epoch 122 2026-02-11T23:31:58.732+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:58.732+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:58.732+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:58.732+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219567776s,-7.219324589s]) 2026-02-11T23:31:58.732+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6832/2699377081 29 ==== osd_ping(ping_reply e122 up_from 19 ping_stamp 2026-02-11T23:31:58.733175+0000/101.408576965s send_stamp 94.189231873s delta_ub -7.219350338s) ==== 2033+0+0 (crc 0 0 0) 0x68bd980 con 0x48cd400 2026-02-11T23:31:58.732+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.3 from_epoch 122 2026-02-11T23:31:58.732+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:58.732+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:58.732+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:58.732+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219607353s,-7.219350338s]) 2026-02-11T23:31:58.732+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6838/3331151400 28 ==== osd_ping(ping_reply e122 up_from 23 ping_stamp 2026-02-11T23:31:58.733175+0000/101.408576965s send_stamp 91.572090149s delta_ub -9.836491585s) ==== 2033+0+0 (crc 0 0 0) 0x68bdb00 con 0x5282000 2026-02-11T23:31:58.732+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.4 from_epoch 122 2026-02-11T23:31:58.732+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:58.732+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:58.732+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:58.732+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836754799s,-9.836491585s]) 2026-02-11T23:31:58.732+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6840/3331151400 28 ==== osd_ping(ping_reply e122 up_from 23 ping_stamp 2026-02-11T23:31:58.733175+0000/101.408576965s send_stamp 91.572059631s delta_ub -9.836518288s) ==== 2033+0+0 (crc 0 0 0) 0x68bd980 con 0x507bc00 2026-02-11T23:31:58.732+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.4 from_epoch 122 2026-02-11T23:31:58.732+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:58.732+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:58.732+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:58.732+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836786270s,-9.836518288s]) 2026-02-11T23:31:58.732+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6822/2365263801 29 ==== osd_ping(ping_reply e122 up_from 14 ping_stamp 2026-02-11T23:31:58.733175+0000/101.408576965s send_stamp 96.841918945s delta_ub -4.566661358s) ==== 2033+0+0 (crc 0 0 0) 0x68bd980 con 0x48cd000 2026-02-11T23:31:58.732+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.2 from_epoch 122 2026-02-11T23:31:58.732+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:58.732+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:58.732+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:58.732+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566941261s,-4.566661358s]) 2026-02-11T23:31:59.043+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.1 127.0.0.1:0/3043114013 29 ==== osd_ping(ping e122 up_from 10 ping_stamp 2026-02-11T23:31:59.044182+0000/99.707931519s send_stamp 99.707931519s delta_ub -2.011536837s) ==== 2033+0+0 (crc 0 0 0) 0x68bdb00 con 0x4aca400 2026-02-11T23:31:59.043+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.1 127.0.0.1:0/3043114013 29 ==== osd_ping(ping e122 up_from 10 ping_stamp 2026-02-11T23:31:59.044182+0000/99.707931519s send_stamp 99.707931519s delta_ub -2.011536837s) ==== 2033+0+0 (crc 0 0 0) 0x68b1b00 con 0x4aca000 2026-02-11T23:31:59.043+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011763334s,-2.011536837s]) 2026-02-11T23:31:59.043+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:59.044182+0000/99.707931519s send_stamp 101.719688416s delta_ub 2.011763334s) -- 0x68bd800 con 0x4aca400 2026-02-11T23:31:59.043+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.1 from_epoch 122 2026-02-11T23:31:59.043+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:59.043+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:59.043+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:59.043+0000 7f416cbfe640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011866570s,-2.011536837s]) 2026-02-11T23:31:59.043+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:31:59.044182+0000/99.707931519s send_stamp 101.719795227s delta_ub 2.011866570s) -- 0x68b0300 con 0x4aca000 2026-02-11T23:31:59.043+0000 7f416cbfe640 20 osd.0 122 get_con_osd_cluster to osd.1 from_epoch 122 2026-02-11T23:31:59.043+0000 7f416cbfe640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:31:59.043+0000 7f416cbfe640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:31:59.043+0000 7f416cbfe640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:31:59.283+0000 7f41633eb640 10 osd.0 122 tick 2026-02-11T23:31:59.283+0000 7f41633eb640 20 osd.0 122 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:31:59.331+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:31:59.331+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:31:59.331+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:29.332317+0000) 2026-02-11T23:31:59.659+0000 7f4162bea640 10 osd.0 122 tick_without_osd_lock 2026-02-11T23:31:59.659+0000 7f4162bea640 0 osd.0 122 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:59.659+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:59.659+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.137 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:59.659+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:59.659, recovery is active?:false restrictions:<....> 2026-02-11T23:31:59.659+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:59.659+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:59.659+0000 7f4162bea640 20 osd.0 122 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:59.659+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:59.659+0000 7f4162bea640 20 osd.0 122 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:32:00.033+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.1 v2:127.0.0.1:6812/3043114013 1249 ==== pg_lease(1.0 pg_lease(ru 108.698791504s ub 116.698928833s int 16.000000000s) e122/122) ==== 58+0+0 (unknown 0 0 0) 0x68bd980 con 0x4acb000 2026-02-11T23:32:00.033+0000 7f416abfa640 15 osd.0 122 enqueue_peering_evt 1.0 epoch_sent: 122 epoch_requested: 122 MLease epoch 122 from osd.1 pg_lease(ru 108.698791504s ub 116.698928833s int 16.000000000s) 2026-02-11T23:32:00.033+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 122 epoch_requested: 122 MLease epoch 122 from osd.1 pg_lease(ru 108.698791504s ub 116.698928833s int 16.000000000s)) class_id 2 prio 255 cost 10 e122) 2026-02-11T23:32:00.033+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:32:00.033+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 122 epoch_requested: 122 MLease epoch 122 from osd.1 pg_lease(ru 108.698791504s ub 116.698928833s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e122) queued 2026-02-11T23:32:00.033+0000 7f414f993640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:32:00.034+0000 7f414f993640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 122 epoch_requested: 122 MLease epoch 122 from osd.1 pg_lease(ru 108.698791504s ub 116.698928833s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e122) pg 0x4ef0000 2026-02-11T23:32:00.034+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:32:00.034+0000 7f414f993640 10 osd.0 pg_epoch: 122 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/122 sis=118) [1,0] r=1 lpr=118 crt=111'600 active mbc={}] do_peering_event: epoch_sent: 122 epoch_requested: 122 MLease epoch 122 from osd.1 pg_lease(ru 108.698791504s ub 116.698928833s int 16.000000000s) 2026-02-11T23:32:00.034+0000 7f414f993640 10 osd.0 pg_epoch: 122 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/122 sis=118) [1,0] r=1 lpr=118 crt=111'600 active mbc={}] PeeringState::proc_lease pg_lease(ru 108.698791504s ub 116.698928833s int 16.000000000s) 2026-02-11T23:32:00.034+0000 7f414f993640 20 osd.0 pg_epoch: 122 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/122 sis=118) [1,0] r=1 lpr=118 crt=111'600 active mbc={}] PeeringState::proc_lease peer_clock_delta_ub -2.011536837s -> ru 110.710327148s 2026-02-11T23:32:00.034+0000 7f414f993640 20 osd.0 pg_epoch: 122 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/122 sis=118) [1,0] r=1 lpr=118 crt=111'600 active mbc={}] PeeringState::proc_lease readable_until now 110.710327148s 2026-02-11T23:32:00.034+0000 7f414f993640 20 osd.0 pg_epoch: 122 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/122 sis=118) [1,0] r=1 lpr=118 crt=111'600 active mbc={}] PeeringState::proc_lease peer_clock_delta_lb -2.011866570s -> ruub 118.710792542s 2026-02-11T23:32:00.034+0000 7f414f993640 20 osd.0 pg_epoch: 122 pg[1.0( v 111'600 (111'590,111'600] local-lis/les=118/119 n=600 ec=27/27 lis/c=118/118 les/c/f=119/119/122 sis=118) [1,0] r=1 lpr=118 crt=111'600 active mbc={}] PeeringState::proc_lease readable_until_ub now 118.710792542s 2026-02-11T23:32:00.034+0000 7f414f993640 20 osd.0 122 get_con_osd_cluster to osd.1 from_epoch 122 2026-02-11T23:32:00.034+0000 7f414f993640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:00.034+0000 7f414f993640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:00.034+0000 7f414f993640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6812/3043114013,v1:127.0.0.1:6813/3043114013] -- pg_lease_ack(1.0 pg_lease_ack(ruub 116.698928833s) e122/122) -- 0x68aa580 con 0x4acb000 2026-02-11T23:32:00.034+0000 7f414f993640 10 log is not dirty 2026-02-11T23:32:00.034+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:32:00.250+0000 7f41633eb640 10 osd.0 122 tick 2026-02-11T23:32:00.250+0000 7f41633eb640 10 osd.0 122 maybe_update_heartbeat_peers forcing update after 20.954588 seconds 2026-02-11T23:32:00.250+0000 7f41633eb640 10 osd.0 122 reset_heartbeat_peers 2026-02-11T23:32:00.250+0000 7f41633eb640 10 osd.0 122 maybe_update_heartbeat_peers updating 2026-02-11T23:32:00.250+0000 7f41633eb640 10 osd.0 122 adding neighbor peer osd.1 2026-02-11T23:32:00.250+0000 7f41633eb640 10 osd.0 122 adding neighbor peer osd.4 2026-02-11T23:32:00.250+0000 7f41633eb640 10 osd.0 122 maybe_update_heartbeat_peers 4 peers, extras 1,2,3,4 2026-02-11T23:32:00.250+0000 7f41633eb640 20 osd.0 122 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:32:00.332+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:32:00.332+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:32:00.332+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:30.332389+0000) 2026-02-11T23:32:00.332+0000 7f415bf00640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-11T23:32:00.351+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.3 127.0.0.1:0/2699377081 26 ==== osd_ping(ping e122 up_from 19 ping_stamp 2026-02-11T23:32:00.351622+0000/95.807563782s send_stamp 95.807563782s delta_ub -7.219324589s) ==== 2033+0+0 (crc 0 0 0) 0x68b0300 con 0x48cc800 2026-02-11T23:32:00.351+0000 7f416cbfe640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219573021s,-7.219324589s]) 2026-02-11T23:32:00.351+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.3 127.0.0.1:0/2699377081 26 ==== osd_ping(ping e122 up_from 19 ping_stamp 2026-02-11T23:32:00.351622+0000/95.807563782s send_stamp 95.807563782s delta_ub -7.219324589s) ==== 2033+0+0 (crc 0 0 0) 0x68bd800 con 0x48cc400 2026-02-11T23:32:00.351+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:32:00.351622+0000/95.807563782s send_stamp 103.027137756s delta_ub 7.219573021s) -- 0x68b1b00 con 0x48cc800 2026-02-11T23:32:00.351+0000 7f416cbfe640 20 osd.0 122 get_con_osd_cluster to osd.3 from_epoch 122 2026-02-11T23:32:00.351+0000 7f416cbfe640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:00.351+0000 7f416cbfe640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:00.351+0000 7f416cbfe640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:32:00.351+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219598770s,-7.219324589s]) 2026-02-11T23:32:00.351+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:32:00.351622+0000/95.807563782s send_stamp 103.027160645s delta_ub 7.219598770s) -- 0x68bdb00 con 0x48cc400 2026-02-11T23:32:00.351+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.3 from_epoch 122 2026-02-11T23:32:00.351+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:00.351+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:00.351+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:32:00.398+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.2 127.0.0.1:0/2365263801 29 ==== osd_ping(ping e122 up_from 14 ping_stamp 2026-02-11T23:32:00.399300+0000/98.507896423s send_stamp 98.507896423s delta_ub -4.566661358s) ==== 2033+0+0 (crc 0 0 0) 0x68bdb00 con 0x48cc000 2026-02-11T23:32:00.398+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566884518s,-4.566661358s]) 2026-02-11T23:32:00.398+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:32:00.399300+0000/98.507896423s send_stamp 103.074783325s delta_ub 4.566884518s) -- 0x68bd800 con 0x48cc000 2026-02-11T23:32:00.398+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.2 127.0.0.1:0/2365263801 29 ==== osd_ping(ping e122 up_from 14 ping_stamp 2026-02-11T23:32:00.399300+0000/98.507896423s send_stamp 98.507896423s delta_ub -4.566661358s) ==== 2033+0+0 (crc 0 0 0) 0x68b1b00 con 0x4641000 2026-02-11T23:32:00.398+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.2 from_epoch 122 2026-02-11T23:32:00.398+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:00.398+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:00.398+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:32:00.398+0000 7f416cbfe640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566896915s,-4.566661358s]) 2026-02-11T23:32:00.398+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:32:00.399300+0000/98.507896423s send_stamp 103.074790955s delta_ub 4.566896915s) -- 0x68b0300 con 0x4641000 2026-02-11T23:32:00.398+0000 7f416cbfe640 20 osd.0 122 get_con_osd_cluster to osd.2 from_epoch 122 2026-02-11T23:32:00.398+0000 7f416cbfe640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:00.398+0000 7f416cbfe640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:00.398+0000 7f416cbfe640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:32:00.438+0000 7f416abfa640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] <== osd.4 v2:127.0.0.1:6836/3331151400 1252 ==== pg_lease(4.0 pg_lease(ru 101.277824402s ub 109.277961731s int 16.000000000s) e122/122) ==== 58+0+0 (unknown 0 0 0) 0x68bd980 con 0x507ac00 2026-02-11T23:32:00.438+0000 7f416abfa640 15 osd.0 122 enqueue_peering_evt 4.0 epoch_sent: 122 epoch_requested: 122 MLease epoch 122 from osd.4 pg_lease(ru 101.277824402s ub 109.277961731s int 16.000000000s) 2026-02-11T23:32:00.438+0000 7f416abfa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 122 epoch_requested: 122 MLease epoch 122 from osd.4 pg_lease(ru 101.277824402s ub 109.277961731s int 16.000000000s)) class_id 2 prio 255 cost 10 e122) 2026-02-11T23:32:00.438+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:32:00.438+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 122 epoch_requested: 122 MLease epoch 122 from osd.4 pg_lease(ru 101.277824402s ub 109.277961731s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e122) queued 2026-02-11T23:32:00.438+0000 7f414b98b640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:32:00.438+0000 7f414b98b640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 122 epoch_requested: 122 MLease epoch 122 from osd.4 pg_lease(ru 101.277824402s ub 109.277961731s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e122) pg 0x5978000 2026-02-11T23:32:00.438+0000 7f414f993640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:32:00.438+0000 7f414b98b640 10 osd.0 pg_epoch: 122 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/120 les/c/f=121/121/122 sis=120) [4,0] r=1 lpr=120 crt=111'600 active mbc={}] do_peering_event: epoch_sent: 122 epoch_requested: 122 MLease epoch 122 from osd.4 pg_lease(ru 101.277824402s ub 109.277961731s int 16.000000000s) 2026-02-11T23:32:00.438+0000 7f414b98b640 10 osd.0 pg_epoch: 122 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/120 les/c/f=121/121/122 sis=120) [4,0] r=1 lpr=120 crt=111'600 active mbc={}] PeeringState::proc_lease pg_lease(ru 101.277824402s ub 109.277961731s int 16.000000000s) 2026-02-11T23:32:00.438+0000 7f414b98b640 20 osd.0 pg_epoch: 122 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/120 les/c/f=121/121/122 sis=120) [4,0] r=1 lpr=120 crt=111'600 active mbc={}] PeeringState::proc_lease peer_clock_delta_ub -9.836518288s -> ru 111.114349365s 2026-02-11T23:32:00.438+0000 7f414b98b640 20 osd.0 pg_epoch: 122 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/120 les/c/f=121/121/122 sis=120) [4,0] r=1 lpr=120 crt=111'600 active mbc={}] PeeringState::proc_lease readable_until now 111.114349365s 2026-02-11T23:32:00.438+0000 7f414b98b640 20 osd.0 pg_epoch: 122 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/120 les/c/f=121/121/122 sis=120) [4,0] r=1 lpr=120 crt=111'600 active mbc={}] PeeringState::proc_lease peer_clock_delta_lb -9.836786270s -> ruub 119.114753723s 2026-02-11T23:32:00.438+0000 7f414b98b640 20 osd.0 pg_epoch: 122 pg[4.0( v 111'600 (111'590,111'600] local-lis/les=120/121 n=600 ec=49/49 lis/c=120/120 les/c/f=121/121/122 sis=120) [4,0] r=1 lpr=120 crt=111'600 active mbc={}] PeeringState::proc_lease readable_until_ub now 119.114753723s 2026-02-11T23:32:00.438+0000 7f414b98b640 20 osd.0 122 get_con_osd_cluster to osd.4 from_epoch 122 2026-02-11T23:32:00.438+0000 7f414b98b640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:00.438+0000 7f414b98b640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:00.438+0000 7f414b98b640 1 -- [v2:127.0.0.1:6802/3294232400,v1:127.0.0.1:6803/3294232400] --> [v2:127.0.0.1:6836/3331151400,v1:127.0.0.1:6837/3331151400] -- pg_lease_ack(4.0 pg_lease_ack(ruub 109.277961731s) e122/122) -- 0x68aa160 con 0x507ac00 2026-02-11T23:32:00.438+0000 7f414b98b640 10 log is not dirty 2026-02-11T23:32:00.438+0000 7f414b98b640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:32:00.671+0000 7f4162bea640 10 osd.0 122 tick_without_osd_lock 2026-02-11T23:32:00.671+0000 7f4162bea640 0 osd.0 122 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:32:00.671+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:32:00.671+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.137 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:32:00.671+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:32:00.672, recovery is active?:false restrictions:<....> 2026-02-11T23:32:00.671+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:32:00.671+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:32:00.671+0000 7f4162bea640 20 osd.0 122 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:32:00.671+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:32:00.671+0000 7f4162bea640 20 osd.0 122 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:32:01.252+0000 7f41633eb640 10 osd.0 122 tick 2026-02-11T23:32:01.252+0000 7f41633eb640 20 osd.0 122 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:32:01.331+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:32:01.331+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:32:01.331+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:31.332470+0000) 2026-02-11T23:32:01.343+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.1 127.0.0.1:0/3043114013 30 ==== osd_ping(ping e122 up_from 10 ping_stamp 2026-02-11T23:32:01.344395+0000/102.008140564s send_stamp 102.008140564s delta_ub -2.011568308s) ==== 2033+0+0 (crc 0 0 0) 0x68b0300 con 0x4aca000 2026-02-11T23:32:01.343+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.1 127.0.0.1:0/3043114013 30 ==== osd_ping(ping e122 up_from 10 ping_stamp 2026-02-11T23:32:01.344395+0000/102.008140564s send_stamp 102.008140564s delta_ub -2.011568308s) ==== 2033+0+0 (crc 0 0 0) 0x68bd800 con 0x4aca400 2026-02-11T23:32:01.343+0000 7f416cbfe640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011759043s,-2.011568308s]) 2026-02-11T23:32:01.343+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:32:01.344395+0000/102.008140564s send_stamp 104.019905090s delta_ub 2.011759043s) -- 0x68b1b00 con 0x4aca000 2026-02-11T23:32:01.343+0000 7f416cbfe640 20 osd.0 122 get_con_osd_cluster to osd.1 from_epoch 122 2026-02-11T23:32:01.343+0000 7f416cbfe640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:01.343+0000 7f416cbfe640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:01.343+0000 7f416cbfe640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:32:01.343+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011786461s,-2.011568308s]) 2026-02-11T23:32:01.343+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:32:01.344395+0000/102.008140564s send_stamp 104.019927979s delta_ub 2.011786461s) -- 0x68bdb00 con 0x4aca400 2026-02-11T23:32:01.343+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.1 from_epoch 122 2026-02-11T23:32:01.343+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:01.343+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:01.343+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:32:01.567+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.4 127.0.0.1:0/3331151400 28 ==== osd_ping(ping e122 up_from 23 ping_stamp 2026-02-11T23:32:01.568948+0000/94.407714844s send_stamp 94.407714844s delta_ub -9.836491585s) ==== 2033+0+0 (crc 0 0 0) 0x68bd980 con 0x507a400 2026-02-11T23:32:01.567+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.4 127.0.0.1:0/3331151400 28 ==== osd_ping(ping e122 up_from 23 ping_stamp 2026-02-11T23:32:01.568948+0000/94.407714844s send_stamp 94.407714844s delta_ub -9.836491585s) ==== 2033+0+0 (crc 0 0 0) 0x68bdb00 con 0x507a800 2026-02-11T23:32:01.567+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836747169s,-9.836491585s]) 2026-02-11T23:32:01.567+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:32:01.568948+0000/94.407714844s send_stamp 104.244461060s delta_ub 9.836747169s) -- 0x68bd680 con 0x507a400 2026-02-11T23:32:01.567+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.4 from_epoch 122 2026-02-11T23:32:01.567+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:01.567+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:01.567+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:32:01.567+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836770058s,-9.836491585s]) 2026-02-11T23:32:01.567+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:32:01.568948+0000/94.407714844s send_stamp 104.244483948s delta_ub 9.836770058s) -- 0x68bd800 con 0x507a800 2026-02-11T23:32:01.567+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.4 from_epoch 122 2026-02-11T23:32:01.567+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:01.567+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:01.567+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:32:01.659+0000 7f4162bea640 10 osd.0 122 tick_without_osd_lock 2026-02-11T23:32:01.659+0000 7f4162bea640 0 osd.0 122 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:32:01.659+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:32:01.659+0000 7f4162bea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:32:01.659+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:32:01.659+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.137 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:32:01.659+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:32:01.659, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:32:01.659+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:32:01.659+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:32:01.659+0000 7f4162bea640 20 osd.0 122 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:32:01.659+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:32:01.659+0000 7f4162bea640 20 osd.0 122 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:32:02.248+0000 7f41633eb640 10 osd.0 122 tick 2026-02-11T23:32:02.248+0000 7f41633eb640 20 osd.0 122 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:32:02.332+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:32:02.332+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:32:02.332+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:32.332545+0000) 2026-02-11T23:32:02.651+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.3 127.0.0.1:0/2699377081 27 ==== osd_ping(ping e122 up_from 19 ping_stamp 2026-02-11T23:32:02.651847+0000/98.107788086s send_stamp 98.107788086s delta_ub -7.219387531s) ==== 2033+0+0 (crc 0 0 0) 0x68bd800 con 0x48cc400 2026-02-11T23:32:02.651+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.3 127.0.0.1:0/2699377081 27 ==== osd_ping(ping e122 up_from 19 ping_stamp 2026-02-11T23:32:02.651847+0000/98.107788086s send_stamp 98.107788086s delta_ub -7.219387531s) ==== 2033+0+0 (crc 0 0 0) 0x68b1b00 con 0x48cc800 2026-02-11T23:32:02.651+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219566822s,-7.219387531s]) 2026-02-11T23:32:02.651+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:32:02.651847+0000/98.107788086s send_stamp 105.327354431s delta_ub 7.219566822s) -- 0x68bdb00 con 0x48cc400 2026-02-11T23:32:02.651+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.3 from_epoch 122 2026-02-11T23:32:02.651+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:02.651+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:02.651+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:32:02.651+0000 7f416cbfe640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219592571s,-7.219387531s]) 2026-02-11T23:32:02.651+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:32:02.651847+0000/98.107788086s send_stamp 105.327377319s delta_ub 7.219592571s) -- 0x68b0300 con 0x48cc800 2026-02-11T23:32:02.651+0000 7f416cbfe640 20 osd.0 122 get_con_osd_cluster to osd.3 from_epoch 122 2026-02-11T23:32:02.651+0000 7f416cbfe640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:02.651+0000 7f416cbfe640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:02.651+0000 7f416cbfe640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:32:02.681+0000 7f4162bea640 10 osd.0 122 tick_without_osd_lock 2026-02-11T23:32:02.681+0000 7f4162bea640 0 osd.0 122 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:32:02.681+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:32:02.681+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.130 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:32:02.681+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:32:02.681, recovery is active?:false restrictions:<....> 2026-02-11T23:32:02.681+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:32:02.681+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:32:02.681+0000 7f4162bea640 20 osd.0 122 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:32:02.681+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:32:02.681+0000 7f4162bea640 20 osd.0 122 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:32:02.699+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.2 127.0.0.1:0/2365263801 30 ==== osd_ping(ping e122 up_from 14 ping_stamp 2026-02-11T23:32:02.699507+0000/100.808105469s send_stamp 100.808105469s delta_ub -4.566764355s) ==== 2033+0+0 (crc 0 0 0) 0x68bdb00 con 0x48cc000 2026-02-11T23:32:02.699+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.2 127.0.0.1:0/2365263801 30 ==== osd_ping(ping e122 up_from 14 ping_stamp 2026-02-11T23:32:02.699507+0000/100.808105469s send_stamp 100.808105469s delta_ub -4.566764355s) ==== 2033+0+0 (crc 0 0 0) 0x68b0300 con 0x4641000 2026-02-11T23:32:02.699+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566870213s,-4.566764355s]) 2026-02-11T23:32:02.699+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:32:02.699507+0000/100.808105469s send_stamp 105.374977112s delta_ub 4.566870213s) -- 0x68bd800 con 0x48cc000 2026-02-11T23:32:02.699+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.2 from_epoch 122 2026-02-11T23:32:02.699+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:02.699+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:02.699+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:32:02.699+0000 7f416cbfe640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566889286s,-4.566764355s]) 2026-02-11T23:32:02.699+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:32:02.699507+0000/100.808105469s send_stamp 105.374992371s delta_ub 4.566889286s) -- 0x68b1b00 con 0x4641000 2026-02-11T23:32:02.699+0000 7f416cbfe640 20 osd.0 122 get_con_osd_cluster to osd.2 from_epoch 122 2026-02-11T23:32:02.699+0000 7f416cbfe640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:02.699+0000 7f416cbfe640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:02.699+0000 7f416cbfe640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:32:03.093+0000 7f416c3fd640 1 --2- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] >> conn(0x5282400 0x5280680 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).accept 2026-02-11T23:32:03.093+0000 7f416cbfe640 1 --2- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] >> conn(0x5282400 0x5280680 unknown :-1 s=BANNER_ACCEPTING pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:32:03.093+0000 7f416cbfe640 10 osd.0 122 ms_handle_fast_authentication new session 0x59548c0 con 0x5282400 entity client.admin addr 2026-02-11T23:32:03.093+0000 7f416cbfe640 1 --2- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] >> 127.0.0.1:0/3647056830 conn(0x5282400 0x5280680 crc :-1 s=READY pgs=3 gs=6 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=client.7961 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-11T23:32:03.093+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== client.7961 127.0.0.1:0/3647056830 1 ==== command(tid 1: {"prefix": "get_command_descriptions"}) ==== 62+0+0 (crc 0 0 0) 0x37002c0 con 0x5282400 2026-02-11T23:32:03.093+0000 7f415cf02640 20 osd.0 122 OSD::ms_dispatch: command(tid 1: {"prefix": "get_command_descriptions"}) 2026-02-11T23:32:03.093+0000 7f415cf02640 20 osd.0 122 _dispatch 0x37002c0 command(tid 1: {"prefix": "get_command_descriptions"}) 2026-02-11T23:32:03.093+0000 7f416bbfc640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> 127.0.0.1:0/3647056830 -- command_reply(tid 1: 0 ) -- 0x672de40 con 0x5282400 2026-02-11T23:32:03.101+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== client.7961 127.0.0.1:0/3647056830 2 ==== command(tid 2: {"prefix": "flush_pg_stats"}) ==== 52+0+0 (crc 0 0 0) 0x672de40 con 0x5282400 2026-02-11T23:32:03.101+0000 7f415cf02640 20 osd.0 122 OSD::ms_dispatch: command(tid 2: {"prefix": "flush_pg_stats"}) 2026-02-11T23:32:03.101+0000 7f415cf02640 20 osd.0 122 _dispatch 0x672de40 command(tid 2: {"prefix": "flush_pg_stats"}) 2026-02-11T23:32:03.101+0000 7f416bbfc640 15 osd.0 122 collect_pg_stats 2026-02-11T23:32:03.101+0000 7f416bbfc640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] -- pg_stats(0 pgs seq 21474836508 v 0) -- 0x4aa1200 con 0x4aca800 2026-02-11T23:32:03.101+0000 7f416bbfc640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> 127.0.0.1:0/3647056830 -- command_reply(tid 2: 0 ) -- 0x37002c0 con 0x5282400 2026-02-11T23:32:03.107+0000 7f415cf02640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] <== client.7961 127.0.0.1:0/3647056830 3 ==== command(tid 3: {"prefix": "flush_pg_stats"}) ==== 52+0+0 (crc 0 0 0) 0x37002c0 con 0x5282400 2026-02-11T23:32:03.107+0000 7f415cf02640 20 osd.0 122 OSD::ms_dispatch: command(tid 3: {"prefix": "flush_pg_stats"}) 2026-02-11T23:32:03.107+0000 7f415cf02640 20 osd.0 122 _dispatch 0x37002c0 command(tid 3: {"prefix": "flush_pg_stats"}) 2026-02-11T23:32:03.107+0000 7f416bbfc640 15 osd.0 122 collect_pg_stats 2026-02-11T23:32:03.107+0000 7f416bbfc640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] -- pg_stats(0 pgs seq 21474836509 v 0) -- 0x4aa1500 con 0x4aca800 2026-02-11T23:32:03.107+0000 7f416bbfc640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> 127.0.0.1:0/3647056830 -- command_reply(tid 3: 0 ) -- 0x672de40 con 0x5282400 2026-02-11T23:32:03.107+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] >> 127.0.0.1:0/3647056830 conn(0x5282400 msgr2=0x5280680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 80 2026-02-11T23:32:03.107+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] >> 127.0.0.1:0/3647056830 conn(0x5282400 msgr2=0x5280680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-11T23:32:03.107+0000 7f416cbfe640 1 --2- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] >> 127.0.0.1:0/3647056830 conn(0x5282400 0x5280680 crc :-1 s=READY pgs=3 gs=6 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-11T23:32:03.107+0000 7f416cbfe640 1 --2- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] >> 127.0.0.1:0/3647056830 conn(0x5282400 0x5280680 crc :-1 s=READY pgs=3 gs=6 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-11T23:32:03.107+0000 7f415cf02640 2 osd.0 122 ms_handle_reset con 0x5282400 session 0x59548c0 2026-02-11T23:32:03.151+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.3 127.0.0.1:0/2699377081 28 ==== osd_ping(ping e122 up_from 19 ping_stamp 2026-02-11T23:32:03.152008+0000/98.607948303s send_stamp 98.607948303s delta_ub -7.219393730s) ==== 2033+0+0 (crc 0 0 0) 0x68bd800 con 0x48cc400 2026-02-11T23:32:03.151+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.3 127.0.0.1:0/2699377081 28 ==== osd_ping(ping e122 up_from 19 ping_stamp 2026-02-11T23:32:03.152008+0000/98.607948303s send_stamp 98.607948303s delta_ub -7.219393730s) ==== 2033+0+0 (crc 0 0 0) 0x68b1b00 con 0x48cc800 2026-02-11T23:32:03.151+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219562531s,-7.219393730s]) 2026-02-11T23:32:03.151+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:32:03.152008+0000/98.607948303s send_stamp 105.827507019s delta_ub 7.219562531s) -- 0x68bdb00 con 0x48cc400 2026-02-11T23:32:03.151+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.3 from_epoch 122 2026-02-11T23:32:03.151+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:03.151+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:03.151+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:32:03.151+0000 7f416cbfe640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219586849s,-7.219393730s]) 2026-02-11T23:32:03.151+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:32:03.152008+0000/98.607948303s send_stamp 105.827529907s delta_ub 7.219586849s) -- 0x68b0300 con 0x48cc800 2026-02-11T23:32:03.151+0000 7f416cbfe640 20 osd.0 122 get_con_osd_cluster to osd.3 from_epoch 122 2026-02-11T23:32:03.151+0000 7f416cbfe640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:03.151+0000 7f416cbfe640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:03.151+0000 7f416cbfe640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:32:03.209+0000 7f415b6ff640 20 osd.0 122 reports for 0 queries 2026-02-11T23:32:03.210+0000 7f415b6ff640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] -- mgrreport(unknown.0 +0-0 packed 3294 daemon_metrics=2) -- 0x68ad180 con 0x4aca800 2026-02-11T23:32:03.210+0000 7f415b6ff640 15 osd.0 122 collect_pg_stats 2026-02-11T23:32:03.210+0000 7f415b6ff640 1 -- [v2:127.0.0.1:6800/3294232400,v1:127.0.0.1:6801/3294232400] --> [v2:127.0.0.1:6808/4279895178,v1:127.0.0.1:6809/4279895178] -- pg_stats(0 pgs seq 21474836510 v 0) -- 0x68aef00 con 0x4aca800 2026-02-11T23:32:03.275+0000 7f41633eb640 10 osd.0 122 tick 2026-02-11T23:32:03.275+0000 7f41633eb640 20 osd.0 122 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:32:03.332+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:32:03.332+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:32:03.332+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:33.332618+0000) 2026-02-11T23:32:03.433+0000 7f4147983640 5 osd.0 122 heartbeat osd_stat(store_statfs(0x0/0x0/0x384000, data 0x53a3e5/0x5e6000, compress 0x0/0x0/0x0, omap 0xa403, meta 0x1a45bfd), peers [1,2,3,4] op hist []) 2026-02-11T23:32:03.433+0000 7f4147983640 20 osd.0 122 check_full_status cur ratio 1, physical ratio 1, new state failsafe 2026-02-11T23:32:03.433+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6816/3043114013,v1:127.0.0.1:6817/3043114013] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:32:03.433359+0000/106.108764648s send_stamp 106.108764648s delta_ub 2.011786461s) -- 0x68bdc80 con 0x48c4400 2026-02-11T23:32:03.433+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6814/3043114013,v1:127.0.0.1:6815/3043114013] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:32:03.433359+0000/106.108764648s send_stamp 106.108764648s delta_ub 2.011786461s) -- 0x68bde00 con 0x4640c00 2026-02-11T23:32:03.433+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6824/2365263801,v1:127.0.0.1:6825/2365263801] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:32:03.433359+0000/106.108764648s send_stamp 106.108764648s delta_ub 4.566889286s) -- 0x68bc780 con 0x48c4000 2026-02-11T23:32:03.433+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6822/2365263801,v1:127.0.0.1:6823/2365263801] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:32:03.433359+0000/106.108764648s send_stamp 106.108764648s delta_ub 4.566889286s) -- 0x68bd380 con 0x48cd000 2026-02-11T23:32:03.433+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6832/2699377081,v1:127.0.0.1:6833/2699377081] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:32:03.433359+0000/106.108764648s send_stamp 106.108764648s delta_ub 7.219586849s) -- 0x68bcc00 con 0x48cd400 2026-02-11T23:32:03.433+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6830/2699377081,v1:127.0.0.1:6831/2699377081] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:32:03.433359+0000/106.108764648s send_stamp 106.108764648s delta_ub 7.219586849s) -- 0x68bc600 con 0x48cd800 2026-02-11T23:32:03.433+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6840/3331151400,v1:127.0.0.1:6841/3331151400] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:32:03.433359+0000/106.108764648s send_stamp 106.108764648s delta_ub 9.836770058s) -- 0x68bc300 con 0x507bc00 2026-02-11T23:32:03.433+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6838/3331151400,v1:127.0.0.1:6839/3331151400] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:32:03.433359+0000/106.108764648s send_stamp 106.108764648s delta_ub 9.836770058s) -- 0x68bc000 con 0x5282000 2026-02-11T23:32:03.433+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6816/3043114013 31 ==== osd_ping(ping_reply e122 up_from 10 ping_stamp 2026-02-11T23:32:03.433359+0000/106.108764648s send_stamp 104.097213745s delta_ub -2.011556149s) ==== 2033+0+0 (crc 0 0 0) 0x68bc300 con 0x48c4400 2026-02-11T23:32:03.433+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.1 from_epoch 122 2026-02-11T23:32:03.433+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:03.433+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:03.433+0000 7f416cbfe640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6824/2365263801 30 ==== osd_ping(ping_reply e122 up_from 14 ping_stamp 2026-02-11T23:32:03.433359+0000/106.108764648s send_stamp 101.542060852s delta_ub -4.566703796s) ==== 2033+0+0 (crc 0 0 0) 0x68bc780 con 0x48c4000 2026-02-11T23:32:03.433+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:32:03.433+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6814/3043114013 31 ==== osd_ping(ping_reply e122 up_from 10 ping_stamp 2026-02-11T23:32:03.433359+0000/106.108764648s send_stamp 104.097236633s delta_ub -2.011532784s) ==== 2033+0+0 (crc 0 0 0) 0x68bc000 con 0x4640c00 2026-02-11T23:32:03.433+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011723995s,-2.011556149s]) 2026-02-11T23:32:03.433+0000 7f416cbfe640 20 osd.0 122 get_con_osd_cluster to osd.2 from_epoch 122 2026-02-11T23:32:03.433+0000 7f416cbfe640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:03.433+0000 7f416cbfe640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:03.433+0000 7f416cbfe640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:32:03.433+0000 7f416cbfe640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566882610s,-4.566703796s]) 2026-02-11T23:32:03.433+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6840/3331151400 29 ==== osd_ping(ping_reply e122 up_from 23 ping_stamp 2026-02-11T23:32:03.433359+0000/106.108764648s send_stamp 96.272247314s delta_ub -9.836519241s) ==== 2033+0+0 (crc 0 0 0) 0x68bc300 con 0x507bc00 2026-02-11T23:32:03.433+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.4 from_epoch 122 2026-02-11T23:32:03.433+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:03.433+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:03.433+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:32:03.433+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836707115s,-9.836519241s]) 2026-02-11T23:32:03.433+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.1 from_epoch 122 2026-02-11T23:32:03.433+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:03.433+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6832/2699377081 30 ==== osd_ping(ping_reply e122 up_from 19 ping_stamp 2026-02-11T23:32:03.433359+0000/106.108764648s send_stamp 98.889419556s delta_ub -7.219348907s) ==== 2033+0+0 (crc 0 0 0) 0x68bc300 con 0x48cd400 2026-02-11T23:32:03.433+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:03.433+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:32:03.433+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011726618s,-2.011532784s]) 2026-02-11T23:32:03.433+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.3 from_epoch 122 2026-02-11T23:32:03.433+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:03.433+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:03.433+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:32:03.433+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219555378s,-7.219348907s]) 2026-02-11T23:32:03.433+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6838/3331151400 29 ==== osd_ping(ping_reply e122 up_from 23 ping_stamp 2026-02-11T23:32:03.433359+0000/106.108764648s send_stamp 96.272270203s delta_ub -9.836497307s) ==== 2033+0+0 (crc 0 0 0) 0x68bc000 con 0x5282000 2026-02-11T23:32:03.433+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.4 from_epoch 122 2026-02-11T23:32:03.433+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:03.433+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:03.433+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:32:03.433+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836709976s,-9.836497307s]) 2026-02-11T23:32:03.433+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6822/2365263801 30 ==== osd_ping(ping_reply e122 up_from 14 ping_stamp 2026-02-11T23:32:03.433359+0000/106.108764648s send_stamp 101.542083740s delta_ub -4.566679478s) ==== 2033+0+0 (crc 0 0 0) 0x68bc300 con 0x48cd000 2026-02-11T23:32:03.433+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.2 from_epoch 122 2026-02-11T23:32:03.433+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:03.433+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:03.433+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6830/2699377081 30 ==== osd_ping(ping_reply e122 up_from 19 ping_stamp 2026-02-11T23:32:03.433359+0000/106.108764648s send_stamp 98.889442444s delta_ub -7.219324112s) ==== 2033+0+0 (crc 0 0 0) 0x68bc000 con 0x48cd800 2026-02-11T23:32:03.433+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:32:03.433+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566896915s,-4.566679478s]) 2026-02-11T23:32:03.433+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.3 from_epoch 122 2026-02-11T23:32:03.433+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:03.433+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:03.433+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:32:03.433+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219548225s,-7.219324112s]) 2026-02-11T23:32:03.651+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.3 127.0.0.1:0/2699377081 29 ==== osd_ping(ping e122 up_from 19 ping_stamp 2026-02-11T23:32:03.652197+0000/99.108131409s send_stamp 99.108131409s delta_ub -7.219324112s) ==== 2033+0+0 (crc 0 0 0) 0x68bc780 con 0x48cc800 2026-02-11T23:32:03.651+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.3 127.0.0.1:0/2699377081 29 ==== osd_ping(ping e122 up_from 19 ping_stamp 2026-02-11T23:32:03.652197+0000/99.108131409s send_stamp 99.108131409s delta_ub -7.219324112s) ==== 2033+0+0 (crc 0 0 0) 0x68bc000 con 0x48cc400 2026-02-11T23:32:03.651+0000 7f416cbfe640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219560623s,-7.219324112s]) 2026-02-11T23:32:03.651+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:32:03.652197+0000/99.108131409s send_stamp 106.327697754s delta_ub 7.219560623s) -- 0x68b0300 con 0x48cc800 2026-02-11T23:32:03.652+0000 7f416cbfe640 20 osd.0 122 get_con_osd_cluster to osd.3 from_epoch 122 2026-02-11T23:32:03.652+0000 7f416cbfe640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:03.652+0000 7f416cbfe640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:03.652+0000 7f416cbfe640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:32:03.652+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219593525s,-7.219324112s]) 2026-02-11T23:32:03.652+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:32:03.652197+0000/99.108131409s send_stamp 106.327728271s delta_ub 7.219593525s) -- 0x68bc600 con 0x48cc400 2026-02-11T23:32:03.652+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.3 from_epoch 122 2026-02-11T23:32:03.652+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:03.652+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:03.652+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:32:03.664+0000 7f4162bea640 10 osd.0 122 tick_without_osd_lock 2026-02-11T23:32:03.664+0000 7f4162bea640 0 osd.0 122 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:32:03.664+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:32:03.664+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.130 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:32:03.664+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:32:03.664, recovery is active?:false restrictions:<....> 2026-02-11T23:32:03.664+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:32:03.664+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:32:03.664+0000 7f4162bea640 20 osd.0 122 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:32:03.664+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:32:03.664+0000 7f4162bea640 20 osd.0 122 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:32:03.799+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.2 127.0.0.1:0/2365263801 31 ==== osd_ping(ping e122 up_from 14 ping_stamp 2026-02-11T23:32:03.799644+0000/101.908241272s send_stamp 101.908241272s delta_ub -4.566679478s) ==== 2033+0+0 (crc 0 0 0) 0x68b0300 con 0x4641000 2026-02-11T23:32:03.799+0000 7f416cbfe640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566868782s,-4.566679478s]) 2026-02-11T23:32:03.799+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.2 127.0.0.1:0/2365263801 31 ==== osd_ping(ping e122 up_from 14 ping_stamp 2026-02-11T23:32:03.799644+0000/101.908241272s send_stamp 101.908241272s delta_ub -4.566679478s) ==== 2033+0+0 (crc 0 0 0) 0x68bc600 con 0x48cc000 2026-02-11T23:32:03.799+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:32:03.799644+0000/101.908241272s send_stamp 106.475112915s delta_ub 4.566868782s) -- 0x68bc780 con 0x4641000 2026-02-11T23:32:03.799+0000 7f416cbfe640 20 osd.0 122 get_con_osd_cluster to osd.2 from_epoch 122 2026-02-11T23:32:03.799+0000 7f416cbfe640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:03.799+0000 7f416cbfe640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:03.799+0000 7f416cbfe640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:32:03.799+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566887856s,-4.566679478s]) 2026-02-11T23:32:03.799+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:32:03.799644+0000/101.908241272s send_stamp 106.475128174s delta_ub 4.566887856s) -- 0x68bc000 con 0x48cc000 2026-02-11T23:32:03.799+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.2 from_epoch 122 2026-02-11T23:32:03.799+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:03.799+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:03.799+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:32:03.933+0000 7f4147983640 5 osd.0 122 heartbeat osd_stat(store_statfs(0x0/0x0/0x384000, data 0x53a3e5/0x5e6000, compress 0x0/0x0/0x0, omap 0xa403, meta 0x1a45bfd), peers [1,2,3,4] op hist []) 2026-02-11T23:32:03.933+0000 7f4147983640 20 osd.0 122 check_full_status cur ratio 1, physical ratio 1, new state failsafe 2026-02-11T23:32:03.933+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6816/3043114013,v1:127.0.0.1:6817/3043114013] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:32:03.933491+0000/106.608894348s send_stamp 106.608894348s delta_ub 2.011726618s) -- 0x68bcf00 con 0x48c4400 2026-02-11T23:32:03.933+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6814/3043114013,v1:127.0.0.1:6815/3043114013] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:32:03.933491+0000/106.608894348s send_stamp 106.608894348s delta_ub 2.011726618s) -- 0x68b0780 con 0x4640c00 2026-02-11T23:32:03.933+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6824/2365263801,v1:127.0.0.1:6825/2365263801] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:32:03.933491+0000/106.608894348s send_stamp 106.608894348s delta_ub 4.566887856s) -- 0x68b0a80 con 0x48c4000 2026-02-11T23:32:03.933+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6822/2365263801,v1:127.0.0.1:6823/2365263801] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:32:03.933491+0000/106.608894348s send_stamp 106.608894348s delta_ub 4.566887856s) -- 0x68b0180 con 0x48cd000 2026-02-11T23:32:03.933+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6832/2699377081,v1:127.0.0.1:6833/2699377081] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:32:03.933491+0000/106.608894348s send_stamp 106.608894348s delta_ub 7.219593525s) -- 0x4c9a480 con 0x48cd400 2026-02-11T23:32:03.933+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6830/2699377081,v1:127.0.0.1:6831/2699377081] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:32:03.933491+0000/106.608894348s send_stamp 106.608894348s delta_ub 7.219593525s) -- 0x4ec3800 con 0x48cd800 2026-02-11T23:32:03.933+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6840/3331151400,v1:127.0.0.1:6841/3331151400] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:32:03.933491+0000/106.608894348s send_stamp 106.608894348s delta_ub 9.836709976s) -- 0x4ecca80 con 0x507bc00 2026-02-11T23:32:03.933+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6838/3331151400,v1:127.0.0.1:6839/3331151400] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:32:03.933491+0000/106.608894348s send_stamp 106.608894348s delta_ub 9.836709976s) -- 0x4ecdb00 con 0x5282000 2026-02-11T23:32:03.933+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6816/3043114013 32 ==== osd_ping(ping_reply e122 up_from 10 ping_stamp 2026-02-11T23:32:03.933491+0000/106.608894348s send_stamp 104.597305298s delta_ub -2.011591911s) ==== 2033+0+0 (crc 0 0 0) 0x4c9a480 con 0x48c4400 2026-02-11T23:32:03.933+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.1 from_epoch 122 2026-02-11T23:32:03.933+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:03.933+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:03.933+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:32:03.933+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011705399s,-2.011591911s]) 2026-02-11T23:32:03.933+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6814/3043114013 32 ==== osd_ping(ping_reply e122 up_from 10 ping_stamp 2026-02-11T23:32:03.933491+0000/106.608894348s send_stamp 104.597312927s delta_ub -2.011578560s) ==== 2033+0+0 (crc 0 0 0) 0x4ec3800 con 0x4640c00 2026-02-11T23:32:03.933+0000 7f416cbfe640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6824/2365263801 31 ==== osd_ping(ping_reply e122 up_from 14 ping_stamp 2026-02-11T23:32:03.933491+0000/106.608894348s send_stamp 102.042167664s delta_ub -4.566731453s) ==== 2033+0+0 (crc 0 0 0) 0x68b0d80 con 0x48c4000 2026-02-11T23:32:03.933+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.1 from_epoch 122 2026-02-11T23:32:03.933+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:03.933+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:03.933+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:32:03.933+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011705160s,-2.011578560s]) 2026-02-11T23:32:03.933+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6822/2365263801 31 ==== osd_ping(ping_reply e122 up_from 14 ping_stamp 2026-02-11T23:32:03.933491+0000/106.608894348s send_stamp 102.042190552s delta_ub -4.566710472s) ==== 2033+0+0 (crc 0 0 0) 0x4ecca80 con 0x48cd000 2026-02-11T23:32:03.933+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.2 from_epoch 122 2026-02-11T23:32:03.933+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:03.933+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:03.933+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:32:03.933+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566864491s,-4.566710472s]) 2026-02-11T23:32:03.933+0000 7f416cbfe640 20 osd.0 122 get_con_osd_cluster to osd.2 from_epoch 122 2026-02-11T23:32:03.933+0000 7f416cbfe640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:03.933+0000 7f416cbfe640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:03.933+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6832/2699377081 31 ==== osd_ping(ping_reply e122 up_from 19 ping_stamp 2026-02-11T23:32:03.933491+0000/106.608894348s send_stamp 99.389526367s delta_ub -7.219363689s) ==== 2033+0+0 (crc 0 0 0) 0x4ecca80 con 0x48cd400 2026-02-11T23:32:03.933+0000 7f416cbfe640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:32:03.933+0000 7f416cbfe640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566894531s,-4.566731453s]) 2026-02-11T23:32:03.933+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.3 from_epoch 122 2026-02-11T23:32:03.933+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:03.933+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:03.933+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:32:03.933+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219543934s,-7.219363689s]) 2026-02-11T23:32:03.933+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6840/3331151400 30 ==== osd_ping(ping_reply e122 up_from 23 ping_stamp 2026-02-11T23:32:03.933491+0000/106.608894348s send_stamp 96.772415161s delta_ub -9.836478233s) ==== 2033+0+0 (crc 0 0 0) 0x4ecca80 con 0x507bc00 2026-02-11T23:32:03.933+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.4 from_epoch 122 2026-02-11T23:32:03.933+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:03.933+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6830/2699377081 31 ==== osd_ping(ping_reply e122 up_from 19 ping_stamp 2026-02-11T23:32:03.933491+0000/106.608894348s send_stamp 99.389556885s delta_ub -7.219339371s) ==== 2033+0+0 (crc 0 0 0) 0x4ecdb00 con 0x48cd800 2026-02-11T23:32:03.933+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:03.933+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:32:03.933+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836675644s,-9.836478233s]) 2026-02-11T23:32:03.933+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.3 from_epoch 122 2026-02-11T23:32:03.933+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:03.933+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:03.933+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:32:03.933+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219557762s,-7.219339371s]) 2026-02-11T23:32:03.933+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6838/3331151400 30 ==== osd_ping(ping_reply e122 up_from 23 ping_stamp 2026-02-11T23:32:03.933491+0000/106.608894348s send_stamp 96.772445679s delta_ub -9.836449623s) ==== 2033+0+0 (crc 0 0 0) 0x4ecdb00 con 0x5282000 2026-02-11T23:32:03.933+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.4 from_epoch 122 2026-02-11T23:32:03.933+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:03.933+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:03.933+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:32:03.933+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836701393s,-9.836449623s]) 2026-02-11T23:32:04.298+0000 7f41633eb640 10 osd.0 122 tick 2026-02-11T23:32:04.298+0000 7f41633eb640 20 osd.0 122 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:32:04.332+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:32:04.332+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:32:04.332+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:34.332692+0000) 2026-02-11T23:32:04.468+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.4 127.0.0.1:0/3331151400 29 ==== osd_ping(ping e122 up_from 23 ping_stamp 2026-02-11T23:32:04.469097+0000/97.307868958s send_stamp 97.307868958s delta_ub -9.836449623s) ==== 2033+0+0 (crc 0 0 0) 0x4ecdb00 con 0x507a800 2026-02-11T23:32:04.468+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836720467s,-9.836449623s]) 2026-02-11T23:32:04.468+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.4 127.0.0.1:0/3331151400 29 ==== osd_ping(ping e122 up_from 23 ping_stamp 2026-02-11T23:32:04.469097+0000/97.307868958s send_stamp 97.307868958s delta_ub -9.836449623s) ==== 2033+0+0 (crc 0 0 0) 0x4ecca80 con 0x507a400 2026-02-11T23:32:04.468+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:32:04.469097+0000/97.307868958s send_stamp 107.144584656s delta_ub 9.836720467s) -- 0x4ec3800 con 0x507a800 2026-02-11T23:32:04.468+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.4 from_epoch 122 2026-02-11T23:32:04.468+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:04.468+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:04.468+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:32:04.468+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836743355s,-9.836449623s]) 2026-02-11T23:32:04.468+0000 7f416abfa640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3331151400 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:32:04.469097+0000/97.307868958s send_stamp 107.144607544s delta_ub 9.836743355s) -- 0x4c9a480 con 0x507a400 2026-02-11T23:32:04.468+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.4 from_epoch 122 2026-02-11T23:32:04.468+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:04.468+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:04.468+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:32:04.711+0000 7f4162bea640 10 osd.0 122 tick_without_osd_lock 2026-02-11T23:32:04.711+0000 7f4162bea640 0 osd.0 122 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:32:04.711+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:32:04.711+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.130 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:32:04.711+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:32:04.711, recovery is active?:false restrictions:<....> 2026-02-11T23:32:04.711+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:32:04.711+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:32:04.711+0000 7f4162bea640 20 osd.0 122 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:32:04.711+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:32:04.711+0000 7f4162bea640 20 osd.0 122 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:32:04.752+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.3 127.0.0.1:0/2699377081 30 ==== osd_ping(ping e122 up_from 19 ping_stamp 2026-02-11T23:32:04.752351+0000/100.208290100s send_stamp 100.208290100s delta_ub -7.219339371s) ==== 2033+0+0 (crc 0 0 0) 0x4ec3800 con 0x48cc400 2026-02-11T23:32:04.752+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219525814s,-7.219339371s]) 2026-02-11T23:32:04.752+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:32:04.752351+0000/100.208290100s send_stamp 107.427818298s delta_ub 7.219525814s) -- 0x4ecdb00 con 0x48cc400 2026-02-11T23:32:04.752+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.3 from_epoch 122 2026-02-11T23:32:04.752+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:04.752+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:04.752+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:32:04.752+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.3 127.0.0.1:0/2699377081 30 ==== osd_ping(ping e122 up_from 19 ping_stamp 2026-02-11T23:32:04.752351+0000/100.208290100s send_stamp 100.208290100s delta_ub -7.219339371s) ==== 2033+0+0 (crc 0 0 0) 0x68b0d80 con 0x48cc800 2026-02-11T23:32:04.752+0000 7f416cbfe640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219542027s,-7.219339371s]) 2026-02-11T23:32:04.752+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2699377081 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:32:04.752351+0000/100.208290100s send_stamp 107.427833557s delta_ub 7.219542027s) -- 0x68b1680 con 0x48cc800 2026-02-11T23:32:04.752+0000 7f416cbfe640 20 osd.0 122 get_con_osd_cluster to osd.3 from_epoch 122 2026-02-11T23:32:04.752+0000 7f416cbfe640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:04.752+0000 7f416cbfe640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:04.752+0000 7f416cbfe640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:32:04.844+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.1 127.0.0.1:0/3043114013 31 ==== osd_ping(ping e122 up_from 10 ping_stamp 2026-02-11T23:32:04.844538+0000/105.508285522s send_stamp 105.508285522s delta_ub -2.011578560s) ==== 2033+0+0 (crc 0 0 0) 0x4ecdb00 con 0x4aca400 2026-02-11T23:32:04.844+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011715174s,-2.011578560s]) 2026-02-11T23:32:04.844+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:32:04.844538+0000/105.508285522s send_stamp 107.519996643s delta_ub 2.011715174s) -- 0x4ec3800 con 0x4aca400 2026-02-11T23:32:04.844+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.1 from_epoch 122 2026-02-11T23:32:04.844+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:04.844+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.1 127.0.0.1:0/3043114013 31 ==== osd_ping(ping e122 up_from 10 ping_stamp 2026-02-11T23:32:04.844538+0000/105.508285522s send_stamp 105.508285522s delta_ub -2.011578560s) ==== 2033+0+0 (crc 0 0 0) 0x68b1680 con 0x4aca000 2026-02-11T23:32:04.844+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:04.844+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:32:04.844+0000 7f416cbfe640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011729717s,-2.011578560s]) 2026-02-11T23:32:04.844+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/3043114013 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:32:04.844538+0000/105.508285522s send_stamp 107.520019531s delta_ub 2.011729717s) -- 0x68b0d80 con 0x4aca000 2026-02-11T23:32:04.844+0000 7f416cbfe640 20 osd.0 122 get_con_osd_cluster to osd.1 from_epoch 122 2026-02-11T23:32:04.844+0000 7f416cbfe640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:04.844+0000 7f416cbfe640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:04.844+0000 7f416cbfe640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:32:04.899+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.2 127.0.0.1:0/2365263801 32 ==== osd_ping(ping e122 up_from 14 ping_stamp 2026-02-11T23:32:04.899801+0000/103.008392334s send_stamp 103.008392334s delta_ub -4.566710472s) ==== 2033+0+0 (crc 0 0 0) 0x4ec3800 con 0x48cc000 2026-02-11T23:32:04.899+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566870213s,-4.566710472s]) 2026-02-11T23:32:04.899+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:32:04.899801+0000/103.008392334s send_stamp 107.575271606s delta_ub 4.566870213s) -- 0x4ecdb00 con 0x48cc000 2026-02-11T23:32:04.899+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.2 127.0.0.1:0/2365263801 32 ==== osd_ping(ping e122 up_from 14 ping_stamp 2026-02-11T23:32:04.899801+0000/103.008392334s send_stamp 103.008392334s delta_ub -4.566710472s) ==== 2033+0+0 (crc 0 0 0) 0x68b0d80 con 0x4641000 2026-02-11T23:32:04.899+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.2 from_epoch 122 2026-02-11T23:32:04.899+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:04.899+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:04.899+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:32:04.899+0000 7f416cbfe640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566887379s,-4.566710472s]) 2026-02-11T23:32:04.899+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:32:04.899801+0000/103.008392334s send_stamp 107.575286865s delta_ub 4.566887379s) -- 0x68b1680 con 0x4641000 2026-02-11T23:32:04.899+0000 7f416cbfe640 20 osd.0 122 get_con_osd_cluster to osd.2 from_epoch 122 2026-02-11T23:32:04.899+0000 7f416cbfe640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:04.899+0000 7f416cbfe640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:04.899+0000 7f416cbfe640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:32:05.332+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:32:05.332+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:32:05.332+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:35.332768+0000) 2026-02-11T23:32:05.336+0000 7f41633eb640 10 osd.0 122 tick 2026-02-11T23:32:05.336+0000 7f41633eb640 20 osd.0 122 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:32:05.399+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] <== osd.2 127.0.0.1:0/2365263801 33 ==== osd_ping(ping e122 up_from 14 ping_stamp 2026-02-11T23:32:05.399969+0000/103.508567810s send_stamp 103.508567810s delta_ub -4.566740990s) ==== 2033+0+0 (crc 0 0 0) 0x4ecdb00 con 0x48cc000 2026-02-11T23:32:05.399+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566898823s,-4.566740990s]) 2026-02-11T23:32:05.399+0000 7f416c3fd640 1 -- [v2:127.0.0.1:6804/3294232400,v1:127.0.0.1:6805/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:32:05.399969+0000/103.508567810s send_stamp 108.075469971s delta_ub 4.566898823s) -- 0x4ec3800 con 0x48cc000 2026-02-11T23:32:05.399+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.2 from_epoch 122 2026-02-11T23:32:05.399+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:05.399+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:05.399+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] <== osd.2 127.0.0.1:0/2365263801 33 ==== osd_ping(ping e122 up_from 14 ping_stamp 2026-02-11T23:32:05.399969+0000/103.508567810s send_stamp 103.508567810s delta_ub -4.566740990s) ==== 2033+0+0 (crc 0 0 0) 0x68b1680 con 0x4641000 2026-02-11T23:32:05.399+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:32:05.399+0000 7f416cbfe640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566910744s,-4.566740990s]) 2026-02-11T23:32:05.399+0000 7f416cbfe640 1 -- [v2:127.0.0.1:6806/3294232400,v1:127.0.0.1:6807/3294232400] --> 127.0.0.1:0/2365263801 -- osd_ping(ping_reply e122 up_from 5 ping_stamp 2026-02-11T23:32:05.399969+0000/103.508567810s send_stamp 108.075477600s delta_ub 4.566910744s) -- 0x68b0d80 con 0x4641000 2026-02-11T23:32:05.399+0000 7f416cbfe640 20 osd.0 122 get_con_osd_cluster to osd.2 from_epoch 122 2026-02-11T23:32:05.399+0000 7f416cbfe640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:05.399+0000 7f416cbfe640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:05.399+0000 7f416cbfe640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:32:05.723+0000 7f4162bea640 10 osd.0 122 tick_without_osd_lock 2026-02-11T23:32:05.723+0000 7f4162bea640 0 osd.0 122 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:32:05.723+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:32:05.723+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.130 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:32:05.723+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:32:05.723, recovery is active?:false restrictions:<....> 2026-02-11T23:32:05.723+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:32:05.723+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:32:05.723+0000 7f4162bea640 20 osd.0 122 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:32:05.723+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:32:05.723+0000 7f4162bea640 20 osd.0 122 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:32:06.233+0000 7f4147983640 5 osd.0 122 heartbeat osd_stat(store_statfs(0x0/0x0/0x384000, data 0x53a3e5/0x5e6000, compress 0x0/0x0/0x0, omap 0xa403, meta 0x1a45bfd), peers [1,2,3,4] op hist []) 2026-02-11T23:32:06.233+0000 7f4147983640 20 osd.0 122 check_full_status cur ratio 1, physical ratio 1, new state failsafe 2026-02-11T23:32:06.233+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6816/3043114013,v1:127.0.0.1:6817/3043114013] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:32:06.233694+0000/108.909103394s send_stamp 108.909103394s delta_ub 2.011729717s) -- 0x5e98000 con 0x48c4400 2026-02-11T23:32:06.233+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6814/3043114013,v1:127.0.0.1:6815/3043114013] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:32:06.233694+0000/108.909103394s send_stamp 108.909103394s delta_ub 2.011729717s) -- 0x5e98180 con 0x4640c00 2026-02-11T23:32:06.233+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6824/2365263801,v1:127.0.0.1:6825/2365263801] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:32:06.233694+0000/108.909103394s send_stamp 108.909103394s delta_ub 4.566910744s) -- 0x5e98300 con 0x48c4000 2026-02-11T23:32:06.233+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6822/2365263801,v1:127.0.0.1:6823/2365263801] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:32:06.233694+0000/108.909103394s send_stamp 108.909103394s delta_ub 4.566910744s) -- 0x5e98480 con 0x48cd000 2026-02-11T23:32:06.233+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6832/2699377081,v1:127.0.0.1:6833/2699377081] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:32:06.233694+0000/108.909103394s send_stamp 108.909103394s delta_ub 7.219542027s) -- 0x5e98600 con 0x48cd400 2026-02-11T23:32:06.233+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6830/2699377081,v1:127.0.0.1:6831/2699377081] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:32:06.233694+0000/108.909103394s send_stamp 108.909103394s delta_ub 7.219542027s) -- 0x5e98780 con 0x48cd800 2026-02-11T23:32:06.233+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6840/3331151400,v1:127.0.0.1:6841/3331151400] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:32:06.233694+0000/108.909103394s send_stamp 108.909103394s delta_ub 9.836743355s) -- 0x5e98900 con 0x507bc00 2026-02-11T23:32:06.233+0000 7f4147983640 1 -- 127.0.0.1:0/3294232400 --> [v2:127.0.0.1:6838/3331151400,v1:127.0.0.1:6839/3331151400] -- osd_ping(ping e122 up_from 5 ping_stamp 2026-02-11T23:32:06.233694+0000/108.909103394s send_stamp 108.909103394s delta_ub 9.836743355s) -- 0x5e98a80 con 0x5282000 2026-02-11T23:32:06.233+0000 7f416cbfe640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6824/2365263801 32 ==== osd_ping(ping_reply e122 up_from 14 ping_stamp 2026-02-11T23:32:06.233694+0000/108.909103394s send_stamp 104.342407227s delta_ub -4.566697598s) ==== 2033+0+0 (crc 0 0 0) 0x5e98300 con 0x48c4000 2026-02-11T23:32:06.233+0000 7f416cbfe640 20 osd.0 122 get_con_osd_cluster to osd.2 from_epoch 122 2026-02-11T23:32:06.233+0000 7f416cbfe640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:06.233+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6814/3043114013 33 ==== osd_ping(ping_reply e122 up_from 10 ping_stamp 2026-02-11T23:32:06.233694+0000/108.909103394s send_stamp 106.897552490s delta_ub -2.011548042s) ==== 2033+0+0 (crc 0 0 0) 0x5e98a80 con 0x4640c00 2026-02-11T23:32:06.233+0000 7f416cbfe640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:06.233+0000 7f416cbfe640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:32:06.233+0000 7f416cbfe640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566874981s,-4.566697598s]) 2026-02-11T23:32:06.233+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.2 v2:127.0.0.1:6822/2365263801 32 ==== osd_ping(ping_reply e122 up_from 14 ping_stamp 2026-02-11T23:32:06.233694+0000/108.909103394s send_stamp 104.342430115s delta_ub -4.566669464s) ==== 2033+0+0 (crc 0 0 0) 0x5e98900 con 0x48cd000 2026-02-11T23:32:06.233+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.2 from_epoch 122 2026-02-11T23:32:06.233+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:06.233+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:06.233+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6820/2365263801 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:32:06.233+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.2 up_from 14 peer_clock_delta [-4.566856384s,-4.566669464s]) 2026-02-11T23:32:06.233+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.1 from_epoch 122 2026-02-11T23:32:06.233+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:06.233+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:06.233+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:32:06.233+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011749029s,-2.011548042s]) 2026-02-11T23:32:06.233+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.1 v2:127.0.0.1:6816/3043114013 33 ==== osd_ping(ping_reply e122 up_from 10 ping_stamp 2026-02-11T23:32:06.233694+0000/108.909103394s send_stamp 106.897583008s delta_ub -2.011522532s) ==== 2033+0+0 (crc 0 0 0) 0x5e98900 con 0x48c4400 2026-02-11T23:32:06.233+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.1 from_epoch 122 2026-02-11T23:32:06.233+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:06.233+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:06.233+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6812/3043114013 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:32:06.233+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.1 up_from 10 peer_clock_delta [-2.011733294s,-2.011522532s]) 2026-02-11T23:32:06.233+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6830/2699377081 32 ==== osd_ping(ping_reply e122 up_from 19 ping_stamp 2026-02-11T23:32:06.233694+0000/108.909103394s send_stamp 101.689758301s delta_ub -7.219337940s) ==== 2033+0+0 (crc 0 0 0) 0x5e98a80 con 0x48cd800 2026-02-11T23:32:06.233+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.3 from_epoch 122 2026-02-11T23:32:06.233+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:06.233+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:06.233+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:32:06.233+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.3 v2:127.0.0.1:6832/2699377081 32 ==== osd_ping(ping_reply e122 up_from 19 ping_stamp 2026-02-11T23:32:06.233694+0000/108.909103394s send_stamp 101.689788818s delta_ub -7.219313145s) ==== 2033+0+0 (crc 0 0 0) 0x5e98900 con 0x48cd400 2026-02-11T23:32:06.233+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219561577s,-7.219337940s]) 2026-02-11T23:32:06.233+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.3 from_epoch 122 2026-02-11T23:32:06.233+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:06.233+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:06.233+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6828/2699377081 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:32:06.233+0000 7f416c3fd640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6838/3331151400 31 ==== osd_ping(ping_reply e122 up_from 23 ping_stamp 2026-02-11T23:32:06.233694+0000/108.909103394s send_stamp 99.072601318s delta_ub -9.836501122s) ==== 2033+0+0 (crc 0 0 0) 0x5e98a80 con 0x5282000 2026-02-11T23:32:06.233+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.3 up_from 19 peer_clock_delta [-7.219548225s,-7.219313145s]) 2026-02-11T23:32:06.233+0000 7f416c3fd640 20 osd.0 122 get_con_osd_cluster to osd.4 from_epoch 122 2026-02-11T23:32:06.233+0000 7f416c3fd640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:06.233+0000 7f416c3fd640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:06.233+0000 7f416abfa640 1 -- 127.0.0.1:0/3294232400 <== osd.4 v2:127.0.0.1:6840/3331151400 31 ==== osd_ping(ping_reply e122 up_from 23 ping_stamp 2026-02-11T23:32:06.233694+0000/108.909103394s send_stamp 99.072624207s delta_ub -9.836474419s) ==== 2033+0+0 (crc 0 0 0) 0x5e98900 con 0x507bc00 2026-02-11T23:32:06.233+0000 7f416c3fd640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:32:06.233+0000 7f416c3fd640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836750031s,-9.836501122s]) 2026-02-11T23:32:06.233+0000 7f416abfa640 20 osd.0 122 get_con_osd_cluster to osd.4 from_epoch 122 2026-02-11T23:32:06.233+0000 7f416abfa640 20 osd.0 122 get_nextmap_reserved map_reservations: {122=1} 2026-02-11T23:32:06.233+0000 7f416abfa640 20 osd.0 122 release_map epoch: 122 2026-02-11T23:32:06.233+0000 7f416abfa640 10 osd.0 122 maybe_share_map: con v2:127.0.0.1:6836/3331151400 our osdmap epoch of 122 is not newer than session's projected_epoch of 122 2026-02-11T23:32:06.233+0000 7f416abfa640 20 osd.0 122 handle_osd_ping new stamps hbstamp(osd.4 up_from 23 peer_clock_delta [-9.836736679s,-9.836474419s]) 2026-02-11T23:32:06.314+0000 7f41633eb640 10 osd.0 122 tick 2026-02-11T23:32:06.314+0000 7f41633eb640 20 osd.0 122 tick last_purged_snaps_scrub 2026-02-11T23:30:18.281208+0000 next 2026-02-13T04:54:26.718708+0000 2026-02-11T23:32:06.332+0000 7f415bf00640 10 monclient: tick 2026-02-11T23:32:06.332+0000 7f415bf00640 10 monclient: _check_auth_tickets 2026-02-11T23:32:06.332+0000 7f415bf00640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:36.332838+0000) 2026-02-11T23:32:06.677+0000 7f4162bea640 10 osd.0 122 tick_without_osd_lock 2026-02-11T23:32:06.677+0000 7f4162bea640 0 osd.0 122 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:32:06.677+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:32:06.677+0000 7f4162bea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.130 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:32:06.677+0000 7f4162bea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:32:06.678, recovery is active?:false restrictions:<....> 2026-02-11T23:32:06.677+0000 7f4162bea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:32:06.677+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:32:06.677+0000 7f4162bea640 20 osd.0 122 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:32:06.677+0000 7f4162bea640 10 osd.0 122 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:32:06.677+0000 7f4162bea640 20 osd.0 122 maybe_send_beacon beacon would be too frequent; skipping